I rent a vps to run fediverse instances. Here's what I've found
I rent a vps to run fediverse instances. Here's what I've found
Hi everybody! I rent a vps with 6 cores, 16 GB ram and 100 GB nvme to run fediverse instances (mastodon, lemmy, matrix, peertube) and a couple related services (nginx proxy manager, nodebb forum, wiki.js, uptime kuma, portainer, tig-stack).
The why
Over the past month they have been running. It has been some work to set them up but since then they run pretty flawlessly. My main goal was to find out how much hardware I really need to run private instances (just me or maybe a handful of friends).
Here are the stats:
Disclaimer: I checked and all services federate correctly.
Resource usage
From my point of view, the 6 cores are totally overkill. A two core would probably suffice. Of the 16 GB of ram I currently need 4. So 8 should suffice. The 100 GB nvme drive is barely handling it (66% full) and thats without backups (they get pulled immediately and then deleted) and with a 48 hour media delete cycle on mastodon (which currently is the largest folder I have).
Individual Space usage (nvme)
- 8.0K heimdall
- 2.0G lemmy
- 7.1G mastodon
- 5.2G matrix
- 123M nodebb
- 236M npm
- 3.5G peertube
- 12K pgadmin
- 8.0K portainer
- 376K tig-stack
- 8.0K uptime-kuma
- 8.0K watchtower
- 12K wiki.js
- 3.6G wordpress
So while I use lemmy the most, mastodon uses far more disk space, matrix also uses more than lemmy although I'm in like 20 chats and my friends don't use matrix except to chat with one another. I have subscribed to maybe 50 communities on lemmy.
Since the graphs in the screenshots lack legends, I'll explain the spikes.
Container CPU usage
- red spike: lemmy postgres
- lightblue: mastodon sidekiq
- yellow: telegraf
- dark-blue: lemmy ui
Container Ram usage
- red line: peertube
- light-blue: mastodon sidekiq
- dark-red: lemmy postgres
- pink: mastodon postgres
Summary
So far I didn't actively try to get the needed resources down except mastodon since I didnt want it to derail my experiment.
Mastodon seems to have the worst data on drive footprint, Peertube seems like a constant ram hog and lemmy spikes in the cpu all the time. So they all have their positives and negatives but nothing major. If you have 2 cores, at least 6 GB of ram and probably 250 GB of disk space to spare you should be good.
I'll work on making the compose files and config files publicly available but right now I'm not there. If you need help spinning up an instance, please let me know. I can send you individual files if needed.
Have a good one! :)
Hey all, don't get too excited. A 2GB Lemmy instance has very little data going through it. My instance, which is just me, and auto-subscribes to communities with "Top" posts using LCS, processes about 10GB of new data a day.
Without subscriptions, the Top, Hot, etc feeds would basically be empty. So if you only want communities you subscribe to, IO will be low, but if you want a full experience, disk and IO will be your bottleneck.
All that being said, thanks to OP for giving the community all this extremely helpful info.
Thank you for sharing that. Personally, I am unlikely to start my own hosting, but hopefully it helps others to make the fediverse stronger
I've been wanting to, but I'm lazy and have adhd. I should easily have the hardware requirements strewn around the house though to slap something together should my focus travel that road.
It's nice to know your options.
ADHD
should my focus travel that road
made me snort.
Also, thanks OP for this post, this is some great quality dense information useful stuff right here.
Thank you for your feedback. Thats precisely my intention with this post. I‘d like hosting your own instance to be as easy as „come up with name, buy domain name, run script, done“. Lot of work until that day but it would make the fediverse infonitely larger imo.
Interesting read! Can I ask how much running this approximately costs?
Looking at DigitalOcean, a similar ish Droplet would be around $100/mo.
At Hetzner, a similar ish shared host would be around €30–40/mo, or dedicated for €50–60/mo.
Not sure where OP is hosting, but they may have gotten cheaper.
On Hetzner, if you go with OP’s recommendation of 2 vCPU, 8 GB of RAM, and do the storage add on for 100 GB of bulk storage, it’d be about €16/mo.
I pay around 10$ per month atm. But you need to factor in backups which I handle with my home server. Those can be acquired as cheap as 2$ a month I believe. It’s not expensive if you know where to look and dont mind making mistakes.
I am also interested in the answer to this and which service the author is using.
I think I found them on serverhunter.com or similar. Just search for serverhunter and you should find many interesting offers. The important part is that a good backup procedure is more important than strong hardware.
Anyone knows why Mastodon is so disk space hungry? Seems kinda stupid.
Is it all the video?
Every image on the federated timeline. It's a lot if users on your instance follow many other users.
I'd say, you have a small instance.
I used to host lemmyonline.com, which has somewhere around 50-100 users.
It was upwards of 50-80g of disk space, and it used a pretty good chunk of bandwidth. CPU/Memory requirements were not very high though.
As I said, I host a small instance for me and a handful of friends. I subscribed to 30-50 communities and post regularly. I don’t think thats in any way comparable to a 50-100 people instance.
Also, my findings were that mastodon has like 2-3 times the storage requirement although the same numbers of users.
"Used to"? It still seems to be online, or do you mean that the other admin handles the hosting now?
The other admin now "owns" this instance, and hosts it in the EU.
I am just a glorified moderator now.
I hosted mastodon once. 20GB disk was full after a few days only. At some point the full disk somehow made the postgres DB delete itself, and then I stopped running my own instance.
For Lemmy, the CPU spikes are probably the score updates...
Great info. Out of curiosity what does your hosting setup say about visitor numbers? Futurology.today uses Cloudflare. They give a figure of about 10k per day for what they call unique visitors. That seems unduly high when you look at how busy our lemmy instance actually is. We have just short of 1K subscribers, so I would assume visitor numbers would be lower than 10k per day.
The cloud flare unique visitors counts requests made by other federated instances. My solo instance reads around 14k unique visitors in the past week.
Good question. i‘d have to check that. So far I‘m not sure how to find that out.
Thank you very much for the breakdown!
This is very useful to know for people new to self-hosting as myself :)
(And for developers wanting to improve their apps potentially)
oh wow thats interesting info. Thanks for sharing!
Thank you very much. I appreciate the good vibes. :)
Thank you very much! great practical insights.
If you plan to do it again, how about to increase the data timespan (e.g. 6 months)? it will also give insight about scale as the time passes?
Thanks! I appreciate it.
Sure, I‘ll probably increase the collection timespan. I just didn’t see the necessity before. The setup is about a month old so there‘s a lot still to learn and to experience.
Right now I‘m trying to figure out if I keep it running as it is with a bit spare disk space or if I start testing backups and maybe put it all in one script, either bash or ansible (have no experience with that).
i second that 6gb ram floor