As some of you have already noticed, image uploads are currently not working on lemm.ee at all. The short explanation is that I have been trying to figure out the best way to enable image uploads without incurring big additional costs in terms of storage, bandwidth, and moderation.
I have come to the conclusion that initially, the most sustainable path will be to enable image uploads only for small images (<100kb) - this should be enough to allow folks to upload avatars for themselves and their communities, but it for sure won't allow uploading many bigger images or gifs for your posts.
For bigger images, I would ask everybody to use an external image hosting service for now, as was usually done on Reddit before they added image uploads.
I am in the process of creating a patch to the image upload code that will allow all Lemmy instance admins to configure a smaller limit on the uploaded image sizes. Once that patch is finished and accepted, I will be deploying it on lemm.ee - at that point, you will be able to finally upload avatars. Sorry for the wait, hopefully it won't be much longer!
"No pornography" rule
There is a new rule landing in your sidebar - "No pornography". I was initially against adding any too-restrictive rules, but after many discussions with other instance admins, I have realized that there is a big amount of additional risk to a community when allowing pornography.
As of writing this post, there are no pornography-related communities on lemm.ee, so I'm hopeful that this rule won't be too controversial. If you were planning to create such a community on lemm.ee, I'm very sorry to ruin your plans and I hope you can find another instance that is more suitable for your community.
The rate of new users joining lemm.ee has certainly been much faster than I expected. For now, the growth has been sustainable, and we can still keep growing at this same rate for a while longer without incurring additional costs.
However, if this growth continues for an extended period of time, then additional costs will become unavoidable. At that point, we will have the option of either closing sign-ups to new users, or upgrading our infrastructure to allow continued growth. The latter can only be achieved through some kind of community funding, so I will be asking the community for input if that time ever comes.
I want to be clear about one thing: if we end up closing new sign-ups, then this will not close your communities to new members. As always, Lemmy users from any other instance will be able to subscribe to your communities on lemm.ee, they will be able to participate as normal and even become moderators. The only thing that will change if we close sign-ups is that new users will not be able to choose lemm.ee as their home instance.
That's all for now, let me know in the comments if you have any feedback or thoughts about any of this!
+1 to the no porn rule. I seriously considered hosting my own instance, but balked at the liability especially in regards to porn. I'm no prude, but having adult material seems like a serious headache.
First of all, thanks for setting up this instance!
I'd suggest to open some form of paid subscription right now, not waiting for the moment when we'll need to make hard choices.
And don't bother with asking for single-time donations - it's not sustainable with so small number of users.
Hey, thanks for creating the instance and keeping it ship shape!
Just a couple questions about timelines
When do you expect the patch for image posting might be done? - I'm not knowledgeable at all about any of this stuff, does the patch mean we'll be able to post images and they'll show up organically/embedded as part of the post? (As opposed to seeing a link to Imgur where you can't see the actual image)
Re growth of lemm.ee - How long would you expect the 'extended period of time' to be if growing rate stays constant? It's exciting to think we could be growing so much/fast!
When do you expect the patch for image posting might be done?
I'm hoping during this week, but can't guarantee anything. I'll keep you guys posted.
does the patch mean we’ll be able to post images and they’ll show up organically/embedded as part of the post?
Yes, as long as the images are <100kb 😀
How long would you expect the ‘extended period of time’ to be if growing rate stays constant?
It's difficult to predict, as we've only been live for a few days, and there are many moving parts here. We have several pieces of our infrastructure which are being affected at different rates by the growth, and only time will tell which one gets close to the limit first.
For example, I am paying for:
Storage for uploaded media
Storage for the database (all posts, user data, comments, content from other instances, etc)
Bandwidth
E-mails
Just plain server resources like CPU/memory for multiple servers
I'm keeping a close watch on resource usage for now, I'll definitely keep updating everybody as time goes by.
Ahh I see, I thought the patch would allow for bigger (say <3-5mb) images to be uploaded. Does that mean it's not in the plans at all to be able to have larger sizes uploaded?
I’m pleased about the porn, fwiw. I forgot to check prior to choosing an instance and thought I might have to move or be constantly blocking. And I didn’t want to move, because I’ve been really impressed with how you’ve implemented this.
Does anyone have a recommendation for photo hosting? Imgur, I guess? This kind of feels like a fun trip down memory lane to the early 00’s when we all had a blog and constantly lectured each other about how hotlinking was stealing :) Looks like photobucket is a paid service now, though.
Sadly, I don't know what the best alternative is. Postimages may be it for non NSFW stuff https://postimages.org/ and another option would be Catbox https://catbox.moe/
Glad to hear you're keeping up with all this new traffic!
Are you willing to talk about the costs you're incurring to run this? Or some details about your deployment? I'm considering spinning up an instance and curious to get a sense of what the overhead is like. I'm guessing you're handing this kind of traffic on...like a single EC2 instance+RDS?
Also, still trying to understand ActivePub -- is it mirroring all content from all other instances locally?
It's not on AWS, but another cloud provider. Currently it looks like this:
Cloudflare for DNS (and all the Cloudflare protections)
Load balancer
2 statless servers for the Lemmy software
1 stateful server for image upload software pict-rs (which is not yet usable by users)
s3-type object storage for uploaded media
RDS-type managed database
Terraform cloud for infrastructure orchestration
Grafana cloud for monitoring
Some mail service for sending e-mails - not settled on a provider yet, it might be Postmark, but their support has been less than helpful so far, so it might end up being something else as well
ActivePub – is it mirroring all content from all other instances locally?
Yes! Everything is cached locally, so even if other instances go down, their content will still be visible here (and vice versa).
Re: ActivePub (If I'm not imposing), what does your current storage volume/growth look like? My long-term concern was that, at some point, smaller instances with few users wouldn't be able to keep up with the cost of data growth or sync overhead. But it does at least look like media is requested from the instance it was created on no matter what instance you're browsing.
However, if this growth continues for an extended period of time
What do you expect the bottleneck to be as the platform/server grows? Is it user accounts taking up space, different people subscribing to different federated servers or the cpu/bandwidth for serving users? Or something else?
Allow me to copy my response to a very similar question from another thread:
It’s difficult to predict, as we’ve only been live for a few days, and there are many moving parts here. We have several pieces of our infrastructure which are being affected at different rates by the growth, and only time will tell which one gets close to the limit first.
For example, I am paying for:
Storage for uploaded media
Storage for the database (all posts, user data, comments, content from other instances, etc)
Bandwidth
E-mails
Just plain server resources like CPU/memory for multiple servers
I’m keeping a close watch on resource usage for now, I’ll definitely keep updating everybody as time goes by.