Lemmy Developer AMA and Dev Update, 2024-01-26, 1500 CEDT
This is a chance for any users, admins, or developers to ask anything they'd like to myself, @[email protected] , SleeplessOne , or @[email protected] about Lemmy, its future, and wider issues about the social media landscape today.
NLNet Funding
First of all some good news: We are currently applying for new funding from NLnet and have reached the second round. If it gets approved then @[email protected] and SleeplessOne will work on the paid milestones, while @dessalines and @nutomic will keep being funded by direct user donations. This will increase the number of paid Lemmy developers to four and allow for faster development.
You can see a preliminary draft for the milestones. This can give you a general idea what the development priorities will be over the next year or so. However the exact details will almost certainly change until the application process is finalized.
@dessalines has been adding moderation abilities to Jerboa, including bans, locks, removes, featured posts, and vote viewing.
In other news there will soon be a security audit of the Lemmy federation code, thanks to Radically Open Security and NLnet.
Support development
@dessalines and @nutomic are working full-time on Lemmy to integrate community contributions, fix bugs, optimize performance and much more. This work is funded exclusively through donations.
If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. Recurring donations are ideal because they allow for long-term planning. But also one-time donations of any amount help us.
Maybe a blog post like "a year in review and what's up for this year"
I'm not talking about bugs or minor tweaks. Just a general where are we, where are we coming from and where are we going to? What are important milestones?
I've just updated the post body with some updates about this, but if we get approved for another year of funding from NLNet, the the two new devs will be working on these milestones in 2024 (still a draft at this point).
Being an open source project, we can afford to be less strict about a roadmap, as anyone (including ourselves) can take on any of the open issues on the issue tracker. Part of the fun of these is getting to pick which things you'd like to work on, and that you personally think are important.
Outside of maintenance-related tasks and merging PRs (which does take a significant chunk of our time) of course @[email protected] and I both have things we'd like to prioritize this year. My main priorities are:
I think a lemmy roadmap for the next year is hard, because scope and even individual features depend on funding (for example, nlnet funds specific features).
I wouldn't put a timeline to it. Just a list of features, broad and specific. As time goes on, they can be marked as "in progress" or "included". New things can be added over time, or made more specific. All without timetables. For now call it a wishlist.
whats that thing where a company has a 'we have never been contacted by law enforement or have been forced to disclose data' sign on their website that theyll take down to implicitly inform users theyve received a request and a silencing order
So many apps die before getting any users. For Lemmy however, when was the first time you really thought "Damn, this thing really might actually take off"?
For me it was long before the reddit migration (which was ~7 months or so ago). I noticed lemmy slowly but surely gaining traction. It felt more dead than it does now, but the trend was slow and steady growth, which is always a great sign. People were using lemmy, liking it, and sticking around.
At the same time, it was clear that we weren't making the mistake of all the other reddit alternatives, by promising to be a free speech haven for bigoted communities. Those people actively did our work for us by warning their communities to stay away from Lemmy and its tankie devs, thereby making Lemmy a much more enjoyable place from the very beginning. That was a crucial test: we were not willing to sacrifice our values for growth's sake.
Lemmy was meant to be a Reddit replacement from the beginning, so it was always supposed to take off. Even in the early days the tech was working quite smoothly and users were happy so there was no real doubt about it. The only thing missing were more users. However I had no idea how a real migration would actually look like, so it was really overwhelming when last year people started to flood in and everything got overloaded and broke down.
What could be done to improve interoperability between federated platforms?
mainly talking about Mastodon since it is the biggest one.
I have seen the Peertube dev is quite nice and approachable. And willing to improve the experience cross-platform.
Have you tried to approach @[email protected]? Is he willing to contribute? How could we get Mastodon to improve the user experience with federated content, eg. communities and article posts?
There have been lots of compatibility improvements with Mastodon from our side. However Mastodon seems to have almost no interest to make improvements from their side. I dont think there is much we can do about that, in the end project maintainers always care about their own users most.
With dansup there was some communication years ago, but it seems he lost interest in Lemmy.
While I agree with the content of that article I don't know if we should give up on Eugen just yet. The Mastodon team has not disclosed what their plan is regarding the groups rework currently on the mastodon roadmap. There is an old proposal here, but I think we have good reason to believe that implementation will be revisited. To that end, it is very important to advocate for the adoption of FEP-1b12 which is the standard that Lemmy uses.
It may also be a good idea to advocate for the adoption of FEP-d36d both here and on lemmy. This is a standard for group-to-group following. Effectively allowing communities to subscribe to other communities.
I think there's a risk of lacking a coherent direction if decision-making is outsourced too much to the community. Furthermore, core developers might lose ownership of the project and then lose interest. As long as it's open source, I'm pretty happy to have the core maintainers develop projects according to their own vision, and the community fork it should this vision differ too much from their own. :)
Mastodon’s main dev isn’t really open. Have a look at the “Ego” part of this article
That article was over 5 years ago now. I would expect that there has been massive change now that Mastodon is way more popular, and the project is way more involved. Also, blocks and mutes do work now.
Interoperability is great, but sadly there isn't really any organized group effort to standardize more aspects / extensions of ActivityPub. AP is really "thin" in that it barely prescribes anything. There's not even a test suite to test whether software complies to the spec of AP.
So everyone kind of does their own thing, and fixes interoperability on a case-by-case basis. This makes it kinda frustrating to spend time on - lemmy already has special cases for many different softwares (peertube, mastodon, ...) and every one increases the complexity.
There are such efforts on SocialHub and on a W3C mailing list. However devs of major Fediverse projects are rarely active there, because they are all busy working on their own software.
Its definitely a concern. IMO the lemmyverse is far too centralized at the moment. The big questions are:
Is there a trend toward centralization, or away from it?
How are people being introduced / onboarded onto lemmy?
What can we do to combat centralization?
(1) I'm honestly unsure, and I'd def appreciate if anyone has done a study of it. We've seen a big growth in single person / smaller topic-focused instances, which is a great thing, but if their communities aren't growing, we need to figure out how to reverse that trend. I'd have no problem with the current large instances, including this one, as long as the long-term-trend is away from them.
(2) Is mostly word-of-mouth, join-lemmy.org, and apps / web-ui's which show an instance by default.
We've made the sort for the join-lemmy.org instances page be by random active users, and tried to emphasize on that page that it doesn't matter which instance you join, since most federate, and can subscribe / connect to any community. I hope that helps, and we need to replicate that wherever we can.
Apps and webUI's mostly just show lemmy.world rn, where they should show random instances. I'm guilty of this in Jerboa as well (showing lemmy.ml by default), and I've just opened up an issue that it should be showing a random server for anonymous users.
But I think we need to do more, and I'd def appreciate yours and anyone else's ideas on how we can combat centralization. We need to get ahead of this problem before it gets worse.
But I think we need to do more, and I’d def appreciate yours and anyone else’s ideas on how we can combat centralization.
I am admin of the biggest Brazilian instance, but I am welcoming more local instances and talking to the admins we should spread the load. But what I notice is the users are concerned they will miss out if they are not in an instance that already have everything.
Could we have an easier way to auto-federate every new communities from a given instance? Even an "auto-federate everything possible" option. as @[email protected] said lemmy DB isn't too big, most instance owners could have it on their servers. And making it opt-in won't hurt the small instances.
I think its totally normal that instance sizes follow a power law distribution. Its similar to many other things, for example there are few large cities, some medium cities and lots of small cities. The wiki article lists many other examples. So I think its fine as long as there are no intentional attempts to lock in users into large instances or limit federation.
The big instances are bad enough but big communities are absolute killer of decentralisation
When you go to /c/books on your server, you don't see an agglomeration of all /c/books on all servers of the fediverse. You only see that server's /c/books, if it even has one.
This is a fatal flaw of lemmy which concentrates power enormously into the hands of the owners.
The default view should be all /c/books on all federated servers, with an easy way to filter only local posts.
Lemmy will turn into reddit if this is not quickly rectified.
I kind of get where you're coming from, but to me it sounds like you're looking for a different experience than what Lemmy is designed for. It seems you are more interested in aggergating all posts about specific topics (like "books"), and strongly limiting the effect of moderation (as nobody would have final say about how to moderate an entire topic). If I correctly understood the experience you're interested in, then for sure the design of Lemmy will not match that.
I don't think it's fair to describe this as a fatal flaw, though. Lemmy is not built around the idea of generic, "ownerless" topics, instead, it's built around communities with clear owners. We have decentralization at the admin and infrastructure level (as in, a single admin does not control the entire network), but this does not really mean we also need to have it at individual community level.
IMO it's totally fine that different people create different communities with extremely similar purposes. The entire internet as a whole also works like this - the internet itself is decentralized, but at the same time people can create different websites with very similar purposes (and even domains!), and it works out fine. For example, it's totally possible for there to exist a news.com, news.co.uk, news.ee, news.fi, etc. Imagine if whenever you navigated to news.fi with your browser, it would also automatically insert content from all the other news websites of all possible domains - it doesn't really seem like a useful feature, but that's kind of analogous to what you're suggesting for Lemmy at the moment.
When you go to /c/books on your server, you don’t see an agglomeration of all /c/books on all servers of the fediverse. You only see that server’s /c/books, if it even has one.
What prevents from visiting /c/books@anotherserver?
Genuinely asking, because this is one of the core concepts of Lemmy and federation
maybe communities should be able to flag that they're the same community as one on another server, and if they mutually do so be combined into one metacommunity that people can search for
Firstly, thank you so much for providing the means for me to cut Reddit out of my life, I feel like I'm engaging with content in a much more deliberate way since, and honestly it's been a massive improvement to my mental health in a way that I was completely oblivious to there even being a problem before.
Anyway, the question—regarding things happening entirely out of your control, what would be the best and worst things that could happen to lemmy from your perspectives? And as an extension, what are your goals for it?
Thx! Its pretty wild to me how much these algorithms, and formats, affect our mental well-being. Those giant US tech companies employing Psychology PhDs to figure out how to keep people angry, engaged, and watching ads, is doing so much harm to so many people, not just in the US, but the whole world, and unfortunately very few countries are doing enough to protect their people from these companies (who also act as surveillance arms of the US state) by blocking facebook and the rest.
I've seen two professors I respected turn into angry children on twitter, in a way that would never happen in real life. Reddit, twitter, and Youtube platform reactionary rage-bait to get people trapped in a downward spiral of negativity. These companies do not care how much damage they do; all that matters to them is their profits.
We don't have those same incentive structures, so we can and should be doing everything we can to make this a positive and enjoyable experience, not about arguing constantly, but about learning, laughing, and understanding.
what would be the best and worst things that could happen to lemmy from your perspectives? And as an extension, what are your goals for it?
The best thing would be that we continue our slow and steady growth. Every user that migrates away from big tech to the fediverse is victory, so while we shouldn't emphasize growth at any cost, its still a good thing when we can get people away from all that negativity.
The biggest concern for me about Lemmy, would be a centralization onto one big server, that tries to replicate all the worst things and behaviors about reddit: its combativeness, xenophobia, bigotry, pro-US-foreign policy agendas, and advertising. There is a noticeable chunk of Lemmy's users who don't really see any problem with those things, they just want a reddit that lets them use 3rd party apps again.
The best thing would be if Reddit goes the way of Digg. Seems that will happen sooner or later. The worst thing, maybe if funding stops and we are unable to keep working on Lemmy. But even then admins could still host Lemmy instances.
The best thing would be if Reddit goes the way of Digg.
Well, it has already. The only reason it hasn't fully imploded & all the users deserted for another site, is because there wasn't an equivalent place to go to.
They were sort of parallel in development but digg blew up and Reddit didn't then Digg took a quick hard turn towards enshitification.
Reddit has done the enshitification but like a parasitic infected spider, it's wandering about and most of the users haven't realised yet that it's an empty shell.
It's slow demise would be better in the long run than a quick collapse like Diggs so it's now putrid culture is not transmitted with an enmass exodus.
Some company (dont know which) wanted to make a one-time donation of 500 Euros to get listed as donor on join-lemmy.org. Rejected because thats only for recurring donors. Does this count as weird?
also people should search the issues on github, a few of these questions already have issues filed with discussions in them, put a thumbs up on a github issue if it's something you want
Back when the first Reddit exodus happened, there was a group heavily DDOSing many of the popular Lemmy instances. While it was a great opportunity to optimize Lemmy, did you ever find out who that attacker was?
I don't think we found any specific groups of people attacking Lemmy. I personally just saw one or two what looked like individuals trying (and succeeding) to take Lemmy down with a few very simple requests that forced Lemmy to do lots of compute (something like fetching the next million posts from page 10000). The fixes for those were simple because it was just missing limits checking.
I'm not sure if there actually was a larger organized attack. Lots of performance issues in Lemmy simply appeared simultaneously and compunded each other with a rapidly growing number of active users and posts.
First, I want to say thank you for the incredible job you already have done in this area. However, do you have any thoughts on further improving some fundamental Lemmy UX painpoints? Examples such as:
That would be awesome if true. It's progressing faster than I thought. I'm still just learning about the scaled sort and enjoying that new feature lol.
As @[email protected] mentioned we have improvements coming down the pipe for linking content across instances.
Community linking and user linking do work currently (for example I just linked phiresky above), and a community example would be [email protected] , but we could improve this by extending it to posts and comments, as well as creating a url link standard that would work across apps.
Can you add one to your list? Linking posts across instances? Like you can do !community@instance and the community will open viewed through your instance. But for linking posts there is no such equivalent. Like if I make an HTTP link it will be through my instance or possibly the one the community is hosted on which would be annoying for users of other instances.
Also, linking communities across instances is possible already, but you can leave it up since it's confusing. I still see a lot of folks try to do the reddit approach if c/community
Turning the fediverse button into an "open on my instance" with similar functionality to subscribing may also be a solution here. Bonus points if it'll also open a comment on mastodon.
For migration we recently added a feature to export your user data. But "real" migrating accounts is something I put on our "todo" list, though it probably also first needs a proposal to define how it should work exactly (should it still work when the original instance is down?) As soon as we start giving users more control over their private key issues start appearing like not having any infrastructure for key rotation / revocation. Without that it will only work when the original instance still exists.
I'm not sure if by tagging users you mean linking / mentioning them? Or adding tags to them like you can tag posts / users on other platform. For tagging in general there's a pending proposal https://github.com/LemmyNet/rfcs/pull/4 . So far it focuses on post tagging though to reduce the scope. I think the goal is going to be to start with one kind of tagging and add more kinds of tagging later.
For improving cross-instance linking (both communities, posts, and users) we also have a open milestone. There's a few spitballing issues about it, but no real concrete proposal on how to build it yet.
What happened with the domain Lemmy.ml when Mali took back controls over its domains and some sites went offline? Are you confident that the .ml domain will be reliable in the future?
We're paid up on our current registrar for a few more years, but I honestly don't know how much that can be trusted. Its pretty difficult to get clear information on which ones are safe, and which ones aren't.
If lemmy.ml goes down, it will be a major annoyance as we migrate domains, and will likely have a day or two of downtime. Even outside of DNS yanks, we do daily DB backups, and have our pictures backed up locally as well, so a full restore is possible, but federation would definitely have issues, and would probably need to do a lot of re-subscribes. Overall tho, it wouldn't destroy the lemmy project or anything, and if people migrate to smaller servers in the process, that would give us a lot more time to code 😄 .
I apologize for that tho , it was something I didn't forsee and wasn't even thought of many years ago when I registered.
Have you considered migrating while you still have control of the domain? You could at least make the domain redirect in that case. Though not sure if federation could somehow pick up that lemmy.ml has moved elsewhere.
I'd be very curious how you guys migrate to a new domain if you ever need to, maybe there will be an open source tool to help with it, and maybe some improvements inside Lemmy's code, like api calls from both domains to initiate the migration
Instance owners currently gets notified when someone has reported a user for spamming or trolling, but frequently it's a user that is not on his instance, so he can't do anything about it. Wouldn't it be better if instance owners got notified only when they can take actual action (like the user being registered on their instance)?
If you've been following our code commits / PRs, we've been adding a lot of mod tools improvements not just lately, but over lemmy's entire life. I would even go so far as to say we have the strongest mod tools of any project in the fediverse, all the more necessary for us because of the community-focus.
Instance owners currently gets notified when someone has reported a user for spamming or trolling, but frequently it’s a user that is not on his instance, so he can’t do anything about it. Wouldn’t it be better if instance owners got notified only when they can take actual action (like the user being registered on their instance)?
Instance admins are responsible for what content their users see, so if a troll is visible to their users and ruining their day, then it should be taken care of everywhere necessary.
I have seen this first hand. I think when someone hits report it needs to go to the moderator of the community. From there the mod should be able to forward it to where it needs to go.
Instance admins should be able to intersect this process.
There's some more context for this in this issue, and we each have different views on it, because there are tradeoffs no matter what.
My personal view (based on experience modding and admin'ing), is that we should prioritize handling a report ASAP, by the first eyeballs that see it, rather than whose jurisdiction it is. On all but the largest communities, admins are generally more active, and more likely to see the report and take action on it.
Check the updated OP. We can definitely use more donations, at the moment we are getting around 4000 Euros per month which is not much for two fulltime devs. And code contributions are also helpful, there is an almost endless amount of open issues.
Thank you! Lemmy is a tremendous contribution to the wider Fediverse, and no amount of "thank yous" is ever enough for people like you writing free software and giving freely to the public domain.
I have been on Lemmy, and around the Fediverse on various accounts since ~2021, and a suggestion I have seen promoted countless times is for communities which federate across instances. e.g. posts to [email protected] will show on [email protected] as long as lemmy.ml and lemmy.world federate with one another. If I remember correctly, each of you have previously opposed this idea for multiple reasons. If you do still oppose such a feature, will you please reiterate why you think this is the wrong direction for Lemmy? Also, have you considered adding a multi-community feature similar to Reddit's multi-reddit feature which allows end-users to combine multiple federated communities into a single page just for them?
I see why people think that's a problem, but in reality, its more of a feature. For example, take communities named !news, that pertain to completely different topics, or locations, based on their instance:
These are all news communities, yet should stand on their own: each with their own creators, moderators, users, rules, posts, comments, culture, and topics. It makes no sense to combine or "merge" them given all these differences, in the same way that it makes no sense to "merge" two completely different users because they have the same name.
Also, have you considered adding a multi-community feature similar to Reddit’s multi-reddit feature which allows end-users to combine multiple federated communities into a single page just for them?
Sure! Multi-Communities are an open issue, that I'm sure someone will take on eventually.
Hi there! Looks like you linked to a Lemmy community using a URL instead of its name, which doesn't work well for people on different instances. Try fixing it like this: [email protected]e
Thank you for making it clear this is a conscious design decision. I very much disagree with it. Multicommunities, like multireddit do not adress this terrible problem. I think lemmy is now doomed to repeat reddit's mistake. Hopefully in 10-15 years the lemmy successor will see the light about this problem.
I'm not aware of philosophical disagreements with that feature, I can just think of logical and technical issues. Like how moderation would federate, etc. If all the mods come to an agreement then the mods on one instance could lock their community and link to the other one. If the mods disagree, then moderation is going to be chaos in any case, no?
I think multi-community views would be a great idea.
With the web being so ad-infested and completely owned by google, people have noted how the TLDR bot means they often don't have to leave their lemmy app at all, and can stay behind its privacy shield.
While of course I do think we can code a lot of functionality directly in to lemmy in a way that we couldn't with reddit, there's undeniably a lot of potential with bots that can do different things for us.
I linked a timezone convert link (before I updated the post), which I think I'd have to do even if we used the UTC offset format. I must be just far away enough from UTC to not know what my offset is at any given time.
Time math gets a bit difficult far enough from UTC. Where I live virtually any event in Europe or Asia will be happening on a different day there than here, so it's not fun to try and figure in one's head.
The only universal solution is to link to a converter site.
Personally I wish everything supported the automaticly-converting timestamps I've seen in Discord which just show up in local time or as a countdown.
There was a big time gap between 0.18.5 and 0.19. Have you considered adopting a release train model, similar to what Rust does? The Bevy game engine has also adopted the idea.
More frequent but smaller releases would probably cause less friction and make upgrading less of a "big thing" and "big things" are always where things go wrong.
0.19 was a bit of a special case because there was a set of breaking updates that had to be done at some point, and trickle releasing breaking changes isn't really great either. Usually hopefully the breaking changes are rare, so releases can be more frequent.
They normally do have smaller releases (18.1, 18.2, 18.3, 18.4, 18.5) but going from 18 to 19 was a big update that also required a database upgrade. Rust releases don't have database upgrades or anything that is not backwards compatible, so it's not really comparable.
I think they would, it would be super cool to do art competitions and have the community pick the designs, could do it once a quarter to help boost funding.
I’d be down for an enamel pin. I’m sure you guys are familiar with the Apollo app (RIP), but in his merch store he had enamel pins made in the style of some of his app icons.
To add, recurring donations, no matter how small, help us plan for the future, as we can then reliably estimate how many developers we can support off them. One-offs donations and merch sales wouldn't help us out in that regard.
Excited, but also extremely stressed out and exhausted. For about 2 months I was getting an average of 4 hours of consistent sleep a night after that happened. We were very happy when things calmed down.
How's development going? Do you have enough funds to pay your salaries? Did the EU fund run out? What's your workload? Is the amount of full-time developers enough to work on new features? Or is it barely enough to keep up?
How do you like Lemmy and the people on it? (As of now)
We are getting about 4000 Euros per month which is not much to pay for two developers, so more donations would definitely be nice. From NLnet Dessalines and I still have a few milestones leftover from 2022 but those should be finished very soon. We could definitely use more developers, its impossible to keep up with all the issues so we have to try and prioritize the most important ones.
The people on Lemmy are generally very nice, so I cant complain.
I don't think it's that large. Text is very small and compressible compared to images. Well it depends on if you mean the actual database storage (uncompressed, with indexes) or a compressed copy of all the posts. You can see the post number in the URL, which on lemmy.world for this post is 11169622. That means there's around 11 million posts total in lemmy.world's database. If you assume each of them takes 0.5kB of storage that would be only ~ 5 GB of posts.
Strengths are federation, comment threads, mod tools, app ecosystem.
Weaknesses are lack of notifications, DB code, and lack of funding / donations. We need to support much more that just one or two devs if we want to take on reddit, which is a multi-billion dollar company with hundreds of employees.
A long time ago, @[email protected] made Jerboa as an Android Native client for Lemmy as an alternative to Boost for Reddit. How happy are you that the OG Boost developer came and made a Lemmy client?
I post a fair amount of video edits. I've had quite a few people say that video playback is far from ideal for not just Lemmy, but the Fediverse as a whole. Is this mostly a 3rd party app thing, or a backend issue? I haven't had much issue myself, but enough people have mentioned it that there is likely an issue somewhere down the line.
Will private messages ever be displayed in a threaded or grouped manner?
Please correct me if I'm wrong, but the current web interface is just a reverse chronological list of all sent and received messages. This can be confusing to follow if one is messaging multiple users over an extended period of time. I think the ability to group messages by user would be useful.
There can definitely be improvements, but I agree that they're low priority. I'm hesitant to put too much work into Lemmy's private messages or its interface, as they're inherently insecure and not E2EE (we even have warnings in the lemmy-ui interface about this).
Its best to rely on messaging apps like matrix and xmpp that were made for that job, which you can add to your profile. We also added a specific matrix_id field to your user settings, which enables a "Send secure message" button.
This is already possible in many Lemmy apps. I built the Quiblr web app and messages are more similar to a messaging app. Many Lemmy apps probably do similar as it's just a front-end change.
Cool! I haven't tried out web apps, but many mobile apps have this feature as well. Would be nice to see this feature merged into the native web interface.
So far it doesn't seem like any company actually wants to compete in this space (longer-form somewhat text-focused communities). Even reddit is trying to become more twitter and less reddit.
There is a ton of decentralized projects that no one has really ever heard of, new ones pop up all the time (I was watching multiple of them in the past). Sadly in most cases it seems like most authors stop working on their projects after a while.
The same ideas have existed for a long time but both decade old projects (ever heard of Freenet? Probably no) and new ones . Many of them are very ambitious and try to replace huge swaths of things (not just file storage but also social aspects, web of trust, etc) but then collapse under the complexity. IPFS is the most well known new project and (good imo) has limited its scope, but sadly (still) suffers from huge scalability issues, some of which are deep in the design.
I think it's really hard to align incentives there - the nicer it is the harder it is to make money with it. So either these projects tend towards control by one entity or they tend towards death.
Really the only one that seems to have a long lasting life so far is torrents. Which are amazing. And Email if you want to count that.
Torrents are truly one of the best inventions of the internet.
They've fully solved the static data distribution problem, in a way that's resilient and practical. I do a few torrent-related side projects, and I'm also super-interested in how we could integrate them into lemmy UIs and apps in order to take on YouTube.
There are different software that could help this happen but I'm not sure how they'd handle all of lemmy's content
There is Gun.js and OrbitDB (OrbitDB runs on IPFS using Helia)
and there is FreeNet which allows hosting of freesites on their network if you have something downloaded to your computer.
Some of the problems could be mitigated by volunteers running relay nodes hosting the kind of content they want to stay up (this would work with Gun.js and OrbitDB since it allows subscribing to specific data) or just general relay nodes that gives the network more capacity (this would be the method if you are working with FreeNet)
Again not sure how easy or hard this would be so I hope the developers share their thoughts.
Edit: When I say relay nodes hosting I don't mean they have control over the data but instead keeps the data online. The data is still stored on the users own device and is shared between users if both are online at the same time.
Since I read a few comments here... What is your oppinion on more democratic platforms? I mean something like electing moderators. (Or dropping them in a democratic process.) Or voting for other things in a community.
(This is more a hypothetical question. I guess with the architecture as is, it can easily be exploited. And there is no way to implement this properly without severe changes and consequences.)
We've talked about it elsewhere, on github and one of these Q and A's, but the main thing is that these things can be easily gamed. IE: making a ton of fake users to vote for a malicious mod.
There's a ton of ways that could be exploited that I can't think of rn. I'd rather not stress test them in lemmy.
Go through the issue tracker for lemmy or lemmy-ui and look for some simple bug or minor feature that you care about. Then look for the relevant part of the code and try to fix it. You can also make a comment or post in the dev chat on matrix if you need help. Honestly there are so many issues which could be solved in less than an hour, especially in lemmy-ui. That way you can make Lemmy better and also get familiar with the code to make larger changes in the future.
First, thx for working on LemmyNade! The ecosystem of apps growing up around lemmy, learning from and benefitting each other, is really great to see.
The main thing would be to get involved with lemmy's back-end code. Even if you're not a back-end developer, its still useful to us to learn from devs about wanted features, API improvements, and bugs. Many app devs have suggested features that I've tried to implement based on their usefulness, because it used to be just myself who was the one requesting and adding things on both sides.
Well I am not 100% sure on what the best method would be.
I think the addition of letting moderators see who is voting on what posts is a nice addition since they would tend to be closer to where the content is.
More generally I was thinking about creating a dashboard that gave the admins statistics about that user.
Similar to how we as users can see how many comments and posts any user makes, it might be useful to allow admins to see statistics about which communities the user interacts with or some rudimentary similarity score with other users?
These are some ideas I thought of on the fly and have not thought about the implications but the gist is something that allows the admins to get a better understanding of what is going on from like a bird eye's view.
Registration applications, and user reports are the best way to handle trolls. The first stops 90% of them, the second means we can ban and remove all their spam at the click of a button.
I don't see how you could prematurely know about spammers or trolls until someone reports them. We don't plan on adding any text-analyzing AI or anything like that into lemmy's codebase.
I don’t see how you could prematurely know about spammers or trolls until someone reports them.
I don't think you can. My suggestion was more focused on how admins make decisions after a report.
Right now they have to do a manual scan of the person's comment history and that is the part I find inefficient.
If it was possible to just show extra high level information on the user it might make it easier for the admin to make a decision.
We don’t plan on adding any text-analyzing AI or anything like that into lemmy’s codebase.
Yeah using AI to try and analyze comments would be overkill and probably prone to manipulation anyways.
Edit: I'm sorta talking more specifically towards banning a user or seeing if what a user is doing is a repeated pattern.
We could definitely use some help with ideas there. Lemmy currently has ~40k active users, and it should be able to support more than ~1 average dev salary, especially if we want to take on a multi-billion dollar company with hundreds of employees like reddit.
I think short event or campaign with push for donations with a pop up that you actually can dismiss. An ad like banner. The biggest problem would be community organization as Lemmy isn't only decentralized horizontally but also vertically. Different front ends, different apps different instances. Most of them wouldn't want to implement an ad that wouldn't benefit them directly. They also have costs with running their piece of lemmy. So some cut for them should be included.
I think a dedicated trustworthy person should be responsible for organizing this campaign as developer time is best spent elsewhere.
Wikipedia, about once a year, has those donation pledge boxes at the top of every article ...they must be somewhat effective since they come back year after year ...better keep them small though to avoid disheartening users. Maybe start small like this trending community line at the top of the user feed.
P.S. : Since we don't want the user to get habituated it's better if it's just a few days once a year.
I think flairs would be the same as user-tagging. There's an open proposal for post-tagging https://github.com/LemmyNet/rfcs/pull/4 and the discussion there was so far to add tagging for one type of thing and then later expand to others (like user tagging).
It's a bit of a complicated feature because it needs decision who can tag whom, and what is the scope (who is it federated two), and how does it transfer / interact with other ActivityPub software.
First of thanks a lot for the effort that you put into creating lemmy. You have created a really friendly and welcoming place!
I have a question regarding licenses. When you started developing lemmy, what were the reasons for your choice of the AGPL? As you are marxist-leninists, did you also look into other licenses like the the Anti-Capitalist Software License?
AGPL was already used by most existing Fediverse platforms, and ensures that all code changes need to be published. Its basically an improvement over GPL which also takes effect when the software is hosted on a server, not running on the user's computer.
The Anti-Capitalist Software License is not an open source software license.
To me, AGPL is the most pragmatic choice. As a hard copy-left license, it enforces derivative works to adopt the same license, unlike the more open and "soft" copy-left licenses that let corporations capture and digitally enclose your labor as they see fit.
Will the source code ever move off of proprietary Microsoft GitHub where users need to have an account to contribute & search code—or certain users are blocked due to US sanctions? If the idea is wanting to stand up against centralized US-corpo-controlled social media for forums, why use that US-megacorpate-controlled code forge / social media platform?
So far these problems are mostly theoretical, in practice Github works fine. But once Forgejo gets federation working we will probably migrate to a selfhosted instance.
I agree that it's not ideal to be hosted on a platform controlled by Microsoft, but it's just a fact that you lose 90+% of contributors if you are anywhere else (there's an article where someone compared, can't find it right now). It's not great that that's how it is, but you need to choose your battles.
I'm not really very concerned, since git itself is decentralized, and if Github starts causing visible problems moving somewhere else is not a huge problem. Also VPNs exist.
Yeah, tbh the worst vendor lock-in part of Github (edit: other than the aformentioned social aspect) is Github Releases. And Lemmy doesn't really use them.
Its unfortunate that we (and it seems like 99% of other Rust projects), do their issue tracking on github. We have multiple mirrors set up for Lemmy, so the code is safe from takedowns, but the issue tracker is a concern.
The main issue I've had is: if we migrate, I want that migration to be permanent, and for me a requirement for that is federated collaboration. I've had codeberg remove a torrent project of mine to comply with German law, and gitlab has most of the same problems of github. Self-hosted gitea instances work, but many people just don't contribute to them when they have to make an account on each one.
You'll see below that Lemmy's two main devs are in favor of migrating our issue tracking to forgejo, once federation gets reliably up and running.
I think it's fine. You won't influence anything about US by using a worse collaboration platform. If Lemmy becomes more and more popular, it may eventually become a real alternative for the masses. That's what the goal should be - not avoiding github.
Some alternative forges offer better features (or less social cruft) & are faster (some are not even limited to Git!)… what you will get is the ability to own the code & community along with set the terms instead of letting Microsoft set them for your community & be the gatekeeper for who gets to have access. If you wanted a corporate, centralized, proprietary forum go back to Reddit/Twitter; if you think that’s a terrible recommendation, in the same spirit you should leave behind corporate, centralized, proprietary code forges.
They were posting spicy memes but thats how the internet works. If you dont like it then dont visit there, just like you wouldnt visit 4chan. Lemmy is open source so anyone can use it for any purpose.
I very much dislike it obviously, and I'm happy that one shut down. There have been others, but for the most part they've stayed away from Lemmy as "that software made by tankies."
Outside of making sure that we don't platform them anywhere, there isn't much we can do. Lemmy is open-source software after all, and a tool can be used for good or ill. As @[email protected] mentioned, coordinating on adding them to our blocklists and isolating them is the best option.
I hope they're not too against it. I know they're extremely left wing, which scared a lot of the centrists on Reddit. If they allow even right wing instances, then it emphasizes the project's lakc of political bent, and encourages more mainstream people to join. The politics can be up to each individual instance to decide whether to defederate with those other instances or not.
But that's just my opinion, I'm also curious how the devs will answer.
I'm sure they're probably not okay with it but also there's not much they can do about it other than defederate .ml - such is the nature of open source software.
Have you put measures into place to assure the quality of future updates? In the past several updates have caused issues. And recently 0.19.x broke federation for the most of us. And it took weeks to fix it and make Lemmy usable again.
We publish multiple release candidates and run them on lemmy.ml before the final release. That allows the community to test changes. We dont have a quality assurance team, and developers are notoriously bad at testing their own code, so I dont see what we can improve in this regard.
developers are notoriously bad at testing their own code, so I dont see what we can improve in this regard.
Sounds like software development... I mean automated tests help. But you're developing a distributed/federated platform. Unit tests won't do it. Maybe infrastructure that spins up a small fleet of instances and checks their ability to federate posts, delete comments and simulates interaction. That'd assure the most important aspects keep working. And I think there are tools for that available. But I get it. It's complicated, there are real-world instances with special (niche) setups, you're constrained, it has to be worth the effort and there are other important things to do.
Maybe just do your best not to break too many things and we (users) can complain and have another discussion only if it's a reoccurring problem. 😉
Jerboa has another major contributor now besides me. But I'd also like to keep a hand in it, simply because I made it and its also the main app I use for lemmy. Besides a few other people, no one has stepped up to get involved with it enough that I'd. Plus I'd eventually like to add it to the LemmyNet repo, as it will soon implement all of the features in lemmy-ui, making it an "official" and native app.
Reddit has been banning communist / leftist communities for a while, so a lot of us were looking at different reddit alternatives. I found that none of them had a great tech stack, none of them had federation, and a few were unwelcoming to contributions. It made sense to do a "what if reddit, but open source, made in rust, and self-hostable". It was a great opportunity to teach myself rust also.
The fact that there is no boss telling me what to work on. Instead I get to decide myself whats most important. Last year before the Reddit migration I was temporarily working for a company, and it was extremely demotivating to be told how to do every little thing as if I were a junior developer.
As a software dev, mainly that people enjoy using it.
99% of the proprietary software work I did for companies was work that was societally useless, and eventually thrown in the trash. Here I get to make software that improves peoples lives in a tiny way, and is a form of social media that hopefully 🤞 doesn't destroy people's mental well-being: is easy to put down, and enjoyable to use.
Are there any plans on adding features that enable easier interaction with other federated platforms like mastodon and peertube (for example being able to comment/interact with peertube videos and mastodon posts)?
You can already interact with Peertube videos and follow their channels. Thats possible because Peertube also federates groups (communities). With Mastodon thats not possible because it doesnt have groups, and Lemmy doesnt support content outside of communities. At least not without a full rewrite, which doesnt make sense considering that KBin and dozens of different microblogging platforms already exist.
Will Lemmy ever become more of an organization? I'm slightly concerned about hostile take overs and or major changes that could be driven by personal views or bias.
Also a organization could facilitate cooperation and organize events.
If Lemmy does become more of an organization, it would be nice to have a level of public assurance over any control exerted by the organization. A lot of people see that the lead developers of Lemmy are communist and shy away from it based merely on that. I have one of the oldest accounts on Lemmy, I've seen plenty of them, and my impression is that they have conducted themselves with only the utmost ethics. However, it can still help newcomers who don't want to feel like someone might be breathing down their neck.
Lemmy is somewhat protected by being an AGPL-licensed project, preventing proprietarization. If there's ever a relicensing effort, ba fearful.
I'm not sure what exactly becoming a organization would entail, but so far I'd say the development part is not really large enough? For me I would start being suspicious when a significant amount of dev power came from compan(ies), but so far no company has shown any interest afaik.
There's already been a few forks, for example lemmynsfw has made some changes on their side, which nutomic is now looking to integrate back into lemmy.
As a multi-national open source dev team, it would only complicate our lives to try to set up a more formal legal structure.
I wouldn't be too afraid of hostile takeovers: this is a dev-run-and-controlled project. People will go where the development is, and the federated nature of lemmy protects against the kind of attacks its possible to make against centralized entities.
When and how are you going to address the thousands of open issues in the Github repository, that contain UI bugs, missing error messages (something looks as if it was sent for example if you send a direct message with too many characters, but actually isn't), backend issues and other assorted bugs?
When we have about a dozen more developers. So far only dessalines and i work on Lemmy fulltime, and besides solving issues we also have to review pull requests, prepare releases and much more. So its just not enough time to keep up with all the new issues let alone resolve the whole backlog.
Thank you. A follow-up question: You sound like most things have to be done by full-time developers. Is there a healthy open-source community around Lemmy development? Do people submit enough pull-requests to fix bugs? Do people from the community contribute a substancial amount? features?
Where is the best place to propose new features for Lemmy?
Edit: And as potential follow-up, where is the best place on Lemmy to propose new features for Lemmy? (Not every Lemmy user has or wants a GitHub account)
if you don't want to make a github account, then probably [email protected]
but you can still search and read the github issues without making an account, so you could check if it already exists, or link the issue if you're starting a lemmy discussion about it
Regarding server architecture - How many users can the Lemmy network, or the fediverse as a total scale to, assuming the average person posts once per day and reads ~50 comments/posts a day?
Lemmy supports horizontal scaling, so in theory it is only limited by the amount of servers you can afford. Of course there are always unpredictable bottlenecks which need to be fixed, but no inherent limitation.
The ActivityPub protocol lemmy uses is (in my opinion) really bad wrt scalability. For example, if you press one upvote, your instance has to make 3000 HTTP requests (one to every instance that cares).
But on the other hand, I recently rewrote the federation queue. Looking at reddit, it has around 100 actions per second. The new queue should be able to handle that amount of requests, and PostgreSQL can handle it (the incoming side) as well.
The problem right now is more that people running instances don't have infinite money, so even if you could in theory host hundreds of millions of users most instances are limited by having a budget of 10-100$ per month.
In the future it could make sense to make a protocol extension to send multiple activities in a single HTTP connection. But for now its probably not worth the effort, considering that it would break compatibility with other Fediverse platforms.
I mainly like Hip-hop, but also good music from other genres. I like all sorts of movies if they are well made, but especially adventure and comedy. Since the quality of Hollywood movies has gone steeply downhill, I mainly watch movies from different countries across the world, and older movies from the 70s or so. I also like video games, at the moment Im playing Baldurs Gate 3 as everyone was praising it on Lemmy (and they were right its a great game).
I'm sry I can't give a good answer there... but to me it seems like when we go a certain length of time (not sure how long) without any breaking changes. That will have proven that the API as it exists is stable and well-formed.
That seems a long way off atm, because of all the features and new fields we've been adding and modifying on existing data structures.
Overkill for me, but to me Motorhead's also one of those singles-type bands where the songs kind of stand individually great on their own, rather than the album. So I usually just relisten to a ~40 song greatest hits of theirs.
Is federated authentication being considered for the future? The federated model of the fediverse is great, but it runs into problems when instances “die”, you want to access different servers as they federate with different things, etc. leading to the need of having multiple accounts. If there were a decentralized network of auth servers, could use the same credentials everywhere.
No, Activitypub isn't based on a migrating authentication / single-sign-on model, but on server-to-server communication.
Instances with many users already on them dying should be a rare occurrence, and its unfortunate that it happens not just for us, but in the whole fediverse.
Lemmy is licensed under AGPL which states that everyone can use it, there are no restrictions based on politics. Besides, "right wing" is not the same as "evil", the real world is much more complicated than that. If you ask me, the whole right-left classification is way too simplistic and doesnt make sense anymore (if it ever did).
I'm not a Lemmy dev but I'm interested in this question so I'm commenting so I remember to check up on this one.
I subscribe to that sub because I feel like it's important to engage with people that I don't agree with. Even though the two main contributors to that sub are peculiar in their views, I haven't seen them break any rules of lemm.ee or post outright hate to any group but democrats.
I know sh.it.justworks had their own drama with a the_donald popup community which led to calls for defederation before the community was banned but if people are posting within the rules and properly moderate their own; we ought to let them post their politics even if we don't agree.
I think the instance admins should handle that. Lemmy itself should be a open and agnostic platform. Admins should use defederation and block specific communities.
(My oppinion, I'm not associated with Lemmy development.)
What's wrong with a "right-wing" community? I'm not right wing but I'm definitely not afraid of them. Anything can be solved with education. Why would you censor based on what side of the political spectrum you're on?
When will there be default view agglomeration of posts sent to identically named communities. For example /c/books.
The current setup cntralizes power into the hands of whoever gets traction first on the platform.
If I go to /c/books on any server, all posts of all federated servers' /c/books should be visible. This way no server owner gets the stranglehold on the community that they host.
Lemmygrad still can send all the kulaks to the gulags. But only when the discussion happening inside their hard drive. Aka "I take my ball and go home"
They do not get to silence the rest of the fediverse/c/books
As I understand your suggestion this would mean one super community might get moderated from 5 different instances and 5 rule sets. It is definitely the right direction but not that easy to design..
Posts are moderated by the delegates of the owner of the hard drive who stores them.
Rule sets are irrelevent make belief justifucation for censors.
I just had a comment deleted "bevayse if rule 3".
I hope you can see with this farcical example that rules do not matter , never have mattered and will never matter. It just the powerful telling you "because I told you so" with extra steps and while giving them the feeling that they are not a bad person.
If it is not the default and automatic, then lemmy is a pointless reddit clone.
You have to filter out what you don't want because it is not possible to undelete what has already been deleted.
Users will just circulate ready made blacklists of spammer and thoughtcriminal communities to automatically remove them all from their feed.
The alternative is that only the biggest instance and the biggest community will matter and writing everywhere else is just a exercise in pointlessness
And on Lemmy some instances duplicate everything. For example beehaw
Are they not allowed to?
Beehaw exists for people who wanted a heavily-moderated space, and they seem to be doing well activity-wise. Do you want to force them with the rest of the instances?
Thank you for your work or this reply. However I do not believe a multireddit analog can solve this issue. As it would not be the default, anyone "escaping" with a multireddit would still find themselves invisible to the larger community who does not use it or even know multicommunities exist or remember to use it for that specific community.
Only a system that shows it all, that user then filter out with shared blacklists, can break the tyranny of moderators.
It becomes a much more acceptable tyranny of the majority, which is only optionnally followed by members.
When do we get advanced moderation features? And for example the ability to block all users from a single instance to prevent for example brigading? I mean for the user, so we don't have to rely on defederation so much.
Are you planning to revamp defederation? I mean it's rather complicated the way it works and the triangle that is the user's instance, the other user's instance and the instance the community is located.
What about features like automatically kicking of moderators / revoking their ownership. In the early days of the Reddit exodus, some people reserved lots of communities just so they'd be the owner of the community, but they don't do anything with it. I think admins mostly already dealt with that. But there are ideas floating around to migitate for things like that and other common annoyances. I think good moderation is key (and the tools that go with that and the whole architecture of the platform should favor a good atmosphere.)
When do we get advanced moderation features? And for example the ability to block all users from a single instance to prevent for example brigading? I mean for the user, so we don’t have to rely on defederation so much.
This could be added to the existing instance block feature, but so far no one has even bothered to open an issue I think.
Are you planning to revamp defederation? I mean it’s rather complicated the way it works and the triangle that is the user’s instance, the other user’s instance and the instance the community is located.
Its very simple and effective in that in prevents all network connections to the blocked instance. So I dont think it makes sense to change that, but other tools can be added on top for more fine-grained restrictions (eg user-level instance blocks in 0.19).
Can there be an option to view deleted/removed comments? (this setting a user could configure in settings)
I can understand why mods want to remove comments, but being able to see the text that was removed could be very useful. At the moment, the only way to do this would be having each post uploaded to internet archive, and hope many other lemmy users do the same thing.