Niche Communities won't be able to reach their true potential until lemmy adds a sort that takes engagement into account.
Niche Communities won't be able to reach their true potential until lemmy adds a sort that takes engagement into account.
No matter which sort you use (except for new), content is recommended to you by activity. Depending on the sort (active, hot, top) it uses a slightly different mixture of votes/comments/time since post to determine the order.
The only exception is scaled, which boosts a little bit midsized communities, but still doesn’t manage to improve visibility of niche ones.
If lemmy is to truly start having active hobbyist communities instead of being 95% lefty US politics, Shitposts, and some tech stuff, it needs a sort that takes into account the user’s engagement.
For example, if I upvote / comment often in a community, there should be an option to have posts from the community be boosted in my feed, even if it’s a tiny community.
Let’s say I’m subscribed to [email protected] and [email protected] because I want to occasionally see news. However, I’m also subscribed to a couple hundred other communities, some of them who don’t manage to get more than a couple upvotes on their biggest posts. And whenever I see them I’m replying/upvoting because I’m passionate about that topic.
My feed shouldn’t be 95% c/news and c/world because those are the most upvoted and commented. I shouldn’t have to scroll down hundreds of posts to find “big” posts in small communities I interact with at any opportunity I get.
That’s why I think it would be beneficial to lemmy if the sort/algorithm took into account your engagement in a way.
It doesn’t have to be complicated, you can have a single number “engagement score” for every community calculated with a basic formula, and that number is used as a boost to the community.
I’m aware that there are some examples of successful niche communities on lemmy. But that’s mainly because either a significant chunk of the lemmy userbase is into that niche (let’s face it the lemmy community is not a representative sample of the world population, we tend to be very similar people), or because the posts on it are simplified image/video type posts which appeal to people who don’t know much about the subject.
Isn't that the difference between "active" and "hot?"
One is just upvotes; the other is votes and comments.
I don't think it would help, to be honest. A dead community not being posted to still won't show up on your feed if you sort by engagement.
read my post, not just the title.
I agree that fediverse needs a personalized "explore" page in general. For example, this is the only plus feature of Bluesky over Mastodon (in terms of technology). It is obvious how big difference it makes.
I generally avoid the evil algorithms found on other social networks, but I hope we see that in Lemmy.
Good algorithm should:
- be open-source for users
- be togglable
- boost engagement boost on positive emotions
- be personalized
- promote niche communities
- promote balanced political debate (probably the hardest)
I think to trying meet all of these without compromises (such as privacy and performance) is basically impossible. How would one boost engagement on positive emotions or personalize without large data mining efforts, model building, and running text classification on every comment or post?
I agree they are good aspirations.
Blackbox echo chamber generators really should be avoided. They add to the angst and anger of the Internet, and of society.
Community search could be improved. And people should learn to actually use it, rather than being spoon fed whatever some programmer they've never met thinks they should eat based on the last 3 things they clicked on.
Mastodon has a pseudo-algorithm (sort of, unless there's a better name) in the Search/Explore -> Posts view, which shows you posts that are trending on your instance. I wonder if Lemmy can come up with something similar based on your instance activity
The biggest problem with lemmy for me is the multiple "duplicate" communities.
There should be a feature to combine them at the client level. So the 3 different "privacy" communities could just be viewed as one on my lemmy client
The "duplicate" communities are housed on different websites. Websites that could very well have their own norms, rules, and culture. Lumping them together and treating them as the same thing is just kind of invasive to them, and promotes bad netiquette.
Just pick one that you like best.
Nah. The different character of the communities and their history makes them unique and special, hiding that for broad appeal is unnecessary.
No need to muddy the waters with weird client-side obfuscations, one big one almost always wins and the other gets reposts, while subscribing to both is trivial if one wishes
The balkanization is a massive problem though because instead of one, active, community we have 3 or 4 dead ones. There needs to be a critical mass of users before communities can afford to start splintering, and that just isn't here.
A few apps have multi-community support where you can group whatever you want, how you want, in one stream. I'm using Summit, but I feel a few other of the bigger apps support it now. I group the AskLemmys, tv/movie communities and different art communities into groups so I can view by category.
Multicommunities would also help with that. News communities would not flood your general feed anymore if you were able to have a specific feed for them
This sounds like the sort of thing that's best solved with a 'favourite' option that pushes posts from favourited communities to the top of the feed. No need to get in there and over-complicate it with bespoke weightings or anything.
You are overcomplicating the issue by suggesting a "favorite" option when there is already a "subscribe" option. At the very least, consider proposing something distinct that helps users discover more of the small communities they are subscribed to, rather than suggesting something that has already been implemented.
Scaled sort was implemented in 0.19.0, is that not sufficient? https://join-lemmy.org/news/2023-12-15_-\_Lemmy_Release_v0.19.0_-_Instance_blocking,_Scaled_sort,_and_Federation_Queue
I mentioned scaled sort in my post. Yes it boosts communities with less activity (in practice this tends to be midsized communities as I mentioned in my post), but it does so generally. What my post is advocating for is a sort that boosts the communities you tend to engage with a lot, not every community that is less active.
No matter which sort you use (except for new),
Yes, sorting by new is best. The rest of the post seems irrelevant.
I wish the web ui (and apps) could work like an old fashioned usenet reader, where it would list your subscribed communities and say how many unread posts each one had. I don't like having all the communities jumbled together. That seems fixable.
Same, I stay away from /all and only follow my subscribed communities by new.
So almost forum like?
Yeah
Sorting by new is usually a waste of my time and has nothing of interest to me.
It works fine for me, everything else is worse.
Then try subscribing to communities that have something of interest to you.
Let’s say I’m subscribed to [email protected] and [email protected] because I want to occasionally see news.
I have this exact problem and it's maddening. Fucking "news", which is mostly just political posts about how shitty Republicans are completely drowns out all of my smaller niche communities!
I don't know how to fix the problem but the USER needs some way to control their feed. We either need to be able to throttle the larger communities or boost the smaller ones.
Back before kbin fell off the internet it had a really neat experimental "collections" feature that would let you make named groups of communities. Collections could be used either privately or made public so other people could subscribe to your curated feed on a topic. The owner could update the collection as needed (e.g. adding or removing communities/magazines as they changed over time).
It's one of the kbin features I miss most on lemmy.
Does anyone know if mbin ever got a copy of that? I know they forked off before it was added to kbin, but I don't know if it ever got integrated later. (I don't see it from a quick glance at moist and fedia, but I haven't dug into the dev history.)
I recalled Kbin having that as well, so thought that Mbin surely must, but someone pointed out to me that no it does not, for whatever reason even though Kbin had it.
PieFed does though.
Try Quiblr. It's a lemmy client with exactly the features you ask for. It checks your engagement, and filters and sorts your feed based on what it learned from your habits.
Just replying here with the community link [email protected]
because it doesn’t seem to be federated with my instance, it will once I subscribe though.
Can I use it on Android?
I haven't tried it, but I think it's on android. Might be a PWA, not sure.
Weird question, but does it not have a Subscribed view? I can't find it anywhere
Hi, I created the Lemmy client Quiblr which includes a For You feed which constantly evolves with the types of posts you interact with. 100% private and on-device (i.e. no data leaves your device).
On quiblr, you can use the "For You" sort like any other sort option
Nice. I will try it. Do I have to keep Quiblr cookies for the option to work?
Quiblr's For You feed should still work without cookies. Let me know if it works for you
PieFed has some features that I find helpful in this regard.
One is the Categories of Communities. You'll only see News in the News category or in the non-category search methods (standard Subscribed/All/New/etc.).
Another is the ability to follow - and arguably more important unfollow - everything. Including communities, posts, people, etc. I once made the mistake of replying to a comment in [email protected], and another in Lemmygrad.ml, and the replies kept coming for WEEKS and WEEKS - I almost quit the Fediverse entirely at that point, and I hear that scenario repeated by others as well. But on PieFed, not only can I unfollow any conversation at any time, but unlike Lemmy it also allows a true blocking of all users from any instance you choose (edit: to clarify, I mean without requiring an admin to do it for you and everyone else on the same instance at the same time - a personal defederation that affects nobody else, just like a block, except that Lemmy doesn't allow blocking users from instances, only communities from instances which is nowhere close to being the same thing).
Ofc nothing is perfect - e.g. I decided to unfollow [email protected] bc I don't want to read 5 of those in a row every morning, and rather would want to savor just one at a time. Also it would be nice to separate comment replies (that seem more urgent, for the sake of an active conversation) from e.g. a new post that you haven't seen yet? But for a true niche community, with let's say less than a handful of posts every day and you want to be notified about every single one? It seems perfect for that.
The UI for PieFed needs far more polish, especially outside of the narrow range of short comments on posts with few of those in number. e.g. far too often the existing notifications don't work as the reply to your comment got buried away onto some other page entirely, in an effort to streamline reading but then that not interacting well with the newer (unfinished?) notifications feature. But while it lacks much polish that Lemmy's UI and apps have, it also has so many features like those mentioned above that Lemmy lacks as well, and may move faster in its development due to using a more common programming language. It is so nice to have choices to pick from:-).
Otherwise on Lemmy you could make alts, like one subscribed to News communities, another for Memes and Shitposts, etc. Blocking users would get super annoying bc you'd have to do it multiple times. Blocking the largest news communities and the accounts that usually fill them, and then sorting by New can help, but it requires enormous curation efforts to get there and even then falls far short of what you asked - e.g. you also, still have to bookmark or otherwise check each one of your highly active communities one by one (edit: I mean niche ones here, bc the chances of seeing a post there on New can still be slim, if you are concerned about seeing EVERY post there rather than just find something to read from across hundreds of subscribed communities, so different solutions for different workflows).
Lemmy is great for checking memes, reading tech or politics news, and liking Linux - but for everything else it needs improvements to be made to support. I'm not going back to Reddit though:-).
I think that Lemmy would benefit from a tag system, one that allows both adding tags to communities but also to posts, the upshot is that these could be handled like hashtags on other federated platforms like Mastodon. Lemmy already does this with posts in a community, but it's just a hashtag of the community name, would be good if users could add their own tags.
Well actually... I neglected to mention that PieFed has hashtags already. Here's an example post showing them. Note the categories at the top and the hashtags below the post. Click them and they seem to work perfectly.
There's plenty of communities where I have no interest in discussing things with the masses. I like smaller communities with thoughtful posters who care more about the subject matter than being heard. That's the beauty of Lemmy. Not everything has to be perfect for everyone.
I'd love custom feeds, it was the one of the things I really liked about reddit
Nice profile picture!
Although there were some proposed solutions for this issue, when scaled sort was implemented, @[email protected] closed all related issues, even when they weren't being solved by scaled sort. So, it's clear that since there are no longer any open issues about this, no one is going to care about solving it. Therefore, it seems like the only option is to accept this fact and learn to cope with it. At this point, I've come to terms with the fact that Lemmy is mainly a platform for shitposts, while Reddit is for everything else. When I look at the feed, I mostly see memes, US politics, and some tech.
Custom feeds may not be the most efficient solution due to scalability concerns. However, an alternative approach could be to make the metadata about the posts (votes, comments, etc) available through an API call. This would enable users to develop their own algorithms for content discovery and potentially create a more personalized experience. Users could then implement, share and install these algorithms using tools like Tampermonkey or other userscript managers.
When I look at the feed, I mostly see memes, US politics, and some tech.
My solution to this (same experience here), was to block all the communities that were flooding with this stuff and anything else I didn't care for, and then just browse All. Now my home feed is pretty nice.
Leftist into tech.
My feed got very overwhelmed by depressing relatable memes that, guess what, had leftist views expressed in the comments, and posts that were not politics but ended up getting into there anyways.
I might be leftist but damn if outrage and despair isn't exhausting, I come to social media for fun, not to be angry and sad and hopeless.
Gave up on All incredibly quickly, only use Subscribed (I explicitly excluded anything political from Subscribed). So much less outrage and despair, so many more cute animals.
If there are still problems you should open a new issue. We cant leave issues open forever because they go stale and dont account for new features. By the way we are planning to implement multi-communities.
I'd rather be allowed to reverse sort on any of the algorithms. It would show content with less engagement and probably and up pushing up content from those niche communities you speak of.
Nah. Algorithms, especially personalized as a way of sorting a feed are just a shite idea. Maybe one of the iOS apps will add something like that but if anything Lemmy being different is a selling point. I got two friends on fedi by telling them that "it has no algorithm" which is a simplification of course but you get the gist. It also really hits home that this is not a corporate product.
When do you start counting it as an algoritm.
The current sorts (except new) are based on formulas, does suddenly adding a personal engagement variable into the formula make it an algorithm?
It's arbitrary but something like
SELECT * FROM posts WHERE datePosted < ( currentDay() - 7) ORDER BY upvotes;doesn't feel like an algorithm as it is now used in common parlance to me.A simple quantitative analysis of an existing metric and (upvotes in the above super simplified example) is just not really the same thing in practice as say: multiple linear regression of hidden backend engagement metrics gathered through things like cursor movements to pick a suggested video that is predicted to optimize the best for watch time and CTR from a list of videos on a balance of personalized and generalized (through tracking trends amongst demographics) favourites topics and other qualities classified and categorised by a whole other black box involving all sorts of classifier models from text to images and so on.
Idk, I didn't take algorithms in CS at uni, so this is just a layman's two cents. I'm happy to be explained to why this isn't a valid perspective.
Algorithms are fine when an algorithm is open, clear and optional. The default sort for many apps/UI is "Active", that's an algorithm. It may be a simple one, but it's an algorithm.
I usually use the scaled sort in order to find niche communities