Meet Leddit, a bot that crossposts Reddit threads - and their comments - to the Fediverse
Meet Leddit, a bot that crossposts Reddit threads - and their comments - to the Fediverse
A Reddit-to-Threadiverse archiver for discussion threads. - GitHub - hjalp/leddit: A Reddit-to-Threadiverse archiver for discussion threads.
Happy July 1st! Starting today, free third-party Reddit apps will no longer be usable, but as much as we don't want to admit it, some of us still miss the content on Reddit, and it can be hard to resist the "just browse Old Reddit with an adblocker" loophole. Before we get into yet another "crossposting bots on the Fediverse" debate, here's a quick summary of what this bot is and is not intended to do:
| Intended | Not intended |
|---|---|
| Allow users to consolidate their link aggregation and discussion feeds onto an open-source, non-proprietary site | "Increase activity" in a community by compensating for the lack of real users |
| Encourage ex-Redditors to spend more time on the Threadiverse when they are able to access their favorite content here | Serve as a "bridge" between Reddit and the Fediverse. Threads are archives and messages will not be synced back to Reddit |
| Preserve thoughtful, valuable and informative content, and make them accessible without a privacy-hostile corporate platform |
(Please don't leave comments on the Reddit archive threads on the demo instance.)
Leddit is a fork of lemmit.online and does not use the Reddit API at all. Unlike lemmit.online which is a public service, Leddit is meant to be self-hosted on a personal instance as syncing comments is a very slow process that will get rate-limited on a normal instance.
Based on my demo instance that syncs posts and comments from two subreddits with a combined subscriber count of about 1 million, this takes about the same amount of time that lemmit.online takes to sync only posts from more than 100 subreddits.
An example of a thread that is automatically created and updated by Leddit can be seen here. The header message and position can be customized. Leddit preserves the comment thread's structure and identifies the OP in the comments.
For shorter threads, all comments are synced, but comments in longer threads that are hidden below "show more comments" are not synced as they consume additional requests to Reddit with very little content in return.
If there's interest, I can also add a feature that allows the bot to archive entire subreddits instead of retrieving the newest posts. Please feel free to ask for support to set up your personal bot and instance in the Leddit Lounge community.
My only question... Is it easy to block?
Yes, you can block any account (bot or real).
So... lemmy.world just defederated lemmit.online because of all the Reddit cross post spam... Will this create a whole bunch of personal instances whose Reddit scrapings will show up in the feeds, meaning the admin of lemmy.world will now have a whole bunch of personal instances they have to watch for and defederate from individually, and ongoing?
i am having a difficult time really understanding the federation architecture. i felt like i was doing pretty well with it... until i read your post.
Stolen from another user:
Reddit sucks. Let's make our own reddit with blackjack and hookers.
Result: there are now 15 competing reddit knockoffs. Each one is small and lonely.
Solution: let the reddit knockoffs talk to each other. This solves the problem of a fractured userbase. This was always reddit's strength: disparate communities united, on the same site. This size and diversity meant there was a high likelihood of someone who knows what they're talking about or someone with a unique perspective stumbling across the post and chiming in.
By understanding the motivation, hopefully the concept of "Federation" makes more sense.
▫️▫️▫️
My own explanation: you can find a message board for pretty much anything somewhere on the internet, but you’d have to go to a different website for each message board you’re interested in separately - they can’t talk to each other. What Reddit did was create message boards (subreddits) that could interact with each other on the same site. The fediverse is the equivalent of Reddit.com. Communities are the subreddits.
I think the complicated bit are the instances - Reddit doesn’t have those. Instances are like servers. You need to join an instance to access the fediverse. It doesn’t matter which instance you register to unless you’re really interested in the alt-right (for example). A couple of alt-right instances have been defederated because they’re pretty hateful. They have splinted off and can make their own message boards/communities/subreddits but you can’t see or interact with them unless you join that instance separately. Unless whatever instance you’ve registered to defederates (unlikely if you’re in a mainstream one like lemmy.world) you don’t need to worry about instances or join a second one.
This just means that users on lemmy.world will see nothing from lemmit.online. Personally I blocked the main bot that was making all the posts so I at least don't see all the spam from it.
i am having a difficult time really understanding the federation architecture. i felt like i was doing pretty well with it... until i read your post.
For a simple explanation, imagine instances as servers. Servers can host the lemmy code, think of it like an opensource reddit. Instances can connect and interact with each other when they federate and they can defederate (break the connection).
This has a lot of potential, very interesting
I can see this causing as much annoyance as the Twitter→Mastodon cross posting was for people there. I think there was some relief when API changes came in and people didn’t have to deal with as much.
a bot that crossposts Reddit threads - and their comments - to the Fediverse
Why?
Explained in the post. A very common complaint I see in the migration discussions is "my hobby's community isn't on the Fediverse/doesn't have an active community like Reddit so I still have to visit Reddit". Unless they intend to participate in that Reddit community (which most users don't), they can bring their community's knowledge here instead of giving Reddit more traffic.
We survived for decades without Reddit and we will survive without it in the future. There’s no shame in checking up on small subs there and a community here. And if there isn’t a community here yet. We can make one.
The problem with a copybot, aside from the potential legality, is that all the threads here will be static/stale. There is no reason to interact with them here as the comms are one way.
Do we really need that much dirt brought through here? I mean, the carpets were just installed. Can’t we wait before we let all that mess through here?
If you're copying people's comments without their consent, without their ability to delete them because it's too resource intensive, that's a GDPR issue.
That sounds like a nuisance. Maybe we could have one instance whose only job is to collect scraped Reddit data, but it shouldn’t have any ties to Lemmy instances where actual users want to post content.
That's exactly what https://lemmit.online is. Just one bot to block, or one instance to defederate from. It doesn't post anywhere else, so interested users can just request a subreddit on there, and follow what they want.
Very interesting
I feel like this is best applied to posts that are likely to contain useful technical, troubleshooting, advice, support topics. Basically things that someone might actually search google for to find on reddit. Anything else, opinion, news, meme, etc is just not worth it. That's the type of stuff that can just grow organically on lemmy.
And even with that I'm torn on the concept.
Can’t wait to read all those Ai bot posts /s. I’m good thanks.
So its a bot that reposts bot reposts of bot reposts?
I appreciate your work, but I don't think Lemmy needs this kind of mirrored content...
I keep blocking every community and bot I find doing this.
This is an interesting idea, I might have to try and spin up a private instance. Do you know if there's a better way to do that then only allowing local connections?
You can spin up a regular instance, check "Close signups" and uncheck "Enable federation" in your admin settings, which will make your instance a private forum that is accessible from the internet.
Thank you, I appreciate the help.