Reddit is now blocking major search engines and AI bots — except the ones that pay
Reddit is now blocking major search engines and AI bots — except the ones that pay
Sorry, Bing users.
The cycle continues:
- Hey you guys can have everything for free
- WTF this is expensive to provide, I think I’m gonna start taking advantage of you guys which someone will pay me to do
- WTF where’s everyone going
- WTF I’m still losing money and always have been
- Screw you guys, screw everybody, I didn’t want y’all anyway
- (fades into irrelevance, gets bought by someone and stripped for parts)
Idk it’s not as pithy as Cory Doctorow’s version I guess
Anyway we’re at step 5 at this point
honestly I'm not convinced step 6 is inevitable. I think enough people are okay with whatever reddit does.
Enough consumers are okay with it, but the core geeks and nerds that created, curated, and moderated the content have jumped ship.
The cruise line is still sailing and there are still drinks and snacks so nobody has noticed the staff have jumped ship. There's management, low level volunteers, and thousands of kids, moms, and dads.
But sooner or later people are gonna get tired of snacks and flip their shit when management tells them the people who know how to make the steaks have just all fuckin ✨ inexplicably disappeared✨
The key capitalistic trick is to time your step 2 just when you have a critical mass on your platform. Upper management has understood that our shitty paywall will remove x% of our users from our platform. But if (100-x)% of our users can pay $y annually, we can sustain our business model and make $z of profit each year. PR will take care of all the backlash but it's all calculated.
I freaking wish..
Tangentially related- I fucking hate discord
Discord is fine for chatting, voice, and iterating quickly on projects. I have no idea why people want to think it's a forum. That's ridiculous.
I fucking hate discord
It's Cancer, have an upvote.
thanks to them for making my deredditification that much easier!
The only way they get my clicks now are when I Google something and they come up.
They really keep making sure that I don't end up there.
libredirect helps with that on desktop
(browser extension that turns links to sites like reddit, youtube, etc into links to redlib, invidious)
Too bad. Hey, crazy idea: let's create an open alternative for reddit with good content! Maybe something in the fediverse or so.
I think you're onto something
that would never work
There are numerous occasions where someone has a lingering question on Reddit that I see and know the answer to. It’s too bad it’s on Reddit because I no longer contribute to that website, and refuse to.
All the decent answers I find are from 5+ years ago. I check the user’s activity and they normally quit the place. Warms the heart.
just begin with site:reddit.com test for ddg and it still works
Are they new posts or old ones? They are blocking new ones, not old ones.
new posts do not work
this post in /r/selfhosted is from 8hr ago: SWEKIT v0.1 - an open source library to build software engineering agents (DEVIN) in a agentic framework agnostic manner!
reddit/redlib: https://redlib.kylrth.com/r/selfhosted/comments/1eb86lf/swekit_v01_an_open_source_library_to_build/
doesn’t appear in DDG results: https://duckduckgo.com/?q=site%3Areddit.com+SWEKIT+v0.1+-+an+open+source+library+to+build+software+engineering+agents+(DEVIN)+in+a+agentic+framework+agnostic+manner!&t=ffit
Based on my testing if you filter results by the last week or last day you get nothing. Past month works.
For old posts. I can't find new posts on DDG. I find them on Google but not on DDG.
I tried brave search begin with site:reddit.com test and it still works
LMAO searching "____ reddit" is the only time I visit their site.
They just really have no clue.
Would lemmy instances do this?
I know they can't afford to now, but hypothetically? A lot of people here don't seem to like data scraping for AI.
Your Lemmy posts are already being scraped for AI
The level of effort it would take to prevent would be infeasible to ask of even a non volunteer admin let alone a volunteer let alone literally all of them
You don't need to scrape. If you want to get all the content on Lemmy, just set up an instance and subscribe to all the top communities, and the instances will just send you all the content.
So there isn't really a way to monetise or block it. I guess you could only federate to a whitelist, but the biggest instances will federate by default with any new instances until they are given a reason to defederate.
Some Lemmy instances disallow indexing in robots.txt, however indexers can choose to ignore that and actually blocking them takes a lot more effort.
Some places on a "budget" like Ao3 just rate limit hard.
I don't like that solution at all though.
The users who wrote the content are going to get a share of the money, right Reddit? Riiight? /s
Brave search got an option for that.
begin with site:reddit.com test is much more accurate to get reddit search on brave search tbh
Someone should make this feature but for ALL public web content you browse. Just download an extension to share the content of pages you browse to everyone (with cross-checking for accuracy), and you can view a fair share of what others have shared based on how much you contributed to the platform yourself. Basically crowd-sourced, unblockable web scraping.
I don't care. Do you?
Yeah, a lot of IT and tech content is still on Reddit and YouTube sucks now. The search itself on Reddit is really bad.
Good, their answers are generally crap, and I wish they wouldn't show in searches anyway.
I mostly feel the opposite.
Reddit is one of the only search results that actually has content made by humans.
I mean, you're right if by humans you mean kids.
Depending on the subject, I encounter more and more threads with <deleted by user> content. And billions and billions and billions of results that are either spam or written by unprofessionals.
The smart crowd is not there anymore. The smart crowd that once was there, has removed the content that Reddit was worth visiting for. Let the Googzz have them and sell ads to each other.
lol - fine by me. My private searx-ng instance already filters out Reddit from the results, and my Pi-holes block all known Reddit domains.
Your fault for using a major search engine honestly
