Google has reportedly removed much of Twitter’s links from its search results after the social network’s owner Elon Musk announced reading tweets would be limited.
Search Engine Roundtable found that Google had removed 52% of Twitter links since the crackdown began last week. Twitter now blocks users who are not logged in and sets limits on reading tweets.
According to Barry Schwartz, Google reported 471 million Twitter URLs as of Friday. But by Monday morning, that number had plummeted to 227 million.
“For normal indexing of these Twitter URLs, it seems like these tweets are dropping out of the sky,” Schwartz wrote.
Platformer reported last month that Twitter refused to pay its bill for Google Cloud services.
Elon going to complain about another conspiracy going on while in reality it’s just that when crawlers are not able to open a certain URL they simply assume that the page doesn’t exist anymore. Google certainly didn’t “retaliate”, bots simply couldn’t find those pages anymore.
The headline is actually wrong. Google did not do anything to Twitter. Twitter fucked up their own SEO by removing access to its content.
This is correct.
Yeah, that’s a pretty easy and reasonable conclusion to come to if you think about if for more than five seconds. I’m not sure Elon has any toes left after he keeps shooting himself in the feet.
The latest in a seemingly never-ending series of self-owns. Apart from the stress it must put on their devs, it’s been entertaining
Twitter still has devs?
They have more than one dev left?
No it’s just one guy called Dev.
And he’s on a H1B visa and can’t leave.
Crawl issues I am sure but also user experience issues. Google is sensitive to sending visitors to sites where metrics indicate users do not, like bounce rates etc. I don’t use twt but if it is the case you have the be logged in to see anything now, a non-logged in user will click a link from Google hit a login page, and use the back button. I would assume Google will see that as a bad search result and use it less.
I wonder if Google prioritized recrawling all those Twitter links.
If I were making a web crawler, I would make it so that if a crawler finds a domain that appears to have changed dramatically or gone offline it will re-crawl the domain and flag already-crawled pages as potentially obsolete.
The word bots triggers the muskrat.
Elon, please buy Reddit and repeat your amazing ideas over there. You are so smart.
Spez is already doing his best work to fuck over the platform. Are you certain Elon could do any better?
Spez already said he admires how Elon is running Twitter.
The best part is that Elon is proving to be a pro at losing money left and right while simultaneously inventing new ways to make a social media platform suck to use.
I mean there’s admiring a ruthless mob boss - and then there is admiring a petty thief that keeps getting arrested and all his plans blow up in this face.
Spez is losing his tiny mind.
Oh, there’s scope to introduce “official” Reddit user badges for 10 bucks a pop and all of sorts of suicidal Musk shenanigans
Wait until you have to be logged in to read any posts on Reddit.
Didn’t we achieve this by turning all subreddit NSFW?
Yes, but when the users and mods who built the site do it, it’s petty and selfish. When the ceo does it, it’s brilliant, compounds user value, and improves the security of the user experience. /s
Doesn’t sound like retaliation to me, it sounds like their scheduled web crawlers are finding that content they used to index is now no longer viewable and this removed from search results. Pretty standard. My guess is that there were 400 million URLs listed and as the crawler uncovers that they are no longer available, that number will keep dropping to reflect only content publicly viewable. If only 500 URLs are now publicly viewable (without logins) then that’s what they will index. Google isn’t a search engine for private companies (unless you pay for the service) they are a public search engine so they make an effort to ensure that only public information is indexed. Some folk game the system (like the old expertsexchange.com) but sooner or later google drops the hammer.
God, I hated expert sexchange so much. It was a blessing when stack overflow started.
I’m only an amateur but I’m happy to give it a go
I am genuinely curious. What’s the role of the new CEO if this turd keeps doing everything he can to burn this ish to the ground?
Gotta have someone to push off the glass cliff
Thanks for sharing, I hadn’t heard of that expression or phenomenon.
Ever heard of the glass cliff? You might even see a name or two on that list that you recognize.
Yup, I remember the absolute outrage when Ellen Pao became CEO of reddit, though it is nice at least to finally see her vindicated on there now. I hadn’t heard of the glass cliff until long after she left, but people have been referencing it left and right on reddit recently. Turns out, Ellen Pao wasn’t the problem…
Great. Do Pinterest next.
And Tiktok. And Quora.
Good. Hopefully they remove links to pinterest, quora and facebook too while they’reat it.
Once upon a time Quora had reasonably accessible information. Now I find it nearly unusable and only go there as a last desperate effort which is generally fruitless. Pinterest is annoying, but generally you can still view some content. What’s annoying is trying to download, copy, or isolate content there. If all you want to do is view an image, Google can typically still pull the image out and make it viewable from the search. The problem only arises if you try to go in and see the original.
I have a chrome plugin to strip any Pinterest results from searches, it’s the absolute worst
Please remove Quora links…
Every time I click a Quora link it takes me to a completely unrelated question to the one I clicked on.
I swear that site is doing this deliberately to make it seem like it has more traffic than it does.
You can omit search results from certain sites by adding -quora.com to your search if you’re feelings motivated
Is there a way to just add that flag to all of my searches? Quora is shit in general
I don’t know of a way to save it as a setting but there may be a plugin you could add to your browser
Has anyone noticed too that if you put AI Blockers on your website Google delists it from their search?
Twitter was an important unifying communications tool during the Arab Spring. The Arab spring was a threat to biz as usual in places like Saudi Arabia. The second largest investor in Twitter is Saudi Arabia.
Saudi Arabia killed and dismembered a journalist from the US, more or less in plain sight. Elon is now killing and dismembering Twitter in plain sight to limit its power as a unifying tool that stands as a demonstrable, active threat to capitalism and oligarchs around the world.
Billionaires do favors for other billionaires. It’s part of why spez is trying to tank Reddit. Remember how dangerous Reddit was to capitalism’s status quo around the time of GME/Robinhood/Antiwork recently.
The specific moment we’re in right now is meant to shatter consolidated organizing power on Reddit as we splinter into several smaller alternative platforms (or for some, disconnect entirely). Not saying we shouldn’t be in Lemmy, but calling out the larger reality of the moment.
Billionaires do favors for other billionaires.
This sounds a lot like Hanlon’s razor. “Never attribute to malice that which is adequately explained by stupidity.”
Do you really believe that people like spez, Zuckerberg, Musk behave like they do because they want to do favors for other billionaires? Isn’t it much more likely that they’re just … disturbed? That they are narcissistic, megalomaniac, maybe idealistic in their own believe. And in being that, they make stupid decisions because they literally work differently than regular folks.
Yeah, I also seriously doubt there’s a big conspiracy happening where ultra rich people are helping each other. Have you looked at those people? Most don’t give a fuck about anyone but themselves.
Musk bought Twitter around the time he was fighting with this guy that had the private jet tracker. I think it’s more reasonable to believe that Musk bought Twitter just to shut that down and now it’s a toy he can play with, where every time he merely touches it, media jumps on it, which feeds his ego massively. And once Twitter is dead, he’ll discard it and move on to the next thing. Like a cat playing with its prey.
Normally I would tend to agree with you, but look where Musk got his “loans” to buy out Twitter. Saudi Arabia and Russia where big “lenders”.
And this is interesting…
https://mastodon.social/@rodhilton/110657916419002616"rodhilton Rod Hilton @[email protected] I have some insider knowledge here that I wanted to share.
This is not happening because Google scrapes Twitter and is now unable to. Google has been a paying customer (with a special negotiated rate) of the Firehose API for nearly a decade. Presumably, that deal was still in effect, barring API rate changes having an impact.
So this decision is solely because the results can no longer be viewed by non-logged-in users.
Great now can they remove Pinterest too please??
Yeah, if the content is walled off from the public, there’s not much point indexing it
It’s a shame it’s come to this, but there are literal crazy people at the helm over at Twitter…
I’m convinced that Elon is intentionally killing the platform.
Would that fit with his own head cannon though?
Elon has been a destroyer behind closed doors it’s sounds like, tough to say if it was just due to how pompous and narcissistic he is or if it was deliberate retaliation for being a whiny little spoiled brat.
He ain’t that smart
Musky is probably wondering how to charge search engines indexing his site for free
This morning, I needed to use image search for a bit and basically 4/5 links leading to Twitter wouldn’t load for various reasons. That was not on Google, but I imagine, they have (had) similar problems.
Well, and for normal search, Twitter results are completely worthless to me, as I don’t have an account. So, at this rate, no results from Twitter would be the optimum.
decimates
Twitter absolutely DESTROYED by RATE LIMITING and ELON
Um actually 🤓 decimating it would be 47 million links 🤓 not 227 million
Um actually a word’s etymology does not define its modern usage. 🤓
It does if people use it correctly
You do realize that there are multiple definitions for the word “decimate”, yes?
Now do Quora.
And Pinterest. I want my quality searches back.
Use google and add “site:Pinterest.com” then search by images. Works better than Pinterest. Also works for Poshmark.
Good advice except you forgot the minus symbol, this tip will ONLY get Pinterest links.
Blocking users who are not logged in has farther reaching consequences that aren’t readily apparent. For example, there was an AMBER Alert a few days ago with a short link to see more info. The link goes back to a Twitter account/tweet. All that time sensitive, useful information was behind a wall where you can’t see it unless you log in. Most people aren’t going to create an account just to do that.
This is exactly why we should be encouraging local libraries, universities, law enforcement, city, and county governments how to set up Mastodon servers.
On the one hand, when you have a duty to inform the public, it no longer makes sense to suffer at the whims of tech billionaires. There was a time, for a decade or two, when these sites prioritized access and predictability, but no more. When you have information that you need to have accessible, the only guarantee is to control it yourself. They can still use corporate social media to get the message out to their network, but link it back to their mastodon account. Roll it into their IT departments just like their email server.
On the other hand, it’s a critical step for the success of the fediverse. Universal email adoption came about because it was used by government and universities. What you could call the original social network is still an open protocol, it’s not owned by any single corporation or government, and still the primary form of communication online. About 2 billion emails have been sent since you started reading this.
I feel like Google is going to have to find a way to effectively index federated content at some point. The only way to really get human information is from sites like Reddit and Twitter. And both of those platforms seem to be dedicated to completely imploding at the moment.
There’s nothing about the content being federated that makes it hard or impossible to index. Each instance is just a website with a public webpage that a bot can read. That all a search engine needs to index it. The worst case scenario is the bot will find the same content on multiple instances.
I did read that the website is loaded entirely through JavaScript and that maybe the Google bot doesn’t execute JavaScript so can’t see the text. I don’t know if that’s still a problem in 2023, though.
This article says it’s not a problem, but I didn’t read past the tl;dr, so maybe there’s a caveat. Like maybe it has to use a popular framework like React or something to work.
https://searchengineland.com/tested-googlebot-crawls-javascript-heres-learned-220157
It’s already indexing it.
duckduckgo (who uses Microsoft’s index I believe) is able to find Lemmy instances already.
problem is since every instance has its own domain you cannot search all of Lemmy or the more obscure fediverse.
lemmy.world,beehaw.org,programming.devare all different “websites”.I append “reddit” to my query when I want to search reddit for a human answer to a question. Can’t do that with Lemmy, unless the instance is branded as Lemmy.
Unless there will be an org or volunteers that indexes federated instances and makes them available to search engines to they can be differentiated, finding stuff in the fediverse might be difficult…
Fuck Google, if Lemmy continues to take off we can just develop better search tools within the fediverse. The wider internet has been colonized, the path forward cannot rely on big tech corporations.
I’m not a programmer/developer so I don’t even understand the scale of the work that has yet to be done. But I am deeply committed to upsetting the status quo, and this platform feels distinctly revolutionary. Can’t wait to see what the future holds for Lemmy.
Ask and ye shall receive. Just saw this post!
