Thu 3 Feb 2022 — 6 min

Since last year Trans Safety Network have (with the gracious support of David Allsopp) been trying to raise awareness of Twitter platform manipulation being undertaken by anti-trans activist groups.

This has gone largely ignored, in part because trying to demonstrate that algorithmic harms are real can be quite challenging, and difficult for many people to follow. A subsequent article by Logically.AI raised this question again, following a similar set of concerns about small tightly clustered networks of accounts having an apparent and worrying capacity to send another anti-trans Twitter hashtag onto the trending page. The second time this happened it stayed there for over a day, and has been trending again and again.

Logically.AI's article was later picked up by the political magazine Private Eye, because in the latter case, the trending hashtag had been cited during a House of Lords debate on trans rights. Clearly it's a concern if these trending metrics are being used by politicians to estimate public sentiment about important policy issues, and those same trending metrics can be absolutely gamed by a small group of people.

This gave cause to anti-trans accounts to speak out today, using the #NotABot hashtag. Many of them, quite likely, felt legitimately upset about the suggestion that this trending activity was due to "bot" activity. It's a sad irony that when algorithmic manipulation happens, the vast majority of the people involved in tweeting about the hashtag are responsible for a tiny minority of the engagement - and at the same time, it feels to them like they're engaging as part of a much bigger movement. They see the numbers go up on the trending board, they tweet about the issue themselves.

This article is not to describe those people who are riding on the wave as bots. In fact, "bot" itself seems like a bit of a messy term to be using at all. What this piece is about is how inauthentic behaviour by a small and dedicated core following a specific pattern can result in a hijack of Twitter's trending page, despite the experience of other people interacting with the site in good faith.

The experiment

One of the problems with debates involving politics is that people are motivated to ignore evidence. We have written down how this exploit works. Logically have described a broad overview of what it looks like analytically. And after seeing anti-trans Twitter responding to Private Eye taking it up, I tried to lay out the mathematics of how the attack works. But one of the things that is missing here is following up on our hypotheses about how Twitter's Trending algorithm works by making a prediction and proving it. Additionally, it helps that we do this without directly referencing any sort of political conflict, in the hopes that readers are able to detach from their emotional attachments to the wider topics of Trans Rights, and simply consider the behaviour of the platform under a particular pattern of abuse.

The short version is this:

Normally most Twitter users will tweet about a cause, and tweet a few other people. But a dedicated core group who all retweet each other as well as tweeting their own tweet scales quadratically and rapidly outpaces the (roughly) linear nature of less deliberately focused platform manipulation. The full details of it are in the blog post referred to above.

After tweeting a link to my quick write up of the attack, of how it scales much faster than ordinary Twitter use, I tweeted this, to underline my point.

This was then replied to by Disability Law scholar Jess O'Thomson suggesting "Maybe get #ThisIsAnExploit trending with a bot lol", so I decided why not.

Within 45 minutes of followers following these simple instructions, the hashtag was trending, with around 4800 posts.

By one hour, we had reached 7100 posts, and within a little over 2 hours, we had hit 17.7k hits at 3rd on the Twitter trending board, beating the extremely popular discussion on Boris Johnson's troubled leadership into 4th place.

3rd place in the United Kingdom, 17.7k tweets. Screenshot gathered by @truesolicitor
3rd place in the United Kingdom, 17.7k tweets. Screenshot gathered by @truesolicitor

So as to avoid confusing our impact with the organic following and interest the Twitter trend ultimately attracted, we pulled the first hour's worth of tweets for analysis.

Thanks to David Allsopp for grabbing the API data for us, one third of the tweets at the hour mark (when it was already trending) were attributable to only 10 Twitter accounts, engaged in the same inauthentic tweeting pattern. Over one half of them were accounted for by the top 20. Around 3/4 of the tweets (over 5000 posts!) were accounted for by the top 50 users engaged in the experiment.

At this point, only 383 accounts had interacted with the hashtag in total, but the vast majority of them had ony interacted with the hashtag a small number of times and did not contribute much to the trend, while the core group was able to drag up the trending count into the Trending page.

Twitter eventually responded to the hashtag by blocking it from the Trending list, demonstrating that Twitter is capable of responding to this sort of inauthentic activity. But in the previous cases we have observed it has chosen not to.

Conclusion

The reason we did this exploit on Twitter in real time was to prove a point about how broken Twitter's trending algorithm is. We didn't choose an anti-gender-critical hashtag because we're not attempting to shame or get into some sort of fight with them. We want Twitter to change the way that they handle this, so that tiny focused groups of hate activists are not able to boost anti-trans messages into Parliament well beyond the scale of their actual movement.

If we can't get Twitter to change, we want politicians and the media to understand how easy it is for this to be manipulated. If this is not handled, it is likely that there will be an arms race between advertising companies, political nudge units and activist campaign groups continuing to exploit this behaviour, making the Trending page useless for everyone.


Tags: