Twitter have decided to run an experiment. They are giving random users twice the character limit — 280 rather than 140 characters. Their motivation was their observation that in Japanese, Korean and Chinese 140 characters conveys alot more information and so people tend to tweet more often. Here is their full statement.
The instructive graph is this:
The conclusion drawn is that Japanese tweeters do not hit their character limit as much as English tweeters. They also claim they see more people tweeting in the less constrained languages. Their conclusion is that not having as tight a character limit makes expression easier and so you get more of it.
What Twitter have just told us is that the world gave them a natural experiment and they liked what they saw. They don’t provide much in the way of details. We got this one graph and nothing about usage and certainly nothing about engagement (do longer tweets get more likes or retweets?).
What was Twitter’s reaction to this? To do an experiment. In other words, they are worried that the natural experiment isn’t telling them enough. Since it is about as clean a natural experiment as you are likely to get in society, we can only speculate what they are missing. Are they concerned that this is something cultural? (They had three cultures do this so that is strange). Moreover, many of those users must also speak English so one has to imagine something could be learned from that. Moreover, English tweeters use emoji, what of these three Asian countries? My point is: the new experiment must be testing a hypothesis. But what is that hypothesis?
The candidate is that by moving to 280 characters, the Japanese and English patterns will converge. So what you want to do is find a setting where you can treat a sub-set of the English-tweeting population in a way that allows you to test your hypothesis.
The obvious way would be to announce, say, a three month trial across the whole of English speaking twitter and observe changes. That would replicate the natural experiment to a degree. Or, alternatively, you might pick a language with a small number of users and conduct the experiment there.
That is not what Twitter did. They decided to randomise across a subset of English users — giving them 280 characters — and leaving the rest out. That strikes me as a bad idea because those random people are not contained. They mix with the 140 people. And you can see today what those 280 people are doing — they are advertising the fact. So now you have them tweeting more because of the experiment and you have others also likely changing their behaviour. And you have the feed with a mix of 280 and 140 tweets which is changing how you scroll through it.
Why is this a terrible idea? Because it is not an experiment that tests what Twitter was likely missing from the information they gained already. Instead, it is an experiment that tests the hypothesis — what if we gave some people twice the limit and threw all of them together with those without? The likelihood that Twitter learns anything with confidence to move to a 280 limit from everyone is very low from this.
By the way, it is worse than that. Apparently, it is easy now to give yourself 280 characters because of the way this experiment was coded. So they aren’t even getting randomisation!
People today are complaining about Twitter changing. That happens every time some social media network does a change. So it is beside the point.
What we should be complaining about is why they are running such an awful experiment and how they came to such a ludicrous decision on that.