Is Kindle Unlimited Distorting Amazon Sales Ranks?

The integrity of sales rankings matters. It’s pretty well documented that bestseller rankings have direct effects on sales. In a world awash in titles, it’s hard to figure out what to read, so rationally lazy people rely on bestseller lists to learn what to read. Until recently, the Amazon top 100 was a meaningful ranking showing the top sellers according to the traditional criterion of, you know, the number of people paying the book’s price to get a copy. For example, Amazon scrupulously excluded free books from the paid bestseller rankings. With the introduction of the Kindle Unlimited (KU) program, Amazon now counts each borrow as a sale for the purpose of calculating sales ranks, raising the possibility that Amazon rankings have become a little less meaningful.

Under the Kindle Unlimited program, subscribers pay $10 per month for the right to borrow up to 10 e-books at a time from the Kindle Unlimited catalog. Today about 700,000 titles are in the program. When a KU subscriber “borrows” a book, the borrower pays no price beyond the already-sunk monthly fee. Once the borrower makes it a tenth of the way into the book, the author gets a slice of the monthly KU kitty. Most recently the per-borrow payment was about $1.50.

Authors and publishers are unsure of whether to participate because it’s not clear whether participation will bring in new readers or simply allow existing readers to pay less for the same books. The big publishing houses were reportedly hesitant about the KU program at launch, so it is natural to worry that the KU program will suffer from the streaming Netflix problem: lots of titles but few that you actually want. Using the fascinating data that author Hugh Howey has made available at authorearnings.com, I can explore this. On 4 separate dates in 2014 – in February, May, July, and October – the operators of authorearnings.com gathered data on 100,000 Kindle titles at Amazon (including bestsellers in a variety of categories). For each title they report its sales rank on those four particular days. The KU program went into effect between the July and October data-gathering dates, and the October data indicate whether a title is included in KU.

What sorts of books are included in the KU catalog? Of about 115 thousand titles in the October data, one fifth are in KU. Perhaps surprisingly, given the presumed Netflix problem, almost a third (32 percent) of the top 100 titles on October 2, 2014 were included in KU. And closer to 40 percent (37.9) of titles ranked 101-1000 were included. So, true, most of what you want is not included; but what’s available in KU is tilted toward the popular, or at least the titles near the top of the bestseller list at Amazon.

Which brings us to the question of whether Amazon sales ranks continue to reflect actual popularity in the KU era. For a start we can ask whether participation in the KU program changes an e-book’s Amazon ranking. Figure 1, which includes only the roughly 40,000 titles appearing in both July and October, tells the story. Because these books were released during or before July, their ranks tend to have fallen by October. For the 31,681 titles that do not join KU, the average rank descends from 210,000 to 246,000. But for the 8,784 titles that enter the KU program, the average rank goes from 200,000 to 165,000. That is, instead of getting 17 percent worse (+36/210), the ranks for KU joiners get 18 percent better (-35/200). Overall, then, joining KU gives about a 35 percent improvement.

We can take a slightly more granular look to see what’s happening throughout the rank distribution. For this we divide the 40,000 titles into 100 groups according to their July ranks, then calculate the groups’ average July and October ranks. In Figure 2 the solid red line shows the ranks for titles that do not join KU, while the blue dashed line shows the ranks for titles that join KU. The important finding is that for any July rank group, the average October rank is better (lower) for the joiners.

How big a deal is this? For example, how many of the Amazon top 100 or top 1000 are in those exalted realms only because of the KU program? When I run a regression of the log rank of a title on an October effect (to account for the average change between July and October) and an indicator for joining the KU program, the coefficient on the latter is -0.38, meaning that the log rank for joiners falls by 38 percent relative to non-joiners. So I adjust the log rank by adding 0.38 to the original log rank for joiners. I then re-rank to create a “corrected top 100” and a “corrected top 1000.” Of the corrected top 100, how many were in Amazon’s top 100? Just over 90 percent. And of the corrected top 1000, how many were in Amazon’s top 1000? About 84 percent.

So it’s clear that the Amazon ranking now measures something rather different than what it measured before the KU program. Amazon is giving authors who opt into the KU program higher apparent sales rankings. I don’t know whether this is fishy. After all, I’m just a professor, not a $143 billion dollar company with patient shareholders. But I do know that when I give higher grades to students who opt into my “Office Hours Unlimited Program,” I get in hot water with the dean.