In a fascinating blog post, Adam Ozimek makes the case that we will see much more individualized pricing (which economists call “first degree price discrimination”) as more data mining becomes available. For instance, in a new working paper, Ben Shiller is able to use big data to massively improve his ability to predict demand for Netflix subscriptions by any given individual:
Adding the full set of variables … including web-browsing histories and variables derived from them, substantially improves prediction – predicted probabilities range from close to zero to 91%….I find that web browsing behavior substantially raises the amount by which person-specific pricing raises variable profits.
As more and more data become available, it’s easy to imagine retailers using these data to offer personalized prices for each consumer, especially for information goods where margins are large and pricing flexibility is greatest. “You want to watch Elysium tonight? Special price just for you!”
However, that’s not the end of the story. Remember that big data can work for consumers, too. Thanks to Google and other tools, consumers also have more and better search engines and recommendation services available. That can intensify competition among sellers, which tends to increase consumers’ surplus.
Digitopoly 
Erik,
I find the big data implications for pricing are often really poorly thought out. We know from really basic economic theory that the ability to first degree price discriminate even when there are only two sellers increases consumer surplus because of how the ability to price discriminate changes the reaction curve (it is very simple to show this in a Hotelling model with linear transport costs, for example). This does not depend on any decline in search costs, and is pretty general to oligopolistic markets.
It’s certainly possible for some types of price discrimination to increase consumers’ surplus but that’s far from typical. Perfect price discrimination eliminates all consumer surplus by definition, and even imperfect price discrimination typically makes it easier for sellers to extract surplus from consumers.
This does not hold with oligopoly, though. An oligopolist always is better off being able to price discrimate holding other firms’ prices constant, but in equilibrium the oligopolist is worse off. The reason is that with uniform pricing, both extract positive rents because of the usual tradeoff (if I drop prices, I gain some demand at the expense of the rival in exchange for lower sales), but with the ability to perfectly price discriminate, I maintain the benefit at the extensive margin with no tradeoff on the intensive margin, hence competition is stricter.
Thisse and Vives AER 1988 has formal proofs of this, and Mark Armstrong has a nice summary of more recent work (http://else.econ.ucl.ac.uk/papers/uploaded/222.pdf) but the point that price discrimination has completely different effects in oligopoly than in monopoly is pretty general.
And let’s not forget price discrimination can allow producers to overcome fixed costs and produce goods that might otherwise not have been profitable. In that situation even those with high valuations can benefit.
“In the long run, history suggests that the invisible hand tends to favor consumers as each new wave of technology is invented and diffuses.”
I am not so sure this applies here. One could argue that the reason consumers usually come out ahead is because they can choose. However, we’re here talking about technology where the supplier controls the choice.
Consumers hate this. More accurately they hate to pay more than others for exactly the same thing at the same time. (But are of course happy to pay less!) The problem is how to prevent them from finding out. There have been articles over the years about this (eg claims that Amazon and airlines charged different amounts depending on browser used or repeat visits).
If the pricing differential is anything but absolutely trivial, then there exists an opportunity for a middleman to help me find out, and places charging me more can forget about my loyalty. The seller doesn’t only have to worry about being found out at the time of sale but also for a period afterwards. Eg if I find out 2 months later about being “over” charged, there will be no future loyalty from me.