The unintended consequences of France’s ban on statistical analysis of Judges

If someone had said that I would be writing a blog post to consider a law that might imprison people for conducting statistical analysis on publicly available data, I would have thought that was unlikely because who would ever propose, let alone enact, such a law?

The other day we got our answer: France! The very country that produced Laplace, Pascal and Guerry!

The law in question is Article 33 of the Justice Reform Act which was amended to read as follows:

The identity data of magistrates and members of the judiciary cannot be reused with the purpose or effect of evaluating, analysing, comparing or predicting their actual or alleged professional practices.

That maximum sentence (yes, criminal sentence) for violating this is 5 years. This puts ‘statistics’ in the category of a crime. Notice that it is actually using the data for a specific purpose and not something else like publishing outcomes that violate privacy.

What this means is that you cannot do statistical analyses that compare judges. For instance, you might examine the decisions by members of the Court of Cassation and show that on average Bertrand Louvel was less likely to overturn appeals in civil matters than Agnès Mouillard. Or you might find that Bertrand Louvel was less likely to overturn appeals in criminal matters where the defendant is a woman than Agnès Mouillard. (Just to be clear, I have not looked at the public data, so this is purely hypothetical.) This is both the sort of analysis that parties might be interested in during a legal process but also analysis that is critical for analysing the performance of the bench.

Clearly, this type of ban is a big problem for LegalTech companies using AI to predict various judicial outcomes in France. The fact that the ban has taken place at all suggests that there is actually meaningful analysis in that data such that you might be concerned about forum shopping. Of course, there is other data that one can use for this purpose. Before judicial data was publicly available, one suspects that informal networks and other data gathering processes may well have existed in France that people used for this same purpose. Those activities remain legal so it is only adding an officially sanctioned dataset for that purpose that causes illegality. As with activities that are easy to do but are criminalised, the likely effect is to push those activities underground or offshore. In other words, it is more a loss in transparency than a loss in ‘service.’

What I want to point out here however is that the main victims of this ban are likely to be the judges themselves. Suppose (and again I am speaking hypothetically), someone was to publish an analysis on the internet claiming to have used the publicly available data to generate a statistical analysis of which judges are more likely to make poor decisions just before lunch. They published that list and the methodology so that anyone could replicate that analysis and confirm whether the purported outcomes were true or not. And just to make it interesting, suppose the entire claims were fake.

Normally, none of this would be a problem. A statistician could replicate the methodology and examine the data to expose the fraud. However, were they to do that, they would be liable for criminal prosecution that would be decided by perhaps the same judges named in the fake analysis. What a pickle?

My point is this: this ban exposes the judges themselves to false claims and removes any means of defence. Indeed, under the law, I wonder if the judges or the government could even mount that defence. If someone ever did the terrible hypothetical I just speculated about, it would surely represent some sort of poison pill for this law. France needs to learn that statistical analysis is a defender of truth and so needs to decide whether it wants that.