Consider for a moment two scenarios.
One, a malicious energy reporter tasked with reviewing an electric car decides he is going to fake the review. Part of this fictional narrative, is that the car needs to run out of battery power sometime in the review. He arrives at one of the charging stations, and instead of plugging in, spends a few minutes circling the parking lot trying to drain the battery.
Second, an energy reporter is tasked with reviewing the potential of a new electric car charging network. He arrived at one of the charging location in the dark, and can’t find the charging station. He drives around the parking lot several times looking for it, before finding it and charging his car.
Here is the thing. As Craig Silverman recently pointed out to me, we actually have no idea, based on the interpretation of the review data released by Tesla, which narrative is true. All the data shows is a car driving around a parking lot. And here in lies the principle lesson from the whole Tesla affair: Data is laden with intentionality, and cannot be removed from the context in which it was derived. We do not know, from these data alone, what happened in that parking lot.
David Brooks touched on this very issue in a recent (somewhat overly maligned in my opinion) column on the limits to big data. While his Italian banking analogy felt misplaced, there is actually a large amount of research backing up his general themes. And his point that data struggles with context, is directly relevant to the Tesla dispute:
Data struggles with context. Human decisions are not discrete events. They are embedded in sequences and contexts. The human brain has evolved to account for this reality. People are really good at telling stories that weave together multiple causes and multiple contexts. Data analysis is pretty bad at narrative and emergent thinking, and it cannot match the explanatory suppleness of even a mediocre novel.
In the case of the Tesla review, it is this context that was both poorly recorded by Broder, and which is missing from the Tesla data analysis. This does not mean the analysis is wrong. But it does mean it’s incomplete.
A couple of further points about the role data played in this journalistic dispute.
First, the early triumphalism against the New York Times in the name of both Telsa and data transparency, were clearly premature. In Tesla’s grand rebuttal, Musk clearly overplayed his rhetorical hand by arguing that the review was faked, but he also overstated both the case he could make with the data, as well as the level of transparency that he was actually providing. Tesla didn’t release the data from the review. Telsa released their interpretation of the data from the review. This interpretation took the form of the graphical representation they choose to give it, as well as the subjective write-up they imposed on it.
What is interesting is that even with this limited and selective data release (ie, without the raw data), entirely different narrative interpretations could be built. Broder and his New York Times team presented one. But Rebecca Greenfield at the Atlantic provided an even more detailed one. There are likely elements of truth scattered across these three interpretations of the data. But they are just that – interpretations.
Second, the only person who can provide the needed context to this data is Broder, the reviewer himself. And the only way he can convey this information is if we trust him. Because of his “problems with precision and judgement,” as the New York Times’ Public Editor Margaret Sullivan put it, his trust was devalued. So the missing journalistic piece to this story is lost. Even in a world of data journalism, trust, integrity and journalistic process still matter. In fact, they matter all the more.
Finally, we can’t lose sight of the outcome Tesla wanted from this. They wanted PR for their new vehicle. So amongst all of the righteous indignation, it is worth noting that journalistic principles are not their core objective – good stories about their products are. These may or may not be aligned. This is why, for example, Broder was given significant support and access during his review trip (some of which ultimately proved to be misguided).
An example of this discrepancy surrounds the one clear reality about the Model S (and presumably electric cars in general) that was revealed in the review – they lose significant charge when not plugged in during cold weather. Now, Tesla would rather this fact had not emerged in the review. But it did. And as Steven Johnson pointed out, this has significant implications, specifically for city drivers. For one, it makes parking the Tesla S on the street in the winter (what many urban dwellers would have to do), largely impractical.
So, to recap. The Tesla Affair reinforces that: data does not equal fact; that context matters enormously to data journalism; that trust and documentation are even more important in a world of data journalism; and that companies will continue to prioritize positive PR over good journalism in reviews of their products.