Data, instinct, and what the US election tells us about both

  • Edward Haigh

On the eve of the US election, FiveThirtyEight—the gold standard of election forecasting sites, run by the esteemed data journalist Nate Silver—confidently predicted a comfortable Biden victory, with the former vice president expected to pick up 348 electoral college votes to Trump’s 190. In reality, the Biden/Harris ticket received only 306 votes—enough to win the White House, but a remarkable distance off from Silver’s prediction. 

The fact that FiveThirtyEight seems to have got it so wrong (again) isn’t actually the most interesting thing. These are extraordinary times we’re living in and, as my own organisation knows only too well, predicting stuff has rarely, if ever, been harder. Whether you’re trying to figure out how organisations will use professional services, or how voters will vote, long-established methodologies need revisiting and recalibrating, and a period of what you might charitably call “fine-tuning” (read: running around going “Oh shit, that didn’t work!”) inevitably follows.  

More interesting is the fact that we were much closer. And when I say we, I mean the seven of us who entered the election 2020 office sweepstake. All of us predicted a Biden win but, averaged out, we predicted 312 electoral votes for Biden and 225 for Trump, which means we got it a lot less wrong than FiveThirtyEight did.  

Some part of our collective success should probably be attributed to luck, and it’s also worth acknowledging that relativity has a role to play here: We were responding to FiveThirtyEight’s prediction in making our own. If they had forecast a Trump landslide, I suspect our prediction would have moved sharply to the right.  

But I can’t quite escape the idea that gut instinct—hunch—played an important part. Now, if it was just our hunch that delivered a more accurate prediction then that would be the end of the story. But did you speak to anyone else who had the same hunch? I did. In fact, everyone I spoke to had the same gut feeling: This is going to be a lot closer than the polling suggests. And I think that raises the question of whether there’s an important role in this world for people who look at data and then, using a combination of experience and instinct, decide how much and which bits of it to listen to, and what to do next. Isn’t that what FiveThirtyEight needed? Someone to say, “Listen guys, I know the data’s telling us that Biden’s going to win easily, but I think it’s going to be closer than that.”  

Of course, that invokes a troubling conversation about who that person is, but isn’t that the same sort of conversation businesses have always had about who should lead them? Someone’s going to have to make a call. Someone’s going to have to translate everything they’ve heard and seen into a decision. And in the course of doing so, they’re almost certainly going to have to use a bit of hunch. 

Perhaps, against the backdrop of a business world that seems hell-bent on rewiring its neural pathways so they’re hooked up to data, that’s why there’s a growing interest in the role of the translator. In fact, as long ago as February 2018 McKinsey published an article titled Analytics translator: The new must-have role in which it advocated having people in organisations who could bridge the gap that so often exists between data and action. Indeed, in the article it predicted that demand for translators might reach two to four million by 2026 in the United States alone, a prediction that doubtless informed its decision to create its own academy to train translators.  

I suspect that will turn out to have been a really smart move, because the lesson that’s surely staring us all in the face right now is that having lots of data, and even lots of data scientists, isn’t enough. Unless you’ve got someone who knows what to do next then the chances of doing the wrong thing, or (perhaps worse) nothing, are too great. So, what does McKinsey have to say about the capabilities these people need? According to the article there are four: domain knowledge, general technical fluency, project management skills, and an entrepreneurial spirit (which it defines as the enthusiasm, commitment, and business savvy to navigate the many technical, political, and organisational roadblocks that can emerge). Sound familiar? I think they sound a lot like the sort of capabilities you’d find in a consultant.  

Perhaps we shouldn’t be surprised that McKinsey sees consultants as the solution to the problem, but I suspect they could be. It seems to me that there’s going to be a role for someone who can synthesise better, more intelligent interpretations of data, and trusted, credible, experienced-based hunches. Indeed, research we’ve just completed for the latest report in our Emerging Trends Programme, The Consultant of Tomorrow, finds data & analytics skills to be the top-ranked skill for consultants to have, in the eyes of clients. And I wonder if a consulting industry that has spent much of the last few years learning to speak the language of data and technology now needs to put those skills to good use by combining them with the language of business that was always its mother tongue to do just that, and become the translator that the world is starting, urgently, to need. 

At least, that’s my hunch.