Data science and leadership: opposite ends of the spectrum?

|Matt Offord

When I left the Royal Navy after 30 years, I set up a business of my own as a Management Consultant. But I had an identity crisis. This is not uncommon for men and women in jobs that have high levels of commitment. One’s job is also one’s identity. Since I was no longer a naval officer, who exactly was I? I had spent much of the last years of my service in research, training and leadership. I thought this is what I should do in my business. But I was undecided because I had also loved the data science techniques I had picked up doing leadership research. I couldn’t decide between business analysis and leadership. My indecision was reflected in my business, my meetings, my website and how I presented myself to prospective clients. They were as confused as I was. After a year of indecision, I had a break through. I was working with a client when I realised that many of the strategic decisions the client was making were based on poor data science. Leaders have to make decisions. Good decisions equals good leadership. Good decisions need good data. Finally, I realised exactly where I wanted to make a contribution and how I need to communicate it.

Last week I went to a seminar at Strathclyde Business School. Chatting afterwards with two MBA students at the Business School, I noticed the familiar drawing together of eye brows when I told them that I develop leadership by helping teams understand data. “Data and leadership are opposite ends of the spectrum, aren’t they?” asked one student? No they are not, but the problem is how we define leadership and how we define data. In 1620 Francis Bacon published the Novum Organum or New Method. Bacon established that science had to be based on facts which could be observed and measured. For this contribution (which was one of many) Bacon has been called the father of empiricism (basically: seeing is believing). We take it for granted today that something has be observed to be believed, it is fundamental to our scientific age. And yet, many areas of life appear to be in a pre-scientific state. Take politics, for example, how many political decisions are made based on careful analysis? Most would agree that too few decisions are based on data. The practice of leadership can often fall into this trap.

The problem with leadership today can be easily summarised by another philosopher, Thomas Carlyle. In 1840 Carlyle conducted a series of lectures called “The Great Man”. These lectures elevated leaders to demi-gods with authoritative, masculine traits and, furthermore, that mere mortals (non-leaders) need to have and obey such individuals. At the time not everyone agreed with this heroic depiction of leadership but it indicated that then, as now, the image of the heroic leader is compelling. Compelling and wrong. Anthropologists, such as Christopher Boehm who wrote “Hierarchy in the Forest” point out that humans are nonplussed about leadership. Leaders, after all, can command perks which followers cannot. Why would people want to give up their share, why would they do as they are told? It is because leaders fulfil a role making decisions and coordinating effort. This increases the likelihood of group survival.

The “Great Man” leader would be stymied in the Information Age. The need to quickly assess all of the data and make a quick decision is paramount to such a leader’s authority and credibility. But this is impossible today with the vast array of data in so many different formats and channels. The “Great Man” would become data-bound like a digital Prometheus. In the Information Age decisions are no longer deferred to the top because there isn’t time to cycle vast quantities of data up and down a chain of command. Decisions are more likely to be made at the edge of a network of well-informed specialists all contributing to a recognised representation of a Volatile, Uncertain, Complex and Ambiguous (VUCA) environment. Leadership now, more than ever, exists between people and within relationships enabled by technology.

But the quality of these decisions will depend on the quality of the information which supports them. This means leadership development needs to move beyond pre-scientific and heroic notions of grandeur to a more pragmatic data based approach to decision making. The leadership which resides within teams needs to make smart decisions as well as quick ones if organisations are to profit from data.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.