One challenge I face in working in analytics is people commonly misunderstand analytics. It seems to stem from a misunderstanding of what data is.
Data is just information.
When you think of data, you probably think of numbers. Lots and lots of numbers, maybe in a spreadsheet. And when you think of analytics, I’m guessing you think of complicated equations and graphs.
That’s a good start, but it is a very incomplete picture. Data, or information, comes in many shapes and sizes.
Analytics, in my mind, is simply the use of data to answer questions.
In basketball, there are often debates between the analytics approach and the eye test approach. The analytics approach focuses on how a player looks on the stats sheet: shooting percentage or effective field goal percentage. The eye test approach focuses on how a player looks on the court: their skill set or confidence.
I think the eye test is just one form of analytics. And the observations an analyst or fan makes (how is the player with the ball in their hands? how is their court vision? how is their shooting form?) are data.
And yet, most people distinguish between “analytics” and common analytical practices like the eye test. Why?
There are two elements that come to my mind about what distinguishes “analytics”:
- Focus on the questions to be answered
- Standardized data
Does the eye test include these factors? Well, not usually. So maybe I’m overselling it to say that’s analytics. But I think it certainly can be.
One of the most important books in the analytics realm is
How to Measure Anything. It is a classic resource for advice on how to measure the, well, seemingly unmeasureable.
The eye test has a high degree of subjectivity. Different observers will have different and biased observations even of the same player. But there are a number of steps you can take, such as a grading rubric or a peer review system, to standardize the observation data.
When it comes to focusing on the questions to be answered, you might need to use different systems for answering different questions. How you compare two basketball players of the same age at the same position may differ from how you compare two basketball players at different positions. Or two players from different eras. But you will need some clear system for standardizing and comparing your data.
I’m using examples from basketball because I spend a lot of time thinking about basketball analytics and it’s a more accessible example than most examples from my job. But I encounter a lot of this at work as well. My colleagues will think that the analytics team is separate from their work.
A successful analytics team needs to seek out their colleagues’ own “eye tests” and incorporate that data for value-producing organization-wide analytics.