How to Leverage UX Analytics for Data-Driven Product Design

Qualitative data offers a direct assessment of the usability of a given product’s design.

Designers watch participants struggle with specific UI elements and infer which aspects of the design are problematic and which work well.

Session recordings are an example of qualitative research.

Quantitative data offers an indirect assessment of a given design system’s usability—in other words, we’re talking numbers.

These numbers can either:

1. be based on the users’ performance of a given task (e.g. task completion times, success rates, number of errors);
2. reflect the participants’ perception of the design’s usability (e.g. satisfaction ratings).

A great example of quantitative research are Google Analytics.

The question may seem intuitive, but it is, in fact, incorrect.

The answer is: neither is better. Because that’s not what this is about.

What you should do is combine qualitative and quantitative data in order to fully understand the reasons and motivations behind the trends and behavioral patterns you’re observing.

This combination of the qualitative and quantitative approaches allows you to evaluate the performance of your UX design and your whole application in a holistic and coherent way.

Qualitative and quantitative research can be used together to further drive the direction of product development.

A good way to combine them is to use these two sets of strategic questions:

1. What happened? → Why did it happen?
2. Is this unusual? → Is it happening to more users?

Note that each of the sets assumes a different relationship between the two types of research.

What happened? → Why did it happen?

This type of research represents a move from quantitative research to qualitative research.

Qualitative research can help you discover variables and explain why certain things happen after you discover them through statistical analysis.

Is this unusual? → Is it happening to more users?

Conversely, the second type of research is an example of moving from qualitative research to quantitative research.

Doing so allows you to discover individual trends and/or preferences, then support these claims using larger samples.

What’s the most common scenario of using UX analytics? Installing the Google Analytics Tracking Code as-is. 

However, there are several problems with going about it this way:

1. It gives you only quantitative data, telling you the “what” without the “why”;
2. Google Analytics needs to be set up correctly in order to give valuable insights (events, segments, etc.);
3. Managing it usually requires a full-time Analyst position to make sense of all of the data;
4. It’s like looking for a needle in a haystack—tracking the wrong metric will be fruitless and eventually prove meaningless.

Fortunately, there’s a better way to do this.

The rule of thumb here should always be: define your goals first, then decide on the metrics.

If you don’t define the goal clearly, it will be impossible to determine the best method of tracking the performance or user experience.

Here are more specific steps you can take:

a) Define the KPIs

Develop a strategy by setting up analytical goals.

b) Select the toolkit

Choose the right set of tools for a particular context—desktop vs. mobile users, new vs. existing users, etc.

c) Create the scenarios

Map the scenarios for measuring results based on existing user flows.

d) Develop team cooperation

Create an implementation brief for the development team, then check the feasibility.

e) Process the data

Draw conclusions and formulate new test hypotheses for further exploration.