Analytics
Analytics is, essentially, about analysing the usage and/or performance of some website, app, or digital product by tracking a variety of interactions and datapoints such as time spent on a particular page. They can tell you what is happening in a UI, but not why.
A classic example of a user analytics tool would be the ever present Google Analytics.
The pros and cons of analytics
Areas where analytics are particularly powerful
- Revealing potential problems reaching goals
- Identifying potential root causes of problems
- Quantitive data to supplement your qualitative research
Benefits
- Quickly validate/invalidate causation/correlation theories
- Help persuade data-orientated stakeholders
Challenges
- Analytics can tell you what, not why.
- Defining and scoping metrics - so much can be measured, but not everything is meaningful.
- Getting to answers - which metrics best answer you question(s)?
- Set up - how do you set up the system to show what you want to find out?
- Depending on what or how much you are tracking, it's easy to cross the line into tracking data which users may not want you to know (ex: in highly regulated industries or people just not wanting you bombarding them with tracking cookies. See the Superhuman email tracking fiasco)
- Depending on the tool and how much you're loading into the UI, you can start to negatively impact performance.
Tips
Do
- [[!Start with why]]
- Step back and think about how analytics data can supplement your current work and research process, instead of jumping right into the tool.
- Remember there are multiple ways to measure. If you start with why and questions to answer, some of them might be better (or only) answered with [[!Qualitative research]] or other methods.
- Create an Analytics measurement plan, starting from overarching goals rather than metrics.
Don't
- Track anything that might be 'interesting' without knowing why or how you will use that information. Analytics can become a ‘distracting black hole of “interesting” data without any actionable insight’
- Focus on the analytics tool instead of what it is there to support.
- Start with what you can measure with analytics, measuring the wrong things or creating an easy way to say ‘oh we succeeded’.