This article was originally published on Olga’s Blog.
Recently I volunteered to assist the Digital Analytics Association with the review and recommendations for enhancement of the results and benchmarking sections of the self-assessment tool – one of the most popular member benefits in DAA. My job was to provide recommendations for the best way to compare and display data to end-users. I reviewed the back-end data, and advised on how to enhance the presentation of the self-assessment results.
The self-assessment involves presenting to members of the association how they compare against their peers, and where they could improve. It provides a way for users to reflect on their professional experience.
Though challenging, this was really good for me, just for the change in perspective alone. The main reason is this: I am used to data exploration with a business goal in mind – to improve conversion, to increase profit, to boost user engagement, etc. But this time I was building the visualization to show people where they are in their career, and how they compare to their peers. This is a very different perspective. It is very different from working through the dataset to find out how a business can benefit from the insights. It is about reporting back to the people whom the data represents.
Volunteering for DAA was a very easy decision, and it ended up being fun. It is good for me sometimes to step out of this ‘business goal’ bubble that I’m in.
Here are a few things that I learned. If you find it useful, please give me comment below.
Tips for visualizing survey data
- Transform before visualizing
Survey data is typically collected with tools like Survey Monkey, Google Forms, or similar. These tools aren't well suited to visualizing the results. Data that is generated by these tools comes in transactional form.
It is necessary to export the data and transform it, to optimize it for analysis and visualization. I used Analytics Canvas for data transformation.
- Get the right tools
The first step is to conduct exploratory analysis to find stories in the data.
To uncover significant relationships in the data, that are useful and can be acted upon, you need a tool that will allow you to quickly and easily slice and dice your data any way you want. A tool that is flexible enough to allow you to explore, without forcing you to fit your data into existing data representation templates.
I use Tableau as a scratchpad for data exploration.
- Go from the highest level of dimensionality to the lowest level
An interesting result may surface right away, or you may come back from your first look at the data empty-handed.
Don't stop there – continue to explore your data as you go from the highest level of dimensionality to the lowest level. The highest level of dimensionality could be: average skill level by country. The lowest level could be: average skill level of people in the US who score the lowest in the question number 3.
You don't have to go through every possible combination out there – just make sure to stay curious and alert. Facts come in, and you need to help them to get out!
- Make smart choices
Some questions are more complicated than others: hierarchical questions, Likert scale questions, multiple choice questions, free-form questions…
These all require a lot of time and careful review. You may choose to combine answers into groups by putting everybody who answers 8 or more into a "very satisfied" category, or you could segment the question into multiple visualizations. It’s your choice.
Do avoid trying to combine everything into one overloaded visual, though.
- Find interesting stories
The most interesting stories are revealed in correlations. For example, if you find that people who rank one question highly, typically rank another low, there might be a story right there that is worth talking about.
Hopefully, the structure of the raw data that your survey tool provides will allow you to do this kind of analysis. Otherwise, you will need to create a whole set of new dimensions: people who choose option 1 in question 1, people who answer yes in question 2, people who scored 8 or more in questions 5 to 8, etc.
Select the lowest level of detail when creating these dimensions.
- Blend data
Another powerful discovery method is to blend the survey data with other sources. In this case, we are able to match survey responses with user profiles, to bring more depth to the story.
- Meet interests of your users
When working through the dataset, think of various interests that your final users might have.
For example, when I was designing the visualization of a self-assessment tool, I highlighted the differences that one member of the association, the survey responder, had from other members.
- Include interactive controls
Allow some flexibility for the end user.
Many visualization tools include interactive controls. The end user can filter out information that they may consider irrelevant, or they may want to dive in to learn more.
For example, the end user may uncheck the "male" option because they are only interested in answers from the female respondents.
- Show the stats
One easy to overlook, but very important, part of the survey result presentation is to include the information about the significance of the findings of the survey.
At the very minimum, include the total number of respondents, and the time period during which the responses were collected.
If your visualization includes a comparison – for example, year over year change – provide information about the statistical significance of the difference, using suitable statistical methods.
- Things to check
Before presenting your visualization to the user, check the following: that fonts are used consistently, that charts include tick marks and gridlines where necessary, that one color palette is used across all the charts, that legends are present where necessary, that Interactive controls are working properly, that headings, subheadings and other text is written in simple language that everyone can understand, and that all charts adjust automatically across various screen sizes and devices.
Thank you for reading this article. If you haven’t taken self-assessment, I encourage you to do so. It not only provides you with the benchmarking information, but also offers links to helpful resources that are selected based on your answers.