Recently I volunteered to assist the Digital Analytics Association with the review and recommendations for enhancement of the results and benchmarking sections of the self-assessment tool – one of the most popular member benefits in DAA.
My job was to provide recommendations for the best way to compare and display data to end users.
I reviewed the back-end data, and advised on how to enhance the presentation of the self-assessment results.
I can't share the results with you here, but here are some of my sketches.
The self-assessment involves presenting to members of the association how they compare against their peers, and where they could improve. It provides a way for users to reflect on their professional experience.
Though challenging, this was really good for me, just for the change in perspective alone.
The main reason is this: I was not doing a data exploration with a business goal in mind – to improve conversion, to increase profit, to boost user engagement, etc. I was building this visualization to show people where they are in their career, and how they compare to their peers.
This is a very different perspective. It is very different from working through the dataset to find out how a business can benefit from the insights.
It is about reporting back to the people whom the data represents. I am essentially creating a mirror that people will look into and say, “yeah, that’s me”. And I’m making it with data.
I realized that I wanted to be sure that people connected with what I was creating. Volunteering for DAA was a very easy decision, and it ended up being fun. It is good for me sometimes to step out of this ‘business goal’ bubble that I’m in.
Here are a few things that I learned. If you find it useful, please give me shout on Twitter, or comment below.
Tips for visualizing survey data
Transform before visualizing Survey data is typically collected with tools like Survey Monkey, Google Forms, or similar. These tools aren't well suited to visualizing the results. Data that is generated by these tools comes in transactional form. It is necessary to export the data and transform it, to optimize it for analysis and visualization.
Get the right tools The first step is to conduct exploratory analysis to find stories in the data. To uncover significant relationships in the data, that are useful and can be acted upon, you need a tool that will allow you to quickly and easily slice and dice your data any way you want. A tool that is flexible enough to allow you to explore, without forcing you to fit your data into existing data representation templates. I use Tableau as a scratchpad for data exploration.
Go from the highest level of dimensionality to the lowest level An interesting result may surface right away, or you may come back from your first look at the data empty-handed. Don't stop there – continue to explore your data as you go from the highest level of dimensionality to the lowest level. The highest level of dimensionality could be: average skill level by country. The lowest level could be: average skill level of people in the US who score the lowest in the question number 3. You don't have to go through every possible combination out there – just make sure to stay curious and alert. Facts come in, and you need to help them to get out!
Make smart choices Some questions are more complicated than others: hierarchical questions , Likert scale questions, multiple choice questions, free- form questions… These all require a lot of time and careful thought. You may choose to combine answers into groups by putting everybody who answers 8 or more into a "very satisfied" category, or you could segment the question into multiple visualizations. It’s your choice. Do avoid trying to combine everything into one overloaded bar chart, though.
Find interesting stories The most interesting stories are revealed in correlations. For example, if you find that people who rank one question highly, typically rank another as low, there might be a story right there that is worth talking about. Hopefully, the structure of the raw data that your survey tool provides will allow you to do this kind of analysis. Otherwise, you will need to create a whole set of new dimensions: people who choose option 1 in question 1, people who answer yes in question 2, people who scored 8 or more in questions 5 to 8, etc. Select the lowest level of detail when creating these dimensions.
Blend data Another powerful discovery method is to blend the survey data with other sources. In this case, we are able to match survey responses with user profiles, to bring more depth to the story.
Meet interests of your users When working through the dataset, think of various interests that your final users might have. For example, when I was designing the visualization of a self-assessment tool, I highlighted the differences that one member of the association, the survey responder, had from other members.
Include interactive controls Allow some flexibility for the end user. Many visualization tools include interactive controls. The end user can filter out information that they may consider irrelevant, or they may want to dive in to learn more. For example, the end user may uncheck the "male" option because they are only interested in answers from the female respondents.
Show the stats One easy to overlook, but very important, part of the survey result presentation is to include the information about the significance of the findings of the survey. At the very minimum, include the total number of respondents, and the time period during which the responses were collected. If your visualization includes a comparison – for example, year over year change – provide information about the statistical significance of the difference, using suitable statistical methods, such as a t-test.
Things to check <