Evaluating Narrative Visualization:
A Survey of Practitioners

Figure 1: Years worked as narrative visualization practitioner

Figure 2: Number of narrative visualizations contributed to

Figure 3: Domains that narrative visualization practitioners work in

List of 'other' domains

• technology
• software product development
• advertising. (some other domains too but mostly advertising)
• a mix across all industries
• Transport, Banking, Industry, Technology
• Technology, sustainability and consultancy
• Retail
• Research communication
• Publically funded research
• Public policy and science Communication
• Professional services
• Media, retail, hospitality
• Marketing
• Market research, not for profit
• Management consulting and survey research
• Jobs
• International politics, history
• I work with USA-based, domestic nonprofit organizations. While these are related to health and eduation, I wouldn't say they are specifically healthcare or education organizations.
• I consult, so across many different domains
• Healthcare, Business, Education, Journalism, Entertainment, etc.
• Government
• Gender [Sex-disaggregated data on a variety of topics like economic opportunities, women's voice and agency, decision-making, health, education, gender-based violence, assets, entrepreneurship, attitudes/norms)
• Finance
• Energy and climate Change
• Ecology, Paleontology, Retail for Tires
• E-commerce insights, Product development roadmap, Company milestone timeline
• Digital/ Web Analytics
• Democracy & Society
• Community planning, environmental justice
• City Planning / Non Profit
• Businesses- mostly presentations of business results/ dashboards
• Business, art
• Business and research
• Business Analytics
• Business & Data Science
• Advertising agency, sometimes journalism

List of 'other' elements that were suggested by practitioners

• Design. Information design, especially. This is fundamentally a design problem.
• An actual narrative. Mapping to the mental model of the readers.
• Effective narrative flow (how you introduce someone to the information, a compelling story to be told using it)
• Quality of design, quality of data, quality of interaction
• Accuracy of portraying data
• What question or problem are you addressing and who is your audience
• The key elements sometimes depend on the audience
• Data chunking is part of it. The most essential step is making good decisions about what to include and exclude. The right content is essential. In health, what I'm hearing is that people want to understand what they should DO about the information they have received. So, next steps.
• Probably could have squeezed into cohesiveness...but consistency of scale.
• Storytelling

Figure 6: Whether a colleague or external experts review the visualization before its release. Click on legend to filter by domain

Figure 7: Whether visualization is reviewed using a set of pre-defined criteria. Click on legend to filter by domain

List of 'other' pre-defined criteria

• Precision and units are set up
• Data accuracy
• Accuracy of interpretation of source material
• We work with research participants in participatory design sessions to assess comprehensibility, actionability, and appeal.
• Presentation matches source data

Figure 9: Informal evaluation methods of review

Figure 10: Reasons given for no colleague or external expert review

Figure 11: Whether end-user testing is used to evaluate the narrative visualization

Figure 12: Reasons given for no end-user testing

List of 'other' methods of end-user evaluation

• I believe that visualizations should be self explanatory. In my opinion a visualization is successful when folks have a dialogue on the message the visualization reveals.
• informal reviews with qual feedback. Eventually I want to include focus groups as I am a moderator.
• Show it to some people, ask them questions.
• Small internal users for testing. Gathering general feedback or asking for feedback on specific vis/data explanation.
• Iterative participatory design sessions
• If deemed necessary, we pass it around the office and sit with the user - a colleague - to see if they "get" it. It's not a rigorous process. We are a newsroom.