8. Conclusion
8.1 Improvement and future work
The thing that could be improved in the tested visualisations is the design of some of them. While the second visualisation performed particularly well in its tasks and was attractive to the users, other visualisations showed some space for improvement. The first visualisation could be redesigned due to the answer provided by one participant, mentioning that having bars that stretch on both sides of the funnel chart is hard to compare – they should be only stretching on one side. The third visualisation is something that definitely needs to be improved and redesigned (one participant stated that she “does not necessarily like these charts”). That fact opens up a space for having a new visualisation designed instead of the pie charts. Also the fourth visualisation is up for a redesign since one participant mentioned that the “clicked” stage does not have to be fulfilled in order to progress to the next one.
In an ideal situation with enough manpower, time and money resources better testing could be conducted. More users would be tested and ratio of the A/B test user groups would be 50/50. Also, due to more time and money resources for the research, different visualisations would be created and the evaluation would be conducted on the two or more different visualisation versions instead of on the old visualisation versus the new one. During the testing
responsible for writing down what the users are saying. That person would also be responsible for taking the notes about users’ facial expressions and body language.
References
[1] C. Ware, Information Visualization: Perception for Design, Morgan Kaufman, 2000. [2] D. R. Voss, “The Development of a Model for Nonverbar Factors Impacting The
Design of Visual Information,” PhD thesis, Wayne State University, Detroit, 2008. [3] R. Mazza, “Introduction to Information Visualisation,” Faculty of Communication
Sciences, University of Lugano, Lugano, 2004.
[4] K. Storbacka, T. Strandvik and C. Grönroos, “Managing Customer Relationships for Profit: The Dynamics of Relationship Quality,” International Journal of Service Industry Management, vol. 5, no. 5, pp. 21-38, 1994.
[5] G. Ellis and A. Dix, “An explorative analysis of user evaluation studies in information visualisation,” ACM, pp. 1-7, 2006.
[6] K. Korać, “Impact of Data Visualisation on Users in CRM Systems,” Topic Proposal for the Master Thesis, University of Twente, June, 2016.
[7] K. Korać, “Data visualisation influence on a CRM system’s value proposition,” I&E Minor Thesis, University of Twente, EIT Digital Master School June 24, 2016. [8] L. M. Given, The SAGE Encyclopedia of Qualitative Research Methods, Thousand
Oaks, CA: SAGE Publications, Inc., 2008.
[9] K. French, “The Data Basics You Need to Know,” 3 December 2014. [Online]. Available: http://www.columnfivemedia.com/data-basics-need-know.
[10] “Numerical Data: What’s The Difference Between Discrete and Continuous?,” Australian Bureau of Statistics, [Online]. Available:
http://www.abs.gov.au/websitedbs/CaSHome.nsf/Home/CaSQ+3B+NUMERICAL+DA TA:+WHAT%27S+THE+DIFFERENCE+BETWEEN+DISCRETE+AND+CONTINUOU S.
[11] “Types of Variable,” Laerd Statistics, [Online]. Available:
https://statistics.laerd.com/statistical-guides/types-of-variable.php.
[12] “What researchers mean by cross-sectional vs. longitudinal studies,” Institute for Work & Health, 2009.
[13] S. Few, “Eenie, Meenie, Minie, Moe: Selecting the Right Graph for Your Message,” 18 September 2014. [Online]. Available:
https://www.perceptualedge.com/articles/ie/the_right_graph.pdf.
[14] P. Ghosh, “Cave paintings change ideas about the origin of art,” BBC News, 8 October 2014. [Online]. Available: http://www.bbc.com/news/science-environment- 29415716.
[15] R. D. Guthrie, “Preface: Reassembling the Bones,” in The Nature of Paleolithic Art, Chicago, University of Chicago Press, 2006 .
[16] E. J. Beh and R. Lombardo, Correspondence Analysis - Theory, Practice and New Strategies, John Wiley & Sons, Ltd, 2014.
[18] M. Friendly, “Milestones in the history of thematic cartography, statistical graphics, and data visualization,” 24 August 2009. [Online]. Available:
http://www.math.yorku.ca/SCS/Gallery/milestone/milestone.pdf.
[19] J. R. Beniger and D. L. Robyn, “Quantitative graphics in statistics: A brief history,” The American Statistician, p. pp. 1–11, 1978.
[20] W. S. Cleveland and R. McGill, “Graphical Perception and Graphical Methods for Analyzing Scientific Data,” Science, vol. 229, pp. 828-833, 1985.
[21] M. Soegaard, “Gestalt principles of form perception,” Interaction Design Foundation, [Online]. Available: https://www.interaction-design.org/literature/book/the-glossary-of- human-computer-interaction/gestalt-principles-of-form-perception.
[22] R. Dewey, “"The Whole is Other than the Sum of the Parts",” [Online]. Available: http://www.intropsych.com/ch04_senses/whole_is_other_than_the_sum_of_the_parts .html.
[23] S. Bradley, “Design Principles: Visual Perception And The Principles Of Gestalt,” 28 March 2014. [Online]. Available:
https://www.smashingmagazine.com/2014/03/design-principles-visual-perception- and-the-principles-of-gestalt/.
[24] H. Stevenson, “Emergence: The Gestalt Approach to Change,” Cleveland Consulting Group, Inc, [Online]. Available:
http://www.clevelandconsultinggroup.com/articles/emergence-gestalt-approach-to- change.php.
[25] L. Mastin, “The Human Memory - Memory Encoding,” 2010. [Online]. Available: http://www.human-memory.net/processes_encoding.html.
[26] M. Dubakov, “Targetprocess - Visual Encoding,” September 2012. [Online]. Available: https://www.targetprocess.com/articles/visual-encoding/.
[27] B. &. Company, “Customer Relationship Management, Guide - Bain & Company,” 10 June 2015. [Online]. Available:
http://www.bain.com/publications/articles/management-tools-customer-relationship- management.aspx.
[28] W. Reinartz, M. Krafft and W. D. Hoye, “The CRM Process: Its Measurement and Impact on Performance,” Journal of Marketing Research, 2004.
[29] B. L. Eisenfeld, E. Kolsky and T. Topolinski, “Gartner Survey: 42 Percent of CRM Software Goes Unused,” Gartner, 28 February 2003. [Online]. Available:
https://www.gartner.com/doc/387369/gartner-survey--percent-crm.
[30] M. Rouse, “customer relationship management (CRM),” Searchcrm - Techtarget, November 2014. [Online]. Available: http://searchcrm.techtarget.com/definition/CRM. [31] M. Rouse, “Sales force automation (SFA),” Whatis - Techtarget, January 2011.
[Online]. Available: http://whatis.techtarget.com/definition/sales-force-automation- SFA.
[32] W. Seliga, “The Three Main Components of a CRM System,” SWC - Technology partners, 9 July 2012. [Online]. Available: https://www.swc.com/blog/customer- relationship-management/the-three-main-components-crm-system.
[34] C. S. Long, R. Khalafinezhad, W. K. W. Ismail and S. Z. A. Rasid, “Impact of CRM Factors on Customer Satisfaction and Loyalty,” Procedia Economics and Finance, vol. 23, pp. 563-567, 2015.
[35] H. Swift, “How to Grow Customer Satisfaction with Your CRM,” RingLead, 17 June 2014. [Online]. Available: https://www.ringlead.com/blog/crm-customer-satisfaction/. [36] J. T. Bowen and S. Chen, “The relationship between customer loyalty and customer satisfaction,” International Journal of Contemporary Hospitality Management, vol. 13, no. 5, pp. 213 - 217, 2001.
[37] R. S. Hassan, A. Nawaz, M. N. Lashari and D. F. Zafar, “Effect of Customer Relationship Management on Customer,” in 2nd Global Conference on Business, Economics, Management and Tourism, Prague, 2014.
[38] X. Liang, “CRM business cloud computing,” in International Conference on Innovative Computing and Cloud Computing, 2011, Beijing, 2013.
[39] Z. Plaksij, “Why Cloud-based CRM Suits the Needs of Small Businesses,”
SuperOffice, 27 July 2016. [Online]. Available: http://www.superoffice.com/blog/cloud- based-crm-for-small-business/.
[40] J. Oetting, “A Simple Guide to Choosing the Right Chart for Your Data,” HubSpot, 15 May 2015. [Online]. Available: http://blog.hubspot.com/marketing/data-visualization- choosing-chart.
[41] S. Coles and J. Rowley, “Creating effective graphs and charts,” Industrial Management & Data Systems, vol. 97, no. 3, pp. 114-118, 1997.
[42] C. Finch, “Advantages & Disadvantages of a Pie Chart,” [Online]. Available: http://www.ehow.com/list_6715678_advantages-disadvantages-pie-chart.html. [43] “Sales Representative Job Description,” Snagajob, [Online]. Available:
http://www.snagajob.com/job-descriptions/sales-representative/.
[44] R. Blewett, “The Importance of Data Visualization to Business Decision Making,” 12 June 2011. [Online]. Available:
http://multipeers.itpeers.com/fotos/gca/the_importance_of_data_visualization_to_busi ness_decision_making_83713439550bf20f8eaddf.pdf.
[45] A. Shen-Hsieh and M. Schindler, “Data Visualization for Strategic Decision Making,”
Case Studies of CHI2002, ACM, 2002 .
[46] J. Haight, “How Data Visualization Empowers Decision Making and Who Is Getting Us There,” 13 October 2014. [Online]. Available: http://bluehillresearch.com/how-data- visualization-empowers-decision-making-and-who-is-getting-us-there/..
[47] “HTML,” Mozilla Development Network, 1 July 2016. [Online]. Available: https://developer.mozilla.org/en-US/docs/Web/HTML.
[48] “JavaScript,” Mozilla Development Network, 6 June 2016. [Online]. Available: https://developer.mozilla.org/en-US/docs/Web/JavaScript.
[49] “What Is Angular?,” AngularJS, [Online]. Available: https://docs.angularjs.org/guide/introduction.
[50] “SVG,” Mozilla Development Network, 9 February 2016. [Online]. Available: https://developer.mozilla.org/en-US/docs/Web/SVG.
[52] M. Bostock, V. Ogievetsky and J. Heer, “D3 Data-Driven Documents,” IEEE
Transactions on Visualization and Computer Graphics, vol. 17, no. 12, pp. 2301-2309 , 2011 .
[53] J. M. Sonberg, “How to Build a Strong A/B Testing Plan That Gets Results,” 2014. [Online]. Available: http://conversionxl.com/how-to-build-a-strong-ab-testing-plan-that- gets-results/.
[54] U. Bar-Joseph, “8 Rules of A/B Testing – The Art in Marketing Science,” Search Engine Watch, 6 August 2014. [Online]. Available:
https://searchenginewatch.com/sew/how-to/2196754/8-rules-of-a-b-testing-the-art-in- marketing-science.
[55] J. Nielsen, “Interviewing Users - Nielsen Norman Group,” 26 July 2010. [Online]. Available: https://www.nngroup.com/articles/interviewing-users/.
[56] J. Ludeman, “Edward Tufte: The DaVinci of Data,” 22 February 2016. [Online]. Available: https://sites.psu.edu/julieludeman/2016/02/22/edward-tufte-the-da-vinci-of- data/.
[57] S. Card, J. Mackinlay and B. Shneiderman, Readings in information visualization: using vision to think, San Francisco: Morgan Kaufmann Publishers Inc., 1999.
Appendix – Summary of users’ answers (encoded)
Asked questions
A feature (control) - old
visualisations
B feature (test) - new visualisations
Test 1 - Funnel chart
A raw table Interactive funnel chart vis.1. What data can you tell us from this presentation?
All the respondents
recognized only information about basic lead information and current status. No conversion rate was recognized here.
All the respondents recognized sales conversion rates and their pipeline progress in the visualisation, among the other information contained within the table like basic lead information, current status, etc.
2. Could you or how fast could you tell us how many leads were in a specific status?
If the number of contacts was relatively low, they could count them; otherwise it is impossible task;
~ “I would clearly be
inconvenient to count all of
them” - participant nr. 2
Respondents easily answered this question by reading the data from the chart. Three of them needed brief
assistance in understanding the flow of the leads; in the end managed to perceive the right information.
3. Could you tell us what was the progress of the leads over the stages in last month?
None of the respondents managed to answer this question.
All the respondents managed to read and comprehend progress of the number of leads over their statuses in pipeline.
4. Could you tell us the conversion rates over the stages?
As before, since they weren’t able to comprehend the leads’ progress over time, respondents were constrained from
interpreting conversion rates over stages.
All respondents successfully perceived and interpreted this information. Some of them had to be provided a bit of extra guidance, but in the end comprehended the information completely.
5. Could you make a comparison of conversions and conversion rates of currently presented time span with some other one?
No respondents was able to answer this question.
All of the respondents managed to answer the question by triggering the comparison switch and comparing current and past time span.
6. What is your opinion about the data and the
visualisation? What do you think of it in terms of being helpful for your work?
Respondents found this information useful, but would prefer some
manipulations with the data for it to be useful.
Data was also found useful for all respondents.
Visualisation is appealing and useful in terms of saving time and making better decisions;
Asked questions
A feature (control) - old
visualisations
B feature (test) - new
visualisations
Test 2 - Line &
Stacked bar chart
A raw table Interactive line and column charts (straight lines and
labeled columns)
1. What data can you tell us from this presentation?
Respondents recognized the number of interactions over time; saw only simple numbers.
Respondents discovered certain patterns and behaviour of number of interactions over time.
2. Could you track the progress of your data easly?
The answer was not positive since the respondents were presented by mere numbers - it was difficult for them to track the progress quickly.
All the respondents answered positively; it was easy for them to track the progress of the data.
Additional curved line chart and empty column chart
3. Could you tell us what is happening with the
interactions in Week 2 of May till the 9th in the line chart (under curved line) and from Sat 5 Dec till Sun 6 Dec?
Respondents detected slight rise and then fall (and vice- versa) of the numbers of interactions even though they are consistent.
Respondents perceived the progression of the data accurately.
4. Could you tell us how many interactions are there per one column (individually and together)?
Respondents managed to answer but had slight delay to scale the columns.
With labeled columns, all of the respondents answered the question in a glimpse.
5. What is your opinion about the data and the visualisation? What do you think of it in terms of being helpful for your work?
Table presentation was useful, but certainly not enough; other visualisation was useful complement but needs some redesign in order to completely convey right information
~ “The data I see here is
useful for me, but it’s hard to understand the big picture
from the table” - participant
nr. 1
Complementing the table, this types of charts were very useful (according to the participants), appealing and time saving and enabled more efficient decision making. Because of the visualisation, data was found more useful by most of the users.
~ “Seeing this data visually I
am actually discovering its
Asked questions
A feature (control) - old
visualisations
B feature (test) - new
visualisations
Test 3 - Pie charts
A raw table Interactive pie charts1. What data (about the datapoints) could you tell us from this visualisation?
By reading the datapoint column, participants were able to perceive how many datapoints are available for one contact but couldn’t find out which datapoints (since abbreviations were used).
Participants recognized the number of available
datapoints for a single contacts and their names.
2. How many datapoints are there? How many of contacts have active datapoints?
Having a large number of contacts, it was impossible for them to count the
distribution of datapoints; the task became time
consuming and inconvenient.
Participants easily answered this question by reading the visualisation - it was possible to see the total number of datapoints and how many of them were active.
3. Could you tell us, out of all active ones how many, what is the presence of all datapoints? How many active datapoints are present (per type)?
Even with a low number of contacts, it was inconvenient for the participants to answer this question since it
required data aggregation and then additional operations (mean).
All the respondents managed to read and comprehend numbers and percentage of active datapoints and datapoints per type.
4. What is your opinion about the data and the visualisation? What do you think of it in terms of being helpful for your work?
Seems useful, but participants could have extracted more inside information and gain more insight. Since it was giving additional info about the leads, the data itself was found useful but respondents would have liked to see it in real context.
Visualisation seems useful, providing easy access to aggregated information (time saving and decision making oriented). Due to this reason, data was also found useful for the participants
- however, one participant was not satisfied with the visualisation (wanted different visualisation for this useful data)
~ “I think the data is useful,
but I believe some other
visualisation would be better” -
Asked questions
A feature (control) - old
visualisations
B feature (test) - new
visualisations
Test 4 - Email (single)
campaign timeline
Visualisation in form of unconnected rings
Static timeline with dates
1. What data could you tell us from this visualisation?
Participants were able to perceive how many emails are in which stage (single campaign - only 1).
Participants recognized the progress of the email single campaign and in which state is it now and which ones were completed and when.
2. How many emails are we talking about here?
One email, but potentially more.
Respondents perceived easily that visualisation is related only to the single campaign.
3. What is the current status of the campaign?
Participants answered that one sent, one was received (which created confusion in their perception about single campaign).
All participants easily perceived current campaign status and put it within a time context.
4. What is your opinion about the data and the visualisation? What do you think of it in terms of being helpful for your work?
Participants found this visualisation useful for having multiple campaigns, but can be confusing for a single campaign. Therefore, there was a confusion in what kind of data is the visualisation conveying.
~ “Why is there one sent and
one open mail if it’s only a single campaign? Oh wait.. ”
- participant nr. 1
Participants found the visualisation attractive and useful for it saves their time and enables them to meet their leads more by observing time between different steps in email campaign (i.e. how long does it take for lead to answer the email).
Opinions about the visualisation
To determine how the visualisation actually helps the users of the system, their opinion was asked as the final question of the interview. Key snippets were taken from their answers and listed below, to provide help in determining the “influence” of data visualisation on users’ performance.
Opinions about the visualisations are following:
Test 1
Version A:● Participant nr. 1 - “Tabular presentation looks good, but I would like to see something done with it.” – encoded:
● Participant nr. 2 - “If I want to extract more information from this data, like when you asked me to count down the statuses, this presentation becomes totally poor and furthermore time consuming. It seems to me this data potential is not fully used with this presentation.” – encoded:
Version B:
● Participant nr. 3 - “I like this presentation. I find it readable and easy. This would save
me a lot of time.” – encoded:
● Participant nr. 4 - “It’s good because it lets you see the progress you have and which is the breaking point in the process (sales pipeline). In that way you can tell your reps to work more on this point.” – encoded:
● Participant nr. 5 - “It would definitely be more helpful to get more detail into this. I like the chart. I don’t necessarily like that is goes to the both ends (funnel chart), makes it
difficult to compare, but it’s definitely the best option to represent these numbers.” – encoded:
● Participant nr. 6 - “This (visualisation) definitely provides a useful insight. I can see which conversion rate is the lowest and undertake certain actions to raise the numbers.” – encoded:
● Participant nr. 7 - “Seeing this I don’t need to count all the statuses and I have the progress of my leads over time easily presented. This is time saving and I can focus more on what do I need to do next.” – encoded:
● Participant nr. 8 - “I like the chart, it’s nice and appealing. I didn’t see the conversion rates (on the right) immediately, maybe that could be highlighted somehow, but seeing them now I can see my progress and focus on some parts that are underperforming.” – encoded:
● Participant nr. 9 - “Definitely a good visualisation, gives me an insight about rates and corresponding problems while saving time. Why should I count the statuses myself? System should do it for me and I should focus on what comes after I see that data.” – encoded:
Test 2
Version A:● Participant nr. 1 - “The data I see here is useful for me, but it’s hard to understand the