The research value of this research is in the experience with combining two data sources and making sure there is a good connection. There is also value in the analytics domain about structuring data, while making sure the structure does not limit the options of a company to collect anything they need. In the field of extracting statistics from a system and displaying them a lot has been learned.
One lesson learned is that usage data will always change and expand. This means that the storage solution for usage data needs to have flexible data format, that allows to add extra fields into the usage data. Another lesson learned is that classifying usage data is very useful for answering analytics questions and building funnels. Without classification of usage events, it is not possible to effectively filter and group the data, meaning certain questions cannot be answered.
During the case study at Staying it became even more clear that analytics questions companies have about their products are unpredictable. This fact needs to be taken into account during the design of the usage data storage. This is another reason for proper structure and classification of usage data events, which helps a lot when a new question by a company is about a data type that has not been used before. There is still a chance that the already stored usage data is not sufficient, but the chance will be lowered significantly.
A lot has been learned about funnels, in the area of collecting the required data and in the visualization domain. For collecting the right data it became clear that for properly tracking users through funnels, it is first required to be able to uniquely identify users. For simple funnels this was not a big issue, often there was a clear way to identify users through the complete funnel (such as an email address for simple email tracking). However for more complicated funnels, like the ones created for Staying, it was required to identify users with multiple identifiers at once. For example, for the long property sign up funnel of Staying, it is required to switch identifier a couple of times during the funnel. First they are identified by the marketing email number, after that a session number on the website, and at the end by the accommodation they created during the sign up. Next to this issue, splitting and merging funnels was a problem that appeared in practice. This has also been solved in this project, making the funnels more versatile.
For funnel display there are a couple of interesting insights. While funnels of other analytics tools display the steps horizontally, displaying them vertically does not degrade readability. The vertical layout also offers better options for displaying context information, which has proven to be quite important for Staying. For example, the guest invite funnel of Staying became a lot more useful for them when they could see the difference between mobile and desktop/laptop users. It became clear that a higher percentage of mobile users stopped, which lead to more testing of the invite flow on mobile. The result is that now the mobile invite flow works a lot better, improving the install rate of the app (which is an important KPI of Staying). Another lesson has been about displaying large context information blocks. If a viewer of the funnel cannot see at least two steps at once, it becomes a lot harder to see where users have stopped in the funnel. So it is important that big context information blocks are collapsed by default, so that they do not take up too much room. As last point, color and font size has proven to be important. The size of text indicates how important it is, together with the thickness and contrast compared to the background. This has been used to highlight the most important numbers in the funnel, while making context information more dim. Colors are used to indicate stopped and continued, which helps to immediately identify ’good’ and ’bad’ paths that users can take.
These insight can help future funnel building and visualization tools, ultimately helping companies to get a better view of the processes in their product. Staying has used the funnels build during this project a lot. The first version were with live data directly, integrated into the admin dashboard. The results were immediately used to steer development, streamline marketing and improve support to users. The guest invite funnel and property invite funnel where used to improve both of these processes. For example, reducing the number of steps and making steps easier to understand. The guest invite funnel has been used by support to contact properties where conversion was low. For example, some properties had little to no content added to their app, causing guests to install the app, but rarely use it after the first install. Next to that the email funnel has been used to identify email delivery problems, leading to improvements to the emails so that they have a lower chance of ending up in the spam folder. The SMS delivery funnel showed that messages to the US and Canada were consistently failing, which identified a crucial issue: a phone number from The Netherlands cannot send message to these countries. This issue has been fixed by using an additional US phone number, improving the conversion of guests in the US and Canada.