OPEN TEXT VS. WORD CLOUDS. Your guide to making the most of your qualitative data

(1)

VS.

WORD CLOUDS

Your guide to making the most of

your qualitative data

(2)

THE TROUBLE WITH QUALITATIVE DATA

Identifying patterns in open ended questions

Qualitative data, which is collected via free-form survey questions like open text or essay boxes, can be a gold mine of information.

It can also create a mountain of data that seems impossible to scale.

While word clouds can show you an instant overview of common words and phrases from your responses, open text analysis will give you invaluable insight that automatic tools like word clouds just can’t match.

(3)

AN ILLUSTRATIVE

EXAMPLE: MOVIE SCENES

Revealing the power of open text analysis

As expected, we discovered that word clouds could give us a very basic overview of the responses, but that was it.

The only way to see more significant trends was to perform open text analysis.

Here’s what we did, and how you can do the same thing with your qualitative data.

3 Open Text vs. Word Clouds

To illustrate the

difference, we surveyed our employees about their favorite movie scenes, then examined the data using both word clouds and open text

analysis.

(4)

OUR MOVIE SURVEY

We asked just two questions, both with the essay answer option:

1.  What is your favorite scene from a movie?

2.  Please describe the scene – what happens? Who are the characters? How does it figure into the plot of the movie? How does it make you feel during and after watching it?

The goal of the survey was to determine whether or not there were any commonalities among the answers. We were curious if violent scenes, romantic scenes,

triumphant scenes, or something else were the ones that resonated with our colleagues.

We distributed the survey via email to all SurveyGizmo employees, and got a 34% response rate.

While not statistically significant, the results are

interesting as a snapshot of our office’s movie preferences (and as an illustration of how to work with long form

survey responses).

(5)

RESULTS VIA WORD CLOUD

The word cloud of question #1 looked like this:

Letting people write out their choices free form means there aren’t many common words. But what we can see is that “Harry” and “Potter” are slightly larger (and therefore appear more often).

We can use this in our analysis later.

(6)

RESULTS VIA WORD CLOUD

The word cloud of question #2 looked like this:

You can see that “Eli,” “Sunday,” and “Plainview,” all appear a disproportionate amount. What does that mean?

(7)

RESULTS VIA WORD CLOUD

If we just trusted the word cloud results, we’d figure that those three words were the most popular ones in our respondents’ descriptions.

This could lead us to assume that that film was the most loved, or that movies with characters named “Eli” or

“Plainview” resonate most strongly with our staff.

Both these conclusions are false, but we had to dive into the individual results to find this out.

It turns out one respondent typed in the full dialogue of their favorite scene from, “There Will Be Blood,” including the names of the characters who were speaking, “Eli

Sunday” and “Plainview.”

(8)

RESULTS VIA WORD CLOUD

This had a HUGE impact on our word cloud results, but we can now filter out that particular response and get an

updated word cloud:

Unfortunately now our word cloud isn’t showing us much at all. Time for something more robust!

(9)

OPEN TEXT ANALYSIS

In SurveyGizmo, you can easily use bucketing to quantify patterns in your qualitative questions. Word clouds can be a good first step in this process, because they can help you choose what “buckets” you want to create.

After you collect responses, you go to the “Reports” option under the “Results” tab.

Choose “Open Text Analysis,” then begin your analysis of each question you want to examine.

In our example these are just essay boxes, but open text analysis also works on multiple choice questions with an

“Other – please specify” option, or any other question for which respondents can enter text.

(10)

OPEN TEXT ANALYSIS

The next thing you need to do is select your “Buckets,”

which are the categories into which you’ll manually drop your responses.

Below are the ones I chose for the movie answers, based in part on the word clouds we looked at earlier and

partially on my own quick glance at the answers.

(11)

OPEN TEXT ANALYSIS

Now the hard part: you read (or scan) each response and choose which bucket(s) it belongs in.

This can be time consuming, but it’s really the only accurate way to determine which categories your responses belong in.

Here are two of my responses and the buckets I put them in:

(12)

WHY CHOOSE OPEN TEXT

Reading all of the responses people type in to your open text questions can be time consuming, but the reporting power that it gives you makes it completely worth it.

The word clouds we looked at earlier were sort of neat, but they didn’t really give us insight into how many people

preferred a particular kind of movie scene.

However, after bucketing all the responses I was able to create these reports:

(13)

WHY CHOOSE OPEN TEXT

Similarly interesting results come from Question 2, in which the respondents described their favorite scene:

If we needed to, we could then do another round of

bucketing on questions that fall into the fighting and death category to try and learn more about patterns within those responses.

Open text analysis offers you practically unlimited ways to examine your qualitative data.

(14)

TEXT ANALYSIS +

Open text analysis helps us create quantifiable segments of our qualitative data, but word clods can do a great job of helping us to figure out what those segments should be.

If you need to choose just one method of reporting, open text analysis will give you much better data than a word cloud.

But ideally you could create a quick word cloud and then use the patterns it reveals to create the buckets for your text analysis. It’s the best of both worlds!

(15)

START A TRIAL

15

Ready to start experimenLng with word clouds and open text analysis yourself?

Try SurveyGizmo free for 7 days:

Open Text vs. Word Clouds