The field of information visualization still hasn’t found a layperson-friendly way to visualize the contents of a text. Important speeches, conference paper titles, or letters in a historical archive most frequently wind up in a word cloud.
Word clouds are visually appealing, interesting, and simple to create using online tools. They highlight the most significant and frequently used words. Furthermore, you can format them in more interesting ways:
Word clouds are simple to create and interesting to look at thanks to online tools, which is probably a big factor in their popularity. When people are having fun, nothing negative occurs. However, word clouds are frequently employed in serious contexts, such as data science projects or the dissemination of scientific information. As for the state of the union speeches, journalists adore them. The issue is that word clouds are ineffective for presenting comparative values and summarizing data.
Some Problems with Word Clouds
Take a look at the speech summary posted on the USA Today website directly above the word cloud:
Obama defended the advancements made over the previous seven years and outlined a plan that will probably not be completed until long after his presidency is over: reversing the effects of climate change, starting a “moonshot” to cure cancer, and starting a grassroots movement to demand political system changes.
Consider the word clouds above and consider how much of this summary is immediately discernible from them. How about another look?
Obama only used the word “cancer” twice, which puts it well below the top 150 words in frequency and eliminates any chance of it showing up in the word cloud. He also discusses science, research, HIV/AIDS, malaria, and Ebola, all of which are used to express his objectives for medical research.
However, even though they cluster together within a single concept, none of these words are sufficiently common on their own to appear in a word cloud. One issue with using raw word counts as summaries is that they have a bias toward drawing your attention to words with few alternatives. The medical research words are hidden in the word clouds, as this example demonstrates. Even though there are references to climate change in the speech (oil, fuel, solar, gas, wind, energy, planet, etc.), you may have already noticed that you can’t find them.
The speech actually covered a wide range of subjects, each with its own sub-vocabulary, such as veterans returning from Afghanistan, the Iran nuclear deal, fighting Ebola, and more. The only subject that stands out from the visualizations is Obama’s discourse on the economy and jobs. The speech is woven from the other prominent words, such as variations of America, new, nation, and people.
Word clouds are inaccurate for a variety of reasons, which we won’t go into here (see these references), but suffice it to say that if you try to guesstimate the frequency of each word, you’ll probably be pretty off. Furthermore, users of word cloud tools frequently alter the size to improve the visualization because they are unaware that the size of the word should be in line with its frequency (see our earlier work). Finally, some word clouds turn out to be downright absurd. One day, we received an email that resembled this one quite a bit:
This word cloud is advertising a biology course that students might take.
How to Create a Better Word Cloud
Which design would be superior then? When comparing the use of keywords within or across speeches, I prefer to pick a small subset of keywords; Barbara Maseda has a nice collection of this kind of design.
But it’s difficult to argue against the appeal of word clouds. We set out to see if we could create a word cloud that was more understandable while still maintaining its aesthetic appeal. We believe that better comprehension of a document’s underlying topics would result from more careful word choice and concept organization. We’re not the first to make this suggestion. This concept has been used in books by text visualization expert Jeff Clark, and Wang et al.’s TopicPanorama project include a word cloud made up of words from up to four different topics.
We conducted a series of thorough studies to evaluate the viability of this proposition. With just two significant modifications to the way word clouds are currently constructed, we demonstrated how they can be made more understandable. First:
Semantically
Organize
It is necessary to group the words taken from the document(s) into a few categories that make sense to the reader. The issue with this step is that the majority of unsupervised text topic discovery algorithms fail to generate clear, distinct categories. Hierarchical clustering, LDA, LSA, etc., are all in our sights. All of these automated algorithms typically fail to separate the words in a document into coherent and distinct semantic groups. So far, manual labor is required to complete this subdivision. Second:
Visually
Subdivide
The words are all mixed up in standard word clouds, which is why the words for business and for the economy are so far apart in the visualizations above. Words that are displayed close to one another suggest that their meanings are connected, meaning that they are about the same subject or belong to the same category. In a word cloud, there are a number of ways to visually convey proximity; we can do any or all of the following:
- Group together words that belong to the same category close to one another.
- Put spaces between words to distinguish them from one another.
- Give words belonging to the same category the same color.
These tactics rely on fundamental ideas from perceptual psychology. The use of this strategy is referred to as WordZones. Let’s give it a try with Obama’s State of the Union address. To determine whether it is simpler to understand, compare it to the first two word clouds.
An alternative for the 2016 State of the Union summary.
Here, we chose a subset of the speech’s words based on a combination of their frequency and their ability to be categorized. More topics could have been displayed, but the design would have grown considerably. There are many other layout options that could be used; this is just one that we found to have high user preference scores.
Word Cloud Layouts: A Controlled Study
If you want to learn more about how we demonstrated in a paper recently published in IEEE TVCG that this type of design is preferable to typical word clouds, continue reading.
Our main conclusions were:
- In comparison to ungrouped layouts, visually grouped layouts perform better in time-constrained category understanding tasks.
- Visual grouping can be achieved by dividing categories with whitespace, by using different colors, or by combining both.
- For analytical tasks, layouts characterized by white space are frequently preferred over those that are more densely packed and disorganized looking.
Four experiments were carried out. For the first three, you had to name every category in the design while adhering to the Taboo rulebook and not using any of the words that were displayed. This is a result of the work itself; up to this point, every word cloud study has used a different dataset. In order to evaluate different word cloud designs in the future, we created and tested a set of 60 categories, each made up of five words. For instance:
In the fourth task, we asked participants to rate four layouts on four different criteria: readability, informativeness, visual appeal, and engagement. They were then asked to compare the four layouts and give their preferences. They were instructed to rate the designs on both aesthetic and functional criteria, assuming the word clouds were on a flyer for a class they were considering enrolling in. This was motivated by the biology course layout mentioned above. Which do you consider to be the best?
Four views, each with a consistent color assigned to a category, were presented to the participants for comparison. Lower left: blank space separators, but with the radial layout; lower right: a word cloud without organization. Upper left: columns with blank space separators; upper right: no blank space; categories organized spatially.
In terms of readability, informativeness, and visual appeal, participants gave the word cloud design poor marks; engagement, however, was where they gave it higher marks because it was more engaging than the other designs. According to all criteria, the Column and Radial views were comparable to one another, suggesting that most situations may favor them equally. Since there was no time constraint, we expected people to prefer the spatial grouping typical of word clouds, but the semantically organized spatial layout came in between, which surprised us.
The Moral of the Story
People will be able to understand the underlying meaning of the document much better if you group the words into coherent groups and arrange them into spatially proximal groups rather than using a traditional word cloud. These arrangements are also preferred (at least for people trying to understand the underlying concepts), as the groups are easier to distinguish by color and by leaving space between them.
The disadvantage of this method is that the categories must currently be created manually through analysis, which takes time and involves making decisions. Word clouds do the same thing, but you could counter that there is a standard method for creating them that avoids editorializing, which may be crucial in political contexts. However, we contend that word clouds are by no means neutral due to their inherent randomness, arbitrary count cutoffs, and misleading relative size of the words.
The next time you need to visualize a document, we hope you will think twice before selecting a word cloud. Use a more intricate algorithm and visualization in its place, or if you must use a word cloud, at least divide the text into zones of meaning using color or space so that everyone can comprehend the main ideas while also appreciating the aesthetic appeal.
