The Cloze test for readability

Too much financial content is hard to understand. It uses technical vocabulary. It’s too wordy. It’s poorly organized. But sometimes it’s hard for authors to tell when content doesn’t work. What can they do?

I learned about the Cloze test from reading “Tips and Tools for Consumer Friendly Disclosure” by University of Georgia’s Professor Brenda J. Cude, a National Association of Insurance Commissioners consumer representative. It works as explained in her slide below:

Let’s experiment to see if the test works.

Cloze test experiment

Here’s a financial disclosure with words blanked out as required by the test. Try to fill them in.

ETFs are subject to ____ fluctuation and the risks ____ their underlying investments. Unlike ____ funds, ETF shares are ____ and sold at market____, which may be higher ____ lower than their NAV, ____ are not individually redeemed ____ the fund.

Options trading entails significant ____ and is not appropriate ____ all investors. Certain complex ____ strategies carry additional risk. ____ trading options, please read ____ and Risks of Standardized____. Supporting documentation for any____, if applicable, will be ____ upon request.

There were 16 blanks in the sample. If you guess 60% (10 words) or more correctly, then the text is considered well-written, according to Cude. Text for which readers score 49% (8 words) or lower needs to be edited, especially if the score is 39% (6 words) or lower, says Cude.

Here is the text of the original disclosure:

ETFs are subject to market fluctuation and the risks of their underlying investments. Unlike mutual funds, ETF shares are bought and sold at market price, which may be higher or lower than their NAV, and are not individually redeemed from the fund.

Options trading entails significant risk and is not appropriate for all investors. Certain complex options strategies carry additional risk. Before trading options, please read Characteristics and Risks of Standardized Options. Supporting documentation for any claims, if applicable, will be furnished upon request.

How did you do on this test? I suspect that many of you have read enough financial disclosures that this text tested well.

My doubts

I wonder how good a test this. I figure that a highly redundant text would score well because there are so many clues to the missing words. However, I wouldn’t enjoy reading such text.

I looked around to see how others view the Cloze test. In “Cloze Test for Reading Comprehension,” the Jakob Nielsen of the Nielsen Norman Group speaks well of the test as a way to measure comprehension. “Cloze Tests provide empirical evidence of how easy a text is to read and understand for a specified target audience. They thus measure reading comprehension, and not just a readability score.”

Nielsen makes the interesting point that comprehension is different than readability. (I’ve discussed readability in posts like “7 factors that affect reading ease.”) After all, making a sentence short doesn’t guarantee that the reader will understand it.

As a result, I conclude that the Cloze test can be a useful tool to use along with others that I discuss on this blog and in my book, Financial Blogging: How to Write Powerful Posts That Attract Clients.

Thanks, Linda Leitz!

I thank Linda Leitz for drawing my attention to Cude’s work in Leitz’s “Tell it to me like I’m an eighth grader” in the NAPFA Advisor.