Evaluating readability as a factor in information security policies

Researchers have designed a number of software readability metrics that evaluate how difficult a passage is to comprehend; yet, little is known about the impact of readability on the interpretation of information security policies (ISPs) and whether experiment of readability may prove to be a useful factor. This paper examines and compares eight ISP documents on nine mechanical readability formula results with outcomes from a human-based comprehension test. The primary focus is to identify if we might rely on a software readability measure for assessing the difficulty of a text document in the domain of Information Security Policies. Our results reveal that traditional readability metrics are ineffective in predicting the human estimation. Nevertheless, readability, as measured using a bespoke readability metric, may yield useful insight upon the likely difficulty that end-users face in comprehending an ISP document. Thereby, our study aims to provide a means to enhance the comprehensibility of ISPs.