The Verizon Business RISK Team recently published a valuable analysis of four years of data on security breaches among their clients entitled “2008 Data Breach Investigations Report.” Wade H. Baker, C. David Hylender and J. Andrew Valentine are the authors; their contributors include my old friend and colleague Dr Peter Tippett, MD, PhD, A. Bryan Sartin, Stan S. Kang, Christopher Novak, and members of the Verizon RISK Team.

Brad Reed has pointed out the main findings recently in Network World and the paper itself includes a good executive summary; therefore, in the next few columns, I will elaborate on the implications of specific points from the report.

Today I want to draw readers’ attention to the methodology of this landmark study.

As most people realize, all published information about data-security breaches (Compare Data Leak Protection products) should be examined with critical faculties fully engaged. Studies and statistics about computer crimes consistently suffer from the following methodological problems:
• Limited ascertainment (the crimes may not be detected).
• Restricted reporting (many organizations don’t want to report breaches at all and there is no centralized reporting facility to collate the data).
• Non-random samples (it is not possible to generalize from the samples to a wider population because the reports come from self-selected reporting organizations).

For more information about these issues, see my paper, “Understanding Computer Crime Studies and Statistics v4.” 

I believe that the study is unique in drawing upon a massive database of more than 500 specific investigations carried out by the Verizon RISK Team over the last four years. As the authors write, “Furthermore, it contains firsthand information on actual security breaches rather than on network activity, attack signatures, vulnerabilities, public disclosures, and media interpretation that form the basis of most publications in the field. While many reports in the security industry rely on surveys as the primary data collection instrument, this data set is inherently more objective.”

Surveys are inherently limited because it is difficult or impossible to determine whether the willingness to participate in the survey is correlated with any particular attributes of the participants; e.g., perhaps those who refuse to participate have worse security than those who participate – or vice versa. We don’t know and cannot know based on the survey results.

M. E. Kabay, PhD, CISSP-ISSMP, is Program Director of the Master of Science in Information Assurance program at Norwich University.