Keynote Talk

From Revenue Assurance to Assurance: The Importance of Measurement in Computer Security

Peter Gutmann


In 1995, Netscape rolled out SSL, the application-level security protocol that's used to secure web browsing, email, FTP, instant messaging, VoIP, and in general anything that needs an encrypted pipe from A to B. SSL is rather crucially dependent for its security on certificates created by third-party CAs, but for the first 1 1/2 decades of its existence no-one had ever tried to measure how effectively these were being handled. When a volunteer-run project by the EFF did finally examine the situation, they found a chaotic mess that still hasn't been fully untangled. Telcos faced the same problem in the 1990s when they found that, to their considerable surprise, their billing systems were incapable of properly managing mobile phone billing. The result was the field of revenue assurance, a systematic effort to measure and evaluate the performance of mobile phone systems, at least as it applied to billing users. This talk looks at various failures of measurement both in and outside the field of computer security, and applies lessons from the area of revenue assurance to computer security mechanisms.


Peter Gutmann is a researcher in the Department of Computer Science at the University of Auckland working on design and analysis of cryptographic security architectures and security usability. He helped write the popular PGP encryption package, has authored a number of papers and RFC's on security and encryption, and is the author of the open source cryptlib security toolkit, "Cryptographic Security Architecture: Design and Verification" (Springer, 2003), and an upcoming book on security engineering. In his spare time he pokes holes in whatever security systems and mechanisms catch his attention and grumbles about the lack of consideration of human factors in designing security systems.