Comparing Phishing Vulnerability in the Lab to the Real World.pdf

POSTER: Comparing Phishing Vulnerability in the Lab to the Real World

PRESENTER: Jeremy Thomas, Carnegie Mellon University

ABSTRACT:
Phishing vulnerability has been assessed in terms of signal detection theory (SDT) in an online experiment. We assess the validity of such SDT measures using the Security Behavior Observatory (SBO), a longitudinal field-study of computer security habits. We (a) replicate the SDT experiment with SBO participants, (b) assess its construct validity, and (c) evaluate its predictive validity. Results similar to the original SDT experiment were reproduced with the SBO sample. There was evidence of construct validity, as the SDT measures were correlated with the Security Behavior Intentions Scale (SeBIS). However, there was no evidence of predictive validity, as the SDT measures were unrelated to negative computer security outcomes observed in the SBO, including presence of malicious URLs, malware, and malicious files. We discuss the implications of these findings and the challenges of comparing behavior on structured experimental tasks and complex real-world settings.

CO-PRESENTER(S):

KEYWORDS: 
Psychometrics; Individual differences; Validation; Signal detection theory; Security behavior; Phishing

PRESENTER BIO:
Jeremy Thomas is a software engineer in Carnegie Mellon University's Cylab and focuses on the Security Behavior Observatory project. This poster is part of Casey Canfield's work as a PhD student at Carnegie Mellon Univerity's Department of Engineering and Public Policy and uses data collected as part of the SBO project.