We gratefully acknowledge Meta Research for their generous support of our project “Addressing biases in measurement of self-reported privacy constructs”, which was selected as part of the 2021 People’s Expectations and Experiences with Digital Privacy RFP. The principal investigator (PI) of the project is Heng Xu, and the co-PI is Nan Zhang.

Project Summary

Understanding people’s expectations and experiences with privacy is a challenge. Many people express heightened privacy concerns yet refuse to take trivial actions to protect privacy. Consequently, researchers and practitioners frequently question whether self-reported privacy constructs reflect people’s “true” beliefs. To this end, there is mounting evidence that self-reported privacy constructs may be suffused with bias. For example, self-reported privacy concerns can be drastically inflated after exposure to news about privacy. Self-reported privacy preferences, on the other hand, can be heavily affected by people seeking cues irrelevant to privacy when they are uncertain about their own preferences. Moreover, different people are known to be subject to different types of biases to different extents, making the distribution of self-reported privacy constructs a mixture of multiple, heterogeneous, distributions rather than a single homogenous one that is (almost) universally assumed in the privacy literature. To properly understand people’s privacy expectations and experiences, it is critical to improve the precision and inclusivity of self-reported privacy constructs by accounting for this heterogeneous bias. In the proposed project, we leverage our recent methodological findings to address this challenge. Specifically, our proposed work aims not to eliminate all biases with a “perfect” measurement but to acknowledge the existence of bias in self-reported data and to develop statistical and computational techniques that can disentangle various types of biases from the observed responses.