An article co-authored by Heng Xu, Nan Zhang, and Le (Betty) Zhou was recently accepted for publication at the Journal of Management. In the article, we provide an overview of the common issues that threaten the validity of inferences drawn in empirical studies from analyzing organic data – that is, data that were not generated according to an explicit research design but were instead captured by digital devices or platforms; examples include contents and social interactions extracted from social networking sites, Twitter feeds, click streams, etc. Specifically, we discussed two main types of validity threats in the usage of organic data. One is caused by the opaqueness of the data generation process (to the researcher/practitioner), and the other by the opaqueness of the automated information-extraction algorithms.
Here is the link to the article.