Using big data for generating firm-level innovation indicators - a literature review

Obtaining indicators on the innovation activities of firms has been a challenge in economic research for a long time. The most frequently used indicators - R&D expenditures and patents - provide an incomplete picture as they represent inputs in the innovation process. Output measurement of innovation has strongly relied on survey data such as the Community Innovation Survey (CIS). However, this type of data suffers from several shortcomings typical of surveys, including incomplete coverage of the business sector, subjectivity concerns, low timeliness, and limited comparability across industries and firms. An alternative that has attracted growing interest is to use big data sources to collect innovation data at the firm level. This paper discusses recent attempts to use digital big data sources including websites and social media to generate firm-level innovation indicators. It summarises the main challenges of using big data and proposes practical guidelines for their use, including a research agenda that should be useful to practitioners as well as users of statistics derived from big data.