The Scale and Scope of AI-Generated Research

Our study begins by mining over 30,000 potential stock return predictor signals from accounting data. These signals are constructed using various combinations of financial statement items from the COMPUSTAT database, representing a comprehensive universe of accounting-based return predictors. We identify 96 signals that demonstrate robust predictive power for stock returns using the Novy-Marx and Velikov (2023) “Assaying Anomalies” protocol. This validation process involves multiple stages of increasingly stringent criteria, including tests for statistical significance, robustness to different portfolio construction methodologies, and controls for 200+ other known stock return predictors.

For each of these validated signals, we use state-of-the-art Large Language Models (LLMs) and “template reports” generated by the “Assaying Anomalies” protocol to programmatically generate three distinct versions of complete academic papers. Each version contains different theoretical justifications while maintaining consistency with the empirical findings. This approach allows us to explore how AI can generate multiple plausible explanations for the same empirical phenomena, mimicking a common practice in academic finance where researchers often develop hypothesis after discovering empirical patterns, a practice known as HARKing (Hypothesizing After Results are Known).

The resulting 288 papers contain all elements expected in academic finance research: abstracts, introductions developing theoretical arguments, comprehensive data descriptions, detailed methodology sections, and contextualized conclusions. The papers also incorporate citations to existing (and, on occasion, imagined) literature and develop plausible economic mechanisms linking the documented patterns to established theories. The generation process involves sophisticated prompt engineering to ensure the papers maintain academic rigor and style while varying in their theoretical approaches.

Key Findings and Implications

Our experiment reveals several critical insights about the potential and pitfalls of AI in academic research. First, an AI-abetted research pipeline can generate papers at unprecedented speed and scale. While the initial data mining and validation took up considerable computational time, around 24 hours for processing the complete dataset, the final paper generation process required less than a minute per paper. Compared to the typical time spent on a paper by researchers, this dramatic acceleration of research production raises questions about how academic institutions should adapt their evaluation processes to handle a potential influx of papers produced with the help of AI.

Second, the quality of generated content is remarkably sophisticated. The LLMs successfully came up with creative names for the signals, avoiding generic terminology and capturing the economic intuition behind each predictor. The main model we used for text generation, Claude 3.5-Sonnet, developed plausible economic mechanisms linking signals to returns, drawing appropriately from established theoretical frameworks in asset pricing. It related the findings in the template reports with existing literature through appropriate citations and maintained internal consistency between theoretical frameworks and empirical results. Perhaps most impressively, the process demonstrated an ability to generate multiple distinct theoretical explanations for the same empirical pattern, often drawing on completely different strands of the finance literature (e.g., behavioral vs risk-based explanations).

Third, the ability to generate multiple theoretical frameworks for the same empirical findings raises significant concerns for research integrity. The technology enables industrial-scale HARKing, creating a potential flood of papers that appear to have strong theoretical foundations but were actually developed post-hoc. This capability could easily lead to manipulation of citation networks, as papers can be generated with strategic citation patterns to boost specific authors or papers. The volume of AI-generated content risks overwhelming traditional peer review processes, which are already strained by current submission volumes.

Implications for Corporate Governance Research

These findings have particular relevance for corporate governance research and practice. The ability to generate multiple plausible theoretical explanations for empirical patterns suggests the need for stronger emphasis on out-of-sample validation and practical implementation of academic papers. This is especially crucial in corporate governance research, where theoretical explanations often involve complex institutional factors and multiple stakeholders.

As AI tools become more prevalent in research production, we may need new standards for disclosing the extent of AI involvement in academic work. This could include requirements to specify which parts of a paper were AI-generated and what constraints or guidelines were used in the generation process. The potential for automated generation of citation networks could fundamentally affect how we evaluate research impact and influence in corporate governance literature, possibly requiring new metrics that are more resistant to strategic manipulation.

Looking Forward

The emergence of AI-powered research tools necessitates a thoughtful response from the academic community. We propose several key recommendations for the path forward. First, we need enhanced validation systems that can verify citations, ensure reference accuracy, and validate theoretical frameworks. These systems should be able to detect citation manipulation and cross-reference theoretical claims with established literature.

Second, the field requires new quality control mechanisms, including automated checking of theoretical consistency and citation networks. This might involve developing AI tools specifically designed to evaluate the originality and coherence of theoretical contributions, perhaps by comparing them against a corpus of existing work in the field.

Third, clear guidelines must be established for disclosing AI involvement in research production. These guidelines should cover not only the use of AI in writing and analysis but also in hypothesis generation and theoretical development. They should be specific enough to be meaningful but flexible enough to accommodate rapid technological advancement.

The implications of our findings extend beyond finance to all fields where researchers develop theoretical frameworks to explain empirical patterns. The integration of AI into academic research challenges the traditional notions of scientific discovery and hypothesis testing. As these technologies continue to evolve, maintaining research integrity while harnessing the benefits of AI-powered tools will be crucial for the future of academic research.