Could ChatGPT help predict capital expenditure plans of a publicly held company, information that would interest investors, competitors, suppliers, regulators, and anyone involved in M&A?
According to a new research study, the early indications are promising.
Professor Michael Weber of Chicago’s Booth School of Business and three Georgia State professors used the generative AI tool to create an investment score that measures managers’ anticipated changes in capital expenditures.
They tapped ChatGPT’s ability to learn context and meaning in text to analyze 74,586 conference call transcripts from 3,878 unique companies. The calls spanned from 2006 to 2020, according to the research paper they released in July, “ChatGPT and Corporate Policies.”
“The investment score also separately forecast future total, intangible, and R&D investments."
"ChatGPT and Corporate Policies"
July 2023
(The Georgia State professors who co-authored the paper are Manish Jha; Jialin Qian; and Baozhong Yang.)
The researchers asked ChatGPT 3.5 to choose one of five choices to answer the question, “How does the firm plan to change its capital spending over the next year?” The scale ranged from “decrease substantially” to “increase substantially.” ChatGPT was also asked to explain each answer and return a “no information provided” if the call did not touch on the subject.
The researchers’ resulting “ChatGPT investment score” bore a “strong, positive correlation with a company’s future investment both in the short term and long term,” the professors wrote, and the correlation continued to hold for as many as nine quarters. Since ChatGPT sometimes “confidently provides inaccurate information,” the researchers said, they randomly checked the explanations it gave for a sample of the conference calls.
Similarly, the researchers found strong correlations by deploying the ChatGPT method to analyze managerial expectations of changes in dividend payments and company headcounts.“The investment score also separately forecasts future total, intangible, and R&D investments,” according to the paper.
The ChatGPT method might have an application for CFOs. “We use a pretty straightforward way to elicit investment plans that have predictive power, so to the extent managers want to figure out at larger scale what firms in their industry are up to, [this] would certainly be a way rather than reading call transcripts,” said Professor Michael Weber in an email to CFO.
However, Weber said, some CFOs might change their language in response. “At the same time, CFOs also face market pressure, which might counterbalance this consideration,” he said.
Survey Validation
The professors validated their results in several ways. For example, they compared investment scores with Duke/Richmond Fed CFO survey responses. CFOs of high scorers in the ChatGPT model indicated through their Duke CFO survey responses their company’s plans to increase capital investment over the next 12 months.
"Surveys have their role and value, but it’s not perceivable to run surveys at large scale given likely monetary constraints; it’s also difficult to reach decision-makers in firms at large scale.”
Michael Weber
Professor, University of Chicago
As another validation, the researchers found that the ChatGPT industry-level average investment scores were consistent with major changes in the U.S. economy during the study period — such as the software/biotech industries increased investment during the COVID-19 pandemic and the hardest-hit industries lowering spending after the financial crisis (retail and wholesale) and at the pandemic’s onset in 2020 (transport/energy).
Despite advances in textual analysis, according to the paper, extracting complicated information such as a firm’s expected investment policy has previously been beyond researchers’ abilities.
The ChatGPT method could expand and complement existing surveys of CFOs and other C-suite executives, “which can be especially helpful given the decline in survey response rates in the United States in the past decade,” Weber said.
Added Weber, “Surveys have their role and value, but it’s not perceivable to run surveys at large scale given likely monetary constraints; it’s also difficult to reach decision-makers in firms at large scale.”