Report an accessibility problem
ASU home My ASU Colleges and schools
Recent announcement
Competitive intelligence

Using Generative AI for Strategic Intelligence Work in RD

Catherine Chen, Karen Walker
June 08, 2023

The Strategic Intelligence (SI) team in Research Development has evaluated ChatGPT-3.5, a generative AI chatbot, for its ability to assist in the performance of intelligence work. We conducted tests to see how ChatGPT could assist in SI project phases, including planning projects, collecting, organizing, and analyzing information, and transforming information into effective products. We examined outputs for reliability and accuracy. Through these case studies, we have identified when it is appropriate to use AI tools and the potential benefits and risks of using such tools in our work. We have also developed a Code of Ethics for working with generative AI, which emphasizes transparency, fairness, and lawfulness. We are continuing to explore how our work can benefit from using such tools and intend to expand our evaluations to additional AI tools, such as Bing and Barely.AI. If you want to cite or quote material in the report, please attribute it to "Chen, Catherine, & Walker, Karen (2023). Using Generative AI for Strategic Intelligence. Arizona State University."

 

Key Findings:

1. ChatGPT can be used to facilitate many, but not all, aspects of the SI project process. It should be used with caution when the analyst is not able to input all needed information into the prompt.

  • Planning: ChatGPT can help an analyst quickly obtain an overview of a field or topic, so long as the topic is not extremely recent.
  • Collecting/organizing information: ChatGPT can organize and parse information efficiently and well, so long as all needed information is provided in the prompt.
  • Analyzing information: ChatGPT’s tendency to generate false data and its lack of access to the most up-to-date information meant that it performed poorly at this task. It should not be used in analysis tasks.
  • Transforming information: ChatGPT performed poorly at transforming information into actionable intelligence that was specific to our stakeholders. However, once an analyst has key points to incorporate into a written report, AI tools can be used to generate drafts that an analyst could use as “jumping-off points.”

 

2. While ChatGPT often provides information that is generally correct, the specific details may be incorrect. As with all sources, analysts must use their knowledge and training to review the data in order to ensure that accurate and reliable intelligence is produced. The need to fact-check information from ChatGPT may reduce potential efficiency gains.

 

 

Access the full report at: https://www.dropbox.com/s/lqv036kg7jwe365/Using%20Generative%20AI%20for%20SI%20June%202023%202.pdf?dl=0