Before my Xmas break, I met ChatGPT (Generative Pre-trained Transformer). Since then, she has been my teacher, wise but admits her mistakes. Also, she is humorous (when I ask her to be) and very patient.
I decided to see whether ChatGPT can actually do the first stage of screening, i.e. title and abstract screening. After negotiating with her for a few hours, I cracked the code and passed her a carefully worded selection criteria based on PECOS: Population, Exposure, Comparator, Outcome and Study design. And there she was. ChatGPT was telling me whether I should exclude or include a particular study after evaluating a study’s title and abstract.
I used lists of studies and criteria related to this protocol:
Vendl C, Taylor MD, Braeunig J, Gibson MJ, Hesselson D, Neely GG, Lagisz M, Nakagawa S. Profiling research on PFAS in wildlife: Protocol of a systematic evidence map and bibliometric analysis. Ecological Solutions and Evidence. 2021 Oct;2(4):e12106.
What amazed me was that ChatGPT matched the study with our criteria and summarized reasons. Wow, this is better than I can do (see examples: one recommending inclusion and the other recommending exclusion = both are spot on!)
I am hoping to work with a computer scientist and see whether some of these processes can be automated for multiple articles. We are entering an exciting but uncertain time. One thing I can say is that I will be trying to incorporate ChatGPT into some parts of my systematic review workflow from now on, not as a replacement for a human screener but as an addition for now.