Large Language Models for Medical Systematic Reviews
This is the website for Appraising the Potential Uses and Harms of LLMs for Medical Systematic Reviews.
This website was originally used as an interface to display LLM-generated evidence summaries to participants during interviews. The original website only included the What is LLM page and the LLM outputs without any comments. Currently, this website has been repurposed to present the research work.
The purpose of the study is to better understand the utility, limitations, and harms of large language models (LLMs) by assessing them in the context of medical systematic review generation with domain experts.
Our research questions are as follows:
- What is the perspective of domain experts with respect to the potential utility of LLMs to aid production of medical systematic reviews?
- Do domain experts anticipate any potential risks from the use of LLMs in this context?
- What can we learn from domain experts which might inform criteria for rigorous evaluation of biomedical LLMs?
Contents
You can learn more about this project by interacting with the following contents.
- Paper
- GitHub - code and full outputs
- What is LLM?
- Methods and generation of evidence summaries from LLMs
- Materials used during interviews with domain experts. Participants viewed these example outputs and provided their comments. Sample of direct quotes from participants have been added after all the interviews were conducted.
(Please feel free to share your thoughts and opionions on these LLM-generated evidence summaries in the comments sections.)
Citation
@article{yun2023appraising,
title={Appraising the Potential Uses and Harms of LLMs for Medical Systematic Reviews},
author={Yun, Hye Sun and Marshall, Iain J and Trikalinos, Thomas and Wallace, Byron C},
journal={arXiv preprint arXiv:2305.11828},
year={2023}
}
The website is powered by Jekyll with Jekyll Gitbook theme and hosted on Netlify.