Introduction Underrepresentation in clinical trials poses significant risks, including insufficient safety and effectiveness of drugs for minority groups. FDA's Diversity Action Plans (DAPs) aim to address this by setting well-motivated enrollment goals stratified by age, sex, race, and ethnicity (FDA, 2024). An important part of setting enrollment goals consists of reviewing scientific literature on incidence/prevalence of a disease in different subpopulations. This step requires expert personnel and dozens of hours. We introduce a novel, AI-based tool leveraging large language models (LLMs) to automatically scan and extract relevant epidemiological data.Methodology The steps performed by our tool are as follows. 1) Obtain a large (hundreds/thousands) collection of paper titles and abstracts for a given indication (e.g., â€œbreast cancerâ€). This can be executed quickly using APIs like Pubmedâ€™s. 2) Rank the abstracts to prioritize those that include numerical results on incidence/prevalence, then filter to keep top N. Here, an embedding model converts text to vectors, and cosine similarity ranks these vectors against a pre-defined vector. The latter is pre-optimized to represent abstracts with data on incidence/prevalence in different groups. 3) Classify the top abstracts as relevant (Y/N) using a relatively fast LLM. 4) For the relevant abstracts, attempt to extract numerical information about incidence/prevalence with a more powerful LLM. The tool can further work at the level of a paperâ€™s PDF, making a summary of the results contained therein and extracting tables.Results We compared the extractions of the tool against a manually-crafted DAP concerning non-small-cell lung cancer (NSCLC), focusing on race and ethnicity (Hispanic: yes/no). For confidentiality reasons, we cannot provide detailed information.Step 1, searching with PubMed API within 2000/1/1 to 2024/8/6 and â€œnon-small cell lung cancer, non-small-cell lung cancer, NSCLC, lung cancerâ€ as keywords, resulted in 318 papers. Of these, 28% matched those cited in the DAP. Typical missed papers had relatively broad abstracts and titles (e.g., â€œKey Statistics for Lung Cancerâ€). Filtering with step 2, with values of N=1000,500,250,100, respectively led to 28%,28%,28%,24% matches remaining. The list of top 250 was submitted to step 3. The LLM took ~2 seconds per abstract and classified 166 papers as relevant, retaining the 28% matching papers. Step 4, with a more precise LLM taking ~10s per abstract, extracted incidence/prevalence information out of 54 papers. Finally, the extractions were analyzed manually, in ~10 minutes.Incidence of NSCLC from extractions (min-max, mean %) were as follows: White: 69-90,81; Black: 6-16,12; Asian: 7-7,7; Hispanic: 3-8,5. All found means of minorities were within the ranges expressed in the DAP, except for Hispanic, which was too low by 1%.Finally, we also tested the tool extracting tables out of 4 random papers: 3/3, 3/3, 2/2, 2/2 tables were respectively extracted.Conclusion We presented a fast, scalable, data-driven solution for literature search in diversity action plans. In our experiment, extracted disease incidence found in minutes of human intervention were comparable with those found in days. Our tool can help professionals be more efficient and effective in defining diversity-aware enrollment goals, ultimately promoting health equity.

Speeding up literature search for diversity action plans with AI

Collaborating for Healthcare Innovation – a focus on Sustainability!

18-20 March 2025
Basel, Switzerland

DIA Europe 2025 will focus on sustainability, innovation and collaboration – three key pillars for shaping the future of healthcare. From the environmental impact of pharmaceuticals and of the drug development lifecycle to sustainable partnerships and sustainable healthcare systems, DIA Europe 2025 will be the prime opportunity to discuss how collaboration and innovation can support sustainability in healthcare and each stakeholder’s role in making it happen.

Join us in the vibrant city of Basel for an unforgettable experience, where the power of collaboration will be on full display to inspire actionable outcomes and mobilise for solutions the life sciences’ community. Witness first-hand the profound impact of united efforts, gain valuable insights from across the drug development lifecycle, and play an active role in shaping the future of the pharmaceutical arena.