Multi-Agentic Oncology Value Scorecard Creation with LLMs
Published:
Project Overview
Based on some great discussions with my previous Pfizer colleagues Brett South, Ajit Jadhav, Jay Ronquillo, Jon Mauer, and Stephen Watt, this project tests whether large language models (LLMs) can reproduce established, human-built oncology value frameworks such as the ISPOR Scorecard and the ASCO Value Framework. I compare three LLM approaches (a multi-agent system, a single-LLM pipeline, and retrieval-augmented generation) on data from ClinicalTrials.gov, PubMed, and OpenFDA, and check their output against the published reference values in Langdon et al. (2016). Later on I added a fourth, MOA-based multi-agent version to improve the synthesis step and make it easier to trace where each number in the final scorecard came from.