Document resource
Introduction Mapping free-text clinical notes to standardised phenotype terms (Human Phenotype Ontology, HPO) is crucial for advancing research and improving patient care in inflammatory bowel disease (IBD). Manual coding is time-consuming, and current automated pipelines often lack comprehensive audit trails, hindering their implementation in regulated clinical environments.Methods We developed a workflow working with an artificial intelligence (AI) startup using a graph-based system in which each step, such as processing text, retrieving information, creating prompts, running the model, and post-processing, is represented in an immutable ledger (a bit like a blockchain). The four agents used here operate in a secure, air-gapped environment with minimal privileges. Five synthetic patient cases (IBD vignettes) were generated using a language model (GPT 5.2). We tested 15 open-weight instruct-tuned large language models (LLMs), including one finetuned Llama 3 model, to see how well they map free text to HPO terms. Among the five clinical vignettes were 91 relevant HPO terms and a potentially unlimited number of opportunities for hallucination. We measured precision, recall (sensitivity), and harmonic F1 scores across three runs and reported the average results along with the total range across runs.Results Table 1 summarises the five best models by F1 score. When selecting the top predictions per vignette, Ministral-3 (a small edge model) outperformed all other model sets on average. GPTOSS models performed admirably, followed closely by Gemma-3-27B-IT and Devstral-2-2-123B. Weaker models included our finetuned Llama 3-8b model, which substantially underperformed (F1: 0.7590, precision: 0.8400, recall: 0.6920), in part because it struggled with particular letter formatting styles. All scores exceeded the F1 score of 0.54 obtained with a prior, simpler, non-graph-based single-agent implementation.Conclusions Graph-based agentic workflows provide a generic, transparent, and secure foundation for automated phenotyping in IBD. Interestingly, model size is largely irrelevant in these workflows, suggesting that auditable agentic graph-based pipelines can also deliver substantially reduced AI carbon footprints. This approach should, in theory, generalise to other gastrointestinal or rare disease cohorts, although further real-world validation is now required.Abstract P188 Table 1Top 5 performersModelModel SizePrecisionRecallF1 ScoreMinistral 33 Billion Parameters0.8662 [0.8588 – 0.8810]0.8059 [0.8022 – 0.8132]0.8349 [0.8295 – 0.8457]GPT OSS120 Billion Parameters0.8323 [0.8261 – 0.8427]0.8352 [0.8242 – 0.8462]0.8336 [0.8306 – 0.8370]Gemma 327 Billion Parameters0.8248 [0.8132 – 0.8391]0.8095 [0.8022 – 0.8132]0.8170 [0.8132 – 0.8202]Qwen3 Coder30 Billion Parameters0.8264 [0.8202 – 0.8315]0.8022 [0.7912 – 0.8132]0.8141 [0.8090 – 0.8222]