Document resource
Background and Importance Large language models (LLMs) such as ChatGPT are increasingly used by healthcare professionals. While AI shows promise for decision support, its application in pharmaceutical validation, especially for high risk medications like Direct Oral Anticoagulants (DOACs), remains largely unexplored in real-life settings.The integration of generative AI in clinical settings has raised significant interest in its potential to support healthcare professionals.Aim and Objectives This study assesses the ability of DrugGPT, a GPT-4-based language model trained on validated medical databases, to evaluate prescriptions of DOACs, a class of high risk medications, and compares its performance to that of hospital clinical pharmacists.Material and Methods Between September and December 2024, 50 inpatient DOAC prescriptions were collected. Each prescription was independently assessed by a clinical pharmacist and three AI models: ChatGPT-3.5, ChatGPT-4o, and DrugGPT. The study evaluated sensitivity and clinical relevance of AI-generated interventions compared to pharmacist decisions, using concordance analysis.Results ChatGPT-3.5 achieved a sensitivity of 67% in identifying clinically relevant interventions, ChatGPT-4o improved to 83%, and DrugGPT reached 100% concordance with the clinical pharmacist’s assessments across the 50 evaluated DOAC prescriptions. In a representative clinical case (elderly patient, apixaban underdosing), both ChatGPT-4o and DrugGPT provided recommendations aligned with pharmacist judgment, while ChatGPT-3.5 failed to identify the dosing error. Despite the improvements, AI models often lacked the capacity to provide pharmacologically justified reasoning or to contextualise recommendations based on full patient history.Conclusion and Relevance DrugGPT shows higher sensitivity and concordance with clinical practice compared to earlier GPT models, suggesting potential as an adjunct tool in pharmaceutical validation. However, limitations remain: AI models operate without access to EMRs, may generate hallucinations, and cannot assume legal or ethical responsibility. Further concerns include reproducibility, environmental impact, and the need for professional oversight. Responsible implementation, robust regulation, and targeted training are essential. DrugGPT exhibits promising capabilities in DOAC prescription validation, outperforming generalist models. Yet, it is not a substitute for clinical pharmacists. AI may serve as a valuable assistant within a supervised, regulated framework, but its integration into pharmaceutical care demands cautious, evidence-based deployment.Conflict of Interest No conflict of interest