2026-09-21T15:05:12Zhttps://ruor.uottawa.ca/server/oai/request

oai:ruor.uottawa.ca:10393/511182025-12-02T08:00:30Zcom_10393_242col_10393_11105

Automated Care Pathway Modeling Using Agentic and Knowledge-Aware LLMs Houshidari, Alireza Van Woensel, William Amyot, Daniel LLMs Process Mining Knowledge-aware process mining Process extraction BPMN Clinical Pathways Clinical Guidelines Clinical pathways (CPWs) translate evidence-based guidance into stepwise care but are often disseminated as free text, obscuring control-flow semantics needed for clarity and computability. Formalizing CPWs as process models - e.g., in the Business Process Model and Notation (BPMN) - improves comprehensibility and enables downstream automation. This thesis designs, implements, and evaluates LLM4CPW, a pipeline for automatic guideline-to-BPMN modeling using large language models (LLMs). We compare two contemporary frameworks - MAO (agentic, multi-role orchestration) and ProMoAI (single-agent with self-refinement loop) - under controlled execution with standardized evaluation. Automated metrics combine node-level and structural similarity (after Dijkman et al.) with graph-edit distance; a clinician provides fidelity ratings with qualitative annotations. We then investigate knowledge-aware modeling via a categorization of recurrent errors and curated clinical statements, testing prompt-only injections versus a dedicated Knowledge Advisor agent placed at different stages of the workflow. Across four Ontario stroke Quality-Based Procedures (QBPs; n = 15 runs per framework), MAO attains higher node similarity (≈ 0.782 vs. ≈ 0.696) and structural similarity (≈ 0.630 vs. ≈ 0.585) than ProMoAI with large effects and p < 0.001, and exhibits markedly lower run-to-run variability; expert ratings also favor MAO. Knowledge-aware variants of MAO yield measurable gains: introducing a Knowledge Advisor after semantic review phase improves node similarity by > 4 points and reduces Graph Edit Distance by ∼ 13 (to∼ 97), with statistically comparable outcomes when placed before review; expert deltas likewise favor the advisor-based designs. Refining statement wording improves medians and interpretability without shifting means. Contributions. This thesis contributes (i) an auditable LLM4CPW pipeline and evaluation protocol; (ii) empirical evidence that agentic orchestration improves fidelity and stability; and (iii) a principled, deployable strategy for knowledge-enhanced modeling via a specialized advisory phase. Collectively, the findings demonstrate the feasibility of reliable, automatically extracted BPMN models from concise clinical guidelines and chart a path toward broader, clinically grounded automation. 2025-12-01T20:26:20Z 2025-12-01T20:26:20Z 2025-12-01 Thesis http://hdl.handle.net/10393/51118 https://doi.org/10.20381/ruor-31573 en Attribution-NonCommercial 4.0 International http://creativecommons.org/licenses/by-nc/4.0/ application/pdf Université d'Ottawa / University of Ottawa