Harvard has a problem // not only Harvard, LLMs hallucinations
The problem with KGs is that knowledge is semi-structured, KGs can contain only structured part or metadata.
Biomedical knowledge is uniquely complex and structured, requiring distinct reasoning strategies compared to other scientific disciplines like physics or chemistry. Biomedical scientists do not rely on a single approach to reasoning; instead, they use various strategies, including rule-based, prototype-based, and casebased reasoning. This diversity calls for flexible approaches that accommodate multiple reasoning strategies while leveraging in-domain knowledge. We introduce KGAREVION, a knowledge graph (KG) based agent designed to address the complexity of knowledge-intensive medical queries. Upon receiving a query, KGAREVION generates relevant triplets by using the knowledge base of the LLM. These triplets are then verified against a grounded KG to filter out erroneous information and ensure that only accurate, relevant data contribute to the final answer. Unlike RAG-based models, this multi-step process ensures robustness in reasoning while adapting to different models of medical reasoning. Evaluations on four gold-standard medical QA datasets show that KGAREVION improves accuracy by over 5.2%, outperforming 15 models in handling complex medical questions. To test its capabilities, we curated three new medical QA datasets with varying levels of semantic complexity, where KGAREVION achieved a 10.4% improvement in accuracy
Medical reasoning presents unique challenges that require integrating multi-source, grounded, and specialized domain knowledge. In this work, we introduced KGAREVION, a KG-based LLM agent that addresses these challenges by combining the non-codified knowledge of LLMs with the structured, codified knowledge of medical concepts stored in KGs. Through its adaptive reasoning and mechanisms for generating, verifying, and revising knowledge, KGAREVION can handle complex medical QA. Experiments across multiple-choice and open-ended tasks, using a variety of datasets — including challenging new benchmarks — demonstrate KGAREVION’s ability to systematically improve accuracy. By grounding LLM-generated knowledge in KGs, KGAREVION ensures contextual relevance and reliability, making it a valuable tool for knowledge-intensive medical QA.
