November 2, 2026 · Lex Hamilton

Why Semantic Structure and Knowledge Graphs Matter for Pharmaceutical Data in AI

Why drug names, units and routes need semantic structure, what knowledge graphs add, and why accuracy scores alone miss errors in pharmaceutical AI.

Last reviewed: November 2026

Semantic structure means giving every piece of drug data a defined meaning and type, so software can tell a drug name from a unit and a route of administration from a body site. A knowledge graph is a database of typed things and the named relationships between them, such as an ingredient, a dosed product and the label section that describes it, so a program can answer a question by following links instead of matching text. Pharmaceutical data needs both because one drug appears under many names and units, and much of what regulators publish about it is written as prose.

Orinyx is an AI verification platform that helps developers build more reliable AI systems, starting with healthcare, by combining semantic validation with evidence-based verification.

A disclosure first: I'm Lex Hamilton, and I build Orinyx. Read this with that interest in mind. Where I describe what Orinyx does, I describe only what has shipped, and I label direction as direction.

What does "semantic" mean for drug data?
What is a knowledge graph, and what does it add over text search?
Why do standard model evaluations miss errors that semantic checks catch?
What does evidence-based verification add?
What are bounty agents, and when do they arrive?
Why does independent verification matter, and what is Utah doing?
Why did clinician feedback matter to the design?
How do I join the beta?
Where this fits: what can any team do without Orinyx?
Frequently asked questions
Further reading

What does "semantic" mean for drug data?

In drug data, semantic means each field has a defined type and each value comes from that type's vocabulary. A dose is a number plus a unit. A route is how a drug enters the body, such as oral or intramuscular. A body site is a place, such as the left thigh. Treat them as interchangeable text and a system can pass an order that no clinician could read.

Three problems recur. The first is names. One ingredient has generic names, brand names, salt forms and combination products. RxNorm, the National Library of Medicine's drug vocabulary, exists to give each medication a normalized name at a stated level of abstraction (Nelson et al., 2011). Its term types keep an ingredient, a brand name, a clinical drug (ingredient, strength and dose form) and a branded drug separate (NLM term types). A bare "metoprolol" fits two products, succinate extended-release and tartrate immediate-release, so resolving it to either one decides something the order never said (what changed at Orinyx).

The second is units and routes. "mb" is not a unit, and a body site written where a route belongs has not said how the drug is given.

The third is prose. FDA's over-the-counter monograph for internal analgesics gives the adult oral acetaminophen dose in one sentence: three alternative schedules, then a cap of 4,000 mg in 24 hours (FDA OTC Monograph M013, section M013.50(d)(2)). To check an order of 1,500 mg every four hours against that cap, software must read "every four hours" as six doses and multiply. Orinyx missed exactly that total in a synthetic test earlier in October, and I published the miss.

What is a knowledge graph, and what does it add over text search?

A knowledge graph is a database of typed things and named relationships between them, so a program can answer a question by following links. Text search finds the string "metoprolol" wherever it appears. A graph can record that a specific extended-release tablet is a product, that its ingredient is metoprolol, that the tartrate product is a different product, and which label section describes each. Given an order that says only "metoprolol", it shows that two products fit, and the system can say so instead of picking one.

Researchers have built graphs like this for drug safety. A 2024 scoping review of 47 peer-reviewed articles found them used to detect or predict adverse reactions from single drugs and from drug-drug interactions. It suggested they have potential to augment signal detection in pharmacovigilance, the monitoring of medicines after they reach patients. It also found that reported performance varied, comparisons with older methods were sparse, and better reference sets were needed (Hauben et al., 2024).

That is where I think the evidence stands. The vocabulary is settled infrastructure. Federal EHR certification criteria have designated RxNorm as the vocabulary for medications since 2014 (NLM), and RxNorm already works like a graph. Its documentation describes concepts joined by named relationships such as has_dose_form, ingredient_of and has_tradename, and says RxNorm "fills out the graph" where a less specific concept is missing (NLM RxNorm overview). Graph-based checking is a different matter. A 2026 scoping review that found six studies applying knowledge graphs to cardiovascular drug safety reported that integrating ontologies such as MedDRA and RxNorm improves mechanistic reasoning, while validation against standard pharmacovigilance methods remains limited (Jafarpour et al., 2026). So the vocabulary is the standard, and graphs built on it are promising and, for prediction, not yet shown to beat the methods they would replace. I would not buy one on the strength of its diagram. I would ask who curated each relationship and what the system does when a relationship is missing.

Orinyx does not run on a knowledge graph today. Orinyx plans to build one underneath the checks, and the plan came out of research after the failures I published in what changed at Orinyx: a matcher that resolved a name to the wrong product, and a bare drug name that fits two. A graph that links an ingredient to its products, dose forms and label sections gives the checker a place to stop and say that two products fit, where a string match picks one. That is direction, not product, and I am giving no date.

Why do standard model evaluations miss errors that semantic checks catch?

A 2025 JAMA systematic review of 519 studies of large language models in health care, published between January 2022 and February 2024, found that 95.4% used accuracy as the primary dimension of evaluation and only 5% used real patient care data. Most evaluations, in other words, measure whether an answer is accurate, and the errors that hurt patients are often errors of meaning. Deployment considerations were measured in 4.6% of studies, and calibration and uncertainty, meaning whether a system's confidence matches how often it is right, in 1.2%. Writing prescriptions was studied in 0.2% (Bedi et al., 2025).

Accuracy also hides how serious the errors are. In a study of AI-written primary care notes, clinicians labeled every sentence across 18 experiments on 450 transcript-note pairs. Of 12,999 note sentences, 191 held hallucinations, content the consultation did not support: 1.47%. Of those, 84, or 44%, were major, meaning they could change a diagnosis or management if left uncorrected. Of 49,590 transcript sentences, 1,712 were omitted from the note: 3.45%, and 16.7% of those omissions were major. Fabrication was the most common hallucination type at 43%, and negation of a clinically relevant fact was 30%, which the authors called the most concerning (Asgari et al., 2025). The study used one model, GPT-4, and the authors' own annotation platform, so I cite it for its method, not as a general error rate. The authors also report that refining prompts and workflows reduced major errors below previously reported human note-taking rates, so these rates describe those experiments, not a fixed property of the technology.

A rate under two in a hundred reads as small. It does not say that nearly half of those errors could change care. The authors also note that common text-overlap metrics compare surface wording and miss clinical nuance. A note can match its transcript and still be clinically wrong (transcript-faithful is not the same as clinically correct), and a higher score does not close that gap (why you can't make clinical AI accurate enough to skip governance).

Semantic validation is the check that runs before accuracy. It asks whether each part of an output is the kind of thing its field can hold: whether a drug name is a drug, a unit is a unit, a route is a route. Orinyx reads each order with a fixed grammar, with no language model involved, and returns UNREADABLE for an order it cannot read as written, listing every part it could not read. A test order, "wafarin 5 mb by intramuscular injection", has a misspelled drug and a unit that is no unit. A single accuracy number records that the order was wrong. It does not say where to look.

What does evidence-based verification add?

Evidence-based verification ties each answer to the source text it was judged against, so a person can check the check. In Orinyx, every supported or flagged answer names the FDA label section or over-the-counter monograph paragraph it used, and a flag carries a severity set by a fixed rule. Every claim gets one of four answers: supported, flagged, unreadable or not verified. Not verified means Orinyx could not check the claim, which is not the same as correct. Supported means one claim agrees with the cited label text, not that the order suits a patient. None of the four means safe.

Verdicts come from fixed rules. No language model decides a verdict, a severity or a score, so a developer can reproduce any answer. A tool that turns "could not check" into silence teaches its readers to stop looking, and the not verified answer exists to prevent that (what changed at Orinyx). How the checks themselves are measured is in benchmarking clinical AI.

What are bounty agents, and when do they arrive?

Bounty agents are the specialized verification agents Orinyx plans to build after the Orinyx MCP launches. Each one tests a different way an AI system can go wrong in healthcare and other regulated settings: whether its outputs are accurate, supported by evidence, semantically consistent and safe. The name comes from bug bounties, where a tester is rewarded for finding the flaw a developer missed. They are direction, not product, and I am giving no date.

The agents are meant to work together as one verification workflow, not as separate products. A medication-claims agent checks a dose, a route and a frequency. A source-evidence agent checks that a claim is supported by the label it cites. A semantic-mapping agent checks that the terms and mappings underneath are read correctly. The first beta is narrower than that picture. It centers on checking medication orders against cited FDA label text, with the semantic checks described above, and it is not an autonomous agent that writes its own tests. Agents for other areas, such as adversarial inputs, coding suggestions, patient messages and drift in mappings, are planned to follow as each one passes testing.

They depend on the two ideas above. An agent can only probe for a wrong product, a wrong unit or a missed dose total if the system knows which products, units and dose limits exist, so semantic structure and the planned knowledge graph come first. Today, fixed rules, not a language model, decide whether an answer is supported, flagged, unreadable or not verified.

Why does independent verification matter, and what is Utah doing?

Utah's AI sandbox is a state program, not a law, and it builds independent evaluation into healthcare AI pilots. The state's Office of Artificial Intelligence Policy runs the sandbox. Its page on third-party evaluators says participating companies are required to report on their own performance and that the office does not rely on those reports alone. That is the worry behind a point I have made before: a system that produces an output and also grades it has a self-assessment, not a check. I made that argument in why an AI system can't audit its own output and extended it to AI judges and supervising agents in who verifies the AI that verifies healthcare AI.

As of November 2, 2026, the office has signed agreements with six organizations in three roles. Stanford's Clinical Excellence Research Center (September 30), mpathic (September 29) and Vega Health (September 28) are clinical and technical evaluators that can review a pilot before approval and while it runs. Clarion AI Partners (September 29) and Glacis Technologies (August 10) provide runtime attestation, meaning signed receipts showing that a specified safety check ran. The page says a receipt does not show that the AI is safe or that the company followed the law. The Coalition for Health AI (October 2) qualifies evaluators and does not evaluate pilots itself.

The company being evaluated pays the evaluator directly. Evaluators must certify that each evaluation was independent, report any patient safety concern whether or not the company passes, and let the office audit their work. The office keeps final decision authority. The page describes the primary goal as an audit, "objective confirmation of key participant claims" (Utah Office of Artificial Intelligence Policy, page read November 2, 2026).

Three limits apply. Each agreement is a memorandum of understanding between the office and one evaluator, and the page cites no statute or rule requiring evaluation of healthcare AI generally. The program is new: the page says the evaluation criteria are still being finalized with each organization. And evaluator certifications do not replace state regulatory judgment or guarantee sandbox admission, advancement or approval. I cite Utah as one state office building its sandbox on the idea that self-reported performance is not enough alone, not as proof that third-party evaluation works.

Orinyx is not one of the six, and it does not certify or approve any product.

Why did clinician feedback matter to the design?

Clinicians read an order for what it asks someone to do. A tool built only by developers will check what is easy to check. A clinical pilot partner has been sending me written comments on synthetic medication-safety cases, and those comments are one input to the design, alongside developer feedback and my own test failures. They are not an endorsement, and this article makes no claim that anyone has approved how Orinyx works.

Developer testing has changed the product too, and one change is already public. A tester's session on October 6 led to the UNREADABLE answer, so that an order nobody can read gets a different response from an order Orinyx simply has no rule for (what changed at Orinyx).

How do I join the beta?

Beta sign-ups are open at orinyx.io/developers. The button reads "Join the Free MCP Waitlist." The Model Context Protocol (MCP) is a standard way for an AI application to call outside tools, and the Orinyx MCP lets an agent send its medication orders and claims to be checked. Use synthetic data only. Never put patient data in the form or in a test. Orinyx does not connect to an EHR, and it makes no clinical decision: Orinyx flags; the clinician decides. I am not giving a date for the beta in this article.

Where this fits: what can any team do without Orinyx?

Any team can do most of this with its own tools, without Orinyx.

List the vocabulary behind every medication field. Where a field holds free text that could hold an RxNorm concept identifier, record it as a known gap.
Give units, routes, body sites and frequencies their own fields, and let "unreadable" be a valid output. A parser that always returns something will sometimes return the wrong thing.
Count "could not check" as a result. Show it and count it, and never let it render as success.
Measure severity, not only error rate, and use realistic data, not only exam-style questions. The note study's major and minor labels are a usable template.
Keep the checker apart from the generator, and make a check a fixed rule wherever a rule exists.
Attach the source to every answer, so a reviewer checks evidence and not the tone of the response.

Orinyx does these parts so a developer does not have to rebuild them: it reads the order, checks it against cited label text and reports what it could not check. The structure matters more than the vendor, and a team can build it without me. If your agent wrote "metoprolol" with no formulation tomorrow, which of your systems would notice that two products fit?

Frequently asked questions

What is a knowledge graph in pharmaceutical data?

A knowledge graph is a database of typed things and the named relationships between them, such as an ingredient, a dosed product and the label section that describes it, so a program can answer a question by following links. A 2024 scoping review of 47 articles found graphs used to predict adverse drug reactions and drug-drug interactions, with promising but variable results and few comparisons with older methods.

Does Orinyx use a knowledge graph?

Not today. Orinyx reads each order with a fixed grammar and checks claims against cited FDA label text with fixed rules, and no language model decides a verdict. A knowledge graph is planned underneath those checks, and bounty agents, specialized agents that each test a different way an AI system can go wrong, are planned after the MCP launch. Both are direction, not product, and there is no date.

Why do accuracy scores miss errors in medical AI?

Accuracy counts how often an answer is right. It does not say how serious the wrong ones are or what kind of error they are. A 2025 JAMA review of 519 studies published from 2022 to early 2024 found 95.4% used accuracy as the primary dimension of evaluation and 5% used real patient care data. In one 2025 note-writing study of GPT-4, 44% of hallucinated sentences were major, meaning they could change diagnosis or management.

What is Utah's third-party evaluator program for healthcare AI?

It is a program of Utah's Office of Artificial Intelligence Policy within its AI sandbox, not a law. Participating companies report on their own performance, and as of November 2, 2026 the office has signed agreements with six organizations: three clinical and technical evaluators, two runtime-attestation providers and one network that qualifies evaluators. The company being evaluated pays the evaluator, and the office keeps the final decision.

Does Orinyx certify healthcare AI?

No. Orinyx does not certify or approve any product. It checks medication orders and claims against cited FDA label text with fixed rules and reports what it could not check. It is not one of the six organizations Utah's AI sandbox has signed as third-party evaluators. Beta sign-ups are open at orinyx.io/developers.