Retrieve from manuals. Do not invent code clauses.

Why this matters

Design standards are controlled documents: edition, jurisdiction, national annex, and mandatory vs commentary language all matter. Language models are fluent. Fluency plus a fake clause number is worse than “I don’t know,” because it looks like research.

Retrieval-augmented generation (RAG) is a way to ground an answer in passages you actually stored. Used carefully, it can help an engineer find the right page faster. Used carelessly, it is a clause-invention machine with a bibliography.

What RAG is — and is not

In the pattern popularised for knowledge-intensive NLP, the system retrieves text from a corpus, then generates a response that may attend to those passages. The retrieval step is the only reason this is different from asking a bare chat model about “the code.”

RAG is not:

• a licensed copy of every standard you wish you had;
• a determination of which document governs the project;
• a replacement for reading the clause in context (exceptions, charging language, pointers to other sections);
• a solver of the inequalities those clauses contain.

If the retrieved passage is missing, stale, or from the wrong book, generation will still produce sentences. That is a reliability problem, not a user-error footnote.

Inventing clauses vs pointing at manuals

Inventing looks like: a numbered identifier, a paraphrase of “required” load combinations or detailing, and no file you can open that contains that language. Students and rushed practitioners paste this into notes. It is indistinguishable from competence until someone opens the real document.

Pointing at a manual looks like: a citation a human can resolve (document title, edition, section/clause id as printed, page or stable fragment if you store one), plus the retrieved excerpt your system actually used, plus a clear label that the excerpt is not a substitute for the official text.

Those are different products. Only the second one belongs near engineering work, and even then only as a finding aid.

A workflow that keeps the standard in charge

1. Decide the governing documents before retrieval (jurisdiction, edition, whether you are in commentary). A model cannot do this from a generic prompt.
2. Restrict the corpus to copies you are allowed to use, with edition metadata on every chunk. Mixing 2010 and 2022 language in one index is a known way to invent hybrids.
3. Retrieve first. If nothing relevant is returned, the system should say so. Do not “fill in” from parametric memory.
4. Show the passage. The reviewer must see the chunk, not only a summary.
5. Human application. Load paths, φ-factors, material specifications, and exceptions are applied by a person to a specific structure. Generation can mis-associate a clause with the wrong member.
6. Record the citation you used, as you would for any other reference. If you cannot cite it, you did not use it.

What to evaluate (if you build or buy this)

Ask for evidence on questions like these. None of them are solved by a marketing demo on a famous clause everyone already knows.

• Recall on held-out queries: given a real question from your practice, does the right section appear in the top retrieved chunks?
• Faithfulness: does the generated sentence stay inside the retrieved text, or does it add numbers and “shall” language that were not there?
• Edition isolation: can the system refuse to answer from the wrong year?
• Abstention: what is the behaviour when the index has no answer — refusal, or a fluent guess?
• Commentary vs provisions: are non-mandatory notes labelled as such in the corpus, or flattened into the same embedding space?

If a vendor cannot talk about those tests, you do not have an evaluation. You have a demo.

Common mistakes

• Pasting a generated “Clause 12.3.x requires…” into a calculation without opening the book.
• Indexing blog posts, forum threads, and unofficial PDFs as if they were the standard.
• Chunking through a table so that a capacity equation is split from its limits of applicability.
• Allowing the generator to cite a clause id that never appeared in the retrieved chunks.
• Treating a high similarity score as “this is the governing provision.”
• Asking RAG for a numeric design (bar size, drift limit application) instead of asking it to locate the provision you will apply yourself.

Limitations

This article does not implement a RAG stack, does not publish a corpus, and does not quote design standards. Copyright and licence terms control what you may index; “it would be useful” is not a licence.

Retrieval quality depends on chunking, metadata, and the query. Even a perfect index cannot choose the right load combination for your building. National annexes, amendments, local amendments, and project specifications routinely override the “main” book. A hub article cannot list them.

Model-only answers about codes remain unreliable even when RAG exists in the same product. Confirm which code path actually ran.

Professional context

Engineers already know how to use a table of contents, a commentary, and a highlighted PDF. RAG is optional infrastructure around that habit. It does not move responsible charge. If your QA procedure requires citing the edition on the drawing, cite the edition you opened — not the edition a model mentioned.

For analysis files drafted with AI, the same rule applies as for clauses: fluency is not evidence. Pair this briefing with the four-role split (calculation vs suggestion vs review vs validation) before you connect an assistant to either manuals or solvers.

References

• Lewis, P., et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv:2005.11401. Describes the retrieve-then-generate pattern. It is not a method for applying building codes.
• NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0) — nist.gov/itl/ai-risk-management-framework. Evaluation language only; not a design standard.
• The governing structural or building code for a project is the official edition adopted by the jurisdiction. This article does not quote or replace it.

Evaluation and limitations

Treat the notes below as evaluation context, not as a verified calculation or code check.

Evaluate a standards RAG system on retrieval and citation faithfulness, not on answer length: can a reviewer open the cited edition and find the passage; is commentary distinguished from mandatory language; what happens when the corpus has no answer. This article is a reliability briefing. It is not a benchmark of any commercial assistant and it quotes no code clauses.