Answering Staff Questions From Your Own Documents: An AI Knowledge Base for Business

TL;DR: An AI knowledge base for business finds the passages in your own documents that answer a staff question, writes a short answer from them, and shows the source. It is worth building when every answer links to its document, people only see what they are allowed to read, and someone measures how often it is wrong.

An AI knowledge base for business answers a simple need: your staff ask the same questions every week, and the answers already exist somewhere in a shared drive, a handbook, an old email thread or a ticket. The question owners in Worcester ask me is not whether the technology works. It is whether they can trust what it says.

That is the right question. An AI knowledge base can be confidently wrong in a way that looks exactly like being right, and a few design choices, all easy to check before you buy or build, decide which you get.

What is an AI knowledge base, in plain words?

An AI knowledge base works in two steps. First, the system searches your own documents for the passages most likely to answer the question. Second, a language model reads those passages and writes a short answer from them. The technical name for this is retrieval-augmented generation, usually shortened to RAG. If you want the vocabulary, Qurtoo keeps a plain definition of retrieval-augmented generation (opens in a new tab).

The idea comes from a 2020 research paper that introduced the term. Its authors noted that language models working from memory alone have trouble "providing provenance for their decisions and updating their world knowledge," and found that adding retrieval produced "more specific, diverse and factual language." (Lewis et al., NeurIPS 2020)

Those phrases describe what a business needs. Provenance means the answer points to where it came from. Updating means that when your vacation policy changes, the answer can change the same day.

The search step looks for meaning, not just keywords, so a question about "time off for a funeral" can find the paragraph titled "bereavement leave." The parts that decide whether an AI knowledge base is safe to use are less visible.

Why must every answer cite the source document?

Because a language model can produce a wrong answer that reads like a right one. NIST, the federal standards agency, calls this confabulation: "the production of confidently stated but erroneous or false content (known colloquially as 'hallucinations' or 'fabrications') by which users may be misled or deceived." NIST also notes that confabulations "are a natural result of the way generative models are designed." (NIST AI 600-1, July 2024)

Retrieval narrows the problem without removing it. In an AI knowledge base, the source is how a staff member checks the answer in ten seconds instead of trusting it blindly.

There is a subtler trap, and NIST names it too. Generated answers "may also include confabulated logic or citations that purport to justify or explain the system's answer, which may further mislead humans into inappropriately trusting the system's output." A citation only helps if it is real and actually says what the answer claims.

Researchers at Stanford measured this in public AI search tools. On average, "a mere 51.5% of generated sentences are fully supported by citations and only 74.5% of citations support their associated sentence." (Liu, Zhang and Liang, Findings of EMNLP 2023) Those were 2023 systems searching the open web, not your files, but the lesson carries: a link next to an answer is a promise someone has to keep.

In practice, a trustworthy answer from an AI knowledge base has three properties:

• It quotes or points to the exact passage, not just the file name, so the reader can see the sentence that supports it.
• It says when it does not know. "I could not find this in the documents you can access" is a good answer. A guess is not.
• It shows the document's date or version, so an answer drawn from a 2019 handbook looks different from one drawn from this year's.

How accurate is an AI knowledge base that answers from your own documents?

More accurate than a general chatbot, and still not perfect. The clearest evidence comes from law.

A peer-reviewed study from Stanford and Yale tested commercial legal research tools built on retrieval. The authors found that "hallucinations are reduced relative to general-purpose chatbots," yet the tools still "each hallucinate between 17% and 33% of the time." (Magesh et al., Journal of Empirical Legal Studies, 2025)

Their definition of a wrong answer is worth borrowing. It includes answers that are simply incorrect, and answers that are "misgrounded," meaning the response "cites a source which does not in fact support its claims." (Stanford HAI, May 2024)

Legal research is a hard case, and a small business handbook is far simpler. I would not expect the same error rate on "how many sick days do part-time staff get," and I would not assume it is zero either. You do not know your own error rate until you measure it.

How do you make sure staff only see what they are allowed to see?

This is the question I would ask any provider first, because the failure is quiet. If an AI knowledge base indexes the whole shared drive and answers everyone from all of it, a question about "the review process" can surface a passage from someone else's performance review. Nobody hacked anything. Brian has written about the step before this one: whether staff can put customer data into AI tools (opens in a new tab) in the first place.

OWASP, the security standards community, lists this among the top risks for AI applications in 2025. "Inadequate or misaligned access controls can lead to unauthorized access to embeddings containing sensitive information," and the model "could retrieve and disclose personal data, proprietary information, or other sensitive content." Its recommended fix: "Implement fine-grained access controls and permission-aware vector and embedding stores." (OWASP LLM08:2025)

OWASP's guidance on sensitive information says the same thing in general terms: "Limit access to sensitive data based on the principle of least privilege. Only grant access to data that is necessary for the specific user or process." (OWASP LLM02:2025)

Massachusetts businesses will recognize the principle. The state's data security regulation calls for access controls that "restrict access to records and files containing personal information to those who need such information to perform their job duties." (201 CMR 17.04) An AI knowledge base that ignores folder permissions works against that rule, even if every file was locked down correctly in the first place.

What good looks like:

• Permissions follow the person asking. The system checks who is signed in and only searches documents that person can already open.
• Permissions follow the source. When someone loses access to a folder, they lose it in the answers too, without a separate cleanup.
• Some collections stay out entirely. HR files, payroll and anything with health information may not belong in a general staff assistant at all.
• Every question and answer is logged, so you can see what was asked and what was shown.

One more risk sits inside the documents themselves. A file can contain text written to steer an AI, which is a form of prompt injection. We explain it in prompt injection attacks explained. OWASP's advice for knowledge bases is to "audit and validate the integrity of the knowledge base for hidden codes and data poisoning" and to "accept data only from trusted and verified sources."

How do you keep the documents current?

An AI knowledge base is only as good as the documents it reads. If the shared drive holds three versions of the expense policy, the system may find all three and blend them. OWASP names this too: "Data federation knowledge conflict errors can occur when data from multiple sources contradict each other."

The fix is mostly housekeeping, and it is the part I see skipped most often:

1. Give every source collection an owner. The office manager owns the handbook. The service lead owns the procedures. Someone has to say "this is current."
2. Retire old versions instead of keeping them beside the new one. Archive them somewhere the search cannot reach.
3. Put a review date on each document. When the date passes, the system can label answers drawn from it as possibly out of date.
4. Re-index on change, not on a schedule nobody remembers. When a document is edited, the search should see the edit within hours.
5. Keep a short list of questions the documents cannot answer. Those gaps are a writing task for a person, not a search problem.

Messy sources produce confident, messy answers. I wrote more about what AI can and cannot do with your business data (opens in a new tab).

How do you measure wrong answers?

With a test of your AI knowledge base that you write before launch and rerun after every change. NIST suggests that organizations "review and verify sources and citations in GAI system outputs during pre-deployment risk measurement and ongoing monitoring activities," and warns against judging a system from "narrow, non-systematic, and anecdotal assessments." (NIST AI 600-1)

For a small business, that can be one spreadsheet:

1. Collect 50 real questions from the people who get interrupted most, including a few the documents cannot answer.
2. Write the correct answer and its source for each. This takes an afternoon and it is the most valuable afternoon in the project.
3. Grade every response in one of four buckets: correct with the right source, correct with the wrong source, wrong, or "I don't know."
4. Count "correct with the wrong source" as wrong. That is the misgrounded case the Stanford researchers flagged, and it is the one that erodes trust.
5. Rerun the same 50 questions whenever documents, settings or the underlying model change, and compare the counts.

Then keep measuring the AI knowledge base after launch. Give staff a one-click "this was wrong" button and read those reports weekly. A rising "I don't know" rate usually points to gaps in the documents; a rising wrong rate means something changed.

If you want the technical pattern behind this, Qurtoo has a playbook for a retrieval test harness that catches regressions (opens in a new tab). For the management side, I also wrote about how to audit what AI is actually doing (opens in a new tab) in a business. Kief Studio calls checks like these quality gates.

When is a plain search box enough instead of an AI knowledge base?

More often than vendors suggest. If your staff mostly look up exact things, such as a part number, a client name, a form title or a policy number, ordinary keyword search does that job well and cannot invent anything.

Research backs this up. A 2021 benchmark that tested retrieval methods across 18 public datasets found that BM25, a classic keyword ranking method, remained a strong baseline. The methods that beat it on average did so at a high computing cost. (Thakur et al., BEIR, NeurIPS 2021)

A simple way to decide:

• A search box is enough when questions use the same words as the documents, the collection is small, and people want the file, not a summary.
• Meaning-based search helps when people describe a situation in their own words and the answer sits under a different heading.
• A written answer with citations helps when the answer is spread across two or three documents, or when new staff do not yet know what to search for.

Many good systems combine the first two. The written answer is the last layer, not the first.

What does this look like for a Worcester or Central Massachusetts business?

In a small business, the person who knows where everything is often runs the business too.

The state government offers a useful local example of the guardrails in practice. In February 2026, Massachusetts announced a phased rollout of an AI assistant across its executive branch of almost 40,000 employees, starting with its technology office. The state's guidance to employees is plain about the limits: the assistant "may occasionally generate incorrect or incomplete information." (WBUR, February 2026; StateScoop, February 2026)

For a small team's AI knowledge base, the same advice scales down. For example:

• A dental office in Shrewsbury might load its front desk procedures so a new hire can answer "how do we handle a missed appointment fee" without interrupting the office manager, while patient records stay out entirely.
• A machine shop in Worcester might index work instructions so the second shift finds the right setup steps, with each instruction's date beside the answer.
• An accounting firm in Westborough might search past client memos, with each staff member seeing only their assigned clients.

How should a small business start an AI knowledge base?

1. Pick one collection with one owner, such as the employee handbook, not the whole drive.
2. Clean it and decide who sees what before anything is indexed.
3. Write your 50 test questions and their answers.
4. Try plain search first. If it answers most of the 50, you may be done.
5. Add cited answers only where plain search falls short, then grade the same 50 questions and decide with numbers.