Quick answer

When the language model and the document base sit inside the firm's network, case files don't go to an outside AI provider. A lawyer asks a question in plain language, the system searches the firm's documents both by meaning and by the exact wording of a provision, answers only from the passages it finds, and cites a source for every sentence. When there's no source, it says “I don't know”. Checking the answer, and responsibility for it, stay with the lawyer, which is exactly what Polish bar guidance on AI requires.

This post shows what that looks like using the fictional law firm from our Knowledge bases page. The firm, its cases and documents are sample data. This is not legal advice.

Why law firms are wary of cloud AI

Professional secrecy

In Poland, the Act on the Bar (Article 6) and the Act on Legal Advisers (Article 3) impose the same duty: keep secret everything learned in connection with providing legal help (Act on the Bar, Act on Legal Advisers, both in Polish). Case files, client correspondence and matter notes are exactly the material you'd want AI to help search. Other jurisdictions have their own versions of this duty, and the logic below applies broadly.

What the Polish bar says

  • The National Chamber of Legal Advisers (KIRP) published recommendations on AI in legal advisers' work in 2025. According to the chamber's own summaries, they call for human oversight of AI output, no entering of privileged information into external tools without proper safeguards, and no trying out new tools on real client matters. The document is meant to be updated over time (KIRP).
  • The Polish Bar Council (NRA) amended the advocates' Code of Ethics on 12 June 2026. Technology, including AI, may only play a supporting role, must not breach professional secrecy, and must not lead the lawyer to stop assessing and checking the results. The advocate remains personally responsible for the content of their work (Rzeczpospolita, 16 June 2026, in Polish).

Personal data

Case files are full of personal data, including special categories. An external AI provider that processes them is a processor and needs a data processing agreement (GDPR Article 28), and any transfer outside the EEA falls under Chapter V of the GDPR (GDPR, EUR-Lex). This can be handled with contracts, but it has to be done deliberately, for every tool.

Researchers from Stanford and Yale tested commercial legal research tools built on RAG (Lexis+ AI, Westlaw AI-Assisted Research, Ask Practical Law AI). They hallucinated between 17% and 33% of the time: less than general-purpose GPT-4, but far from zero (Magesh et al., 2025). “AI for lawyers” on the label isn't enough. What counts is whether you can check the answer against the source.

What it looks like: seven stages from scan to citation

Below is the scenario from our knowledge base story. A fictional law firm keeps contracts, letters and statutes across several departments. The whole chain runs on its own server.

1. A document comes in

A contract arrives as a scan, a letter as a DOCX, a statute as a PDF. OCR turns images into text (Polish and English), then we clean up footers, page numbers and hyphenation. Each document is split into passages along its own structure: section, paragraph, article. Every passage gets its full heading path and one sentence of context, so “Section 4(2)” doesn't float in a vacuum. We explain why OCR decides the quality of the whole base in our post on modern OCR.

2. Passages and meaning

The embedding model reads each passage together with its headings and turns it into a vector that captures its meaning. Passages with similar meaning end up close together, even when they use different words. Each one gets metadata: document, article, department, version. It also goes into a full-text index that handles Polish inflection. We compare embedding models for Polish documents in a separate post on Polish embedding models and rerankers.

3. Searching like a lawyer

A lawyer searches two ways at once: by meaning and by the exact wording of a provision. So does the system. The question is classified first, then we search the vectors and the full text in parallel, merge the two lists, and a reranking model reads each question and passage pair and sets the final order. Full text matters here: case references, article numbers and names are things vectors catch less reliably.

4. An answer with evidence

Passages go into the model's context in order of importance, within a budget set for the type of question. The model writes only from what it was given and cites a source for every sentence. Clicking a citation opens the document and highlights the sentence it came from. When no passage clears the relevance threshold, the answer is “I don't know”. We explain how that threshold works in our post on a knowledge base that says “I don't know”.

5. Knowledge that changes

A new version of a statute or an amendment to a contract doesn't mean rebuilding the whole base. The system recognises the document by a stable identifier, compares its passages with the previous version and recomputes only the ones that changed. The new version becomes current only once it's fully processed, and the old one stays in the history. For a lawyer this matters: an answer should say which version of the document it came from.

6. Who sees what

Every passage carries the department it belongs to in its metadata. The filter works inside the index, before anything is searched, so another department's files never reach the ranking or the model's context. Every question goes into the audit log: who asked, when, and which sources the answer relied on. We go deeper into permissions in our post on secure RAG data access.

7. On your premises, not in the cloud

The whole chain (OCR, indexes, embedding model, reranker and language model) runs on a server in the firm's network, on NVIDIA Blackwell GPUs. Questions, documents and answers never leave, and the audit log stays with the firm. The models are open models from the Qwen, Gemma, GLM and Mistral families, running without any external API.

One question, step by step

To make it concrete, let's follow one fictional question from a lawyer in the real estate department: “What notice period does the lease in client A's matter set, and did the amendment change it?”

StepWhat the system doesWhat the lawyer sees
ClassificationRecognises a question about a specific document and a change to itNothing, it happens in the background
Permission filterLimits the search to the real estate departmentNothing, but other departments' files are out of reach
SearchVectors find passages about termination; full text catches “notice period” and the client's nameNothing
Reranking and thresholdPicks the lease clause and the amendment clause, drops similar passages from other leasesNothing
AnswerTwo or three sentences, each cited: lease § X, amendment § YAn answer with citations [1], [2]
VerificationClicking a citation opens the document at the cited spotThe lawyer reads the original sentence and decides

If the amendment weren't in the base, the system shouldn't guess. It should answer from the lease and say it found no amending document.

Cloud or the firm's own server: a fair comparison

CriterionGeneral cloud AI chatOn-prem knowledge base
Where files and questions liveWith the provider, often outside your countryOn a server in the firm's network
ProcessorThe AI provider, with a data processing agreementNo external model provider
Source of answersThe model's general knowledge plus whatever you paste inOnly the firm's documents, with a citation per sentence
PermissionsDepends on the tool, usually per accountDepartment filter inside the index
AuditDepends on the plan and providerLog of questions and sources kept by the firm
Upfront costLow, a subscriptionHigher: a GPU server or a managed service, plus setup
MaintenanceThe provider's jobThe firm's or a partner's job

The cloud has real advantages: a low barrier to entry and no hardware to look after. There are also business plans with data processing agreements. But for a firm that wants to search its own files rather than draft emails, an on-prem knowledge base solves two problems at once: files stay put, and answers have sources.

What stays with the lawyer

No technology takes over professional responsibility. Polish bar guidance says so explicitly, and a well-designed system makes it easier to live up to:

  • Checking. A citation for every sentence cuts verification to seconds, but someone still has to do it.
  • Legal judgment. The system finds and summarises what's in the documents. It doesn't decide what that means for the client.
  • Telling the client. Under the amended ethics rules, an advocate should name the tools used in a client's matter when the client asks. The audit log helps answer that.
  • Team skills. Since February 2025, the EU AI Act has required organisations that use AI to take steps on their staff's AI literacy (Article 4). Regulation (EU) 2026/1744, the so-called Digital Omnibus, softened this into a duty to take measures rather than guarantee a particular level (AI Act, EUR-Lex). How a specific system is classified under the AI Act, including Annex III, is worth assessing with a lawyer.

This is not legal advice. Discuss any specific deployment with your data protection officer and your bar.

Checklist for a law firm

  • You know which categories of documents go into the base and which don't.
  • The whole chain (OCR, indexes, models) runs inside the firm's network, or in infrastructure covered by a data processing agreement.
  • Outbound traffic from the AI server is blocked.
  • Department permissions work inside the index, before search.
  • Every sentence in an answer cites a sentence in a document.
  • The system answers “I don't know” when it finds no source.
  • The answer names the document version.
  • The audit log records questions, answers and sources.
  • The tool was tested on an anonymised sample before going near real matters.
  • The team knows how to check answers and where the tool's limits are.

How to check it on your own documents

We start with a sample: a few hundred documents from one department, anonymised or processed on the firm's premises, plus a list of questions your lawyers really ask, each with the source they expect. We build the base, and the result is a report: where the answer hits the source, where the system says “I don't know”, and why. A test like this also follows the advice not to try new tools on real client matters.

The full scenario, with animations of every stage, is on our Knowledge bases page. You can order a trial base through the demo.

Sources