Private AI for law firms: why client files should stay in the building

The question behind the question

When partners ask "can we use AI?", the real question is usually narrower. It's "what happens to our client files if we do?" That instinct is correct. A firm's duty of confidentiality doesn't pause because a tool is convenient, and a client's file doesn't stop being sensitive because a vendor's terms of service call it "content."

Most legal AI today is cloud software. Your documents leave the firm, get processed on servers you'll never see, and sit under someone else's retention schedule, someone else's subprocessors, and someone else's breach surface. Plenty of vendors are careful. But careful or not, you've added a company to the list of people who hold your client's records, and you did it for every matter you run through the tool. For some clients that's a disclosure conversation. For others it's a dealbreaker you find out about later.

The other way to do it

Nothing about modern AI requires the cloud. A system that reads documents, builds chronologies, and answers questions about a record can run entirely on a machine that sits in your office. That's the architecture we chose for the Tiber River Legal Workbench, and the difference isn't a feature on a comparison chart. It's the whole posture.

• Your documents never transit. Not to us, not to a subprocessor, not to a model vendor. The runtime makes no outside network connections.
• Nothing trains on your client data. The model arrives already trained. Your matters don't improve anyone else's product.
• There is no vendor copy of your files to breach. We never hold them in the first place.
• You can check all of it. No egress isn't a promise buried in a contract. It's a property of the box, and your own people can test it.

An honest caveat: local isn't a magic word

It would be easy to stop there, and it would be a little dishonest. A server in a closet, run badly, is worse than a well-run cloud. Unpatched, unlogged, and open to everyone in the office isn't data sovereignty. It's just risk that stays home.

So the honest claim is narrower than "local, therefore safe." An appliance done right gives you custody plus controls, with evidence behind both. When a client, an insurer, or a court asks how your firm supervises its AI, you answer from a log, not from a vendor's brochure.

Five questions to ask any legal AI vendor

Including us. If you're evaluating tools this year, these five questions sort the field quickly:

1. Where, physically, are our documents processed?
2. What do you retain, for how long, and who are your subprocessors?
3. Is our data used to train or improve any model, and how would we know?
4. Can we independently verify that nothing leaves our custody?
5. What record exists of who accessed what, and can we produce it later for a client, an insurer, or a court?

A cloud vendor can have good answers to the second and third. Only hardware you control gives a clean answer to all five.

In practice you'll read most of those answers off a vendor's security page rather than hear them out loud, and those pages are written to sound reassuring whether or not they say much. We've since written a guide to reading one for scope instead of adjectives.

Related reading: The verification tax: the real reason lawyers don't trust AI, how to read an AI vendor's security page, what "your files never leave the building" means, technically, and why AI output should wait for attorney review.