AI with Guardrails: How We Run LLMs Without Leaking Family Data

There's a folder on my laptop with everything that matters. Insurance policies, tax returns, the kids' medical records, our mortgage documents. Years of family life in PDFs and scans.

I can't search any of it. Windows doesn't index the contents. I've tried uploading documents to cloud services, but they're not searchable either -- not in any meaningful way. And even if they were, I'd be handing my family's most sensitive data to a company headquartered in a country that might request access to it for reasons I'll never know about.

As a Canadian, that doesn't sit well with me. I wanted a safe, accessible space for family data -- one where I could actually ask questions and get answers -- without having to trust that a server in Virginia is treating my documents the way I'd want them treated.

That's why I built Archevi. And that's where things got complicated.

The part I didn't think about

I built the system. A document store, a search engine powered by AI, a chat interface where you could ask "When does our home insurance renew?" and get a real answer pulled from your actual policy.

Speeds were great. The experience was exactly what I wanted.

What I hadn't thought about was what happens when a user uploads a document with their Social Insurance Number on it. Or a credit card statement. Or a medical record with diagnostic codes. The AI needs to read those documents to be useful. But every time it does, fragments of that data travel to external services -- the large language models that power the responses.

I knew that to truly help families, I had to think about the security for them. Most people aren't going to audit what an AI chatbot sends to its backend. They shouldn't have to.

How the privacy layer actually works

If you asked me to explain this to a neighbour, I'd say it works in three steps.

First, when you upload a document -- say, your car insurance policy -- the relevant data is extracted and stripped from the raw file. We don't store entire pages; we store the structured information that matters.

Second, that data sits in a secure database. Your documents, your data, stored in one place.

Third, when you ask the AI a question, the request is anonymised before it leaves. Personal names, account numbers, anything identifiable gets replaced with placeholders. The AI sees "James Chen has a policy with Lakeside Insurance expiring on March 15." It never sees your name or your insurer. The answer comes back with placeholders, and we swap the real details back in before you see the response.

Why Groq and Cohere, not OpenAI

I used Claude to do an environment search for the best combination of speed and safety. Three things stood out.

Cohere has a positive Canadian reputation. That mattered to me. Groq is the fastest inference engine available right now, and when you're building a chat experience, speed is everything. And both offered generous free tiers for testing and building, which meant I could iterate without burning through a budget before the product was ready.

The models themselves are tools. The privacy layer is what makes them safe to use.

The uncomfortable truth

At some point, you're connecting to external services via API. There's no way around it -- unless you're running your own language models on your own hardware, and even then you've got supply chain risks. Every external connection is a potential exposure until you get the anonymisation right.

You also take a hit to response speed. Adding a security layer means every request goes through extra processing before it reaches the AI and again after the response comes back. It's not dramatic -- we're talking a couple of extra seconds -- but it's real.

Worth it? Absolutely. But I won't pretend it's free.

What it looks like now

Every query that passes through Archevi's AI goes through the privacy layer. No exceptions. Even casual questions like "what did I upload last week?" pass through the same anonymisation pipeline. The system doesn't distinguish between sensitive and non-sensitive -- it treats everything as if it matters, because to the family that uploaded it, it does.

As someone who stores his own family's documents in this system, that gives me peace of mind. My experience is customised to my family's actual data, but none of that personal information reaches the large language models. It stays where it should -- with us.

That's what "AI with guardrails" means in practice. Not a marketing checkbox. Not a privacy policy nobody reads. An actual layer of engineering between your family's data and the outside world.