RAG readiness checklist: 6 steps before you build

This is Chapter 0 of a series about a real RAG project for a SaaS company. I’ve recently been diving deep into RAG systems, as part of the technical setup work I do for AI-native tools. I’m writing this while the project is still happening.

As for my background: I'm a tech, automation, and integrations specialist, an ex-developer who works closely with companies to wire their systems together like CRMs, data pipelines, internal tools so people spend less time on manual work and more time on the job they were actually hired for. RAG is the newest thing on that list.

What’s RAG?

You've probably already heard about Agentic RAG, which takes this idea a step further. An agent is a program that can independently make decisions, plan iteratively, and execute tasks toward a goal, rather than just answering one prompt and stopping. Applied to RAG, that means the retrieval step stops being a single, rigid pass and becomes something the system can revisit. It runs from rigid, one-way pipelines on one end, to free-form agents that can feel like a black box on the other, with structured workflows sitting in between as a middle ground. That middle ground is roughly where I expect to land for this client, though I haven't made that call yet either. Full agentic freedom sounds appealing until you're the one explaining to compliance why the system decided, on its own, to check a fourth document nobody approved.

Why I spent a week not building anything

Dear diary, this day week was a bad day week…

First of all, I like to be prepared, and I've learned a few things in my career. One of them is to constantly ask myself: am I doing this because it's the right choice, or because I'm just letting my dev heart chase the shiny object? RAG is a pretty new idea, so there's a lot of noise about it on LinkedIn, yet not much well-structured information about successful implementations at real companies. So I started to dive deep into the topic, investigating, learning, trying new services. Before touching anything technical, I'm actually spending about one week not building anything at all. And it feels wrong the whole time 😫. Everyone involved, including me, wants to see a demo. My impostor syndrome is kicking in hard here too. At some point I stop being sure whether I'm really helping the client, or just stalling. Remember kids, whenever in that situation, the only way out is to just start from basics, baby steps.

How to prepare a company for RAG: 6 steps

I can’t keep thinking forever. I’m supposed to deliver this so let’s focus on how this idea actually comes to execution. Leadership wants RAG for a real reason and it’s to help a team find answers inside a large pile of internal policy documents, without digging through folders every time. That reason isn’t hype, and it isn’t a request for a quick or simple fix either. But having a real reason isn’t the same as having a measurable problem. Nobody has turned it into something you could put a number on yet, a target, a metric, a clear definition of what “solved” would look like. Adam Brotman and Andy Sack make a similar point in AI First. The technology moves fast, and that speed tempts people to skip the step where you turn a good reason into something you can actually measure.

So here’s the plan I’m actually following, step by step, before touching anything technical.

1. Check where the company actually stands with AI, not where it thinks it stands.

This is still happening. It’s where we started, and we’re in the middle of it right now. Not everyone in the company is at the same point. Leadership is ready to spend money, while the people who’d actually use the tool haven’t touched anything like it yet. In my conversations with other Internal Automations leaders at SaaS companies, this comes up again and again. Almost every company faces the same problem. Some parts of the business understand and use AI, other parts don’t, and that gap causes most of the friction. Some people call this a maturity curve, moving from ad hoc experiments toward something the whole company actually uses. Most companies I work with look further along on paper than they really are in practice. That gap causes a common failure: a pilot that works well in one team, then never leaves that team. Finding where the real gap sits tells me more than any conversation about vector databases would.

2. Do the boring homework before the fun part starts (Gauge).

I found the idea in Jared Matfess’s book on AI adoption: he calls this stage Gauge. It’s the boring part nobody wants to do. Not the model, not the interface, just the unglamorous background checks: Who actually owns this data? Is anyone maintaining it? Would people even trust an answer that came from it? It’s the kind of thing you skip because it feels like a delay, and then it comes back later as a much bigger delay. You’re taking a debt (either technical or process you have to pay soon). Maybe two teams keep separate copies of the same file. Maybe nobody can say who actually owns the data. Maybe there’s a compliance problem nobody has noticed yet. The goal isn’t a full audit of every system. It’s finding the few problems that would stop the project in month three, if I don’t catch them in week one.

Score it, don’t just guess.

To make that less of a gut call, I’m using a more structured yet still simple method: the AI Compatibility Score from the book Architecting Enterprise AI Strategies. The framework scores an organization across eight dimensions: how modern the legacy systems are, whether data can be traced end to end and trusted, how ready the cloud and integration layer is, how solid privacy and governance are, whether MLOps and observability practices exist, how well AI work lines up with actual business strategy, how ready the culture is for change, and how the company compares to others in its sector. Each dimension gets a number from 1 to 5.

The scale itself is easy to follow. A 1 (Initial) means there’s basically no structured capability, people just improvise and risk is high. A 2 (Emerging) means early efforts exist, but they’re fragmented and depend on a couple of people rather than an actual process. A 3 (Developing) means there’s a real process now, even if it’s not applied the same way everywhere. A 4 (Established) means the whole company does this consistently, not just one team. A 5 (Leading) means the company isn’t only consistent, it’s proactive and actively getting better at it over time, ahead of its peers.

With eight dimensions on a 1-5 scale, the maximum total is 40. Harjika splits that into four bands. A score of 8 to 16 is Fragile Readiness, where ambition is running ahead of what the company can actually support. 17 to 28 is Developing Maturity, real progress but uneven, with some areas dragging the rest down. 29 to 36 is Established Foundations, the basics are genuinely solid, though it still takes ongoing work. 37 to 40 is an AI-Compatible Enterprise, where structure and culture are both actually ready, not just the ambition.

3. Write down what success looks like, before any code gets written.

I also still need to answer a simple question properly. What does success actually look like here? Not a slogan, an actual sentence I could write down and show someone. For this client, that means describing what “someone on the team can find the right answer in seconds, instead of digging through folders” would actually look like, and how we’d know it’s true. If I can’t write that sentence honestly, I don’t have a use case yet. I have a technology looking for a reason to exist.

Part of writing that sentence honestly means sitting down with the actual team that would use this, and collecting real questions, the kind they’d genuinely type on a normal working day, not the polished examples I’d come up with on my own. That list becomes the first, roughest version of a test set, long before there’s any system to test it against.

4. Get the right small group in the room.

There’s also a small group that needs to exist here, people from a few different departments, because someone has to decide what data is fair to use, and who’s responsible if something goes wrong. I’m still not fully happy with how this part is going. It feels more like following a template than solving our specific problem, and I want to come back to that in a later post.

5. Pick two or three small pilots, not one big bet.

Once success is written down and the right group is in the room, it’s a good time to ask what we actually build first. The safe answer is picking two or three small candidates instead of one big bet. Small enough to move fast. Big enough that a result would actually matter. Each one with a clear way to know when to stop, using the same definition of “success” from step 3, not a new one invented on the spot. The honest version isn’t glamorous. Mostly it means writing down, in advance, what “done” and “worked” mean, so nobody argues about it six weeks later.

6. Build or buy: the RAG platform decision

None of this was about RAG specifically yet. Once RAG is actually the technology on the table, the same care needs to go one step further. The final boss is the question. Build the pipeline yourself, or buy a managed platform?

Building it yourself gives you control over every piece, the vector database, the embedding model, the language model. That control matters if you’re in an environment cut off from the public internet for security reasons, or if you need something tuned very specifically to your own use case. But it’s expensive in a way that’s easy to underestimate. Ofer Mendelevitch and Forrest Sheng Bao note in Hands-On RAG for Production that

initial cost estimates for do-it-yourself RAG are notoriously unreliable, with real production costs often ending up 3 to 5 times higher than planned.

There’s also the risk of every team quietly building its own pipeline, its own vector store, its own access rules, until nobody in IT can say which of several internal tools is breaking a GDPR rule.

Buying a managed platform flips those trade-offs. Someone else runs the infrastructure and handles updates, so you move faster and spend less time on things that aren’t your actual product. Platforms also tend to offer one consistent way of enforcing security and governance, instead of five different teams inventing their own rules. Costs are usually bundled, sometimes literally sold as a block of “credits” covering API calls, storage, and compute together, which makes budgeting more predictable. The real trade-off is that you depend on that vendor, and switching away later isn’t free.

Let’s break it down

That’s the real build versus buy question here. Not whether you can build a RAG demo, almost anyone can now, but whether several teams each doing that on their own actually costs less than one shared, well-managed platform. I haven’t picked a side yet for this client. It’s less a yes-or-no question and more a question of how much control is actually worth paying for, and I don’t have a final answer on that yet.

One thing I’m not covering here on purpose. It’s how I’ll actually check whether any of this worked once it’s built. That comes later, once there’s something real to measure.

None of this is the exciting part of RAG. It’s not about chunking strategy or which embedding model to pick. But it decides whether the technical work ahead solves a real problem, or just becomes an expensive way to find out, too late, that the organization wasn’t ready yet.