How We Run Discovery for AI Initiatives

The short version of our discovery process

The shape of the sprint depends on the client and the use case, but the logic is fairly consistent.

We start by clarifying the business pressure behind the idea. Why this initiative, why now, and what would make it valuable enough to justify the investment? From there, we narrow the use case into a specific workflow or decision. “We want an AI assistant” isn’t a project yet. “We want to reduce triage time for this class of inbound request by extracting key fields, checking them against policy rules, and preparing a draft response for human review” is much closer.

Once the use case is clear enough, we map the current workflow. We want to understand where the work happens today, where the pain is, where AI might enter, who acts on the output, and what breaks if the output is wrong. Then we review the data and systems. This is not just asking whether data exists. We want to look at representative samples, schemas, access paths, ownership, permission boundaries, quality issues, labeling assumptions, and how the data changes over time.

After that, we assess feasibility and risk. Can the data support the use case? How should the output be evaluated? What error rate can the business tolerate? What needs human review? What are the cost, latency, security, compliance, integration, and maintenance constraints?

Only then do we sketch solution options. Sometimes that means a generative AI application. Sometimes it means retrieval, deterministic rules, a workflow redesign, a human-in-the-loop system, classical ML, or a data-readiness phase before any product work begins.

The output should be simple enough to act on: a sharper problem statement, a feasibility view, data readiness notes, a solution sketch, an evaluation approach, a delivery roadmap, key risks, team requirements, and a recommendation.

In other words: what to build first, what not to build yet, and what assumptions could still break the project.

AI Discovery Readiness Checklist

Before an AI initiative moves into implementation, these are the dimensions we want to see clearly enough to make a responsible decision.

Dimension

Key question

Business objective

What measurable outcome should improve?

Workflow fit

Where will AI enter the process, and what changes after it does?

User clarity

Who uses it, who trusts it, and who is accountable?

Data readiness

What data exists, who owns it, how reliable is it, and can we use it?

Evaluation plan

How will we know whether the system is good enough?

Risk profile

What happens when the system is wrong, uncertain, slow, or unavailable?

Integration path

How does this connect to current tools, permissions, and processes?

Delivery ownership

Who maintains, monitors, and improves it after launch?

This checklist is intentionally practical. A founder, CTO, product lead, or innovation team should be able to look at it and quickly see where the initiative is strong, where it is weak, and where the team is relying on assumptions.

It also reflects how we think at Eventum. We are not treating AI delivery as model selection. We are looking at the business case, the workflow, the data, the risk, the evaluation plan, and the operating model around the system.

If too many of these answers are unclear, the next step is probably not implementation. It is better discovery, narrower scoping, or data readiness work.

Why discovery has to be different for AI

Traditional product discovery usually focuses on users, requirements, priorities, integrations, and delivery constraints. AI discovery includes all of that, but it adds a second layer of uncertainty: whether the system can behave reliably enough under real-world conditions.

With conventional software, once the logic is defined, you can usually reason directly about what the system will do. With AI, especially generative AI, the behavior is less deterministic. The same class of input can produce different outputs. Edge cases are harder to enumerate. A system can work well on a carefully chosen demo set and still degrade when exposed to messy production data.

That is why we care so much about evaluation. “Good enough” cannot be a feeling. In some projects, a reviewed sample set and clear acceptance criteria are enough to get started. In others, we may need a labeled benchmark, false positive and false negative analysis, confidence thresholds, human scoring, regression tests, and production monitoring.

The goal is not to over-engineer evaluation before anything has been built. The goal is to avoid building a system whose quality can only be judged by anecdotes.

Security and compliance also need to come in early when the workflow is sensitive. Data retention, access control, provider constraints, auditability, permission boundaries, and compliance exposure are not implementation details to clean up later. They can determine whether the use case is viable at all.

This is one reason demos are so misleading. A demo proves that something can work under selected conditions. It does not prove that the system can be trusted, integrated, monitored, secured, maintained, and improved inside the business.

The questions that expose risk fastest

Some discovery questions are useful because they cut through vague enthusiasm quickly.

“What happens when the system is wrong?” is usually the most important one. It forces the team to separate harmless errors from expensive ones, reversible mistakes from irreversible ones, and low-risk recommendations from high-risk automation.

“Who trusts this output enough to act on it?” reveals whether adoption risk may be higher than model risk. A system can be technically strong and still fail because the user does not trust it, does not understand it, or does not want the workflow to change.

“Where does the ground truth come from?” exposes whether evaluation is possible or whether the team is going to argue from anecdotes.

“Who will maintain this six months from now?” tells us whether we are building a product, an experiment, or a future orphan.

And one of my favorites: “Are we automating the work, or helping a human make a better decision?” A lot of teams say “automation” when what they really need is decision support. Those are different products. They have different interfaces, risks, review processes, and evaluation standards.

Discovery is where that distinction should be made, not after a prototype has already shaped everyone’s expectations.

Model choice comes later

The most common wrong starting point is model-first thinking.

A client has read about a model, seen a competitor announcement, watched an internal demo, or been told by a vendor that a platform can solve their problem. So the conversation starts with, “Should we use this model?” or “Can we build an agent for this?”

I understand why that happens. The technology is visible. The business problem is messier. The model gives everyone something concrete to talk about.

But it is still backwards.

Model choice is downstream of problem framing. The better first question is: what decision, workflow, or outcome are we trying to improve?

Once we know that, model choice becomes much easier and much less ideological. If the use case is low-risk, text-heavy, and tolerant of variability, maybe a generative model is appropriate. If the task is structured, repetitive, and safety-critical, maybe rules or classical automation are better. If latency, cost, or data sensitivity matter more than flexibility, that changes the answer again.

We are not anti-model. We are anti-premature model selection.

A more complex system is not more advanced if it creates more uncertainty than the workflow can absorb.

Discovery decides what the first build should test

Discovery is not a substitute for building. It’s how we decide what the first meaningful build should test.

If the main uncertainty is user adoption, the first step may be a workflow prototype. If it’s data quality and model reliability, the first step is data inspection and evaluation sets with proper experimental protocols, respectively. If we’re concerned with operational risk instead, the first step is a review, escalation, and monitoring design.

If a team builds a prototype because they are unsure, but the prototype isn’t actually testing the thing they are unsure about. It’s proving that a model can produce plausible output. This doesn’t prove that the data is reliable, that users will trust the system, or that the workflow will be resilient at scale.

At Eventum, we try to be direct about this: We’re not trying to prove that AI can do something. In most cases, it can do something. We are trying to prove that this system should exist.

A common pattern: document automation

Document automation is one of the clearest examples.

A client comes in saying, “We need AI to extract information from documents.” On the surface, that sounds like a model problem: OCR if needed, an LLM, some prompting, a validation layer, and structured data pushed downstream.

But when you slow it down, the real questions are not model questions. What documents are we talking about? Are they consistent templates or highly variable? Which fields actually matter? Which fields are deterministic? Which require interpretation? What happens when a value is missing? Who reviews uncertain outputs? What is the cost of extracting the wrong value? Is there ground truth we can use for evaluation?

In many cases, only part of the workflow really needs AI. Some fields can be parsed deterministically, some can be handled with rules, and some could be redesigned out of the workflow entirely.

The model becomes one component in a larger system, not the center of the universe.

I have seen the same before-and-after pattern with internal assistants. The first version is usually vague: “We want an AI assistant to help our operations team handle incoming requests”. After discovery, that can become something much sharper: “We want to reduce triage time for one class of inbound requests by extracting relevant fields, checking them against known policy rules, suggesting a category, and preparing a draft response for human review”.

That is a different project. It has boundaries: one request class, specific fields, known policy checks, a human review step, and a measurable outcome.

That is what good discovery should create.

Go / No-Go / Re-Scope Decision Matrix

Discovery should not always end with “build this”. The recommendation depends on what we find. Here’s a simple decision matrix we use internally for client recommendations:

Recommendation

When it applies

Build

Clear business value, usable data, manageable risk, and a realistic path to deployment.

Prototype first

Promising use case, but uncertain data, UX, model performance, integration, or user adoption.

Re-scope

The idea is directionally useful but too broad, risky, or expensive in its current form.

Do not build yet

The problem is unclear, data is missing, ownership is weak, or the proposed AI layer does not improve the workflow.

Use simpler automation

The task can be solved more safely with rules, workflow redesign, search, analytics, or classical ML.

The last category matters more than people think. If the workflow is deterministic, use deterministic logic. If the user needs known information from controlled documents, search or retrieval may be enough. If the task is prediction on structured historical data, classical ML may be better than generative AI. If the process itself is broken, workflow redesign may create more value than AI.

Applying AI (or some pre-determined solution regardless of project parameters) is not our goal. It’s applying it in an efficient manner to guarantee better business performance.

What happens after discovery

A useful discovery output allows the client to choose what to do next with clarity and it does so by making the next steps unambiguous: build the first slice, prototype a narrower workflow, do the data-readiness work before product development, choose a simpler non-generative approach, or pause because the idea is not ready yet.

That last answer should not be treated as failure. A failed AI initiative does not just waste money. It makes the organization more skeptical of the next AI initiative, including the good ones. Avoiding a bad build is one of the most valuable outcomes discovery can produce.

Discovery also shapes the implementation team. A company may start with “we need an AI engineer,” but after discovery, the bottleneck may be data engineering, backend integration, MLOps, UX, compliance, or domain expertise. The right team depends on the shape of the problem, not the job title someone guessed at the beginning.

That is why we see discovery as part of delivery. It affects architecture, roadmap, evaluation, staffing, risk, and go/no-go decisions.

Final thought

Enthusiasm is cheap in AI right now, but judgment is not.

Good AI work starts with problem framing, data reality, workflow understanding, evaluation discipline, and a credible path to production. The model matters, but it is not the whole system; often it is not even the hardest part.

The goal is not to produce a prototype that impresses people in a meeting. The goal is to build something the business can actually use, trust, maintain, and improve.

If you are considering an AI initiative and are not sure whether to build, prototype, re-scope, or pause, Eventum can help you run a focused discovery sprint. We turn vague AI ambition into a practical delivery plan: the first build, the risks, the data requirements, the team shape, and the decision criteria. And if the idea is not ready, we will tell you that before you spend months proving it the expensive way.