Article | November 11, 2026
An enterprise architecture blueprint for using Databricks as a governed platform for data, analytics, semantic intelligence and responsible AI
By Arun Khandelwal, Enterprise Data & AI Architect, DXC Technology
The past decade was defined by efforts to modernize the enterprise data estate. The next will be defined by how effectively organizations convert that estate into intelligence. Advances in cloud, analytics and generative AI have opened up new opportunities, but they've also exposed ongoing problems with data, governance, semantic consistency and operational control.
Most enterprises already have the data they need. What they lack is a reliable way to turn it into answers people trust, at the speed decisions are made. Filling that gap demands an architecture that connects data, analytics and AI within a common framework, so every decision and action is grounded in the same trusted commercial context. This article studies how enterprises can move beyond classic data platforms and build an intelligence engine that delivers trusted insights, governed AI and measurable business outcomes.
At 9:07 on Monday morning, a CFO asks, "Why did margin fall in Northern Europe last week, and which customer segments changed?" In a fragmented estate, that question becomes a chain of tickets. Finance debates definitions, analysts export spreadsheets and engineering reconciles sources, and the answer arrives after the decision window has closed. A data lake may hold every raw ingredient and still fail, because nobody can agree which table, join, filter or measure is definitive.
A mature data intelligence platform changes the whole journey. The CFO's identity determines which sources they can see. Certified definitions settle what "margin" and "customer segment" mean. A governed compute service evaluates the question and shows its citations, logic and lineage in the response. If the CFO then asks for an action, such as launching a sales intervention, the system calls an approved tool under explicit policy and, for anything material, waits for human approval. The platform's real promise is traceable decision flow.
Figure 1. The enterprise data platform journey to traceable decision flow
Almost everything about the physical platform changes over time. Open table formats replace closed database structures, elastic compute replaces fixed capacity and batch ETL expands into streaming, federation and sharing. Consumption grows beyond dashboards to include natural-language analysis, models, agents and APIs.
The disciplines that made the warehouse trustworthy stay the same, though. Dimensional modeling, reconciliation, stewardship, service levels and change control all carry on being essential. Automation simply makes weak ownership harder to hide.
The strategic change is from fractured data work to a governed system that can repeatedly turn enterprise context into decisions. Swapping one database for another won't get you there. Whichever platform an enterprise chooses, it becomes the common execution plane for data and AI only when the organization runs it as a product, with its own operating model. Used as a landing zone for lifted-and-shifted extract, transform and load (ETL) jobs, it simply relocates the old problems.
Four mutually reinforcing foundations need to be in place:
Building only the storage layer recreates yesterday's silos on cheaper infrastructure.
On top of those foundations, the strongest architectures standardize four contracts. Each settles a question that would otherwise be renegotiated project by project.
Medallion layers (the raw, refined and business-ready zones often labeled bronze, silver and gold) are useful for managing quality. They don't create an operating model on their own. Enterprise value appears only when every promoted asset has an owner, a contract, a consumer and a service level.
Together, these items define the target state. A user can ask a material business question, get an answer grounded in certified definitions and authorized data, inspect its evidence and lineage and trigger an approved additional action. Meanwhile, the platform team can explain the service's reliability, security and unit cost.
Reaching that state depends less on technology choices than on the following leadership decisions:
Name the executive questions, operational decisions, latency needs and regulatory boundaries before choosing services. Teams that load data first and decide what to ask later usually end up with a costly store of raw data that nobody trusts or uses.
The platform team owns the paved roads (standard, preapproved ways to build and deploy), while domain teams own meaning, quality and adoption.
Proven modeling patterns still make data understandable, whatever the platform underneath.
Agreed definitions of terms such as “margin” or “active customer,” each with an owner and formal sign-off, are what let people and AI tools trust self-service answers.
Raw compute consumption means little to the business. Measure cost per pipeline run, certified query, active consumer, model call or decision service.
Adopt new capabilities by controlled release rings, with exit criteria and a fallback path.
Without citations, evaluation, permission enforcement and human control, an AI answer is only a demo.
Make governed semantics, approved tools, evaluation and human-control patterns shared platform capabilities, built once and reused by every assistant.
Databricks brings these foundations together in one platform. Its lakehouse runs on open table formats, Unity Catalog provides the governance plane for data and AI assets and its semantic and AI services (including Genie for natural-language analysis) work from the same governed definitions.
Still, no credible architect should claim one platform wins every workload. Databricks is most differentiated when open data, large-scale engineering, BI, data science and AI must work over governed assets under one operating model. Other patterns can fit better elsewhere:
Seven questions help decide which pattern should lead. The more “yes” answers, the stronger the case for the lakehouse as the primary platform. Mostly “no” answers point toward a cloud warehouse or coexistence.
Federated coexistence is often the healthiest pattern during a transition. A phased program might keep an existing cloud warehouse for stable dashboards while Databricks becomes the engineering, advanced analytics and AI platform. Federation provides governed query access where moving data is unnecessary, and ingestion is reserved for data that needs performance, history, cross-domain joins or AI reuse. That's a sound design choice as long as boundaries, semantic authority, lineage and exit conditions are documented. It turns harmful only when both platforms independently define the same business truth.
Move data when you need performance, history, reuse or ownership in open tables. Federate when freshness or sovereignty matters more than repeated cross-source computation. Share when another organization or region needs governed access without owning a copy.
In some situations, another platform should lead. A small, SQL-only team on a stable SaaS warehouse may not justify a wider platform operating model. Primary transaction processing belongs on a proven operational database (Databricks' own Lakebase should be assessed case by case). Microsecond decision paths need caches, online feature stores or serving systems placed close to the application. Disconnected sovereignty zones may require an approved local platform that exchanges governed products at the boundary. And if the organization won't fund product owners, stewards, platform engineering and operations, consolidating technology alone won't produce trusted self-service. The goal is a coherent data and AI estate, regardless of which vendors it includes.
Many platform narratives promise universal performance gains and savings. This article deliberately avoids those claims, and four practical realities explain why.
Any vendor or customer results cited here are historical outcomes and shouldn't be read as forecasts for another organization.
Enterprise AI fails less often because of the base model than because the surrounding system lacks trusted context, deterministic tools, permissions, evaluation data and feedback. On Databricks, Unity Catalog supplies semantics and permissions, and Genie, Agent Bricks and custom agents form the interactive layer. AI Search, model services and governed tool connections do the work, while MLflow 3 traces and evaluates each step, and Unity AI Gateway enforces policy, routing and budgets. The architect's job is to assemble these into a decision system that produces evidence, instead of a collection of chat interfaces.
Figure 2. Governed AI architecture
Unity Catalog's semantic layer provides the building blocks. Metric views define reusable measures, so the same KPI logic serves SQL, dashboards and AI. Pages record authoritative business concepts, domains organize data products according to purpose and certification marks the assets the organization endorses.
Genie Ontology (documented as a Public Preview capability at the time of writing) combines that human-modeled layer with context inferred from assets and usage, ranking authority by source, use and freshness while enforcing source permissions. In its June 2026 launch blog, Databricks reported an internal 28-question benchmark in which Genie answered 84.5% of questions correctly on the first attempt, compared with 52.4% for the strongest anonymized general-purpose coding agent. It's an interesting vendor result, though it isn't independent proof or a client guarantee.
Context can materially improve answer quality, but coverage can only be measured against your own questions, definitions, permissions and risk threshold. Build a golden question set (real business questions with verified answers) and score it before widening access.
The human control an AI system needs must follow from what it can do, regardless of the underlying technology.
What the AI does: Summarizes, explains, drafts or queries read-only data.
Required control: The user verifies output before relying on it.
What the AI does: Ranks, predicts or proposes an action.
Required control: Evidence and policy checks, with an accountable person deciding.
What the AI does: Drafts a ticket, plan or transaction.
Required control: A person reviews the full payload before submission.
What the AI does: Executes a bounded, low-impact change.
Required control: Preauthorized scope, rate limits, audit, monitoring and rollback.
What the AI does: Has a financial, legal, employment, safety or irreversible effect.
Required control: Named human approval and independent control (the agent can't authorize itself).
These five tiers also describe maturity, which varies by use case. One organization might run an automated fraud block, a finance recommendation service and a read-only HR assistant at the same time, so a single enterprise "AI maturity score" would hide very different consequences. Before moving any use case up a tier, look for evidence: verified answers for assistants; accuracy and override data for recommendations; and tested approval, recovery and kill-switch controls for anything that acts. Then keep evaluating whenever models, prompts, semantics or tools change.
Databricks explicitly warns that AI-generated responses can be inaccurate. Design the experience so checking evidence is normal behavior, not a disclaimer hidden at the end.
In the future state, high-value decisions and selected actions draw on reusable, governed intelligence capabilities. Business goals set the portfolio, certified context constrains interpretation and every model and tool call passes through enforceable control points. Evaluation and feedback produce evidence, while telemetry connects quality, risk, adoption and cost.
The guiding principle is to separate the pace of innovation from the strength of control. Models, prompts and orchestration will keep changing quickly, so identity, semantics, action boundaries, ownership and release gates must stay durable.
An AI strategy is effective only when you can trace it from a board-level outcome to a specific decision, with a designated owner, defined autonomy, measurable evidence and reusable capabilities. This approach prevents generic goals like "use GenAI" from resulting in disconnected prototypes.
For example, consider an insurer aiming to reduce claims leakage without compromising service. The key decision is: “Which claims do we review before settlement?” The AI provides recommendations within the claims workbench and includes a clear escalation path. Certified semantics, document retrieval, an approved scoring tool and end-to-end tracing ensure repeatability. Metrics such as recommendation validity, override reasons, leakage trends and unit cost help leaders assess safety and effectiveness.
Speed in a modernization program comes from a repeatable path from source system to an adopted, governed product, not from moving the most tables in the first quarter. Executives can manage that path through three investment horizons, each producing a usable capability and earning the right to expand autonomy. Timings are indicative and must follow local readiness and risk.
Within those horizons, delivery runs through eight waves (Figure 4), each ending with evidence, an owner and an explicit go or no-go decision.
Self-service changes roles. Engineers build reusable product patterns, analysts publish governed metrics, stewards manage measurable workflows and platform engineers run the paved roads. Business users gain conversational access and the responsibility to inspect evidence and report errors. Training, office hours, champions and response cycles belong in the architecture budget from day one.
The platform can reduce duplication, speed delivery, make governance and costs more transparent and increase the reuse of analytics and AI. None of that is automatic, so a defensible business case separates hypotheses from evidence.
Start by baselining today's run-rate, people effort, decision lead time and duplicate copies. Tie every benefit to a specific architectural change and an accountable owner (generic percentage savings won't survive scrutiny from finance). Fund the platform foundation and each product wave separately, and release each new round of funding only when the previous wave shows real adoption and the legacy systems it replaces have been switched off. Track leading indicators monthly and realized outcomes quarterly, then stop or redesign any product that isn't being used.
For the board, seven measures show whether the platform is producing trusted decisions at a sustainable cost:
The most expensive failures tend to stem from structural decisions that create duplication, bottlenecks, ambiguous meaning, unowned cost or unsafe AI actions. Each of these belongs on the architecture review checklist:
Enterprise leaders now face a greater challenge than modernizing data platforms: transforming data into trusted intelligence that drives better decisions, faster actions and measurable value. Concentrating exclusively on migration, consolidation or AI experimentation may result in advanced technology without meaningful business impact.
The practical path is clear. Identify critical decisions, define common terms, assign ownership and establish governance. Build reliable semantic foundations prior to scaling AI. Demonstrate value through select high-impact decision processes, then expand successful practices.
Organizations that consistently turn enterprise knowledge into trusted decisions and governed actions will gain a competitive advantage, regardless of the size of their data estate.
This is the shift from managing a data estate to building an intelligence engine.
Arun Khandelwal is an Enterprise Data & AI Architect at DXC Technology. With 22 years of experience in addressing complex business challenges through innovation and strong execution, Arun has a proven track record in re-architecting, redesigning and migrating legacy systems to modern platforms. Connect with Arun on LinkedIn.
