Enterprise data and AI platforms compared

Enterprise data and AI platforms compared (October 2026)

Enterprise AI rollouts stall less on models than on data: whether agents can find the right data, understand what it means, trust its quality, and act on it under governance. This note compares the vendors and tools that make up that data layer, layer by layer, with statuses as of October 2026 (GA = generally available). The detailed notes live in the data platforms folder; BI front-ends are in bi-tools-landscape-2026.

The stack in one picture

Layer	Question it answers	Main players
1. Platform	Where does the data live and run?	Databricks, Snowflake, Microsoft Fabric/OneLake, Google BigQuery, AWS, Oracle, Teradata, Cloudera, ClickHouse, SAP BDC
2. Open formats and catalogs	Can several engines share one copy under one policy?	Apache Iceberg, Delta Lake, Unity Catalog, Polaris, Horizon, Lakekeeper
3. Transformation and semantics	What do the numbers mean, and are they tested?	dbt, SQLMesh, Coalesce, Cube, AtScale, vendor semantic layers
4. Ingestion and orchestration	How does data arrive and stay fresh?	Fivetran, Airbyte, Matillion, Confluent, Informatica, Talend, Airflow, Dagster, Prefect
5. Governance and quality	Who may use it, where did it come from, is it correct?	Purview, Unity Catalog, Atlan, Collibra, Alation, DataHub, OpenMetadata, Monte Carlo, Soda, Great Expectations
6. Context and ontology	What is a “customer” here, and what can an agent do with it?	Palantir Ontology, Databricks Genie Ontology, Microsoft Fabric IQ, knowledge graphs
7. Agents and apps	Who asks, who acts?	Genie One, Snowflake Intelligence/CoWork, Copilot, Foundry, Agent Bricks, Palantir AIP

Layer 1: platforms

	Foundation	Governance	AI building blocks	Business-user entry	MCP / agent hooks	Caveats (status)
Databricks	Lakehouse (Delta, Iceberg), Lakebase Postgres	Unity Catalog; Unity Gateway budgets and guardrails	Agent Bricks (component status not verified), Genie Code, Lakeflow	Genie One (status per Data + AI Summit 2026 recap, secondary: GA; mobile and macOS labels not verified)	Managed MCP; Omnigent harness	Genie Ontology Public Preview; Lakehouse//RT Beta; LTAP announced
Snowflake	Managed warehouse and lake, Iceberg tables, Snowflake Postgres	Horizon Catalog; AI Agent Identity	Cortex AI functions in SQL, Cortex Agents (GA status not verified), Openflow	Snowflake Intelligence / CoWork (GA status not verified)	Snowflake-managed MCP server	Cortex AI Gateway reported as preview (date not verified); other AI feature statuses not verified
Microsoft	OneLake (Delta, Iceberg interop, shortcuts, mirroring)	Entra, Purview	Foundry, Fabric data agents (GA per Microsoft docs read; not re-checked), Copilot in Fabric (status not verified)	Copilot, Power BI	Fabric IQ MCP and OneLake catalog MCP (status not verified)	Fabric IQ and ontology reported as preview (not re-verified)
Google BigQuery	Serverless warehouse, BigLake/Iceberg	Knowledge Catalog (renamed from Dataplex Universal Catalog, 2026-04-10)	Gemini in BigQuery, BigQuery ML and AI functions, Data Engineering Agent	Conversational Analytics (agents in Gemini Enterprise: preview)	Remote BigQuery MCP server	AI feature statuses not verified
AWS	S3 and S3 Tables (Iceberg), Redshift, Glue	Lake Formation, AgentCore Identity and Policy	Bedrock Data Automation, AgentCore, SageMaker Unified Studio	Amazon Quick (evolved from QuickSight)	MCP/A2A in AgentCore	Multiple services to assemble (opinion); status per component not verified
Oracle	Autonomous AI Database, AI Vector Search	Database security model	Select AI	Oracle Analytics	MCP servers (status not verified)	GA dates not verified
Teradata, Cloudera, ClickHouse, SAP BDC	Enterprise warehouse / hybrid data platform / real-time analytics / SAP data cloud	Own catalogs	MCP servers (licence and status not verified)	via partners	see notes	not verified; see each note
Palantir	Not a data lake: Foundry integrates data and Ontology sits on top	Markings, permissions, audit	AIP Logic, Chatbot Studio (formerly Agent Studio), Evals	AIP applications	Reads external lakehouses without copying	Closed platform; GA status of AIP features not stated in docs read

Notes: databricks-data-intelligence-platform, snowflake-ai-data-cloud, microsoft-onelake-and-fabric-data-platform, google-bigquery-data-platform, aws-data-and-ai-platform, oracle-ai-data-platform, teradata-vantage, cloudera-data-platform, clickhouse, sap-business-data-cloud, palantir-foundry-and-aip.

Read-across. In the author’s view (opinion), Databricks and Snowflake are converging feature for feature (catalog, semantic layer, agent builder, business-user agent, transactional Postgres). In the author’s view (opinion), heritage differs: Databricks started data-science and open-format first, Snowflake SQL-first and managed. In the author’s view (opinion), Microsoft sells a broad bundle whose context layer (Fabric IQ) is still in preview. Palantir is a different kind of product: the decision-and-action layer that can sit on any of the others.

Layer 2: open formats and catalogs

• Apache Iceberg: spec v1-v3 (v4 in development), REST catalog; release 1.12.0 on 2026-09-30; adopted by Snowflake, Databricks, AWS S3 Tables, BigQuery and OneLake. Delta Lake: releases 4.4.1 (2026-10-05); UniForm exposes Delta tables as Iceberg. See apache-iceberg, delta-lake, apache-hudi-and-paimon.
• Catalogs make one policy work across engines: Unity Catalog OSS, Apache Polaris (1.8.0 on 2026-09-28), Lakekeeper, Gravitino, Snowflake Horizon. See open-lakehouse-catalogs.
• Engines and local analytics: Trino/Starburst, Dremio (now part of SAP), DuckDB/MotherDuck. See trino-and-starburst, dremio, duckdb-and-motherduck.
• Postgres for agents: Databricks Lakebase and Neon, Snowflake Postgres, Supabase, and the cloud Postgres services, because agents need transactional state next to analytical data. See postgres-for-ai-agents.

Layer 3: transformation and semantic layers

• dbt is a widely adopted transformation tool (adoption share not verified): tested, documented models plus a semantic layer (MetricFlow) that grounds text-to-SQL agents. dbt Labs and Fivetran announced a merger on 2025-10-13, completed on 2026-06-01; dbt v2.0.0 was released on GitHub on 2026-09-14 (GA announcement reported for 2026-09-16 at dbt Summit; that date is from a search lead only, and the 2026-06-01 merger release still listed dbt Core v2.0 as alpha); a dbt MCP server exists. See dbt, dbt-labs, dbt-learning-path.
• SQLMesh (Tobiko Data, acquired by Fivetran on 2025-09-03, contributed to the Linux Foundation on 2026-03-25), Coalesce, Cube, AtScale: see sqlmesh-and-tobiko, coalesce-transform, cube-semantic-layer, atscale.
• Open Semantic Interchange: a 2025 industry initiative now incubating at Apache as Ossie (incubator since 2026-06-22); aims to let semantic definitions move between tools. See open-semantic-interchange.
• Vendor semantic layers (Snowflake semantic views, Databricks metric views, Power BI semantic models, LookML, Fabric IQ) are compared in semantic-layers-compared. Practical point: whichever you pick, agent answer quality tracks how well these definitions are curated.

Layer 4: ingestion, streaming and orchestration

Tool	Role	Notable 2025-26 fact
fivetran	Managed ELT	Merged with dbt Labs (completed 2026-06-01)
airbyte	Open-source ELT	2.0 released 2025-10-15; agents and an MCP server listed
matillion	Cloud ELT	Branding as Maia by Matillion: not verified
confluent-and-kafka	Streaming	IBM deal announced 2025-12-08 (about $11B,$31 per share in cash); completed 2026-03-17 (IBM newsroom)
informatica-idmc	Enterprise integration and data management	Acquired by Salesforce (see informatica)
talend-data-fabric	Integration	Now Qlik Talend (see qlik)
apache-airflow, dagster, prefect	Orchestration	Release line not verified; Prefect is acquiring Dagster Labs (announced 2026-07-13 on Dagster’s blog; open source to be actively developed and Dagster+ continues, about 40 staff join Prefect, founder Nick Schrock departs; closing date and terms not stated)

What changes for AI: unstructured documents and embeddings pipelines, freshness expectations for agents, and pipelines triggered by agents. See data-pipelines-for-ai.

Layer 5: governance, lineage and quality

• Catalogs and governance: microsoft-purview, databricks-unity-catalog, atlan, collibra, alation, informatica, ataccama, and open source datahub and openmetadata (both ship MCP servers). Side-by-side in data-catalogs-compared.
• Quality and observability: monte-carlo-data, soda-data-quality, great-expectations.
• Cost and runtime control is the newest part: Databricks’ Unity AI Gateway (GA per the Data + AI Summit 2026 recap) offers spend budgets, rate limits and MCP server management, Snowflake has a Cortex AI Gateway (preview), AWS pairs AgentCore Policy with Lake Formation, Microsoft uses Purview and Entra. These are the “CFO controls” for token spend.

Layer 6: context and ontology

• Palantir Ontology (in the author’s view the longest-established of these; not verified) offers objects, links, actions that write back to systems, functions, interfaces.
• Databricks Genie Ontology (Public Preview; enabled by default since 2026-08-13, and ontology snippets available to all customers the same day, per the Databricks AI/BI release notes): a context layer that maps an organisation for Genie; read-focused context, not writeback.
• Microsoft Fabric IQ ontology (preview): entity types, relationships, rules bound to OneLake data.
• Knowledge graphs (Neo4j, Stardog, TigerGraph, GraphRAG) are the open alternative. See knowledge-graph-and-ontology-platforms, ontology, graphrag.

Consolidation to watch (2025-26)

Fivetran with dbt Labs (and Tobiko/SQLMesh; Fivetran also became steward of Great Expectations on 2026-05-13); IBM with Confluent; Salesforce with Informatica; Qlik with Talend; SAP with Dremio (announced 2026-05-04, completed 2026-07-06, per SAP News); Prefect with Dagster (announced); Databricks with Neon and Snowflake with Crunchy Data (both for Postgres); ClickHouse with Langfuse; Domo’s platform sold to Progress (domo-platform). Expect vendor roadmaps, licences and brands in layers 3-5 to keep moving; check status before committing.

What an AI rollout needs, by layer

1. Governed, reachable data (layers 1-2): one copy where possible, open formats, one catalog policy.
2. Meaning (layers 3 and 6): tested models, a curated semantic layer, an ontology where agents must act.
3. Trust (layers 4-5): freshness, lineage, quality checks, access control and PII handling.
4. Control (layer 5): per-agent identity, token budgets, audit.
5. Agents last (layer 7): start with read-only Q&A on a certified domain, then add actions.
   See data-readiness-for-enterprise-ai.

Where to start, by situation (judgement)

• Microsoft shop: OneLake plus Fabric data agents (GA) today; plan for Fabric IQ and Foundry as they leave preview.
• Lakehouse or data-science-led: Databricks; accept that Genie Ontology is still Public Preview.
• SQL-led, minimal ops: Snowflake; check the preview status of the specific Cortex features you need.
• Google or AWS first: BigQuery or the AWS stack, with Iceberg as the neutral storage format to keep options open.
• Operational decisions with writeback (supply chain, maintenance, defence): Palantir on top of any of the above; product partnerships are announced with Databricks (2025-03-13, Databricks newsroom) and Microsoft (2024-08-08, Microsoft newsroom: Palantir software and Azure OpenAI in Azure Government and classified clouds).
• Regardless of platform: pick dbt (or the vendor’s semantic layer), a catalog with an MCP server, and a quality tool before scaling agents.

Open items

• Verification 2026-10-07 (spot-check of dated claims only, not the whole note): IBM-Confluent, dbt-Fivetran, Tobiko, SQLMesh/Linux Foundation, Ossie, Prefect-Dagster, Knowledge Catalog rename, Iceberg/Polaris/Delta/Airbyte releases confirmed; not checked: Layer 1 table cells (Snowflake, Microsoft, AWS, Oracle, Teradata, Cloudera, SAP statuses), Genie One mobile/macOS preview labels, Matillion/Maia branding, Airflow 3.3.x, Domo/Progress, ClickHouse-Langfuse, Databricks-Neon and Snowflake-Crunchy deals. No verified: date set for that reason.

• dbt v2 GA date: GitHub shows v2.0.0 on 2026-09-14; the 2026-09-16 announcement date is unconfirmed by a primary page.

• Vendor statuses come from release notes and docs fetched in October 2026; many vendor sites blocked automated fetching, so some GA/preview labels rest on one source. Each linked note lists its own open items.

• Palantir: the Databricks and Microsoft partnerships are confirmed from the vendors’ newsrooms; Palantir’s FY2025 revenue and segments come from its 10-K (see palantir), and its 2026 quarterly results are not quoted here.

• Prefect-Dagster: terms and closing date are not published. IBM-Confluent: confirmed from the IBM newsroom completion release.

• No independent benchmark compares answer quality across these agents; do not infer quality from feature lists.

• Pricing is deliberately absent; see each vendor’s pricing page.

Sources (checked 2026-10-07)

• Confirmed 2026-10-07: https://www.fivetran.com/press/fivetran-dbt-labs-complete-merger-to-create-the-data-infrastructure-for-trusted-ai-agents (completed 2026-06-01, announced 2025-10-13) ; https://www.getdbt.com/blog/dbt-labs-and-fivetran-merge-announcement (2025-10-13) ; https://newsroom.ibm.com/2026-03-17-IBM-Completes-Acquisition-of-Confluent,-Making-Real-Time-Data-the-Engine-of-Enterprise-AI-and-Agents ; https://www.confluent.io/blog/ibm-to-acquire-confluent/ (2025-12-08, $31 per share) ; https://www.fivetran.com/press/fivetran-acquires-tobiko-data-to-power-the-next-generation-of-advanced-ai-ready-data-transformation (2025-09-03) ; https://www.linuxfoundation.org/press (SQLMesh, 2026-03-25) ; https://incubator.apache.org/projects/ossie.html (2026-06-22) ; https://dagster.io/blog/prefect-is-acquiring-dagster (2026-07-13) ; https://docs.cloud.google.com/dataplex/docs/release-notes (Knowledge Catalog, 2026-04-10) ; https://docs.databricks.com/aws/en/ai-bi/release-notes/2026 (Genie Ontology) ; GitHub releases APIs for apache/iceberg, apache/polaris, delta-io/delta, dbt-labs/dbt-core, airbytehq/airbyte.

• Databricks release notes and docs: https://docs.databricks.com/aws/en/ai-bi/release-notes/2026 , https://docs.databricks.com/aws/en/release-notes/product/2026/june , https://docs.databricks.com/aws/en/ai-gateway/

• Palantir docs: https://www.palantir.com/docs/foundry/aip/overview/ , https://www.palantir.com/docs/foundry/ontology/overview/

• Data + AI Summit 2026 recap (secondary): https://www.flexera.com/blog/perspectives/databricks-data-ai-summit-2026-recap-genie-one-ltap-lakehouse-rt-and-every-major-launche/

• dbt Labs and Fivetran: https://www.getdbt.com/blog/dbt-labs-and-fivetran-merge-announcement , https://www.fivetran.com/press/fivetran-dbt-labs-complete-merger-to-create-the-data-infrastructure-for-trusted-ai-agents

• The per-layer facts come from the linked notes, which cite their own primary sources (vendor docs, release notes, GitHub releases, Microsoft Learn).