Databricks Data Intelligence Platform

What it is

Databricks’ lakehouse platform: open-format storage (Delta Lake, Iceberg) with Spark/SQL compute, one governance layer (Unity Catalog), pipelines and orchestration (Lakeflow), a managed Postgres (Lakebase) and an agent platform (Agent Bricks). BI/NL-query surfaces are covered elsewhere: BI and Databricks Genie.

Maker, ownership and history

Vendor note: Databricks. Per Wikipedia (secondary, fetched 2026-10-07): MosaicML acquired June 2023 (reported $1.4B),Tabular2024(reportedover$1B), Neon 2025 (reported around $1B);afundingroundinAugust2026reportedata$190B post-money valuation (see Open items).

Editions/deployment

Runs on AWS, Azure and Google Cloud; serverless and classic compute; Databricks Free Edition exists. Billing is consumption based (DBUs plus cloud cost). Pricing: https://www.databricks.com/product/pricing

Core architecture

• Storage: data stays in customer cloud object storage as Delta Lake (Parquet plus transaction log) or Iceberg tables; managed or external tables.
• Compute: Spark/Photon clusters, SQL warehouses, serverless; Lakeflow (Connect, Spark Declarative Pipelines, Jobs, Designer) for ingestion and ETL.
• Governance: Unity Catalog, three-level namespace catalog.schema.object, access policies, lineage, audit, data classification and quality monitoring, sharing via OpenSharing; enabled by default in workspaces created after 2023-11-08. An open-source Unity Catalog exists (github.com/unitycatalog/unitycatalog).
• Lakebase: fully managed Postgres inside the platform (autoscaling, scale-to-zero, instant branching, Unity Catalog integration); origin is the Neon acquisition, and Lakebase is also reachable via Neon’s own docs.

Role in an enterprise AI rollout

Single copy of governed data for training, RAG context and feature data; Unity Catalog governs tables, models, functions and agent tools; Lakebase gives agents and apps a transactional store next to analytical data; Agent Bricks hosts the agents. See vector embeddings for the retrieval side and the enterprise platform comparison.

AI features as of October 2026 (docs.databricks.com release notes)

• Agent Bricks: “the Databricks agent developer platform” (Agent Runtime for any framework, Databricks Sandbox for code execution, managed memory, MLflow Tracing). CLI in Beta (Sept 2026); code-first docs added Oct 2026.
• Genie family: Genie Code scheduled tasks GA (Sept 2026); Genie Code CLI Beta (Oct 2026); Genie Code builds document-processing pipelines (Sept 2026). Genie spaces: see Databricks Genie.
• Lakeflow Designer: GA June 2026; pipeline unit testing Beta (July 2026); managed tables in pipelines Beta (Sept 2026).
• Lakebase: ai_search over Lakebase synced tables Beta (Sept 2026); storing Postgres changes as Delta with full history is public preview (Lakebase docs).
• Unity Catalog: Python UDFs GA (Sept 2026), JDBC connections GA (Oct 2026), external secrets Beta (Sept 2026), Unity Catalog Skills Beta (Aug 2026), customer-managed keys GA (Apr 2026).
• Data + AI Summit 2026 (June), per Wikipedia (secondary): Lakehouse//RT real-time engine, LTAP (transaction and analytical processing on one data copy), Genie One, Genie ZeroOps, Omnigent (open-source agent meta-harness). GA status of these was not confirmed in the release notes read (Open items).
• Mosaic AI model serving: not re-verified this pass.

Integrations

Iceberg and Delta readers (Fabric OneLake catalog federation lets Unity Catalog in Azure Databricks query OneLake data), dbt, Fivetran-style ingestion, MLflow, Lakebase for apps. Compare: snowflake-ai-data-cloud, microsoft-onelake-and-fabric-data-platform, aws-data-and-ai-platform.

Strengths and weaknesses vs peers (opinion)

• Strong: open formats, one governance plane across data and AI, engineering depth, fast release cadence.
• Weaker: breadth of surface area and naming churn (Genie, Agent Bricks, Lakeflow rebrands) raises learning cost; consumption billing needs cost governance.

Self-learning

• Databricks free on-demand courses (Databricks Fundamentals, AI Agent Fundamentals, Generative AI Fundamentals) and free lab pathways for data engineers, ML engineers, analysts: https://www.databricks.com/learn/training/home (opened 2026-10-07)
• Platform docs: https://docs.databricks.com/aws/en/
• Lakebase: https://docs.databricks.com/aws/en/oltp/
• Unity Catalog: https://docs.databricks.com/aws/en/data-governance/unity-catalog/
• Agent Bricks: https://docs.databricks.com/aws/en/generative-ai/agent-bricks/
• Release notes: https://docs.databricks.com/aws/en/release-notes/product/
• Certification names/prices not verified (Open items).

Sources (fetched 2026-10-07)

• https://docs.databricks.com/aws/en/release-notes/product/
• https://docs.databricks.com/aws/en/oltp/
• https://docs.databricks.com/aws/en/data-governance/unity-catalog/
• https://docs.databricks.com/aws/en/generative-ai/agent-bricks/
• https://www.databricks.com/learn/training/home
• https://en.wikipedia.org/wiki/Databricks (secondary)
• https://learn.microsoft.com/en-us/fabric/fundamentals/whats-new (OneLake federation)

Open items

• Neon deal date and price, the $190B valuation and DAIS 2026 announcements rest on Wikipedia only; no primary press release retrieved (press page returned navigation only).
• Whether LTAP and Lakehouse//RT are GA, preview or announced: unverified.
• Mosaic AI model serving current status, Databricks certification names: not verified.