Open-source alternative to Palantir Foundry.
Compose best-of-breed OSS tools. Unify them with an Ontology Layer.
Quick Start · Architecture · Roadmap · Full Design Doc
Status: Phase 0 — Infrastructure Skeleton
Architecture design is complete (v0.4.0). Docker Compose infrastructure is ready. The Ontology Layer (custom code) has not been implemented yet.
Palantir Foundry is powerful, but:
- Expensive — enterprise pricing starts in the millions.
- Vendor lock-in — your data and workflows are trapped in Palantir's ecosystem.
- No real free tier — the free version can't connect external DBs, is single-user, and caps at 50 objects.
Individual OSS tools are excellent, but fragmented:
- A metric defined in your BI tool is invisible to your operational app.
- There's no shared semantic model across tools.
- Integration is ad-hoc, brittle, and tool-specific.
OpenFoundry bridges this gap. It deploys best-of-breed open-source tools and connects them through an Ontology Layer — a semantic control plane that gives every tool a shared understanding of your data.
- Compose, Don't Build — Deploy proven OSS. The only custom code is the Ontology Layer + Portal.
- Ontology as the Glue — Every tool speaks the same semantic language.
- Shared Data Plane — All consumers query one SQL surface (
ontology.*via Trino). - Vertical First — Complete one end-to-end workflow before integrating more tools.
- Everything is Replaceable — Swap any tool without breaking the ontology contract.
┌─────────────────────────────────────────────────────────────────┐
│ UNIFIED PORTAL (Backstage) │
│ Ontology Explorer │ Pipeline Monitor │ BI │ Apps │ AI Agent │
└────────────────────────────┬────────────────────────────────────┘
│
┌────────────────────────────▼────────────────────────────────────┐
│ ONTOLOGY LAYER (only custom code) │
│ Registry · Compiler · Query API · Action Engine │
└──┬──────┬───────┬───────┬───────┬────────┬───────┬──────┬──────┘
▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼
Airbyte Dagster dbt Trino Superset Appsmith OpenMD Keycloak
+Iceberg Grafana NocoDB Jupyter OPA
+MinIO Soda Loki Valkey
| Role | Tool | Replaces (Foundry) |
|---|---|---|
| Ingestion | Airbyte | Data Connection |
| Orchestration | Dagster | Scheduling |
| Transformation | dbt | Transforms / Code Repos |
| Storage | Apache Iceberg + MinIO | Foundry Datasets |
| Query Engine | Trino | Internal Query Layer |
| BI Dashboard | Apache Superset | Contour |
| Time-series | Grafana | Quiver |
| Low-code Apps | Appsmith | Workshop |
| Notebook | JupyterHub | Code Workbook |
| Spreadsheet | NocoDB | Fusion |
| Metadata | OpenMetadata | Compass / Data Lineage |
| Data Quality | Soda | Data Health |
| Auth / SSO | Keycloak | Multipass |
| Policy / RBAC | OPA | Markings / RBAC |
| Event Bus | Valkey (Pub/Sub + Streams) | Webhooks & Streaming |
| Observability | Prometheus + Loki + Grafana | Platform Monitoring |
| AI / LLM | LangChain + LangGraph | AIP |
| Portal | Backstage | Carbon / Compass |
| Ontology | Custom (only custom code) | Entire Ontology |
The Ontology Layer is what turns a collection of tools into a platform. It has four components:
- Registry — Define entity types, properties, links, and actions in YAML. Version-controlled with Git.
- Compiler — Reads the registry and generates Trino views, metadata tags, BI datasets, and SDK clients.
- Query API — Translates entity queries into Trino SQL. Supports filtering, pagination, and link traversal.
- Action Engine — The only write path. Executes business actions (Dagster jobs, Airbyte syncs, DB writeback) with idempotency, audit logging, and event publishing.
Define Student once, and it's available everywhere — dashboards, operational apps, portal search, and AI agents — with the same meaning.
Three-tier authorization with a single policy source of truth:
| Layer | Role | Scope |
|---|---|---|
| L1 — Gate | Schema/table-level blocking (Trino + OPA) | Block bronze.*, silver.*. Allow ontology.* only. |
| L2 — View Isolation | PII isolation (view split + service accounts) | ontology.* = no PII. ontology_secure.* = PII included. |
| L3 — Fine-grained | Row/field policies + audit (Query API) | Teacher sees only their classroom's students. |
- Docker & Docker Compose v2
- 4+ CPU cores, 16GB+ RAM
git clone https://github.com/smilebank7/OpenFoundry.git
cd OpenFoundry
cp .env.example .env
# Edit .env to change default passwords# Minimal setup — Lakehouse + Auth + Monitoring
docker compose --profile core --profile observability up -d| Service | URL |
|---|---|
| Trino | http://localhost:8080 |
| Keycloak | http://localhost:8180 |
| MinIO Console | http://localhost:9001 |
| Grafana | http://localhost:3001 |
| Prometheus | http://localhost:9090 |
| OPA | http://localhost:8181 |
| Profile | Services | When to use |
|---|---|---|
core (required) |
PostgreSQL, MinIO, Valkey, Trino, Keycloak, OPA | Always |
observability |
Prometheus, Loki, Grafana, Node Exporter | Always (recommended) |
pipeline |
Dagster (webserver + daemon) | Phase 1+ |
catalog |
OpenMetadata + Elasticsearch | Phase 1+ |
consumer |
Superset | Phase 4+ |
openfoundry |
Ontology API, Portal (not yet built) | Phase 2+ |
Minimum: core = 4 cores / 16GB RAM. Full stack: 8 cores / 32GB RAM.
Phase 0 Phase 1 Phase 2 Phase 3 Phase 4 Phase 5
Lakehouse Pipeline Ontology MVP Actions & Portal & Governance
Spine Golden Path Events Consumers & Polish
─────────────────────────────────────────────────────────────────────────────────────
██████ ░░░░░░░░░░░░ ░░░░░░░░░░░░░ ░░░░░░░░░░░░ ░░░░░░░░░░░ ░░░░░░░░░
WE ARE
HERE
- PostgreSQL, MinIO, Valkey, Trino + Iceberg (JDBC catalog)
- Keycloak realm with OIDC clients and test users
- OPA policy for schema-level access control (L1 Gate)
- Prometheus + Grafana + Loki monitoring stack
- dbt project initialization with Trino profile
- Docker Compose with 6 profiles
- Airbyte: Source DB → MinIO Bronze sync
- Dagster orchestration: Airbyte → dbt → Soda pipeline
- dbt Silver transformations
- Soda data quality checks
- OpenMetadata ingestion
- Iceberg maintenance jobs (snapshot expiry, orphan cleanup, compaction)
- Registry: YAML parsing, JSON Schema validation, cross-entity validation
- Compiler: Trino view DDL generation (
ontology.*/ontology_secure.*split) - Query API: Entity CRUD, filter DSL, cursor pagination, depth-1 link expansion
- SDK Codegen: OpenAPI spec → TypeScript & Python clients
- Action Engine: Idempotency keys, outbox pattern, audit logging
- Valkey Pub/Sub (real-time UI) + Streams (durable integration events)
- Writeback adapters + Airbyte sync-now triggers
- Backstage portal with Ontology Explorer plugin
- Superset dashboards via ontology views
- Appsmith operational apps
- LangChain AI agent prototype
- Fine-grained OPA policies
- Performance optimization
- Kubernetes Helm deployment
OpenFoundry/
├── docker-compose.yml # 6 profiles, ~17 services
├── .env.example # Environment variables template
├── LICENSES.md # All tool licenses & versions
│
├── docs/
│ ├── architecture.md # Full architecture guide (v0.4.0, 744 lines)
│ ├── mvp-spec.md # MVP spec: API contracts, schemas, test plan
│ └── palantir-foundry-feature-map.md # ~70 Foundry features mapped
│
├── trino/etc/ # Trino coordinator + Iceberg catalog config
├── keycloak/ # Realm export (roles, clients, test users)
├── opa/policies/ # L1 Trino gate policy (Rego)
├── prometheus/ # Scrape config
├── grafana/provisioning/ # Datasources (Prometheus + Loki)
├── loki/ # Log aggregation config
├── dbt/ # dbt project (Trino/Iceberg profile)
└── scripts/ # Init scripts (PostgreSQL multi-DB)
| Document | Description |
|---|---|
| Architecture Guide | Full design: tool selection, auth model, deployment, risks (v0.4.0) |
| MVP Specification | API contracts, JSON Schema, DB schemas, test plan |
| Foundry Feature Map | ~70 Palantir Foundry features mapped to OSS alternatives |
| Licenses | All tools with licenses, Docker images, and versions |
This project is in early development. Contributions, feedback, and ideas are welcome.
- Check the Roadmap for current status
- Read the Architecture Guide for design context
- Open an issue to discuss before submitting PRs
Apache License 2.0 — see LICENSE for details.
The Ontology Layer and Portal are original code under this license. Individual OSS tools retain their own licenses — see LICENSES.md.