- Notifications
You must be signed in to change notification settings - Fork4
A universal metrics layer. Compatible with definitions in LookML, MetricFlow, Cube with DuckDB, Snowflake, Clickhouse, Bigquery & more!
License
sidequery/sidemantic
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
SQL-first semantic layer for consistent metrics across your data stack. Import from Cube, dbt MetricFlow, LookML, Hex, Rill, Superset, and Omni. Supports DuckDB, MotherDuck, PostgreSQL, BigQuery, Snowflake, ClickHouse, Databricks, and Spark SQL.
- SQL query interface: Write familiar SQL that gets rewritten to use semantic layer
- Automatic joins: Define relationships, joins happen automatically via graph traversal
- Multi-format adapters: Import from 8 semantic layer formats (Cube, dbt, Looker, Hex, and more)
- SQLGlot-powered: Dialect-agnostic SQL generation with transpilation support
- Type-safe: Pydantic models with validation
- Pre-aggregations: Automatic query routing to materialized rollups
- Predicate pushdown: Filters pushed into CTEs for improved performance
- Segments: Reusable named filters with template placeholders
- Metric-level filters: Auto-applied filters for consistent business logic
- Jinja2 templating: Conditional logic and loops in SQL
- Inheritance: Extend models and metrics
- Hierarchies: Parent/child dimensions with drill-down API
- Relative dates: Natural language like "last 7 days", "this month"
- Ungrouped queries: Raw row access without aggregation
- Multi-hop joins: Automatic 2+ hop join discovery
- Aggregations: sum, avg, count, count_distinct, min, max
- Ratios: revenue / order_count
- Derived formulas: (revenue - cost) / revenue
- Cumulative: running totals, rolling windows
- Time comparisons: YoY, MoM, WoW with LAG window functions
- Conversion funnels: signup → purchase rate
Import semantic models from:
- Sidemantic (native)
- Cube
- MetricFlow (dbt)
- LookML (Looker)
- Hex
- Rill
- Superset (Apache)
- Omni
See theAdapter Compatibility section for detailed feature support.
Sidemantic includes powerful CLI tools for working with your semantic layer:
Interactive workbench for exploring and querying your semantic layer:
# Try the demo (no setup required!)uvx sidemantic workbench --demo# Or with your own modelssidemantic workbench semantic_models/
Features:
- Tree browser with hover tooltips showing full metadata
- Tabbed SQL editor with syntax highlighting and 4 example queries
- Table and chart views with automatic axis selection for time-series
- Chart types: Bar, Line, and Scatter plots
- Keyboard shortcuts: Ctrl+R to run, Ctrl+C to quit
- Demo mode: Try it instantly with
--demoflag (includes sample data from multiple formats)
Execute SQL queries from the command line and get CSV output:
# Query to stdoutsidemantic query examples/multi_format_demo/ --sql"SELECT orders.total_revenue, customers.region FROM orders"# Query to filesidemantic query examples/multi_format_demo/ -q"SELECT orders.total_revenue FROM orders" -o results.csv# Pipe to other toolssidemantic query examples/multi_format_demo/ -q"SELECT * FROM orders"| head -5
Perfect for:
- Shell scripts and automation
- Piping to other tools (jq, csvkit, etc.)
- Generating reports
- CI/CD workflows
Expose your semantic layer over the PostgreSQL wire protocol:
# Start server (demo mode)sidemantic serve --demo# Start with your modelssidemantic serve semantic_models/ --port 5433# With authenticationsidemantic serve semantic_models/ --username admin --password secret
Connect with any PostgreSQL client:
psql -h 127.0.0.1 -p 5433 -U admin -d sidemantic
Note: Requirespip install sidemantic[serve]
Perfect for:
- BI tools (Tableau, Power BI, Looker, Metabase)
- SQL clients (DBeaver, DataGrip, pgAdmin)
- Python libraries (psycopg2, SQLAlchemy)
- Any PostgreSQL-compatible tool
# Validate all definitionssidemantic validate semantic_models/# Quick infosidemantic info semantic_models/# MCP server for AI integrationsidemantic mcp-serve semantic_models/
Sidemantic supportsthree definition syntaxes: YAML, SQL, and Python. Choose your preference!
YAML:
# semantic_layer.ymlmodels: -name:orderstable:ordersprimary_key:order_idrelationships: -name:customertype:many_to_oneforeign_key:customer_iddimensions: -name:statustype:categoricalsql:status -name:order_datetype:timesql:created_atgranularity:daymetrics: -name:revenueagg:sumsql:amount -name:order_countagg:countmetrics: -name:total_revenuesql:orders.revenue
SQL:
-- semantic_layer.sqlMODEL (name orders, table orders, primary_key order_id);RELATIONSHIP (name customer, type many_to_one, foreign_key customer_id);DIMENSION (name status, type categorical, sql status);DIMENSION (name order_date, typetime, sql created_at, granularity day);METRIC (name revenue, agg sum, sql amount);METRIC (name order_count, agg count);
Python:
fromsidemanticimportSemanticLayer,Model,Dimension,Metric,Relationshiplayer=SemanticLayer()orders=Model(name="orders",table="orders",primary_key="order_id",relationships=[Relationship(name="customer",type="many_to_one",foreign_key="customer_id")],dimensions=[Dimension(name="status",type="categorical",sql="status"),Dimension(name="order_date",type="time",sql="created_at",granularity="day"), ],metrics=[Metric(name="revenue",agg="sum",sql="amount"),Metric(name="order_count",agg="count"), ])layer.add_model(orders)
fromsidemanticimportSemanticLayer# Load semantic layerlayer=SemanticLayer.from_yaml("semantic_layer.yml")# Query with familiar SQL - automatically rewrittenresult=layer.sql(""" SELECT revenue, status FROM orders WHERE status = 'completed'""")df=result.fetchdf()
Alternative: Python API
fromsidemanticimportSemanticLayer,Model,Metric,Dimension,Relationshiplayer=SemanticLayer()orders=Model(name="orders",table="orders",primary_key="order_id",relationships=[Relationship(name="customer",type="many_to_one",foreign_key="customer_id") ],dimensions=[Dimension(name="status",type="categorical",sql="status"),Dimension(name="order_date",type="time",sql="created_at",granularity="day"), ],metrics=[Metric(name="revenue",agg="sum",sql="amount"),Metric(name="order_count",agg="count"), ])layer.add_model(orders)# Programmatic queryresult=layer.query(metrics=["orders.revenue"],dimensions=["orders.status"],filters=["orders.status = 'completed'"])df=result.fetchdf()
Generate JSON Schema for autocomplete in VS Code, IntelliJ, etc:
uv run python -m sidemantic.schema
Add to your YAML files:
# yaml-language-server: $schema=./sidemantic-schema.jsonThe easiest way to load semantic models from any format:
fromsidemanticimportSemanticLayer,load_from_directory# Point at a directory with mixed formats (Cube, LookML, Hex, MetricFlow, etc.)layer=SemanticLayer(connection="duckdb:///data.db")load_from_directory(layer,"semantic_models/")# That's it! Automatically:# - Discovers all semantic layer files# - Detects format (Cube, Hex, LookML, MetricFlow, Sidemantic)# - Parses with the right adapter# - Infers relationships from foreign key naming (customer_id -> customers)# - Ready to query!result=layer.query(metrics=["orders.revenue"],dimensions=["customers.region"])
For more control, you can use adapters directly:
fromsidemantic.adapters.cubeimportCubeAdapterfromsidemantic.adapters.metricflowimportMetricFlowAdapterfromsidemantic.adapters.sidemanticimportSidemanticAdapter# From Cubecube_adapter=CubeAdapter()graph=cube_adapter.parse("cube_schema.yml")# From MetricFlow (dbt)mf_adapter=MetricFlowAdapter()graph=mf_adapter.parse("semantic_models.yml")# From native Sidemanticnative_adapter=SidemanticAdapter()graph=native_adapter.parse("semantic_layer.yml")
Define ratios, formulas, cumulative metrics with automatic dependency detection:
models: -name:orderstable:ordersprimary_key:order_idmetrics:# Model-level aggregations -name:revenueagg:sumsql:amount -name:completed_revenueagg:sumsql:amountfilters:["status = 'completed'"]# Graph-level metricsmetrics:# Simple reference (dependencies auto-detected) -name:total_revenuesql:orders.revenue# Ratio -name:conversion_ratetype:rationumerator:orders.completed_revenuedenominator:orders.revenue# Derived (dependencies auto-detected from formula!) -name:profit_margintype:derivedsql:"(revenue - cost) / revenue"# Cumulative -name:running_totaltype:cumulativesql:orders.revenuewindow:"7 days"
Python alternative
Metric(name="total_revenue",sql="orders.revenue")Metric(name="conversion_rate",type="ratio",numerator="orders.completed_revenue",denominator="orders.revenue")Metric(name="profit_margin",type="derived",sql="(revenue - cost) / revenue")Metric(name="running_total",type="cumulative",sql="orders.revenue",window="7 days")
Define relationships once, query across models:
models: -name:orderstable:ordersprimary_key:order_idrelationships: -name:customertype:many_to_oneforeign_key:customer_id -name:customerstable:customersprimary_key:customer_idrelationships: -name:regiontype:many_to_oneforeign_key:region_id
Query spans 2 hops automatically:
# Automatically joins orders -> customers -> regionsresult=layer.sql(""" SELECT orders.revenue, regions.region_name FROM orders""")
- many_to_one: Many records in THIS table → one record in OTHER table (e.g., orders → customer)
- one_to_many: One record in THIS table → many records in OTHER table (e.g., customer → orders)
- one_to_one: One record in THIS table → one record in OTHER table (e.g., order → invoice)
models: -name:orderssegments: -name:completedsql:"{model}.status = 'completed'"description:"Only completed orders" -name:high_valuesql:"{model}.amount > 100"# Use in querieslayer.compile(metrics=["orders.revenue"], segments=["orders.completed"])
metrics: -name:completed_revenueagg:sumsql:amountfilters:["{model}.status = 'completed'"]# Auto-applied!
metrics: -name:taxed_revenueagg:sumsql:"{% if include_tax %}amount * 1.1{% else %}amount{% endif %}"# Use with parameterslayer.compile(metrics=["orders.taxed_revenue"], parameters={"include_tax":True})
models: -name:base_salestable:salesdimensions:[...] -name:filtered_salesextends:base_sales# Inherits all dimensions!segments:[...]
# Define hierarchyDimension(name="country",type="categorical")Dimension(name="state",type="categorical",parent="country")Dimension(name="city",type="categorical",parent="state")# Navigate hierarchymodel.get_hierarchy_path("city")# ['country', 'state', 'city']model.get_drill_down("country")# 'state'model.get_drill_up("city")# 'state'
# Natural language date filterslayer.compile(metrics=["orders.revenue"],filters=["orders_cte.created_at >= 'last 7 days'"])# Auto-converts to: created_at >= CURRENT_DATE - 7# Supports: "last N days/weeks/months", "this/last/next month/quarter/year", "today", etc.
# Get raw rows without aggregation (for detail views)sql=layer.compile(metrics=["orders.revenue"],dimensions=["orders.customer_id"],ungrouped=True# Returns raw rows)
| Format | Import | Notes |
|---|---|---|
| Sidemantic (native) | ✅ | Full feature support |
| Cube | ✅ | No native segments |
| MetricFlow (dbt) | ✅ | No native segments or hierarchies |
| LookML (Looker) | ✅ | Liquid templating (not Jinja) |
| Hex | ✅ | No segments or cross-model derived metrics |
| Rill | ✅ | No relationships, segments, or cross-model metrics; single-model only |
| Superset (Apache) | ✅ | No relationships in datasets |
| Omni | ✅ | Relationships in separate model file |
This table shows which Sidemantic features are supported when importing from other formats:
| Feature | Sidemantic | Cube | MetricFlow | LookML | Hex | Rill | Superset | Omni | Notes |
|---|---|---|---|---|---|---|---|---|---|
| Models | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | All formats support models/tables |
| Dimensions | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | All formats support dimensions |
| Simple Metrics | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | All formats support sum, count, avg, min, max |
| Time Dimensions | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | All formats support time dimensions with granularity |
| Relationships | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | Rill/Superset: single-model only; Omni: in model file |
| Derived Metrics | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | All formats support calculated metrics |
| Metric Filters | ✅ | ✅ | ❌ | ✅ | ✅ | ❌ | ✅ | Rill has basic support; Superset lacks filters | |
| Ratio Metrics | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | Rill/Superset don't have native ratio metric type |
| Segments | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | Only Cube and LookML have native segment support |
| Cumulative Metrics | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | Cube has rolling_window; MetricFlow has cumulative; others lack native support |
| Time Comparison | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | Only MetricFlow has native time comparison metrics |
| Jinja Templates | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | LookML uses Liquid templating | |
| Hierarchies | ✅ | ❌ | ❌ | ❌ | ❌ | Cube/LookML/Omni: via drill_fields | |||
| Inheritance | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | Only LookML has native extends support |
| Metadata Fields | ✅ | ✅ | ✅ | Label and description support varies by format | |||||
| Parameters | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | Sidemantic-only feature |
| Ungrouped Queries | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | Sidemantic-only feature |
Legend:
- ✅ Full support - feature fully supported on import
⚠️ Partial support - feature works with limitations- ❌ Not supported - feature not available in source format
Sidemantic supports multiple databases:
| Database | Status | Installation |
|---|---|---|
| DuckDB | ✅ | Built-in (default) |
| MotherDuck | ✅ | Built-in (cloud DuckDB) |
| PostgreSQL | ✅ | pip install sidemantic[postgres] |
| BigQuery | ✅ | pip install sidemantic[bigquery] |
| Snowflake | ✅ | pip install sidemantic[snowflake] |
| ClickHouse | ✅ | pip install sidemantic[clickhouse] |
| Databricks | ✅ | pip install sidemantic[databricks] |
| Spark SQL | ✅ | pip install sidemantic[spark] |
Connection examples:
# DuckDB (default)connection:duckdb:///data.duckdb# MotherDuck (cloud DuckDB - requires MOTHERDUCK_TOKEN env var)connection:duckdb://md:database_name# PostgreSQLconnection:postgres://user:pass@localhost:5432/analytics# BigQueryconnection:bigquery://project-id/dataset-id# Snowflakeconnection:snowflake://user:pass@account/database/schema?warehouse=wh# ClickHouseconnection:clickhouse://user:pass@localhost:8123/default# Databricksconnection:databricks://token@server/http-path?catalog=main# Spark SQLconnection:spark://localhost:10000/default
See thedocumentation for complete connection string formats and features.
- SQL query interface with automatic rewriting
- Core semantic layer with SQLGlot generation
- Relationship-based automatic joins (many_to_one, one_to_many, one_to_one)
- Multi-hop join discovery
- Derived metrics with automatic dependency detection
- Cumulative metrics (running totals, rolling windows)
- Conversion funnel metrics
- Time comparison metrics (YoY, MoM, WoW)
- Segments (reusable filters)
- Metric-level filters
- Jinja2 templating
- Model and metric inheritance
- Hierarchies with drill-down API
- Relative date parsing
- Ungrouped queries (raw row access)
- Metadata fields (format, drill_fields, non-additivity, defaults)
- Native YAML format
- Adapters for 8 semantic layer formats (Cube, MetricFlow, LookML, Hex, Rill, Superset, Omni)
- DuckDB integration
- Pre-aggregations with automatic query routing
- Predicate pushdown with SQLGlot parsing
- PostgreSQL wire protocol server for broader client compatibility
- Pre-aggregation materialization and refresh scheduling
- Additional database engine support (Postgres, MySQL, Snowflake, BigQuery, etc.)
- REST API endpoints for HTTP-based queries
Seeexamples/ directory:
sql_query_example.py- SQL query interface demonstrationbasic_example.py- Core usage patternssidemantic/orders.yml- Native YAML examplecube/orders.yml- Cube format examplemetricflow/semantic_models.yml- MetricFlow format example
Run tests:
uv run pytest -v
About
A universal metrics layer. Compatible with definitions in LookML, MetricFlow, Cube with DuckDB, Snowflake, Clickhouse, Bigquery & more!
Topics
Resources
License
Contributing
Security policy
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Packages0
Uh oh!
There was an error while loading.Please reload this page.