Live Dataframes
The best way to work with live data. Use livedataframes to easily analyze, transform, and visualize real‑time data. For teams creating data-intensive apps at scale.
Why is working with fast data always so slow?
Most query engines are built for processing batch data.
Deephaven’s query engine is purpose-built for processing fast real‑time data.
Micro-batching is inefficient, making real-time data slow and usable only for simple cases.
Deephaven efficiently tracks changes to incrementally calculate results.
Stream processors are only good forETL, not analytics or apps.
Deephaven is good for a wide range of real-time workflows and applications.
Fast data in the browser is hard, and you have to solve that problem yourself.
Deephaven is full-stack, and includes a complete library of Python components designed for real-time.
Deephaven is a new type of query engine.Live dataframes are a powerful new abstraction unique to Deephaven. They are column-oriented, structured tables that process data incrementally based on table changes (deltas) instead of micro-batching. This approach enables millisecond response times for complex data, even under high throughput, while also reducing hardware utilization.
Users define adirected acyclic graph (DAG) to represent a query's execution plan. Dataframes update incrementally, with deltas propagating to downstream nodes both within the process and over the wire, supporting batch, streaming, and hybrid workflows. Users can submit queries through server-side interfaces or by using gRPC-based, idiomatic client APIs in Python, Java, Go, R, C++, or JavaScript.
Live dataframes combine high performance, versatility, and hardware efficiency by leveraging vectorized structures, lazy evaluation, caching, and parallel processing. Deephaven's optimized DAG ensures fast and accurate query execution, even as data changes. Running within the JVM, with sophisticated bridging to Python and an embedded SQL Calcite interface, Deephaven provides numerous patterns for efficiently addressing large, dynamic datasets and complex queries. If your application requires calculating, displaying, or transforming large volumes of rapidly changing data, Deephaven is an excellent choice.
Deephaven is a new type of query engine. Instead of recalculating everything from scratch when data changes, live dataframes update by processing only the changes (deltas). When you make a query, Deephaven builds a map (directed acyclic graph) that guides changed data through your query, while doing the least amount of work along the way. This is good for things that need live data, high speeds, and scale. If you need to calculate, show, or manipulate a lot of data that changes quickly, Deephaven is a great choice.
| Timestamp | Shape | Vertices | Color |
|---|
| Shape | Vertices |
|---|
| Vertices |
|---|
Supported data sources
- Event streams
- Kafka
- Redpanda
- Message queues
- WebSockets
- REST, JSON
- Logs
- Vendor feeds
- Multicast
- Custom
- Batch Data
- Iceberg
- Parquet
- S3
- CSV
- Arrow, Arrow Flight SQL
- ODBC / JDBC / ADBC
- Vendor databases
Deephaven query engine
- Join
- Filter
- Select
- Sort
- Group
- Ungroup
- Aggregate
- Formulas
- Roll-up
- Partition
- Time-series
- As-of join
- Range join
- User-defined functions
- Windows
- Cumulative
- Rolling
- Ring
- Blink
Uses for live dataframes
- Data apps
- Dashboards
- Analytics
- Data grids
- Real-time charting
- Real-time AI
- Downstream consumers
- Server to server
- Pipelines
- Iceberg ETL
- Business use cases
- Front office
- Research
- Simulation
- Risk
- Compliance
- Operations
- Monitoring
Our customers
Used for data-intensive streaming applications. Mostly finance, mostly critical.
- Top 3 bulge bracket bank
- Top 3 largest investment bank
- Top 3 largest hedge fund in the world
- Top 5 crypto exchange
- Top 5 public stock exchange
- Top 5 largest derivative player
- Top 10 multi-manager fund
- Top 10 quant fund
- ...
Community users
- Quants
- Data scientists
- Java, Python data devs
- Smaller companies
Case studies:
Wunderkind case study (PDF)
Run locally
Deephaven isopen-core and can be run locally. To get started, run the following command:
Versatile building blocks for teams that care about real-time data
An easy mental model
Write queries and develop applications as if you're working with static batch tables—but with Deephaven, you seamlessly inherit real-time updates. No extra engineering. No complex workarounds. Whether you're working with Kafka topics or CSV files, the experience is the same—simple and powerful.
A query engine with significant range
Deephaven empowers diverse teams with unmatched versatility. Seamlessly integrate stream and batch data, perform complex analysis, and leverage Python's ecosystem. Scale effortlessly from single-engine to large clusters. Build headless or interactive applications.
APIs you will love
Deephaven's API supports imperative and declarative interfaces in Python, SQL, and Java. Barrage, its wire protocol based on Apache Arrow Flight, enables efficient data exchange with gRPC endpoints. Share tables easily across instances and networks using simple URIs. Polyglot APIs are available in JavaScript, Python, C++, C#, Java, R, and Go.
Engineering for demanding enterprises
Sophisticated teams face challenges with data management, scalability, and application development. Deephaven, battle-tested for over a decade by demanding financial workloads on Wall Street, addresses these challenges with a modular architecture. Integrate only what you need, ensuring seamless interoperability and allowing you to focus on business outcomes.
Plug-and-play UI/UX
Deephaven integrates with Jupyter and VS Code, offering interactive table and plotting widgets. Connect Excel to receive live data. Build and share dynamic dashboards with drag-and-drop tools or the open-sourcedeephaven.ui Python UI framework. Empower users with interactive explorations and intuitive interfaces.
44 engineers,0 Marketing,1 sales person
Focused on making real‑time data easier to use.
Pricing
Community
Free
Available on GitHub- Core Engine
- Deephaven Community License
- Best and only version of the engine.
- APIs for stream ingestion and batch.
- Authorization and authentication hooks.
- “App-mode” for delivering microservices.
- Query engine to power pipelines.
- Integration for Deephaven front-ends.
- Designed to accommodate your engineering.
- Included open-source building blocks.
- Apache 2.0 licensed building blocks:
- BarrageGitHub
- Apache Arrow Flight extension.
- gRPC API.
- Server-to-server and client-server comms.
- jpyGitHub
- Bi-directional Python-Java bridge.
- Embed one in the other, respectful of paradigms.
- Multi-threaded, fast, versatile.
- Web Client UIGitHub
- React-based IDE front-end.
- Best-in-class quadrillion row data grid.
- Data interrogation, REPL, widgets.
- Composable npm packages.
- VS Code ExtensionGitHub
- Speeds local query development.
- Run code on local and remote Deephaven workers.
- Display real-time tables and plots.
- Resource management for applications.
- Deephaven ExpressGitHub
- Real-time wrapper for Plotly Express.
- Accepts live dataframes as source.
- Updating versions of popular graphs.
- Control of the figure and formatting.
- Deephaven.uiGitHub
- Python library for building browser data apps.
- Like Streamlit, Reflex or Dash but fast & real-time.
- Build dashboards & UIs to parameterize queries.
- BarrageGitHub
Enterprise
$200K - 2M+USD/yr
Contact sales- Everything in Community, plus:
- Multi-user, ACL permissions, pivot tables, clustered system for analytics, autonomous applications, auditing, SSO, and UIs at scale.
- Dedicated support and pro services.
- Live source data at scale
- Scale across 1000s of cores and TBs of streams
- Centralized ingestion of external feeds.
- Real-time persistence of live dataframes.
- Fanout at scale.
- Column-oriented random access of live data.
- Partition-, group-, index-management tooling.
- Cataloging, discovery of live and batch.
- Smart solutions for access control.
- Application development and management
- "Persistent queries" to make live apps easy.
- Programmatic and UI interfaces for app-dev.
- Dependency management.
- Central control and dispatching.
- Replicas, sharding, failover.
- System engineering for modern enterprises
- Kubernetes, Podman, Helm deployment.
- Config and schema solutions via ETCD.
- Turnkey integration with auth technologies.
- Best-of-breed real-time & batch access controls.
- Slick proxying via Envoy.
- Tooling for integrations with monitoring services.
- Robust audit infrastructure.
- Gear for multi-environment management.
- Modern SBOM.
- Shareable data apps and dashboards
- Create and share dashboards across your org.
- Permissioned dashboard access.
- Embed widgets and dashboards in your own apps.
- Enterprise only features
- Pivot tables
- Excel sync plugin