Timeplus 3.0: The New Era of Real-Time Data Pipelines, Simplified for Massive Scale
- Ting Wang
- 20 hours ago
- 6 min read
50GB/s processing throughput. Zero replication. Zero lag.
Table of Contents:
Opportunities: Our Journey So Far
When we started building Timeplus in 2022, our goal was simple: make real-time data processing simple, fast, and accessible. Three years later, the world has changed faster than anyone imagined. The explosion of real-time analytics and telemetry, and especially the rise of LLMs, Agents and Physical AI, has been making real-time data more pervasive, more critical and operating at an unprecedented scale. Live data context now powers training, inference, and monitoring—running continuously, everywhere.
But with this massive, machine-level scale comes chaos: bursty traffic, unpredictable throughput, noisy or poor quality data, and drifting schemas. Processing must still stay low-latency, long-running, and resilient, while operating securely and efficiently.
Across industries, we’ve seen strong patterns repeat among our customers:
A gaming company computing the real-time feature from millions of player actions per second, updating risk scores, recommendations, and fraud detection in real-time.
A network provider scaling their telemetry pipelines to handle throughput spikes from 5MB/s to 500MB/s, managing 80M+ cardinalities in real-time.
A trading platform processing 2.7B+ transactions, 2B+ accounts and 8B+ rows, adding new blocks every 12s with 10~250ms latency from ingesting, processing, and querying.
Enterprises now face a new imperative for their real-time data pipelines:
Compute fast, not just connect: Pipelines must transform raw streams into high-quality, context-rich data, filtering out noise and avoiding unnecessary costs. They shouldn’t just move data - they should compute, transform and trigger in-pipeline insights in motion.
Scale efficiently and autonomously: Not only scale out when data bursts, also scale back when traffic slows, elastic auto-scaling for unpredictable workloads without over-provisioning.
Simplify pipeline development & operations: Building and operating 100s-1000s pipelines requires declarative (not imperative) definition, CI/CD-ready, and end-to-end observability, giving users full visibility and control in resources, latency, lag and backpressure.
Run close to enterprise data across Edge, Cloud, BYOC or hybrid environments, with full security, compliance, efficiency and sovereignty.
In this new world, bundling everything into a traditional batch-oriented data warehouse is neither realistic nor affordable. These monolithic “big” stacks are breaking down as models, telemetry, and personalization now depend on fresh, continuously aggregated context. Legacy designs were not built for this speed, scale and adaptability.
With this new imperative, Timeplus version 3.0 further evolves from our core strength in low-latency, high-cardinality, ultra-efficient processing in edge and on-prem environments to power a new era of real-time data pipelines, simplified for cloud scale.
Enter Timeplus 3.0: Scaling Real-Time Data Pipelines for Analytics, Telemetry & AI
Our innovation at Timeplus has always been driven by customer feedback and real-world production experience:
1.0 | Ultra-fast, single-binary SQL engine for stateful and stateless computing. |
2.0 | Multi-Raft consensus and MPP clustering for edge and on-prem workloads. |
3.0 | Industry-first Converged MPP + Cloud-Native architecture, scaling elastically and processing 50 ~ 100 GB+/s in a 16 node cluster (8 vCPU / 32 GB memory per node), with full 360-degree observability across BYOC and Cloud environments. |
Timeplus 3.0 is built for a new era of real-time data pipeline workloads for analytics, telemetry, and AI, which demand ultra efficient, scalable and observable systems by design. Powered by a highly efficient vectorized stream processing engine built in modern C++, Timeplus can scale seamlessly from low-latency, zero dependency workloads at the Edge to massive, high-throughput and Cloud-scale production. That marks a major leap beyond the complexity and operational burden of traditional real-time data stacks.
HIGHLIGHT:
Per-Stream Shared-Nothing to Shared-Storage
Timeplus 3.0 introduces an industry-first elastic design that expands from low-latency MPP (shared-nothing) mode to a highly scalable, zero-replication shared-storage model. This new design is unlocking near-infinite scalability with a truly disaggregated model. Timeplus users can configure two modes per different use cases, workloads, even different streams.
Zero-Replication NativeLog
The new Write-Ahead Log supports object storage (e.g., S3) as its primary backend, enabling active-active writes with zero replication overhead and GB/s throughput. Elastic and efficient, with no cross-AZ replication costs or EBS IOPS waste.
CREATE DISK s3_plain_disk disk(
type = 's3_plain',
endpoint = 'http://localhost:9000/disk/shards/',
access_key_id = 'minioadmin',
secret_access_key = 'minioadmin'
);
CREATE STREAM shared_disk_stream(i int, s string)
SETTINGS
shared_disk='s3_plain_disk',
ingest_batch_max_bytes=67108864,
ingest_batch_timeout_ms=200,
fetch_threads=1;Zero-Replication Query State Checkpoint
Materialized Views now support checkpointing query states directly to cloud object storage. Just like Zero Replication NativeLog, this dramatically improves stability, elasticity, and cost-efficiency across the cluster.
Enhanced Cluster Elasticity & Stability
Overall cluster elasticity and stability have been significantly enhanced in this release. Faster recovery, smoother scaling, and more resilient fault tolerance.
HIGHLIGHT:
Declarative Real-Time SQL
Timeplus 3.0 continues to advance its declarative SQL engine to let developers build and operate end-to-end real-time pipelines, entirely in SQL/UDF. From sources, transformation, to sinks, task scheduling and in-pipeline alerting, everything can be expressed declaratively, without external dependence layers.
In-pipeline Alerts
In-pipeline alerts can monitor any streaming data and automatically trigger actions when certain conditions are met. When streaming queries detect events of interest, alerts can trigger notifications via email or Slack, send data to downstream systems like Kafka, or execute Python UDF for any automated response.
Example — Detect new stars on GitHub and send a Slack notification:
CREATE ALERT default.test
BATCH 10 EVENTS WITH TIMEOUT 5s
LIMIT 1 ALERTS PER 15s
CALL alert_action_proton_new_star
AS SELECT actor FROM github_events WHERE repo='timeplus-io/proton' AND type='WatchEvent'
CREATE OR REPLACE FUNCTION alert_action_proton_new_star(actor string) RETURNS string LANGUAGE PYTHON AS $$
import json
import requests
def alert_action_proton_new_star(value):
for i in range(len(value)):
github_id=value[i]
requests.post("https://hooks.slack.com/services/T123/B456/other_id", data=json.dumps({"text": f"New 🌟 for Timeplus Proton from https://github.com/{github_id}"}))
return value
$$
Scheduled Task
A scheduled task runs a historical query at defined intervals and writes results to a target native stream or external systems (e.g., ClickHouse). When combined with UDFs, scheduled tasks can move data between external systems, enrich streams, or generate periodic summaries - complementing Materialized Views, which operate data processing continuously. Scheduled Tasks provide built-in checkpointing and fault recovery, allowing stateful incremental workloads without external schedulers.
Example — Collect node status every 5 seconds:
-- Create a task to collect the node statuses
CREATE TASK refresh_node_states
SCHEDULE 5s
TIMEOUT 2s
INTO node_states
AS
SELECT cluster_id, node_id, node_state FROM system.cluster;Powerful UDF in Python
Timeplus 3.0 natively supports User-Defined Functions (UDFs) in Python, in addition to JavaScript, SQL, and Remote UDFs. Developers can extend pipelines with custom business logic, from real-time AI/ML inference to custom transformations, enrichments, and alert actions. Whether you’re embedding an AI model for predictions or defining your own transformation logic, UDFs make the pipeline truly programmable, without breaking out of SQL.
HIGHLIGHT:
Best Developer & Ops Experience
Brand-new Timeplus Console with a new Light-Mode UI
A completely redesigned, simplified management console with a new brighter Light Mode that gives developers and operators full visibility into their real-time pipelines, powered by built-in system metrics, lineage graphs, materialized view insights, and cluster health monitoring.
Simplified Runtime
Legacy components such as timeplus_web and kv_service have been fully removed, resulting in a leaner, faster, and easier-to-manage runtime with fewer moving parts and lower operational overhead.
Timeplus + dbt
Our new dbt integration combines dbt’s popular transformation and version-control workflows with Timeplus’s ultra-fast real-time pipeline engine, enabling developers to use dbt’s versioning, testing, documentation, and lineage tracking directly on data streaming pipeline.
BYOC
Timeplus 3.0 introduces BYOC mode, keeping your data sovereign, close to your infrastructure, and under your control while simplifying provisioning and system operation. BYOC gives enterprises the flexibility to deploy real-time processing inside their own cloud, without moving or replicating sensitive data. (Currently available with AWS support)
Benefits:
Data sovereignty & compliance: Data never leaves your VPC.
Network efficiency: Reduced egress cost drastically.
Operational control: Integrate with your existing IAM, monitoring, and scaling policies.
Full security isolation: Your keys, your cluster, your boundary.
More details in our docs
Major Upgrade in Community Version - Proton 3.0
Two years after open sourcing Proton, our core engine, we’re thrilled to announce Proton 3.0 - the biggest upgrade yet for the community edition. This release brings full-fledged streaming connectivity, processing and routing capabilities to every developer, with unmatched performance and efficiency in a single binary.
With Proton 3.0, building real-time pipelines is now faster, simpler and more fun than ever, with the same efficiency and performance proven in other large enterprise deployments.
First vectorized streaming SQL engine in modern C++ under Apache 2.0
High-throughput, Low-latency, High-Cardinality
Full streaming processing end-to-end: ETL, join and aggregation, Alert and Task
Native connections with Kafka, Redpanda, Pulsar, ClickHouse, Splunk, Elastic, MongoDB, S3, Apache Iceberg etc.
Native Python/JavaScript UDF/UDAF support
Single binary with zero dependencies
👉 Try the Proton 3.0 Community Edition on GitHub and start building real-time pipelines today!
Roadmap & Vision
Real-time pipelines are now mission-critical infrastructure. At Timeplus our mission is to simplify and scale real-time data processing for every organization, faster, cheaper and more efficient. But our vision goes even further. The future isn’t just moving data, it's about computing: processing, transforming, and acting on data instantly, to power real-time intelligence everywhere.
Explore and try the new Timeplus 3.0!
👉 Timeplus Enterprise 3.0: Download
👉 Timeplus Community (“Proton”) 3.0: Visit GitHub Repo
👉 Timeplus Demos: Visit Demo Center
About Timeplus:
Timeplus is a fast, efficient real-time data pipeline built for analytics, telemetry and AI/ML.
We are the only vectorized streaming SQL platform that unifies stateful and stateless processing, combining both columnar and row-based storage. All in a single, zero-dependency, and efficient binary.
Deploy and scale anywhere: edge, cloud, BYOC, or hybrid environments.
