Skip to content

Siddhi - Cloud Native Stream Processor

Jenkins Build Status GitHub Siddhi Core Release GitHub Release Date GitHub last commit codecov License

Siddhi is a cloud native Streaming and Complex Event Processing engine that understands Streaming SQL queries in order to capture events from diverse data sources, process them, detect complex conditions, and publish output to various endpoints in real time.

Siddhi can run as an embedded Java library, and as a microservice on bare metal, VM, Docker and natively in Kubernetes. It also has a graphical and text editor for building Streaming Data Integration and Streaming Analytics applications.

Distributions

Kubernetes Docker Binary Java Source

And more installation options

Overview

Supported Use Cases

Streaming Data Integration Streaming Data Analytics Alerts & Notifications Adaptive Decision Making
  • Retrieve data from various event sources (Kafka, JMS, HTTP, CDC, etc).
  • Transform events to and from multiple event formats (JSON, XML, Text, Avro, etc).
  • Data preprocessing & cleaning.
  • Join multiple data streams.
  • Integrate streaming data with databases (RDBMS, Cassandra, HBase, Redis, etc) and services.
  • Calculate aggregations over windows such as time, length, and session.
  • Long duration time series aggregations with granularities from seconds to years.
  • Analyze trends (rise, fall, turn, tipple bottom).
  • Realtime predictions with pre trained machine learning models (PMML, Tensorflow).
  • Learn and predict at runtime using online machine learning models.
  • Generate alerts based on thresholds.
  • Correlate data to find missing and erroneous events.
  • Detect temporal event patterns.
  • Detect non-occurrence of events.
  • Publish data to multiple event sinks (Email, messaging systems, services, databases).
  • Static rule processing.
  • Adaptive stateful rule processing.
  • Decision making through synchronous RPC.
  • Query state from tables, windows and aggregations.
  • Static and online machine learning based decision making.

Siddhi is free and open source, released under Apache Software License v2.0.

Why use Siddhi ?

If you are a Siddhi user, we would love to hear more on how you use Siddhi? Please share your experience and feedback via the Siddhi user Google group.

Get Started!

Get started with Siddhi in a few minutes by following the Siddhi Quick Start Guide

Development Environment

Siddhi provides tooling that supports following features to develop and test stream processing applications:

  • Text Query Editor with syntax highlighting and advanced auto completion support.
  • Event Simulator and Debugger to test Siddhi Applications.
  • Graphical Query Editor with drag and drop query building support.

Graphical Query Editor

Text Query Editor

Siddhi Versions

  • Latest Stable Release of Siddhi v5.x.x : v5.0.0 built on Java 8 & 11.
  • Get Siddhi API information here.

Contact us

How to Contribute

Find the detail information on asking questions, providing feedback, reporting issues, building and contributing code on How to contribute? section.

Roadmap

  • Support Kafka
  • Support NATS
  • Siddhi Runner Distribution
  • Siddhi Tooling (Editor)
  • Siddhi Kubernetes CRD
  • Periodic incremental state persistence
  • Support Prometheus for metrics collection
  • Support high available Siddhi deployment with NATS via Kubernetes CRD
  • Support distributed Siddhi deployment with NATS via Kubernetes CRD

Siddhi is joint research project initiated by WSO2 and University of Moratuwa, Sri Lanka.

Top