Skip to content

Siddhi Extensions

Siddhi provides an extension architecture to enhance its functionality by incorporating other libraries and custom logic in a seamless manner. Each extension has a namespace that can be used to identify and specifically access the functionality of the relevant extension.

Following are the supported Siddhi extension types:

Available Extensions

All the Siddhi extensions are released under Apache 2.0 License.

Execution Extensions

Name Description Latest
Tested
Version
execution-string Provides basic string handling capabilities such as concat, length, replace all, etc. 5.0.7
execution-regex Provides basic RegEx execution capabilities. 5.0.5
execution-math Provides useful mathematical functions. 5.0.4
execution-time Provides time related functionality such as getting current time, current date, manipulating/formatting dates, etc. 5.0.4
execution-map Provides the capability to generate and manipulate map data objects. 5.0.4
execution-json Provides the capability to retrieve, insert, and modify JSON elements. 2.0.4
execution-unitconversion Converts various units such as length, mass, time and volume. 2.0.2
execution-reorder Orders out-of-order event arrivals using algorithms such as K-Slack and alpha K-Stack. 5.0.3
execution-unique Retains and process unique events based on the given parameters. 5.0.4
execution-list Provides the capability to send an array object inside Siddhi stream definitions and use it within queries. 1.0.0
execution-streamingml Performs streaming machine learning (clustering, classification and regression) on event streams. 2.0.3
execution-tensorflow Provides support for running pre-built TensorFlow models. 2.0.2
execution-pmml Evaluates Predictive Model Markup Language (PMML). It is under GPL license. 5.0.1

Input/Output Extensions

Name Description Latest
Tested
Version
io-http Receives and publishes events via http and https transports, calls external services, and serves incoming requests and provide synchronous responses. 2.1.2
io-nats Receives and publishes events from/to NATS. 2.0.6
io-kafka Receives and publishes events from/to Kafka. 5.0.4
io-email Receives and publishes events via email using smtp, pop3 and imap protocols. 2.0.4
io-cdc Captures change data from databases such as MySQL, MS SQL, Postgresql, H2 and Oracle. 2.0.3
io-tcp Receives and publishes events through TCP transport. 3.0.4
io-googlepubsub Receives and publishes events through Google Pub/Sub. 2.0.2
io-rabbitmq Receives and publishes events from/to RabbitMQ. 3.0.2
io-file Receives and publishes event data from/to files. 2.0.3
io-jms Receives and publishes events via Java Message Service (JMS), supporting Message brokers such as ActiveMQ 2.0.2
io-prometheus Consumes and expose Prometheus metrics from/to Prometheus server. 2.1.0
io-grpc Receives and publishes events via gRpc. 1.0.2
io-mqtt Allows to receive and publish events from/to mqtt broker. 3.0.0
io-sqs Subscribes to a SQS queue and receive/publish SQS messages. 3.0.0
io-s3 Allows you to publish/retrieve events to/from Amazon AWS S3. 1.0.1
io-gcs Receives/publishes events from/to Google Cloud Storage bucket. 1.0.0

Data Mapping Extensions

Name Description Latest
Tested
Version
map-json Converts JSON messages to/from Siddhi events. 5.0.4
map-xml Converts XML messages to/from Siddhi events. 5.0.3
map-text Converts text messages to/from Siddhi events. 2.0.4
map-avro Converts AVRO messages to/from Siddhi events. 2.0.5
map-keyvalue Converts events having Key-Value maps to/from Siddhi events. 2.0.4
map-csv Converts messages with CSV format to/from Siddhi events. 2.0.3
map-binary Converts binary events that adheres to Siddhi format to/from Siddhi events. 2.0.4
map-protobuf Converts protobuf messages to/from Siddhi events.. 1.0.1

Store Extensions

Name Description Latest
Tested
Version
store-rdbms Optimally stores, retrieves, and manipulates data on RDBMS databases such as MySQL, MS SQL, Postgresql, H2 and Oracle. 7.0.0
store-mongodb Stores, retrieves, and manipulates data on MongoDB. 2.0.3
store-redis Stores, retrieves, and manipulates data on Redis. 3.1.1
store-elasticsearch Stores, retrieves, and manipulates data on Elasticsearch. 3.1.2

Script Extensions

Name Description Latest
Tested
Version
script-js Allows writing user defined JavaScript functions within Siddhi Applications to process events. 5.0.2

Configure System Parameters for Extensions

Siddhi extensions cater use-case specific logic that is not available by default in Siddhi. Some of these extensions have system parameter configurations to define/modify their behavior. These extensions usually have default values for the parameters, but when needed, they can be overridden by configuring the parameters in Siddhi Configuration yaml and passing it at startup. Refer more details on this in here

Writing Custom Extensions

Custom extensions can be written in order to cater use case specific logics that are not by default available in Siddhi core, or in existing extensions.

The following Maven Archetypes can be used to generate the necessary Maven project to create the relevant extensions.

Writing Execution Executions

Maven archetype for execution execution that is used to generate the extension types such as:

  • Function extension, by extending io.siddhi.core.executor.function.FunctionExecutor.
  • Aggregate Function extension, by extending io.siddhi.core.query.selector.attribute.aggregator.AttributeAggregatorExecutor.
  • Window extension, by extending io.siddhi.core.query.processor.stream.window.WindowProcessor.
  • Stream Function extension, by extending io.siddhi.core.query.processor.stream.function.StreamFunctionProcessor.
  • Stream Processor extension, by extending io.siddhi.core.query.processor.stream.StreamProcessor.

The CLI command to generate the Maven project for the extension is as follows.

1
2
3
4
5
mvn archetype:generate
    -DarchetypeGroupId=io.siddhi.extension.archetype
    -DarchetypeArtifactId=siddhi-archetype-execution
    -DgroupId=io.siddhi.extension.execution
    -Dversion=1.0.0-SNAPSHOT

When the command is executed, the following properties will be prompted. At last, confirm that all the property values are correct by typing Y for yes, and N if they are not.

Property Description Mandatory Default Value
_nameOfFunction The name of the custom function (function, aggregate function, window, stream function, or stream processor) that is being created. Yes -
_nameSpaceOfFunction The namespace of the function that groups similar custom functions. Yes -
groupIdPostfix Postfix for the group ID. (As a convention, the function namespace is used). No ${_nameSpaceOfFunction}
artifactId Artifact ID for the project. No siddhi-execution-${_nameSpaceOfFunction}
classNameOfFunction Class name of the Function. No ${_nameOfFunction}Function
classNameOfAggregateFunction Class name of the Aggregate Function. No ${_nameOfFunction}AggregateFunction
classNameOfWindow Class name of the Window. No ${_nameOfFunction}Window
classNameOfStreamFunction Class name of the Stream Function. No ${_nameOfFunction}StreamFunction
classNameOfStreamProcessor Class name of the Stream Processor. No ${_nameOfFunction}StreamProcessor

Writing Input/Output Executions

Maven archetype for input/output execution that is used to generate the extension types such as:

  • Source extension, by extending io.siddhi.core.stream.input.source.Source.
  • Sink extension, by extending io.siddhi.core.stream.output.sink.Sink.

The CLI command to generate the Maven project for the extension is as follows.

1
2
3
4
5
mvn archetype:generate
    -DarchetypeGroupId=io.siddhi.extension.archetype
    -DarchetypeArtifactId=siddhi-archetype-io
    -DgroupId=io.siddhi.extension.io
    -Dversion=1.0.0-SNAPSHOT

When the command is executed, the following properties will be prompted. At last, confirm that all the property values are correct by typing Y for yes, and N if they are not.

Property Description Mandatory Default Value
_IOType The name of the custom IO type that is being created. Yes -
groupIdPostfix Postfix for the group ID. (As a convention, the name of the IO type is used). No ${_IOType}
artifactId Artifact ID for the project. No siddhi-io-${_IOType}
classNameOfSink Class name of the Sink. No ${_IOType}Sink
classNameOfSource Class name of the Source. No ${_IOType}Source

Writing Data Mapping Executions

Maven archetype for data mapping execution that is used to generate the extension types such as:

  • Source Mapper extension, by extending io.siddhi.core.stream.output.sink.SourceMapper.
  • Sink Mapper extension, by extending io.siddhi.core.stream.output.sink.SinkMapper.

The CLI command to generate the Maven project for the extension is as follows.

1
2
3
4
5
mvn archetype:generate
    -DarchetypeGroupId=io.siddhi.extension.archetype
    -DarchetypeArtifactId=siddhi-archetype-map
    -DgroupId=io.siddhi.extension.map
    -Dversion=1.0.0-SNAPSHOT

When the command is executed, the following properties will be prompted. At last, confirm that all the property values are correct by typing Y for yes, and N if they are not.

Property Description Mandatory Default Value
_mapType The name of the custom mapper type that is being created. Yes -
groupIdPostfix Postfix for the group ID. (As a convention, the name of the mapper type is used). No ${_mapType}
artifactId Artifact ID of the project. No siddhi-map-${_mapType}
classNameOfSinkMapper Class name of the Sink Mapper. No ${_mapType}SinkMapper
classNameOfSourceMapper Class name of the Source Mapper. No ${_mapType}SourceMapper

Writing Store Executions

Maven archetype to generate store extension, by extending either io.siddhi.core.table.record.AbstractRecordTable or io.siddhi.core.table.record.AbstractQueryableRecordTable. Here, the former allows Siddhi queries to perform conditional filters on the store while the latter allows performing both the conditional filters and select aggregations directly on the store.

The CLI command to generate the Maven project for the extension is as follows.

1
2
3
4
5
mvn archetype:generate
   -DarchetypeGroupId=io.siddhi.extension.archetype
   -DarchetypeArtifactId=siddhi-archetype-store
   -DgroupId=io.siddhi.extension.store
   -Dversion=1.0.0-SNAPSHOT

When the command is executed, the following properties will be prompted. At last, confirm that all the property values are correct by typing Y for yes, and N if they are not.

Property Description Mandatory Default Value
_storeType The name of the custom store that is being created. Yes -
groupIdPostfix Postfix for the group ID. (As a convention, the name of the store type is used). No ${_storeType}
artifactId Artifact ID for the project. No siddhi-store-${_storeType}
className Class name of the store. No ${_storeType}EventTable

Writing Script Executions

Maven archetype to generate script extension, by extending either io.siddhi.core.function.Script.

The CLI command to generate the Maven project for the extension is as follows.

1
2
3
4
5
mvn archetype:generate
    -DarchetypeGroupId=io.siddhi.extension.archetype
    -DarchetypeArtifactId=siddhi-archetype-script
    -DgroupId=io.siddhi.extension.script
    -Dversion=1.0.0-SNAPSHOT

When the command is executed, the following properties will be prompted. At last, confirm that all the property values are correct by typing Y for yes, and N if they are not.

Property Description Mandatory Default Value
_nameOfScript The name of the custom script that is being created. Yes -
groupIdPostfix Postfix for the group ID. (As a convention, the name of the script type is used). No ${_nameOfScript}
artifactId Artifact ID for the project. No siddhi-script-${_nameOfScript}
classNameOfScript Class name of the Script. No Eval${_nameOfScript}
Top