

Siddhi 5.2 as a Docker Microservice¶

This section provides information on running Siddhi Apps on Docker.

Siddhi Microservice can run one or more Siddhi Applications with required system configurations. Here, the Siddhi application (.siddhi file) contains stream processing logic and the necessary system configurations can be passed via the Siddhi configuration .yaml file.

Steps to Run Siddhi Docker Microservice is as follows.

Pull the the latest Siddhi Runner image from Siddhiio Docker Hub.
```
docker pull siddhiio/siddhi-runner-alpine:latest
```

Start SiddhiApps with the runner config by executing the following docker command.

docker run -it -v <local-siddhi-file-path>:<siddhi-file-mount-path> -v <local-conf-file-path>:<conf-file-mount-path> siddhiio/siddhi-runner-alpine:latest -Dapps=<siddhi-file-mount-path> -Dconfig=<conf-file-mount-path>

E.g.,

docker run -it -v /home/me/siddhi-apps:/apps -v /home/me/siddhi-configs:/configs siddhiio/siddhi-runner-alpine:latest -Dapps=/apps/Foo.siddhi -Dconfig=/configs/siddhi-config.yaml

Running multiple SiddhiApps in one runner instance.

To run multiple SiddhiApps in one runtime instance, have all SiddhiApps in a directory, mount the directory and pass its location through -Dapps parameter as follows,
-Dapps=<siddhi-apps-directory>

Always use absolute path for SiddhiApps and runner configs.

Providing absolute path of SiddhiApp file, or directory in -Dapps parameter, and when providing the Siddhi runner config yaml on -Dconfig parameter while starting Siddhi runner.

Siddhi Tooling

You can also use the powerful Siddhi Editor to implement and test steam processing applications.

Configuring Siddhi

To configure databases, extensions, authentication, periodic state persistence, and statistics for Siddhi as Docker Microservice refer Siddhi Config Guide.

Samples¶

Running Siddhi App¶

Following SiddhiApp collects events via HTTP and logs the number of events arrived during last 15 seconds.

Always listen on 0.0.0.0 with the Siddhi Application running inside a docker container.

If you listen on localhost inside the container, nothing outside the container can connect to your application. That includes blocking port forwarding from the docker host and container to container networking.

Copy the above SiddhiApp, and create the SiddhiApp file CountOverTime.siddhi.

Run the SiddhiApp by executing following commands from the distribution directory

docker run -it -p 8006:8006 -v <local-absolute-siddhi-file-path>/CountOverTime.siddhi:/apps/CountOverTime.siddhi siddhiio/siddhi-runner-alpine -Dapps=/apps/CountOverTime.siddhi

Test the SiddhiApp by calling the HTTP endpoint using curl or Postman as follows
- Publish events with curl command:
  Publish few json to the http endpoint as follows,
```
curl -X POST http://localhost:8006/production \
  --header "Content-Type:application/json" \
  -d '{"event":{"name":"Cake","amount":20.12}}'
```
- Publish events with Postman:
  1. Install 'Postman' application from Chrome web store
  2. Launch the application
  3. Make a 'Post' request to 'http://localhost:8006/production' endpoint. Set the Content-Type to 'application/json' and set the request body in json format as follows,
```
{
  "event": {
    "name": "Cake",
    "amount": 20.12
  }
}
```

Runner logs the total count on the console. Note, how the count increments with every event sent.

[2019-04-11 13:36:03,517]  INFO {io.siddhi.core.stream.output.sink.LogSink} - CountOverTime : TotalCountStream : Event{timestamp=1554969963512, data=[1], isExpired=false}
[2019-04-11 13:36:10,267]  INFO {io.siddhi.core.stream.output.sink.LogSink} - CountOverTime : TotalCountStream : Event{timestamp=1554969970267, data=[2], isExpired=false}
[2019-04-11 13:36:41,694]  INFO {io.siddhi.core.stream.output.sink.LogSink} - CountOverTime : TotalCountStream : Event{timestamp=1554970001694, data=[1], isExpired=false}

Running with runner config¶

When running SiddhiApps users can optionally provide a config yaml to Siddhi runner to manage configurations such as state persistence, databases connections and secure vault.

Following SiddhiApp collects events via HTTP and store them in H2 Database.

The runner config can be configured with the relevant datasource information and passed when starting the runner

Copy the above SiddhiApp, & config yaml, and create corresponding the SiddhiApp file ConsumeAndStore.siddhi and TestDb.yaml files.

Run the SiddhiApp by executing following command

docker run -it -p 8006:8006 -p 9443:9443 -v <local-absolute-siddhi-file-path>/ConsumeAndStore.siddhi:/apps/ConsumeAndStore.siddhi -v <local-absolute-config-yaml-path>/TestDb.yaml:/conf/TestDb.yaml siddhiio/siddhi-runner-alpine -Dapps=/apps/ConsumeAndStore.siddhi -Dconfig=/conf/TestDb.yaml

Test the SiddhiApp by calling the HTTP endpoint using curl or Postman as follows
- Publish events with curl command:
  Publish few json to the http endpoint as follows,
```
curl -X POST http://localhost:8006/production \
  --header "Content-Type:application/json" \
  -d '{"event":{"name":"Cake","amount":20.12}}'
```
- Publish events with Postman:
  1. Install 'Postman' application from Chrome web store
  2. Launch the application
  3. Make a 'Post' request to 'http://localhost:8006/production' endpoint. Set the Content-Type to 'application/json' and set the request body in json format as follows,
```
{
  "event": {
    "name": "Cake",
    "amount": 20.12
  }
}
```
Query Siddhi Store APIs to retrieve 10 records from the table.
- Query stored events with curl command:
  Publish few json to the http endpoint as follows,
```
curl -X POST https://localhost:9443/stores/query \
  -H "content-type: application/json" \
  -u "admin:admin" \
  -d '{"appName" : "ConsumeAndStore", "query" : "from ProductionTable select * limit 10;" }' -k
```
- Query stored events with Postman:
  1. Install 'Postman' application from Chrome web store
  2. Launch the application
  3. Make a 'Post' request to 'https://localhost:9443/stores/query' endpoint. Set the Content-Type to 'application/json' and set the request body in json format as follows,
```
{
  "appName" : "ConsumeAndStore",
  "query" : "from ProductionTable select * limit 10;"
}
```
The results of the query will be as follows,
```
{
  "records":[
    ["Cake",20.12]
  ]
}
```

Running with environmental/system variables¶

Templating SiddhiApps allows users to provide environment/system variables to siddhiApps at runtime. This can help users to migrate SiddhiApps from one environment to another (E.g from dev, test and to prod).

Following templated SiddhiApp collects events via HTTP, filters them based on amount greater than a given threshold value, and only sends the filtered events via email.

Here the THRESHOLD value, and TO_EMAIL are templated in the TemplatedFilterAndEmail.siddhi SiddhiApp.

The runner config is configured with a gmail account to send email messages in EmailConfig.yaml by templating sending EMAIL_ADDRESS, EMAIL_USERNAME and EMAIL_PASSWORD.

Copy the above SiddhiApp, & config yaml, and create corresponding the SiddhiApp file TemplatedFilterAndEmail.siddhi and EmailConfig.yaml files.

Set the below environment variables by passing them during the docker run command:

 THRESHOLD=20
 TO_EMAIL=<to email address> 
 EMAIL_ADDRESS=<gmail address>
 EMAIL_USERNAME=<gmail username>
 EMAIL_PASSWORD=<gmail password>

Or they can also be passed as system variables by adding them to the end of the docker run command .

-DTHRESHOLD=20 -DTO_EMAIL=>to email address> -DEMAIL_ADDRESS=<gmail address> 
 -DEMAIL_USERNAME=<gmail username> -DEMAIL_PASSWORD=<gmail password>

Run the SiddhiApp by executing following command.

docker run -it -p 8006:8006 -v <local-absolute-siddhi-file-path>/TemplatedFilterAndEmail.siddhi:/apps/TemplatedFilterAndEmail.siddhi -v <local-absolute-config-yaml-path>/EmailConfig.yaml:/conf/EmailConfig.yaml -e THRESHOLD=20 -e TO_EMAIL=<to email address> -e EMAIL_ADDRESS=<gmail address> -e EMAIL_USERNAME=<gmail username> -e EMAIL_PASSWORD=<gmail password> siddhiio/siddhi-runner-alpine -Dapps=/apps/TemplatedFilterAndEmail.siddhi -Dconfig=/conf/EmailConfig.yaml

Test the SiddhiApp by calling the HTTP endpoint using curl or Postman as follows
- Publish events with curl command:
  Publish few json to the http endpoint as follows,
```
curl -X POST http://localhost:8006/production \
  --header "Content-Type:application/json" \
  -d '{"event":{"name":"Cake","amount":2000.0}}'
```
- Publish events with Postman:
  1. Install 'Postman' application from Chrome web store
  2. Launch the application
  3. Make a 'Post' request to 'http://localhost:8006/production' endpoint. Set the Content-Type to 'application/json' and set the request body in json format as follows,
```
{
  "event": {
    "name": "Cake",
    "amount": 2000.0
  }
}
```

Check the to.email for the published email message, which will look as follows,

Subject : High Cake production!

Hi, 

High production of Cake, with amount 2000.0 identified. 

For more information please contact production department. 

Thank you

Top