Operation Guide

Last Updated：2020-10-10

Typical Application Scenarios of Edge Stream Computing is described in the above. This paper introduces how to use the edge stream computing.

Simulation Scenario Description

Use simulator to send the simulation data to edge hub uninterruptedly at the frequency of one piece of data per second.
The format of simulation data is: {"humidity":6.3426914,"temperature":11.457714,"timestamp":1576207523}
The cloud is configured with edge stream computing task, to calculate the statistical value of temperature, including:
- 10-minute mean
- 10-minute maximum
- 10-minute minimum
- 10-minute computation sample number
Distribute the edge stream computing task to edge core device.
Subscribe the edge hub, and verify the stream computing result.

Prerequisite Preparation

You should have a Baidu AI Cloud. If not, click Register
Enable Baidu IntelliEdge (BIE) service by clicking Enable.
A Raspberry Pi should be available. Raspberry Pi 4 is used in this experiment. Other models of Raspberry Pi (Raspberry Pi 3 Model B+) or virtual machines of Ubuntu16.04 can also be used.
The Raspberry Pi has been connected to the cloud BIE. For the configuration process, please see Getting Started.
There is a data simulator application, and this paper provides the data simulator application to be used in the tutorial. If you have development capability, you can simply modify baetyl-timer to realize one edge data simulator application.

In this tutorial, you can learn

1.Data simulator application configuration 2.Edge stream computing configuration

Configuration Guide

Apart from the agent module used for connecting BIE cloud management suite, the following three modules also need to be configured:

The function of each module is described as follows:

localhub: Local MQTT Broker on the edge side, local data interaction center.
temperature-simulator-2: Data simulator module, sending the simulation data to localhub uninterruptedly. The format of simulation data is shown as follows:
```
{ 
    "humidity":36.97492, 
    "temperature":92.05849, 
    "timestamp":1576215005 
} 
```
statistics_1min_temperature: Stream computing task module, to be responsible for real-time subscription of localhub data, calculate the subscribed stream data, and return the computation result to localhub module. The data format of stream computing return is shown as below:
```
{ 
    "avg_temp":55.232093,
    "count_temp":60,
    "max_temp":97.903465,
    "min_temp":2.287275,
    "ts":1576214940000 
} 
```

Data flow chart between module

Module 1: Localhub

Localhub configuration is as shown in the following figure:

Port mapping needing addition of 1883:1883

Conf is baetyl module configuration

Data and log are empty directory storage volume

mt-localhub-conf configuration is shown as below:

listen: 
  - 'tcp://0.0.0.0:1883' 
principals: 
  - username: test 
    password: hahaha 
    permissions: 
      - action: pub 
        permit: 
          - '#' 
      - action: sub 
        permit: 
          - '#' 
logger: 
  path: var/log/baetyl/localhub-service.log 
  level: debug

Module 2: Temperature-simulator-2

Firstly, enter into Baidu IntelliEdge > Module list, click Create a Module, as shown in the following figure:

Name: temperature-simulator
Image address:
- armv7l: hub.baidubce.com/intelliedge/temperature-simulator:2.0
- amd64(x86): hub.baidubce.com/intelliedge/temperature-simulator:3.0
- arm64: hub.baidubce.com/intelliedge/temperature-simulator:4.0

Enter into Baidu IntelliEdge > Core list > Core details, click Add a Service, and the configuration of temperature-simulator-2 is as shown in the following figure:

Conf is baetyl module configuration
Log is empty directory storage volume

The configuration of timer-conf is shown as below:

hub: 
  address: 'tcp://localhub:1883' 
  username: test 
  password: hahaha 
  clientid: timer-1 
timer: 
  #Appoint the data transmission interval 
  interval: 1s 
publish: 
  #Appoint the topic sent to hub 
  topic: testtopic/update 
logger: 
  path: var/log/baetyl/service.log 
  level: debug

hub: To connect with localhub as mqtt client

interval: 1s: The simulation data is sent once a second.

topic: testtopic/update: The simulation data is sent to the topic, and stream computing task will subscribe this topic.

Module 3: Statistics_1min_temperature

The configuration of edge stream computing is mainly in two steps, respectively, Create edge stream computing task and Add stream computing service to edge core .

Step1:Create Edge Stream Computing Task

Create a data source

Enter into Rule engine > Real-time computation > Data source list, and click Create a data source, as shown in the following figure:

Operating environment: Edge stream computing task needs to be created here, so edge is selected.
Name: Data source name, and src_data is to be filled here.
Type: MQTT
Attributes:
- Server: tcp://localhub:1883, the link address of localhub module
- Topic: testtopic/update: the simulation value generated by data simulator is sent to this topic, and stream computing task needs to subscribe the data of such topic for computation.
- User name, password: To be consistent with principals configured by localhub.
Format: Json
Field: The field selected may be identification from JSON, and the format of JSON is already given in simulation scenario description.
Message with timestamp Select Yes.
Timestamp field Use the timestamp field in the original data, and it is timestamp here, which can be found out in the field above.
Watermark: Use the default value, 2 seconds.

Create the data destination

Enter into Rule engine > Real-time computation > Data destination list, and click Create a data destination, as shown in the following figure:

Operating environment: Edge
Name: Data destination name: target_data to be filled here.
Type: MQTT
Attributes:
- Server: tcp://localhub:1883, the link address of localhub module.
- Topic: testtopic/streamdata, a topic the stream computing result needs to return, and the stream computing result may be checked by subscribing this topic.
- User name, password: To be consistent with [principals configured by localhub](# module1: localhub)
Format: Json

Create tasks

Enter into Rule engine > Real-time computation > Task list, and click Create a task, as shown in the following figure:

Operating environment: Edge
Name: Task name: statistics_1min_temperature to be filled here.
Time type: EVENTTIME, indicating the timestamp of the service data, to be consistent with data source configuration.
Degree of concurrence: Default value, 1
SQL:

INSERT INTO target_data 
SELECT 
      TUMBLE_START(rowtime, INTERVAL '1' MINUTE) AS ts, 
      MAX(temperature) AS max_temp, 
      MIN(temperature) AS min_temp, 
      AVG(temperature) AS avg_temp, 
      COUNT(temperature) AS count_temp 
FROM src_data 
GROUP BY 
      TUMBLE(rowtime, INTERVAL '1' MINUTE)

After completion of configuration, Debug mode may be enabled for verification, as shown in the following figure:

Input the following four lines of json in src_data:

{"humidity":36.97492,"temperature":92.05849,"timestamp":1576215005} 
{"humidity":8.728794,"temperature":9.086535,"timestamp":1576215004} 
{"humidity":37.186928,"temperature":34.015495,"timestamp":1576215003} 
{"humidity":5.9257016,"temperature":46.55856,"timestamp":1576215002}

Click Test run, to obtain the following result, indicating that the stream computing SQL configuration is correct.

{"ts":1576215000000,"max_temp":92.05849,"min_temp":9.086535,"avg_temp":45.429770000000005,"count_temp":4}

Step2: Add Stream Computing Service to Edge Core

Add stream computing service

Enter into Baidu IntelliEdge > Core list > Core details, and on the interface of core details, click Add a service, and select stream computing module, as shown in the following figure:

Service type: Stream computing module.
Task: Select the created edge stream computing task in the drop-down box, statistics_1min_temperature.
Service name: Use the default valuestatistics_1min_temperature.
Stream computing module uses system default storage volume, and you have no need to configure the storage volume manually.

Update the stream computing service

After change of stream computing task, click Check stream computing service, and update are monitored automatically, and a prompt Update or Not are given, as shown in the following figure.

Verify the Edge Stream Computing Result

Check the Service Start Status

After completion of module configuration, the configuration is published as an official edition, and then Distribute operation is carried out, as shown in the following figure:

In the first distribution, edge core device needs to download the image, it takes several minutes to start all the services, and the real-time service status can be checked timely on the cloud, as shown in the following figure:

You also may login the Raspberry Pi remotely, to check the service startup situation via `docker ps’ order, as shown in the following figure:

Check the Stream Computing Result

Start MQTTBox, subscribe testtopic/update and testtopic/streamdata, these two topics, and check the simulation data produced by data simulator as well as the stream computing result based on simulation data, as shown in the following figure:

testtopic/update: Simulation data, one piece of record per second
testtopic/streamdata: Stream computing results, containing the maximum, minimum, mean, and computation sample number. As it is the one-minute statistical result, the computation sample number is 60, and it is conforming to the reality.

Application Scenario Overview

Configuration File Description

百度智能云

Intelligent Edge