As stated in Getting Started, in specific deployment and use of BAETYL, the storage volume is a frequently occurring word. Especially for the BIE cloud console, storage volume is a concept that is necessary and must be mastered (understood thoroughly).

Storage volume: It refers to the directory used by service. It can be a read-only directory, such as the directory where resources such as configurations, certificates and scripts are stored, or it can also be a writable directory, such as log, data and other persistent directories. Storage volume mounting: Service is operating within the container, and there are two demands at this time. One is obtainment of service configuration files, and the other is persistence of service data. They are realized by mounting the storage volume, and through mapping between host directories and directories in the container, configuration obtainment and data persistence are realized.

In the following content, explain the design concept of storage volumes, template introduction and creation, template application and storage volume application.

Design Concept

Storage volume is an abstraction of all resources, used to manage resources such as configurations, certificates and AI models. Storage volume can realize resource isolation and sharing. Only when a service is bound with a storage volume can the resources in the storage volume be used; one storage volume can be bound with multiple services to realize sharing.

Note: Service refers to a set of operating programs controlled by BAETYL to provide certain specific functions, such as message routing services, function computing services and microservices.

Official Template

Currently, the official platform provides 5 types of templates - BAETYL module configuration, computing configuration, certificate configuration, custom and empty directory. The definitions and contents of various templates are as follows:

• storage volume template for BAETYL module configuration: Provide templates for the configuration file of each service module of BAETYL, such as hub, function-manager, function-python27, function-python36, function-node85, remote-mqtt and timer.

• Storage volume template for computing configuration: Provide configuration templates for specific computing services of the function-manager service, including CFC function template and BOS file template.

  • CFC function template: Provide direct import interface from Baidu AI Cloud CFC for scripts of function computing applications such as Python27, Python36 and Nodejs85. Please note that before application of this template, the related function scripts should have been created in Baidu AI Cloud CFC and are in the same region (Beijing or Guangzhou, they cannot reference each other if they are in different regions).
  • BOS file template: It provides import interface from Baidu AI Cloud BOS for using AI models. It should be noted that before application of this template, it requires that the model files to be imported should be uploaded to Baidu AI Cloud BOS storage in advance and are in the same region (Beijing (BOS includes Baoding) or Guangzhou (BOS includes Suzhou), they cannot reference each other if they are in different regions). Currently, it only supports import through ZIP compression package.

• Storage volume template for certificate configuration: It is mainly used to provide the SSL/TLS security certification certificate required to establish a connection with the Baetyl-hub service or to establish a connection with the Baidu AI Cloud IoTHub platform through the Baetyl-remote-mqtt module.

  • Client certificate template issued through BIE: It is used to issue the client certificate and CA required for establishing SSL/TLS connection between end device or MQTT Client and the Baetyl-hub service.
  • Server certificate template issued through BIE: It is used to issue the server certificate and CA required for establishing SSL/TLS bidirectional secure connection between end device or MQTT Client and the Baetyl-hub service. It should be noted that when this storage volume is used, its reference path in the configuration file of the Baetyl-hub module must be consistent with its mapping path in the Docker container when it is mounted to the edge core.
  • Client certificate template of Baidu AI Cloud IoT Hub: It is mainly used to issue client certificate files required for establishing SSL/TLS secure connection between the Remote module of the edge core and the Baidu AI Cloud IoTHub platform. It should be noted that for application of this storage volume, it requires to select the “certificate” method for authentication when establishing users under endpoint in the Baidu AI Cloud IoTHub platform, upload the cert-and-keys.txt file obtained from the IoTHub platform and conduct automatic splitting through this storage volume template (client certificate and CA).
• Custom storage volume template: It can be used to create files in various formats, such as xml, toml and txt.
• Storage volume template with empty directory: It only equals to a directory and cannot store files. It is mainly used to mount module logs and storage applications for module data persistence.

Create a Module Application or Storage Volume

Creating a storage volume mainly means creating a storage volume required for deployment of various module services of BAETYL. Based on different demand scenarios, different storage volume templates should be used to create different types of storage volumes, mount them to related edge cores and then distribute them to local edge devices for operation. For example, for the Baetyl-hub module, usually 3 storage volumes are created, i.e. storage volume for configuration files, storage volume for data persistence and storage volume for logs; for the Baetyl-function-manager module (same to the Remote module), usually 2 storage volumes are created, i.e. storage volume for configuration files and storage volume for logs; for the Baetyl-function-python27 (same to function-python36 and function-node85) module, usually 2 storage volumes are created, i.e. storage volume for configuration files and storage volume for CFC functions. If SSL/TLS secure certificate connection is required, for the Baetyl-hub module, Baetyl-function-manager module, Baetyl-timer module and Baetyl-remote-mqtt module, certificate storage volume also has to be mounted.

The complete process of creating a storage volume (“Release a new version” is not required for storage volumes with empty directory) is as follows:

Take an example for each storage volume mentioned above to introduce how to create it in the following content.

Create a Storage Volume for BAETYL Configuration Files

Here, the example of creating storage volumes for configuration files of the Baetyl-hub module is given for introduction.

First, click “Create a storage volume” to start creating a storage volume, as shown in the figure below.

Then, select baetyl module configuration for the storage volume template, name it bie-demo-hub-conf and click “OK”.

You will find that the service.yml file has been generated by default in the storage volume, and it is the default configuration file name in each service module of BAETYL.

Next, enter the configuration information of the Baetyl-hub module:

listen: 
  - tcp://0.0.0.0:1883
principals: 
  - username: test 
    password: hahaha 
    permissions: 
      - action: 'pub' 
        permit: ['#'] 
      - action: 'sub' 
        permit: ['#'] 
subscriptions: 
  - source: 
      topic: 't' 
    target: 
      topic: 't/topic' 
logger: 
  path: var/log/baetyl/service.log 
  level: "debug" 

The definition of configuration is that you have an access identity with a user name of test and a password of hahaha, you have all pub and sub privileges, and the hub will forward the topic t to the topic t/topic. For more configuration resolutions, please see Configuration File Description.

Save and release the new version.

Create a Storage Volume for Computing Configuration

Here, take CFC function template and BOS file template as examples for introduction.

Use a CFC function template

For using a CFC function template to create a code storage volume, here the example of importing Python27 script from CFC is given for description.

First, click “Create a storage volume”, select CFC function template as the template, name it bie-demo-compute-code and click “OK”.

Select to import a CFC function, and select the function which has been created in advance in Baidu AI Cloud CFC. Currently, it only supports to import Python27, Python36 and Nodejs85 scripts.

Create a simple piece of code in CFC. Currently, it only supports to import Python27, Python36 and Nodejs85 functions.

def handler(event, context): 
    event["result"] = event["x"] + event["y"] + event["z"] 
    return event 

Then, release the new version.

Note: When using the CFC function storage volume template, the function scripts should be created in CFC in advance and they should be in the same region (Beijing or Guangzhou).

Use a BOS file template

Similar to importing a CFC function, using a BOS file storage volume template is to import related content files from the Baidu object storage service, and usually they are files with large size.

Click “Create a storage volume”, name it bos-volume, select “BOS file template” and click “OK”.

Then, continue to click “Import a BOS Zip package”, select the corresponding Bos Bucket and Bos Key (file names stored in BOS), and click “OK”.

If the above operations are smooth, the Zip package demo_meter_V4.zip can be successfully imported. After correct import, the cloud platform can automatically decompress the imported Zip package and display the decompressed result, as shown in the figure below.

You can see that the ‘var’ directory has been successfully imported from BOS, and then release the new version. It should be noted that similar to CFC function templates, importing from BOS also requires to upload the Zip package to be imported to BOS in advance, and they should be in the same region (Guangzhou (BOS includes Suzhou) or Beijing (BOS includes Baoding)).

Create a Storage Volume for Certificate Configuration

Storage volume template for certificate configuration mainly includes 3 types - client certificate issued through BIE, server certificate issued through BIE and client certificate of Baidu AI Cloud IoT Hub. Take examples respectively for description in the following content.

Client certificate issued through BIE

Client certificate issued through BIE is mainly used when SSL/TLS connection is adopted for end device or MQTT Client which is connected to the Baetyl-hub service. Using this template, it can provide users with officially issued security certificates through BIE. The related steps are as follows:

Click “Create a storage volume”, enter the name baetyl-hub-client-cert, and click “OK”.

Then, continue to click “Generate a certificate”, enter the domain name (it can be left blank or be the service name connected to the Baetyl-hub module or be the domain name of the machine where the custom Hub module is located), and click “OK”.

If the above operations are correct, you can see the three certificates - ca.pem, client.pem and client.key are generated.

As shown in the figure above, after the certificates are created, click “Download a storage volume ZIP package” to download and decompress the generated certificates for establishing TLS/SSL connection.

Server certificate issued through BIE

The steps are completely the same as those of client certificate issued through BIE, but just select the template of “server certificate issued through BIE” when a storage volume is created, and the creation process is shown below.

Slightly different from client certificate issued through BIE, after the certificates are created, you do not have to “Download a storage volume ZIP package”, instead, you have to release the new version and mount it to the Baetyl-hub service module, used for certification of clients which are connected to the Baetyl-hub service.

Client certificate of Baidu AI Cloud IoT Hub

Different from the client and server certificates issued through BIE mentioned above, the client certificate of Baidu AI Cloud IoT Hub is generated here, used for certification of SSL/TLS connection when synchronizing data to Baidu AI Cloud IoT Hub through the Remote module.

It should be particularly noted that when this storage volume template is used to create a storage volume, you have to create related endpoint on the Baidu AI Cloud IoT Hub platform, select the authentication by certificates method for authentication of user connection when creating a user and download the integrative file (cert-and-keys.txt) of authentication certificates provided for Baidu AI Cloud IoT Hub. For related content, see Baidu AI Cloud IoT Official Documentation.

Then, switch to the page to create a Baidu IntelliEdge storage volume, click “Create a storage volume”, enter the name remote-iothub-cert, and click “OK”.

Continue to click “Upload a certificate file”, and read the “Privacy Policy” and “Baidu AI Cloud User Service Agreement”. If there is no doubt, click “Confirm to upload”, select “cert-and-keys.txt” (download it from the Baidu AI Cloud IoT Hub platform) to be uploaded locally, and click “OK”.

If the above operations are normal, you can see the three split certificate files - ca.pem, client.pem and client.key (mainly used for SSL /TLS authentication when the Remote module is connected to the Baidu AI Cloud IoT Hub), and then release the new version.

Create a Custom Storage Volume

As the name suggests, custom storage volume is for easier use of storage volumes by users. Its format and content are not constrained or restricted by the BAETYL framework (for example, configuration files in each module are in yaml format). Users can place files of any format and store any content.

Here, an example of user loading a custom module is given for description. If you want to import a custom module based on the BAETYL framework, but the configuration file of the module is in toml format, you can select to create and mount a custom storage volume when creating a storage volume for configuration file of this custom module.

Same as the steps to create other storage volumes mentioned above, click “Create a storage volume”, enter the name of the storage volume cutomize-module-conf, select “custom storage volume template”, and click “OK”.

Then, click “Create an editable file”, name the new file service.toml, enter the related configuration information in the editable box, click “Save”, and then release the new version.

With this, the storage volume for configuration file of a custom module is created.

In addition, as shown in the figure above, it is not difficult to find that after the custom storage volume template is selected, you can not only create an editable file (on-line editing), but also upload a ZIP package, which facilitates the smooth import of user’s custom modules.

Create a Storage Volume with Empty Directory

As mentioned above, template of storage volume with empty directory is used for recording module logs and making data persistent. Here, an example of establishing a log storage volume of the Baetyl-hub module is given for description.

First, click “Create a storage volume”, select “storage volume with empty directory” as the storage volume template, name it bie-demo-hub-log, and click “OK”.

After the storage volume with empty directory is created, a new version (V1) will be released. Modified files cannot be added in it, and it only acts as a “folder/directory”.

Application of the Storage Volume

As mentioned above, after a storage volume is created, it is mainly used to be mounted in the edge core service so that services in each module can use the resources in the storage volume, and multiple services can share the same storage volume.

Generally, you can create a storage volume first (recommended), then create an edge core, add services in each module for the edge core, mount the corresponding storage volume for each service, generate a new configuration for the edge core and then distribute it.

The related process is shown in the figure below:

In addition, for the core that is already in the application, if you want to update the storage volume configuration of a service, the process to update the corresponding storage volume version is as follows.

The related operation diagram is shown below.

With this, the content about application of storage volumes is finished.