This article outlines a real muck throwing and spilling case encountered in the transportation field and describe the pain points encountered and the edge AI solution based on Baidu IntelliEdge.

Scenario Description

In urban transportation, muck may be thrown, spill, drip and leak from the truck, causing great pollution to the urban environment. As shown in the figure below, the dripping muck causes flying dust and affect air quality, and sanitation workers have to clean it up on the road.

Some serious throwing, spilling, dripping and leaking problems can also cause traffic accidents. As shown in the figure below, the first case where a truck driver was prosecuted due to a traffic accident caused by throwing and spilling of muck occurred in Nanjing.

Muck throwing and spilling is a typical problem in urban sanitation management, which is mainly reflected in the following aspects:

• Muck throwing and spilling is not detected in time. In most cases, only when people passing by make a report will it be known that muck throwing and spilling occurs at a specific location.
• There are too many roads in the city, and it is impossible to detect the muck throwing and spilling problem by monitoring all roads in real time.
• -There are many muck transportation vehicles, and it is impossible to detect the muck throwing and spilling problem by analyzing road videos uploaded by vehicles in real time through the cloud. The amount of video data computed and processed by the cloud is too large, and the cost is high.

The best way is to let the vehicle itself actively detect and report the muck throwing and spilling problem. This is a typical scenario of edge computing video AI. Through the edge video AI, muck throwing and spilling problem can be quickly detected and located and handled in collaboration with urban management units.

Solution

Overall solution architecture is shown as below:

The solution mainly includes the following processes:

Each process is described as follows:

1. Model training

   • Muck throwing and spilling identification model via AI model training platform

2. Building of edge environment

   • Install a video camera in the tail of muck truck.
   • Install an edge device AI-BOX on the muck truck, and connect it to the rear camera. AI-BOX may be the EdgeBoard edge AI calculation box in Baidu AI market.

3. Edge device connected to BIE

   • The edge device is deployed with Baetyl edge computing framework, and connected to Baidu AI Cloud. Please see Quickly Learn about Baidu IntelliEdge

4. The cloud is deployed with an edge application module, and each module (i.e., main function) is shown as follows:

   • video-infer

     • Video camera data is connected, and frame extraction is conducted to video, to change the video processing into the image processing.
     • AI inference is conducted to image with muck throwing and spilling model, and function-manger module is called via grpc interface to be responsible for the post-processing operation of AI inference result of post-processing function function-python36-opencv.
     • Based on the result of post-processing function return, it is to determine whether to save the image and send the message. For example, when the post-processing function return message shows the existence of muck throwing and spilling, frame extracting image should be saved locally, and muck throwing and spilling message should be sent to Hub module.

   • function-manger

     • Open grpc interface, used for calling by video-infer module
     • Manage function-python36-opencv function

   • function-python36-opencv

     • AI interface result post-processing function takes responsibility to conduct extra post-processing operation to AI interface result, for example, for the AI interface result of object recognition, only when the score is more than 0.6, it is needed to save the frame extracting image, and send the message to hub module. When the score is less than 0.6, frame
     • -The decision of post-processing is returned to video-infer module, and this module is responsible for specific implementation.

   • hub

     • Receive the interface result message ofvideo-infer.

   • remote-mqtt

     • When the throwing and spilling message in hub module is reported to cloud Internet of Things platform, and cloud business system will take action based on the message.

   • remote-object

     • The frame extracting image of throwing and spilling is reported to cloud object for storage, as the input of AI model training platform, to optimize the AI model continuously.

5. Distribute a configuration

   • After application module is configured in the cloud, the overall configuration is published as an official edition, and then the overall official edition is distributed to edge device. See Distribute an Application Configuration

6. Carry out object detection

   • Corresponding to the work of aforementioned video-infer,function-manger and function-python36-opencv modules respectively.

7. Report the result

   • Corresponding to the work of aforementioned ‘remote-mqtt’ and ‘remote-object’ modules respectively.

Description: For the complete configuration guide of application in step 4, please see Operation Guide.