The prediction service aims to run the user "model" by Web Server, and provide the predicting features. Currently, only the Https access mode is supported. The system assigns an access address to the prediction service running successfully. The user can access the prediction service by the "access address".

The premise of creating the prediction service is to import the well-trained model to the model repository. For the specific mode, please refer to the part of model repository.

Create Online Prediction Service

After the model is trained and released to the model repository, you can deploy the online prediction service by the online prediction service functional module of the BML platform.

In the "Prediction Service"-> "Online Prediction" page in the navigation bar, click "Create Online Prediction". In the page of pop-up box, select "Create New Service", fill in related parameters and click "Confirm".

Parameter description:

Creation mode: You can select to create a new prediction service or add a new version on the original prediction service.

Service name: Name of prediction service

Service version: Version No. of prediction service

Model: If you select "Yes", it means that the model in the model repository is selected to create the prediction service. If you select "No", it means that the custom image is selected to create the prediction service. Currently, the model in the model repository is used by default to create the prediction service. The system automatically matches the service type according to the used model.

Proportion: The platform distributes the traffic as per the proportion, and is used for multiple versions of AB Test. If there is only one version of the prediction, the traffic is 100% regardless of the proportion; if there are multiple versions, the traffic is distributed as per the proportion. For example, if the proportions of the two versions are 3 and 7 respectively, 30% and 70% of the traffic is distributed to the two versions respectively as per the proportion of 3:7.

Number of running duplicates: The prediction service can include multiple duplicates, and each copy is equivalent to ensure the high availability of the service. The fees charged increases proportionally as the number of duplicates increases.

If you want to add a version on one prediction service, you can click [Add Version] in the operation.

Then you can open the pop-up window of [Add Online Prediction Version], and fill in a new four-bit version number. Select a new model, set the proportion, computing resource and duplicates, and click "Confirm" to add a new version.

Prediction Service List

The information of the prediction service list is composed by the prediction services submitted historically, and displays the basic information of the services.

As shown in the figure, the following operations are supported in the page of prediction service list:

Add version: Add a new version based on one service, namely, the created page of creation is the new version of the existing service.

View the service quality: View the PV, total handling time, total traffic, proportion of successful assess of the service, and other service quality information.

Stop: The prediction service is stopped. After the user doesn't use the related service any more, this operation is available. After the prediction service is not, fees are not charged.

Restart: Re-create the stopped services or the services failing to be created, and recover the corresponding service to the available status.

Testing: Enter the data to be predicted by the interface. Test whether the corresponding prediction results meet the expectations.

Deletion: Delete the corresponding prediction service. The deleted prediction service cannot be recovered. The corresponding record is also deleted and can't be queried. The service can be deleted only when it is under the status of "Stopped", "Creation Failed", etc.

Online Debugging

After the the service is created successfully, you can click『Test』to debug the prediction service.

Details of Prediction Service

After creating the prediction service, you can click the service name to enter the service details. The tags are divided into two switchable tags: basic information and service quality.

The service quality displays : PV (prediction requests), total handling time, total traffic, and the proportion of successful assess. The middle broken line graph/bar graph includes: PV, total traffic (sumTraffic), average traffic of each request (avgTraffic), total handling time (sumHandleTime), average handling time of each request (avgHandleTime) at each time point. The middle pie chart includes: proportions of status codes 5xx(status5Proportion), 4xx (status4Proportion), 3xx(status3Proportion) and 2xx(status2Proportion) at each time point. Support the query of corresponding data according to time.

The basic information includes the log path, access address and version list.

The version list displays the details of multiple versions: Status, model file, resource package, traffic proportion, service type, expectation and number of actual running instances, operation.

If the status is abnormal, it means that the actual number of running duplicates is less than expected number of running duplicates. Fees are charged for the duplicates in the running or abnormal status.

You can click "Edit" to modify the prediction service, including changes of model, traffic proportions of all versions, resource package and number of running duplicates, etc. If you need a high availability, you can properly increase the number of duplicates. If a lot of computation is required, you can change to a package with a higher availability. If you want to adjust the AB Test traffic proportion, you can make the adjustment by modifying the proportion.

If you need to roll back, you can click "Configuration History" to expand the pop-up window of configuration history; select the configuration of a certain historical time and click "Recover" to roll back.

Version Details

In the version list, click the version No. of one service version the expand the details of the version service. The details include the basic information and duplicate information of the service.

Duplicate Details

Click the duplicate ID of the page of version details to enter the page of copy details to view the duplicate information and the container list.

Container Details

Click the container name to display the container details in the page of browser new label. The container details mainly include the container log, stdout and stderr output in the running process of the container. You can turn the pages by the buttons in the page.