Model Object Definition

Cluster

Parameter Name	Parameter Type	Parameter Description
applications	List<Application>	Information on component to install (e.g. hive, pig, and hbase)
autoTerminate	Boolean	Whether to automatically release the cluster after running all steps
id	String	The cluster ID is a fixed-length string which only contains uppercase and lowercase letters, numbers, hyphens (-), and underlines (_)
imageType	String	Image type, and see [BMR Image](#BMR Image) for more information
imageVersion	String	Image version, and see [BMR Image](#BMR Image) for more information
logUri	String	BOS path used to upload the operating log
name	String	Cluster name
status	ClusterStatus	Information on cluster status

Application

Parameter Name	Parameter Type	Parameter Description
name	String	Component name (e.g. hive, pig, hbase, and hue), and see [BMR Image](#BMR Image) for more information
properties	Object	Component properties, depending on different components
version	String	Component version, and see [BMR Image](#BMR Image) for more information

Component properties

The hive component’s properties are defined as follows:

Parameter Name	Parameter Type	Parameter Description
metastore	String	How to enable hive metastore. default means the use of the cluster’s metastore service, and mysql means the use of the user’s mysql services.
host	String	mysql server address, and this field does not appear when metastore is default
port	int	mysql server port and this field does not appear when metastore is default
database	String	Name of database used by metastore, and this field does not appear when metastore is default
userName	String	User name used to connect mysql server, and this field does not appear when metastore is default

hbase component, pig component and hue component have no properties fields

ClusterStatus

Parameter Name	Parameter Type	Parameter Description
creationDateTime	String	Cluster creation time, conforming to Date and Time Format Constraints
endDateTime	String	Cluster end time, conforming to Date and Time Format Constraints
readyDateTime	String	Cluster ready time, conforming to Date and Time Format Constraints
state	String	[Cluster Status](#Cluster Status)

Step

Parameter Name	Parameter Type	Parameter Description
actionOnFailure	String	[Step Failure Policy](#Step Failure Policy)
id	String	The step ID is a fixed-length string which only contains uppercase and lowercase letters, numbers, hyphens (-), and underlines (_)
name	String	Step name
properties	Object	[Step Description](#Step Description), depending on the step type
status	StepStatus	Information on step status
type	String	[Step Type](# Step Type)
logUri	String	Step log path

StepStatus

Parameter Name	Parameter Type	Parameter Description
creationDateTime	String	Step submission time, conforming to [Date and Time Format Constraints](#Date and Time Format Constraints)
endDateTime	String	Step end time, conforming to [Date and Time Format Constraints](#Date and Time Format Constraints)
startDateTime	String	Step starting time, conforming to [Date and Time Format Constraints](#Date and Time Format Constraints)
state	String	[Step State](#Step State

Enumeration Type Definition

Instance Group Type

Coding	Description
Master	Master node
Core	Computing node with hdfs deployment
Task	Computing node without hdfs, and suitable for expansion

Cluster Status

Coding	Description
Starting	Initializing
Running	Running
Waiting	Waiting
Terminating	Releasing
Terminated	Released
TerminatedWithErrors	Released with errors
Suspending	Service stopping
Suspended	Service stopped
Resuming	Recovering

Step Type

Coding	Description
Streaming	MapReduce Streaming program
Hive	HQL program
Pig	pig step
Java	User-defined Java program
Spark	Spark step

Step Failure Policy

Coding	Description
Continue	Continue other steps
TerminateCluster	Release cluster after step failure
CancelAndWait	Cancel other steps not executed yet after step failure and set the cluster in waiting status

Step Status

Coding	Description
Pending	Pending
Running	Running
Completed	Completed
Cancelled	Cancelled
Failed	Failed

BMR Image

Image Type	Image Version	Supported Component
BMR	1.0.0	hadoop 2.7 spark 1.6.0 hive 1.2.0 pig 0.15.1 hue 3.10.0 hbase 1.1.2
BMR	1.1.0	hadoop 2.7 hive 1.2.0 pig 0.15.1 hue 3.10.0 spark 2.1.0 hbase 1.1.2
BMR	1.2.0	hadoop 2.7 hive 1.2.0 pig 0.15.1 hue 3.10.0 spark 2.1.0 hbase 1.1.2 ranger 0.5.0
BMR	2.0.0	hadoop 3.1 hive 3.1.0 spark 2.3.2 pig 0.17.0 hue 4.4.0 presto 0.219 hbase 2.0.2 azkaban 3.58.0 zeppelin 0.8.0 flink 1.8.2 druid 0.12.1 impala 3.2.0
BMR	2.1.0	hadoop 3.1 hive 3.1.0 spark 2.4.2 pig 0.17.0 hue 4.4.0 presto 0.219 hbase 2.0.2 azkaban 3.58.0 zeppelin 0.8.0 flink 1.8.2 druid 0.12.1 impala 3.2.0 kafka 2.0.1

Instance Specifications

For instance specifications, please see Instance Specifications

Cloud Disk Type

Coding	Type	Description
ssd	High-performance cloud disk	System disk and data disk support and select this type
premium_ssd	ssd cloud disk	System disk and data disk support this type
sata	Last-generation cloud disk	Data disk supports this type

InstanceGroupConfig Data Structure

Parameter Name	Parameter Type	Required	Parameter Description
instanceCount	Int	Yes	Number of virtual machines in the instance group
instanceType	String	Yes	Type of virtual machines in the instance group; for more information, please see [Instance Specifications](#Instance Specifications)
type	String	Yes	Master, Core or Task; for more information, please see [Instance Group Type](#Instance Group Type)
name	String	No	Instance group name, which is blank by default
rootDiskSizeInGB	int	Yes	Capacity of system disk
rootDiskMediumType	String	Yes	Medium type of system disk
cds	List<CdsItem>	No	List of data disks and cloud disks

Notes: The instanceGroups field includes three instance groups, and the type is Master, Core, or Task; the instanceCount is 1 for Master instance group, not less than 2 and not more than 20 for Core instance group, and not more than 20 for Task instance group.

ApplicationConfig Data Structure

Parameter Name	Parameter Type	Required	Parameter Description
name	String	Yes	Component name (e.g. hive, pig, hbase, and hue), and see [BMR Image](#BMR Image) for more information
version	String	Yes	Component version, and see [BMR Image](#BMR Image) for more information
properties	Object	No	[Component Properties](#Component Properties), depending on different components

Component Properties

The hive component’s properties are defined as follows:

Parameter Name	Parameter Type	Required	Parameter Description
metastore	String	No	How to enable hive metastore. Set default if using the cluster’s metastore service, and set mysql if using the user’s mysql services. Default setting: default
host	String	No	mysql server address, which must be provided if metastore selects mysql
port	int	No	mysql server port, which must be provided if metastore selects mysql
database	String	No	Name of database used by metastore, which must be provided if metastore selects mysql
userName	String	No	User name used to connect mysql server, which must be provided if metastore selects mysql
password	String	No	Password used to connect mysql server, which must be provided if metastore selects mysql

The hbase component’s properties are defined as follows:

Parameter Name	Parameter Type	Required	Parameter Description
backupEnabled	Boolean	No	Whether to enable hbase backup function. The default is false
backupLocation	String	No	BOS path for hbase backup, which must be provided if backupEnabled is true
backupIntervalInMinutes	Int	No	Backup interval, which must be provided if backupEnabled is true
backupStartDateTime	String	No	String conforming to [Date and Time Format Constraints](#Date and Time Format Constraints), which must be provided if backupEnabled is true
restoreEnabled	Boolean	No	true means restoring from hbase backup, and false means enabling a new hbase. The default is false
restoreLocation	String	No	BOS path used to restore hbase backup files, which must be provided if restoreEnabled is true
restoreVersion	String	No	Version of backup files when the user restores hbase, which must be provided if restoreEnabled is true

The pig components and hue components do not need properties.

StepConfig Data Structure

Parameter Name	Parameter Type	Required	Parameter Description
actionOnFailure	String	Yes	[Step Failure Policy](#Step Failure Policy)
properties	Object	Yes	[Step Description](#Step Description), depending on the step type
type	String	Yes	[Step Type](#Step Type)
name	String	No	Step name, which is my-step by default

Step Description

Streaming step’s properties are defined as follows:

Parameter Name	Parameter Type	Required	Parameter Description
input	String	Yes	Input path
mapper	String	Yes	mapper program
output	String	Yes	Output path
arguments	String	No	hadoop streaming execution parameters
reducer	String	No	reducer program

Hive step’s properties are defined as follows:

Parameter Name	Parameter Type	Required	Parameter Description
script	String	Yes	Storage path of hql script on the BOS
arguments	String	No	hql script execution parameters
input	String	No	Predefined input path, which is referred to via ${INPUT} in the hql script
output	String	No	Predefined output path, which is referred to via ${OUTPUT} in the hql script

Pig step’s properties are defined as follows:

Parameter Name	Parameter Type	Required	Parameter Description
script	String	Yes	Storage path of pig script on the BOS
arguments	String	No	pig script execution parameters
input	String	No	Predefined input path, which is referred to via ${INPUT} in the pig script
output	String	No	Predefined output path, which is referred to via ${OUTPUT} in the pig script

Java step’s properties are defined as follows:

Parameter Name	Parameter Type	Required	Parameter Description
jar	String	Yes	Storage path of custom java program on the BOS
mainClass	String	Yes	Main entry of custom java program
arguments	String	No	Executive parameters of custom java program

Spark step’s properties are defined as follows:

Parameter Name	Parameter Type	Required	Parameter Description
jar	String	Yes	Storage path of custom program on the BOS
submitOptions	String	Yes	Parameters of spark-submit script
arguments	String	No	Execution parameters of custom program

ModifyInstanceGroupConfig Data Structure

Parameter Name	Parameter Type	Required	Parameter Description
id	String	Yes	Instance group id
instanceCount	int	Yes	Number of instances in the modified instance group

InstanceGroup Data Structure

Parameter Name	Parameter Type	Parameter Description
id	String	The instance group ID is a fixed-length string which only contains uppercase and lowercase letters, numbers, hyphens (-), and underlines (_)
instanceType	String	Type of virtual machines in the instance group; for more information, please see [Instance Specifications](#Instance Specifications)
name	String	Instance group name
type	String	Master, Core or Task; for more information, please see [Instance Group Type](#Instance Group Type)
requestedInstanceCount	int	Number of virtual machines in the instance group

Instance Data Structure

Parameter Name	Parameter Type	Parameter Description
id	String	Instance ID
privateIpAddress	String	Instance private ip
publicIpAddress	String	Instance public ip, and this field is a blank string in all nodes other than master nodes

CdsItem Data Structure

Parameter Name	Parameter Type	Parameter Description
sizeInGB	int	Disk size
mediumType	String	Disk medium type: premium_ssd (ssd cloud disk), ssd (high-performance cloud disk)