What if the step fails?

In the step list of the cluster, you can find step failure logs of syslog, stderr, and stdout. The syslog log records step running operation, the stderr log records the reason for step failure, and the stdout log records information output during step running. By viewing the stderr log, you can find out the reason for step failure and repair it and then restart step.

Why does the step fail?

1: Step's input directory does not exist in BOS or output directory exists, which makes reduce tasks fail to read or write data, leading to step failure.

Solution: Please make sure the input directory exists in BOS and output directory does not exist.

2: The error exists in the step.

Solution: If you submit Custom JAR, Spark, or Pig step, your custom parameters do not conform to specifications. You can query the task log and find out the error to repair the step.

3: Error exists in input parameters.

Solution: The spelling error exists in the keyword of input parameters. You can query the task log and find out the error to repair the step.

Why does step submission fail?

1: Parameter setup does not conform to the standard format.

Solution: Based on the error message returned by page, you can refer to parameter configurations for different steps described in beginner's guide, and make modifications.

2: The number of steps is more than 256.

Solution: Follow instructions to create a cluster, and add steps to cluster.

Why is the cluster automatically terminated?

It indicates some accidents happen when the cluster processes steps, for example, the input bos address does not exist, or the user has no permission to process the address. The user selects cluster destruction in "Action After Failure" during the setup of the cluster, so the cluster is automatically terminated after the step fails. This situation generally happens when the user's step setup does not conform to specifications. You can check the errlog to reset the step.