百度智能云

All Product Document

          MapReduce

          Create Cluster

          BMR provides two ways to create clusters: the first is to create custom clusters, and the second is to create clusters by using predefined system templates.

          Create Custom Cluster

          1.Log in to the console, and then select "Product Service > Baidu MapReduce (BMR)" to enter the cluster list page.

          2.(Optional) Select the region. For the region description, please see Region Selection Introduction. Clusters created in different regions are independent of each other.

          3.Click "Create clusters" to enter the cluster creation page.

          4.Choose the payment mode:

          • Prepaid: Payment of orders during the billing cycle of a natural year or natural month, payment before service, and lower price than postpayment.
          • Post-paid: Real-time billing by minutes, deduction by hours, and service before payment.

          5.In "Basic configuration" section, you can refer to following instructions to configure necessary information of cluster:

          • Cluster name: Enter the cluster name.
          • Administrator password: Used for SSH remote connection to cluster and login to Hue and other web page applications.
          • Confirm administrator password: Re-enter administrator password.
          • Short message switch: It is on by default.
          • Log: It is on by default. Enter the log address (i.e., BOS path) when it is on. Data written in BOS can be used for the presentation of job log, diagnosis and tuning of cluster, and high availability guarantee of cluster. You must have write permission to the address. Click "Create Bucket" if you do not create a bucket on BOS, in other words, if bucket-name does not exist. For more information, please see Create Bucket.
          • Automatic termination: It is off by default. After processing all data, the cluster keeps running and needs manual release. When the function is on, the cluster is terminated after processing data. The temporary cluster is created.
          • Timeout setting: The function is frozen by default, and it is activated after the automatic termination function is on. After timeout function is activated, if cluster running time is longer than timeout set by you, the system automatically reminds you and processes cluster according to "Action After Timeout" setup.

          6.In "Basic setting of the cluster" section, you can refer to following configuration instructions:

          • Image version: Select a suitable image version.
          • Built-in template/Optional application: You can use "Built-in Template" to quickly add applications corresponding to Hadoop, HBase, and Druid, and you can add custom applications.

          Note:

          1. Relational storage fees are not charged if you select default as Metastore when adding hive and druid applications. Fees are charged based on RDS standards if you select MySQL as Metastore. RDS instance means the created RDS instance, and you need to input the database’s service address, service port, database, user name, and password information.
          2. Hbase in 2.0.0 image template supports to mount BOS as underlying storage, allowing you to obtain higher data reliability, service availability, and elastic expandability of storage. Please be noted that, during the configuration of the BOS path, the directory at any level under the path is not configured to be Hbase’s storage path. In other words, the directory at any level under the path has no Hbase metadata, or the Hbase services can fail.
          • Security mode: Whether to enable the cluster’s Kerberos authentication function. For more information, please see Cluster Security Mode.
          • High availability: After it is enabled, the cluster has 2 master nodes to support the high availability of Resource Manager, Name Node, and Kerberos.

          7.In the "Node Configuration" section, you can select the type of node configuration or modify the number of nodes.

          Note: Maximum 5 active clusters for a single user in a single region. Setup range of nodes in a single cluster:

          • 1 MASTER node for the non-high-availability cluster and 2 MASTER nodes for the high-availability cluster.
          • 3 CORE nodes by default, 2 at minimum and 20 at maximum, and ticket application for more than 20 nodes.
          • 0 Task node by default, 50 at maximum, and ticket application for more than 50 nodes. The console allows you to add a group of Task nodes for a cluster, and you need to plan the setup of packages.
          • 0 Client node by default, 10 at maximum, and ticket application for more than 10 nodes. The console allows you to add a group of Client nodes for a cluster, and you need to plan the setup of packages.

          8.By clicking "Complete", you can see the created cluster is "Initializing" in the cluster list page. The cluster is created successfully when the cluster status changes to "Waiting" after a moment.

          Create Cluster Templates

          1.In "Product Service>Baidu MapReduce-Cluster Template" page, click "Create Template" to enter the template creation page.

          Note:

          Create the postpaid cluster template, but not the prepaid cluster template.

          2.Complete the "Basic configuration", "Basic setting of the cluster" and "Node Configuration". For more information, please see steps 5, 6, and 7 of Create Custom Clusters.

          3.Click "Complete" to create the template successfully. View the created template in the cluster template page.

          image.png

          Create Clusters by Using System Predefined Templates

          1.Log in to the console, select "Product Service->Baidu MapReduce BMR", and click "Cluster Template" to enter the cluster template page.

          2.The cluster template page provides three system-predefined cluster templates: spark, hive, and hadoop. Select a template and then click the "Create Cluster" button for such a template to enter the cluster creation page.

          3.You can keep the default template configurations or modify the cluster setups. After setting the cluster administrator password, click "Complete" to start the creation of cluster.

          4.You can see the created cluster is "Initializing" in the cluster list page. The cluster is created successfully when the cluster status changes to "Waiting" after a moment.

          Creates private DNS cluster

          Private DNS is an independent network product provided by Baidu AI Cloud, which can realize automatic resolution between domain name (host name ) and IP. Mutual access between different clusters and between BMR cluster and BCC need to be achieved by domain name (host name ). This feature can be used in data transmission. For details, refer to the Cloud DNS.

          The BMR cluster with private DNS enabled will automatically associate with private DNS products and will automatically join the private domain named "novalocal" (if no such private domain, it will be created automatically ).

          1.Log in to the console and select "Product Services > Baidu MapReduce BMR" to go to the Cluster List page. 2.Click "Create Cluster" to go to the Create Cluster page. 3.In cluster basic settings, click the "Private DNS" button.

          Notes

          1. With this feature enabled, Cloud DNS (private DNS ) products will be charged based on used size. At present, Cloud DNS (private DNS ) is for free in the Open Beta Test stage. Refer to the product pricing for details.
          2. Temporarily, this feature cannot be turned off once it is enabled. Please make reasonable cluster planning. When releasing a cluster, in order to ensure the normal use of other clusters (if any ), the associated private DNS domain will not automatically release. Please go to Cloud DNS console to release related resources after confirming that no other cluster is using this feature, so as to avoid unnecessary billing.

          3.When creating a BMR cluster using a sub-user, the private DNS feature need to be enabled. You need to enter the Identity and Access Management (IAM) Center to grant this user with the "LDFullControlPolicy" privileges. Otherwise, this feature will not be available.

          4.This feature is still in the whitelist Open Beta Test stage. To use this feature, please submit a Ticket to apply for the Whitelist privileges for Baidu AI Cloud resolution (private DNS resolution ).

          Previous
          Introduction to Open-Source Components
          Next
          Manage Existing Clusters