Create LoadBalancer_Service Through YAML
This document introduces in detail how to create a LoadBalancer service under CCE.
Note: The following annotation may not be effective for versions below 1.16.3. Contact the administrator for handling the ticket.
Kubernetes official tutorial: Services
Quick Start
When a user creates a LoadBalancer service, CCE creates a BLB and binds the BLB to an EIP.
Take the creation of a simple Nginx for example:
---
kind: Service
apiVersion: v1
metadata:
name: nginx-service
spec:
selector:
app: nginx
type: LoadBalancer
ports:
- name: nginx-port
port: 80
targetPort: 80
protocol: TCP
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
selector:
matchLabels:
app: nginx
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80(1) Create
$ kubectl create -f nginx.yaml
(2) Query EIP
IP 8.8.8.8 is the EIP of Nginx.
$ kubectl get svc
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-service 1.1.1.1 8.8.8.8 80:30274/TCP 5m(3) Query BLB
$ kubectl get svc nginx-service -o jsonpath={.metadata.annotations}
map[service.beta.kubernetes.io/cce-load-balancer-id:lb-xxxxxx]lb-xxxxxx is the id of the BLB for this Service.
(4) Access Test
$ curl -i http://8.8.8.8
LoadBalancer Service with Three External Traffic Policies
CCE provides LoadBalancer Service with Three External Traffic Policies, which are called Cluster mode, Local mode, and mode of directly connecting LB with Pod in this document.
For the Cluster LB Service, when it receives the data packet, BLB sends the data packet to a node in the cluster. The node forwards it to a Pod in the cluster for the second time. The host node of the target Pod and the node to forward data packet may not be the same one.
When the local LB Service receives the data packet, BLB sends the data packet to the node where the target Pod is deployed, and then the node forwards it to its Pod.
For the LB Service in the LB direct connection Pod mode, when it receives the data packet, the load balancer directly sends the data packet to each Pod. Compared with the first two modes, one forwarding operation on nodes is saved in this mode.
In the VPC routing network mode, the path to forward the data packet of the LoadBalancer Service in three modes is shown in the figure:
In the VPC-CNI network mode, the path to forward the data packet of the LoadBalancer Service in three modes is shown in the figure:
Cluster Mode
If you want to use a Service in Cluster mode, you should specify externalTrafficPolicy: Cluster when creating a Service, as shown in the example below:
apiVersion: v1
kind: Service
metadata:
name: service-example-cluster
annotations:
prometheus.io/scrape: "true"
spec:
selector:
app: nginx
type: LoadBalancer
externalTrafficPolicy: Cluster
sessionAffinity: None
ports:
- name: nginx
protocol: TCP
port: 80
targetPort: 80Local Mode
If you want to use a Service in Cluster mode, you should specify externalTrafficPolicy: Local when creating a Service, as shown in the example below:
apiVersion: v1
kind: Service
metadata:
name: service-example-local
annotations:
prometheus.io/scrape: "true"
spec:
selector:
app: nginx
type: LoadBalancer
externalTrafficPolicy: Local
sessionAffinity: None
ports:
- name: nginx
protocol: TCP
port: 80
targetPort: 80LB Directly-connected to Pod
To use the Service by directly connecting LB to Pod, you should add Annotation service.beta.kubernetes.io/cce-load-balancer-backend-type: "eni" to create the service, as shown in the example below:
For detailed instructions for use, refer to Use Mode of Directly-connected Pod for LoadBalancer Service.md.
apiVersion: v1
kind: Service
metadata:
name: service-example-direct
annotations:
prometheus.io/scrape: "true"
service.beta.kubernetes.io/cce-load-balancer-backend-type: "eni"
spec:
selector:
app: nginx
type: LoadBalancer
sessionAffinity: None
ports:
- name: nginx
protocol: TCP
port: 80
targetPort: 80Advanced Configuration
Fixed EIP
When the user deletes the Service and recreates it, the EIP changes. Thus, all other services that depend on this IP need to be changed, and CCE provides a way to fix this EIP.
Solutions for fixed EIP:
(1) The user purchases an EIP instance on Baidu AI Cloud in advance
(2) When creating the Service, set loadBalancerIP to this EIP
(3) Create Service. At this moment, EXTERNAL-IP is this EIP
(4) Delete Service. CCE only unbinds this EIP but does not release this EIP. The user can continue to use the EIP next time.
The example is as follows:
---
kind: Service
apiVersion: v1
metadata:
name: nginx-service-eip-with-load-balancer-ip
spec:
selector:
app: nginx-eip-with-load-balancer-ip
type: LoadBalancer
loadBalancerIP: 8.8.8.8
ports:
- name: nginx-port
port: 80
targetPort: 80
protocol: TCP
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment-eip-with-load-balancer-ip
spec:
selector:
matchLabels:
app: nginx-eip-with-load-balancer-ip
replicas: 1
template:
metadata:
labels:
app: nginx-eip-with-load-balancer-ip
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80The EXTERNAL-IP found in this way is the EIP:
kubectl get svc nginx-service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-service-eip-with-loadBalancerIP LoadBalancer 1.1.1.1 8.8.8.8 80:30601/TCP 1mDo Not Allocate EIP (i.e., BLB in VPC)
When the user uses it:
(1) Set Service.Spec.Type=LoadBalancer;
(2) Add annotations to the Service, i.e., service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true".
The example is as follows:
---
kind: Service
apiVersion: v1
metadata:
name: nginx-service-blb-internal-vpc
annotations:
service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true"
spec:
selector:
app: nginx-blb-internal-vpc
type: LoadBalancer
ports:
- name: nginx-port
port: 80
targetPort: 80
protocol: TCP
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment-blb-internal-vpc
spec:
selector:
matchLabels:
app: nginx-blb-internal-vpc
replicas: 1
template:
metadata:
labels:
app: nginx-blb-internal-vpc
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80The EXTERNAL-IP found in this way can only be accessed in the VPC:
kubectl get svc nginx-service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-service-blb-internal-vpc LoadBalancer 1.1.1.1 2.2.2.2 80:30601/TCP 1mNote: This intranet BLB can only be used normally between clusters in a VPC. A problem occurs when using the intranet BLB in the same cluster. Recommend you use the Service's ClusterIP directly in the same cluster.
Custom EIP Configuration
EIP Supports Configuration Types:
Prepaid
| Item | Limit |
|---|---|
| Public Network Bandwidth | 1-200Mbps，Int |
| Duration | [1,2,3,4,5,6,7,8,9,12,24,36], time unit, month |
Postpaid
| Billing mode | Public Network Bandwidth | Examples of Fees |
|---|---|---|
| ByTraffic | 1~200Mbps，Int | Configuration cost: ￥0.00032/minute; Traffic Cost: ￥0.76/GB |
| ByBandwidth | 1-200Mbps，Int | Configuration cost (1Mbps as an example): ￥0.00094/minute |
How To Use: Set the corresponding Annotation when creating the Service as follows:
// Payment mode. By default, the payment mode is Postpaid. Optional payment modes: Postpaid and Prepaid modes.
ice.beta.kubernetes.io/cce-elastic-ip-bandwidth-in-mbps:
// Billing mode. The default billing mode is ByTraffic. The optional billing mode includes ByBandwidth and ByBandwidth.
service.beta.kubernetes.io/cce-elastic-ip-billing-method:"ByTraffic"
// The public network bandwidth is expressed in Mbps, which defaults to 100. For the prepay and bandwidth EIPs, the bandwidth value is limited to an integer within a range of 1-200. For the traffic EIP, the bandwidth value is limited to an integer within a range of 1-200.
service.beta.kubernetes.io/cce-elastic-ip-bandwidth-in-mbps:"100"
// For the prepaid mode, the duration must be set to [1,2,3,4,5,6,7,8,9,12,24,36] months (unit). For the postpaid mode, this setting is invalid.
service.beta.kubernetes.io/cce-elastic-ip-reservation-length:"36"Post-payment example:
kind: Service
apiVersion: v1
metadata:
name: nginx-service
annotations:
service.beta.kubernetes.io/cce-elastic-ip-payment-timing: "Postpaid"
service.beta.kubernetes.io/cce-elastic-ip-billing-method: "ByTraffic"
service.beta.kubernetes.io/cce-elastic-ip-bandwidth-in-mbps: "200"
spec:
selector:
app: nginx
type: LoadBalancer
ports:
- name: http
port: 80
targetPort: 80Prepayment example (make sure the balance is sufficient. Otherwise, it becomes invalidated):
kind: Service
apiVersion: v1
metadata:
name: nginx-service
annotations:
service.beta.kubernetes.io/cce-elastic-ip-payment-timing: "Prepaid"
service.beta.kubernetes.io/cce-elastic-ip-bandwidth-in-mbps: "10"
service.beta.kubernetes.io/cce-elastic-ip-reservation-length:"1"
spec:
selector:
app: nginx
type: LoadBalancer
ports:
- name: http
port: 80
targetPort: 80Notes Default configuration: The default configuration is postpaid + byTraffic + 100M bandwidth.
Fixed EIP: You cannot update the configuration of the fixed EIP. Thus, go to the console to modify it by yourself.
The user updates the configuration of the Service EIP (i.e., manually edit the annotation). The configuration you can update includes public network bandwidth.
Prepaid
(1) In the prepaid mode, automatic renewal is not currently supported due to the limitation of the EIP API. Thus, the user needs to renew on the console.
(2) For the prepayment, you do not need to set the billing method.
(3) When you delete the Service, the prepaid EIP is not released but is released after expiration.
UDP-Service
Modify spec.ports.protocol to UDP to use the feature of the UDP Service. The example is as follows:
---
apiVersion: v1
kind: Service
metadata:
name: udp-server-demo-svc
labels:
app: udp-server-demo
spec:
type: LoadBalancer
ports:
- name: udp-server-demo-port
port: 3005
targetPort: 3005
protocol: UDP
selector:
app: udp-server-demo
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: udp-server-demo
labels:
app: udp-server-demo
spec:
replicas: 1
selector:
matchLabels:
app: udp-server-demo
template:
metadata:
labels:
app: udp-server-demo
spec:
containers:
- name: udp-server-demo
image: hub.baidubce.com/jpaas-public/udp-server-demo:latest
ports:
- containerPort: 3005
protocol: UDP(1) Deploy the udp test service
$ kubectl apply -f udp.yaml(2) The UDP Service is successfully created.
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 172.16.0.1 <none> 443/TCP 6h
udp-server-demo-svc LoadBalancer 172.16.122.139 10.10.10.10 3005:31441/UDP 1m(3) View the service log.
$ kubectl logs -f udp-server-demo-6fdf5d796f-h6595
Received: HealthCheck
Get Health Check, response OK
Received: HealthCheck
Get Health Check, response OK
Received: HealthCheck
Get Health Check, response OKNote: According to the requirements for Baidu AI Cloud Load Balance (BLB), for the services that listen to UDP, the UDP health check must be passed before BLB forwards the traffic to the backend. Therefore, the user’s backend UDP service needs to respond to the health check string. See details: Introduction of UDP Health Check.
Specify Subnet for Service BLB
To facilitate users to manage network-related resources, you can specify a subnet for BLB when creating a Service. When users use the subnet:
(1) Set Service.Spec.Type=LoadBalancer
(2) Add annotations to the Service and specify the subnet ID, namely service.beta.kubernetes.io/cce-load-balancer-subnet-id: "sbn-*"
The example is as follows:
---
kind: Service
apiVersion: v1
metadata:
name: nginx-service-blb-subnet-id
annotations:
service.beta.kubernetes.io/cce-load-balancer-subnet-id: "sbn-123456"
spec:
selector:
app: nginx
type: LoadBalancer
ports:
- name: nginx-port
port: 80
targetPort: 80
protocol: TCP
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment-blb-subnet-id
spec:
selector:
matchLabels:
app: nginx
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80After the service deployment, you can go to the Baidu AI Cloud BLB page to query the subnet where the created BLB is located for verification. Note: Only the subnet specified when creating the Service is supported. After the service creation, you cannot edit the Service to add or modify the annotation.
Create LB Service for Specified BLB
Add annotations to the Service, namely service.beta.kubernetes.io/cce-load-balancer-id: "lb-xxxxxxxx". Specify the BLB of the LB. The example is as follows:
kind: Service
apiVersion: v1
metadata:
name: nginx-service-blb-assigned-id
annotations:
service.beta.kubernetes.io/cce-load-balancer-id: "xxxxxx"
spec:
selector:
app: nginx
type: LoadBalancer
ports:
- name: nginx-port
port: 80
targetPort: 80
protocol: TCPBLB is not retained during the deletion. If you need to retain the BLB, use Delete Service to retain the BLB annotation.
Delete Service and Retain BLB
Add annotations to the Service, namely service.beta.kubernetes.io/cce-load-balancer-reserve-lb: "true". Specify the BLB is retained when deleting the Service. The example is as follows:
kind: Service
apiVersion: v1
metadata:
name: nginx-service-blb-reserve-lb
annotations:
service.beta.kubernetes.io/cce-load-balancer-reserve-lb: "true"
spec:
selector:
app: nginx
type: LoadBalancer
ports:
- name: nginx-port
port: 80
targetPort: 80
protocol: TCP