CCE集群使用Jenkins持续发布

更新时间：2025-08-21

概览

持续构建与发布是企业研发工作中必不可少的一个步骤，目前大多公司都采用 Jenkins 集群来搭建符合需求的 CI/CD 流程，Jenkins的持续发布流程可以跟Kubernetes集群更好的对接，更好的发挥它的部署优势，这篇文档可以指定用户把Jenkins发布流程跟CCE集群集成。

操作步骤

1. 设置 Jenkins 存储目录

在 Kubenetes 环境下所起的应用都是一个个Docker镜像，为了保证应用重启的情况下数据安全，所以需要将Jenkins的数据目录持久化到存储中。这里用的是CCE提供的多种持久化存储之一，方便在Kubernetes环境下应用启动节点转义数据一致。当然也可以选择存储到本地，但是为了保证应用数据一致，需要设置Jenkins固定到某一 Kubernetes节点。

参考容器引擎CCE-操作指南-存储管理一节。

选取任意一种方式部署生成PVC，记录下PVC的名字。

2. 部署 Jenkins Server 到 Kubernetes

service-account.yaml

Plain Text

1---
2apiVersion: v1
3kind: ServiceAccount
4metadata:
5  name: jenkins
6 
7---
8kind: Role
9apiVersion: rbac.authorization.k8s.io/v1beta1
10metadata:
11  name: jenkins
12rules:
13- apiGroups: [""]
14  resources: ["pods"]
15  verbs: ["create","delete","get","list","patch","update","watch"]
16- apiGroups: [""]
17  resources: ["pods/exec"]
18  verbs: ["create","delete","get","list","patch","update","watch"]
19- apiGroups: [""]
20  resources: ["pods/log"]
21  verbs: ["get","list","watch"]
22- apiGroups: [""]
23  resources: ["events"]
24  verbs: ["watch"]
25- apiGroups: [""]
26  resources: ["secrets"]
27  verbs: ["get"]
28 
29---
30apiVersion: rbac.authorization.k8s.io/v1beta1
31kind: RoleBinding
32metadata:
33  name: jenkins
34roleRef:
35  apiGroup: rbac.authorization.k8s.io
36  kind: Role
37  name: jenkins
38subjects:
39- kind: ServiceAccount
40  name: jenkins

jenkins.yaml

Plain Text

1# jenkins
2 
3---
4apiVersion: apps/v1
5kind: StatefulSet
6metadata:
7  name: jenkins
8  labels:
9    name: jenkins
10spec:
11  selector:
12    matchLabels:
13      name: jenkins
14  serviceName: jenkins
15  replicas: 1
16  updateStrategy:
17    type: RollingUpdate
18  template:
19    metadata:
20      name: jenkins
21      labels:
22        name: jenkins
23    spec:
24      terminationGracePeriodSeconds: 10
25      serviceAccountName: jenkins
26      containers:
27        - name: jenkins
28          image: hub.baidubce.com/jpaas-public/jenkins-github:v0
29          imagePullPolicy: Always
30          ports:
31            - containerPort: 8080
32            - containerPort: 50000
33          resources:
34            limits:
35              cpu: 1
36              memory: 1Gi
37            requests:
38              cpu: 0.5
39              memory: 500Mi
40          env:
41            - name: LIMITS_MEMORY
42              valueFrom:
43                resourceFieldRef:
44                  resource: limits.memory
45                  divisor: 1Mi
46            - name: JAVA_OPTS
47              # value: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
48              value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
49          volumeMounts:
50            - name: jenkins-home
51              mountPath: /var/jenkins_home
52          livenessProbe:
53            httpGet:
54              path: /login
55              port: 8080
56            initialDelaySeconds: 60
57            timeoutSeconds: 5
58            failureThreshold: 12 # ~2 minutes
59          readinessProbe:
60            httpGet:
61              path: /login
62              port: 8080
63            initialDelaySeconds: 60
64            timeoutSeconds: 5
65            failureThreshold: 12 # ~2 minutes
66      securityContext:
67        fsGroup: 1000
68      volumes:
69        - name: jenkins-home
70          persistentVolumeClaim:
71            claimName: myjenkinspvc
72 
73---
74apiVersion: v1
75kind: Service
76metadata:
77  name: jenkins
78spec:
79  type: NodePort
80  selector:
81    name: jenkins
82  # ensure the client ip is propagated to avoid the invalid crumb issue when using LoadBalancer (k8s >=1.7)
83  #externalTrafficPolicy: Local
84  ports:
85    - name: http
86      port: 80
87      targetPort: 8080
88      protocol: TCP
89    - name: agent
90      port: 50000
91      protocol: TCP

注意：

jenkins.yaml文件中的字段claimName需要改成1、设置Jenkins存储目录中生成的PVC名字

在CCE Kubernetes集群中执行以下命令

Plain Text

1kubectl create -f service-account.yaml
2kubectl create -f jenkins.yaml

依次生成以下内容代表创建成功

3. 初始化配置 Jenkins

此时Jenkins Master服务已经部署启动起来了，并且将端口暴露到80:30427，50000:31598，此时可以通过浏览器打开http://<Node_IP>:30427访问 Jenkins页面了。

在浏览器上完成Jenkins的初始化插件安装过程，并配置管理员账户信息，这里忽略过程，初始化完成后界面如下：

注意：

初始化过程中，让输入 /var/jenkins_home/secret/initialAdminPassword初始密码时，可以直接到挂载的PVC持久化目录里面读取，或者直接进入到容器的内部进行获取

kubectl exec -it jenkins-0 cat /var/jenkins_home/secrets/initialAdminPassword

4. Jenkins 安装 Kubernetes Plugin 插件

管理员账户登录Jenkins Master页面，点击 "系统管理" -> "插件管理" —> "可选插件" —> "Kubernetes" 勾选安装即可。

安装完毕后，点击"系统管理" —> "系统设置" —> "新增一个云" —> 选择 "Kubernetes"，然后填写Kubernetes和Jenkins配置信息。

说明：

1、Name处默认为kubernetes，也可以修改为其他名称，如果这里修改了，下边在执行Job时指定podTemplate()参数cloud为其对应名称，否则会找不到，cloud默认值取：kubernetes
2、Kubernetes URL处填写了https://kubernetes.default这里填写了 Kubernetes Service对应的DNS记录，通过该DNS记录可以解析成该Service的Cluster IP。

注意： 也可以填写https://kubernetes.default.svc.cluster.local完整DNS记录，因为它要符合<svc_name>.<namespace_name>.svc.cluster.local的命名方式，或者直接填写外部Kubernetes的地址https://<ClusterIP>:<Ports>。

3、Jenkins URL处填写了http://jenkins.default，跟上边类似，也是使用 Jenkins Service对应的DNS记录, 同时也可以用http://<ClusterIP>:<Node_Port>方式，例如我这里可以填http://x.x.x.x:30427也是没有问题的，这里的30427就是对外暴露的NodePort。
4、配置完毕，可以点击"Test Connection"按钮测试是否能够连接的到 Kubernetes，如果显示Connection test successful则表示连接成功，配置没有问题

5. 非集群内的 Jenkins 连接 Kubernetes 集群

填写 Kubernetes 配置内容

以一个kubeconfig文件为例

Plain Text

1apiVersion: v1
2clusters:
3- cluster:
4    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURwRENDQW95Z0F3SUJBZ0lVUldSdmNwRkxNaTVaUFZJUVllL2o2WkxsZlJJd0RRWUpLb1pJaHZjTkFRRUwKQlFBd2FqRUxNQWtHQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbAphVXBwYm1jeEREQUtCZ05WQkFvVEEyczRjekVVTUJJR0ExVUVDeE1MWTJ4dmRXUnVZWFJwZG1VeEV6QVJCZ05WCkJBTVRDbXQxWW1WeWJtVjBaWE13SGhjTk1qQXdOVEU0TURjeE5qQXdXaGNOTWpVd05URTNNRGN4TmpBd1dqQnEKTVFzd0NRWURWUVFHRXdKRFRqRVFNQTRHQTFVRUNCTUhRbVZwU21sdVp6RVFNQTRHQTFVRUJ4TUhRbVZwU21sdQpaekVNTUFvR0ExVUVDaE1EYXpoek1SUXdFZ1lEVlFRTEV3dGpiRzkxWkc1aGRHbDJaVEVUTUJFR0ExVUVBeE1LCmEzVmlaWEp1WlhSbGN6Q0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUtuNmNNMWYKQzlGMTFVVG1jVFlEQmZJMGlBUnJ2N3RtUEhDNU02NHBQNlViRVhIQ2lMNEFIUEVHK29tdVZKWHZ6MExhNVZRagp2SjZUblc0K2h4UjNuT2pGSmhGbldVZDcrWUtBM1Fic05qUVNybFNLTVhqRVlTSTA2M3NGS1YzNFZNUERXR3ByClVvVlZWZjduVmVkY0FhRTdzeFg3cXFOeERDS3Fjc2cxWCtDNFFrK01zaExKaUdyRnNnRC8rOHNUVkRzVzRoTEMKZ01zZ1R5WlVwRDlmM2hBTXQ0dzduV3RiWURsOFlvZnhEcU9tYndpVEx3VlNyaXAvZHlVeW9BZXhxbFFWaUgrMwpTNXY5cXRnTXVRRjhUMVNPUkwzcldsMUNYR0JPU3k0YTVBenA0dUx6TTlnbjJzdzZIU3gySG1aZlpxUlI1SXdICkFnQU9ydStFZ1lXVmNoOENBd0VBQWFOQ01FQXdEZ1lEVlIwUEFRSC9CQVFEQWdFR01BOEdBMVVkRXdFQi93UUYKTUFNQkFmOHdIUVlEVlIwT0JCWUVGSjBNOXlyc2F5SXEzZnVuYkhWWHJiZUZZL3RBTUEwR0NTcUdTSWIzRFFFQgpDd1VBQTRJQkFRQkE0UzBoVkpyOTBmNjNPU0tRMzVrUGNvTE9ObTg0TVNUbkI1OHE3alJHOHBtRHVvc09TUXUwCjZpUE9CZVBMOE92eEpObzhRcHBXOWJCR1N1ZFRKMG5CUnMyTmFjWU1BM0FqeUM0Y09nOFFJNGxHcUJWQllvVEcKa0I0cVlsYjl4dVJ2bnBJenVUWkd0RkVYQkpkZXFGZzZzSno2THRIZzFiUmM2cGJGZThuVUZZYnRSMzRPeGl2Qgp5WUNUKzNSbEpxTmhXcjlJK2djNDhMeXNheTBnL3BmQ1lsSzdXQm8rSWFZTXF1Z2Zmc2RQS2dISkU0Tm0xei9pCmlsNnhYMWV3R1RYMEFsMENKN2tNeitHa3hZalFHcHY1UG5iNzRpOHZZMm5uODVhUnliZVZjeENMZkhTazBTN2kKL0hjTXRkRjdxMVE5bndqMlhiclMxRmRCTEVSWGFGQTkKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
5    server: https://1.1.1.1:6443
6  name: kubernetes
7contexts:
8- context:
9    cluster: kubernetes
10    user: kubernetes-admin
11  name: kubernetes-admin@kubernetes
12current-context: kubernetes-admin@kubernetes
13kind: Config
14preferences: {}
15users:
16- name: kubernetes-admin
17  user:
18    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUsdafZJQ0FURS0tLS0tCUJBZ0lVQlRMK3lWNXdNbjNvamdkdzRKSmFBa0RpNTZrd0RRWUpLb1pJaHZjTkFRRUwKQlFBd2FqRUxNQWtHQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbAphVXBwYm1jeEREQUtCZ05WQkFvVEEyczRjekVVTUJJR0ExVUVDeE1MWTJ4dmRXUnVZWFJwZG1VeEV6QVJCZ05WCkJBTVRDbXQxWW1WeWJtVjBaWE13SGhjTk1qQXdOVEU0TURjeE5qQXdXaGNOTXpBd05URTJNRGN4TmpBd1dqQjcKTVFzd0NRWURWUVFHRXdKRFRqRVFNQTRHQTFVRUNCTUhRbVZwU21sdVp6RVFNQTRHQTFVRUJ4TUhRbVZwU21sdQpaekVYTUJVR0ExVUVDaE1PYzNsemRHVnRPbTFoYzNSbGNuTXhGREFTQmdOVkJBc1RDMk5zYjNWa2JtRjBhWFpsCk1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrYldsdU1JSUJJakFOQmdrcWhraUc5dzBCQVFFRkFBT0MKQVE4QU1JSUJDZ0tDQVFFQTErQjh2aXJGU0ZTQVJEQk4yOWxzRHN2WDlxaHJRTU82WnlCK0E5bWNybUtQZG5NOQozdU80T3FGQ1RjNktxTVNDQStDUHVKdlQwZXFJM29UUWZRMHZmWW40UndVNFhIZ21IbWQ2Nk50SXlQZUdzZGtGClRJc1FxaElzVm1VQS9ZNXBEb2h1Yk9OQWlGbDFOaEliUTNqU3JmMy9MYzVLY3RYM2ZOSGhxOE1hblpYYkVmRUsKY1NsdVI5Um4rQkdET1pubGNMZTkzRWRxME9EeFRsaXV6aFNYektUQUhyZnRLbGlIS3hyVXVkRlFZcGpHaFgrTgpLT2pRTFAzdkUwNjYzbklibGxZb2trYkxpZUVnd3lkcDNWUFpHSXI2aE5kZnd5bEdpU2lHMmw0N045bG1lVVhoCmt6MjZoWG5OeDZXRzdvL2d1aXBLdUpWeHZmT1NXS0tHWmJyZmhRSURBUUFCbzRISE1JSEVNQTRHQTFVZER3RUIKL3dRRUF3SUZvREFkQmdOVkhTVUVGakFVQmdnckJnRUZCUWNEQVFZSUt3WUJCUVVIQXdJd0RBWURWUjBUQVFILwpCQUl3QURBZEJnTlZIUTRFRmdRVVZUQm5ESjJYRUtCMkRRTlJZY2c5SFlsL2VCQXdId1lEVlIwakJCZ3dGb0FVCm5RejNLdXhySWlyZCs2ZHNkVmV0dDRWaiswQXdSUVlEVlIwUkJENHdQSWNFckJJQUFZY0VaRURtSUljRVpFYWsKeUljRVpFRG1INGNFWkVha3dZY0VaRURtSG9jRVpFYWt2b2NFWkVEbUlZY0VhZ3hkSm9jRVpFSUFiREFOQmdrcQpoa2lHOXcwQkFRc0ZBQU9DQVFFQUNFc2JBOWh6cHp6YWdwL2RMemQremx2anJVQ0dqNVFBQjNmQlN0OEN4Z2loCnhKNzBLeWV5TDdVVExOZDRNd09Tcjd0VGxhMTFmZTE4VkRvRUNDV0NzY0FabXZEK014eVJ2alFaL1NjY29xMGwKRkc5RFlBU3dOL2RhMURaeTY2L0FJdllaWEV1WmxQbGtSNnlrQ1F4MHJNYjJtdVdSemdJTS9MUTNwSFVYenJ4cQo3cUlreEtJeWlHZC8yUDdvcFBtSzMvemJIU2wyVnNnc1pFL2tJb1ZNQ3BYc1NEaml2aEpHNUlya1lyamNSa2JzCmV2cDI2b0M0K0lKNkNYNjh0TDROTkFxNzFoRmh1QTJ1TGhVLzFhc3NOY0w4RFpFNnFvTlFpWUZXQjVheE81NEsKZXlxRkM3dDBTbEVNYTFRVnNlbklwTVlTRmRlYTZselhtN3lXSFVCVEdBPT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
19    client-key-data:LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVkbjRSdlR3Nka0YKVVBL1k1cERvaHViT05BaUZsMU5oSWJRM2pTcmYzL0xjNUtjdFgzZk5IaHE4TWFuWlhiRWZFSwpjU2x1UjlSbitCR0RPWm5sY0xlOTNFZHEwT0R4VGxpdXpoU1h6S1RBSHJmdEtsaUhLeHJVdWRGUVlwakdoWCtOCktPalFMUDN2RTA2NjNuSWJsbFlva2tiTGllRWd3eWRwM1ZQWkdJcjZoTmRmd3lsR2lTaUcybDQ3TjlsbWVVWGgKa3oyNmhYbk54NldHN28vZ3VpcEt1SlZ4dmZPU1dLS0daYnJmaFFJREFRQUJBb0lCQUZRVUhlR2ZIT2xYNkFFbQo0eHd1YVZTMTllNGVtRzlJRERDZ1NoUkx6Q2RyUWI3N2tXeGZPdUN5Ny9VSDdaOWZzZGU2dlo5RUtkTEhTdm1ICnR3QU5nNktjZXZPR1IvWlQ0VnpVSnQzTWttT2JiSDJXTGVjcS9wbU9ySEFWdExZTW9rUkR4T1pwK0RkaXlERUEKQ2xoVUZaSW9yQnQyRGk3OXdQOS9heXFQdWEwSCttK0NpV2JSYzhodkRSZWtIZnd2b2pXTWtzNVNUNkZhSmVzNwpXQW1GNnVTaEs2Y3ovZjV5QjgvNjVmQ0tvYVN2RFlkdk1HVHZ2U3BqRlBoZVlScHRnREVWd2VJMElaRFBXdDQ4CmpnWklpQjVLbmpOdXprc1l6eEUvWnRhV05PS3QxVmtKYTduQ1M2SitOM0JESDFwOE9QVWk0ZVpDVTRzd05kRVMKVDRoV0ZZRUNnWUVBOGthR0Y3aGxmSEhBZkwya3MwNUJNb01mTktOUFFkSkxYV01ZUVk0L0FwYWJlaWhOVzZRSwpabDJ1cjNoRkFZZ3B6ZFp3RkUzbEx5VmhnODZSS05XNVZvSWcwUW5qWEpBc1U5VGt1KzNzVXRqWnFHTkNmdWRHClB1dG1qWFNwR1NvQWQxZDdwQkNkbEdMYWZBd1ArREI3VkFCNHRTQklVd1NHY3hDb2pUOU92T0VDZ1lFQTVCc2gKcnpOTHZJcU81MlpyL2xlZ3BWRC96aFhZWmNhaDhQcFFLNUY1VkJNa3MyalRNL25uVXdvYzl1QXllUlYxbm1lTApNajlXTmI2QkhZNXNRZXZaNUFPQ2pHWGxyL24zd29qSitTTVpyQnlLMVVqdjBVakdXcVlncHBXMUxhM21MREV5CmpGbDBhcnB5MllRSkxoK1VoU3R0c2ZKU25qdVVmTXJVR3pRUTh5VUNnWUIyRDJmSXAxTE5FYUY3Sis3YWNZZlQKMVpHZlZQV0tYYS9jRWkzL3hCRndjWFBTVTFGZkZ0RDZrU3hPMVl6SzhrOXN2dEpmRXBaY0l4c2gzOGRjM3NreQpIcmRmSmpKbEtOeHcvWTE1QnJmaXAwbHBoUFVpWWhFWkdCMGhVWGdWaXlJdkJiSjZnSjVKY09LSEVGbTMxK2hCClJ2bUxTZS8waElBQUVsNFFkb2tvQVFLQmdRQzd6Q3FiVjV3UENmUkZSdW02YU9KMXVJNGlXWkhqbVBsU3NJSzQKbS9oTDQ4YmZmbm9EM01jNmNxVU9DOThDR1V6UXNXYkVZNmpTYnBsV2dCOVkxcGg1UlBxQ0pKSkpvMzc3eGlxaQoxdWNYOEJmTktWTm45b1ozc3paR2NCTE9ITkhYcUZsNWUxeUJVaWVrTlRScHFNNWFKVHNXdWU2VEgzSk1tNkN0CkZOeXZrUUtCZ0VlY05hN2UzNTRJcEtlMGowZ0dzWXZuVXNFOVRKSkRzdmhjYmRhL0E5TEZvV2tRNGFHVmt2SlgKSG5yb29ZWDg2OU1FM2YyaHNxUmVsMng2a1phZWF6ZWNnWld0RzkvR2kzWXRqVWJZVktzU2pWZDVhSkFrM2k1cwpEVWRmMFZUU1QrR0hjRDUzL2NkK3M3WnA1WkFzdWx1czdnQmFPWkdzOUx6VXY5aHIrRzJWCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==

序号1：保持不变既可，默认是kubernetes
序号2：填写kubeconfig文件中的clusters.cluster.server的地址
序号3: 获取kubeconfig文件中的certificate-authority-data的内容并转化成base64 encoded文件

Plain Text

1ecbo xxx | base64 -d > /opt/crt/ca.crt

将ca.crt的内容填写到jenkins kubernetes的Kubernetes服务证书key栏中获取kubeconfig中client-certificate-data和client-key-data的内容并转化成base64 encoded文件

Plain Text

1echo xxxxx== | base64 -d > /opt/crt/client.crt
2echo xxxxx== | base64 -d > /opt/crt/client.key
3 
4 
5# 生成Client P12认证文件cert.pfx，并下载至本地
6openssl pkcs12 -export -out /opt/crt/cert.pfx -inkey /opt/crt/client.key -in /opt/crt/client.crt -certfile /opt/crt/ca.crt
7Enter Export Password:
8Verifying - Enter Export Password:
9 
10# 注：自定义一个password并牢记

序号4: 在云kubernetes中添加凭证

注意： Upload certificate上次刚生成并下载至本地的cert.pfx文件，Password值添加生成cert.pfx文件时输入的密钥,然后在序号4中选择这个证书

最后点击连接测试：出现代表测试成功

6. 测试并验证

好了，通过Kubernetes安装Jenkins Master完毕并且已经配置好了连接，接下来，我们可以配置一个Job测试以下是否会成功发布

pipeline 类型支持

创建一个Pipeline类型Job并命名为my-k8s-jenkins-pipeline，然后Pipeline脚本处填写一个简单的测试脚本如下：

Plain Text

1def label = "mypod-${UUID.randomUUID().toString()}"
2podTemplate(label: label, cloud: 'kubernetes') {
3    node(label) {
4        stage('Run shell') {
5            sh 'sleep 130s'
6            sh 'echo hello world.'
7        }
8    }
9}

执行构建，此时去构建队列里面，可以看到有一个构建任务，点击立即构建之后，等初始化成功之后，会成功发布，我们通过kubectl命令行，可以看到整个自动创建和删除过程。

注意： 示例中使用的镜像，如果直接拉取会有超时的情况，可以使用以下命令在机器上提前执行

Plain Text

1docker pull hub.baidubce.com/jpaas-public/jenkins/jnlp-slave:v0
2
3
4docker tag hub.baidubce.com/jpaas-public/jenkins/jnlp-slave:v0 jenkins/jnlp-slave:4.0.1-1

通过 CCE Ingress 实现虚机和容器服务的统一接入

CCE-访问公网实践

容器引擎 CCE

容器引擎 CCE

CCE集群使用Jenkins持续发布

概览

操作步骤

1. 设置 Jenkins 存储目录

2. 部署 Jenkins Server 到 Kubernetes

3. 初始化配置 Jenkins

4. Jenkins 安装 Kubernetes Plugin 插件

5. 非集群内的 Jenkins 连接 Kubernetes 集群

6. 测试并验证