基于kubernetes构建企业Jenkins,CI,CD平台

1.Ci,CD介绍

开发-编译-测试-部署

持续集成(Continuous Integration,ci):代码合并、构建、部署、测试都在一起,不断地执行这个过程,并对结果反馈

持续部署(Continuous Deployment,cd):部署到测试环境、预生产环境、生产环境

持续交付(Continuous Delivery,cd):将最终产品发布到生产环境,给用户使用。

1.1项目环境

角色:K8s Harbor Git Jenkins
IP地址:k8s(118.190.201.11,118.190.201.12,118.190.201.13) Harbor,Git(118.190.201.14) Jenkins(部署在k8s平台)
配置:2C+ 4G

1.2准备环境

#所有节点
[root@linux-node1 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
118.190.201.11 linux-node1 linux-node1.example.com
118.190.201.12 linux-node2 linux-node2.example.com
118.190.201.13 linux-node3 linux-node3.example.com
[root@linux-node1 ~]# getenforce 
Disabled
[root@linux-node1 ~]# systemctl status firewalld.service 
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: inactive (dead)

1.3搭建Harbor仓库

##安装docker此处省略
##地址https://github.com/goharbor/harbor/releases
tar xvf harbor-offline-installer-v1.6.1.tgz
cd harbor/
##vi harbor.cfg 
#hostname = 118.190.201.14
./prepare
./install.sh
##安装docker-compose
sudo curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
docker-compose --version

1.4Git版本代码仓库

#安装git
yum install git
#创建Git用户并设置密码
useradd git
passwd git
#创建仓库
su - git
mkdir demon.git
cd demon.git
git --bare init
#配置客户端与Git服务器SSH免交互认证
#测试
git clone git@118.190.201.14:/home/git/demon.git
git add .
git common -m "test"
git push origin master

1.5搭建NFS服务器提供存储

##官网地址
https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client/deploy
#所有节点安装nfs
yum install -y nfs-utils
[root@linux-node4 ~]# cat /etc/exports
/ifs/kubernetes *(rw,no_root_squash)
systemctl start nfs #nfs服务器启动就行
#git clone https://github.com/xiejincheng/k8s-deploy.git
cd k8s-deploy/nfs-client  #修改server地址 path路径
kubectl apply -f .

1.6部署Jenkins服务

参考地址:https://github.com/xiejincheng/kubernetes-plugin/tree/master/src/main/kubernetes

#git clone https://github.com/xiejincheng/k8s-deploy.git
cd k8s-deploy/jenkins  #修改使用nfs动态供给
kubectl apply -f .

1.7登录Jenkins web界面之后选择安装插件

插件下载地址:http://updates.jenkins-ci.org/download/plugins/

http://118.190.201.12:30006/

选择安装git 及 pipeline kubernetes插件

1.8安装DNS服务

参考地址:https://github.com/xiejincheng/kubernetes/tree/master/cluster/addons/dns/coredns

#修改内容clusterIP clusterDemon limitMemory
git clone https://github.com/xiejincheng/k8s-deploy.git
cd k8s-deploy/coredns
kubectl apply -f coredns.yaml

1.9Jenkins在kubernetes中动态创建代理

JNLP(Java Network Launching Protocol )是java提供的一种可以通过浏览器直接执行java应用程序的途径,它使你可以直接通过一个网页上的url连接打开一个java应用程序。

选择在线安装插件: 系统管理-管理插件

搜索git pipeline kubernetes进行安装

配置Jenkins支持Kubernetes插件

系统管理-系统设置-点击之后移动到最低部-添加cloud-选择kubernetes

pod默认是有权限的所以不需要添加证书key

补充:可以通过UI的形式添加pod模板创建Jenkins slave 建议使用pipeline脚本去添加,方便管理

1.10构建Jenkins Slave镜像

#参考地址https://github.com/jenkinsci/docker-jnlp-slave
[root@linux-node1 jenkins-slave]# vim Dockerfile
FROM centos:7
MAINTAINER cnts@163.com
RUN yum install -y java-1.8.0-openjdk maven curl git libtool-ltdl-devel && \
    yum clean all && \
    rm -fr /var/cace/yum/* && \
    mkdir -p /usr/share/jenkins

COPY slave.jar /usr/share/jenkins/slave.jar
COPY jenkins-slave /usr/bin/jenkins-slave
COPY settings.xml /etc/maven/settings.xml
RUN chmod +x /usr/bin/jenkins-slave

ENTRYPOINT ["jenkins-slave"]

##获取slave.jar包
"""http://jenkins-server/jnlpJars/slave.jar 下载"""
http://118.190.201.12:30006/jnlpJars/slave.jar  ##下载好上传对应目录下
##构建
docker build -t 118.190.201.14/project/jenkins-slave-jdk:1.8 .
##settings文件存放路径
https://github.com/xiejincheng/k8s-deploy/blob/master/jenkins-slave/settings.xml
##docker仓库配置可信任
[root@linux-node1 jenkins-slave]# cat /etc/docker/daemon.json 
{
    "registry-mirrors": ["http://f1361db2.m.daocloud.io"],
    "insecure-registries": ["118.190.201.14"]
}
##推送镜像
[root@linux-node1 jenkins-slave]# docker login 118.190.201.14
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[root@linux-node1 jenkins-slave]# docker push 118.190.201.14/project/jenkins-slave-jdk:1.8

2.Jenkins构建Pipeline流水线发布

1.jenkins pipeline是一套插件,支持Jenkins中实现集成和持续交付管道;
2.pipeline通过特定语法对简单到复杂的传输管道进行建模
  .声明式:遵循Groovy相同语法。pipeline{}
  .脚本式:支持Groovy大部分功能,非常灵活。node{}
3.Jenkins pipeline的定义被写入一个文本文件,称为Jenkinsfile

使用pipeline语法生成

http://118.190.201.12:30006/job/demon/pipeline-syntax/

##仓库地址
[root@linux-node1 ~]# git clone git@118.190.201.14:/home/git/demo.git
[root@linux-node1 ~]# cd demo/
[root@linux-node1 demo]# cat .git/config 
[core]
       repositoryformatversion = 0
       filemode = true
       bare = false
       logallrefupdates = true
[remote "origin"]
       url = git@118.190.201.14:/home/git/demo.git
       fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
       remote = origin
       merge = refs/heads/master

添加用户密钥认证,添加私钥内容cat /root/.ssh/id_rsa

添加私钥内容cat /root/.ssh/id_rsa

添加凭据之后,点击生产脚本

添加到pipeline脚本里面执行测试

##参考模板文件地址:https://github.com/xiejincheng/kubernetes-plugin
jenkisfile地址:
https://github.com/xiejincheng/k8s-deploy/blob/master/jenkinsfile

保存Harbor的用户名与密码到Jenkins中

凭据-系统-点击全局凭据(下三角)-添加凭据

然后把生成的ID填写到pipeline脚本中def docker_registry_auth = "76aba3bd-cc8c-45a2-95cf-b5ed9aac5002"

以及git的ID保存到pipeline脚本中def git_auth = "f4550165-e7c6-473d-9d12-ed5170cba02c"

2.1提交代码到git仓库中

[root@linux-node4 ~]# su - git
Last login: Thu Oct 10 23:35:48 CST 2019 from linux-node1 on pts/1
[git@linux-node4 ~]$ mkdir java-demo.git
[git@linux-node4 ~]$ cd java-demo.git/
[git@linux-node4 java-demo.git]$ git --bare init
Initialized empty Git repository in /home/git/java-demo.git/
##代码克隆指定git-url
#git clone https://github.com/xiejincheng/tomcat-java-demo.git
#cd tomcat-java-demo/
#cat .git/config 
[core]
      repositoryformatversion = 0
      filemode = true
      bare = false
      logallrefupdates = true
[remote "origin"]
      url = git@118.190.201.14:/home/git/java-demo.git
      fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
      remote = origin
      merge = refs/heads/master
[root@linux-node1 tomcat-java-demo]# git add .
[root@linux-node1 tomcat-java-demo]# git commit -m "java-demo test"
[root@linux-node1 tomcat-java-demo]# git config --global user.name "Your Name"
[root@linux-node1 tomcat-java-demo]# git config --global user.email you@example.com
[root@linux-node1 tomcat-java-demo]# git push origin master

设置参数化构建

Pipeline demon-配置-参数化构建-字符参数-保存

点击构建

2.2构建到k8s平台

##现在节点进行测试镜像是否正常
[root@linux-node2 ~]# docker run -d -p 8888:8080 118.190.201.14/welcome/demo:3
##然后浏览器进行访问http://IP:port

kubernetes Continuous Deploy插件:用于将资源配置部署到kubernetes
##插件参考地址:https://plugins.jenkins.io/kubernetes-cd

安装插件:系统管理-插件管理-可用-搜索kueretes-选择kubernetes Continuous Deploy插件进行安装

添加kebeconfig凭据

凭据内容在master节点
ls /root/.kube/config 
##生成config文件如下步骤

1.设置集群参数

设置证书是因为API Server是通过RBAC进行授权,RBAC预定义了一些角色,所有要设置参数

 [root@linux-node1 /usr/local/src/ssl]# kubectl config set-cluster kubernetes \
   --certificate-authority=/opt/kubernetes/ssl/ca.pem \
   --embed-certs=true \
   --server=https://118.190.201.11:6443
Cluster "kubernetes" set.

2.设置客户端认证参数

 [root@linux-node1 /usr/local/src/ssl]# kubectl config set-credentials admin \
   --client-certificate=/opt/kubernetes/ssl/admin.pem \
   --embed-certs=true \
   --client-key=/opt/kubernetes/ssl/admin-key.pem
User "admin" set.

3.设置上下文参数

 [root@linux-node1 /usr/local/src/ssl]#  kubectl config set-context kubernetes \
   --cluster=kubernetes \
   --user=admin
Context "kubernetes" created.

4.设置默认上下文

 [root@linux-node1 /usr/local/src/ssl]# kubectl config use-context kubernetes
Switched to context "kubernetes".

然后把生成的ID填写到def k8s_auth = "生成id"

##deploy文件
https://github.com/xiejincheng/k8s-deploy/blob/master/deploy.yml
##将deploy这个文件提交到代码仓库
git add .
git commit -m "deploy.yml"
git push origin master
##创建认证凭据pipeline中的secret_name = "registry-pull-secret"
# kubectl create secret docker-registry registry-pull-secret --docker-username=admin --docker-password=Harbor12345 \
  --docker-email=cnts@163.com --docker-server=118.190.201.14

然后使用pipeline脚本部署到k8s平台

2.3pipeline脚本与源代码一起管理

#把Jenkinsfile文件放到代码版本仓库
git add .
git commit -m "Jenkinsfile"
git push origin master

 

0
如无特殊说明,文章均为本站原创,转载请注明出处

该文章由 发布

这货来去如风,什么鬼都没留下!!!