Kubernetes 部署 gitlab

敏捷开发和devops潮流下，gitlab的使用率直线上升，自身所带有的CI/CD工具也很齐全，深受开发人群的喜爱，我们将在kubernetes 集群环境中部署一套gitlab环境，作为代码管理和使用自动化构建部署功能，准备在生产环境中使用

环境准备

k8s集群（阿里云专业版）
postgres 数据库

redis 缓存服务

gitlab 应用

关于postgres、redis的部署请查看之前相关部署文章

gitlab 部署

版本 gitlab 中文版 11.1.4

gitlab 本身是无状态服务，但是其中的repository仓库，配置等文件需要进行持久化存储，容器目录为：

/home/git/data
/etc/gitlab/
/var/opt/gitlab
/var/log/gitlab 可选

编写deployment 部署文件，gitlab.yaml,提前创建好pv 资源datadir-gitlab，这里使用的是阿里云的nas服务，配置gitlab时，请修改example.com 为自身域名，邮箱配置参数按实际情况进行修改，ssh端口这里暴露为nodeport类型32222，使用SLB的22端口代理进来。

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: datadir-gitlab
  namespace: gitlab-cicd
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: nas
  resources:
    requests:
      storage: 50Gi
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: gitlab
  namespace: gitlab-cicd
  labels:
    name: gitlab
spec:
  replicas: 1
  template:
    metadata:
      name: gitlab
      labels:
        name: gitlab
    spec:
      containers:
      - name: gitlab
        image: twang2218/gitlab-ce-zh:11.1.4
        imagePullPolicy: IfNotPresent
        env:
        - name: TZ
          value: Asia/Shanghai
        - name: GITLAB_TIMEZONE
          value: Beijing
        - name: GITLAB_SECRETS_DB_KEY_BASE
          value: long-and-random-alpha-numeric-string
        - name: GITLAB_SECRETS_SECRET_KEY_BASE
          value: long-and-random-alpha-numeric-string
        - name: GITLAB_SECRETS_OTP_KEY_BASE
          value: long-and-random-alpha-numeric-string
        - name: GITLAB_ROOT_PASSWORD
          value: admin123456
        - name: GITLAB_ROOT_EMAIL
          value: 
        - name: GITLAB_HOST
          value: xxx.example.com
        - name: GITLAB_PORT
          value: "80"
        - name: GITLAB_SSH_PORT
          value: "22"
        - name: GITLAB_NOTIFY_ON_BROKEN_BUILDS
          value: "true"
        - name: GITLAB_NOTIFY_PUSHER
          value: "false"
        - name: GITLAB_BACKUP_SCHEDULE
          value: daily
        - name: GITLAB_BACKUP_TIME
          value: 01:00
        - name: GITLAB_OMNIBUS_CONFIG
          value: |
            ## time_zone
            gitlab_rails['time_zone'] = 'Asia/Shanghai'
            ## postgres
            postgresql['enable'] = false
            gitlab_rails['db_adapter'] = "postgresql"
            gitlab_rails['db_encoding'] = "utf-8"
            gitlab_rails['db_database'] = "gitlab"
            gitlab_rails['db_username'] = "postgres"
            gitlab_rails['db_password'] = "xxx"
            gitlab_rails['db_host'] = "postgres-svc"
            gitlab_rails['db_port'] = 5432
            ## redis
            redis['enable'] = false
            gitlab_rails['redis_host'] = "redis"
            gitlab_rails['redis_port'] = 6379
            #gitlab_rails['redis_password'] = "redis_password"
            gitlab_rails['redis_database'] = 0
            ## pages
            pages_external_url "http://page.example.com/"
            gitlab_pages['enable'] = true
            gitlab_rails['pages_path'] = "/var/opt/gitlab/pages"
            gitlab_pages['external_http'] = ['1.1.1.2:80']
            #gitlab_pages['access_control'] = true
            ## gitlab ssh,http克隆地址，默认为hostname
            external_url "http://gitlab.example.com/"
            ## email setting
            gitlab_rails['smtp_enable'] = true
            gitlab_rails['smtp_address'] = "smtp.qq.com"
            gitlab_rails['smtp_port'] = 465
            gitlab_rails['smtp_user_name'] = ""
            gitlab_rails['smtp_password'] = "xxx"
            gitlab_rails['smtp_authentication'] = "login"
            gitlab_rails['smtp_enable_starttls_auto'] = true
            gitlab_rails['smtp_tls'] = true
            gitlab_rails['gitlab_email_from'] = ''
            gitlab_rails['smtp_domain'] = "smtp.qq.com"
            postgresql['enable'] = false
            postgres_exporter['enable'] = false
            redis['enable'] = false
            prometheus['enable'] = false
            alertmanager['enable'] = false
            node_exporter['enable'] = false
            redis_exporter['enable'] = false
            prometheus_monitoring['enable'] = false
        ports:
        - name: http
          containerPort: 80
        - name: ssh
          containerPort: 22
        volumeMounts:
        - mountPath: /home/git/data
          name: data
        - mountPath: /etc/gitlab
          name: etc-gitlab
        - mountPath: /var/opt/gitlab    
          name: opt-gitlab
        livenessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 900
          timeoutSeconds: 5
        readinessProbe:
          httpGet:
            path: /
            port: 80
          initialDelaySeconds: 5
          timeoutSeconds: 1
      volumes:
      - name: data
        persistentVolumeClaim:
            claimName: datadir-gitlab
      - name: etc-gitlab
        persistentVolumeClaim:
            claimName: etc-gitlab
      - name: opt-gitlab
        persistentVolumeClaim:
            claimName: opt-gitlab

---
apiVersion: v1
kind: Service
metadata:
  name: gitlab
  namespace: gitlab-cicd
  labels:
    name: gitlab
spec:
  type: NodePort
  ports:
    - name: http
      port: 80
      targetPort: http
    - name: ssh
      port: 22
      targetPort: ssh
      nodePort: 32222
  selector:
    name: gitlab

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: gitlab
  namespace: gitlab-cicd
  annotations:
    nginx.ingress.kubernetes.io/affinity: "cookie"
spec:
  rules:
  - host: gitlab.example.com
    http:
      paths:
      - backend:
          serviceName: gitlab
          servicePort: http
        path: /

注意：本次部署使用的数据服务应提前部署好，若没有部署请查看>> <<文章部署 , 连接地址可以直接使用svc-name.namespace.svc 的方式，修改自定义的账号密码

执行gitlab.yaml部署文件

$ kubectl apply -f gitlab.yaml
persistentvolumeclaim/datadir-gitlab configured
deployment.apps/gitlab configured
service/gitlab configured
ingress.extensions/gitlab configured

查看pod 运行状态：

$ kubectl get pod -n gitlab-cicd
NAME                              READY   STATUS    RESTARTS   AGE
gitlab-54548c6969-ghvff           1/1     Running   0          2h
gitlab-ci-runner-0                1/1     Running   0          2h
gitlab-ci-runner-1                1/1     Running   0          2h
redis-8477595b9c-qh6th            1/1     Running   0          77d
stolon-keeper-0                   1/1     Running   0          1d
stolon-keeper-1                   1/1     Running   0          1d
stolon-keeper-2                   1/1     Running   0          1d
stolon-proxy-db976479d-5r6qs      1/1     Running   0          1d
stolon-proxy-db976479d-8x46s      1/1     Running   0          1d
stolon-sentinel-54579c7dd-bk76h   1/1     Running   0          1d
stolon-sentinel-54579c7dd-cwtm2   1/1     Running   0          1d

运行成功后，浏览器访问 http://gitlab.example.com 输入上面配置初始root 用户密码 admin123456,登录成功后测试gitlab运行稳定性，进行相关配置，gitlab默认是公开注册，需要配置使用邮箱验证；创建新项目，克隆仓库和上传文件，最后不要忘记进行忘记密码操作，测试邮件服务是否配置成功

注意：

gitlab.yaml 部署文件中，注意GITLAB_OMNIBUS_CONFIG 配置项的参数，这里我关闭了镜像默认启动的grafana、altermanager、prometheus 等服务，并开启了pages服务，配置pages服务时注意不要使用gitlab.example.com 子域名，可以配置page.example.com

如要需要开启自定义域名，需要配置gitlab_pages[‘external_http‘] 参数

其他相关配置参数请查阅gitlab官网 相关文档