金系——Devops

使用Gitlab+Rancher实现自动部署测试环境

2018-07-27  本文已影响114人  qixuezhiren

以前项目使用jenkins打包部署环境，用了一段时间，发现jenkins太过臃肿了，java进程动不动就占用几个G的内存，实在吃不消。

最近项目代码全部迁移到了gitlab做统一管理。查看了官方文档，发现自带的ci/cd功能，只需要把部署过程定义在.gitlab-ci.yml文件中即可，十分方便。实际过程使用，使用很简单，所有工作，全部在.gitlab-ci.yml完成即可，比jenkins好太多了。

以下是过程记录，会对部分内容做解释，以便记忆。

Gitlab-runner部署

Centos7 安装Gitlab-runner

Gitlab-runner建议安装在一台独立的服务器上，我这里是docker、rancher安装在一起


$ curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh | sudo bash

$ yum install gitlab-runner -y

Gitlab-runner注册

gitlab支持多种方式的runner，通常使用以下两种配置

独享的runner配置：Project -> Settings -> CI/CD -> Runners settings

共享的runner配置：Admin area -> Overview -> Runners


$ gitlab-runner register    # 注册runner

Running in system-mode.                           



Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):

http://10.10.0.3/  # gitlab服务器地址

Please enter the gitlab-ci token for this runner:

U-pS5ZVAeq-MhFmkxCyJ    # 访问gitlab的token，见runner配置页

Please enter the gitlab-ci description for this runner:

[public]:  # runner描述

Please enter the gitlab-ci tags for this runner (comma separated):

public-tag  # runner标记

Registering runner... succeeded                    runner=U-pS5ZVA

Please enter the executor: ssh, docker+machine, kubernetes, docker, docker-ssh, parallels, shell, virtualbox, docker-ssh+machine:

shell      # 选择执行器，这里选择shell

Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!

$ cat /etc/gitlab-runner/config.toml # 配置文件，配置成功后自动生成

测试


$ usermod -aG docker gitlab-runner      # 加入docker组

$ sudo -u gitlab-runner -H docker info  # 测试docker命令

$ gitlab-ci-multi-runner verify        # 检查runner状态

$ su - gitlab-runner                    # 切换到gitlab-runner用户，默认使用该用户部署

$ ls /home/gitlab-runner/builds/43eee0a1/0/test/deploy_text    # 项目目录

Gitlab部署示例

测试示例

项目结构

- apps

    - backend      # 后台项目目录

        - api      # 后台项目APP源代码目录

            - Dockerfile

            - run.py

            - requirements.txt

        - web

        - app

    - frontend      # 前端项目目录

        - wx        # 前端项目APP源代码目录

        - web

- config            # 公共项目配置文件

    - nginx_web.conf

    - supervisord_web.conf

    - ...

- doc              # 项目文档及API等

    - spring1

    - sprint2

- .gitignore        # git的忽略文件

- .gitlab-ci.yml    # gitlab部署文件

- docker-compose.yml            # 测试环境docker-compose

- rancher-compose.yml          # 测试环境rancher-compose

- docker-compose-pro.yml        # 生产环境docker-compose

- rancher-compose-pro.yml      # 生产环境rancher-compose

容器命名规范

仓库地址/项目路径/分支名称/APP名称:v打包ID

registry.registry:5000/test/deploy-text/master/test:v217

registry.registry:5000/test/deploy-text/master/test:latest

所有字母全部小写和 : - / 三种合法的字符

.gitlab-ci.yml


variables:

  #CI_DEBUG_TRACE: "true"      # 调试使用

  REGISTRY_HOST: "registry.registry:5000"  # 本地镜像

  BASE_IMAGE: "$REGISTRY_HOST/$CI_PROJECT_PATH/$CI_COMMIT_REF_NAME" # 镜像前缀

  TEST_APP: "test"

before_script:  # gitlab流水线过程执行前的脚本

  - export TEST_IMAGE="$BASE_IMAGE/$TEST_APP:v$CI_PIPELINE_ID"  # 当前版本的镜像

  - export TEST_IMAGE_LATEST="$BASE_IMAGE/$TEST_APP:latest"    # 最新的镜像

stages:

  - build  # 定义build阶段

build_test:

  stage: build  # job属于build阶段

  script:

    - docker build -t $TEST_IMAGE ./backend/app/.

    - docker tag $TEST_IMAGE $TEST_IMAGE_LATEST

    - docker push $TEST_IMAGE

    - docker push $TEST_IMAGE_LATEST

  tags:

    - public-tag    # runner配置了tags，则必须指定tags

实际项目示例

  1. git-runner用户需要执行rancher config
  1. dockerfile add命令只支持相对路径

gitlab.yml

定时任务配置：Project -> Settings -> CI/CD -> Schedules


variables:

  #CI_DEBUG_TRACE: "true"

  REGISTRY_HOST: "registry.registry:5000"

  BASE_IMAGE: "$REGISTRY_HOST/$CI_PROJECT_PATH/$CI_COMMIT_REF_NAME"

  STACK: "cht-v$CI_COMMIT_REF_NAME"

  WEB_APP: "web"

  API_APP: "api"

before_script:

  - export WEB_IMAGE="$BASE_IMAGE/$WEB_APP:v$CI_PIPELINE_ID"

  - export WEB_IMAGE_LATEST="$BASE_IMAGE/$WEB_APP:latest"

  - export API_IMAGE="$BASE_IMAGE/$API_APP:v$CI_PIPELINE_ID"

  - export API_IMAGE_LATEST="$BASE_IMAGE/$API_APP:latest"

  - export TAG=$CI_COMMIT_REF_NAME

  - cp -rf ./config/*api.conf ./apps/backend/api/

  - cp -rf ./config/*web.conf ./apps/backend/web/

  - cp -rf ./apps/frontend/web/dist/ ./apps/backend/web/www

stages:

  - build

  - deploy

build_web:

  stage: build

  script:

    - docker build -t $WEB_IMAGE ./apps/backend/web/.

    - docker tag $WEB_IMAGE $WEB_IMAGE_LATEST

    - docker push $WEB_IMAGE

    - docker push $WEB_IMAGE_LATEST

  tags:

    - public-tag

  only:

    - schedules    # 不使用默认的commit触发，该由定时任务触发构建，减少无意义的构建

  except:

    - master

build_api:

  stage: build

  script:

    - docker build -t $API_IMAGE ./apps/backend/api/.

    - docker tag $API_IMAGE $API_IMAGE_LATEST

    - docker push $API_IMAGE

    - docker push $API_IMAGE_LATEST

  tags:

    - public-tag

  only:

    - schedules

  except:

    - master

deploy_test:

  stage: deploy

  script:

    - rancher up -d -u -p -c -s $STACK  # 调用rancher的命令部署

  tags:

    - public-tag

  only:

    - schedules

  except:

    - master

docker-compose.yml


version: '2'

services:

  web:

    image: "${WEB_IMAGE_LATEST}"    # 使用gitlab-ci.yml export的环境变量

    stdin_open: true

    tty: true

    restart: always

    links:

      - pgsql:pgsql

      - redis:redis

    volumes:

      - /data/${TAG}/static_file:/data/www/static_file

      - /data/${TAG}/dm_log:/data/log

    labels:

      io.rancher.container.pull_image: always

      io.rancher.scheduler.affinity:host_label: io.rancher.server.role=agent1

  api:

    image: "${API_IMAGE_LATEST}"

    stdin_open: true

    tty: true

    restart: always

    links:

      - pgsql:pgsql

      - redis:redis

    volumes:

      - /data/${TAG}/static_file:/data/www/static_file

      - /data/${TAG}/wx_log:/data/log

    labels:

      io.rancher.container.pull_image: always

      io.rancher.scheduler.affinity:host_label: io.rancher.server.role=agent1

  idgen:

    image: registry.registry:5000/cht2idgen

    stdin_open: true

    tty: true

    restart: always

    environment:

      PROJECT_NAME: 16youlu_cht2idgen

      SYSLOG_HOST: 10.10.0.63

      SYSLOG_PORT: 12201

      SYSLOG_LEVEL: DEBUG

    links:

      - redis:redis

    labels:

      io.rancher.container.pull_image: always

      io.rancher.scheduler.affinity:host_label: io.rancher.server.role=agent1

  pgsql:

    image: postgres

    environment:

      POSTGRES_DB: test

      POSTGRES_PASSWORD: test123456

      POSTGRES_USER: test

    stdin_open: true

    tty: true

    restart: always

    volumes:

      - /data/${TAG}/pgsql_data:/var/lib/postgresql/data

    labels:

      io.rancher.container.pull_image: always

      io.rancher.scheduler.affinity:host_label: io.rancher.server.role=agent1

  redis:

    image: redis

    stdin_open: true

    tty: true

    restart: always

    labels:

      io.rancher.container.pull_image: always

      io.rancher.scheduler.affinity:host_label: io.rancher.server.role=agent1

参考资料

https://docs.gitlab.com/runner/configuration/advanced-configuration.html

https://docs.gitlab.com/ce/ci/variables/README.html

上一篇下一篇

猜你喜欢

热点阅读