gitlab CI环境搭建

gitlab CI环境搭建

摘要：主要记录基于docker 的gitlabci环境搭建步骤和示例,顺带介绍相关的基本概念.

基本概念

• CI

From wiki

In software Pipeline
一次 Pipeline 其实相当于一次构建任务，里面可以包含多个流程，如安装依赖、运行测试、编译、部署测试服务器、部署生产服务器等流程。engineering, continuous integration (CI) is the practice of merging all developer working copies to a shared mainline several times a day.Each check-in is then verified by an automated build, allowing teams to detect problems early.

• pipeline: 一个完整的构建任务, 可以包括多个流程, 如安装依赖、运行测试、编译、部署等流程
• Stages: 构建阶段，pipeline中提及的若干流程
• Jobs: 表示构建作业任务，某Stage中执行的作业
• GitLab Runner: 实际的构造pipeline执行者, 以下会有详述
• .gitlab-ci.yml: 项目根目录下描述CI pipeline执行的文件, 见相关小节

prerequisites

本文主要关注基于docker(k8s暂未搭建)搭建gitlab CI环境. 需要先搭建好docker(或k8s)相关环境:

• docker
  • docker for mac, 带kubernetes

Install gitlab

• docker 安装启动gitlab

sudo docker pull gitlab/gitlab-ce:latest
sudo docker run --detach \
    --hostname localhost \
    --network=host \
    --publish 443:443 --publish 80:80 --publish 22:22 \
    --name gitlab \
    --restart always \
    --volume /tmp/gitlab/config:/etc/gitlab \
    --volume /tmp/gitlab/logs:/var/log/gitlab \
    --volume /tmp/gitlab/data:/var/opt/gitlab \
    gitlab/gitlab-ce:latest

# if error occurred: 
# Failed asserting that mode permissions on "/var/opt/gitlab/git-data/repositories" is 2770
mkdir -p /tmp/gitlab/data/git-data/repositories
sudo chmod g+s /tmp/gitlab/data/git-data/repositories
docker exec -it gitlab update-permissions

参数说明：
--hostname: 设置容器的hostname
--network: 设置容器运行的网络模式, 此处为host模式, 主机网络.
--publish或-p: 端口映射转发规则（本地80:容器Http访问端口, 本地443: Https访问端口, 本地端口22: ssh）
--name：容器名称
--restart always：crash或机器启动后总要重启
--volume or -v: 文件目录或磁盘挂载, (/tmp/gitlab/data: app data，/tmp/gitlab/logs: 日志路径，/tmp/gitlab/config: 配置文件)
-e：配置环境变量

• 可修改ssh端口(默认22如果已经占用, 修改为30102), /tmp/gitlab/config/gitlab.rb: gitlab_rails['gitlab_shell_ssh_port'] = 30102

• 本地环境修改modify URL

  • sudo hostname gitlab.test.com
  • 修改/var/opt/gitlab/gitlab-rails/etc/gitlab.yml
  • 或/etc/gitlab/gitlab.rb

• 浏览器访问url, 注册填写root用户密码, 这里因为默认邮箱是无法使用的, 直接设置密码后用root登录即可.

http://localhost:80

初次见面

创建项目

• 见上图, 先创建一个组devops, 设置权限后再建立项目citest(略)

CI pipeline

see gitlabci

Starting from version 8.0, GitLab Continuous Integration (CI) is fully integrated into GitLab itself and is enabled by default on all projects.

配置 CI/CD

增加 .gitlab-ci.yml

• 在项目根目录下(比如我的是$GOPATH/src/gitlab.test.com/devops/citest)
• a demo gitlab-ci.yml file

image: alpine:latest
lint:
  script: "echo lint"
test:
  script: "echo testing"

• 其中image,script是保留字(reserved keywords), image是构建项目需要使用的基础镜像, script 是CI要执行的脚本命令的意思, 类似其他保留字:

extends, stage, services, image, before_script, after_script, artifacts, environment, variables, parellel等, 下篇准备对此做个总结, 其实gitlab官网说明挺详细的.

配置docker gitlab Runner

• docker runner, see gitlab
• see gitlab-runner register
• 交互式注册, seeregister runner

# https://docs.gitlab.com/runner/install/docker.html
mkdir -p /tmp/gitlab-runner/config
docker pull gitlab/gitlab-runner
docker run -d --network=host --name gitlab-runner --restart always \
  -v /tmp/gitlab-runner/config:/etc/gitlab-runner \
  -v /var/run/docker.sock:/var/run/docker.sock \
  gitlab/gitlab-runner:latest
  
# register a runner interactively, 交互式注册
docker exec -it gitlab-runner /bin/bash
gitlab-runner register --help
gitlab-runner register

# 注册填写url/token/name/executer/tag等, 从项目settings拷贝url跟toker创建一个专有runner
copy url && token from http://gitlab.test.com/devops/citest/settings/ci_cd
# http://192.168.1.101/admin/runners
# docker中不要用localhost
# executor 这一步使用 docker

# The token can be found in /etc/gitlab-runner/config.toml
grep "token" /etc/gitlab-runner/config.toml

# stop
docker stop gitlab-runner && docker rm gitlab-runner

• 命令注册, seeregister runner in one line

docker run --rm -t -i -v /tmp/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register \
  --non-interactive \
  --executor "docker" \
  --docker-image alpine:latest \
  --url "https://gitlab.test.com/" \
  --registration-token "PROJECT_REGISTRATION_TOKEN" \
  --description "docker-runner" \
  --tag-list "docker" \
  --run-untagged \
  --locked="false"``

注意: url与token来自这里：

settings->CI/CD->Specific Runners

• delploy runner in k8s （来自网上, 未验证）

# deployment.yml

# docker run --rm -t -i alpine sh -c "apk add --no-cache curl; curl -X POST -I http://gitlab.test.com/api/v4/runners"

#!/bin/bash -x
# docker run --rm -t -i -v /opt/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register
# One-line registration command
token="Y9xTAzfiGmvWjGjSP9SV"
if [ $# -gt 1 ];then
    token=$1
fi

docker run --rm -t -i --network=host -v /opt/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register \
  --non-interactive \
  --url "http://gitlab.test.com/" \
  --registration-token ${token} \
  --description "docker-search-engine-runner" \
  --executor "docker" \
  --tag-list "sfd,tsfd,gpu" \
  --run-untagged \
  --locked="false" \
  --docker-image alpine:latest

tips

• 使用tag在.gitlab-ci.yml中指定runner
• image指定docker 镜像
• services 指定需要访问的服务
• gitlab unregister --all-runners

references

• https://docs.gitlab.com/ee/ci/
• 基于Gitlab CI搭建持续集成环境
• https://www.jianshu.com/p/2b43151fb92e
• https://docs.gitlab.com/runner/executors/kubernetes.html
• https://docs.gitlab.com/runner/install/kubernetes.html
• https://github.com/denverdino/k8s-for-docker-desktop