kubernetes实践之二十七：Harbor

一：简介

Harbor是一个用于存储和分发Docker镜像的企业级Registry服务器。

镜像的存储harbor使用的是官方的docker registry(v2命名是distribution)服务去完成。harbor在docker distribution的基础上增加了一些安全、访问控制、管理的功能以满足企业对于镜像仓库的需求。harbor以docker-compose的规范形式组织各个组件，并通过docker-compose工具进行启停。

docker的registry是用本地存储或者s3都是可以的，harbor的功能是在此之上提供用户权限管理、镜像复制等功能，提高使用的registry的效率。Harbor的镜像拷贝功能是通过docker registry的API去拷贝，这种做法屏蔽了繁琐的底层文件操作、不仅可以利用现有docker registry功能不必重复造轮子，而且可以解决冲突和一致性的问题。

二：Harbor架构


三：主要组件

• Proxy：对应启动组件nginx。它是一个nginx反向代理，代理Notary client（镜像认证）、Docker client（镜像上传下载等）和浏览器的访问请求（Core Service）给后端的各服务；
• UI（Core Service）：对应启动组件harbor-ui。底层数据存储使用mysql数据库，主要提供了四个子功能： 
  
  • UI：一个web管理页面ui；
  • API：Harbor暴露的API服务；
  • Auth：用户认证服务，decode后的token中的用户信息在这里进行认证；auth后端可以接db、ldap、uaa三种认证实现；
  • Token服务（上图中未体现）：负责根据用户在每个project中的role来为每一个docker push/pull命令issuing一个token，如果从docker client发送给registry的请求没有带token，registry会重定向请求到token服务创建token。
• Registry：对应启动组件registry。负责存储镜像文件，和处理镜像的pull/push命令。Harbor对镜像进行强制的访问控制，Registry会将客户端的每个pull、push请求转发到token服务来获取有效的token。
• Admin Service：对应启动组件harbor-adminserver。是系统的配置管理中心附带检查存储用量，ui和jobserver启动时候需要加载adminserver的配置；
• Job Sevice：对应启动组件harbor-jobservice。负责镜像复制工作的，他和registry通信，从一个registry pull镜像然后push到另一个registry，并记录job_log；
• Log Collector：对应启动组件harbor-log。日志汇总组件，通过docker的log-driver把日志汇总到一起；
• Volnerability Scanning：对应启动组件clair。负责镜像扫描
• Notary：对应启动组件notary。负责镜像认证
• DB：对应启动组件harbor-db，负责存储project、 user、 role、replication、image_scan、access等的metadata数据。

四：安装

1、安装python-pip
yum -y install epel-release
yum -y install python-pip

2、安装docker-compose
pip install docker-compose
待安装完成后，执行查询版本的命令，即可安装docker-compose
docker-compose version

3.安装Harbor
wget https://storage.googleapis.com/harbor-releases/release-1.4.0/harbor-offline-installer-v1.4.0.tgz
tar -zxvf harbor-offline-installer-v1.4.0.tgz

配置文件 harbor.cfg

点击(此处)折叠或打开

 ## Configuration file of Harbor

 #The IP address or hostname to access admin UI and registry service.
 #DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
 hostname = 120.79.156.135

 #The protocol for accessing the UI and token/notification service, by default it is http.
 #It can be set to https if ssl is enabled on nginx.
 ui_url_protocol = http

 #Maximum number of job workers in job service
 max_job_workers = 3

 #Determine whether or not to generate certificate for the registry's token.
 #If the value is on, the prepare script creates new root cert and private key
 #for generating token to access the registry. If the value is off the default key/cert will be used.
 #This flag also controls the creation of the notary signer's cert.
 customize_crt = on

 #The path of cert and key files for nginx, they are applied only the protocol is set to https
 ssl_cert = /mnt/harbor/cert/server.crt
 ssl_cert_key = /mnt/harbor/cert/server.key

 #The path of secretkey storage
 secretkey_path = /mnt/harbor

 #Admiral's url, comment this attribute, or set its value to NA when Harbor is standalone
 admiral_url = NA

 #Log files are rotated log_rotate_count times before being removed. If count is 0, old versions are removed rather than rotated.
 log_rotate_count = 50
 #Log files are rotated only if they grow bigger than log_rotate_size bytes. If size is followed by k, the size is assumed to be in kilobytes.
 #If the M is used, the size is in megabytes, and if G is used, the size is in gigabytes. So size 100, size 100k, size 100M and size 100G
 #are all valid.
 log_rotate_size = 200M

 #NOTES: The properties between BEGIN INITIAL PROPERTIES and END INITIAL PROPERTIES
 #only take effect in the first boot, the subsequent changes of these properties
 #should be performed on web ui

 #************************BEGIN INITIAL PROPERTIES************************

 #Email account settings for sending out password resetting emails.

 #Email server uses the given username and password to authenticate on TLS connections to host and act as identity.
 #Identity left blank to act as username.
 email_identity =

 email_server = smtp.mydomain.com
 email_server_port = 25
 email_username = [email protected]
 email_password = abc
 email_from = admin
 email_ssl = false
 email_insecure = false

 ##The initial password of Harbor admin, only works for the first time when Harbor starts.
 #It has no effect after the first launch of Harbor.
 #Change the admin password from UI after launching Harbor.
 harbor_admin_password = Weinong$2017

 ##By default the auth mode is db_auth, i.e. the credentials are stored in a local database.
 #Set it to ldap_auth if you want to verify a user's credentials against an LDAP server.
 auth_mode = db_auth

 #The url for an ldap endpoint.
 ldap_url = ldaps://ldap.mydomain.com

 #A user's DN who has the permission to search the LDAP/AD server.
 #If your LDAP/AD server does not support anonymous search, you should configure this DN and ldap_search_pwd.
 #ldap_searchdn = uid=searchuser,ou=people,dc=mydomain,dc=com

 #the password of the ldap_searchdn
 #ldap_search_pwd = password

 #The base DN from which to look up a user in LDAP/AD
 ldap_basedn = ou=people,dc=mydomain,dc=com

 #Search filter for LDAP/AD, make sure the syntax of the filter is correct.
 #ldap_filter = (objectClass=person)

 # The attribute used in a search to match a user, it could be uid, cn, email, sAMAccountName or other attributes depending on your LDAP/AD
 ldap_uid = uid

 #the scope to search for users, 0-LDAP_SCOPE_BASE, 1-LDAP_SCOPE_ONELEVEL, 2-LDAP_SCOPE_SUBTREE
 ldap_scope = 2

 #Timeout (in seconds) when connecting to an LDAP Server. The default value (and most reasonable) is 5 seconds.
 ldap_timeout = 5

 #Verify certificate from LDAP server
 ldap_verify_cert = true

 #Turn on or off the self-registration feature
 self_registration = on

 #The expiration time (in minute) of token created by token service, default is 30 minutes
 token_expiration = 30

 #The flag to control what users have permission to create projects
 #The default value "everyone" allows everyone to creates a project.
 #Set to "adminonly" so that only admin user can create project.
 project_creation_restriction = everyone

 #************************END INITIAL PROPERTIES************************

 #######Harbor DB configuration section#######

 #The address of the Harbor database. Only need to change when using external db.
 db_host = mysql

 #The password for the root user of Harbor DB. Change this before any production use.
 db_password = Weinong$2017

 #The port of Harbor database host
 db_port = 3306

 #The user name of Harbor database
 db_user = root

 ##### End of Harbor DB configuration#######

 #The redis server address. Only needed in HA installation.
 redis_url =

 ##########Clair DB configuration############

 #Clair DB host address. Only change it when using an exteral DB.
 clair_db_host = postgres

 #The password of the Clair's postgres database. Only effective when Harbor is deployed with Clair.
 #Please update it before deployment. Subsequent update will cause Clair's API server and Harbor unable to access Clair's database.
 clair_db_password = password

 #Clair DB connect port
 clair_db_port = 5432

 #Clair DB username
 clair_db_username = postgres

 #Clair default database
 clair_db = postgres

 ##########End of Clair DB configuration############

 #The following attributes only need to be set when auth mode is uaa_auth
 uaa_endpoint = uaa.mydomain.org
 uaa_clientid = id
 uaa_clientsecret = secret
 uaa_verify_cert = true
 uaa_ca_cert = /path/to/ca.pem


 ### Docker Registry setting ###
 #registry_storage_provider can be: filesystem, s3, gcs, azure, etc.
 registry_storage_provider_name = filesystem
 #registry_storage_provider_config is a comma separated "key: value" pairs, e.g. "key1: value, key2: value2".
 #Refer to https://docs.docker.com/registry/configuration/#storage for all available configuration.
 registry_storage_provider_config =

配置文件docker-compose.yml

点击(此处)折叠或打开

 version: '2'
 services:
   log:
     image: vmware/harbor-log:v1.4.0
     container_name: harbor-log
     restart: always
     volumes:
       - /mnt/harbor/log/:/var/log/docker/:z
       - ./common/config/log/:/etc/logrotate.d/:z
     ports:
       - 127.0.0.1:1514:10514
     networks:
       - harbor
   registry:
     image: vmware/registry-photon:v2.6.2-v1.4.0
     container_name: registry
     restart: always
     volumes:
       - /mnt/harbor/registry:/storage:z
       - ./common/config/registry/:/etc/registry/:z
     networks:
       - harbor
     environment:
       - GODEBUG=netdns=cgo
     command:
       ["serve", "/etc/registry/config.yml"]
     depends_on:
       - log
     logging:
       driver: "syslog"
       options:
         syslog-address: "tcp://127.0.0.1:1514"
         tag: "registry"
   mysql:
     image: vmware/harbor-db:v1.4.0
     container_name: harbor-db
     restart: always
     volumes:
       - /mnt/harbor/database:/var/lib/mysql:z
     networks:
       - harbor
     env_file:
       - ./common/config/db/env
     depends_on:
       - log
     logging:
       driver: "syslog"
       options:
         syslog-address: "tcp://127.0.0.1:1514"
         tag: "mysql"
   adminserver:
     image: vmware/harbor-adminserver:v1.4.0
     container_name: harbor-adminserver
     env_file:
       - ./common/config/adminserver/env
     restart: always
     volumes:
       - /mnt/harbor/config/:/etc/adminserver/config/:z
       - /mnt/harbor/secretkey:/etc/adminserver/key:z
       - /mnt/harbor/:/data/:z
     networks:
       - harbor
     depends_on:
       - log
     logging:
       driver: "syslog"
       options:
         syslog-address: "tcp://127.0.0.1:1514"
         tag: "adminserver"
   ui:
     image: vmware/harbor-ui:v1.4.0
     container_name: harbor-ui
     env_file:
       - ./common/config/ui/env
     restart: always
     volumes:
       - ./common/config/ui/app.conf:/etc/ui/app.conf:z
       - ./common/config/ui/private_key.pem:/etc/ui/private_key.pem:z
       - ./common/config/ui/certificates/:/etc/ui/certificates/:z
       - /mnt/harbor/secretkey:/etc/ui/key:z
       - /mnt/harbor/ca_download/:/etc/ui/ca/:z
       - /mnt/harbor/psc/:/etc/ui/token/:z
     networks:
       - harbor
     depends_on:
       - log
       - adminserver
       - registry
     logging:
       driver: "syslog"
       options:
         syslog-address: "tcp://127.0.0.1:1514"
         tag: "ui"
   jobservice:
     image: vmware/harbor-jobservice:v1.4.0
     container_name: harbor-jobservice
     env_file:
       - ./common/config/jobservice/env
     restart: always
     volumes:
       - /mnt/harbor/job_logs:/var/log/jobs:z
       - ./common/config/jobservice/app.conf:/etc/jobservice/app.conf:z
       - /mnt/harbor/secretkey:/etc/jobservice/key:z
     networks:
       - harbor
     depends_on:
       - ui
       - adminserver
     logging:
       driver: "syslog"
       options:
         syslog-address: "tcp://127.0.0.1:1514"
         tag: "jobservice"
   proxy:
     image: vmware/nginx-photon:v1.4.0
     container_name: nginx
     restart: always
     volumes:
       - ./common/config/nginx:/etc/nginx:z
     networks:
       - harbor
     ports:
       - 80:80
       - 443:443
       - 4443:4443
     depends_on:
       - mysql
       - registry
       - ui
       - log
     logging:
       driver: "syslog"
       options:
         syslog-address: "tcp://127.0.0.1:1514"
         tag: "proxy"
 networks:
   harbor:
     external: false

安装Harbor
./install.sh

可以使用docker-compose来管理Harbor的生命周期。 一些有用的命令列出如下:
docker-compose ps  查看


docker-compose stop  停止
docker-compose start 启动
docker-compose down  删除，利用./install.sh可以重新安装

五：验证

1.碰到的问题

a. 错误提示

点击(此处)折叠或打开

 /usr/lib/python2.7/site-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.21.1) or chardet (2.2.1) doesn

原因： python库中urllib3 (1.21.1) or chardet (2.2.1) 的版本不兼容
解决办法：
pip uninstall urllib3
pip uninstall  chardet
pip install requests

b.错误提示： “harbor failed to initialize the system: read /etc/adminserver/key: is a directory”
原因： harbor.cfg中的 secretkey_path和docker-compose.yml中的设置不一致

2.ui 界面

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/28624388/viewspace-2153546/，如需转载，请注明出处，否则将追究法律责任。

转载于:http://blog.itpub.net/28624388/viewspace-2153546/