harbor/registry设置Ceph对象存储
纯干货分享
http://jiangjiang.space 回忆书签，转载请注明出处，谢谢
1.创建ceph账户和swift子账户
安装ceph（略）
创建ceph账户
radosgw-admin subuser create --uid demo --subuser=demo:swift --access=full --secret=secretkey --key-type=swift
创建swift api 子账户
$ radosgw-admin key create --subuser=demo:swift --key-type=swift --gen-secret
{
"user_id": "demo",
"display_name": "ceph rgw demo user",
"email": "",
"suspended": 0,
"max_buckets": 1000,
"auid": 0,
"subusers": [
{
"id": "demo:swift",
"permissions": "full-control"
}
],
"keys": [
{
"user": "demo",
"access_key": "5Y1B1SIJ2YHKEHO5U36B",
"secret_key": "nrIvtPqUj7pUlccLYPuR3ntVzIa50DToIpe7xFjT"
}
],
"swift_keys": [
{
"user": "demo:swift",
"secret_key": "aCgVTx3Gfz1dBiFS4NfjIRmvT0sgpHDP6aa0Yfrh"
}
],
"caps": [],
"op_mask": "read, write, delete",
"default_placement": "",
"placement_tags": [],
"bucket_quota": {
"enabled": false,
"max_size_kb": -1,
"max_objects": -1
},
"user_quota": {
"enabled": false,
"max_size_kb": -1,
"max_objects": -1
},
"temp_url_keys": []
}
创建存储筒：
用S3客户端创建一个存储筒，将来push上来的镜像就存在这里。
纯干货分享
http://jiangjiang.space 回忆书签，转载请注明出处，谢谢
2. harbor中的registry
Harbor中包含一套标准的docker registry， 修改registry设置只需要修改这个registry的配置文件和compose文件即可。
按照harbor的compose文件
- harbor
registry:
image: vmware/registry:2.6.2-photon
container_name: registry
restart: always
volumes:
- /data:/storage:z
- ./common/config/registry/:/etc/registry/:z ## 这里看到registry的配置文件
修改./common/config/registry/config.yml文件：
#filesystem:
# rootdirectory: /storage
swift:
username: demo:swift # 刚才创建的swift子账户
password: xxxxxxxxtzhNH # 刚才生成出的key
authurl: http://ceph01/auth/v1.0 # swift api url
tenant: demo # 这个一定要写，否则报错
container: regswfit # 注意这里的container并非dockercontainer
注释 filesystem，由于registry直接用S3方式连接有问题，因此用swift api接口对接。
增加swift的配置， 注意这些键值一个都不能少， 后面会有少写key的报错信息。
我是一点一点磕碰出来的配置。
注意，这里的container不是docker container， 其实就是S3的存储筒， 这个筒要事先创建好，否则也报错。
3. 重启registry容器
# 找到 registry 容器
docker ps
1ea3ee9c0f8e vmware/registry:2.6.2-photon "/entrypoint.sh se..." 3 hours ago Up 2 hours
# 停止容器
docker stop 1ea
# 启动容器
docker start 1ea
观察日志看是否有报错：
tail -f /var/log/harbor/日期/registry.log
# 报错对应信息见附录
4. 测试harbor
推送镜像
docker push registry.demo.cn/pub/ubuntu:16.04
The push refers to a repository [registry.demo.cn/pub/ubuntu]
49907af65b0a: Pushed
4589f96366e6: Pushed
b97229212d30: Pushed
cd181336f142: Pushed
0f5ff0cf6a1c: Pushed
5. 附录
报错1：
Nov 2 18:02:54 172.29.248.1 registry[1095]: panic: No container parameter provided
Nov 2 18:02:54 172.29.248.1 registry[1095]:
Nov 2 18:02:54 172.29.248.1 registry[1095]: goroutine 1 [running]:
Nov 2 18:02:54 172.29.248.1 registry[1095]: panic(0xb7dc40, 0xc4203f9570)
Nov 2 18:02:54 172.29.248.1 registry[1095]: #011/usr/local/go/src/runtime/panic.go:500 +0x1a1
Nov 2 18:02:54 172.29.248.1 registry[1095]: github.com/docker/distribution/registry/handlers.NewApp(0x1067820, 0xc4203851d0, 0xc4202bf180, 0x1067820)
Nov 2 18:02:54 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/registry/handlers/app.go:123 +0x3908
Nov 2 18:02:54 172.29.248.1 registry[1095]: github.com/docker/distribution/registry.NewRegistry(0x7f13ea120198, 0xc4203851d0, 0xc4202bf180, 0xe, 0x0, 0x0)
Nov 2 18:02:54 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/registry/registry.go:86 +0x213
Nov 2 18:02:54 172.29.248.1 registry[1095]: github.com/docker/distribution/registry.glob..func1(0x108f1a0, 0xc42036acd0, 0x1, 0x1)
Nov 2 18:02:54 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/registry/registry.go:55 +0x106
Nov 2 18:02:54 172.29.248.1 registry[1095]: github.com/docker/distribution/vendor/github.com/spf13/cobra.(*Command).execute(0x108f1a0, 0xc42036ac80, 0x1, 0x1, 0x108f1a0, 0xc42036ac80)
Nov 2 18:02:54 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:495 +0x190
Nov 2 18:02:54 172.29.248.1 registry[1095]: github.com/docker/distribution/vendor/github.com/spf13/cobra.(*Command).Execute(0x108f340, 0xc4201b7f40, 0xc4200001a0)
Nov 2 18:02:54 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:560 +0x3c3
Nov 2 18:02:54 172.29.248.1 registry[1095]: main.main()
Nov 2 18:02:54 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/cmd/registry/main.go:24 +0x2d
原因： 没有写 container（也就是s3 得存储桶）
报错2：
Nov 2 18:09:31 172.29.248.1 registry[1095]: panic: Failed to create container registry (Bad Request)
Nov 2 18:09:31 172.29.248.1 registry[1095]:
Nov 2 18:09:31 172.29.248.1 registry[1095]: goroutine 1 [running]:
Nov 2 18:09:31 172.29.248.1 registry[1095]: panic(0xb7dc40, 0xc420295190)
Nov 2 18:09:31 172.29.248.1 registry[1095]: #011/usr/local/go/src/runtime/panic.go:500 +0x1a1
Nov 2 18:09:31 172.29.248.1 registry[1095]: github.com/docker/distribution/registry/handlers.NewApp(0x1067820, 0xc420397800, 0xc42037ca80, 0x1067820)
Nov 2 18:09:31 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/registry/handlers/app.go:123 +0x3908
Nov 2 18:09:31 172.29.248.1 registry[1095]: github.com/docker/distribution/registry.NewRegistry(0x7f383810c128, 0xc420397800, 0xc42037ca80, 0xe, 0x0, 0x0)
Nov 2 18:09:31 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/registry/registry.go:86 +0x213
Nov 2 18:09:31 172.29.248.1 registry[1095]: github.com/docker/distribution/registry.glob..func1(0x108f1a0, 0xc42030da30, 0x1, 0x1)
Nov 2 18:09:31 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/registry/registry.go:55 +0x106
Nov 2 18:09:31 172.29.248.1 registry[1095]: github.com/docker/distribution/vendor/github.com/spf13/cobra.(*Command).execute(0x108f1a0, 0xc42030d9e0, 0x1, 0x1, 0x108f1a0, 0xc42030d9e0)
Nov 2 18:09:31 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:495 +0x190
Nov 2 18:09:31 172.29.248.1 registry[1095]: github.com/docker/distribution/vendor/github.com/spf13/cobra.(*Command).Execute(0x108f340, 0xc4201b9f40, 0xc4200001a0)
Nov 2 18:09:31 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:560 +0x3c3
Nov 2 18:09:31 172.29.248.1 registry[1095]: main.main()
Nov 2 18:09:31 172.29.248.1 registry[1095]: #011/go/src/github.com/docker/distribution/cmd/registry/main.go:24 +0x2d
原因： 没有写tenant
报错3：
Swift authentication failed: HTTP Error: 405: 405 Method Not Allowed
原因： 存储筒不存在
报错4：
20190115 更新
原因： docker registry 开启了redirect配置(helm/k8s,harbor1.7环境)
panic: invalid type for redirect config: configuration.Parameters{"disable":interface {}(nil)}
设置 redirect: false
纯干货分享
http://jiangjiang.space 回忆书签，转载请注明出处，谢谢