码迷，mamicode.com
首页 > 其他好文 > 详细

Docker registry垃圾回收

时间：2018-11-27 11:10:26      阅读：558      评论：0      收藏：0      [点我收藏+]

标签：ora   val   for   epo   rom   name   type   his   diff   

Docker registry垃圾回收

通过：

docker run -p 5000:5000 -v /netdata/xxxx/registry:/var/lib/registry registry.cn-hangzhou.aliyuncs.com/term/registry-gc:2.7 registry garbage-collect /var/lib/registry/config.yml

搭建好registry服务后，docker push alpine:3.8镜像至registry服务。其中alpine的dockerfile内容如下：

FROM scratch
ADD rootfs.tar.xz /
CMD ["/bin/sh"]

registry.cn-hangzhou.aliyuncs.com/term/registry-gc:2.7的dockerfile如下:

FROM alpine:3.4

RUN set -ex     && apk add --no-cache ca-certificates apache2-utils

COPY registry /bin/registry
COPY config.yml /etc/docker/registry/config.yml

VOLUME ["/var/lib/registry"]
EXPOSE 5000

COPY docker-entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

CMD ["/etc/docker/registry/config.yml"]

config.yml内容如下:

version: 0.1
log:
  fields:
    service: registry
storage:
  cache:
    blobdescriptor: inmemory
  filesystem:
    rootdirectory: /var/lib/registry
  delete:
    enabled: true
  maintenance:
    readonly:
      enabled: false
http:
  addr: :5000
  headers:
    X-Content-Type-Options: [nosniff]
health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3

后端存储

镜像推送至registry后，可查看registry后端的本地存储目录结构如下：

docker
`-- registry
    `-- v2
        |-- blobs
        |   `-- sha256
        |       |-- 01
        |       |   `-- 0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc
        |       |       `-- data
        |       |-- 1e
        |       |   `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667
        |       |       `-- data
        |       `-- 5a
        |           `-- 5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f
        |               `-- data
        `-- repositories
            `-- alpine
                |-- _layers
                |   `-- sha256
                |       |-- 0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc
                |       |   `-- link
                |       `-- 5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f
                |           `-- link
                |-- _manifests
                |   |-- revisions
                |   |   `-- sha256
                |   |       `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667
                |   |           `-- link
                |   `-- tags
                |       `-- 3.8
                |           |-- current
                |           |   `-- link
                |           `-- index
                |               `-- sha256
                |                   `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667
                |                       `-- link
                `-- _uploads

查看Manifest

查看alpine:3.8的manifest文件内容：

curl -i -H "Accept:application/vnd.docker.distribution.manifest.v2+json" http://127.0.0.1:5000/v2/alpine/manifests/3.8

Response Header:

HTTP/1.1 200 OK
Content-Length: 526
Content-Type: application/vnd.docker.distribution.manifest.v2+json
Docker-Content-Digest: sha256:1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667
Docker-Distribution-Api-Version: registry/2.0
Etag: "sha256:1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667"
X-Content-Type-Options: nosniff
Date: Tue, 04 Sep 2018 11:59:14 GMT

Response body:

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 1512,
      "digest": "sha256:0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc"
   },
   "layers": [
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 12456,
         "digest": "sha256:5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f"
      }
   ]
}

执行垃圾回收

sudo docker run --rm --name gctest -v /home/term/registry:/var/lib/registry registry:2.5.1 garbage-collect  -d /var/lib/registry/config.yml

得到的结果如下：

alpine
alpine: marking manifest sha256:1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667
alpine: marking blob sha256:5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f
alpine: marking configuration sha256:0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc

3 blobs marked, 0 blobs eligible for deletion

发现所有的blobs都被mark, 无blobs可清理

删除Manifest

curl -X DELETE http://127.0.0.1:5000/v2/alpine/manifests/sha256:1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667

sha256:1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667从获取Manifest时的Header: Docker-Content-Digest里获取。

调用manifest API删除manifest文件后，再次查看registry后端存储目录结构:

docker
`-- registry
    `-- v2
        |-- blobs
        |   `-- sha256
        |       |-- 01
        |       |   `-- 0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc
        |       |       `-- data
        |       |-- 1e
        |       |   `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667
        |       |       `-- data
        |       |-- 5a
        |       |   `-- 5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f
        |       |       `-- data
        |       `-- a3
        |           `-- a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
        |               `-- data
        `-- repositories
            `-- alpine
                |-- _layers
                |   `-- sha256
                |       |-- 0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc
                |       |   `-- link
                |       |-- 5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f
                |       |   `-- link
                |       `-- a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
                |           `-- link
                |-- _manifests
                |   |-- revisions
                |   |   `-- sha256
                |   |       `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667
                |   `-- tags
                `-- _uploads

对比之前的目录，发现alpine下的tags目录内容已被删

再次垃圾回收

再次执行垃圾回收：

sudo docker run --rm --name gctest -v /home/term/registry:/var/lib/registry registry:2.5.1 garbage-collect  -d /var/lib/registry/config.yml

得到结果如下：

alpine

0 blobs marked, 4 blobs eligible for deletion
blob eligible for deletion: sha256:0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc
blob eligible for deletion: sha256:1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667
blob eligible for deletion: sha256:5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f
blob eligible for deletion: sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4

以上为dry-run方式执行。

真正执行gc时，日志如下:

time="2018-09-05T01:50:32Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/08/0873c923e00e0fd2ba78041bfb64a105e1ecb7678916d1f7776311e45bf5634b" go.version=go1.6.3 instance.id=ccafa0cc-4335-4535-b16a-9bff7886c896
time="2018-09-05T01:50:32Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/11/11cd0b38bc3ceb958ffb2f9bd70be3fb317ce7d255c8a4c3f4af30e298aa1aab" go.version=go1.6.3 instance.id=ccafa0cc-4335-4535-b16a-9bff7886c896
time="2018-09-05T01:50:32Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/1e/1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667" go.version=go1.6.3 instance.id=ccafa0cc-4335-4535-b16a-9bff7886c896
time="2018-09-05T01:50:32Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/5a/5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f" go.version=go1.6.3 instance.id=ccafa0cc-4335-4535-b16a-9bff7886c896
time="2018-09-05T01:50:32Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/8e/8e3ba11ec2a2b39ab372c60c16b421536e50e5ce64a0bc81765c2e38381bcff6" go.version=go1.6.3 instance.id=ccafa0cc-4335-4535-b16a-9bff7886c896
time="2018-09-05T01:50:32Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/a3/a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" go.version=go1.6.3 instance.id=ccafa0cc-4335-4535-b16a-9bff7886c896
time="2018-09-05T01:50:32Z" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/01/0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc" go.version=go1.6.3 instance.id=ccafa0cc-4335-4535-b16a-9bff7886c896

gc只清除blobs下的数据，但不会清除repositories里的内容及blobs下对应的目录。

docker
`-- registry
    `-- v2
        |-- blobs
        |   `-- sha256
        |       |-- 01
        |       |-- 08
        |       |-- 11
        |       |-- 1e
        |       |-- 5a
        |       |-- 8e
        |       `-- a3
        `-- repositories
            `-- alpine
                |-- _layers
                |   `-- sha256
                |       |-- 0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc
                |       |   `-- link
                |       |-- 11cd0b38bc3ceb958ffb2f9bd70be3fb317ce7d255c8a4c3f4af30e298aa1aab
                |       |   `-- link
                |       |-- 5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f
                |       |-- 8e3ba11ec2a2b39ab372c60c16b421536e50e5ce64a0bc81765c2e38381bcff6
                |       `-- a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
                |           `-- link
                |-- _manifests
                |   |-- revisions
                |   |   `-- sha256
                |   |       |-- 0873c923e00e0fd2ba78041bfb64a105e1ecb7678916d1f7776311e45bf5634b
                |   |       `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667
                |   `-- tags
                `-- _uploads

删除blobs

调用registry api删除blob：

curl -i -X DELETE http://127.0.0.1:5000/v2/alpine/blobs/sha256:5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f

Response Header:

HTTP/1.1 202 Accepted
Content-Length: 0
Docker-Distribution-Api-Version: registry/2.0
X-Content-Type-Options: nosniff
Date: Wed, 05 Sep 2018 01:05:20 GMT
Content-Type: text/plain; charset=utf-8

Reponse body:

docker
`-- registry
    `-- v2
        |-- blobs
        |   `-- sha256
        |       |-- 01
        |       |   `-- 0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc
        |       |       `-- data
        |       |-- 1e
        |       |   `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667
        |       |       `-- data
        |       |-- 5a
        |       |   `-- 5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f
        |       |       `-- data
        |       `-- a3
        |           `-- a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
        |               `-- data
        `-- repositories
            `-- alpine
                |-- _layers
                |   `-- sha256
                |       |-- 0107b3d497f0786d2d586d6093015ac5c64140794798fe140728f78c21af7bdc
                |       |   `-- link
                |       |-- 5a83255105eaf8d24ad660befe32d7b9d449889ffde99b3ec59caf8edee59a3f
                |       `-- a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
                |           `-- link
                |-- _manifests
                |   |-- revisions
                |   |   `-- sha256
                |   |       `-- 1e7481c6b78ad4b425f1985e3324117075ab3ebe2ef99d54a578d0f8c1fe0667
                |   `-- tags
                `-- _uploads

对比之前目录结构，发现:docker/registry/v2/repositoires/<name>/_layers/<algorithm>/<digest>/link文件被删。

Config内容

{
    "architecture": "amd64",
    "config": {
        "ArgsEscaped": true,
        "AttachStderr": false,
        "AttachStdin": false,
        "AttachStdout": false,
        "Cmd": [
            "/bin/sh"
        ],
        "Domainname": "",
        "Entrypoint": null,
        "Env": [
            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
        ],
        "Hostname": "",
        "Image": "sha256:80c02d9956b63482b0e4344c56cc029d8c330e01a3f6093516b0a60b83118b9c",
        "Labels": null,
        "OnBuild": null,
        "OpenStdin": false,
        "StdinOnce": false,
        "Tty": false,
        "User": "",
        "Volumes": null,
        "WorkingDir": ""
    },
    "container": "bf174046ba695f7f6c1d841dd2894b2c34a96d73bb832c102b3ba2b6a0d20120",
    "container_config": {
        "ArgsEscaped": true,
        "AttachStderr": false,
        "AttachStdin": false,
        "AttachStdout": false,
        "Cmd": [
            "/bin/sh",
            "-c",
            "#(nop) ",
            "CMD [\"/bin/sh\"]"
        ],
        "Domainname": "",
        "Entrypoint": null,
        "Env": [
            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
        ],
        "Hostname": "bf174046ba69",
        "Image": "sha256:80c02d9956b63482b0e4344c56cc029d8c330e01a3f6093516b0a60b83118b9c",
        "Labels": {},
        "OnBuild": null,
        "OpenStdin": false,
        "StdinOnce": false,
        "Tty": false,
        "User": "",
        "Volumes": null,
        "WorkingDir": ""
    },
    "created": "2018-09-04T09:32:26.252138866Z",
    "docker_version": "17.06.2-ce",
    "history": [
        {
            "created": "2018-09-04T09:32:25.454551817Z",
            "created_by": "/bin/sh -c #(nop) ADD file:babb191309280a09293bff66ca4aead1ecad6119d9add3a803d489fa866d5282 in / "
        },
        {
            "created": "2018-09-04T09:32:26.252138866Z",
            "created_by": "/bin/sh -c #(nop)  CMD [\"/bin/sh\"]",
            "empty_layer": true
        }
    ],
    "os": "linux",
    "rootfs": {
        "diff_ids": [
            "sha256:2a7f9859cf97588d6533a9ac63c988b34beff0cc2758182b70e7a8cc8383d7f3"
        ],
        "type": "layers"
    }
}

结论

  1. 要想调用registry API，请打开deletable=true的开关，可通过环境变量: REGISTRY_STORAGE_DELETE_ENABLED=true指定, 也可在config.yml里配置
  2. 镜像清除，删除manifest即可，无需调用API删除layer，真正Blob清理需执行garbage-collect.
  3. 执行garbage-collect前，请将regsitry的readonly=true开关打开，重启registry，保证gc时，registry只读。此时不允许docker push镜像，否则会清理掉gc过程中新push的blobs。清理完成后，将readonly开关关闭,再次重启registry。
  4. 目前registry暂无GC API，只能通过脚本执行GC工作。

坑点

  1. 目前registry不支持通过环境变量设置readonly=true，在registry:2.5.1版本试过，会导致registry起不来，报错:
goroutine 1 [running]:
panic(0xc2a220, 0xc820438620)
    /usr/local/go/src/runtime/panic.go:481 +0x3e6
github.com/docker/distribution/registry/handlers.NewApp(0x7fd101c70ad8, 0xc8203ba780, 0xc82000d600, 0x7fd101c70ad8)
    /go/src/github.com/docker/distribution/registry/handlers/app.go:135 +0xc44
github.com/docker/distribution/registry.NewRegistry(0x7fd101c70b18, 0xc8203ba780, 0xc82000d600, 0xc82000d600, 0x0, 0x0)
    /go/src/github.com/docker/distribution/registry/registry.go:86 +0x2b0
github.com/docker/distribution/registry.glob.func1(0x14cc660, 0xc820347f50, 0x1, 0x1)
    /go/src/github.com/docker/distribution/registry/registry.go:55 +0x2a9
github.com/docker/distribution/vendor/github.com/spf13/cobra.(*Command).execute(0x14cc660, 0xc820347f00, 0x1, 0x1, 0x0, 0x0)
    /go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:495 +0x6d4
github.com/docker/distribution/vendor/github.com/spf13/cobra.(*Command).Execute(0x14cc800, 0x0, 0x0)
    /go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:560 +0x180
main.main()

须通过额外挂载配置文件解决

Docker registry垃圾回收

标签：ora   val   for   epo   rom   name   type   his   diff   

原文地址：https://www.cnblogs.com/blogabc/p/10024765.html
踩
(0)
赞
(0)
   
举报
评论 一句话评论（0）
登录后才能评论！
分享档案
更多>
2021年07月29日 (22)
2021年07月28日 (40)
2021年07月27日 (32)
2021年07月26日 (79)
2021年07月23日 (29)
2021年07月22日 (30)
2021年07月21日 (42)
2021年07月20日 (16)
2021年07月19日 (90)
2021年07月16日 (35)
周排行
mamicode.com排行更多图片
更多
友情链接
兰亭集智   国之画   百度统计   站长统计   阿里云   chrome插件   新版天听网
关于我们 - 联系我们 - 留言反馈
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码！