nydus vs estargz image format对比
背景
OCIv2 vs OCIv1
OCIv1是开源容器 规定的镜像格式标准(image format spec).
OCIv2是解决OCIv1部分问题后，解决的镜像标准改进版（希望向下兼容美好愿望）
目前希望解决的事项：
- 减少重复： 解决大tar文件， 变成bucket块
- 规范表示（可再现图像构建）： 通过
metaServer元数据服务来管理， 表示image各个层的兼容性，依赖关系。 - 显式（和最小）文件系统对象和元数据: 可显示的元数据系统
- 可挂载文件系统格式： 类似于FUSE 的文件系统，解决tar文件不规范性
- 镜像清单： 后续的扫描工具等，可以按此
清单实现标准化工具等 - 延迟获取支持: 解决多租户下，镜像文件隔离
- 可扩展性: 新的文件格式，替换tar格式
- 可验证性和可修复性： 容器运行时，可以判断镜像文件损坏与恢复
- 减少上传: 更改注册逻辑，制作镜像层依赖，不做实际上传
- 限制不受信任的存储： 每一次文件进行加密和验证
OCIv1目前格式：
# docker run --network host dongjiang1989/skopeo:latest --insecure-policy copy docker://nginx:latest oci:nginx:test
Getting image source signatures
Copying blob sha256:e3b6889c89547ec9ba653ab44ed32a99370940d51df956968c0d578dd61ab665
Copying blob sha256:52d2b7f179e32b4cbd579ee3c4958027988f9a8274850ab0c7c24661e3adaac5
Copying blob sha256:fd9f026c631046113bd492f69761c3ba6042c791c35a60e7c7f3b8f254592daa
Copying blob sha256:055fa98b43638b67d10c58d41094d99c8696cc34b7a960c7a0cc5d9d152d12b3
Copying blob sha256:96576293dd2954ff84251aa0455687c8643358ba1b190ea1818f56b41884bdbd
Copying blob sha256:a7c4092be9044bd4eef78f27c95785ef3a9f345d01fd4512bc94ddaaefc359f4
Copying blob sha256:da761d9a302b21dc50767b67d46f737f5072fb4490c525b4a7ae6f18e1dbbf75
Copying config sha256:f85e46e0f752f080a53656ee53282fdaa34dfd33ae723d19a522896019d77649
Writing manifest to image destination
Storing signatures
# ll
总用量 68852
-rw-r--r-- 1 root root 628 8月 28 11:36 055fa98b43638b67d10c58d41094d99c8696cc34b7a960c7a0cc5d9d152d12b3
-rw-r--r-- 1 root root 29124563 8月 28 11:36 52d2b7f179e32b4cbd579ee3c4958027988f9a8274850ab0c7c24661e3adaac5
-rw-r--r-- 1 root root 958 8月 28 11:36 96576293dd2954ff84251aa0455687c8643358ba1b190ea1818f56b41884bdbd
-rw-r--r-- 1 root root 371 8月 28 11:36 a7c4092be9044bd4eef78f27c95785ef3a9f345d01fd4512bc94ddaaefc359f4
-rw-r--r-- 1 root root 1405 8月 28 11:36 da761d9a302b21dc50767b67d46f737f5072fb4490c525b4a7ae6f18e1dbbf75
-rw-r--r-- 1 root root 1214 8月 28 11:36 e3b6889c89547ec9ba653ab44ed32a99370940d51df956968c0d578dd61ab665
-rw-r--r-- 1 root root 8146 8月 28 11:36 eea7b3dcba7ee47c0d16a60cc85d2b977d166be3960541991f3e6294d795ed24
-rw-r--r-- 1 root root 41338560 8月 28 11:36 fd9f026c631046113bd492f69761c3ba6042c791c35a60e7c7f3b8f254592daa
-rw-r--r-- 1 root root 1778 8月 28 11:36 manifest.json
-rw-r--r-- 1 root root 33 8月 28 11:36 version
# find . -type f | xargs file -z
./96576293dd2954ff84251aa0455687c8643358ba1b190ea1818f56b41884bdbd: POSIX tar archive (gzip compressed data)
./e3b6889c89547ec9ba653ab44ed32a99370940d51df956968c0d578dd61ab665: POSIX tar archive (gzip compressed data)
./version: ASCII text
./eea7b3dcba7ee47c0d16a60cc85d2b977d166be3960541991f3e6294d795ed24: ASCII text, with very long lines, with no line terminators
./055fa98b43638b67d10c58d41094d99c8696cc34b7a960c7a0cc5d9d152d12b3: POSIX tar archive (gzip compressed data)
./fd9f026c631046113bd492f69761c3ba6042c791c35a60e7c7f3b8f254592daa: POSIX tar archive (gzip compressed data)
./52d2b7f179e32b4cbd579ee3c4958027988f9a8274850ab0c7c24661e3adaac5: POSIX tar archive (gzip compressed data)
./manifest.json: ASCII text
./a7c4092be9044bd4eef78f27c95785ef3a9f345d01fd4512bc94ddaaefc359f4: POSIX tar archive (gzip compressed data)
./da761d9a302b21dc50767b67d46f737f5072fb4490c525b4a7ae6f18e1dbbf75: POSIX tar archive (gzip compressed data)
nydus-snapshotter 使用
$ vi kind-config.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
ipFamily: dual
nodes:
- role: control-plane
image: kindest/node:v1.23.4
extraMounts:
- hostPath: ./containerd-config.toml
containerPath: /etc/containerd/config.toml
- hostPath: /dev/fuse
containerPath: /dev/fuse # fuse挂载
$ vi containerd-config.toml
version = 2
[debug]
level = "debug"
[plugins."io.containerd.grpc.v1.cri".containerd]
discard_unpacked_layers = false
disable_snapshot_annotations = false
snapshotter = "overlayfs"
default_runtime_name = "runc"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
[plugins."io.containerd.grpc.v1.cri"]
sandbox_image = "registry.k8s.io/pause:3.6"
构建环境：
$ kind create cluster --config=kind-config.yaml --name nydus
Creating cluster "nydus" ...
✓ Ensuring node image (kindest/node:v1.23.4) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-nydus"
You can now use your cluster with:
kubectl cluster-info --context kind-nydus
Thanks for using kind! 😊
安装 nydus-snapshotter
$ helm install --wait --timeout 10m --dependency-update --create-namespace --namespace nydus-system -f ../config-nydus.yaml nydus-snapshotter charts/nydus-snapshotter
NAME: nydus-snapshotter
LAST DEPLOYED: Fri Aug 25 16:29:16 2023
NAMESPACE: nydus-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Thank you for installing nydus-snapshotter.
Your release is named nydus-snapshotter.
To learn more about the release, try:
$ kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-64897985d-7x82l 1/1 Running 0 18m
kube-system coredns-64897985d-rj4fw 1/1 Running 0 18m
kube-system etcd-nydus-control-plane 1/1 Running 0 18m
kube-system kindnet-kd5cv 1/1 Running 0 18m
kube-system kube-apiserver-nydus-control-plane 1/1 Running 0 18m
kube-system kube-controller-manager-nydus-control-plane 1/1 Running 0 18m
kube-system kube-proxy-p42wj 1/1 Running 0 18m
kube-system kube-scheduler-nydus-control-plane 1/1 Running 0 18m
local-path-storage local-path-provisioner-5ddd94ff66-g6jf9 1/1 Running 0 18m
nydus-system nydus-snapshotter-q5stt 1/1 Running 0 3m2s
生态
- 历史OCIv1 image转换： Nerdctl.
- 构建新的格式OCIv2：Buildkit.
- 使用 docker buildx 插件
- 运行时：nydus-snapshotter 扩展 containerd生态
- 其他：对接Dragonfly P2P 系统； 兼容stargz格式； nydus-image自有client;
Nydus vs eStargz
| Nydus | eStargz | |
|---|---|---|
| 格式转换 | ✅ | ✅ |
| 构建OCIv2新格式-Buildkit、nerdctl、Kaniko | ✅ | ✅🚀 |
| 镜像兼容OCIv1 | ✅ | ✅ |
| 支持kubernetes运行 | ✅ | ✅ |
| image转换工具 | ✅ | ✅ |
| 注册表转换 | ❌ | ✅ |
| Dragonfly P2P | ✅🚀 | ❌ |
| 周边生态 比如podman、docker | ✅ | ❌ |
要求
- 内核版本 5.x, cgroup v2；
- kubernetes 1.23+ 需要支持oci interface；
- 涉及到image overlay的组成，需要使用docker buildx编译，本地docker build都不可以；
- 历史镜像需要通过ctl工具转换；
其他
「如果这篇文章对你有用,请随意打赏」
如果这篇文章对你有用,请随意打赏
使用微信扫描二维码完成支付
