Skip to content

Kubernetes deploy daemon ignores registry mappings (app_api:daemon:registry:add) #1039

Description

@oleksandr-nc

Problem

A deploy daemon can carry registry mappings (occ app_api:daemon:registry:add <daemon> --registry-from ghcr.io --registry-to registry.example.com), so ExApp images are taken from a mirror or a private registry instead of the registry named in the ExApp's info.xml. This is what air-gapped and registry-restricted environments rely on.

The mapping is only applied by the Docker deploy backend. With a Kubernetes deploy daemon the command succeeds, registry:list shows the mapping, and nothing else happens: the ExApp Deployment is still created with the original image reference, so the cluster tries to pull from the upstream registry.

Steps to reproduce

  1. Register a Kubernetes deploy daemon (kubernetes-install).
  2. occ app_api:daemon:registry:add <daemon> --registry-from ghcr.io --registry-to registry.example.com
  3. occ app_api:daemon:registry:list <daemon> lists ghcr.io -> registry.example.com.
  4. Deploy any ExApp whose image lives on ghcr.io, e.g. run Test Deploy.
  5. kubectl -n <exapps-namespace> get deploy -o wide shows the image as ghcr.io/..., not registry.example.com/....

Expected

The Deployment is created with registry.example.com/<image>:<tag>, the same way the Docker backend rewrites the image before pulling.

Cause

DockerActions resolves the mapping in buildBaseImageName(), buildExtendedImageName() and shouldPullImage(). KubernetesActions::buildImageName() concatenates image_src/image_name:image_tag from info.xml and never looks at deploy_config['registries']. The registry:* commands do not check the daemon type, so they accept a Kubernetes daemon without any hint that the mapping is unused there.

Affects main, stable35 and stable34 (every branch that has the Kubernetes backend).

Proposal

  • Apply the daemon's registry mapping in KubernetesActions when the create payload for HaRP is built, ideally through one shared helper so both backends resolve image names identically.
  • Decide what the special local target means on Kubernetes. AppAPI does not pull there, the kubelet does, and ExApp pods use imagePullPolicy: IfNotPresent, so keeping the image name unchanged (a preloaded image is then used) looks like the natural equivalent.
  • Cover it with a unit test for the Kubernetes image name, and mention Kubernetes in the registry mapping documentation.

Related, to decide while at it

A mirror that needs authentication also requires pull credentials in the cluster. HaRP does not set imagePullSecrets on ExApp pods today, so this currently has to be solved outside (for example imagePullSecrets on the namespace's default ServiceAccount). Worth documenting, or supporting as a HaRP option.

Workarounds until this is supported

  • Configure a registry mirror for the upstream registry in the container runtime of the cluster nodes (containerd hosts.toml). Transparent for all ExApps.
  • Rewrite the image registry at admission time for the ExApps namespace, e.g. with a Kyverno mutate policy (sample).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    daemondeployRelated ExApp deploymentenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions