We have a Tekton pipeline task in Openshift which uses Kaniko to build a container image and then push it to a external registry.
The build step of the task always fails with the error:
error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "xxxxxxx.io/xxxxxxxxxxx/frontend:frontend-secure-pipeline-csd6ur": POST https://xxxxxxx.io/v2/xxxxxxxxxxx/frontend/blobs/uploads/: DENIED: You are not authorized to access the specified resource. See https://cloud.ibm.com/docs/Registry?topic=Registry-troubleshoot-access-denied; [map[Action:pull Class: Name:xxxxxxxxxxx/frontend Type:repository] map[Action:push Class: Name:xxxxxxxxxxx/frontend Type:repository]]
The task yaml is (annotations, comments and later steps removed for brevity)...
apiVersion: tekton.dev/v1beta1kind: Taskmetadata: annotations: name: source-to-image-cache namespace: openshift-pipelines resourceVersion: '259117173' uid: xxxxxxxxxxxxxxxxxxxxxspec: params: - description: The path to the dockerfile to build name: pathToDockerfile type: string - default: . description: >- The build context used by Kaniko (https://github.com/GoogleContainerTools/kaniko#kaniko-build-contexts) name: pathToContext type: string - default: '' description: Image tag name: imageTag type: string - default: sbu-pipeline description: Is the name of the base image registry namespace secret name: base-reg-secret-name type: string - default: sbu-pipeline description: Is the name of the built image registry namespace secret name: build-reg-secret-name type: string - default: '' description: >- This is the URL to save the built image to - used with imageRespositoryPath below name: imageRegistryUrl type: string - default: '' description: >- This is the repository path at the imageRegistryUrl to save the built image to name: imageRepositoryPath type: string results: - description: The image SHA code for the built application name: image-digest type: string steps: - image: 'uk.icr.io/sbu-pipeline/alpine-curl-jq:6' imagePullPolicy: IfNotPresent name: merge-json resources: {} script: > #!/usr/bin/env bash set +x printf "\nSorting out config.json for Kaniko.\n" diff /home/.dockerwip/base.dockerconfigjson /home/.dockerwip/build.dockerconfigjson -q myDiff=$? if [ "$myDiff" -gt 0 ]; then # Hopefully short-term fix until Kaniko supports multiple auths... cp /home/.dockerwip/base.dockerconfigjson /home/.docker/config.json else cp /home/.dockerwip/base.dockerconfigjson /home/.docker/config.json fi printf "\nFinished sorting out config.json for Kaniko.\n" securityContext: runAsUser: 0 volumeMounts: - mountPath: /home/.dockerwip/base.dockerconfigjson name: base-registry-creds subPath: base.dockerconfigjson - mountPath: /home/.dockerwip/build.dockerconfigjson name: build-registry-creds subPath: build.dockerconfigjson - mountPath: /home/.docker name: docker-config - resources: {} name: build command: - /kaniko/executor env: - name: DOCKER_CONFIG value: /kaniko/.docker/ securityContext: runAsUser: 0 imagePullPolicy: IfNotPresent volumeMounts: - mountPath: /kaniko/.docker/ name: docker-config image: 'gcr.io/kaniko-project/executor:v1.16.0' args: - >- --dockerfile=$(workspaces.source.path)/$(inputs.params.pathToDockerfile) - >- --destination=$(params.imageRegistryUrl)/$(params.imageRepositoryPath):$(inputs.params.imageTag) - '--context=$(workspaces.source.path)/$(inputs.params.pathToContext)' - '--single-snapshot=true' - '--image-name-with-digest-file=$(workspaces.source.path)/image-digest' - '--cache=true' - '--cache-copy-layers=true' - '--use-new-run=true' volumes: - emptyDir: {} name: docker-config - name: base-registry-creds secret: items: - key: .dockerconfigjson path: base.dockerconfigjson secretName: $(params.base-reg-secret-name) - name: build-registry-creds secret: items: - key: .dockerconfigjson path: build.dockerconfigjson secretName: $(params.build-reg-secret-name) workspaces: - name: source
I've tested the credentials locally and everything seems fine. I can mount the secret containing the docker credentials in another container and then successfully login, pull and push images. The error only occurs with Kaniko. It seems Kaniko is not reading the DOCKER_CONFIG environment variable. I've tried multiple different Kaniko versions without success.
Any suggestions on possible fixes or next steps in debugging?