Plugins / Docker Buildx

Docker Buildx

by Woodpecker Authors

dockerimagecontainerbuild

plugin to build multiarch Docker images with buildx

Overview

Woodpecker CI plugin to build multiarch Docker images with buildx. This plugin is a fork of thegeeklab/drone-docker-buildx which itself is a fork of drone-plugins/drone-docker.

Features

Build without push
Use custom registries
Build based on existing tags when needed
Push to multiple registries/repos
Use remote builders

It will automatically generate buildkit configuration to use custom CA certificate if the normal image is used or following conditions are met:

Setting buildkit_config is not set
Custom registry/logins value is provided
File exists /etc/docker/certs.d/<registry-value>/ca.crt

To mount custom CA certificates, the Woodpecker env var WOODPECKER_BACKEND_DOCKER_VOLUMES with value /etc/ssl/certs:/etc/ssl/certs:ro,/etc/docker/certs.d:/etc/docker/certs.d:ro can be used.

Settings

Settings Name	Default	Description
`dry-run`	`false`	disables docker push
`repo`	none	sets repository name for the image (can be a list)
`username`	none	sets username to authenticates with
`password`	none	sets password / token to authenticates with
`aws_access_key_id`	none	sets AWS_ACCESS_KEY_ID for AWS ECR auth
`aws_secret_access_key`	none	sets AWS_SECRET_ACCESS_KEY for AWS ECR auth
`aws_region`	`us-east-1`	sets AWS_DEFAULT_REGION for AWS ECR auth
`password`	none	sets password / token to authenticates with
`email`	none	sets email address to authenticates with
`registry`	`https://index.docker.io/v1/`	sets docker registry to authenticate with
`dockerfile`	`Dockerfile`	sets dockerfile to use for the image build
`tag`/`tags`	none	sets repository tags to use for the image
`platforms`	none	sets target platform for build
`provenance`	none	sets provenance for build
`remote-builders`	none	sets remote builders for build
`ssh-key`	none	sets an ssh key to connect to remote builders

auto_tag

If set to true, it will use the default_tag ("latest") on tag event or default branch. If it's a tag event it will also assume sem versioning and add tags accordingly (x, x.x and x.x.x). If it's not a tag event, and no default branch, automated tags are skipped.

Examples

publish-next-agent:
  image: woodpeckerci/plugin-docker-buildx
  settings:
    repo: woodpeckerci/woodpecker-agent
    dockerfile: docker/Dockerfile.agent.multiarch
    platforms: windows/amd64,darwin/amd64,darwin/arm64,freebsd/amd64,linux/amd64,linux/arm64/v8
    tag: next
    username:
      from_secret: docker_username
    password:
      from_secret: docker_password
  when:
    branch: ${CI_REPO_DEFAULT_BRANCH}
    event: push

publish:
  image: woodpeckerci/plugin-docker-buildx
  settings:
    platforms: linux/386,linux/amd64,linux/arm/v6,linux/arm64/v8,linux/ppc64le,linux/riscv64,linux/s390x
    repo: codeberg.org/${CI_REPO_OWNER}/hello
    registry: codeberg.org
    tags: latest
    username: ${CI_REPO_OWNER}
    password:
      from_secret: cb_token

docker-build:
  image: woodpeckerci/plugin-docker-buildx
  settings:
    repo: codeberg.org/${CI_REPO_OWNER}/hello
    registry: codeberg.org
    dry-run: true
    output: type=oci,dest=${CI_REPO_OWNER}-hello.tar

Advanced Settings

Settings Name	Default	Description
`mirror`	none	sets a registry mirror to pull images
`storage_driver`	none	sets the docker daemon storage driver
`storage_path`	`/var/lib/docker`	sets the docker daemon storage path
`bip`	none	allows the docker daemon to bride ip address
`mtu`	none	sets docker daemon custom mtu setting
`custom_dns`	none	sets custom docker daemon dns server
`custom_dns_search`	none	sets custom docker daemon dns search domain
`insecure`	`false`	allows the docker daemon to use insecure registries
`ipv6`	`false`	enables docker daemon IPv6 support
`experimental`	`false`	enables docker daemon experimental mode
`debug`	`false`	enables verbose debug mode for the docker daemon
`daemon_off`	`false`	disables the startup of the docker daemon
`buildkit_debug`	`false`	enables debug output of buildkit
`buildkit_config`	none	Can only be changed for insecure image. Sets content of the docker buildkit TOML config
`buildkit_driveropt`	none	Can only be changed for insecure image. Adds one or multiple `--driver-opt` buildx arguments for the default buildkit builder instance
`tags_file`	none	overrides the `tags` option with values in a file named `.tags`; multiple tags can be specified separated by a newline
`context`	`.`	sets the path of the build context to use
`auto_tag`	`false`	generates tag names automatically based on git branch and git tag, tags supplied via `tags` are additionally added to the `auto_tags` without suffix
`default_suffix`/`auto_tag_suffix`	none	generates tag names with the given suffix
`default_tag`	`latest`	overrides the default tag name used when generating with `auto_tag` enabled
`label`/`labels`	none	sets labels to use for the image in format `<name>=<value>`
`default_labels`/`auto_labels`	`true`	sets docker image labels based on git information
`build_args`	none	sets custom build arguments for the build
`build_args_from_env`	none	forwards environment variables as custom arguments to the build
`secrets`	none	Sets the build secrets for the build
`quiet`	`false`	enables suppression of the build output
`target`	none	sets the build target to use
`cache_from`	none	sets configuration for cache source
`cache_to`	none	sets configuration for cache export
`cache_images`	none	a list of images to use as cache.
`pull_image`	`true`	enforces to pull base image at build time
`compress`	`false`	enables compression of the build context using gzip
`config`	none	sets content of the docker daemon json config
`purge`	`true`	enables cleanup of the docker environment at the end of a build
`no_cache`	`false`	disables the usage of cached intermediate containers
`add_host`	none	sets additional host:ip mapping
`output`	none	sets build output in format`type=<type>[,<key>=<value>]`
`logins`	none	option to log into multiple registries
`env_file`	none	load env vars from specified file
`ecr_create_repository`	`false`	creates the ECR repository if it does not exist
`ecr_lifecycle_policy`	none	AWS ECR lifecycle policy
`ecr_repository_policy`	none	AWS ECR repository policy
`ecr_scan_on_push`	none	AWS: whether to enable image scanning on push
`http_proxy`	none	Set an http proxy if needed. It is also forwarded as build arg called "HTTP_PROXY".
`https_proxy`	none	Set an https proxy if needed. It is also forwarded as build arg called "HTTPS_PROXY".
`no_proxy`	none	Set (sub-)domains to be ignored by proxy settings. It is also forwarded as build arg called "NO_PROXY".

Multi registry push example

Only supported with woodpecker >= 1.0.0 (next-da997fa3).

settings:
  repo: a6543/tmp,codeberg.org/6543/tmp
  tag: demo
  logins:
    - registry: https://index.docker.io/v1/
      username: a6543
      password:
        from_secret: docker_token
      mirrors:
        - "my-docker-mirror-host.local"
    - registry: https://codeberg.org
      username: "6543"
      password:
        from_secret: cb_token
    - registry: https://<account-id>.dkr.ecr.<region>.amazonaws.com
      aws_region: <region>
      aws_access_key_id:
        from_secret: aws_access_key_id
      aws_secret_access_key:
        from_secret: aws_secret_access_key

Using `plugin-docker-buildx` behind a proxy

When performing a docker build behind a corporate proxy one needs to pass through the proxy settings to the plugin.

variables:
  # deployment targets
  - &publish_repos "codeberg.org/test"
  # logins for deployment targets
  - publish_logins: &publish_logins
      - registry: https://codeberg.org
        username:
          from_secret: CODEBERG_USER
        password:
          from_secret: CODEBERG_TOKEN

steps:
  test:
    image: woodpeckerci/plugin-docker-buildx:2
    privileged: true
    settings:
      dry-run: true
      repo: *publish_repos
      dockerfile: Dockerfile.multi
      platforms: linux/amd64
      auto_tag: true
      logins: *publish_logins
      # Adding custom dns server to lookup internal Docker Hub mirror.
      # custom_dns:
      #   - 192.168.55.31
      #   - 192.168.55.32
      # Adding an optional Docker Hub mirror for the nested dockerd.
      # mirror: https://my-mirror.example.com
      http_proxy: "http://X.Y.Z.Z:3128"
      https_proxy: "http://X.Y.Z.Z:3128"
      no_proxy: ".my-subdomain.com"

Using cache images

You can provide a list of images to use for cache. These cache images are built with mode=max, image-manifest=true, and oci-mediatypes=true. This is to provide better usage of cache and better compatibility with image stores like Harbor.

steps:
  build:
    image: woodpeckerci/plugin-docker-buildx
    settings:
      repo: hari/radiant
      cache_images:
        - hari/radiant:cache
        - harbor.example.com/hari/radiant:cache
      logins:
        - registry: https://index.docker.io/v1/
          username: hari
          password:
            from_secret: docker_password
        - registry: https://harbor.example.com
          username: hari
          password:
            from_secret: harbor_password

Using other cache types

You can specify cache_to and cache_from to use specific settings. For example you can configure an s3 object as cache.

More details can be found in the docker docs.

steps:
  build:
    image: woodpeckerci/plugin-docker-buildx
    settings:
      repo: hari/radiant
      cache_to: type=s3,region=east,bucket=mystuff,name=radiant-cache
      cache_from: type=s3,region=east,bucket=mystuff,name=radiant-cache

Using remote builders

When building for multiple platforms, you might want to offload some builds to a remote server, to avoid emulation. To support this, provide a list build servers to remote-builders. These servers will need key authentication, so you will also need to provide a (private) SSH key.

build:
  image: woodpeckerci/plugin-docker-buildx
  settings:
    platforms: linux/amd64,linux/arm64
    repo: codeberg.org/${CI_REPO_OWNER}/hello
    registry: codeberg.org
    dry-run: true
    ssh-key:
      from_secret: ssh_key
    remote-builders: root@my-amd64-build-server,root@my-arm64-build-server

If you want to mix local and remote builders, the list can include "local":

build:
  image: woodpeckerci/plugin-docker-buildx
  settings:
    platforms: linux/amd64,linux/arm64
    repo: codeberg.org/${CI_REPO_OWNER}/hello
    registry: codeberg.org
    dry-run: true
    ssh-key:
      from_secret: ssh_key
    remote-builders: local,root@my-arm64-build-server