Github Actions Docker Pipeline
Something that I have been using more and more is Github Actions. It is an easy to use tool to run tests and build in CI. One of the nice things about it is that is is more module than GitLab Pipeline. There is also a marketplace of company and user created actions so there are many options to choose when you are looking to build something out.
I have used them to build docker pipelines for container building and deployment. I found that actions workflow were great to keep a simple and clean workflow for my pypy-flask docker image. The full workflow file is available in the repo but I will be breaking it down below.
Full file:
name: Docker
on:
push:
jobs:
Slim:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Login to Docker
uses: docker/login-action@v1
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login To GitHub
uses: docker/login-action@v1
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.CR_PAT }}
- name: Login To GitLab
uses: docker/login-action@v1
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
with:
registry: registry.gitlab.com
username: ${{ secrets.GITLAB_USER }}
password: ${{ secrets.GITLAB_TOKEN }}
- name: Docker Meta
id: meta
uses: docker/metadata-action@v3
with:
images: cyb3rjak3/pypy-flask,ghcr.io/cyb3r-jak3/pypy-flask,registry.gitlab.com/cyb3r-jak3/pypy-flask
tags: |
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=sha
labels: |
org.label-schema.vcs-url=https://github.com/Cyb3r-Jak3/pypy-flask.git
org.label-schema.schema-version=1.0.0-rc1
- name: Set up QEMU
uses: docker/setup-qemu-action@v1.2.0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1.3.0
- name: Cache Docker layers
uses: actions/cache@v2.1.6
with:
path: /tmp/.buildx-cache
key: buildx-slim-${{ github.sha }}
restore-keys: buildx-slim
- name: Slim Build and Push
uses: docker/build-push-action@v2.5.0
with:
platforms: linux/amd64,linux/arm64
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache
push: ${{ startsWith(github.ref, 'refs/tags/v') }}
file: Dockerfile
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
Alpine:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Login to Docker
uses: docker/login-action@v1
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Login To GitHub
uses: docker/login-action@v1
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.CR_PAT }}
- name: Login To GitLab
uses: docker/login-action@v1
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
with:
registry: registry.gitlab.com
username: ${{ secrets.GITLAB_USER }}
password: ${{ secrets.GITLAB_TOKEN }}
- name: Docker Meta
id: meta
uses: docker/metadata-action@v3
with:
images: cyb3rjak3/pypy-flask,ghcr.io/cyb3r-jak3/pypy-flask,registry.gitlab.com/cyb3r-jak3/pypy-flask
flavor: |
suffix=-alpine
tags: |
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=sha
labels: |
org.label-schema.vcs-url=https://github.com/Cyb3r-Jak3/pypy-flask.git
org.label-schema.schema-version=1.0.0-rc1
- name: Set up QEMU
uses: docker/setup-qemu-action@v1.2.0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1.3.0
- name: Cache Docker layers
uses: actions/cache@v2.1.6
with:
path: /tmp/.buildx-cache
key: buildx-alpine-${{ github.sha }}
restore-keys: buildx-alpine
- name: Alpine Build and Push
uses: docker/build-push-action@v2.5.0
with:
platforms: linux/amd64,linux/arm64
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache
push: ${{ startsWith(github.ref, 'refs/tags/v') }}
file: alpine.Dockerfile
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
It looks like a lot but the same job is close to being repeated twice to have better caching and means they run in parallel.
Breaking it down
Main Step
# Name of the Workflow
name: Docker
# When to run the workflow
on:
# Run on all push
push:
# List of the jobs
jobs:
# Name of the job
Slim:
# Operating system to run. Can be ubuntu, windows or mac
runs-on: ubuntu-latest
# List of steps for the job
steps:
After setting up the workflow there are some starter steps for checking out the repo and logging in.
Where there is uses
in a step it means that the step is using a custom workflow.
# Checkout the repo
- name: Checkout
uses: actions/checkout@v2
# Log into DockerHub
- name: Login to Docker
uses: docker/login-action@v1
# The IF statement means that only tags that start with v will run this step
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
# With passes action specific input. Used here with the login creds for DockerHub
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
# Login into GitHub's container registry
- name: Login To GitHub
uses: docker/login-action@v1
# The IF statement means that only tags that start with v will run this step
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
# With passes action specific input. Used here with the login creds for GitHub
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.CR_PAT }}
# Login in GitLab's Container registry
- name: Login To GitLab
uses: docker/login-action@v1
# The IF statement means that only tags that start with v will run this step
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
# With passes action specific input. Used here with the login creds for GitLab
with:
registry: registry.gitlab.com
username: ${{ secrets.GITLAB_USER }}
password: ${{ secrets.GITLAB_TOKEN }}
Next the metadata is setup for building the container
- name: Docker Meta
# ID allows us to use the output of the step in later steps
id: meta
uses: docker/metadata-action@v3
# With passes action specific input. Used here to pass the baseline images, tags to generate and labels to add
with:
images: cyb3rjak3/pypy-flask,ghcr.io/cyb3r-jak3/pypy-flask,registry.gitlab.com/cyb3r-jak3/pypy-flask
# The '|' allows for multi line entries.
tags: |
type=ref,event=pr
type=semver,pattern=
type=semver,pattern=.
type=sha
labels: |
org.label-schema.vcs-url=https://github.com/Cyb3r-Jak3/pypy-flask.git
org.label-schema.schema-version=1.0.0-rc1
Once all the metadata has been generated then the build environment is setup
- name: Set up QEMU
# Using docker action
uses: docker/setup-qemu-action@v1.2.0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1.3.0
# Using the caching action means we can pass items between workflow runs to speed up runs where not much has changed
- name: Cache Docker layers
# Using docker action
uses: actions/cache@v2.1.6
with:
# Path of the files to cache / restore to
path: /tmp/.buildx-cache
# The cache key to save the files as
key: buildx-slim-$
# The cache key to restore with. It will restore from any key that starts with buildx-slim
restore-keys: buildx-slim
Finally the build step
- name: Slim Build and Push
uses: docker/build-push-action@v2.5.0
with:
# Using buildx to build on multiple platforms
platforms: linux/amd64,linux/arm64
# Using the cached layers
cache-from: type=local,src=/tmp/.buildx-cache
# Saving layers to cache
cache-to: type=local,dest=/tmp/.buildx-cache
# Only push to the registries if the git ref is a tag that starts with v
push: ${{ startsWith(github.ref, 'refs/tags/v') }}
# The Dockerfile to use when building
file: Dockerfile
# The tags for the container. Used from the meta step
tags: ${{ steps.meta.outputs.tags }}
# The labels for the container. Used from the meta step
labels: ${{ steps.meta.outputs.labels }}
There is a repeat job that runs for the alpine Dockerfile in the repo only adding the suffix of alpine.
Wrap up
Using custom actions means that it is much simpler to build out workflows. I don’t have to worry about generating all the tags or labels for each image. It also means that it is easier to copy and paste between repos as only a few lines need to be changed. I highly recommend Github actions for people who are just starting with CI/CD solutions
Further Reading
Actions used
In order of use