Github Actions Docker Pipeline
Something that I have been using more and more is Github Actions. It is an easy to use tool to run tests and build in CI. One of the nice things about it is that is is more module than GitLab Pipeline. There is also a marketplace of company and user created actions so there are many options to choose when you are looking to build something out.
I have used them to build docker pipelines for container building and deployment. I found that actions workflow were great to keep a simple and clean workflow for my pypy-flask docker image. The full workflow file is available in the repo but I will be breaking it down below.
Full file:
1name: Docker
2
3on:
4 push:
5
6
7jobs:
8 Slim:
9 runs-on: ubuntu-latest
10 steps:
11
12 - name: Checkout
13 uses: actions/checkout@v2
14
15 - name: Login to Docker
16 uses: docker/login-action@v1
17 if: ${{ startsWith(github.ref, 'refs/tags/v') }}
18 with:
19 username: ${{ secrets.DOCKER_USERNAME }}
20 password: ${{ secrets.DOCKER_PASSWORD }}
21
22 - name: Login To GitHub
23 uses: docker/login-action@v1
24 if: ${{ startsWith(github.ref, 'refs/tags/v') }}
25 with:
26 registry: ghcr.io
27 username: ${{ github.repository_owner }}
28 password: ${{ secrets.CR_PAT }}
29
30 - name: Login To GitLab
31 uses: docker/login-action@v1
32 if: ${{ startsWith(github.ref, 'refs/tags/v') }}
33 with:
34 registry: registry.gitlab.com
35 username: ${{ secrets.GITLAB_USER }}
36 password: ${{ secrets.GITLAB_TOKEN }}
37
38 - name: Docker Meta
39 id: meta
40 uses: docker/metadata-action@v3
41 with:
42 images: cyb3rjak3/pypy-flask,ghcr.io/cyb3r-jak3/pypy-flask,registry.gitlab.com/cyb3r-jak3/pypy-flask
43 tags: |
44 type=ref,event=pr
45 type=semver,pattern={{version}}
46 type=semver,pattern={{major}}.{{minor}}
47 type=sha
48 labels: |
49 org.label-schema.vcs-url=https://github.com/Cyb3r-Jak3/pypy-flask.git
50 org.label-schema.schema-version=1.0.0-rc1
51
52 - name: Set up QEMU
53 uses: docker/[email protected]
54
55 - name: Set up Docker Buildx
56 uses: docker/[email protected]
57
58 - name: Cache Docker layers
59 uses: actions/[email protected]
60 with:
61 path: /tmp/.buildx-cache
62 key: buildx-slim-${{ github.sha }}
63 restore-keys: buildx-slim
64
65 - name: Slim Build and Push
66 uses: docker/[email protected]
67 with:
68 platforms: linux/amd64,linux/arm64
69 cache-from: type=local,src=https://blog.cyberjake.xyz/tmp/.buildx-cache
70 cache-to: type=local,dest=/tmp/.buildx-cache
71 push: ${{ startsWith(github.ref, 'refs/tags/v') }}
72 file: Dockerfile
73 tags: ${{ steps.meta.outputs.tags }}
74 labels: ${{ steps.meta.outputs.labels }}
75
76 Alpine:
77 runs-on: ubuntu-latest
78 steps:
79
80 - name: Checkout
81 uses: actions/checkout@v2
82
83 - name: Login to Docker
84 uses: docker/login-action@v1
85 if: ${{ startsWith(github.ref, 'refs/tags/v') }}
86 with:
87 username: ${{ secrets.DOCKER_USERNAME }}
88 password: ${{ secrets.DOCKER_PASSWORD }}
89
90 - name: Login To GitHub
91 uses: docker/login-action@v1
92 if: ${{ startsWith(github.ref, 'refs/tags/v') }}
93 with:
94 registry: ghcr.io
95 username: ${{ github.repository_owner }}
96 password: ${{ secrets.CR_PAT }}
97
98 - name: Login To GitLab
99 uses: docker/login-action@v1
100 if: ${{ startsWith(github.ref, 'refs/tags/v') }}
101 with:
102 registry: registry.gitlab.com
103 username: ${{ secrets.GITLAB_USER }}
104 password: ${{ secrets.GITLAB_TOKEN }}
105
106 - name: Docker Meta
107 id: meta
108 uses: docker/metadata-action@v3
109 with:
110 images: cyb3rjak3/pypy-flask,ghcr.io/cyb3r-jak3/pypy-flask,registry.gitlab.com/cyb3r-jak3/pypy-flask
111 flavor: |
112 suffix=-alpine
113 tags: |
114 type=ref,event=pr
115 type=semver,pattern={{version}}
116 type=semver,pattern={{major}}.{{minor}}
117 type=sha
118 labels: |
119 org.label-schema.vcs-url=https://github.com/Cyb3r-Jak3/pypy-flask.git
120 org.label-schema.schema-version=1.0.0-rc1
121
122 - name: Set up QEMU
123 uses: docker/[email protected]
124
125 - name: Set up Docker Buildx
126 uses: docker/[email protected]
127
128 - name: Cache Docker layers
129 uses: actions/[email protected]
130 with:
131 path: /tmp/.buildx-cache
132 key: buildx-alpine-${{ github.sha }}
133 restore-keys: buildx-alpine
134
135 - name: Alpine Build and Push
136 uses: docker/[email protected]
137 with:
138 platforms: linux/amd64,linux/arm64
139 cache-from: type=local,src=https://blog.cyberjake.xyz/tmp/.buildx-cache
140 cache-to: type=local,dest=/tmp/.buildx-cache
141 push: ${{ startsWith(github.ref, 'refs/tags/v') }}
142 file: alpine.Dockerfile
143 tags: ${{ steps.meta.outputs.tags }}
144 labels: ${{ steps.meta.outputs.labels }}
It looks like a lot but the same job is close to being repeated twice to have better caching and means they run in parallel.
Breaking it down
Main Step
1
2# Name of the Workflow
3name: Docker
4
5# When to run the workflow
6on:
7 # Run on all push
8 push:
9
10# List of the jobs
11jobs:
12 # Name of the job
13 Slim:
14 # Operating system to run. Can be ubuntu, windows or mac
15 runs-on: ubuntu-latest
16 # List of steps for the job
17 steps:
After setting up the workflow there are some starter steps for checking out the repo and logging in.
Where there is uses in a step it means that the step is using a custom workflow.
1 # Checkout the repo
2 - name: Checkout
3 uses: actions/checkout@v2
4
5 # Log into DockerHub
6 - name: Login to Docker
7 uses: docker/login-action@v1
8 # The IF statement means that only tags that start with v will run this step
9 if: ${{ startsWith(github.ref, 'refs/tags/v') }}
10 # With passes action specific input. Used here with the login creds for DockerHub
11 with:
12 username: ${{ secrets.DOCKER_USERNAME }}
13 password: ${{ secrets.DOCKER_PASSWORD }}
14
15 # Login into GitHub's container registry
16 - name: Login To GitHub
17 uses: docker/login-action@v1
18 # The IF statement means that only tags that start with v will run this step
19 if: ${{ startsWith(github.ref, 'refs/tags/v') }}
20 # With passes action specific input. Used here with the login creds for GitHub
21 with:
22 registry: ghcr.io
23 username: ${{ github.repository_owner }}
24 password: ${{ secrets.CR_PAT }}
25
26 # Login in GitLab's Container registry
27 - name: Login To GitLab
28 uses: docker/login-action@v1
29 # The IF statement means that only tags that start with v will run this step
30 if: ${{ startsWith(github.ref, 'refs/tags/v') }}
31 # With passes action specific input. Used here with the login creds for GitLab
32 with:
33 registry: registry.gitlab.com
34 username: ${{ secrets.GITLAB_USER }}
35 password: ${{ secrets.GITLAB_TOKEN }}
Next the metadata is setup for building the container
1
2 - name: Docker Meta
3 # ID allows us to use the output of the step in later steps
4 id: meta
5 uses: docker/metadata-action@v3
6 # With passes action specific input. Used here to pass the baseline images, tags to generate and labels to add
7 with:
8 images: cyb3rjak3/pypy-flask,ghcr.io/cyb3r-jak3/pypy-flask,registry.gitlab.com/cyb3r-jak3/pypy-flask
9 # The '|' allows for multi line entries.
10 tags: |
11 type=ref,event=pr
12 type=semver,pattern={{version}}
13 type=semver,pattern={{major}}.{{minor}}
14 type=sha
15 labels: |
16 org.label-schema.vcs-url=https://github.com/Cyb3r-Jak3/pypy-flask.git
17 org.label-schema.schema-version=1.0.0-rc1
Once all the metadata has been generated then the build environment is setup
1
2 - name: Set up QEMU
3 # Using docker action
4 uses: docker/[email protected]
5
6 - name: Set up Docker Buildx
7 uses: docker/[email protected]
8
9 # Using the caching action means we can pass items between workflow runs to speed up runs where not much has changed
10 - name: Cache Docker layers
11 # Using docker action
12 uses: actions/[email protected]
13 with:
14 # Path of the files to cache / restore to
15 path: /tmp/.buildx-cache
16 # The cache key to save the files as
17 key: buildx-slim-${{ github.sha }}
18 # The cache key to restore with. It will restore from any key that starts with buildx-slim
19 restore-keys: buildx-slim
Finally the build step
1 - name: Slim Build and Push
2 uses: docker/[email protected]
3 with:
4 # Using buildx to build on multiple platforms
5 platforms: linux/amd64,linux/arm64
6 # Using the cached layers
7 cache-from: type=local,src=https://blog.cyberjake.xyz/tmp/.buildx-cache
8 # Saving layers to cache
9 cache-to: type=local,dest=/tmp/.buildx-cache
10 # Only push to the registries if the git ref is a tag that starts with v
11 push: ${{ startsWith(github.ref, 'refs/tags/v') }}
12 # The Dockerfile to use when building
13 file: Dockerfile
14 # The tags for the container. Used from the meta step
15 tags: ${{ steps.meta.outputs.tags }}
16 # The labels for the container. Used from the meta step
17 labels: ${{ steps.meta.outputs.labels }}
There is a repeat job that runs for the alpine Dockerfile in the repo only adding the suffix of alpine.
Wrap up
Using custom actions means that it is much simpler to build out workflows. I don't have to worry about generating all the tags or labels for each image. It also means that it is easier to copy and paste between repos as only a few lines need to be changed. I highly recommend Github actions for people who are just starting with CI/CD solutions
Further Reading
Actions used
In order of use