GITGUD

neiam /gitgud

Git Gud

public · Issues · Labels · Forks · Compare · Actions success · Packages

runner

90f3abb · gmorell · 2026-05-22 03:50

8 files +265 -8

Files changed

added .forgejo/workflows/build.yml
+38 −0
@@ -0,0 +1,38 @@
1 +name: build
2 +on:
3 + push:
4 + branches: [master]
5 + # tags: ['v*']
6 +
7 +jobs:
8 + build:
9 + runs-on: self-hosted
10 + steps:
11 + - uses: actions/checkout@v4
12 +
13 + - uses: docker/setup-buildx-action@v3
14 +
15 + - uses: docker/login-action@v3
16 + with:
17 + registry: ${{ vars.CI_REGISTRY }}
18 + username: ${{ secrets.CI_REGISTRY_USER }}
19 + password: ${{ secrets.CI_REGISTRY_PASSWORD }}
20 +
21 + - uses: docker/metadata-action@v5
22 + id: meta
23 + with:
24 + images: ${{ vars.CI_REGISTRY_IMAGE }}
25 + tags: |
26 + type=sha,format=long
27 + type=ref,event=branch
28 + type=raw,value=latest,enable={{is_default_branch}}
29 +
30 + - uses: docker/build-push-action@v6
31 + with:
32 + context: .
33 + file: ./Dockerfile
34 + push: true
35 + tags: ${{ steps.meta.outputs.tags }}
36 + labels: ${{ steps.meta.outputs.labels }}
37 + cache-from: type=registry,ref=${{ vars.CI_REGISTRY_IMAGE }}:buildcache
38 + cache-to: type=registry,ref=${{ vars.CI_REGISTRY_IMAGE }}:buildcache,mode=max
modified PROGRESS.md
+6 −0
@@ -1125,7 +1125,13 @@ Closed slices 14, 15, 16, 17 in this session. Final task graph:
1125 1125 | 120 | done | Org members management (slice 55) |
1126 1126 | 121 | done | gix NIF: commit + log + tree (slice 56) |
1127 1127 | 122 | done | libcluster + L1/L2 distributed Nebulex (slice 57) |
1128 +| 123 | done | Runner-register UI: generate-config + Quadlet + Kubernetes tabs (slice 58) |
1129 +| 124 | done | Canonical `.forgejo/workflows/build.yml` (ports `.gitlab-ci.yml` to Actions + buildx) |
1130 +| 125 | done | zot deployment manifest `zot.yml` (PVC-backed, ggr.neiam.co, fsGroup init) |
1131 +| 126 | done | `mix gitgud.registry.keygen` task + Packages.Token / ZOT_WEBHOOK_SECRET runtime env wiring |
1128 1132
1129 1133 End-of-session: 501 tests pass on the NIF backend with the new Multilevel cache. Next pickups: in-browser PR conflict resolution, or filling in the remaining gix NIF ops (diff_stats / merge_base / merge_tree / commit_tree / update_ref) once the gix API shakes out.
1130 1134 Embedded SSH daemon verified end-to-end against `git push --set-upstream origin master` from a real client (after fixing two bugs: spawning git with the dashed subcommand name failed; `:stderr_to_stdout` on the Port corrupted the pkt-line stream).
1131 1135
1136 +End-of-session (registry full-cycle slice): runner-register UI now matches Forgejo's binary + docker install docs (`generate-config` step, `--config` flag on daemon) and gained Podman Quadlet + Kubernetes deployment styles. Ported the legacy `.gitlab-ci.yml` to a canonical Actions workflow at `.forgejo/workflows/build.yml` (buildx + metadata-action + registry build cache). Wrote `zot.yml` in the same heavily-commented Traefik/cert-manager style as `app.yml`, backed by a 100Gi `zot-data` PVC on `openebs-zfspv` (S3/Garage driver kept available in a comment). Added `mix gitgud.registry.keygen` — emits PKCS#1 private + SubjectPublicKeyInfo public PEMs (verified against `openssl rsa -pubout`) plus kubectl one-liners for splitting them across the `elixir-args` and `zot` Secrets. `runtime.exs` now reads `GITGUD_REGISTRY_PRIVATE_KEY` → `Packages.Token` and `ZOT_WEBHOOK_SECRET` → `PackagesWebhookController`, and `app.yml` exposes both env vars (both `optional: true` so pods survive before/after key provisioning). End-to-end push → workflow → runner → registry path is now wired; what remains is operator-side: keygen, `kubectl apply -f zot.yml`, DNS for `ggr.neiam.co`, and providing the new secrets to `elixir-args`.
1137 +
modified README.md
+42 −4
@@ -59,6 +59,7 @@ external tools (Forgejo runner, Garage, zot) rather than reinvent them.
59 59 - **CI**
60 60 - Speaks the **Forgejo Actions runner protocol** (Twirp) — point any modern `forgejo-runner` at it; protocol path autodetected (`ping.v1`, `runner.v1`, with/without `_apis/`)
61 61 - Repo-scoped + org-scoped runner registration with one-shot tokens minted from the settings UI
62 + - **Five deployment styles** offered in the registration UI: Native binary, Docker, Podman, Podman Quadlet, Kubernetes (Deployment + dind sidecar) — copy-pasteable `register` + `daemon` snippets per style
62 63 - Re-run / cancel buttons on the workflow show page
63 64 - Per-task HS256 JWT minted on `FetchTask`, gates artifact + cache endpoints
64 65 - **GitHub Actions cache server v1** with hit matching (primary exact → fallback prefix, current ref → default ref) + age/size eviction worker
@@ -70,7 +71,10 @@ external tools (Forgejo runner, Garage, zot) rather than reinvent them.
70 71 - `GitGud.Storage.Garage` — S3-flavored ([Garage](https://garagehq.deuxfleurs.fr/))
71 72 - Add another with one `@behaviour` impl
72 73 - **Packages**
73 − - OCI registry hosted in [zot](https://zotregistry.dev/); local mirror of `(package, version)` rows via the webhook at `/_webhooks/zot`
74 + - OCI registry hosted in [zot](https://zotregistry.dev/) — deployment manifest ships in `zot.yml` (PVC-backed by default, S3/Garage driver kept in a comment block)
75 + - The forge owns `/v2/token`: `docker login` → Basic-auth handshake → RS256-signed JWT scoped to `repository:<owner>/<name>:pull,push` (`Packages.Token`); zot validates with the matching public PEM
76 + - Keypair minted by `mix gitgud.registry.keygen` (PKCS#1 private, SubjectPublicKeyInfo public — verified against `openssl rsa -pubout`); prints kubectl one-liners for splitting it across the `elixir-args` and `zot` Secrets
77 + - Local mirror of `(package, version)` rows via the webhook at `/_webhooks/zot` (Bearer-token guarded by `ZOT_WEBHOOK_SECRET`)
74 78 - Browseable at `/r/<owner>/<name>/packages` with per-package version history
75 79
76 80 ## Architecture
@@ -107,7 +111,9 @@ external tools where they exist:
107 111
108 112 - **forgejo-runner** for CI execution (we just speak its protocol)
109 113 - **Garage** (or any S3) for production object storage
110 −- **zot** for OCI registry if you want one (we don't run a registry)
114 +- **zot** for the OCI registry — deployed alongside via `zot.yml`; the
115 + forge owns `/v2/token` auth and a Postgres mirror of registry
116 + metadata, zot owns the manifests + blobs
111 117 - **gix** (Rust) for git operations via a Rustler NIF, with a `git`
112 118 CLI fallback for anything not yet wired through the NIF
113 119
@@ -147,7 +153,9 @@ These all live in `config/*.exs`. Key ones:
147 153 | `GitGud.Cache` | Nebulex cache sizing | 50k entries / 200MB |
148 154 | `GitGud.Accounts.TwoFactor` `required` / `grace_period_days` | Instance-wide MFA enforcement + grace window | `false` / `14` |
149 155 | `GitGud.Workflows.Workers.CacheEviction` `max_age_days` / `per_repo_max_mb` | `actions/cache` pruning knobs | `90` / `500` / `dry_run: false` |
150 −| `GitGudWeb.PackagesWebhookController` `shared_secret` | Bearer-token gate on the zot webhook | `nil` (open in dev) |
156 +| `GitGudWeb.PackagesWebhookController` `shared_secret` | Bearer-token gate on the zot webhook (env: `ZOT_WEBHOOK_SECRET`) | `nil` (open in dev) |
157 +| `GitGud.Packages.Token` `private_key_pem` | RS256 PEM that signs OCI registry Bearer tokens (env: `GITGUD_REGISTRY_PRIVATE_KEY`) | `nil` (registry auth disabled) |
158 +| `GitGud.Packages.Token` `issuer` / `audience` / `key_id` / `ttl_seconds` | JWT claim defaults zot must agree on | `gitgud-registry` / `container_registry` / `gitgud-registry-1` / `300` |
151 159
152 160 To swap to S3-flavored storage in prod:
153 161
@@ -224,9 +232,39 @@ via `RunnerService/Register`; the runner is then assignable to a repo /
224 232 org pool. On each job poll we mint an HS256 JWT per task that gates the
225 233 artifact + cache endpoints under `/api/actions/_apis/...`.
226 234
235 +The registration UI (`/r/<owner>/<name>/settings/runners` or
236 +`/orgs/<handle>/settings/runners`) emits ready-to-paste snippets for
237 +**Native binary, Docker, Podman, Podman Quadlet, and Kubernetes** —
238 +all built around the official `forgejo-runner generate-config →
239 +register → daemon --config runner-config.yml` flow per the
240 +[Forgejo install docs](https://forgejo.org/docs/latest/admin/actions/installation/).
241 +
227 242 The runner is **external**. A native runner is on the roadmap but the
228 243 external one works today and is well-tested by the Forgejo community.
229 244
245 +### OCI registry
246 +
247 +When you want pushes from your workflows (`docker/build-push-action`,
248 +`kaniko`, etc.) to land in your own registry, deploy zot alongside the
249 +app:
250 +
251 +```sh
252 +mix gitgud.registry.keygen \
253 + --out-priv registry-priv.pem \
254 + --out-pub registry-pub.pem
255 +
256 +# private half → elixir-args.gitgud_registry_private_key
257 +# public half → zot.registry-pub.pem
258 +# (the task prints kubectl one-liners for both)
259 +
260 +kubectl apply -f zot.yml
261 +```
262 +
263 +Add `GITGUD_REGISTRY_PRIVATE_KEY` to the app pod (already wired in
264 +`app.yml`) and a DNS record for your registry hostname pointing at the
265 +ingress. Workflows then push to `<registry-host>/<owner>/<repo>:<tag>`
266 +and the auth handshake flows through the forge.
267 +
230 268 ## What's not done
231 269
232 270 See [ROADMAP.md](./ROADMAP.md). Highlights:
@@ -240,7 +278,7 @@ See [ROADMAP.md](./ROADMAP.md). Highlights:
240 278 - Reputation aging for new federated hosts
241 279
242 280 See [PROGRESS.md](./PROGRESS.md) for the slice-by-slice history of how
243 −the forge was built, and what each shipped — currently through slice 55.
281 +the forge was built, and what each shipped — currently through slice 58.
244 282
245 283 ## Project conventions
246 284
modified config/config.exs
+11 −0
@@ -65,6 +65,17 @@ config :git_gud, GitGud.Federation.Workers.Retention,
65 65 # loopback-only dev where the URL is the only thing keeping it private.
66 66 config :git_gud, GitGudWeb.PackagesWebhookController, shared_secret: nil
67 67
68 +# OCI registry Bearer-token signing key. The private PEM is wired in
69 +# `config/runtime.exs` from GITGUD_REGISTRY_PRIVATE_KEY. zot validates
70 +# signatures against the matching public PEM at /etc/zot/registry-pub.pem;
71 +# `kid` must agree on both sides or zot rejects with `invalid signature`.
72 +config :git_gud, GitGud.Packages.Token,
73 + issuer: "gitgud-registry",
74 + audience: "container_registry",
75 + key_id: "gitgud-registry-1",
76 + ttl_seconds: 300,
77 + private_key_pem: nil
78 +
68 79 config :git_gud, GitGud.Workflows.Workers.CacheEviction,
69 80 # Drop entries with `last_used_at` (or `archived_at` when never read)
70 81 # older than this many days.
modified config/runtime.exs
+18 −0
@@ -125,6 +125,24 @@ if config_env() == :prod do
125 125 config :git_gud, registration_mode: mode
126 126 end
127 127
128 + # OCI registry token signing key. Generated by
129 + # `mix gitgud.registry.keygen`; the matching public PEM lives in the
130 + # `zot` Secret (mounted at /etc/zot/registry-pub.pem inside the zot
131 + # pod). Unset → `Packages.Token.issue/2` raises on first `/v2/token`
132 + # request, but the rest of the app boots fine, so we don't fail
133 + # boot here either.
134 + if pem = System.get_env("GITGUD_REGISTRY_PRIVATE_KEY") do
135 + config :git_gud, GitGud.Packages.Token, private_key_pem: pem
136 + end
137 +
138 + # zot → Packages mirror webhook shared secret. zot sends it as
139 + # `Authorization: Bearer <secret>`; `PackagesWebhookController`
140 + # secure-compares against this value. Unset = no auth (open
141 + # endpoint); fine on a private cluster network, otherwise set it.
142 + if zot_secret = System.get_env("ZOT_WEBHOOK_SECRET") do
143 + config :git_gud, GitGudWeb.PackagesWebhookController, shared_secret: zot_secret
144 + end
145 +
128 146 # The secret key base is used to sign/encrypt cookies and other secrets.
129 147 # A default value is used in config/dev.exs and config/test.exs but you
130 148 # want to use a different value for prod and you most likely don't want
modified lib/git_gud_web/controllers/runner_controller.ex
+19 −4
@@ -332,10 +332,25 @@ defmodule GitGudWeb.RunnerController do
332 332 # ── helpers ──────────────────────────────────────────────────────────
333 333
334 334 defp authenticate(conn) do
335 − with ["Bearer " <> token] <- Plug.Conn.get_req_header(conn, "authorization"),
336 − %RunnerSchema{} = runner <- Runners.find_by_token(token) do
337 − runner
338 − else
335 + # forgejo-runner / act_runner send their post-Register credential
336 + # via the `x-runner-token` header on every Twirp call. Some forks
337 + # (and our own `forgejo-runner` interceptor in newer builds) also
338 + # accept `Authorization: Bearer <token>`. Try both so we work with
339 + # whatever version of runner is registering.
340 + token =
341 + case Plug.Conn.get_req_header(conn, "x-runner-token") do
342 + [t | _] when t != "" ->
343 + t
344 +
345 + _ ->
346 + case Plug.Conn.get_req_header(conn, "authorization") do
347 + ["Bearer " <> t] -> t
348 + _ -> nil
349 + end
350 + end
351 +
352 + case token && Runners.find_by_token(token) do
353 + %RunnerSchema{} = runner -> runner
339 354 _ -> nil
340 355 end
341 356 end
added lib/mix/tasks/gitgud.registry.keygen.ex
+116 −0
@@ -0,0 +1,116 @@
1 +defmodule Mix.Tasks.Gitgud.Registry.Keygen do
2 + @shortdoc "Generate the RS256 keypair the OCI registry auth flow needs."
3 +
4 + @moduledoc """
5 + mix gitgud.registry.keygen [--out-priv PATH] [--out-pub PATH] [--bits N]
6 +
7 + Generates the RSA keypair used by `GitGud.Packages.Token` to sign
8 + bearer tokens for the OCI distribution flow. The private half goes
9 + into `elixir-args.gitgud_registry_private_key` (the Phoenix app
10 + reads it via `GITGUD_REGISTRY_PRIVATE_KEY`); the public half goes
11 + into `zot.registry-pub.pem` (mounted at `/etc/zot/registry-pub.pem`
12 + inside the zot pod — see `zot.yml`).
13 +
14 + By default both PEMs are printed to stdout with delimiters. Pass
15 + `--out-priv` and/or `--out-pub` to write to files instead.
16 +
17 + mix gitgud.registry.keygen
18 + mix gitgud.registry.keygen --out-priv registry-priv.pem --out-pub registry-pub.pem
19 + mix gitgud.registry.keygen --bits 2048 # default is 4096
20 + """
21 +
22 + use Mix.Task
23 +
24 + @switches [out_priv: :string, out_pub: :string, bits: :integer]
25 +
26 + @impl Mix.Task
27 + def run(argv) do
28 + {opts, _, _} = OptionParser.parse(argv, switches: @switches)
29 +
30 + bits = Keyword.get(opts, :bits, 4096)
31 + Application.ensure_all_started(:crypto)
32 + Application.ensure_all_started(:public_key)
33 +
34 + # PKCS#1 "BEGIN RSA PRIVATE KEY" for the private half (what
35 + # `openssl genrsa` produces; Joken/JOSE handle it natively),
36 + # SubjectPublicKeyInfo "BEGIN PUBLIC KEY" for the public half
37 + # (what zot's bearer-auth `cert:` loader parses with
38 + # ParsePKIXPublicKey).
39 + rsa_private = :public_key.generate_key({:rsa, bits, 65_537})
40 + priv_pem = encode_pem(:RSAPrivateKey, rsa_private)
41 +
42 + rsa_public = rsa_public_from_private(rsa_private)
43 + pub_pem = encode_pem(:SubjectPublicKeyInfo, rsa_public)
44 +
45 + emit(opts[:out_priv], "private", priv_pem)
46 + emit(opts[:out_pub], "public", pub_pem)
47 +
48 + Mix.shell().info("""
49 +
50 + ── Next steps ──────────────────────────────────────────────────
51 +
52 + 1) Put the PRIVATE PEM into the `elixir-args` Secret under key
53 + `gitgud_registry_private_key`. With kubectl:
54 +
55 + kubectl -n git-gud create secret generic elixir-args \\
56 + --from-file=gitgud_registry_private_key=registry-priv.pem \\
57 + --dry-run=client -o yaml | kubectl apply -f -
58 +
59 + Then add this env entry to app.yml's container spec (next to
60 + SECRET_KEY_BASE etc.):
61 +
62 + - name: GITGUD_REGISTRY_PRIVATE_KEY
63 + valueFrom:
64 + secretKeyRef:
65 + name: elixir-args
66 + key: gitgud_registry_private_key
67 +
68 + 2) Put the PUBLIC PEM into the `zot` Secret under key
69 + `registry-pub.pem`:
70 +
71 + kubectl -n git-gud create secret generic zot \\
72 + --from-file=registry-pub.pem=registry-pub.pem \\
73 + --dry-run=client -o yaml | kubectl apply -f -
74 +
75 + (or paste it into the `registry-pub.pem` stringData field in
76 + zot.yml if you don't use kubectl-apply for secrets.)
77 +
78 + 3) Restart both Deployments so they pick up the new key material:
79 +
80 + kubectl -n git-gud rollout restart deployment/git-gud
81 + kubectl -n git-gud rollout restart deployment/zot
82 +
83 + Rotate by re-running this task and repeating steps 1–3. The two
84 + halves must always come from the same generation — a `kid`
85 + mismatch causes 401s with `invalid signature`.
86 + """)
87 + end
88 +
89 + defp emit(nil, label, pem) do
90 + Mix.shell().info("""
91 +
92 + ── #{String.upcase(label)} KEY ─────────────────────────────────
93 + #{String.trim_trailing(pem)}
94 + """)
95 + end
96 +
97 + defp emit(path, label, pem) do
98 + File.write!(path, pem)
99 + File.chmod!(path, if(label == "private", do: 0o600, else: 0o644))
100 + Mix.shell().info("Wrote #{label} key to #{path} (mode #{if label == "private", do: "0600", else: "0644"})")
101 + end
102 +
103 + defp encode_pem(asn1_type, entity) do
104 + entry = :public_key.pem_entry_encode(asn1_type, entity)
105 + :public_key.pem_encode([entry])
106 + end
107 +
108 + # `:public_key.generate_key({:rsa, ...})` returns the full
109 + # `RSAPrivateKey` record; positions 3 and 4 are modulus + public
110 + # exponent, which is all SubjectPublicKeyInfo needs.
111 + defp rsa_public_from_private(rsa_private) do
112 + modulus = elem(rsa_private, 2)
113 + public_exp = elem(rsa_private, 3)
114 + {:RSAPublicKey, modulus, public_exp}
115 + end
116 +end
modified test/git_gud_web/controllers/runner_controller_test.exs
+15 −0
@@ -116,6 +116,21 @@ defmodule GitGudWeb.RunnerControllerTest do
116 116 assert decoded.task == nil
117 117 end
118 118
119 + test "FetchTask accepts x-runner-token header (real forgejo-runner shape)", %{
120 + conn: conn,
121 + runner_token: token
122 + } do
123 + body = FetchTaskRequest.encode(%FetchTaskRequest{tasks_version: 0}) |> IO.iodata_to_binary()
124 +
125 + resp =
126 + conn
127 + |> put_req_header("content-type", "application/protobuf")
128 + |> put_req_header("x-runner-token", token)
129 + |> post("/api/actions/_apis/runner.v1.RunnerService/FetchTask", body)
130 +
131 + assert resp.status == 200
132 + end
133 +
119 134 test "FetchTask without token → 401", %{conn: conn} do
120 135 body = FetchTaskRequest.encode(%FetchTaskRequest{}) |> IO.iodata_to_binary()
121 136

Parents: 60c9058