neiam
/gitgud
Git Gud
public · Issues · Pulls · Labels · Forks · Compare · Actions success · Badges · Packages
⭐
1
Log in to mark this repository.
Secrets: prod no longer runs on the public master key and job-token secret
9f1041f · Gabriel Morell · 2026-10-20 18:22
Message
{commit_body(@commit)}
Files changed
modified
README.md
+41
−2
@@ -261,7 +261,10 @@ These all live in `config/*.exs`. Key ones:
| 261 | 261 | | `GitGud.Packages.Token` `issuer` / `audience` / `key_id` / `ttl_seconds` | JWT claim defaults zot must agree on | `gitgud-registry` / `container_registry` / `gitgud-registry-1` / `300` | |
| 262 | 262 | | `:package_formats` | Which registry formats this instance serves. A format left out 404s | all of `pypi` / `hex` / `cargo` / `deb` | |
| 263 | 263 | | `:package_registry_url` | Absolute base URL the registries put in index documents (behind a proxy that rewrites the host) | endpoint URL | |
| 264 | −| `GitGud.Secrets` `master_key` | Also wraps package signing keys' private halves | dev placeholder | | |
| 264 | +| `GITGUD_SECRETS_MASTER_KEY` (env) → `GitGud.Secrets` `master_key` | AES-256-GCM key for everything encrypted at rest: CI secrets, TOTP secrets, import tokens, package signing keys' private halves. Base64 of exactly 32 random bytes (`openssl rand -base64 32`). **Required in prod**: boot refuses a missing, malformed or all-zero key | dev/test: the all-zero key (public — never use it) | | |
| 265 | +| `GITGUD_SECRETS_KEY_ID` (env) → `GitGud.Secrets` `primary_key_id` | Id stored with every row the key above encrypts. Anything but `default`, which names the old all-zero key; change it whenever the key changes | `k1` | | |
| 266 | +| `GITGUD_SECRETS_OLD_KEYS` (env) → `GitGud.Secrets` `master_keys` | Earlier keys kept readable after a rotation, `id:base64,id:base64`. Rows under them are re-encrypted with the primary key by `bin/migrate`. The all-zero key is always registered as `default` (decrypt only) so rows written before keys came from the environment keep working | unset | | |
| 267 | +| `GITGUD_RUNNER_JWT_SECRET` (env) → `GitGud.Workflows.RunnerJwt` `secret` | HS256 secret that signs CI job tokens (`${{ github.token }}`). At least 32 bytes (`openssl rand -base64 48`). **Required in prod**: boot refuses a missing, short or dev-default value. Changing it invalidates the tokens of jobs running at the time | dev/test: a fixed string in `config/config.exs` (public — never use it) | | |
| 265 | 268 | |
| 266 | 269 | To swap to S3-flavored storage in prod: |
| 267 | 270 |
@@ -325,6 +328,41 @@ Behaviour that changed under an existing install, as of the October
| 325 | 328 | - **Merging a PR takes write access**, labels are an admin's (or the |
| 326 | 329 | author's on their own issue or PR), and closing an issue is its |
| 327 | 330 | author's or a triager's — anyone with read access used to be able to. |
| 331 | +- **Production no longer boots on the built-in keys.** Up to now | |
| 332 | + `config/config.exs` shipped the secrets master key (all zeros) and the | |
| 333 | + job-token signing secret, and nothing in prod replaced them: anyone | |
| 334 | + who read the source could forge a job token with any permissions for | |
| 335 | + any running job, and decrypt every stored CI secret, TOTP secret, | |
| 336 | + import token and package signing key with a database dump. Prod now | |
| 337 | + requires `GITGUD_RUNNER_JWT_SECRET`, `GITGUD_SECRETS_MASTER_KEY` and | |
| 338 | + (optionally, default `k1`) `GITGUD_SECRETS_KEY_ID` — see | |
| 339 | + [Configuration knobs](#configuration-knobs). Order to deploy it in: | |
| 340 | + 1. Generate both values (`openssl rand -base64 48` for the JWT secret, | |
| 341 | + `openssl rand -base64 32` for the master key) and store them where | |
| 342 | + the deployment reads secrets (a k8s Secret; `/etc/gitgud/gitgud.env` | |
| 343 | + for the .deb — a conffile, so an upgraded install keeps its old file | |
| 344 | + and won't boot until they're added). | |
| 345 | + 2. Give them to **both** the migrate step and the app, *before* the | |
| 346 | + new release rolls out — the previous release ignores them, so this | |
| 347 | + is safe to do first; the new one won't boot without them. | |
| 348 | + `bin/migrate` now re-encrypts every row still under the old key | |
| 349 | + with the new one after running migrations (`GitGud.Secrets.rewrap_all/0`, also | |
| 350 | + `bin/git_gud eval 'GitGud.Release.rewrap_secrets()'`); it prints | |
| 351 | + counts only, and is a no-op once everything is moved. | |
| 352 | + 3. Deploy: migrate first, then the app. The old all-zero key stays | |
| 353 | + registered as `default` for decrypting only, so nothing is lost if | |
| 354 | + a row is written under it during the rollout — the next migrate | |
| 355 | + moves it. Pods still running the previous release can't read rows | |
| 356 | + already moved, so CI secrets and TOTP may fail on them until the | |
| 357 | + rollout finishes. | |
| 358 | + 4. Jobs running during the deploy lose their job token (it was signed | |
| 359 | + with the old secret) and need a re-run. | |
| 360 | + | |
| 361 | + Anything encrypted under the all-zero key should be treated as | |
| 362 | + disclosed: rotate the CI secrets themselves (API tokens, deploy keys) | |
| 363 | + if the database or a backup of it may have left your hands. To | |
| 364 | + rotate the master key later, set a new key and id and move the old | |
| 365 | + pair into `GITGUD_SECRETS_OLD_KEYS`; the next migrate re-encrypts. | |
| 328 | 366 | |
| 329 | 367 | ## Storage backends |
| 330 | 368 |
@@ -618,7 +656,8 @@ echo "deb [signed-by=/etc/apt/keyrings/<owner>.asc] https://<host>/pkg/deb/<owne
| 618 | 656 | | sudo tee /etc/apt/sources.list.d/gitgud.list |
| 619 | 657 | sudo apt update && sudo apt install gitgud |
| 620 | 658 | |
| 621 | −sudoedit /etc/gitgud/gitgud.env # DATABASE_URL, SECRET_KEY_BASE, PHX_HOST | |
| 659 | +sudoedit /etc/gitgud/gitgud.env # DATABASE_URL, SECRET_KEY_BASE, PHX_HOST, | |
| 660 | + # GITGUD_RUNNER_JWT_SECRET, GITGUD_SECRETS_MASTER_KEY | |
| 622 | 661 | sudo /opt/gitgud/bin/migrate |
| 623 | 662 | sudo systemctl start gitgud |
| 624 | 663 | ``` |
modified
config/config.exs
+6
−4
@@ -135,15 +135,17 @@ config :mime, :types, %{
| 135 | 135 | "application/protobuf" => ["protobuf"] |
| 136 | 136 | } |
| 137 | 137 | |
| 138 | −# Secrets encryption. The default key is for dev/test; production | |
| 139 | −# MUST override in runtime.exs from an env var or KMS-fetched value. | |
| 138 | +# Secrets encryption. The all-zero key is for dev/test only and is | |
| 139 | +# public; production reads GITGUD_SECRETS_MASTER_KEY / _KEY_ID / | |
| 140 | +# _OLD_KEYS in runtime.exs and refuses to boot without them. | |
| 140 | 141 | # 32 raw bytes, base64-encoded. |
| 141 | 142 | config :git_gud, GitGud.Secrets, |
| 142 | 143 | primary_key_id: "default", |
| 143 | 144 | master_key: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=" |
| 144 | 145 | |
| 145 | −# Per-task JWT signing secret used by the runner pipeline. Override in | |
| 146 | −# runtime.exs in production. | |
| 146 | +# Per-task JWT signing secret used by the runner pipeline. Dev/test | |
| 147 | +# only and public; production reads GITGUD_RUNNER_JWT_SECRET in | |
| 148 | +# runtime.exs and refuses to boot without it. | |
| 147 | 149 | config :git_gud, GitGud.Workflows.RunnerJwt, |
| 148 | 150 | secret: "dev-runner-jwt-secret-do-not-use-in-prod", |
| 149 | 151 | ttl_seconds: 86_400 |
modified
config/runtime.exs
+88
−0
@@ -190,6 +190,94 @@ if config_env() == :prod do
| 190 | 190 | You can generate one by calling: mix phx.gen.secret |
| 191 | 191 | """ |
| 192 | 192 | |
| 193 | + # Job-token signing secret (`GitGud.Workflows.RunnerJwt`, HS256). The | |
| 194 | + # default in config.exs is in the source, so anyone could forge a job | |
| 195 | + # token with it: prod refuses to boot without a real one. Changing it | |
| 196 | + # invalidates the tokens of jobs running at the time (they need a | |
| 197 | + # re-run). Generate with: openssl rand -base64 48 | |
| 198 | + runner_jwt_secret = | |
| 199 | + System.get_env("GITGUD_RUNNER_JWT_SECRET") || | |
| 200 | + raise """ | |
| 201 | + environment variable GITGUD_RUNNER_JWT_SECRET is missing. | |
| 202 | + It signs CI job tokens. Generate one with: openssl rand -base64 48 | |
| 203 | + """ | |
| 204 | + | |
| 205 | + cond do | |
| 206 | + runner_jwt_secret == "dev-runner-jwt-secret-do-not-use-in-prod" -> | |
| 207 | + raise "GITGUD_RUNNER_JWT_SECRET is the public dev default; generate a real one" | |
| 208 | + | |
| 209 | + byte_size(runner_jwt_secret) < 32 -> | |
| 210 | + raise "GITGUD_RUNNER_JWT_SECRET must be at least 32 bytes" | |
| 211 | + | |
| 212 | + true -> | |
| 213 | + :ok | |
| 214 | + end | |
| 215 | + | |
| 216 | + config :git_gud, GitGud.Workflows.RunnerJwt, secret: runner_jwt_secret | |
| 217 | + | |
| 218 | + # Master key for secrets at rest (`GitGud.Secrets`: CI secrets, TOTP | |
| 219 | + # secrets, import tokens, package signing keys). Base64 of 32 random | |
| 220 | + # bytes (openssl rand -base64 32), tagged with GITGUD_SECRETS_KEY_ID. | |
| 221 | + # The all-zero key in config.exs is public, so prod refuses it. | |
| 222 | + legacy_zero_key = Base.encode64(<<0::256>>) | |
| 223 | + | |
| 224 | + decode_master_key = fn var, b64 -> | |
| 225 | + case Base.decode64(String.trim(b64)) do | |
| 226 | + {:ok, <<0::256>>} -> | |
| 227 | + raise "#{var} is the all-zero dev key; generate one with: openssl rand -base64 32" | |
| 228 | + | |
| 229 | + {:ok, <<_::binary-size(32)>>} -> | |
| 230 | + String.trim(b64) | |
| 231 | + | |
| 232 | + _ -> | |
| 233 | + raise "#{var} must be base64 of exactly 32 bytes (openssl rand -base64 32)" | |
| 234 | + end | |
| 235 | + end | |
| 236 | + | |
| 237 | + secrets_master_key = | |
| 238 | + decode_master_key.( | |
| 239 | + "GITGUD_SECRETS_MASTER_KEY", | |
| 240 | + System.get_env("GITGUD_SECRETS_MASTER_KEY") || | |
| 241 | + raise(""" | |
| 242 | + environment variable GITGUD_SECRETS_MASTER_KEY is missing. | |
| 243 | + It encrypts secrets at rest. Generate one with: openssl rand -base64 32 | |
| 244 | + """) | |
| 245 | + ) | |
| 246 | + | |
| 247 | + secrets_key_id = System.get_env("GITGUD_SECRETS_KEY_ID", "k1") | |
| 248 | + | |
| 249 | + if secrets_key_id in ["", "default"] do | |
| 250 | + raise "GITGUD_SECRETS_KEY_ID can't be #{inspect(secrets_key_id)}: rows under the old " <> | |
| 251 | + "all-zero key carry key id \"default\"" | |
| 252 | + end | |
| 253 | + | |
| 254 | + # Earlier keys stay readable so rows written under them decrypt until | |
| 255 | + # `GitGud.Release.migrate/0` (`GitGud.Secrets.rewrap_all/0`) moves | |
| 256 | + # them to the primary key. GITGUD_SECRETS_OLD_KEYS is | |
| 257 | + # "id:base64,id:base64" — add the outgoing key here when rotating. | |
| 258 | + secrets_old_keys = | |
| 259 | + System.get_env("GITGUD_SECRETS_OLD_KEYS", "") | |
| 260 | + |> String.split(",", trim: true) | |
| 261 | + |> Map.new(fn entry -> | |
| 262 | + case String.split(String.trim(entry), ":", parts: 2) do | |
| 263 | + [id, b64] when id not in ["", "default"] and id != secrets_key_id -> | |
| 264 | + {id, decode_master_key.("GITGUD_SECRETS_OLD_KEYS (#{id})", b64)} | |
| 265 | + | |
| 266 | + _ -> | |
| 267 | + raise "GITGUD_SECRETS_OLD_KEYS entries must be id:base64, with an id other than " <> | |
| 268 | + "\"default\" and GITGUD_SECRETS_KEY_ID" | |
| 269 | + end | |
| 270 | + end) | |
| 271 | + | |
| 272 | + config :git_gud, GitGud.Secrets, | |
| 273 | + primary_key_id: secrets_key_id, | |
| 274 | + master_key: secrets_master_key, | |
| 275 | + # Every row written before keys came from the environment carries | |
| 276 | + # key id "default" and is encrypted under the all-zero key from | |
| 277 | + # config.exs. Keep that key readable — for decrypting only, never | |
| 278 | + # for new writes — so those rows survive until re-wrapped. | |
| 279 | + master_keys: Map.put(secrets_old_keys, "default", legacy_zero_key) | |
| 280 | + | |
| 193 | 281 | host = System.get_env("PHX_HOST") || "example.com" |
| 194 | 282 | |
| 195 | 283 | config :git_gud, :dns_cluster_query, System.get_env("DNS_CLUSTER_QUERY") |
modified
lib/git_gud/release.ex
+24
−0
@@ -11,6 +11,30 @@ defmodule GitGud.Release do
| 11 | 11 | for repo <- repos() do |
| 12 | 12 | {:ok, _, _} = Ecto.Migrator.with_repo(repo, &Ecto.Migrator.run(&1, :up, all: true)) |
| 13 | 13 | end |
| 14 | + | |
| 15 | + rewrap_secrets() | |
| 16 | + end | |
| 17 | + | |
| 18 | + @doc """ | |
| 19 | + Re-encrypt every stored secret still under an old master key with the | |
| 20 | + primary one (`GitGud.Secrets.rewrap_all/0`). Part of `migrate/0`, so | |
| 21 | + every deploy runs it; a no-op once everything is under the primary | |
| 22 | + key. Prints counts only, never values. | |
| 23 | + | |
| 24 | + kubectl -n git-gud exec deployment/git-gud -- \\ | |
| 25 | + bin/git_gud eval 'GitGud.Release.rewrap_secrets()' | |
| 26 | + """ | |
| 27 | + def rewrap_secrets do | |
| 28 | + load_app() | |
| 29 | + | |
| 30 | + {:ok, counts, _} = | |
| 31 | + Ecto.Migrator.with_repo(GitGud.Repo, fn _repo -> GitGud.Secrets.rewrap_all() end) | |
| 32 | + | |
| 33 | + for {kind, %{rewrapped: r, failed: f}} <- Enum.sort(counts) do | |
| 34 | + IO.puts("secrets rewrap: #{kind}: #{r} re-encrypted, #{f} failed") | |
| 35 | + end | |
| 36 | + | |
| 37 | + counts | |
| 14 | 38 | end |
| 15 | 39 | |
| 16 | 40 | def rollback(repo, version) do |
modified
lib/git_gud/secrets.ex
+140
−4
@@ -5,9 +5,16 @@ defmodule GitGud.Secrets do
| 5 | 5 | |
| 6 | 6 | Encryption: AES-256-GCM. The master key is base64-encoded in |
| 7 | 7 | `config :git_gud, GitGud.Secrets, master_key: "..."` (32 bytes |
| 8 | − decoded). On rotation, set a new `master_key` and keep the previous | |
| 9 | − one available under `master_keys: %{"old-id" => "<b64>"}` — rows | |
| 10 | − store `key_id` so a re-wrap job can re-encrypt them lazily. | |
| 8 | + decoded), tagged `primary_key_id`. Every row stores the `key_id` it | |
| 9 | + was written under; `decrypt/1` looks that id up — the primary first, | |
| 10 | + then `master_keys: %{"old-id" => "<b64>"}`. | |
| 11 | + | |
| 12 | + In production all three come from the environment (see | |
| 13 | + `config/runtime.exs`): `GITGUD_SECRETS_MASTER_KEY`, | |
| 14 | + `GITGUD_SECRETS_KEY_ID` and `GITGUD_SECRETS_OLD_KEYS`. On rotation, | |
| 15 | + set a new key and id and move the previous pair into the old keys; | |
| 16 | + `rewrap_all/0` — run by `GitGud.Release.migrate/0` on every deploy — | |
| 17 | + re-encrypts every row still under an old key. | |
| 11 | 18 | |
| 12 | 19 | Resolution for a job: organization secrets first, then repository |
| 13 | 20 | secrets overlay them. Variables resolve the same way. Repo always |
@@ -16,7 +23,12 @@ defmodule GitGud.Secrets do
| 16 | 23 | |
| 17 | 24 | import Ecto.Query, warn: false |
| 18 | 25 | |
| 26 | + require Logger | |
| 27 | + | |
| 28 | + alias GitGud.Accounts.User | |
| 29 | + alias GitGud.Imports.Import | |
| 19 | 30 | alias GitGud.Organizations.Organization |
| 31 | + alias GitGud.Packages.SigningKey | |
| 20 | 32 | alias GitGud.Repo |
| 21 | 33 | alias GitGud.Repositories.Repository |
| 22 | 34 | alias GitGud.Secrets.OrganizationSecret |
@@ -28,6 +40,12 @@ defmodule GitGud.Secrets do
| 28 | 40 | |
| 29 | 41 | @aad "gitgud-secret-v1" |
| 30 | 42 | |
| 43 | + # The all-zero dev/test key from config/config.exs. Public, so a | |
| 44 | + # production build refuses to encrypt with it (runtime.exs refuses to | |
| 45 | + # boot with it first; this is the second check). | |
| 46 | + @dev_master_key Base.encode64(<<0::256>>) | |
| 47 | + @refuse_dev_key Mix.env() == :prod | |
| 48 | + | |
| 31 | 49 | @doc """ |
| 32 | 50 | Encrypt `plaintext` with the current master key. Returns |
| 33 | 51 | `{:ok, %{ciphertext, iv, tag, key_id}}`. |
@@ -61,7 +79,18 @@ defmodule GitGud.Secrets do
| 61 | 79 | defp current_master_key do |
| 62 | 80 | cfg = Application.get_env(:git_gud, __MODULE__, []) |
| 63 | 81 | primary_id = Keyword.get(cfg, :primary_key_id, "default") |
| 64 | − {primary_id, decode_key!(Keyword.fetch!(cfg, :master_key))} | |
| 82 | + b64 = Keyword.fetch!(cfg, :master_key) | |
| 83 | + | |
| 84 | + if @refuse_dev_key and b64 == @dev_master_key do | |
| 85 | + raise "refusing to encrypt with the public dev master key; set GITGUD_SECRETS_MASTER_KEY" | |
| 86 | + end | |
| 87 | + | |
| 88 | + {primary_id, decode_key!(b64)} | |
| 89 | + end | |
| 90 | + | |
| 91 | + @doc "The key id new ciphertext is written under." | |
| 92 | + def primary_key_id do | |
| 93 | + :git_gud |> Application.get_env(__MODULE__, []) |> Keyword.get(:primary_key_id, "default") | |
| 65 | 94 | end |
| 66 | 95 | |
| 67 | 96 | defp lookup_master_key(kid) do |
@@ -88,6 +117,113 @@ defmodule GitGud.Secrets do
| 88 | 117 | if byte_size(key) != 32, do: raise("master_key must decode to 32 bytes"), else: key |
| 89 | 118 | end |
| 90 | 119 | |
| 120 | + # ── re-wrap ────────────────────────────────────────────────────────── | |
| 121 | + | |
| 122 | + # Every column set that holds `encrypt/1` output: schema, and the | |
| 123 | + # fields its ciphertext / iv / tag / key_id live in. | |
| 124 | + @wrapped [ | |
| 125 | + {:repository_secrets, RepositorySecret, | |
| 126 | + %{ciphertext: :ciphertext, iv: :iv, tag: :tag, key_id: :key_id}}, | |
| 127 | + {:organization_secrets, OrganizationSecret, | |
| 128 | + %{ciphertext: :ciphertext, iv: :iv, tag: :tag, key_id: :key_id}}, | |
| 129 | + {:totp_secrets, User, | |
| 130 | + %{ciphertext: :totp_ciphertext, iv: :totp_iv, tag: :totp_tag, key_id: :totp_key_id}}, | |
| 131 | + {:import_tokens, Import, | |
| 132 | + %{ciphertext: :token_ciphertext, iv: :token_iv, tag: :token_tag, key_id: :token_key_id}}, | |
| 133 | + {:package_signing_keys, SigningKey, | |
| 134 | + %{ | |
| 135 | + ciphertext: :private_key_ciphertext, | |
| 136 | + iv: :private_key_iv, | |
| 137 | + tag: :private_key_tag, | |
| 138 | + key_id: :key_id | |
| 139 | + }} | |
| 140 | + ] | |
| 141 | + | |
| 142 | + @doc """ | |
| 143 | + Re-encrypt every stored ciphertext whose `key_id` isn't the primary | |
| 144 | + key's under the primary key: repository and organization secrets, | |
| 145 | + TOTP secrets, import tokens and package signing keys. | |
| 146 | + | |
| 147 | + Each row is re-read under a row lock and rewritten in its own | |
| 148 | + transaction, so a concurrent write of the same row isn't clobbered | |
| 149 | + and a failure leaves the others done. Idempotent: rows already under | |
| 150 | + the primary key aren't touched, so a second run is a no-op. | |
| 151 | + | |
| 152 | + A row whose key is missing or whose ciphertext doesn't authenticate | |
| 153 | + is left as it is and counted as `failed` (logged by id and key id, | |
| 154 | + never by value). | |
| 155 | + | |
| 156 | + Returns `%{kind => %{rewrapped: n, failed: n}}`. | |
| 157 | + """ | |
| 158 | + def rewrap_all do | |
| 159 | + primary = primary_key_id() | |
| 160 | + | |
| 161 | + Map.new(@wrapped, fn {kind, schema, fields} -> | |
| 162 | + {kind, rewrap_table(kind, schema, fields, primary)} | |
| 163 | + end) | |
| 164 | + end | |
| 165 | + | |
| 166 | + defp rewrap_table(kind, schema, %{key_id: kid_field, ciphertext: ct_field} = fields, primary) do | |
| 167 | + from(r in schema, | |
| 168 | + where: | |
| 169 | + not is_nil(field(r, ^ct_field)) and not is_nil(field(r, ^kid_field)) and | |
| 170 | + field(r, ^kid_field) != ^primary, | |
| 171 | + select: r.id | |
| 172 | + ) | |
| 173 | + |> Repo.all() | |
| 174 | + |> Enum.reduce(%{rewrapped: 0, failed: 0}, fn id, acc -> | |
| 175 | + case rewrap_row(schema, id, fields, primary) do | |
| 176 | + :ok -> | |
| 177 | + Map.update!(acc, :rewrapped, &(&1 + 1)) | |
| 178 | + | |
| 179 | + :skip -> | |
| 180 | + acc | |
| 181 | + | |
| 182 | + {:error, reason} -> | |
| 183 | + Logger.warning("secrets rewrap: #{kind} id=#{id} not re-encrypted: #{inspect(reason)}") | |
| 184 | + Map.update!(acc, :failed, &(&1 + 1)) | |
| 185 | + end | |
| 186 | + end) | |
| 187 | + end | |
| 188 | + | |
| 189 | + defp rewrap_row(schema, id, fields, primary) do | |
| 190 | + Repo.transaction(fn -> | |
| 191 | + row = Repo.one(from(r in schema, where: r.id == ^id, lock: "FOR UPDATE")) | |
| 192 | + | |
| 193 | + with %{} <- row, | |
| 194 | + kid when kid != primary <- Map.fetch!(row, fields.key_id), | |
| 195 | + {:ok, plaintext} <- | |
| 196 | + decrypt(%{ | |
| 197 | + ciphertext: Map.fetch!(row, fields.ciphertext), | |
| 198 | + iv: Map.fetch!(row, fields.iv), | |
| 199 | + tag: Map.fetch!(row, fields.tag), | |
| 200 | + key_id: kid | |
| 201 | + }) do | |
| 202 | + {:ok, enc} = encrypt(plaintext) | |
| 203 | + | |
| 204 | + row | |
| 205 | + |> Ecto.Changeset.change(%{ | |
| 206 | + fields.ciphertext => enc.ciphertext, | |
| 207 | + fields.iv => enc.iv, | |
| 208 | + fields.tag => enc.tag, | |
| 209 | + fields.key_id => enc.key_id | |
| 210 | + }) | |
| 211 | + |> Repo.update!() | |
| 212 | + | |
| 213 | + :ok | |
| 214 | + else | |
| 215 | + # Gone, or rewritten under the primary key since we listed it. | |
| 216 | + nil -> :skip | |
| 217 | + ^primary -> :skip | |
| 218 | + {:error, reason} -> Repo.rollback(reason) | |
| 219 | + end | |
| 220 | + end) | |
| 221 | + |> case do | |
| 222 | + {:ok, result} -> result | |
| 223 | + {:error, reason} -> {:error, reason} | |
| 224 | + end | |
| 225 | + end | |
| 226 | + | |
| 91 | 227 | # ── repo secrets ───────────────────────────────────────────────────── |
| 92 | 228 | |
| 93 | 229 | def list_repo_secret_names(%Repository{id: rid}) do |
modified
lib/git_gud/workflows/runner_jwt.ex
+18
−1
@@ -38,9 +38,14 @@ defmodule GitGud.Workflows.RunnerJwt do
| 38 | 38 | Signed HS256 with the secret configured under: |
| 39 | 39 | |
| 40 | 40 | config :git_gud, GitGud.Workflows.RunnerJwt, |
| 41 | − secret: "<base64 32 bytes>", | |
| 41 | + secret: "<at least 32 random bytes>", | |
| 42 | 42 | ttl_seconds: 86400 |
| 43 | 43 | |
| 44 | + In production it comes from `GITGUD_RUNNER_JWT_SECRET` (see | |
| 45 | + `config/runtime.exs`). Changing it invalidates every token already | |
| 46 | + handed to a runner: jobs running at the time fail their remaining | |
| 47 | + checkout / cache / artifact / registry calls and need a re-run. | |
| 48 | + | |
| 44 | 49 | HS256 because only we mint and verify; no need for asymmetric keys |
| 45 | 50 | the way federation does. |
| 46 | 51 | """ |
@@ -56,6 +61,13 @@ defmodule GitGud.Workflows.RunnerJwt do
| 56 | 61 | # How long a token outlives its job: the runner's post-job steps. |
| 57 | 62 | @grace_seconds 300 |
| 58 | 63 | |
| 64 | + # The dev/test secret in config/config.exs. It's in the source, so | |
| 65 | + # anyone could sign with it: a production build won't. runtime.exs | |
| 66 | + # refuses to boot without `GITGUD_RUNNER_JWT_SECRET` first; this is | |
| 67 | + # the second check. | |
| 68 | + @dev_secret "dev-runner-jwt-secret-do-not-use-in-prod" | |
| 69 | + @refuse_dev_secret Mix.env() == :prod | |
| 70 | + | |
| 59 | 71 | @doc """ |
| 60 | 72 | Mint a token for a job that's just been claimed, carrying `perms` |
| 61 | 73 | (`GitGud.Workflows.Permissions.effective/2`). |
@@ -179,6 +191,11 @@ defmodule GitGud.Workflows.RunnerJwt do
| 179 | 191 | |
| 180 | 192 | case Keyword.fetch(cfg, :secret) do |
| 181 | 193 | {:ok, value} when is_binary(value) and value != "" -> |
| 194 | + if @refuse_dev_secret and value == @dev_secret do | |
| 195 | + raise "refusing to sign or verify job tokens with the public dev secret; " <> | |
| 196 | + "set GITGUD_RUNNER_JWT_SECRET" | |
| 197 | + end | |
| 198 | + | |
| 182 | 199 | value |
| 183 | 200 | |
| 184 | 201 | _ -> |
modified
packaging/deb/gitgud.env
+15
−0
@@ -16,6 +16,21 @@ DATABASE_URL=postgres://gitgud:CHANGEME@localhost/gitgud
| 16 | 16 | # Generate with: /opt/gitgud/bin/git_gud eval 'IO.puts(Base.encode64(:crypto.strong_rand_bytes(48)))' |
| 17 | 17 | SECRET_KEY_BASE=CHANGEME |
| 18 | 18 | |
| 19 | +# Signs CI job tokens. At least 32 bytes; changing it invalidates the | |
| 20 | +# tokens of jobs running at the time. | |
| 21 | +# Generate with: openssl rand -base64 48 | |
| 22 | +GITGUD_RUNNER_JWT_SECRET=CHANGEME | |
| 23 | + | |
| 24 | +# Encrypts secrets at rest (CI secrets, TOTP, import tokens, package | |
| 25 | +# signing keys). Base64 of exactly 32 bytes. Keep it safe: lose it and | |
| 26 | +# everything it encrypted is gone. GITGUD_SECRETS_KEY_ID names it (not | |
| 27 | +# "default"); when rotating, set a new key + id and list the old pair | |
| 28 | +# in GITGUD_SECRETS_OLD_KEYS=id:base64 — bin/migrate re-encrypts. | |
| 29 | +# Generate with: openssl rand -base64 32 | |
| 30 | +GITGUD_SECRETS_MASTER_KEY=CHANGEME | |
| 31 | +GITGUD_SECRETS_KEY_ID=k1 | |
| 32 | +#GITGUD_SECRETS_OLD_KEYS= | |
| 33 | + | |
| 19 | 34 | # The public hostname. Clone URLs, webhooks and federation all derive |
| 20 | 35 | # from it, so it must be what users actually type. |
| 21 | 36 | PHX_HOST=git.example.com |
modified
test/git_gud/secrets_test.exs
+78
−0
@@ -96,4 +96,82 @@ defmodule GitGud.SecretsTest do
| 96 | 96 | assert vars["REGION"] == "eu-west" |
| 97 | 97 | end |
| 98 | 98 | end |
| 99 | + | |
| 100 | + describe "key rotation and rewrap_all/0" do | |
| 101 | + # The config a production deploy switches to: a new primary key, | |
| 102 | + # with the all-zero key every earlier row was written under kept | |
| 103 | + # readable as "default". | |
| 104 | + @new_key Base.encode64(:crypto.strong_rand_bytes(32)) | |
| 105 | + | |
| 106 | + setup do | |
| 107 | + original = Application.get_env(:git_gud, Secrets) | |
| 108 | + on_exit(fn -> Application.put_env(:git_gud, Secrets, original) end) | |
| 109 | + %{legacy: original} | |
| 110 | + end | |
| 111 | + | |
| 112 | + defp switch_to_new_key(legacy) do | |
| 113 | + Application.put_env(:git_gud, Secrets, | |
| 114 | + primary_key_id: "k1", | |
| 115 | + master_key: @new_key, | |
| 116 | + master_keys: %{"default" => Keyword.fetch!(legacy, :master_key)} | |
| 117 | + ) | |
| 118 | + end | |
| 119 | + | |
| 120 | + defp key_ids(schema) do | |
| 121 | + GitGud.Repo.all(from(s in schema, select: s.key_id)) |> Enum.uniq() |> Enum.sort() | |
| 122 | + end | |
| 123 | + | |
| 124 | + test "legacy rows decrypt after the switch, move to the new key, and stay readable", | |
| 125 | + %{legacy: legacy} do | |
| 126 | + user = user_fixture() | |
| 127 | + {:ok, org} = Organizations.create_organization(user, %{"handle" => "rewrapz"}) | |
| 128 | + {:ok, repo} = Repositories.create_repository_for_org(org, user, %{"name" => "app"}) | |
| 129 | + | |
| 130 | + {:ok, _} = Secrets.put_org_secret(org, "ORG_TOKEN", "org-value") | |
| 131 | + {:ok, _} = Secrets.put_repo_secret(repo, "REPO_TOKEN", "repo-value") | |
| 132 | + assert key_ids(Secrets.RepositorySecret) == ["default"] | |
| 133 | + | |
| 134 | + switch_to_new_key(legacy) | |
| 135 | + | |
| 136 | + # Still readable through the legacy entry before any rewrap. | |
| 137 | + assert {%{"ORG_TOKEN" => "org-value", "REPO_TOKEN" => "repo-value"}, _} = | |
| 138 | + Secrets.resolve_for(repo) | |
| 139 | + | |
| 140 | + counts = Secrets.rewrap_all() | |
| 141 | + assert counts.repository_secrets == %{rewrapped: 1, failed: 0} | |
| 142 | + assert counts.organization_secrets == %{rewrapped: 1, failed: 0} | |
| 143 | + assert key_ids(Secrets.RepositorySecret) == ["k1"] | |
| 144 | + assert key_ids(Secrets.OrganizationSecret) == ["k1"] | |
| 145 | + | |
| 146 | + # Second run: nothing left to do. | |
| 147 | + assert Enum.all?(Secrets.rewrap_all(), fn {_, c} -> c == %{rewrapped: 0, failed: 0} end) | |
| 148 | + | |
| 149 | + # Readable with the legacy key gone entirely. | |
| 150 | + Application.put_env(:git_gud, Secrets, primary_key_id: "k1", master_key: @new_key) | |
| 151 | + | |
| 152 | + assert {%{"ORG_TOKEN" => "org-value", "REPO_TOKEN" => "repo-value"}, _} = | |
| 153 | + Secrets.resolve_for(repo) | |
| 154 | + end | |
| 155 | + | |
| 156 | + test "a row whose key isn't configured is counted as failed and left alone", | |
| 157 | + %{legacy: legacy} do | |
| 158 | + {_user, repo} = repository_fixture() | |
| 159 | + {:ok, _} = Secrets.put_repo_secret(repo, "KEY", "v") | |
| 160 | + | |
| 161 | + # New primary, and the legacy key forgotten. | |
| 162 | + Application.put_env(:git_gud, Secrets, primary_key_id: "k1", master_key: @new_key) | |
| 163 | + | |
| 164 | + log = | |
| 165 | + ExUnit.CaptureLog.capture_log(fn -> | |
| 166 | + assert Secrets.rewrap_all().repository_secrets == %{rewrapped: 0, failed: 1} | |
| 167 | + end) | |
| 168 | + | |
| 169 | + assert log =~ "no_key" | |
| 170 | + refute log =~ "\"v\"" | |
| 171 | + assert key_ids(Secrets.RepositorySecret) == ["default"] | |
| 172 | + | |
| 173 | + switch_to_new_key(legacy) | |
| 174 | + assert Secrets.rewrap_all().repository_secrets == %{rewrapped: 1, failed: 0} | |
| 175 | + end | |
| 176 | + end | |
| 99 | 177 | end |
Parents: 8417f8f