neiam
/gitgud
Git Gud
public · Issues · Pulls · Labels · Forks · Compare · Actions success · Badges · Packages
⭐
1
Log in to mark this repository.
Serve static binaries to clients that have no package manager
b0551cd · Gabriel Morell · 2026-09-16 18:20
Message
{commit_body(@commit)}
Files changed
modified
README.md
+4
−2
@@ -98,12 +98,13 @@ reinvent them.
| 98 | 98 | - `GitGud.Storage.Disk` (default) — single-node dev, single-node prod, NFS-shared multi-node |
| 99 | 99 | - `GitGud.Storage.Garage` — S3-flavored ([Garage](https://garagehq.deuxfleurs.fr/)) |
| 100 | 100 | - Add another with one `@behaviour` impl |
| 101 | −- **Package registries** — five formats: four served by the forge on | |
| 102 | − one substrate, plus OCI in zot | |
| 101 | +- **Package registries** — several formats served by the forge on one | |
| 102 | + substrate, plus OCI in zot | |
| 103 | 103 | - **PyPI** ([PEP 503](https://peps.python.org/pep-0503/) + [PEP 691](https://peps.python.org/pep-0691/)) — `pip install --index-url …/simple/`, `twine upload`. Both serialisations off one data structure, negotiated on `Accept` with real q-values; PEP 592 yank state; name normalisation applied before every lookup |
| 104 | 104 | - **Hex** ([registry v2](https://github.com/hexpm/specifications/blob/main/registry-v2.md)) — `mix hex.repo add` + `{:dep, "~> 1.0", repo: "…"}`. Protobuf encoded per request out of Postgres rather than rebuilt by `mix hex.registry build`, so publishing is an insert instead of O(everything) |
| 105 | 105 | - **Cargo** ([sparse index](https://doc.rust-lang.org/cargo/reference/registry-index.html)) — `cargo publish`, `add`, `yank`. Prefix-sharded index with real `ETag`/`304` revalidation, `features2`/`v: 2` splitting, and the `explicit_name_in_toml` → `package` rename translation |
| 106 | 106 | - **apt** ([DebianRepository/Format](https://wiki.debian.org/DebianRepository/Format)) — signed `InRelease`, `Packages.xz`, pool. **Namespace-wide repositories at `_all`**: one `deb` line and one keyring file for everything an org (or a user) publishes, rather than one of each per repository. **Browsable**, the way a real mirror is — click down through `dists/` and `pool/` in a browser, in the forge's own theme, with the listing derived from what the repository actually serves so it can neither advertise a 404 nor show a repository you couldn't read |
| 107 | + - **raw** — static files with a JSON manifest, for clients that have no package manager: a microcontroller doing OTA, a shell script, an installer. `manifest.json` lists every version with an absolute URL and a SHA-256 per file and a `latest` pointer, so the whole client protocol is "fetch one document, then fetch a URL out of it". Publish with `curl --upload-file` to `/<version>/<filename>`; one package per repository, and `latest` means the last thing published rather than the highest version string, since raw bytes have no version grammar to sort by | |
| 107 | 108 | - **OCI** hosted in [zot](https://zotregistry.dev/) — the forge owns `/v2/token` (`docker login` → Basic-auth handshake → RS256 JWT scoped to `repository:<owner>/<name>:pull,push`); zot owns the manifests and blobs. Deployment manifest in `zot.yml`; keypair from `mix gitgud.registry.keygen`; `(package, version)` mirrored via `/_webhooks/zot` |
| 108 | 109 | - **Publish from CI with no stored secret** — `${{ github.token }}` is accepted by every registry and authorized to write its own repository only, exactly as at the OCI token endpoint |
| 109 | 110 | - **Signing keys are per namespace**, not per repository — one `signed-by=` entry or one `--public-key` covers every repository an org publishes. Served at `/pkg/_keys/:format/:owner`; private halves encrypted at rest under the `GitGud.Secrets` master key |
@@ -399,6 +400,7 @@ exact lines to paste.
| 399 | 400 | /pkg/cargo/<owner>/<repo>/ cargo (sparse+…) |
| 400 | 401 | /pkg/deb/<owner>/<repo> apt |
| 401 | 402 | /pkg/deb/<owner>/_all apt, whole namespace |
| 403 | +/pkg/raw/<owner>/<repo>/manifest.json raw (then latest/<file>) | |
| 402 | 404 | /pkg/_keys/<format>/<owner> the namespace's public key |
| 403 | 405 | ``` |
| 404 | 406 |
modified
lib/git_gud/packages/format.ex
+3
−2
@@ -38,7 +38,7 @@ defmodule GitGud.Packages.Format do
| 38 | 38 | |
| 39 | 39 | alias GitGud.Repositories.Repository |
| 40 | 40 | |
| 41 | − @formats ~w(oci pypi hex cargo deb rpm fdroid) | |
| 41 | + @formats ~w(oci pypi hex cargo deb rpm fdroid raw) | |
| 42 | 42 | |
| 43 | 43 | # Format identifier → adapter module. `oci` is absent on purpose: zot |
| 44 | 44 | # is the authoritative store for container images and serves the |
@@ -51,7 +51,8 @@ defmodule GitGud.Packages.Format do
| 51 | 51 | "cargo" => GitGud.Packages.Formats.Cargo, |
| 52 | 52 | "deb" => GitGud.Packages.Formats.Deb, |
| 53 | 53 | "rpm" => GitGud.Packages.Formats.Rpm, |
| 54 | − "fdroid" => GitGud.Packages.Formats.Fdroid | |
| 54 | + "fdroid" => GitGud.Packages.Formats.Fdroid, | |
| 55 | + "raw" => GitGud.Packages.Formats.Raw | |
| 55 | 56 | } |
| 56 | 57 | |
| 57 | 58 | @typedoc """ |
added
lib/git_gud/packages/formats/raw.ex
+278
−0
@@ -0,0 +1,278 @@
| 1 | +defmodule GitGud.Packages.Formats.Raw do | |
| 2 | + @moduledoc """ | |
| 3 | + Static binaries with a manifest, for consumers that have no package | |
| 4 | + manager at all. | |
| 5 | + | |
| 6 | + GET /pkg/raw/<owner>/<repo>/manifest.json what exists | |
| 7 | + GET /pkg/raw/<owner>/<repo>/latest/<file> the newest bytes | |
| 8 | + GET /pkg/raw/<owner>/<repo>/<version>/<file> one version's bytes | |
| 9 | + PUT /pkg/raw/<owner>/<repo>/<version>/<file> publish | |
| 10 | + | |
| 11 | + The other five formats each serve an ecosystem: something on the far | |
| 12 | + end knows how to read a sparse index, a `Release` file, a `repomd`. | |
| 13 | + This one exists for the far end that knows nothing — a microcontroller | |
| 14 | + doing OTA, a shell script, an installer — where the whole protocol a | |
| 15 | + client can be expected to implement is "fetch a small JSON document, | |
| 16 | + then fetch a URL out of it". | |
| 17 | + | |
| 18 | + So the manifest is the format. It is one document listing every | |
| 19 | + version and every file, each with the absolute URL to fetch and the | |
| 20 | + SHA-256 to check it against, and a `latest` pointer at the top: | |
| 21 | + | |
| 22 | + {"name": "lambent-flex-c6", | |
| 23 | + "latest": "0.1.0+ci20260916.g1a2b3c4", | |
| 24 | + "versions": { | |
| 25 | + "0.1.0+ci20260916.g1a2b3c4": { | |
| 26 | + "published_at": "2026-09-16T17:00:00Z", | |
| 27 | + "yanked": false, | |
| 28 | + "files": [{"name": "lambent-flex-c6.bin", | |
| 29 | + "size": 1312768, | |
| 30 | + "sha256": "9f86d0…", | |
| 31 | + "content_type": "application/octet-stream", | |
| 32 | + "url": "https://…/0.1.0+ci20260916.g1a2b3c4/lambent-flex-c6.bin"}]}}} | |
| 33 | + | |
| 34 | + ## One package per repository | |
| 35 | + | |
| 36 | + Every other format takes the package name from the artifact it is | |
| 37 | + handed — a wheel, a `.deb`, an APK all carry their own. Raw bytes | |
| 38 | + carry nothing, so the package is named after the repository and a | |
| 39 | + repository that ships several binaries ships them as several *files* | |
| 40 | + of one version. That keeps the URL free of a name nobody would have | |
| 41 | + a source for, and keeps `manifest.json` a single document rather than | |
| 42 | + an index of indexes. | |
| 43 | + | |
| 44 | + ## `latest` is the last thing published, not the highest version | |
| 45 | + | |
| 46 | + There is no version grammar here to sort by: `2026-09-16`, `v1.2.3` | |
| 47 | + and `0.1.0+ci…g1a2b3c4` are all plausible and none of them compares | |
| 48 | + to the others. So `latest` is whatever was published most recently, | |
| 49 | + which is both well-defined and the thing CI actually means. Rolling a | |
| 50 | + fleet back is then republishing the older bytes under a new version, | |
| 51 | + rather than hoping a comparison function agrees with you. | |
| 52 | + | |
| 53 | + Consequently `latest` is a reserved word in the version slot: a | |
| 54 | + version literally named `latest` is unreachable at its own URL. | |
| 55 | + | |
| 56 | + ## No signing key | |
| 57 | + | |
| 58 | + Every other signature-bearing format here pins a key because its | |
| 59 | + client fetches indexes it cannot otherwise trust. A raw consumer | |
| 60 | + fetches the manifest and the bytes from the same origin over TLS, and | |
| 61 | + checks the SHA-256 the manifest gave it — which catches a truncated | |
| 62 | + transfer and a mismatched publish. Anything stronger than that is | |
| 63 | + secure boot's job, on the device, not a detached signature this | |
| 64 | + registry could offer. | |
| 65 | + """ | |
| 66 | + | |
| 67 | + @behaviour GitGud.Packages.Format | |
| 68 | + | |
| 69 | + alias GitGud.Packages | |
| 70 | + alias GitGud.Packages.Package | |
| 71 | + alias GitGud.Packages.PackageFile | |
| 72 | + alias GitGud.Packages.PackageVersion | |
| 73 | + alias GitGud.Repositories.Repository | |
| 74 | + | |
| 75 | + @format "raw" | |
| 76 | + | |
| 77 | + # The version slot that means "resolve it for me". | |
| 78 | + @latest "latest" | |
| 79 | + | |
| 80 | + @doc """ | |
| 81 | + Media type for a stored artifact. | |
| 82 | + | |
| 83 | + Deliberately shallow: the point of this registry is bytes, and a | |
| 84 | + client that cared about the type would be using a format that has | |
| 85 | + one. Enough of a table to keep a browser from offering to download a | |
| 86 | + checksum file, and `application/octet-stream` for everything else. | |
| 87 | + """ | |
| 88 | + @impl true | |
| 89 | + def content_type(filename) when is_binary(filename) do | |
| 90 | + case filename |> Path.extname() |> String.downcase() do | |
| 91 | + ".json" -> "application/json" | |
| 92 | + ".txt" -> "text/plain; charset=utf-8" | |
| 93 | + ".md" -> "text/markdown; charset=utf-8" | |
| 94 | + ".sha256" -> "text/plain; charset=utf-8" | |
| 95 | + ".csv" -> "text/csv; charset=utf-8" | |
| 96 | + ".gz" -> "application/gzip" | |
| 97 | + ".zip" -> "application/zip" | |
| 98 | + ".tar" -> "application/x-tar" | |
| 99 | + _ -> "application/octet-stream" | |
| 100 | + end | |
| 101 | + end | |
| 102 | + | |
| 103 | + def content_type(_filename), do: "application/octet-stream" | |
| 104 | + | |
| 105 | + # ── reads ─────────────────────────────────────────────────────────── | |
| 106 | + | |
| 107 | + @impl true | |
| 108 | + def index(repo, ctx) | |
| 109 | + | |
| 110 | + # The manifest, at the registry root and under its own name. Both, | |
| 111 | + # because a person poking at the URL in a browser lands on the root | |
| 112 | + # and a device has a full URL written into its firmware — and a | |
| 113 | + # device's URL should say what it fetches. | |
| 114 | + def index(repo, %{path: path} = ctx) when path in [[], ["manifest.json"]] do | |
| 115 | + {:ok, "application/json", manifest(repo, ctx), | |
| 116 | + [ | |
| 117 | + # A manifest that a cache pins is a fleet that never updates. | |
| 118 | + # Revalidation is cheap; the bytes it points at are immutable | |
| 119 | + # and cache as aggressively as anything wants to. | |
| 120 | + {"cache-control", "no-cache"} | |
| 121 | + ]} | |
| 122 | + end | |
| 123 | + | |
| 124 | + def index(repo, %{path: [@latest, filename]}) do | |
| 125 | + with {:ok, package} <- fetch_package(repo), | |
| 126 | + %PackageVersion{} = version <- newest(package) do | |
| 127 | + serve(version, filename) | |
| 128 | + else | |
| 129 | + _ -> {:error, :not_found} | |
| 130 | + end | |
| 131 | + end | |
| 132 | + | |
| 133 | + def index(repo, %{path: [version, filename]}) do | |
| 134 | + with {:ok, package} <- fetch_package(repo), | |
| 135 | + %PackageVersion{} = version <- Packages.get_version(package, version) do | |
| 136 | + serve(version, filename) | |
| 137 | + else | |
| 138 | + _ -> {:error, :not_found} | |
| 139 | + end | |
| 140 | + end | |
| 141 | + | |
| 142 | + def index(_repo, _ctx), do: {:error, :not_found} | |
| 143 | + | |
| 144 | + defp serve(%PackageVersion{} = version, filename) do | |
| 145 | + case Enum.find(Packages.list_files(version), &(&1.filename == filename)) do | |
| 146 | + %PackageFile{} = file -> {:blob, file} | |
| 147 | + nil -> {:error, :not_found} | |
| 148 | + end | |
| 149 | + end | |
| 150 | + | |
| 151 | + # ── writes ────────────────────────────────────────────────────────── | |
| 152 | + | |
| 153 | + @impl true | |
| 154 | + def store(repo, ctx, body) | |
| 155 | + | |
| 156 | + # PUT, because the URL names the thing being created and publishing | |
| 157 | + # the same bytes twice is a no-op — `publish_file/5` is idempotent on | |
| 158 | + # (version, filename) and refuses a filename whose bytes changed. | |
| 159 | + # POST is accepted at the same path for callers whose HTTP client | |
| 160 | + # makes PUT awkward. | |
| 161 | + def store(repo, %{method: method, path: [version, filename]}, body) | |
| 162 | + when method in ["PUT", "POST"] and is_binary(body) do | |
| 163 | + cond do | |
| 164 | + version == @latest -> | |
| 165 | + {:error, {400, "'#{@latest}' is reserved; publish under a real version"}} | |
| 166 | + | |
| 167 | + body == "" -> | |
| 168 | + {:error, {400, "empty body"}} | |
| 169 | + | |
| 170 | + true -> | |
| 171 | + publish(repo, version, filename, body) | |
| 172 | + end | |
| 173 | + end | |
| 174 | + | |
| 175 | + def store(_repo, %{path: path}, _body) when length(path) != 2, | |
| 176 | + do: {:error, {400, "publish to /<version>/<filename>"}} | |
| 177 | + | |
| 178 | + # A multipart body arrives as a params map (Plug.Parsers decoded it | |
| 179 | + # before the controller saw it). Nothing publishing raw bytes has a | |
| 180 | + # reason to send one, and guessing which part is the artifact is how | |
| 181 | + # the wrong bytes get published. | |
| 182 | + def store(_repo, _ctx, _body), | |
| 183 | + do: {:error, {400, "send the file as the request body, not as a form"}} | |
| 184 | + | |
| 185 | + defp publish(%Repository{} = repo, version, filename, body) do | |
| 186 | + attrs = %{ | |
| 187 | + filename: filename, | |
| 188 | + body: body, | |
| 189 | + content_type: content_type(filename) | |
| 190 | + } | |
| 191 | + | |
| 192 | + case Packages.publish_file(repo, @format, repo.name, version, attrs) do | |
| 193 | + {:ok, _package, _version, file} -> | |
| 194 | + {:ok, "application/json", | |
| 195 | + Jason.encode_to_iodata!(%{ | |
| 196 | + "name" => repo.name, | |
| 197 | + "version" => version, | |
| 198 | + "filename" => file.filename, | |
| 199 | + "size" => file.size, | |
| 200 | + "sha256" => file.sha256 | |
| 201 | + })} | |
| 202 | + | |
| 203 | + {:error, :file_exists} -> | |
| 204 | + {:error, :file_exists} | |
| 205 | + | |
| 206 | + {:error, reason} -> | |
| 207 | + {:error, reason} | |
| 208 | + end | |
| 209 | + end | |
| 210 | + | |
| 211 | + # ── the manifest ──────────────────────────────────────────────────── | |
| 212 | + | |
| 213 | + defp manifest(%Repository{} = repo, ctx) do | |
| 214 | + versions = | |
| 215 | + case fetch_package(repo) do | |
| 216 | + {:ok, package} -> Packages.list_versions(package) | |
| 217 | + :error -> [] | |
| 218 | + end | |
| 219 | + | |
| 220 | + Jason.encode_to_iodata!(%{ | |
| 221 | + "name" => repo.name, | |
| 222 | + # `null` rather than an omitted key: a client that has to | |
| 223 | + # distinguish "nothing published yet" from "I fetched the wrong | |
| 224 | + # URL" can, and one that doesn't care reads the same falsy value | |
| 225 | + # either way. | |
| 226 | + "latest" => latest_version(versions), | |
| 227 | + "versions" => | |
| 228 | + Map.new(versions, fn version -> | |
| 229 | + {version.version, describe(version, ctx)} | |
| 230 | + end) | |
| 231 | + }) | |
| 232 | + end | |
| 233 | + | |
| 234 | + defp describe(%PackageVersion{} = version, ctx) do | |
| 235 | + %{ | |
| 236 | + "published_at" => version.pushed_at && DateTime.to_iso8601(version.pushed_at), | |
| 237 | + "yanked" => version.yanked, | |
| 238 | + "files" => | |
| 239 | + Enum.map(Packages.list_files(version), fn file -> | |
| 240 | + %{ | |
| 241 | + "name" => file.filename, | |
| 242 | + "size" => file.size, | |
| 243 | + "sha256" => file.sha256, | |
| 244 | + "content_type" => file.content_type, | |
| 245 | + # Absolute, so a device stores one URL (the manifest's) and | |
| 246 | + # follows what it is given rather than assembling paths. | |
| 247 | + "url" => "#{ctx.base_url}/#{encode(version.version)}/#{encode(file.filename)}" | |
| 248 | + } | |
| 249 | + end) | |
| 250 | + } | |
| 251 | + end | |
| 252 | + | |
| 253 | + # `list_versions/1` is newest-published first, so this is the head — | |
| 254 | + # skipping anything yanked, since a yanked version is exactly one an | |
| 255 | + # updater should not roll onto. | |
| 256 | + defp newest(%Package{} = package), do: package |> Packages.list_versions() |> newest() | |
| 257 | + | |
| 258 | + defp newest(versions) when is_list(versions), do: Enum.find(versions, &(not &1.yanked)) | |
| 259 | + | |
| 260 | + defp latest_version(versions) do | |
| 261 | + case newest(versions) do | |
| 262 | + %PackageVersion{version: version} -> version | |
| 263 | + nil -> nil | |
| 264 | + end | |
| 265 | + end | |
| 266 | + | |
| 267 | + defp fetch_package(%Repository{} = repo) do | |
| 268 | + case Packages.get_package(repo, @format, repo.name) do | |
| 269 | + %Package{} = package -> {:ok, package} | |
| 270 | + nil -> :error | |
| 271 | + end | |
| 272 | + end | |
| 273 | + | |
| 274 | + # Version strings carry `+` (build metadata) and filenames carry all | |
| 275 | + # sorts of things; both are single path segments, so encode them as | |
| 276 | + # such rather than trusting them to be URL-clean. | |
| 277 | + defp encode(segment), do: URI.encode(segment, &URI.char_unreserved?/1) | |
| 278 | +end |
modified
lib/git_gud/packages/usage.ex
+68
−0
@@ -598,6 +598,65 @@ defmodule GitGud.Packages.Usage do
| 598 | 598 | } |
| 599 | 599 | end |
| 600 | 600 | |
| 601 | + # ── raw ───────────────────────────────────────────────────────────── | |
| 602 | + | |
| 603 | + def for_format("raw", ctx) do | |
| 604 | + url = base_url(ctx, "raw") | |
| 605 | + | |
| 606 | + %{ | |
| 607 | + consume: [ | |
| 608 | + step( | |
| 609 | + "Everything published, with a URL and a checksum per file", | |
| 610 | + "shell", | |
| 611 | + """ | |
| 612 | + curl -fsSL #{url}/manifest.json | |
| 613 | + """ | |
| 614 | + ), | |
| 615 | + step( | |
| 616 | + "Fetch the newest bytes without resolving a version first", | |
| 617 | + "shell", | |
| 618 | + """ | |
| 619 | + curl -fsSLO #{url}/latest/<filename> | |
| 620 | + """ | |
| 621 | + ), | |
| 622 | + step( | |
| 623 | + "…or one version, which never changes once published", | |
| 624 | + "shell", | |
| 625 | + """ | |
| 626 | + curl -fsSLO #{url}/<version>/<filename> | |
| 627 | + """ | |
| 628 | + ) | |
| 629 | + ], | |
| 630 | + publish: [ | |
| 631 | + step( | |
| 632 | + "Publish a file under a version", | |
| 633 | + "shell", | |
| 634 | + """ | |
| 635 | + curl -fsSL --fail \\ | |
| 636 | + -H "Authorization: $GITGUD_TOKEN" \\ | |
| 637 | + --upload-file ./build/firmware.bin \\ | |
| 638 | + "#{url}/1.2.3/firmware.bin" | |
| 639 | + """ | |
| 640 | + ) | |
| 641 | + ], | |
| 642 | + ci: [ | |
| 643 | + step( | |
| 644 | + "Publish from a workflow", | |
| 645 | + "yaml", | |
| 646 | + """ | |
| 647 | + - name: Publish the build | |
| 648 | + run: | | |
| 649 | + version="1.2.3+ci$(date +%Y%m%d).g${GITHUB_SHA::7}" | |
| 650 | + curl -fsSL --fail \\ | |
| 651 | + -H "Authorization: ${{ github.token }}" \\ | |
| 652 | + --upload-file ./build/firmware.bin \\ | |
| 653 | + "#{url}/$version/firmware.bin" | |
| 654 | + """ | |
| 655 | + ) | |
| 656 | + ] | |
| 657 | + } | |
| 658 | + end | |
| 659 | + | |
| 601 | 660 | def for_format(_format, _ctx), do: nil |
| 602 | 661 | |
| 603 | 662 | @doc """ |
@@ -643,6 +702,12 @@ defmodule GitGud.Packages.Usage do
| 643 | 702 | def install_line("rpm", _ctx, pkg, version), |
| 644 | 703 | do: "sudo dnf install #{pkg.name}-#{version}" |
| 645 | 704 | |
| 705 | + # No package manager to name, so the useful line is the fetch — and | |
| 706 | + # the version's own URL rather than `latest/`, since a reader looking | |
| 707 | + # at a version list means that one. | |
| 708 | + def install_line("raw", ctx, _pkg, version), | |
| 709 | + do: "curl -fsSLO #{base_url(ctx, "raw")}/#{version}/<filename>" | |
| 710 | + | |
| 646 | 711 | def install_line(_format, _ctx, _pkg, _version), do: nil |
| 647 | 712 | |
| 648 | 713 | @doc """ |
@@ -661,6 +726,9 @@ defmodule GitGud.Packages.Usage do
| 661 | 726 | def summary("fdroid"), |
| 662 | 727 | do: "An F-Droid app repository — add it once, install and update from the phone." |
| 663 | 728 | |
| 729 | + def summary("raw"), | |
| 730 | + do: "Static files with a JSON manifest — for clients with no package manager." | |
| 731 | + | |
| 664 | 732 | def summary("oci"), do: "Container images, served by the bundled OCI registry." |
| 665 | 733 | def summary(_), do: nil |
| 666 | 734 |
added
test/git_gud_web/raw_registry_test.exs
+228
−0
@@ -0,0 +1,228 @@
| 1 | +defmodule GitGudWeb.RawRegistryTest do | |
| 2 | + @moduledoc """ | |
| 3 | + The raw registry, over HTTP. | |
| 4 | + | |
| 5 | + The client this format exists for is a microcontroller with a few | |
| 6 | + hundred lines of C and no package manager, so the assertions are the | |
| 7 | + ones such a client depends on and cannot work around: the manifest | |
| 8 | + names a `latest`, every file carries a URL that can be fetched | |
| 9 | + verbatim and a SHA-256 that matches the bytes on the other end, and | |
| 10 | + published bytes never change under a filename that was already served. | |
| 11 | + """ | |
| 12 | + use GitGudWeb.ConnCase, async: false | |
| 13 | + | |
| 14 | + alias GitGud.Packages | |
| 15 | + alias GitGud.Repositories.Storage | |
| 16 | + | |
| 17 | + import GitGud.ForgeFixtures | |
| 18 | + | |
| 19 | + @firmware <<0xE9, 0x06, 0x02, 0x1F>> <> :crypto.strong_rand_bytes(1024) | |
| 20 | + | |
| 21 | + setup do | |
| 22 | + {user, repo} = repository_fixture(%{visibility: "public"}) | |
| 23 | + {:ok, _t, token} = Packages.create_registry_token(repo, "ci", ["pull", "push"]) | |
| 24 | + | |
| 25 | + {:ok, | |
| 26 | + user: user, | |
| 27 | + repo: repo, | |
| 28 | + token: token, | |
| 29 | + root: "/pkg/raw/#{Storage.owner_handle(user)}/#{repo.name}"} | |
| 30 | + end | |
| 31 | + | |
| 32 | + defp publish(conn, root, token, version, filename, body) do | |
| 33 | + conn = if token, do: put_req_header(conn, "authorization", token), else: conn | |
| 34 | + | |
| 35 | + conn | |
| 36 | + |> put_req_header("content-type", "application/octet-stream") | |
| 37 | + |> put("#{root}/#{URI.encode(version, &URI.char_unreserved?/1)}/#{filename}", body) | |
| 38 | + end | |
| 39 | + | |
| 40 | + defp manifest(conn, root), do: Jason.decode!(get(conn, root <> "/manifest.json").resp_body) | |
| 41 | + | |
| 42 | + describe "publishing" do | |
| 43 | + test "stores the bytes and reports the digest", %{conn: conn, root: root, token: token} do | |
| 44 | + conn = publish(conn, root, token, "1.0.0", "firmware.bin", @firmware) | |
| 45 | + | |
| 46 | + assert conn.status == 200 | |
| 47 | + body = Jason.decode!(conn.resp_body) | |
| 48 | + assert body["version"] == "1.0.0" | |
| 49 | + assert body["filename"] == "firmware.bin" | |
| 50 | + assert body["size"] == byte_size(@firmware) | |
| 51 | + assert body["sha256"] == Base.encode16(:crypto.hash(:sha256, @firmware), case: :lower) | |
| 52 | + end | |
| 53 | + | |
| 54 | + test "needs a token", %{conn: conn, root: root} do | |
| 55 | + assert publish(conn, root, nil, "1.0.0", "firmware.bin", @firmware).status == 401 | |
| 56 | + end | |
| 57 | + | |
| 58 | + test "the same bytes twice is a no-op, not a conflict", %{ | |
| 59 | + conn: conn, | |
| 60 | + root: root, | |
| 61 | + token: token | |
| 62 | + } do | |
| 63 | + publish(conn, root, token, "1.0.0", "firmware.bin", @firmware) | |
| 64 | + | |
| 65 | + assert publish(build_conn(), root, token, "1.0.0", "firmware.bin", @firmware).status == 200 | |
| 66 | + end | |
| 67 | + | |
| 68 | + test "different bytes under a published filename are refused", %{ | |
| 69 | + conn: conn, | |
| 70 | + root: root, | |
| 71 | + token: token | |
| 72 | + } do | |
| 73 | + publish(conn, root, token, "1.0.0", "firmware.bin", @firmware) | |
| 74 | + | |
| 75 | + conn = publish(build_conn(), root, token, "1.0.0", "firmware.bin", @firmware <> "more") | |
| 76 | + assert conn.status == 409 | |
| 77 | + end | |
| 78 | + | |
| 79 | + test "'latest' is reserved in the version slot", %{conn: conn, root: root, token: token} do | |
| 80 | + conn = publish(conn, root, token, "latest", "firmware.bin", @firmware) | |
| 81 | + | |
| 82 | + assert conn.status == 400 | |
| 83 | + assert conn.resp_body =~ "reserved" | |
| 84 | + end | |
| 85 | + | |
| 86 | + test "an empty body is refused", %{conn: conn, root: root, token: token} do | |
| 87 | + assert publish(conn, root, token, "1.0.0", "firmware.bin", "").status == 400 | |
| 88 | + end | |
| 89 | + end | |
| 90 | + | |
| 91 | + describe "the manifest" do | |
| 92 | + test "is empty but well-formed before anything is published", %{conn: conn, root: root} do | |
| 93 | + body = manifest(conn, root) | |
| 94 | + | |
| 95 | + assert body["latest"] == nil | |
| 96 | + assert body["versions"] == %{} | |
| 97 | + end | |
| 98 | + | |
| 99 | + test "lists every version, with a fetchable url and a checksum", %{ | |
| 100 | + conn: conn, | |
| 101 | + root: root, | |
| 102 | + token: token, | |
| 103 | + user: user, | |
| 104 | + repo: repo | |
| 105 | + } do | |
| 106 | + publish(conn, root, token, "1.0.0", "firmware.bin", @firmware) | |
| 107 | + | |
| 108 | + body = manifest(build_conn(), root) | |
| 109 | + assert body["name"] == repo.name | |
| 110 | + assert body["latest"] == "1.0.0" | |
| 111 | + | |
| 112 | + version = body["versions"]["1.0.0"] | |
| 113 | + refute version["yanked"] | |
| 114 | + assert version["published_at"] =~ ~r/^\d{4}-/ | |
| 115 | + | |
| 116 | + assert [file] = version["files"] | |
| 117 | + assert file["name"] == "firmware.bin" | |
| 118 | + assert file["size"] == byte_size(@firmware) | |
| 119 | + assert file["sha256"] == Base.encode16(:crypto.hash(:sha256, @firmware), case: :lower) | |
| 120 | + assert file["content_type"] == "application/octet-stream" | |
| 121 | + | |
| 122 | + # The URL is the contract: a device follows it verbatim. | |
| 123 | + handle = Storage.owner_handle(user) | |
| 124 | + assert file["url"] =~ "/pkg/raw/#{handle}/#{repo.name}/1.0.0/firmware.bin" | |
| 125 | + assert %URI{path: path} = URI.parse(file["url"]) | |
| 126 | + assert get(build_conn(), path).resp_body == @firmware | |
| 127 | + end | |
| 128 | + | |
| 129 | + test "'latest' is the most recently published, not the highest string", %{ | |
| 130 | + conn: conn, | |
| 131 | + root: root, | |
| 132 | + token: token | |
| 133 | + } do | |
| 134 | + publish(conn, root, token, "2.0.0", "firmware.bin", @firmware) | |
| 135 | + publish(build_conn(), root, token, "1.0.0", "firmware.bin", @firmware <> "older") | |
| 136 | + | |
| 137 | + assert manifest(build_conn(), root)["latest"] == "1.0.0" | |
| 138 | + end | |
| 139 | + | |
| 140 | + test "skips a yanked version when choosing latest", %{ | |
| 141 | + conn: conn, | |
| 142 | + root: root, | |
| 143 | + token: token, | |
| 144 | + repo: repo | |
| 145 | + } do | |
| 146 | + publish(conn, root, token, "1.0.0", "firmware.bin", @firmware) | |
| 147 | + publish(build_conn(), root, token, "1.1.0", "firmware.bin", @firmware <> "bad") | |
| 148 | + | |
| 149 | + package = Packages.get_package(repo, "raw", repo.name) | |
| 150 | + {:ok, _} = package |> Packages.get_version("1.1.0") |> Packages.set_yanked(true, "bricks") | |
| 151 | + | |
| 152 | + body = manifest(build_conn(), root) | |
| 153 | + assert body["latest"] == "1.0.0" | |
| 154 | + assert body["versions"]["1.1.0"]["yanked"] | |
| 155 | + end | |
| 156 | + | |
| 157 | + test "is served at the registry root as well", %{conn: conn, root: root, token: token} do | |
| 158 | + publish(conn, root, token, "1.0.0", "firmware.bin", @firmware) | |
| 159 | + | |
| 160 | + assert Jason.decode!(get(build_conn(), root).resp_body)["latest"] == "1.0.0" | |
| 161 | + end | |
| 162 | + | |
| 163 | + test "is not cached, or a fleet never sees an update", %{conn: conn, root: root} do | |
| 164 | + conn = get(conn, root <> "/manifest.json") | |
| 165 | + | |
| 166 | + assert get_resp_header(conn, "cache-control") == ["no-cache"] | |
| 167 | + end | |
| 168 | + end | |
| 169 | + | |
| 170 | + describe "downloads" do | |
| 171 | + setup %{conn: conn, root: root, token: token} do | |
| 172 | + publish(conn, root, token, "1.0.0", "firmware.bin", @firmware) | |
| 173 | + :ok | |
| 174 | + end | |
| 175 | + | |
| 176 | + test "a public repository serves bytes to anyone", %{root: root} do | |
| 177 | + conn = get(build_conn(), root <> "/1.0.0/firmware.bin") | |
| 178 | + | |
| 179 | + assert conn.status == 200 | |
| 180 | + assert conn.resp_body == @firmware | |
| 181 | + end | |
| 182 | + | |
| 183 | + test "'latest' resolves without the client knowing a version", %{root: root, token: token} do | |
| 184 | + publish(build_conn(), root, token, "1.1.0", "firmware.bin", @firmware <> "newer") | |
| 185 | + | |
| 186 | + assert get(build_conn(), root <> "/latest/firmware.bin").resp_body == @firmware <> "newer" | |
| 187 | + end | |
| 188 | + | |
| 189 | + test "an unknown version or filename is a 404", %{root: root} do | |
| 190 | + assert get(build_conn(), root <> "/9.9.9/firmware.bin").status == 404 | |
| 191 | + assert get(build_conn(), root <> "/1.0.0/nothing.bin").status == 404 | |
| 192 | + end | |
| 193 | + | |
| 194 | + test "a version with a build-metadata '+' round-trips", %{root: root, token: token} do | |
| 195 | + version = "0.1.0+ci20260916.g1a2b3c4" | |
| 196 | + publish(build_conn(), root, token, version, "firmware.bin", @firmware <> "ci") | |
| 197 | + | |
| 198 | + [file] = manifest(build_conn(), root)["versions"][version]["files"] | |
| 199 | + assert %URI{path: path} = URI.parse(file["url"]) | |
| 200 | + assert get(build_conn(), path).resp_body == @firmware <> "ci" | |
| 201 | + end | |
| 202 | + end | |
| 203 | + | |
| 204 | + describe "a private repository" do | |
| 205 | + test "challenges an anonymous reader", %{} do | |
| 206 | + {user, private} = repository_fixture(%{visibility: "private"}) | |
| 207 | + root = "/pkg/raw/#{Storage.owner_handle(user)}/#{private.name}" | |
| 208 | + | |
| 209 | + conn = get(build_conn(), root <> "/manifest.json") | |
| 210 | + | |
| 211 | + assert conn.status == 401 | |
| 212 | + assert [_challenge] = get_resp_header(conn, "www-authenticate") | |
| 213 | + end | |
| 214 | + | |
| 215 | + test "serves the manifest to a token", %{} do | |
| 216 | + {user, private} = repository_fixture(%{visibility: "private"}) | |
| 217 | + {:ok, _t, token} = Packages.create_registry_token(private, "ci", ["pull"]) | |
| 218 | + root = "/pkg/raw/#{Storage.owner_handle(user)}/#{private.name}" | |
| 219 | + | |
| 220 | + conn = | |
| 221 | + build_conn() | |
| 222 | + |> put_req_header("authorization", token) | |
| 223 | + |> get(root <> "/manifest.json") | |
| 224 | + | |
| 225 | + assert conn.status == 200 | |
| 226 | + end | |
| 227 | + end | |
| 228 | +end |
Parents: 8d41fd5