defmodule GitGud.Packages.Format do
@moduledoc """
The one thing that actually differs between package registries.
PyPI, Hex, Cargo and apt share everything that was expensive to
build: blob storage (`GitGud.Storage`), namespacing (`Package` /
`PackageVersion` / `PackageFile`, per repository), auth tokens
(`GitGud.Packages.RegistryToken`), and a UI. What differs is the
*index shape* a client fetches and the *upload framing* a publish
tool sends. That is this behaviour, and there is one implementation
per format — the same move `GitGud.Storage` makes for backends.
Everything is served under `/pkg/:format/:owner/:name/*rest`;
`GitGudWeb.PackageRegistryController` resolves the repository,
authenticates, and hands the remaining path to the adapter.
## `index/2`
Serves every read: index documents, metadata, and artifact
downloads. The `ctx` carries the path below the repository root plus
the raw `Plug.Conn`, because several formats negotiate on request
headers — PEP 691 picks a serialisation off `Accept`, Cargo's sparse
index revalidates with `If-None-Match`.
## `store/3`
Serves every write, not just uploads: the `ctx` carries the HTTP
method, so Cargo's `DELETE …/yank` and `PUT …/unyank` land here too.
The third argument is the already-extracted upload body — either a
`%Plug.Upload{}` (twine's multipart) or a raw binary (Cargo's
length-prefixed octet-stream).
## `content_type/1`
The media type to serve a stored artifact with, given its filename.
Clients rarely care, but `pip` and `cargo` both log it.
"""
alias GitGud.Repositories.Repository
@formats ~w(oci pypi hex cargo deb rpm fdroid raw)
# Format identifier → adapter module. `oci` is absent on purpose: zot
# is the authoritative store for container images and serves the
# distribution API itself, so `oci` exists only as a `format` value
# the packages UI can list alongside everything else. Each remaining
# format appears here as its adapter lands.
@default_adapters %{
"pypi" => GitGud.Packages.Formats.Pypi,
"hex" => GitGud.Packages.Formats.Hex,
"cargo" => GitGud.Packages.Formats.Cargo,
"deb" => GitGud.Packages.Formats.Deb,
"rpm" => GitGud.Packages.Formats.Rpm,
"fdroid" => GitGud.Packages.Formats.Fdroid,
"raw" => GitGud.Packages.Formats.Raw
}
@typedoc """
Request context handed to an adapter.
* `:path` — path segments below `/pkg/:format/:owner/:name`
* `:conn` — the live `Plug.Conn`, for header negotiation
* `:method` — `"GET"`, `"PUT"`, `"POST"`, `"DELETE"`
* `:trailing_slash?` — whether the raw request path ended in `/`
(Plug's segment list drops it, and PEP 503 makes it load-bearing)
* `:base_url` — absolute URL of this repository's registry root,
no trailing slash
* `:principal` — the authenticated principal, see
`GitGud.Packages.Auth`
"""
@type ctx :: %{
path: [String.t()],
conn: Plug.Conn.t(),
method: String.t(),
trailing_slash?: boolean(),
base_url: String.t(),
principal: term()
}
@typedoc """
What an adapter hands back from `index/2`.
* `{:ok, content_type, body}` — render it
* `{:ok, content_type, body, headers}` — …with extra response
headers (`ETag`, `Cache-Control`)
* `{:blob, package_file}` — serve the stored artifact
* `{:redirect, url_or_path}` — 301
* `:not_modified` — 304, the client's validator still holds
* `{:error, reason}` — mapped to a status by the controller
"""
@type index_result ::
{:ok, String.t(), iodata()}
| {:ok, String.t(), iodata(), [{String.t(), String.t()}]}
| {:blob, GitGud.Packages.PackageFile.t()}
| {:redirect, String.t()}
| :not_modified
| {:error, term()}
@typedoc """
What an adapter hands back from `store/3`. The body is the
format's own success document — Cargo wants a `warnings` object,
twine is happy with an empty 200.
"""
@type store_result ::
{:ok, String.t(), iodata()}
| {:reply, integer(), String.t(), iodata()}
| {:error, term()}
@callback index(Repository.t(), ctx()) :: index_result()
@callback store(Repository.t(), ctx(), term()) :: store_result()
@callback content_type(String.t()) :: String.t()
@doc """
The ecosystem's canonical form of a package name. Optional — a
format that doesn't specify one gets case folding.
"""
@callback normalize(String.t()) :: String.t()
@doc """
Serve a namespace-wide index at `/pkg/:format/:owner/#{"_all"}/*rest`,
aggregating every repository in the namespace the caller may read.
Optional. It earns its place where the namespace is the unit a
consumer actually configures — apt, where five repositories mean
five `deb` lines and five keyring entries for what an administrator
thinks of as one publisher. A format without it 404s there, which is
the honest answer.
`repos` is already filtered by the caller's read access, so an
adapter renders what it is given and makes no visibility decisions
of its own.
"""
@callback namespace_index(namespace :: struct(), repos :: [Repository.t()], ctx()) ::
index_result()
@optional_callbacks normalize: 1, namespace_index: 3
@doc """
The repository-slot marker that means "the whole namespace".
It can never collide with a repository: `Repository`'s name format
requires a leading alphanumeric.
"""
def namespace_marker, do: "_all"
@summary_header "x-package-summary"
@description_header "x-package-description"
@doc """
What a publish request says the package is, from its headers.
* `X-Package-Summary` — one line
* `X-Package-Description` — percent-encoded, because a header can't
carry the newlines a description has. `jq -rR @uri` does it.
A value that doesn't decode is taken as written.
Returns only the keys that were sent, for `Packages.publish_file/5`'s
`:about`. Adapters whose artifacts describe themselves have no reason
to call this; the ones that carry nothing (an APK, a raw file) do.
"""
@spec about(ctx() | map()) :: %{optional(String.t()) => String.t()}
def about(%{conn: %Plug.Conn{} = conn}) do
[
{"summary", Plug.Conn.get_req_header(conn, @summary_header)},
{"description",
conn |> Plug.Conn.get_req_header(@description_header) |> Enum.map(&percent_decode/1)}
]
|> Enum.flat_map(fn
{key, [value | _]} -> [{key, value}]
{_key, []} -> []
end)
|> Map.new()
end
def about(_ctx), do: %{}
defp percent_decode(value) do
URI.decode(value)
rescue
ArgumentError -> value
end
@doc "Format identifiers this forge knows about."
def formats, do: @formats
@doc """
The live adapter table. Defaults to every format this build knows
how to serve; a deployment that only wants, say, Hex can narrow it:
config :git_gud, :package_formats, %{"hex" => GitGud.Packages.Formats.Hex}
A format that isn't in the table 404s, which is a better answer than
serving an index nobody meant to publish.
"""
def adapters, do: Application.get_env(:git_gud, :package_formats, @default_adapters)
@doc "Resolve a format identifier to its adapter module."
def adapter(format) when is_binary(format) do
case Map.fetch(adapters(), format) do
{:ok, mod} -> {:ok, mod}
:error -> :error
end
end
def adapter(_), do: :error
@doc """
Canonical form of a package name for `format`, used for every
lookup. Each ecosystem specifies its own; where one doesn't, case
folding is the safe floor.
"""
def normalize(format, name) do
case adapter(format) do
{:ok, mod} ->
if Code.ensure_loaded?(mod) and function_exported?(mod, :normalize, 1),
do: mod.normalize(name),
else: String.downcase(name)
:error ->
String.downcase(name)
end
end
@doc """
Media type to serve `filename` with, asked of `format`'s adapter.
Falls back to `application/octet-stream`, which is what every client
here treats an unknown artifact as anyway.
"""
def adapter_content_type(format, filename) do
case adapter(format) do
{:ok, mod} -> mod.content_type(filename)
:error -> "application/octet-stream"
end
end
end
neiam
/gitgud
Git Gud
public · Issues · Pulls · Labels · Forks · Compare · Actions success · Badges · Packages
⭐
1
Log in to mark this repository.
8.2 KiB · text
History
a625083