defmodule GitGud.Packages.Formats.Fdroid do
@moduledoc """
An F-Droid app repository, per git repository *and* per namespace.
# everything neiam publishes — one repo entry on the phone
https://<host>/pkg/fdroid/neiam/_all/repo
# or one repository on its own
https://<host>/pkg/fdroid/neiam/gitgud/repo
The trailing `/repo` is required, not stylistic — see `repo_url/1`.
Like the apt adapter, this is static documents over HTTP with one
signature on top. `_all` is the shape that matters: a phone should
get one repository for a publisher, not one per Android project, and
`Format.namespace_marker/0` exists for exactly this.
Paths below either root:
/entry.jar signed; what a v2 client fetches first
/entry.json the same content, unsigned — for debugging
/index-v2.json the index proper, verified by hash from entry
/index-v1.jar signed; the whole index, for older clients
/index-v1.json the same, unsigned
/icons/icon.png the namespace avatar
/icons/<pkg>.<code>.png an app's launcher icon
/<package>_<code>.apk
## Both index versions, because the failure is invisible
A client that can't speak v2 falls back to `index-v1.jar` *silently*,
after fetching a perfectly good `entry.jar` — and reports the 404 it
gets. So a repository with a flawless v2 index reads as "invalid
repository / NotFoundException", which points at nothing in
particular. Serving both is the only way that failure doesn't happen.
## Where the fields come from
An APK carries no metadata file, so everything is read out of the
binary at publish time by `Fdroid.Apk` — package name, version code
and name, SDK levels, permissions, ABIs, and the signing
certificate's SHA-256. See `docs/fdroid-registry.md` for the formats.
Per-app `name` and the launcher icon come from `resources.arsc`, via
`Fdroid.Arsc`: `android:label` and `android:icon` in the manifest are
resource ids, and the table is what maps them to a string and a file.
A release build makes that the only route — R8 rewrites every
resource path, so this forge's own app ships `res/o-.png` and no
`mipmap` directory to guess from. Where the table can't be read the
index falls back to the package name and no icon.
`summary`, `description` and the Links section come from the *forge*,
not the APK, which carries none of it. `summary` and `description`
are the package's own (`Package.about_changeset/2`) — set on the
packages page, or sent with the upload as `X-Package-Summary` and
`X-Package-Description` (see `Format.about/1`). Unset, the
repository's description stands in for both. The repository's page is `sourceCode`, its issues are
`issueTracker`, its default branch's log is `changelog`, and its
namespace is `authorName` / `authorWebSite`. Both index versions
spell those fields identically and neither localizes them, unlike
`name`, `summary` and `description`.
`license` is the SPDX id detected from the repository's licence file
(`GitGud.Repositories.License`), which v2 omits when detection
couldn't tell and v1 spells `Unknown`, having no way to say it.
`webSite` is deliberately left unset: on a forge it would be the
repository page again, and F-Droid would show the same link twice
under two labels.
The *repository* icon is the namespace's avatar, via `GitGud.Avatars`
— an uploaded one if the org has set one, and the identicon the
profile page draws otherwise. Seeded on the namespace rather than the
repository: a repo under `neiam` shows neiam, which is what `_all`
shows anyway.
## signer, and why it is the load-bearing field
F-Droid records the SHA-256 of each APK's signing certificate and
refuses an update whose signer differs from what is installed. It is
the thing standing between a repository and someone replacing an app
on a phone. It is read from the APK rather than asserted.
## Timestamps
Milliseconds since the epoch, which is what F-Droid's index uses
throughout — seconds would be silently interpreted as 1970 and every
client would treat the repository as permanently stale.
"""
@behaviour GitGud.Packages.Format
alias GitGud.Packages
alias GitGud.Packages.Formats.Fdroid.Apk
alias GitGud.Avatars
alias GitGud.Packages.Formats.Fdroid.Jar
alias GitGud.Packages.PackageFile
alias GitGud.Repositories.Repository
alias GitGud.Repositories.Storage
@format "fdroid"
# The index-v2 format version F-Droid's own tooling emits.
@index_version 20_002
# index-v1's own version field. Unrelated to the above beyond looking
# similar — this one is the format the older client understands.
@index_v1_version 20_001
@impl true
def content_type(_filename), do: "application/vnd.android.package-archive"
# ── reads ────────────────────────────────────────────────────────────
@impl true
def index(repo, ctx), do: serve([repo], repo_label(repo), repo, strip_repo_prefix(ctx))
@impl true
def namespace_index(namespace, repos, ctx),
do: serve(repos, Storage.owner_handle(namespace), namespace, strip_repo_prefix(ctx))
@doc """
Drop the `repo/` or `fdroid/repo/` the client adds for itself.
F-Droid rewrites the URL you give it. `RepoUriGetter` appends
`/fdroid/repo` to any address that doesn't already end in `/repo` or
`/archive` — so `…/pkg/fdroid/neiam/_all` is fetched as
`…/pkg/fdroid/neiam/_all/fdroid/repo/entry.jar`, which 404s, which
the client reports as "invalid repository".
So `repo_url/1` advertises an address ending in `/repo`, which the
client leaves alone. This accepts both anyway: anyone who added the
bare URL before has the rewritten form stored as their repository
address and would otherwise never recover.
"""
def strip_repo_prefix(%{path: ["fdroid", "repo" | rest]} = ctx), do: %{ctx | path: rest}
def strip_repo_prefix(%{path: ["repo" | rest]} = ctx), do: %{ctx | path: rest}
def strip_repo_prefix(ctx), do: ctx
@doc """
The address to hand a client, which is `base_url` plus `/repo`.
The suffix is not decoration. Without it F-Droid rewrites the address
and fetches a path that doesn't exist; with it, `pathSegments.last()
== "repo"` and the client uses what it was given.
"""
@spec repo_url(String.t()) :: String.t()
def repo_url(base_url), do: String.trim_trailing(base_url, "/") <> "/repo"
defp serve(repos, label, scope, ctx)
defp serve(repos, label, scope, %{path: ["index-v2.json"]} = ctx) do
{:ok, "application/json", index_v2(repos, label, ctx, scope)}
end
# The unsigned entry, which is what `curl` wants and what no client
# will accept. Kept because "is the index right" and "is the
# signature right" are different questions and debugging them
# together is miserable.
defp serve(repos, label, scope, %{path: ["entry.json"]} = ctx) do
{:ok, "application/json", entry(repos, label, ctx, scope)}
end
defp serve(repos, label, scope, %{path: ["entry.jar"]} = ctx) do
signed_entry(repos, label, scope, ctx)
end
# index-v1, for clients that don't speak v2.
#
# Not optional in practice. A client that can't find this falls back
# to it *silently* after fetching a perfectly good entry.jar, and
# reports the 404 — so a repository with a flawless v2 index reads as
# "invalid repository / NotFoundException", which points at nothing.
defp serve(repos, label, scope, %{path: ["index-v1.jar"]} = ctx) do
document = index_v1(repos, label, ctx, scope) |> IO.iodata_to_binary()
# SHA-256, not the SHA-1 that fdroidserver historically used here.
#
# A deliberate divergence, and measured: a SHA-1-signed JAR handed
# to a current verifier comes back "treated as unsigned, because it
# is signed with a weak algorithm that is now disabled". On a device
# that is a far worse failure than the 404 this path exists to fix —
# an index that looks present and unsigned rather than absent.
#
# SHA-1 was never a weakness for an index (second-preimage
# resistance is the property that matters), but the platforms have
# stopped caring about that distinction, so this follows them.
case Jar.build(scope, [{"index-v1.json", document}], digest: :sha256) do
{:ok, jar, _fingerprint} -> {:ok, "application/java-archive", jar}
{:error, reason} -> {:error, {500, "could not sign the index: #{inspect(reason)}"}}
end
end
# The unsigned form, for the same reason entry.json is served.
defp serve(repos, label, scope, %{path: ["index-v1.json"]} = ctx),
do: {:ok, "application/json", index_v1(repos, label, ctx, scope)}
# The repository icon: the namespace's identicon, the same grid the
# profile page draws, baked into a PNG because a client has no
# stylesheet to resolve the web avatar's CSS variables against.
defp serve(repos, _label, scope, %{path: ["icons", filename]}) do
{content_type, body} = icon(scope)
if filename == icon_filename(content_type) do
{:ok, content_type, body}
else
case find_app_icon(repos, filename) do
{app_type, app_icon} -> {:ok, app_type, app_icon}
nil -> {:error, {404, "no such icon"}}
end
end
end
# An APK, by the filename the index points at.
defp serve(repos, _label, _scope, %{path: [filename]}) do
case Enum.find(apks(repos), &(&1.file.filename == filename)) do
%{file: %PackageFile{} = file} -> {:blob, file}
nil -> {:error, {404, "no such file in this repository"}}
end
end
defp serve(_repos, _label, _scope, %{path: []}),
do: {:error, {404, "add this URL as a repository in F-Droid; there is no index page"}}
defp serve(_repos, _label, _scope, _ctx),
do: {:error, {404, "not found in this repository"}}
# ── index documents ──────────────────────────────────────────────────
@doc false
# index-v2: a repo block, then every package keyed by package name,
# each with its versions keyed by the APK's sha256.
def index_v2(repos, label, ctx), do: index_v2(repos, label, ctx, nil)
@doc false
def index_v2(repos, label, ctx, scope) do
apks = apks(repos)
packages = Enum.group_by(apks, & &1.info["package"])
Jason.encode_to_iodata!(%{
"repo" => %{
"name" => %{"en-US" => label},
"description" => %{"en-US" => description(repos, label)},
"address" => repo_url(ctx.base_url),
"timestamp" => repo_timestamp(apks),
"icon" => %{"en-US" => icon_file(scope)},
"categories" => %{},
"mirrors" => []
},
"packages" => Map.new(packages, fn {name, group} -> {name, package_entry(name, group)} end)
})
end
# index-v2 records every file by hash and size, the icon included, so
# a client can tell a changed icon from an unchanged one without
# fetching it. Deterministic rendering is what makes that work.
defp icon_file(scope) do
{content_type, body} = icon(scope)
%{
"name" => "/icons/" <> icon_filename(content_type),
"sha256" => :sha256 |> :crypto.hash(body) |> Base.encode16(case: :lower),
"size" => byte_size(body)
}
end
@doc """
Re-read published APKs and record the app name and launcher icon that
the index shows.
Both are read at upload and stored with the file, so an app published
before the adapter could resolve `resources.arsc` carries neither and
nothing re-reads it afterwards: the entry shows a package id and no
icon until the app publishes again. For an app that isn't building
today, that is never.
Options:
* `:all` — every version rather than the newest of each app. The
index takes the name and icon from the highest versionCode, so
the default reads one APK per app instead of pulling twenty out
of storage to change nothing anyone can see.
* `:dry_run` — report without writing.
* `:on_file` — called with a report map per APK read.
Adds a label and an icon and nothing else: everything else the file
records was written when it was published, and a re-read is the wrong
place to discover a disagreement about, say, a versionCode that the
index orders by and the filename encodes.
Returns `{read, written}`. Idempotent — an icon already in storage is
stored under the same content-addressed key.
"""
@spec backfill_icons(keyword()) :: {non_neg_integer(), non_neg_integer()}
def backfill_icons(opts \\ []) do
all? = Keyword.get(opts, :all, false)
dry? = Keyword.get(opts, :dry_run, false)
on_file = Keyword.get(opts, :on_file, fn _ -> :ok end)
backfill_candidates(all?)
|> Enum.reduce({0, 0}, fn file, {read, written} ->
case backfill_file(file, dry?) do
{:ok, report} ->
on_file.(report)
{read + 1, if(dry?, do: written, else: written + 1)}
{:error, report} ->
on_file.(report)
{read + 1, written}
end
end)
end
# Every published APK, or the newest of each app. "Newest" is by
# versionCode, which is what the index sorts on; a file whose metadata
# predates this adapter has none and sorts last rather than breaking
# the comparison.
defp backfill_candidates(true), do: published_apk_files()
defp backfill_candidates(false) do
published_apk_files()
|> Enum.group_by(& &1.package_version.package_id)
|> Enum.map(fn {_package_id, files} -> Enum.max_by(files, &backfill_version_code/1) end)
|> Enum.sort_by(& &1.id)
end
defp published_apk_files do
import Ecto.Query
from(f in PackageFile,
join: v in assoc(f, :package_version),
join: p in assoc(v, :package),
where: p.format == ^@format,
preload: [package_version: {v, package: p}]
)
|> GitGud.Repo.all()
end
defp backfill_version_code(%PackageFile{metadata: %{"version_code" => code}})
when is_integer(code),
do: code
defp backfill_version_code(%PackageFile{}), do: -1
# Only these come from the re-read.
@backfilled ~w(label icon_key icon_sha256 icon_size icon_ext)
defp backfill_file(%PackageFile{} = file, dry?) do
with {:ok, body} <- Packages.read_file(file),
{:ok, info} <- Apk.read(body) do
icon =
case Apk.icon(body) do
{:ok, bytes, ext} -> {byte_size(bytes), ext}
:none -> :none
end
report = %{
package: info.package,
filename: file.filename,
label: info.label,
icon: icon,
written?: not dry?
}
unless dry? do
additions = info |> apk_metadata(body) |> Map.take(@backfilled)
file
|> Ecto.Changeset.change(metadata: Map.merge(file.metadata || %{}, additions))
|> GitGud.Repo.update!()
# The version carries the same map at publish time; kept in step
# so the two don't disagree about what an APK holds.
file.package_version
|> Ecto.Changeset.change(
metadata: Map.merge(file.package_version.metadata || %{}, additions)
)
|> GitGud.Repo.update!()
end
{:ok, report}
else
{:error, reason} ->
{:error, %{filename: file.filename, error: reason}}
end
end
@doc """
Everything the index knows about an APK: what the manifest and
resource table say, plus the launcher icon, stored.
Public because `mix gitgud.backfill_fdroid_icons` re-reads APKs that
were published before the adapter could resolve an icon, and a
backfill that computed metadata its own way would be a second
implementation to keep in step.
"""
@spec apk_metadata(map(), binary()) :: map()
def apk_metadata(info, body) when is_map(info) and is_binary(body) do
info |> string_keys() |> Map.merge(store_icon(body))
end
# The launcher icon pulled out of the APK at publish time, keyed by
# content so re-publishing the same image stores it once.
defp store_icon(body) do
with {:ok, bytes, ext} <- Apk.icon(body),
{:ok, meta} <- Packages.store_icon(bytes, ext, "fdroid-icons") do
meta
else
# An icon is decoration. Failing the publish over one would
# trade a working release for a missing picture.
_ -> %{}
end
end
defp icon_content_type("webp"), do: "image/webp"
defp icon_content_type("jpg"), do: "image/jpeg"
defp icon_content_type(_), do: "image/png"
# Versions published before icons carried their format are PNG: that
# is what the old extractor could produce.
defp icon_ext(info), do: info["icon_ext"] || "png"
# `<package>.<versionCode>.png`, which is what fdroidserver emits and
# what a v1 client expects to find under /icons/ — with whatever
# extension the APK actually carried, since AGP has emitted WebP by
# default for years and a client renders what the name claims.
defp app_icon_name(%{info: info}) do
if info["icon_key"], do: "#{info["package"]}.#{info["version_code"]}.#{icon_ext(info)}"
end
# The Links section of an app's page: where its source is, where its
# bugs go, who publishes it. F-Droid has fields for all of it and an
# APK carries none of them — but the forge knows, because the app is
# published from a repository on it.
#
# Both index versions spell these the same way and neither localizes
# them, unlike `name`, `summary` and `description`.
#
#
# Taken from the newest APK's repository: an app is published from
# one, and `_all` aggregates a namespace where in principle two could
# publish the same package name. The version a phone would install is
# the one whose links belong on the page.
defp links(group) do
repo = Enum.max_by(group, & &1.info["version_code"]).repo
url = repo_page_url(repo)
%{
"sourceCode" => url,
"issueTracker" => url <> "/issues",
# Not a changelog file — the commit log of the default branch,
# which is the closest honest thing a forge has and more than the
# nothing that was there before.
"changelog" => changelog_url(repo, url),
"authorName" => Storage.repo_handle(repo),
"authorWebSite" => namespace_url(repo)
}
end
# The SPDX id `GitGud.Repositories.License` detected for the
# repository this app is published from, or nil when it couldn't tell.
defp license(group), do: Enum.max_by(group, & &1.info["version_code"]).repo.license
defp repo_page_url(%Repository{} = repo),
do: "#{forge_url()}/r/#{Storage.repo_handle(repo)}/#{repo.name}"
# Users and organizations have different profile routes, and a
# namespace is one or the other.
defp namespace_url(%Repository{organization: %{handle: handle}}),
do: "#{forge_url()}/orgs/#{handle}"
defp namespace_url(%Repository{} = repo),
do: "#{forge_url()}/u/#{Storage.repo_handle(repo)}"
defp changelog_url(%Repository{default_branch: branch}, url)
when is_binary(branch) and branch != "",
do: url <> "/commits/" <> branch
defp changelog_url(_repo, _url), do: nil
defp forge_url, do: String.trim_trailing(GitGudWeb.Endpoint.url(), "/")
# What the publisher said about the package, and failing that the
# repository's description: it is what its author wrote, and an APK
# carries nothing of the kind. The newest APK's package, like the
# links — under `_all` two repositories could publish one name.
defp full_description(group) do
newest = Enum.max_by(group, & &1.info["version_code"])
present(newest.package.description) ||
case newest.repo do
%Repository{description: description} -> present(description)
_ -> nil
end
end
defp present(text) when is_binary(text), do: if(String.trim(text) == "", do: nil, else: text)
defp present(_), do: nil
# F-Droid shows the summary under the app's name in a list, where a
# paragraph doesn't fit. An explicit one as written; otherwise the
# description's first line, or a word-boundary cut of it.
#
# Dropped when it would only repeat the name — a repository described
# as "GitGud" publishing an app labelled "GitGud" — because the line
# under the name saying the name again is worse than no line.
@summary_length GitGud.Packages.Package.summary_max()
defp summary(name, group) do
newest = Enum.max_by(group, & &1.info["version_code"])
summary =
present(newest.package.summary) ||
case full_description(group) do
nil -> nil
description -> description |> String.split("\n", parts: 2) |> hd() |> shorten()
end
if summary && same_text?(summary, app_label(name, group)), do: nil, else: summary
end
defp same_text?(a, b) do
normalize = fn text ->
text |> String.trim() |> String.trim_trailing(".") |> String.downcase()
end
normalize.(a) == normalize.(b)
end
defp shorten(text) do
text = String.trim(text)
if String.length(text) <= @summary_length do
text
else
text
|> String.slice(0, @summary_length)
|> String.split(" ")
|> Enum.drop(-1)
|> Enum.join(" ")
|> Kernel.<>("…")
end
end
# v2 localizes the human-readable fields and nothing else. A nil stays
# nil so `drop_nils/1` can take the key out entirely — an empty
# `{"en-US" => null}` is a key a client has to interpret.
defp localized(nil), do: nil
defp localized(text), do: %{"en-US" => text}
# `android:label` off the newest APK, resolved through its resource
# table at publish time. Falls back to the package name, which is what
# every version published before labels were read still carries.
defp app_label(name, group) do
newest = Enum.max_by(group, & &1.info["version_code"])
case newest.info["label"] do
label when is_binary(label) -> if String.trim(label) == "", do: name, else: label
_ -> name
end
end
defp app_icon_file(group) do
newest = Enum.max_by(group, & &1.info["version_code"])
info = newest.info
if info["icon_key"] do
%{
"en-US" => %{
"name" => "/icons/" <> app_icon_name(newest),
"sha256" => info["icon_sha256"],
"size" => info["icon_size"]
}
}
end
end
defp find_app_icon(repos, filename) do
Enum.find_value(apks(repos), fn apk ->
if apk.info["icon_key"] && app_icon_name(apk) == filename do
case GitGud.Storage.get(Packages.artifact_bucket(), apk.info["icon_key"]) do
{:ok, bytes} -> {icon_content_type(icon_ext(apk.info)), bytes}
{:error, _} -> nil
end
end
end)
end
# Whatever the namespace's avatar is — an upload if there is one,
# the identicon otherwise. `Avatars.render/1` makes that one call
# rather than a branch here.
defp icon(scope),
do: scope |> icon_owner() |> Avatars.render() |> then(fn {:ok, t, b} -> {t, b} end)
defp icon_owner(nil), do: %GitGud.Accounts.User{handle: "gitgud"}
defp icon_owner(scope), do: GitGud.Packages.Signing.namespace(scope)
# The index names the icon by filename, so the extension has to match
# what is actually served — a JPEG at `icon.png` is a file every
# client will fetch and some will refuse to draw.
defp icon_filename("image/jpeg"), do: "icon.jpg"
defp icon_filename("image/gif"), do: "icon.gif"
defp icon_filename("image/webp"), do: "icon.webp"
defp icon_filename(_png), do: "icon.png"
defp package_entry(name, group) do
added = group |> Enum.map(& &1.added) |> Enum.min()
updated = group |> Enum.map(& &1.added) |> Enum.max()
%{
"metadata" =>
%{
"added" => added,
"lastUpdated" => updated,
"name" => %{"en-US" => app_label(name, group)},
# Dropped when detection couldn't tell, which is the
# difference between "we don't know" and claiming a licence
# the repository doesn't state.
"license" => license(group),
"summary" => localized(summary(name, group)),
"description" => localized(full_description(group)),
"icon" => app_icon_file(group)
}
|> Map.merge(links(group))
|> drop_nils(),
"versions" => Map.new(group, fn apk -> {apk.file.sha256, version_entry(apk)} end)
}
end
defp version_entry(%{file: file, info: info, added: added}) do
%{
"added" => added,
"file" => %{
# Leading slash: F-Droid resolves it against the repo address.
"name" => "/" <> file.filename,
"sha256" => file.sha256,
"size" => file.size
},
"manifest" =>
%{
"versionCode" => info["version_code"],
"versionName" => info["version_name"],
"usesSdk" => %{
"minSdkVersion" => info["min_sdk"],
"targetSdkVersion" => info["target_sdk"]
},
# A list because an APK signed by a rotated key chain has
# several; we read the current one, so it is a list of one.
"signer" => %{"sha256" => [info["signer"]]},
"usesPermission" =>
Enum.map(info["permissions"] || [], &%{"name" => &1, "maxSdkVersion" => nil}),
"nativecode" => info["nativecode"] || []
}
|> drop_nils()
}
end
@doc false
# index-v1: one flat document rather than v2's entry-plus-index. The
# shapes differ enough that translating between them would be more
# code than emitting both from the same rows.
def index_v1(repos, label, ctx), do: index_v1(repos, label, ctx, nil)
@doc false
def index_v1(repos, label, ctx, scope) do
apks = apks(repos)
by_package = Enum.group_by(apks, & &1.info["package"])
Jason.encode_to_iodata!(%{
"repo" => %{
"timestamp" => repo_timestamp(apks),
"version" => @index_v1_version,
"name" => label,
"icon" => icon_filename(elem(icon(scope), 0)),
"address" => repo_url(ctx.base_url),
"description" => description(repos, label)
},
# Present and empty. The client reads them unconditionally.
"requests" => %{"install" => [], "uninstall" => []},
"apps" => Enum.map(by_package, fn {name, group} -> v1_app(name, group) end),
"packages" =>
Map.new(by_package, fn {name, group} ->
# Newest first: the client takes the head as the suggested
# version, so the order is load-bearing rather than cosmetic.
{name,
group
|> Enum.sort_by(& &1.info["version_code"], :desc)
|> Enum.map(&v1_package/1)}
end)
})
end
defp v1_app(name, group) do
added = group |> Enum.map(& &1.added) |> Enum.min()
updated = group |> Enum.map(& &1.added) |> Enum.max()
newest = Enum.max_by(group, & &1.info["version_code"])
%{
"packageName" => name,
"name" => app_label(name, group),
"summary" => summary(name, group) || "",
"description" => full_description(group) || "",
# v1 has no notion of "we looked and couldn't tell", and a client
# renders whatever string it is given.
"license" => license(group) || "Unknown",
"categories" => [],
"added" => added,
"lastUpdated" => updated,
"suggestedVersionCode" => to_string(newest.info["version_code"]),
"suggestedVersionName" => newest.info["version_name"],
# v1 names the icon relative to <address>/icons/, so a bare
# filename rather than v2's path-and-hash object.
"icon" => app_icon_name(newest)
}
|> Map.merge(links(group))
|> drop_nils()
end
defp v1_package(%{file: file, info: info, added: added}) do
%{
"apkName" => file.filename,
"hash" => file.sha256,
"hashType" => "sha256",
"versionCode" => info["version_code"],
"versionName" => info["version_name"],
"size" => file.size,
"minSdkVersion" => info["min_sdk"],
"targetSdkVersion" => info["target_sdk"],
# The certificate again. v1 spells it `signer`, and a client
# refuses an update whose signer differs from what's installed.
"signer" => info["signer"],
"added" => added,
"nativecode" => info["nativecode"] || [],
# `[name, maxSdkVersion]` pairs, which is the shape v1 uses —
# not v2's list of objects.
"uses-permission" => Enum.map(info["permissions"] || [], &[&1, nil])
}
|> drop_nils()
end
@doc false
# entry.json points at index-v2.json and pins it by hash, so the
# signature on entry.jar covers the index transitively without the
# index itself having to be wrapped in a JAR.
def entry(repos, label, ctx), do: entry(repos, label, ctx, nil)
@doc false
def entry(repos, label, ctx, scope) do
apks = apks(repos)
document = index_v2(repos, label, ctx, scope) |> IO.iodata_to_binary()
Jason.encode_to_iodata!(%{
"timestamp" => repo_timestamp(apks),
"version" => @index_version,
"index" => %{
"name" => "/index-v2.json",
"sha256" => :sha256 |> :crypto.hash(document) |> Base.encode16(case: :lower),
"size" => byte_size(document),
"numPackages" => apks |> Enum.map(& &1.info["package"]) |> Enum.uniq() |> length()
}
})
end
# The signed entry: entry.json wrapped in a JAR whose signature the
# client verifies against the fingerprint it pinned when the
# repository was added.
defp signed_entry(repos, label, scope, ctx) do
document = entry(repos, label, ctx, scope) |> IO.iodata_to_binary()
case Jar.build(scope, [{"entry.json", document}]) do
{:ok, jar, _fingerprint} ->
{:ok, "application/java-archive", jar}
{:error, reason} ->
{:error, {500, "could not sign the index: #{inspect(reason)}"}}
end
end
@doc """
The SHA-256 of this repository's signing certificate — the
`?fingerprint=` a client pins when adding the repository.
Generating the key if there isn't one yet, because the first person
to ask is usually the packages page trying to print the URL.
"""
@spec fingerprint(struct()) :: {:ok, String.t()} | {:error, term()}
def fingerprint(scope) do
with {:ok, der} <- Jar.certificate(scope), do: {:ok, Jar.fingerprint(der)}
end
# ── writes ───────────────────────────────────────────────────────────
@impl true
def store(repo, ctx, body)
def store(repo, %{method: "POST", path: path} = ctx, body)
when is_binary(body) and (path == ["upload"] or path == []) do
with {:ok, info} <- Apk.read(body) do
# Extracted once, here, rather than on every icon request — the
# alternative is pulling a 19 MB APK out of storage to serve
# 30 KB of PNG.
metadata = apk_metadata(info, body)
attrs = %{
filename: Apk.filename(info),
body: body,
content_type: content_type(nil),
metadata: metadata,
version_metadata: metadata,
uploader: uploader(ctx),
about: GitGud.Packages.Format.about(ctx)
}
case Packages.publish_file(
repo,
@format,
info.package,
version_label(info),
attrs
) do
{:ok, _package, _version, _file} ->
{:ok, "application/json",
Jason.encode_to_iodata!(%{
"package" => info.package,
"versionCode" => info.version_code,
"versionName" => info.version_name,
"signer" => info.signer,
"filename" => Apk.filename(info)
})}
# The generic "a file with that name is already published"
# doesn't point anywhere useful here, and this has exactly one
# cause worth naming: a versionCode that didn't move. Android
# refuses an update that doesn't increase it and F-Droid orders
# by it, so a repeated one is a bug in the build rather than a
# publishing accident.
{:error, :file_exists} ->
{:error,
{409,
"#{Apk.filename(info)} is already published with different content — " <>
"versionCode #{info.version_code} has been used before. Bump it: Android " <>
"won't install an update whose versionCode hasn't increased."}}
{:error, %Ecto.Changeset{} = changeset} ->
{:error, {400, about_errors(changeset)}}
{:error, reason} ->
{:error, {400, inspect(reason)}}
end
end
end
def store(_repo, %{method: "POST"}, _body),
do: {:error, {400, "POST the raw .apk to /upload"}}
def store(_repo, %{method: method}, _body),
do: {:error, {405, "#{method} is not supported by the F-Droid repository"}}
# ── internals ────────────────────────────────────────────────────────
# Every published APK across `repos`, with its decoded metadata.
defp apks(repos) do
Enum.flat_map(repos, fn %Repository{} = repo ->
# Preloaded once here rather than per app: the links an app's page
# shows are the repository's, and naming its namespace needs both
# associations.
repo = GitGud.Repo.preload(repo, [:owner, :organization])
repo
|> Packages.list_packages(@format)
|> Enum.flat_map(fn package ->
package |> Packages.list_package_files() |> Enum.map(&{package, &1})
end)
|> Enum.flat_map(fn {package, %PackageFile{} = file} ->
info = file.metadata || %{}
# A file whose metadata predates this adapter, or was written by
# something else, has no package name and cannot be indexed.
# Skipping beats emitting a null key that breaks the whole
# document for every other app in the repository.
if is_binary(info["package"]) do
[
%{
file: file,
package: package,
repo: repo,
repo_name: repo.name,
info: info,
added: added_at(file)
}
]
else
[]
end
end)
end)
|> Enum.sort_by(&{&1.info["package"], &1.info["version_code"]})
end
defp added_at(%PackageFile{inserted_at: %DateTime{} = at}),
do: DateTime.to_unix(at, :millisecond)
defp added_at(_), do: 0
@doc false
# When this repository last changed — the newest publish, not `now`.
#
# It has to be derived from content, for two reasons. `entry.json`
# pins `index-v2.json` by hash, so a document that differs per
# request could never match the hash a client was given, and every
# fetch would look like tampering. And F-Droid compares this against
# the timestamp it already holds to refuse rollbacks, which only
# means anything if it moves when the content does and not otherwise.
#
# An empty repository reports 0 rather than the current time: it has
# never been updated, and saying "just now" every time anyone asks is
# both false and a rollback the next real publish has to climb out of.
defp repo_timestamp([]), do: 0
defp repo_timestamp(apks), do: apks |> Enum.map(& &1.added) |> Enum.max()
# `versionName (versionCode)` — the version string a human reads in
# the packages UI. versionCode alone sorts correctly but tells you
# nothing; versionName alone is not unique.
defp version_label(%{version_name: nil, version_code: code}), do: to_string(code)
defp version_label(%{version_name: name, version_code: code}), do: "#{name} (#{code})"
# Preloaded here rather than assumed: the controller does preload,
# but a missing association should not be a 500 from an index.
defp repo_label(%Repository{} = repo) do
repo = GitGud.Repo.preload(repo, [:owner, :organization])
"#{Storage.repo_handle(repo)}/#{repo.name}"
end
defp description([%Repository{description: d}], _label) when is_binary(d) and d != "", do: d
defp description(_repos, label), do: "Android packages published by #{label}."
# A summary header that's too long is the likely one; say which header
# and what the limit is, rather than dumping a changeset.
defp about_errors(changeset) do
changeset
|> Ecto.Changeset.traverse_errors(fn {msg, opts} ->
Enum.reduce(opts, msg, fn {k, v}, acc ->
String.replace(acc, "%{#{k}}", to_string(v))
end)
end)
|> Enum.map_join("; ", fn {field, msgs} ->
"X-Package-#{String.capitalize(to_string(field))} #{Enum.join(msgs, ", ")}"
end)
end
defp string_keys(info), do: Map.new(info, fn {k, v} -> {to_string(k), stringify(v)} end)
# `signer_scheme` is an atom; everything else is already JSON-shaped.
defp stringify(value) when is_atom(value) and not is_boolean(value) and not is_nil(value),
do: Atom.to_string(value)
defp stringify(value), do: value
defp drop_nils(map), do: Map.reject(map, fn {_k, v} -> is_nil(v) end)
defp uploader(%{principal: {:user, user, _scopes}}), do: user
defp uploader(_), do: nil
end
neiam
/gitgud
Git Gud
public · Issues · Pulls · Labels · Forks · Compare · Actions success · Badges · Packages
⭐
1
Log in to mark this repository.
36.3 KiB · text
History
b868907