GITGUD

neiam neiam /gitgud

Git Gud

public · Issues · Pulls · Labels · Forks · Compare · Actions success · Badges · Packages

⭐ 1

Log in to mark this repository.

22.6 KiB · text History b2aad05

F-Droid app repository — plan

Serving an F-Droid repository out of the forge, the way apt already is, so that pkg/fdroid/<owner>/_all is one repo line in an F-Droid client and CI publishes into it the same way it publishes .debs today.

Spec details are quoted inline deliberately: they were the expensive part to establish, and an implementer shouldn’t have to re-derive them.

The framing

This is one more GitGud.Packages.Format adapter, not a subsystem. Slice 70 left more in place than it looks:

Need Already there
Route /pkg/:format/:owner/:name/*rest — dispatches fdroid unchanged
Namespace view Format.namespace_marker/0 (_all) + the optional namespace_index/3 callback, built for exactly this
Per-namespace signing key SigningKey, minted on first use, private half AES-GCM in Postgres via GitGud.Secrets
Public key endpoint /pkg/_keys/:format/:owner
Auth RegistryToken + HTTP Basic — which the F-Droid client speaks for private repos
Blobs, retention, pruning, disk usage, UI unchanged

namespace_index/3 already carries the rationale for _all:

It earns its place where the namespace is the unit a consumer actually configures — apt, where five repositories mean five deb lines and five keyring entries for what an administrator thinks of as one publisher.

An F-Droid repo is the same story. A phone should get one repo entry for neiam, not one per Android project.

What actually has to be built

Four pieces. Two are parsers, one is a signer, one is JSON assembly.

Phase A — reading an APK

The analogue of Formats.Deb.Control, and bigger, because an APK does not carry a text stanza.

AndroidManifest.xml is binary

An APK is a zip (:zip opens it), but the manifest inside is Android’s compiled resource format, not XML text. Layout:

ResChunk_header      type u16, headerSize u16, size u32
  RES_XML_TYPE              0x0003   the file itself
  RES_STRING_POOL_TYPE      0x0001
  RES_XML_RESOURCE_MAP_TYPE 0x0180
  RES_XML_START_ELEMENT     0x0102
  RES_XML_END_ELEMENT       0x0103

String pool header adds stringCount u32, styleCount u32, flags u32, stringsStart u32, stylesStart u32, then a stringCount-long u32 offset array, then the data.

  • flags &&& 0x100 (UTF8_FLAG) picks the encoding. Both appear in the wild — AAPT2 emits UTF-8 for most resources but the manifest of the APK this was written against is UTF-16.
  • UTF-16: u16 length, then that many UTF-16LE code units, then a u16 NUL. A length with the high bit set is a two-unit length.
  • UTF-8: u8 UTF-16 length, u8 byte length, bytes, u8 NUL — each length independently extensible by its own high bit.

A start element is ResXMLTree_node (lineNumber u32, comment u32) then ResXMLTree_attrExt:

ns u32, name u32, attributeStart u16, attributeSize u16,
attributeCount u16, idIndex u16, classIndex u16, styleIndex u16

then attributeCount × ResXMLTree_attribute:

ns u32, name u32, rawValue u32, size u16, res0 u8, dataType u8, data u32

ns/name/rawValue are string-pool indices, 0xFFFFFFFF for absent. Data types that matter here:

dataType Meaning
0x03 TYPE_STRING data is a string-pool index
0x10 TYPE_INT_DEC data is the integer
0x11 TYPE_INT_HEX ditto
0x12 TYPE_INT_BOOLEAN non-zero is true
0x01 TYPE_REFERENCE @0x… resource id — not resolvable without resources.arsc

Attributes are matched by name string, not by resource id. The resource map chunk maps pool index → resource id, and a fully correct reader would use it; the names are in the pool regardless, and matching them avoids carrying a resource-id table. The cost is that an APK built by a toolchain that strips names would read as empty — none do.

What gets pulled out:

Field From
packageName <manifest package>
versionCode <manifest android:versionCode>
versionName <manifest android:versionName>
minSdkVersion, targetSdkVersion <uses-sdk>
uses-permission <uses-permission android:name>
nativecode lib/<abi>/ entries in the zip, not the manifest

android:label and android:icon are usually TYPE_REFERENCE into resources.arsc, so resolving them needs an ARSC reader. That was deferred once as “pretty listings” and then turned out to be the whole listing: a release build shows no name but the package id and no icon at all, because R8’s resource shrinking renames every resource path. This repo’s own app ships res/0Z.png, res/o-.png and no mipmap directory, so the convention the adapter used first — highest-density res/mipmap-*/ic_launcher.png — can only ever work on a debug build.

An app’s other metadata isn’t in the APK at all. F-Droid’s Links section — source code, issue tracker, changelog, author — and the description are things a publisher writes down, and on a forge they are already written down: the repository is the source, its issues are the tracker, and its default branch’s log is the nearest honest changelog. The description is the package’s, not the repository’s: one tree can publish a phone app and a server, and they want different words. It is set on the packages page or sent with the upload (see Phase D), and the repository’s description only stands in when nothing was. A summary that would just repeat the app’s name is left out rather than shown under it. Both index versions spell these fields the same way (sourceCode, issueTracker, changelog, authorName, authorWebSite) and neither localizes them, while name, summary and description are {"en-US": …} maps in v2 and plain strings in v1. webSite is left unset on purpose: it would be the repository page a second time.

GitGud.Packages.Formats.Fdroid.Arsc reads the table: the global string pool, then one ResTable_type chunk per configuration, keyed 0xPPTTEEEE. An icon id resolves to several files — one per density, plus the adaptive-icon XML at anydpi — so file_paths/2 ranks them highest-density first with XML last, since a <adaptive-icon> is the best icon on a phone and an unservable one in an index.

zipalign, and why :zip can’t read an icon

:zip.unzip/2 raises function_clause on a release APK. zipalign pads local file headers so stored entries start on a 4-byte boundary, and the padding goes in the extra field as bare zero bytes — Erlang’s parser walks extra fields expecting {id, size} pairs and dies on the remainder (:zip.parse_extra(<<0>>)). Every uncompressed member is padded that way, which is every PNG. AndroidManifest.xml is deflated and needs no padding, which is why nothing hit this until the icon did. Fdroid.Zip reads one member: names and offsets from the central directory (which :zip.list_dir/1 reads fine), then the local header and the bytes behind it.

Why this is hand-written rather than rusty-axml

A Rustler NIF is landing anyway for Phase C, so an AXML crate would cost nothing extra in deployment. rusty-axml was the candidate (0.2.1, Apache-2.0, ~16k recent downloads, last release 2025-07). It was evaluated by running it against this repo’s own manifest rather than by reading its docs, the same way the sequoia backend was chosen:

<manifest> attrs = {"package": "org.neiam.gitgud.app",
                    "android:versionCode": "(type 0x10) 0x1",
                    "android:versionName": "0.1.0", ...}
<uses-sdk> attrs = {"android:minSdkVersion": "(type 0x10) 0x1a", ...}

It discards typed values. attributes() is a HashMap<String, String>, and parse_start_element stringifies an integer into "(type 0x10) 0x1" at parse time — the dataType is gone, and there is no typed accessor anywhere in the public API. Every number this adapter needs is an integer attribute, and F-Droid compares versionCode numerically to decide whether something is an update.

Recovering 1 from "(type 0x10) 0x1" means scraping a debug format: correct today, silently wrong the day that format! changes, and wrong in the direction of “the client stops seeing updates” rather than the direction of a crash.

Its other helpers (find_node_by_name matches android:name, not the element type; get_requested_permissions) are shaped for security analysis of components, not for indexing.

So the parser stays in Elixir: it returns version_code: 1 typed from dataType, it is tested against a real AAPT2 manifest including every truncation of it, and it adds no dependency to a code path that reads attacker-supplied input.

The signing certificate is not where you’d expect

F-Droid’s index carries signer — the SHA-256 of the APK’s signing certificate — and the client refuses an update whose signer differs from what’s installed. So it has to be extracted.

The obvious place is META-INF/*.RSA (v1 / JAR signing). It is not there. Checked against this repo’s own APK:

META-INF/*.RSA, MANIFEST.MF  → absent
APK Sig Block 42             → present, block id 0x7109871a (Scheme v2)

minSdk is 26, and AGP omits v1 signing above 24. So the cert comes from the APK Signing Block, which sits between the last zip entry and the central directory:

EOCD (PK\x05\x06) → central directory offset
 [cd_offset - 16, cd_offset)      magic "APK Sig Block 42"
 [cd_offset - 24, cd_offset - 16) u64 size-of-block (repeated at its start)
 block = u64 size ‖ { u64 len ‖ u32 id ‖ value }* ‖ u64 size ‖ magic

IDs: 0x7109871a is Scheme v2, 0xf05368c0 Scheme v3, 0x42726577 is apksigner’s padding block. A v2/v3 value is:

signers: len-prefixed sequence of
  signer: len-prefixed
    signed data:  len-prefixed
    signatures:   len-prefixed
    public key:   len-prefixed
  where signed data = digests ‖ certificates ‖ additional attributes
  and certificates = len-prefixed sequence of len-prefixed X.509 DER

Every length is a little-endian u32. The first certificate of the first signer is the signing cert; sha256 of its DER is signer.

Both paths get implemented — v1 for anything with minSdk < 24, the signing block for everything else — and v1 is checked second, because an APK carrying both is signed by the same key either way and the block is authoritative.

Verification is out of scope. The forge reads the certificate; it does not verify the signature over the APK. The client does that, and Android does it again at install. Re-implementing v2 digest verification would be a third format for no added guarantee.

Phase B — the index documents

Mechanical assembly from PackageVersion rows, the same shape as the Deb adapter’s release_document / packages_document.

Serve index-v2, plus index-v1 for older clients:

/pkg/fdroid/<owner>/_all/entry.jar          signed; points at index-v2
/pkg/fdroid/<owner>/_all/index-v2.json      verified by hash from entry
/pkg/fdroid/<owner>/_all/index-v1.jar       signed; legacy clients
/pkg/fdroid/<owner>/_all/<apk filename>     the artifact

Per F-Droid’s Security Model: index-v1 signs with SHA1withRSA, index-v2 with SHA256withRSA, and index-v2 accepts “any algorithm supported by apksigner and android-23 and newer”. SHA-1 there is not a weakness — the relevant property is second-preimage resistance — but new work signs v2 and only emits v1 for reach.

Phase C — signing the index

The one genuine unknown, and the same shape of gap OpenPGP was for apt.

A signed JAR is a zip containing:

META-INF/MANIFEST.MF   per-entry SHA-256 digests
META-INF/CERT.SF       digest of the manifest, and of each entry's stanza
META-INF/CERT.RSA      PKCS#7 / CMS detached SignedData over CERT.SF

:public_key speaks X.509 and PKCS but not CMS SignedData. That is precisely the hole native/gitgud_apt was built to fill for OpenPGP.

Route taken: neither. The NIF turned out to be unnecessary once OpenBao landed, and Formats.Fdroid.Pkcs7 emits the DER directly.

Two things decided it. The key is not ours to hold — transit means a NIF could not simply sign; it would hand bytes back to be signed elsewhere and be called again to assemble, splitting the work exactly where the hard part was already done. And a JAR signature needs no signedAttrs: jarsigner has always also emitted the form where the signature covers the signed content directly, which here is the bytes of CERT.SF. Java’s verifier accepts both, and the simpler form is one call to Signing.sign/4 over a buffer we already have.

What was left is a fixed DER structure. No build dependency, no system library, nothing added to either image stage — cheaper still than the sequoia backend accepted for apt.

jarsigner -verify is the test that matters. Hand-written ASN.1 fails by looking correct, so checking our own arithmetic proves nothing. It caught three real bugs: subject must precede subjectPublicKeyInfo in a TBSCertificate; the certificate must be deterministic, since a random serial and a now validity window mint a different fingerprint on every call and the fingerprint is what users pin; and a repository scope has to resolve to its namespace, or /acme/tools and /acme/_all would be signed by different keys.

The repository’s self-signed certificate is built in Formats.Fdroid.Jar and signed through the same backend, so under OpenBao even the certificate’s own signature comes from transit. Serial and validity are derived rather than generated, because its SHA-256 is the repository fingerprint and that must never move.

Sign at publish, not at read

The Deb adapter signs InRelease on every request. At NIF speed that is free. It would not be at any other speed, and F-Droid clients poll indexes.

So fdroid materialises entry.jar and index-v1.jar into GitGud.Storage when an APK is published and serves them as blobs — {:blob, package_file} is already an index_result variant, so the adapter contract needs nothing new. Signing then runs once per publish instead of once per poll.

This also keeps the fallback open (below) without a rewrite.

If this had gone wrong

jarsigner/apksigner in their own container, the way zot is the fallback-shaped dependency for OCI and aptly was named for apt. It works, and sign-at-publish makes the latency irrelevant — but it is a worse trade here than aptly was for apt:

  • aptly replaces an entire repository format.
  • A signer sidecar replaces only the CMS step. Phases A, B and the zip assembly stay.

And it costs a key-custody split: the private key lives encrypted in Postgres today, and a JVM wants a keystore file. If it happens, the key stays in Postgres and is handed over loopback per request, never persisted in the container, which must have no ingress — it will sign whatever it is given.

Any shim written for this is a small Rust daemon, not a JVM wrapper. The JVM is the thing being avoided; adding one to escape 150 lines of Rust is the wrong direction.

Phase D — publishing from CI

The existing deb job is the template, down to the verify step:

- name: Publish the APK
  run: |
    curl --fail-with-body -sS \
      -H "Authorization: Bearer ${{ github.token }}" \
      -H "Content-Type: application/vnd.android.package-archive" \
      --upload-file android/app/build/outputs/apk/release/app-release.apk \
      "$FORGE_URL/pkg/fdroid/$REPOSITORY/upload"

What F-Droid shows under the app’s name and on its page can ride along with the upload. X-Package-Summary is one line of at most 80 characters; X-Package-Description is percent-encoded, since a header can’t carry the newlines a description has. A header that isn’t sent leaves the stored value alone, so a job that never sets them never clears what someone typed on the packages page:

    curl --fail-with-body -sS \
      -H "X-Package-Summary: $summary" \
      -H "X-Package-Description: $(urlencode "$description")" \
      …

This repository’s own app does exactly that from android/DESCRIPTION.md — summary on the first line, a blank line, then the description — with a few lines of bash doing the encoding, so the runner needs no jq. See the android job in .forgejo/workflows/build.yml.

A summary that fails validation refuses the whole upload with a 400 naming the header, rather than publishing the APK without it.

Consuming it is one repo entry:

$FORGE_URL/pkg/fdroid/<owner>/_all/repo?fingerprint=<sha256 of the repo cert>

The trailing /repo is required. F-Droid rewrites the address you give it: RepoUriGetter appends /fdroid/repo to anything not already ending in /repo or /archive. Without it the client fetches …/_all/fdroid/repo/entry.jar, gets a 404, falls back to v1, gets another 404, and reports “invalid repository” — while every URL you try by hand returns 200. The adapter accepts both prefixes anyway, so a device that stored the rewritten address recovers on its own.

Confirmed against the client source (RepoUriGetter.getUri): the fingerprint query parameter is read, lowercased, and stripped from the stored address, and fdroidrepos:// / fdroid.link forms are normalised the same way. The packages page emits a QR alongside the URL, since F-Droid’s add-repository screen has a scanner and nobody types 64 hex characters correctly.

Prerequisite: a release keystore

android/README.md notes there isn’t one. A debug-signed APK can be published but not meaningfully consumed — and the signer fingerprint would be the shared Android debug key, which every developer has.

This is the APK signing key, distinct from the index signing key. Keep them apart:

Key Lives Signs
APK release keystore CI secrets (GitGud.Secrets) the app
Repo index key SigningKey, per namespace the index

Key custody — OpenBao

F-Droid’s docs say: “the repo signing keys should not ever be on a public server.” A forge that serves its own index cannot follow that literally. It can, however, stop holding the key.

GitGud.Packages.Signing is the seam: sign/4 takes bytes and returns a signature, and an implementation is free to never possess key material. Two backends:

Backend Key lives Used by
Signing.Local Postgres, AES-GCM via GitGud.Secrets everything, by default
Signing.OpenBao inside OpenBao transit, non-exportable fdroid, opt-in

For F-Droid the key is generated inside OpenBao’s transit engine with exportable: false and allow_plaintext_backup: false. No API returns it — not to the app, not to an operator. The forge sends bytes to /v1/transit/sign/gitgud-fdroid-<namespace> and gets a signature back.

The property that buys: compromising the forge yields the ability to sign for as long as the compromise lasts, not a key that outlives it, and every signature is in an audit log. That is what F-Droid’s guidance is actually after.

It is opt-in, and only fdroid opts in. OpenBao is a second stateful service to unseal and back up; an instance that doesn’t serve an F-Droid repository never contacts it, never needs it deployed, and fails no health check for its absence. Configuration is gated on OPENBAO_ADDR being set at all:

config :git_gud, GitGud.Packages.Signing,
  formats: %{"fdroid" => GitGud.Packages.Signing.OpenBao}

Deployment is a separate Argo Application (openbao/ in the deployment repo) rather than part of git-gud’s, so not deploying it is the default and requires no edit.

The transit call, exactly

POST /v1/transit/sign/<key>
{ "input": "<base64 of the bytes>",
  "hash_algorithm": "sha2-256",       // "sha1" for index-v1
  "signature_algorithm": "pkcs1v15",  // CMS wants v1.5, not PSS
  "prehashed": false }
→ { "data": { "signature": "vault:v1:<base64>" } }

Two details worth not rediscovering:

  • Send raw bytes, not a digest. hash_algorithm: "none" with prehashed: true is documented as yielding a PKCSv1_5_NoOID signature; CMS needs the DigestInfo-wrapped PKCSv1_5_DERnull form, which is what the default path produces.
  • The vault:v<n>: prefix is stripped. A CMS structure has nowhere to record a key version, so rotation needs key_version threaded through deliberately if it ever matters.

The APK key is a different key, and lives somewhere else

Key Signs Lives in Property
Index entry.jar OpenBao transit, non-exportable never leaves
APK the app Forgejo secrets; backed up in OpenBao KV reaches the build container

Keeping them apart is the point. An attacker with the index key can lie about which versions exist; they cannot forge an app, because Android verifies the APK signature at install against the certificate recorded in signer. The APK key is the one that actually matters, and it is also the one that cannot be protected as well — which is exactly why they should never be the same key.

Why the APK key isn’t in transit. apksigner has no external-signer hook — no PKCS#11, no callback — so the key must reach whatever process builds the APK. Transit’s guarantee is unavailable here in principle, not by omission.

Why CI doesn’t fetch it from OpenBao either. It could: KV would give central custody, audit and revocation. But the runner is outside the cluster and OpenBao has no ingress by design, so reaching it would mean an ingress path plus a second auth mechanism — for a secret whose exposure at the moment of use is identical either way. OpenBao KV holds the durable backup, reached by a human with kubectl; the Forgejo secret is the operational copy. Revisit if more secrets need centralising; one doesn’t justify the ingress.

The recipe is in android/README.md.

There is no recovery. Android checks this certificate on every update, so a lost APK key means every install has to be uninstalled and redone by hand. Back it up before it exists anywhere else.

Order

A → B → C → D. Extraction first, because it is provable against an APK this repo already builds. Then the index documents, served unsigned, checked with curl. Then signing, last, because it is the piece with a fallback. CI publishing is the smallest step and lands once there is something to publish into.

Sources