neiam
/gitgud
Git Gud
public · Issues · Pulls · Labels · Forks · Compare · Actions success · Badges · Packages
⭐
1
Log in to mark this repository.
Bring PROGRESS and ROADMAP current: slice 73
278311f · Gabriel Morell · 2026-09-12 04:53
Message
{commit_body(@commit)}
Files changed
modified
PROGRESS.md
+59
−0
@@ -12,6 +12,65 @@ intermediate "I was here when I stopped" state) lives in the
| 12 | 12 | |
| 13 | 13 | --- |
| 14 | 14 | |
| 15 | +## Slice 73 — Import adoption, reconcile-mode upstream sync, listing timestamps | |
| 16 | + | |
| 17 | +(Slice 72 — matrix builds — shipped in the commit history but was never | |
| 18 | +written up here.) | |
| 19 | + | |
| 20 | +### Adopting an existing repository instead of skipping it | |
| 21 | + | |
| 22 | +- **A name collision is not automatically a conflict.** An import into a namespace that already holds `utils` used to skip the item outright. Half the time that's a re-run, or a repo someone pushed here by hand before the migration caught up — the same project arriving twice. The other half it's a genuinely different project that happens to share a name, and merging those would be much worse than skipping. | |
| 23 | +- **The forge already had the test for "same project": a shared root commit** (`repository_roots`, `list_related_repositories/2`). `Repositories.link_upstream/3` fetches the remote into a `refs/link-check/*` scratch namespace, compares root sets, and deletes the scratch refs. Related → the local repo is pointed at the remote's URL as its upstream. Unrelated → `{:error, :unrelated_history}` and the item skips with a reason that says so. | |
| 24 | +- **Adoption links, it does not sync.** The adopted repo has history of its own, so no ref is touched: it is set to `"reconcile"` sync mode and the operator syncs from repo settings when they're ready. Importing must never be a way to silently overwrite work that was already here. | |
| 25 | +- **New item status `linked`** (badge + count tile on the import page) — "nothing was cloned, an existing repository was attached" is neither `imported` nor `skipped`. | |
| 26 | +- A failed lineage fetch resolves to `skipped`, not a retry: a name collision has always been a permanent condition for an item, and burning three attempts on an unreachable remote to re-learn that helps nobody. | |
| 27 | + | |
| 28 | +### `upstream_sync_mode`: mirror or reconcile | |
| 29 | + | |
| 30 | +- **`sync_upstream/3` grew a second mode.** `"mirror"` is what it always did and what an imported repo still gets — force-update everything, prune what upstream deleted. `"reconcile"` fast-forwards the branches that can fast-forward, creates branches upstream has and we don't, and touches nothing else. Divergence is *reported*, never resolved by force. | |
| 31 | +- **`upstream_status/2` is the dry run** — same classification (`created` / `fast_forward` / `up_to_date` / `ahead` / `diverged` / `local_only`), no writes. Repo settings calls it behind "Check without changing anything". | |
| 32 | +- **Upstream lands in `refs/upstream-sync/*` and stays there.** That's what a later reconciliation merges against, and the ref is what keeps those objects alive through a `git gc`. | |
| 33 | +- **`overwrite_branch_from_upstream/2`** is the one resolution that needs no merge UI: take upstream's side of a single branch, deliberately, from the settings panel. Keeping our side needs no action at all. Merging the two in the browser is the roadmap item this work is groundwork for. | |
| 34 | + | |
| 35 | +### Scheduled upstream sync — anonymous, with a seam for auth | |
| 36 | + | |
| 37 | +- **`upstream_auto_sync` per repository, hourly, opt-in.** `GitGud.Repositories.Workers.UpstreamSync` is both the cron entry (walks the opted-in repos, enqueues one job each) and the per-repo job, so one unreachable upstream delays only itself. | |
| 38 | +- **Anonymous on purpose.** Nothing retains a credential past the import that created the repo, and inventing per-repo credential storage is a security decision, not a scheduling one. `credentials_for/1` is the single seam: it returns `nil` today, and every call path, error message and the settings copy already route through it. | |
| 39 | +- **A private upstream is diagnosed, not retried.** git's phrasings for "this remote wants a login" are matched and turned into one sentence in `upstream_sync_error`, surfaced on the settings page. A scheduled job has nobody watching it, so a silent failure is indistinguishable from an upstream that simply hasn't moved. | |
| 40 | +- Failures don't burn Oban attempts: the next tick is soon enough, and a permanently private upstream would otherwise fail three times an hour forever. | |
| 41 | +- Safe to run unattended precisely because of `upstream_sync_mode` — a `reconcile` repo can't lose local work to it. | |
| 42 | + | |
| 43 | +### gix 0.66 → 0.87, and the last five ops | |
| 44 | + | |
| 45 | +Sequenced before the in-browser merge work on the grounds that the merge editor will lean on exactly these primitives. | |
| 46 | + | |
| 47 | +- **The ops the roadmap listed are now native**: `merge_base`, `merge_tree`, `commit_tree`, `update_ref`, `diff_stats`. Only `root_commits/1` and `diff/4` still fall through to the Cli backend. | |
| 48 | +- **The upgrade was the prerequisite.** gix 0.66 has no merge-base or tree-merge API at all — the roadmap's "once the gix API shakes out" was the real blocker, not the binding work. 0.87 has both. API churn absorbed: commit signatures are raw header bytes now (`author()` / `committer()` parse, `time()` parses again), `Sorting::ByCommitTimeNewestFirst` became `ByCommitTime(CommitTimeOrder::NewestFirst)`, `lookup_entry` lost its buffer argument, `EntryMode.0` became `.value()`, and `default-features = false` now needs `sha1` named explicitly. | |
| 49 | +- **`merge_tree` reports `{:error, :conflict}`** off `has_unresolved_conflicts(TreatAsUnresolved::git())` — the same line `git merge-tree --write-tree` draws with its exit code 1. | |
| 50 | +- **`commit_tree` writes the object and nothing else** (gix's own `commit_as` moves a ref too), and appends the trailing newline `git commit-tree -m` guarantees. Without it the two backends produce different commit objects for identical input — which is what the parity test asserts, byte for byte. | |
| 51 | +- **`diff_stats` counts lines through the blob-diff platform**, with binary files contributing a changed-file row and no counts, matching `--numstat`'s `-` columns. | |
| 52 | +- Parity tests per op against the Cli backend, including a no-common-ancestor merge-base, a genuine merge conflict, and `update_ref`'s expected-old constraint rejecting a wrong value without moving the ref. | |
| 53 | + | |
| 54 | +### The forge packages itself (showcasing slice 70) | |
| 55 | + | |
| 56 | +- **CI builds an installable `.deb` of the release and publishes it to this forge's own apt repository** — `deb` job in `.forgejo/workflows/build.yml`, authenticated with `${{ github.token }}`, no stored credential. The most honest test of a registry is using it. | |
| 57 | +- **A `release-export` stage in the Dockerfile** (`FROM scratch`, carrying only `_build/prod/rel/git_gud`) hands the release tree to the packaging script via `--output type=local`. The `deb` job `needs: [build]` so it reuses that job's buildx cache instead of compiling the release cold a second time, and the runner needs no Elixir or Rust toolchain. | |
| 58 | +- **`packaging/deb/build-deb.sh` is what CI runs**, and it takes a release directory — so the same package builds by hand from `mix release`. A packaging bug is reproducible without pushing a commit. | |
| 59 | +- **The package is a self-contained OTP release** (ERTS bundled, so no `erlang` dependency and no distro OTP skew) plus a systemd unit and one conffile. `Depends` is only what the release genuinely can't carry: `git` and `curl` (the Cli backend and the push hooks shell out to them), `openssh-client` for `ssh-keygen`, and the C/locale runtime. | |
| 60 | +- **Enabled but not started, and purge keeps the data.** The shipped env file has `CHANGEME` in it, so auto-starting would only produce a crash loop; and `/var/lib/gitgud` holds every repository and package on the machine, so `postrm purge` prints where they are rather than deleting them. | |
| 61 | +- **Two suites, one build.** trixie and forky, published as `…+trixie` / `…+forky`: the same bytes under two versions, because a `.deb` is identified by `<name>_<version>_<arch>.deb` and one file can only sit in one suite's index. They are not separately compiled — a trixie-linked release runs on forky, and `hexpm/elixir` publishes no forky-based builder image to compile a native one against. `SUITE=` in `build-deb.sh` is the whole of the difference. | |
| 62 | +- **Verified by running it, which found a bug.** The exported release was booted in a bare `debian:trixie-slim` container with exactly the package's `Depends` installed: it starts, which is what that list is a claim about. It also came up with native name encoding `latin1`, because systemd (and a bare container) start a process with no locale — Elixir warns on every boot and non-ASCII filenames mangle. The unit now sets `LANG`/`LC_ALL=C.UTF-8`, which glibc carries, so unlike the Dockerfile's `en_US.UTF-8` nothing has to run `locale-gen` and `locales` drops off `Depends`. | |
| 63 | +- **`CI_FORGE_URL` joins `CI_REGISTRY` as a built-in CI var** — the forge's own base URL, so a workflow can reach `/pkg/...` and the REST API with the per-job token and no per-repo config. Both stay overridable; `CI_*` is deliberately not a reserved prefix. | |
| 64 | +- `test/git_gud_web/gitgud_deb_package_test.exs` renders `control.in` with the same substitutions the script makes and pushes a package built from it through the forge's own apt registry — a broken control stanza fails the suite rather than `apt install`. The dpkg-deb assembly itself is CI's job; this machine has no Debian tooling. | |
| 65 | + | |
| 66 | +### Last-touched column on file listings | |
| 67 | + | |
| 68 | +- Repo overview and tree listings show when each entry was last touched, ISO 8601 in the tooltip. | |
| 69 | +- **One history walk per listing, in the NIF.** `Git.last_touched/4` is `git log --name-status` folded to one answer per entry: newest-first walk, a name is touched when its oid under the directory differs from the oid in *every* parent (git's own TREESAME test), each name drops out of the pending set on its first touch. Caches by commit → directory tree and tree → entry oids, so a tree is decoded once even though it's also visited as a parent. ~2 ms vs ~20 ms for the CLI equivalent on this repo; the per-row `git log -1` it replaced was N processes. | |
| 70 | +- The gix walk needed `Sorting::ByCommitTimeNewestFirst` explicitly — the breadth-first default answers a merge with whichever side the queue reached first, which is not what `git log` says. Caught by a parity test against the Cli backend over a merge whose sides both changed the entry. | |
| 71 | + | |
| 72 | +--- | |
| 73 | + | |
| 15 | 74 | ## Slice 71 — Org dashboard |
| 16 | 75 | |
| 17 | 76 | `/orgs/:handle/dashboard` — the signed-in dashboard narrowed to one organization. |
modified
ROADMAP.md
+6
−6
@@ -9,8 +9,9 @@ See [PROGRESS.md](./PROGRESS.md) for what's done.
| 9 | 9 | |
| 10 | 10 | ## Now — in flight |
| 11 | 11 | |
| 12 | −Nothing in flight. Slice 71 (org dashboard) closed most recently; | |
| 13 | −slice 70 (private package registries) is the substantial one before it. | |
| 12 | +Nothing in flight. Slice 73 (import adoption, reconcile-mode upstream | |
| 13 | +sync, the remaining gix NIF ops) closed most recently; slice 70 (private | |
| 14 | +package registries) is the substantial one before it. | |
| 14 | 15 | |
| 15 | 16 | Federation feature set is closed through slice 42 (reviews over AP |
| 16 | 17 | shipped in slice 38, suggested-blocks in slice 38, actionable reports |
@@ -23,6 +24,7 @@ queue in slice 42). Remaining federation items are discretionary
| 23 | 24 | |---|---|---| |
| 24 | 25 | | `${{ gitgud.token }}` in CI + docs | From **G Notes** below. The per-job token is ours, not GitHub's, and every registry and the OCI endpoint now accept it — the name should say so. Keep `${{ github.token }}` working: it is what `forgejo-runner` injects as `GITHUB_TOKEN`, and every existing workflow uses it | small | |
| 25 | 26 | | Package retention / GC | Nothing prunes old package versions. Yanking deliberately keeps the bytes so existing pins keep resolving, which means storage only grows | medium | |
| 27 | +| Credentials for scheduled upstream sync | Slice 73 ships the scheduler, anonymous only: a private upstream reports "needs credentials" and waits for a human. Storing a per-repo credential is the open question (scope, rotation, who may read it) — `UpstreamSync.credentials_for/1` is the single seam it plugs into | small-medium | | |
| 26 | 28 | | Namespace-wide indexes beyond apt | `namespace_index/3` is optional on `GitGud.Packages.Format` and only `Deb` implements it. A namespace-wide PyPI index is the same shape and needs no new routing | small-medium | |
| 27 | 29 | |
| 28 | 30 | ## Soon — medium investments |
@@ -30,9 +32,7 @@ queue in slice 42). Remaining federation items are discretionary
| 30 | 32 | | Item | Why | Size | |
| 31 | 33 | |---|---|---| |
| 32 | 34 | | Import issues / PRs / wikis alongside git data | Slice 59 imports git data only. Issues need author mapping for users who don't exist locally; PRs need ref rewriting | large | |
| 33 | −| Scheduled mirror sync | Re-sync is manual today. A cron would need retained credentials and a divergence policy | medium | | |
| 34 | −| Resolve PR conflicts in-browser | Today a conflict means "fix it locally and push" — a 3-way merge editor would close the loop | medium-large | | |
| 35 | −| gix NIF: fill in diff_stats, merge_base, merge_tree, commit_tree, update_ref | 10-100× perf for repo-page hot paths; Cli fallback works today. log / commit / tree landed in slice 56 | medium | | |
| 35 | +| Reconcile divergence in-browser | One 3-way merge editor, two callers: a PR whose merge conflicts, and an upstream branch that moved on both sides. Slice 73 built the detection and the non-merge resolutions — `upstream_status/2` classifies every branch, `refs/upstream-sync/*` holds upstream's tip so it stays mergeable, and "take upstream's side" is a button — plus the merge primitives it will run on (`merge_base`, `merge_tree`, `commit_tree`, `update_ref` are all native now). What's missing is the editor itself and the same shape for PR conflicts | medium-large | | |
| 36 | 36 | | Package signing key rotation | Keys mint themselves on first use and are re-wrapped by the `GitGud.Secrets` master-key rotation, but there is no "roll the keypair" flow — because rolling it breaks every consumer that pinned the old one. Needs a publish-both-then-retire story before it is safe | medium | |
| 37 | 37 | |
| 38 | 38 | ## Eventually — bigger / discretionary |
@@ -58,7 +58,7 @@ These are documented in the auto-memory under `project-git-gud-moderation-deferr
| 58 | 58 | |
| 59 | 59 | | Item | Notes | |
| 60 | 60 | |---|---| |
| 61 | −| Wire NIF for the remaining git ops | log / commit / tree landed in slice 56; `diff_stats`, `merge_base`, `merge_tree`, `commit_tree` and `update_ref` still fall back to the CLI per-op | | |
| 61 | +| `root_commits/1` + `diff/4` in the NIF | The last two ops still falling through to the Cli backend. Everything else is native as of slice 73 | | |
| 62 | 62 | | Test fixture cleanup | `tmp/test/repos/` wiped on suite start; consider per-test isolation if parallelism grows | |
| 63 | 63 | | Documentation as code grows | This file + PROGRESS.md; append on each slice close | |
| 64 | 64 | | `mix precommit` enforces warnings-as-errors | **Currently red**, and has been for a while — 17 sites across 14 files, in three families: unreachable clauses after a type got narrowed (`duration/2`, `format_deny/1`, `actor_name/1`, …), one unused `require Logger`, and genuine type mismatches where `attach_label/2` / `detach_label/2` / `check_review_block/3` are handed a map where a struct is expected. None is a runtime bug today, but the gate is worthless while it is red, and each new warning hides behind the existing noise. Worth one pass with `mix compile --force --warnings-as-errors` | |
Parents: d82bd83