neiam
/gitgud
Git Gud
public · Issues · Pulls · Labels · Forks · Compare · Actions success · Badges · Packages
⭐
1
Log in to mark this repository.
Plan: dependency security decisions, and what an OSV mirror costs
29569b7 · Gabriel Morell · 2026-10-22 19:43
Message
{commit_body(@commit)}
Files changed
modified
docs/plans/dependency-security.md
+72
−10
@@ -1,7 +1,8 @@
| 1 | 1 | # Plan: dependency security for repositories on the forge |
| 2 | 2 | |
| 3 | 3 | Status: **proposed** (2026-10-22). Nothing here is built yet except |
| 4 | −step 0; the open questions at the end have recommended defaults. | |
| 4 | +step 0. Decisions so far are under "Decisions"; open questions 3, 5, 6, | |
| 5 | +7 and 8 still have only recommended defaults. | |
| 5 | 6 | |
| 6 | 7 | Three things, in the order they depend on each other: |
| 7 | 8 |
@@ -99,12 +100,12 @@ New endpoint: `POST /api/mobile/v1/repos/:owner/:name/issues` (`title`,
| 99 | 100 | |
| 100 | 101 | ### Who the issue or pull request is from |
| 101 | 102 | |
| 102 | −One forge-wide CI account, created by migration: handle `gitgud-ci` | |
| 103 | +One forge-wide CI account, created by migration: handle `gg-security` | |
| 103 | 104 | (configurable), `is_bot: true` (new column), no password, no email |
| 104 | 105 | login, no SSH keys, can't be logged into, never federated as a person. |
| 105 | 106 | Issues and PRs a job opens are authored by it, and their history |
| 106 | 107 | (`GitGud.Events`) and audit entries carry the job |
| 107 | −(`Audit.Actor.job/2`: "run #12 / job deps"). The UI shows "gitgud-ci | |
| 108 | +(`Audit.Actor.job/2`: "run #12 / job deps"). The UI shows "gg-security | |
| 108 | 109 | (from run #12)" linking to the run. |
| 109 | 110 | |
| 110 | 111 | Why one account rather than `author_id: nil` plus a label: every path |
@@ -195,6 +196,45 @@ the EEF's for Hex, PyPA, Go, npm and more. Two ways to ask it:
| 195 | 196 | Config: `config :git_gud, GitGud.Advisories, source: :mirror | :api | |
| 196 | 197 | :off` (default `:mirror`), plus the ecosystems to fetch. |
| 197 | 198 | |
| 199 | +### What the mirror costs (measured 2026-10-22, osv-scanner 2.6.0) | |
| 200 | + | |
| 201 | +OSV's per-ecosystem dumps, zipped, as osv-scanner keeps them: | |
| 202 | + | |
| 203 | +| ecosystem | size | | ecosystem | size | | |
| 204 | +|---|---|---|---|---| | |
| 205 | +| npm | 209 MB | | Maven | 10 MB | | |
| 206 | +| PyPI | 35 MB | | RubyGems | 5 MB | | |
| 207 | +| Go | 12 MB | | crates.io | 3.5 MB | | |
| 208 | +| Packagist | 11 MB | | NuGet | 2.6 MB | | |
| 209 | +| Hex | 0.8 MB | | Pub | <0.1 MB | | |
| 210 | + | |
| 211 | +About 290 MB for all of them, three quarters of it npm (most of npm's | |
| 212 | +is malicious-package reports). The dumps change daily. | |
| 213 | + | |
| 214 | +One scan, offline, loads the whole dump of each ecosystem it finds: | |
| 215 | + | |
| 216 | +| repository | time | memory | | |
| 217 | +|---|---|---| | |
| 218 | +| this one (Hex + crates.io, 4.3 MB of dumps) | ~1 s | 80 MB | | |
| 219 | +| one npm lockfile (209 MB dump) | ~13 s | 600 MB | | |
| 220 | + | |
| 221 | +So the cost is npm's, and it's per scan, not per package. Hence: | |
| 222 | + | |
| 223 | +- Fetch only the ecosystems some repository on the instance has | |
| 224 | + (known from the last scans), with `If-Modified-Since`. Without npm, | |
| 225 | + a few MB a day; with it, ~210 MB a day (~6.5 GB a month in). | |
| 226 | +- Nightly rescans run as **one** osv-scanner invocation over every | |
| 227 | + repository's exported lockfiles (a directory per repository), so each | |
| 228 | + dump loads once a night, not once per repository. | |
| 229 | +- Per-push scans are debounced (a minute) and batched the same way, and | |
| 230 | + run one at a time (`:advisories` queue, concurrency 1), so an npm | |
| 231 | + repository costs one 600 MB spike, not one per push in flight. | |
| 232 | + | |
| 233 | +The API instead costs nothing locally and needs no storage, but sends | |
| 234 | +each scan's package names and versions to osv.dev, makes scanning | |
| 235 | +depend on a third-party service being up, and for a private repository | |
| 236 | +discloses its dependency inventory. | |
| 237 | + | |
| 198 | 238 | ### Scanning |
| 199 | 239 | |
| 200 | 240 | Shell out to osv-scanner rather than parse lockfiles in Elixir: it |
@@ -209,11 +249,11 @@ runtime dependency; absent, the feature reports itself off.
| 209 | 249 | repository after the mirror refreshes — new advisories arrive without |
| 210 | 250 | anyone pushing. |
| 211 | 251 | - **How:** export only the lockfiles at the head commit (`git archive` |
| 212 | − of the matching paths) into a temp dir, run | |
| 252 | + of the matching paths) into a temp dir per repository, run | |
| 213 | 253 | `osv-scanner scan source -r --offline-vulnerabilities --format json` |
| 214 | − there with a timeout and no network. Repositories' own files never | |
| 215 | − run as code (no `--call-analysis`). Oban queue `:advisories`, | |
| 216 | − concurrency 2. | |
| 254 | + over the batch with a timeout and no network, and split the results | |
| 255 | + by source path. Repositories' own files never run as code (no | |
| 256 | + `--call-analysis`). Oban queue `:advisories`, concurrency 1. | |
| 217 | 257 | |
| 218 | 258 | ### Alerts |
| 219 | 259 |
@@ -224,9 +264,10 @@ its GHSA/CVE aliases), `ecosystem`, `package`, `version`,
| 224 | 264 | `first_seen_at`, `fixed_at`. A rescan opens what's new, marks `fixed` |
| 225 | 265 | what's gone, leaves `dismissed` alone (unless the version changes). |
| 226 | 266 | |
| 227 | −- **Who sees them:** a repository's writers and admins, and the org's | |
| 228 | − admins — an open alert is a map to what's exploitable, so not every | |
| 229 | − reader of a public repository (GitHub draws the same line). | |
| 267 | +- **Who sees them:** an org repository's alerts, the org's admins and | |
| 268 | + members; a personal repository's, its owner and collaborators. Not | |
| 269 | + every reader of a public repository: an open alert is a map to what's | |
| 270 | + exploitable. | |
| 230 | 271 | - **Where:** a "Security" tab on the repository |
| 231 | 272 | (`/r/:owner/:name/security[/:state]`, path segments), a roll-up at |
| 232 | 273 | `/orgs/:handle/security`, and the dashboard's attention panel. |
@@ -241,6 +282,14 @@ what's gone, leaves `dismissed` alone (unless the version changes).
| 241 | 282 | - **API and CLI:** `GET /api/mobile/v1/repos/:owner/:name/security-alerts` |
| 242 | 283 | behind a new `security:read` scope, `gg security`. |
| 243 | 284 | |
| 285 | +## Android | |
| 286 | + | |
| 287 | +The app has a version catalog (`gradle/libs.versions.toml`) but no | |
| 288 | +lockfile, so neither `audit.yml` nor C can read it. Turning on Gradle's | |
| 289 | +dependency locking (`dependencyLocking { lockAllConfigurations() }`, | |
| 290 | +then `./gradlew dependencies --write-locks`) commits a | |
| 291 | +`gradle.lockfile` osv-scanner reads. Nice to have; step 7. | |
| 292 | + | |
| 244 | 293 | ## Phasing (each ships on its own) |
| 245 | 294 | |
| 246 | 295 | 0. `audit.yml` for the forge itself. **Done.** |
@@ -254,12 +303,25 @@ what's gone, leaves `dismissed` alone (unless the version changes).
| 254 | 303 | 5. **C2** — the OSV mirror, nightly rescans, notifications and digest. |
| 255 | 304 | 6. **C3** — the org roll-up, dismissals in the audit log, the API and |
| 256 | 305 | `gg security`, the "open an update PR" link. |
| 306 | +7. **Android** — Gradle dependency locking, so the app is scanned too. | |
| 257 | 307 | |
| 258 | 308 | ## Done when |
| 259 | 309 | |
| 260 | 310 | `mix precommit` passes with no new warnings; one commit per step; |
| 261 | 311 | nothing pushed without asking (pushing deploys). |
| 262 | 312 | |
| 313 | +## Decisions (2026-10-22) | |
| 314 | + | |
| 315 | +- **The CI account is `gg-security`**, one for the whole forge (open | |
| 316 | + question 1). | |
| 317 | +- **Job-token pull requests trigger `pull_request` workflows**, with the | |
| 318 | + one-hop guard — the update PR's own run is what shows the | |
| 319 | + vulnerabilities closed (open question 2). | |
| 320 | +- **Alerts are for admins and members** — the org's for an org | |
| 321 | + repository, the owner and collaborators for a personal one (open | |
| 322 | + question 4). | |
| 323 | +- **Android lockfiles: nice to have** — step 7 (open question 9). | |
| 324 | + | |
| 263 | 325 | ## Open questions |
| 264 | 326 | |
| 265 | 327 | 1. A forge-wide CI account (recommended), or one per organization, or |
Parents: b05f3a1