GITGUD

neiam neiam /gitgud

Git Gud

public · Issues · Pulls · Labels · Forks · Compare · Actions success · Badges · Packages

⭐ 1

Log in to mark this repository.

Actions: hand a job its needs' results and outputs, and skip instead of cancel

15e553d · Gabriel Morell · 2026-10-17 15:33

20 files +664 -116
Message 
{commit_body(@commit)}

Files changed

modified README.md
+3 −1
@@ -92,6 +92,7 @@ reinvent them.
92 92 - **`on: pull_request`** for pull requests within one repository: `types:` `opened`, `synchronize` (the head moved) and `reopened` (the default; other activity types never fire). `branches:` / `branches-ignore:` filter the **target** branch, and `paths:` the PR's diff from its merge-base. Workflows are read from the PR's head commit, so a PR can change its own CI, and the job tests the head itself at `refs/pull/<N>/head` — not a merge with the target, as GitHub's `refs/pull/<N>/merge` does. The job gets `github.event_name`, `head_ref` and `base_ref` (the source and target branches), `ref_name` `<N>/head` and an empty `ref_type`. **Fork and federated pull requests don't run yet**: their head is code the target's maintainers haven't reviewed, and a job is handed the target repository's secrets and a token that can publish its packages; GitHub runs them without secrets and with a read-only token, and until that exists here they don't run at all. `pull_request_target` isn't supported
93 93 - **`paths:` / `paths-ignore:`** with GitHub's pattern rules (`*` stays within a directory, `**` crosses them, `?`, `[a-z]`; in `paths:`, `!pattern` un-matches and a later positive pattern re-matches). A push runs if any changed file is selected by `paths:`, or if not every changed file is matched by `paths-ignore:`; both set → `paths:` wins, with a warning. Changed files are the ref's old → new diff (renames count as both paths); a new branch is diffed from where it left the default branch. Tag pushes and manual runs ignore path filters, and when the changed files can't be worked out (no default branch to compare with, an unknown base, a git error, more than 3000 files) the filter passes — a run is never skipped because the forge couldn't look
94 94 - **The job's context** follows Forgejo's: `github.event_name`, `ref`, `ref_name`, `ref_type`, `sha`, `repository` (`owner/name`), `repository_id`, `repository_owner(_id)`, `actor(_id)`, `run_id`, `run_number`, `run_attempt` (1, bumped by re-running the job; re-running the whole run makes a new run and a new `run_id`), `workflow`, `workflow_ref` (`owner/name/<path>@<ref>`), `job`, `server_url`, `api_url`, `token`. **`github.event`** is GitHub's webhook payload, trimmed: every event has `repository` and `sender`; `push` has `ref`, `before` (40 zeros for a new ref), `after`, `created`, `deleted`, `forced` (always `false`: not tracked), `compare`, `commits` (at most 20) and `head_commit`, `pusher`; `pull_request` has `action`, `number` and `pull_request` (`title`, `body`, `html_url`, `user`, `head`/`base` with `ref`, `sha`, `label`, `repo`); `workflow_dispatch` has `inputs`, `ref`, `workflow`. **`secrets.GITHUB_TOKEN`** (and `GITEA_TOKEN`, `FORGEJO_TOKEN`) is the job token. The workflow's top-level `env:`, `defaults:`, `permissions:`, `run-name:` and `on:` reach the runner with the job
95 + - **Jobs that depend on jobs**, as on GitHub: `${{ needs.<id>.outputs.* }}` and `needs.<id>.result` (`success`, `failure`, `cancelled`, `skipped`; for a matrix, the worst leg's, with the legs' outputs merged) reach the dependent job. A job waits until all its needs finish; if any didn't succeed it is **skipped** — and its own dependents with it — unless its `if:` calls `always()`, `failure()` or `cancelled()`, so cleanup and "report the failure" jobs run. A job its own `if:` skipped shows as skipped, not green. Job-level `continue-on-error: true` (or `${{ matrix.* }}` that comes out `true`) keeps a failure from skipping its dependents, failing the run or fail-fasting its matrix siblings, though the job still shows as failed; any other expression there counts as false. Skipped jobs don't fail a run. Unlike GitHub, **cancelling a run cancels its waiting jobs too**, `always()` or not
95 96 - Re-run / cancel buttons on the workflow show page
96 97 - **The job token** (`${{ github.token }}`, an HS256 JWT minted per job on `FetchTask`) works only while its job runs and for five minutes after (for the runner's post-job cache save and artifact upload), and only for that attempt of the job — a leaked token no longer stays valid for a day. It clones and fetches its own repository, reads and publishes its packages, reads its metadata over the API, dispatches workflows (its own repository's, or one that granted it), and reaches the artifact + cache endpoints of its run. **`permissions:`** narrows it as on GitHub: workflow level, replaced by a job's own; `read-all`, `write-all`, `{}` or a map in which unnamed scopes are `none`. `contents` (`read` fetch, `write` also push — to its own repository, subject to branch protection like any direct push, attributed to whoever started the run), `packages` (`read` / `write`) and `actions` (`write` to dispatch) mean something; `metadata` is always read; the others (`issues`, `pull-requests`, `statuses`, `checks`, `deployments`, `id-token`, …) are accepted and ignored, since the token can't do those things here anyway. With no `permissions:` the token keeps `contents: read, packages: write, actions: write` — GitHub's permissive default would also let it push. **A push made with the job token starts no workflows** (no `push` run, no pull request `synchronize`), as on GitHub, so a workflow that pushes can't trigger itself forever; webhooks and pull request heads still update
97 98 - **GitHub Actions cache server v1**, scoped to the repository. Restore order: the primary key exactly, then each fallback key as a prefix (newest first), each tried on the run's own ref, then — for a `pull_request` run — the PR's base branch, then the default branch; a save goes to the run's own ref. Upload chunks are placed at their `Content-Range` offsets (`actions/cache` sends four at once) and assembled on finalize; a finalized entry is immutable. Age/size eviction worker, which also clears uploads abandoned for a day. **Not reached by a default forgejo-runner**: with `cache.enabled` (its default) the runner starts its own cache server and points `ACTIONS_CACHE_URL` at it, ignoring the URL gitgud sends; jobs only use this one if the runner has `cache.enabled: false` and `ACTIONS_CACHE_URL` set in `runner.envs`. `actions/cache@v4`'s cache service v2 isn't implemented
@@ -365,7 +366,8 @@ else — `${{ matrix.arch }}` inside a step — is evaluated by the runner
365 366 against a matrix context pinned to that leg's single value.
366 367
367 368 `needs: build` waits for **every** leg, and a downstream job sees one
368 −`needs.build` however many machines produced it. `fail-fast` is on by
369 +`needs.build` however many machines produced it: the worst leg's
370 +result, and the legs' outputs merged. `fail-fast` is on by
369 371 default: the first leg to fail cancels its siblings that haven't
370 372 started, while one already running is left to finish rather than
371 373 killed. `max-parallel` is not implemented — every leg is claimable as
modified lib/git_gud/workflows.ex
+165 −82
@@ -1352,7 +1352,8 @@ defmodule GitGud.Workflows do
1352 1352 # steps:
1353 1353 # * job succeeded → step succeeded
1354 1354 # * step never started → skipped
1355 − # * step was in flight → inherit the job's terminal status
1355 + # * step was in flight → inherit the job's terminal status (a skipped
1356 + # job's steps are skipped)
1356 1357 defp finalize_steps(%WorkflowJob{id: jid, status: job_status}) do
1357 1358 now = DateTime.utc_now() |> DateTime.truncate(:second)
1358 1359
@@ -1365,6 +1366,7 @@ defmodule GitGud.Workflows do
1365 1366 cond do
1366 1367 job_status == "success" -> "success"
1367 1368 step.status == "pending" -> "skipped"
1369 + job_status == "skipped" -> "skipped"
1368 1370 job_status in ["failure", "cancelled"] -> job_status
1369 1371 true -> "failure"
1370 1372 end
@@ -1384,10 +1386,12 @@ defmodule GitGud.Workflows do
1384 1386 # sixteen legs is to find out whether they all pass, and once one
1385 1387 # hasn't you have your answer. Only siblings that haven't started are
1386 1388 # cancelled; a leg already running is left to finish, since killing a
1387 − # mid-flight job loses its logs for no gain.
1389 + # mid-flight job loses its logs for no gain. A leg that fails under
1390 + # `continue-on-error` doesn't count, as on GitHub: that's what marking a
1391 + # leg experimental is for.
1388 1392 defp maybe_fail_fast(%WorkflowJob{status: status} = job)
1389 1393 when status in ~w(failure cancelled) do
1390 − if job.matrix && Matrix.fail_fast?(strategy_of(job)) do
1394 + if job.matrix && Matrix.fail_fast?(strategy_of(job)) && effective_result(job) != "success" do
1391 1395 from(j in WorkflowJob,
1392 1396 where:
1393 1397 j.workflow_run_id == ^job.workflow_run_id and j.job_id == ^job.job_id and
@@ -1406,13 +1410,17 @@ defmodule GitGud.Workflows do
1406 1410
1407 1411 @doc """
1408 1412 After a job reaches a terminal state, walk every sibling job in the
1409 − same run whose `needs:` array contains this job's `job_id` and
1410 − promote them from `blocked`:
1413 + same run whose `needs:` array contains this job's `job_id` and, once
1414 + every leg of every need is terminal, decide it as GitHub does:
1411 1415
1412 − * all listed needs are now `success`/`skipped` → `queued`
1413 − * any listed need is `failure`/`cancelled` → `cancelled`
1414 − (cascading failure, matches GitHub Actions default)
1415 − * mixed-but-not-failed (still some pending needs) → leave `blocked`
1416 + * a need that doesn't exist → `cancelled` (author error)
1417 + * every need concluded `success` (`effective_result/1`) → `queued`
1418 + * otherwise, the job's `if:` calls `always()`, `failure()` or
1419 + `cancelled()` → `queued`, and the runner evaluates the `if:`
1420 + against the real `needs` results
1421 + * otherwise → `skipped`, which resolves its own dependents in turn
1422 +
1423 + While some need is still in flight the job stays `blocked`.
1416 1424
1417 1425 Public so tests can drive it directly.
1418 1426 """
@@ -1431,41 +1439,19 @@ defmodule GitGud.Workflows do
1431 1439 Enum.filter(siblings, fn j -> completed_jid in j.needs and j.status == "blocked" end)
1432 1440
1433 1441 Enum.each(dependents, fn dep ->
1434 − needs_states = Enum.map(dep.needs, fn jid -> Map.get(by_jid, jid) end)
1435 −
1436 − cond do
1437 − Enum.any?(needs_states, &(&1 in [nil, []])) ->
1438 − # A referenced need doesn't exist as a sibling — workflow
1439 − # author error. Cancel rather than block forever.
1440 − {:ok, _} = update_job_status(dep, "cancelled")
1442 + # Skipping an earlier dependent resolves its own dependents first,
1443 + # and one of those may be later in this list: re-read rather than
1444 + # decide it twice.
1445 + dep = Repo.get!(WorkflowJob, dep.id)
1441 1446
1442 − Enum.any?(needs_states, fn legs ->
1443 − Enum.any?(legs, &(&1.status in ~w(failure cancelled)))
1444 − end) ->
1447 + case dep.status == "blocked" && dependency_verdict(dep, by_jid) do
1448 + :cancel ->
1445 1449 {:ok, _} = update_job_status(dep, "cancelled")
1446 1450
1447 − Enum.all?(needs_states, fn legs ->
1448 − Enum.all?(legs, &WorkflowJob.satisfies_dependency?(&1.status))
1449 − end) ->
1450 − # Snapshot upstream results/outputs now, while they're terminal.
1451 − # build_needs/1 reads this instead of re-reading upstream status
1452 − # at fetch time (which raced and could yield RESULT_UNSPECIFIED,
1453 − # making act skip the job on `if: success()`).
1454 − #
1455 − # A matrix's legs collapse to one entry, because `needs.build`
1456 − # is one name downstream however many machines produced it:
1457 − # outputs merge, and the status is the worst of them.
1458 − snapshot =
1459 − Map.new(dep.needs, fn jid ->
1460 − legs = Map.get(by_jid, jid)
1461 −
1462 − {jid,
1463 − %{
1464 − "status" => aggregate_status(legs),
1465 − "outputs" => Enum.reduce(legs, %{}, &Map.merge(&2, &1.outputs || %{}))
1466 − }}
1467 − end)
1451 + :skip ->
1452 + {:ok, _} = update_job_status(dep, "skipped")
1468 1453
1454 + {:queue, snapshot} ->
1469 1455 {:ok, _} =
1470 1456 dep
1471 1457 |> Ecto.Changeset.change(status: "queued", needs_snapshot: snapshot)
@@ -1473,8 +1459,7 @@ defmodule GitGud.Workflows do
1473 1459
1474 1460 broadcast({:job_status, rid, Repo.get!(WorkflowJob, dep.id)})
1475 1461
1476 − true ->
1477 − # Some upstream is still pending. Leave blocked.
1462 + _blocked_or_decided ->
1478 1463 :ok
1479 1464 end
1480 1465 end)
@@ -1482,51 +1467,129 @@ defmodule GitGud.Workflows do
1482 1467 :ok
1483 1468 end
1484 1469
1485 − # `success` only when every leg succeeded; `skipped` when they all
1486 − # skipped. Anything mixed reports `success`, because reaching here
1487 − # means no leg failed and a downstream `if: success()` should run.
1470 + # What `dep`'s needs, as they stand in `by_jid`, make of it: `:blocked`
1471 + # while one is in flight, then `{:queue, snapshot}`, `:skip` or
1472 + # `:cancel` (see `resolve_dependents/1`).
1473 + #
1474 + # The snapshot is taken now, while the needs are terminal, and is what
1475 + # the runner gets as `Task.needs` (`RunnerController.build_needs/1`)
1476 + # rather than re-reading upstream status at fetch time, which raced. A
1477 + # matrix's legs collapse to one entry, because `needs.build` is one name
1478 + # downstream however many machines produced it: outputs merge, and the
1479 + # result is the worst of them.
1480 + defp dependency_verdict(%WorkflowJob{} = dep, by_jid) do
1481 + needs_states = Enum.map(dep.needs, fn jid -> Map.get(by_jid, jid) end)
1482 +
1483 + cond do
1484 + Enum.any?(needs_states, &(&1 in [nil, []])) ->
1485 + :cancel
1486 +
1487 + not Enum.all?(needs_states, fn legs ->
1488 + Enum.all?(legs, &WorkflowJob.terminal?(&1.status))
1489 + end) ->
1490 + :blocked
1491 +
1492 + true ->
1493 + snapshot =
1494 + Map.new(dep.needs, fn jid ->
1495 + legs = Map.get(by_jid, jid)
1496 +
1497 + {jid,
1498 + %{
1499 + "status" => aggregate_status(legs),
1500 + "outputs" => Enum.reduce(legs, %{}, &Map.merge(&2, &1.outputs || %{}))
1501 + }}
1502 + end)
1503 +
1504 + cond do
1505 + Enum.all?(snapshot, fn {_, need} -> need["status"] == "success" end) ->
1506 + {:queue, snapshot}
1507 +
1508 + status_function_if?(dep.definition) ->
1509 + {:queue, snapshot}
1510 +
1511 + true ->
1512 + :skip
1513 + end
1514 + end
1515 + end
1516 +
1517 + @doc """
1518 + The result a job's dependents see: `success` for a job that failed
1519 + under `continue-on-error: true`, its status otherwise. The job itself
1520 + still shows as failed.
1521 +
1522 + `continue-on-error` is honoured when it's literally true, or `"true"`
1523 + once `${{ matrix.* }}` is substituted (`continue-on-error: ${{
1524 + matrix.experimental }}`). Any other expression is the runner's to
1525 + evaluate, and can't be seen from here: it counts as false.
1526 + """
1527 + def effective_result(%WorkflowJob{status: "failure"} = job) do
1528 + case Matrix.substitute(job.definition["continue-on-error"], job.matrix || %{}) do
1529 + flag when flag in [true, "true"] -> "success"
1530 + _ -> "failure"
1531 + end
1532 + end
1533 +
1534 + def effective_result(%WorkflowJob{status: status}), do: status
1535 +
1536 + @doc """
1537 + Whether a job's `if:` calls a status-check function — `always()`,
1538 + `failure()` or `cancelled()`, negated or not. Only such a job runs
1539 + after a need failed, was cancelled or was skipped.
1540 + """
1541 + def status_function_if?(%{"if" => condition}) when is_binary(condition),
1542 + do: Regex.match?(~r/\b(always|failure|cancelled)\s*\(\s*\)/, condition)
1543 +
1544 + def status_function_if?(_definition), do: false
1545 +
1546 + # `needs.<id>.result` for a matrix: the worst leg's effective result —
1547 + # failure, then cancelled, then success; `skipped` only when every leg
1548 + # skipped.
1488 1549 defp aggregate_status(legs) do
1550 + results = Enum.map(legs, &effective_result/1)
1551 +
1489 1552 cond do
1490 − Enum.all?(legs, &(&1.status == "skipped")) -> "skipped"
1491 − Enum.all?(legs, &(&1.status == "success")) -> "success"
1492 − true -> "success"
1553 + "failure" in results -> "failure"
1554 + "cancelled" in results -> "cancelled"
1555 + "success" in results -> "success"
1556 + true -> "skipped"
1493 1557 end
1494 1558 end
1495 1559
1560 + # A job failed the run when it failed without `continue-on-error`. A
1561 + # job cancelled on its own (fail-fast, a runner that gave up) fails it
1562 + # too, as on GitHub; only a cancelled run concludes `cancelled`.
1563 + # Skipped jobs don't fail anything.
1496 1564 defp maybe_complete_run(%WorkflowJob{workflow_run_id: rid}) do
1497 − statuses =
1498 − from(j in WorkflowJob, where: j.workflow_run_id == ^rid, select: j.status)
1499 − |> Repo.all()
1565 + jobs = from(j in WorkflowJob, where: j.workflow_run_id == ^rid) |> Repo.all()
1566 + run = Repo.get!(WorkflowRun, rid)
1500 1567
1501 1568 cond do
1502 1569 # `blocked` is still pending — its upstream is in flight.
1503 − Enum.any?(statuses, &(&1 in ~w(queued running blocked))) ->
1504 − run = Repo.get!(WorkflowRun, rid)
1570 + Enum.any?(jobs, &(&1.status in ~w(queued running blocked))) ->
1505 1571 update_run_commit_status(run)
1506 1572 :ok
1507 1573
1508 − Enum.any?(statuses, &(&1 in ~w(failure cancelled))) ->
1509 − run =
1510 − Repo.get!(WorkflowRun, rid)
1511 − |> WorkflowRun.status_changeset("failure")
1512 − |> Repo.update!()
1574 + Enum.any?(jobs, &(effective_result(&1) == "failure")) ->
1575 + finish_run(run, "failure")
1513 1576
1514 − update_run_commit_status(run)
1515 − broadcast_run_status(run)
1577 + Enum.any?(jobs, &(&1.status == "cancelled")) ->
1578 + finish_run(run, if(run.status == "cancelled", do: "cancelled", else: "failure"))
1516 1579
1517 1580 true ->
1518 − run =
1519 − Repo.get!(WorkflowRun, rid)
1520 − |> WorkflowRun.status_changeset("success")
1521 − |> Repo.update!()
1522 −
1523 − update_run_commit_status(run)
1581 + run = finish_run(run, "success")
1524 1582 notify_run_webhook(run)
1525 − broadcast_run_status(run)
1526 1583 run
1527 1584 end
1528 1585 end
1529 1586
1587 + defp finish_run(%WorkflowRun{} = run, status) do
1588 + run = run |> WorkflowRun.status_changeset(status) |> Repo.update!()
1589 + update_run_commit_status(run)
1590 + broadcast_run_status(run)
1591 + end
1592 +
1530 1593 # Notify the run's LiveView that the run reached a terminal state so it
1531 1594 # can flip the status badge + show the re-run button without a reload.
1532 1595 # (The per-job broadcast races ahead of `maybe_complete_run`, so the
@@ -1705,19 +1768,12 @@ defmodule GitGud.Workflows do
1705 1768 legs ->
1706 1769 by_jid = Enum.group_by(siblings, & &1.job_id)
1707 1770
1708 − needs_ok? =
1709 − fn job ->
1710 − job.needs == [] or
1711 − Enum.all?(job.needs, fn jid ->
1712 − case Map.get(by_jid, jid) do
1713 − [_ | _] = upstream ->
1714 − Enum.all?(upstream, &WorkflowJob.satisfies_dependency?(&1.status))
1715 −
1716 − _ ->
1717 − false
1718 − end
1719 − end)
1720 − end
1771 + # Decided as `resolve_dependents/1` would decide it now, against
1772 + # the needs as they stand (and their results snapshotted afresh).
1773 + verdict = fn
1774 + %WorkflowJob{needs: []} -> {:queue, %{}}
1775 + job -> dependency_verdict(job, by_jid)
1776 + end
1721 1777
1722 1778 Repo.transaction(fn ->
1723 1779 updated =
@@ -1743,10 +1799,17 @@ defmodule GitGud.Workflows do
1743 1799 ]
1744 1800 )
1745 1801
1802 + {status, snapshot} =
1803 + case verdict.(job) do
1804 + {:queue, snapshot} -> {"queued", snapshot}
1805 + _ -> {"blocked", job.needs_snapshot}
1806 + end
1807 +
1746 1808 {:ok, row} =
1747 1809 job
1748 1810 |> Ecto.Changeset.change(
1749 − status: if(needs_ok?.(job), do: "queued", else: "blocked"),
1811 + status: status,
1812 + needs_snapshot: snapshot,
1750 1813 conclusion: nil,
1751 1814 started_at: nil,
1752 1815 completed_at: nil,
@@ -1767,6 +1830,23 @@ defmodule GitGud.Workflows do
1767 1830 |> case do
1768 1831 {:ok, updated} ->
1769 1832 Enum.each(updated, &broadcast({:job_status, run.id, &1}))
1833 +
1834 + # A job whose needs are all done but wouldn't let it run is
1835 + # skipped (or cancelled) again rather than left blocked on
1836 + # upstreams that won't finish twice, which would hold the run
1837 + # open for good.
1838 + Enum.each(updated, fn
1839 + %WorkflowJob{status: "blocked"} = row ->
1840 + case verdict.(row) do
1841 + :skip -> {:ok, _} = update_job_status(row, "skipped")
1842 + :cancel -> {:ok, _} = update_job_status(row, "cancelled")
1843 + _ -> :ok
1844 + end
1845 +
1846 + _ ->
1847 + :ok
1848 + end)
1849 +
1770 1850 broadcast_ci_change(run.id)
1771 1851 {:ok, Repo.get!(WorkflowRun, run.id)}
1772 1852
@@ -1792,9 +1872,12 @@ defmodule GitGud.Workflows do
1792 1872 Repo.transaction(fn ->
1793 1873 now = DateTime.utc_now(:second)
1794 1874
1875 + # Blocked jobs are cancelled with the rest, even one whose `if:`
1876 + # says `always()` or `cancelled()`. GitHub would still run those
1877 + # once their needs finish; here a cancelled run runs nothing more.
1795 1878 Repo.update_all(
1796 1879 from(j in WorkflowJob,
1797 − where: j.workflow_run_id == ^run.id and j.status in ["queued", "running"]
1880 + where: j.workflow_run_id == ^run.id and j.status in ["queued", "running", "blocked"]
1798 1881 ),
1799 1882 set: [status: "cancelled", conclusion: "cancelled", completed_at: now]
1800 1883 )
modified lib/git_gud/workflows/runner_payload.ex
+4 −2
@@ -38,8 +38,10 @@ defmodule GitGud.Workflows.RunnerPayload do
38 38 # `if: success()` evaluates false, silently skipping the job. That
39 39 # broke every multi-stage (test -> image) workflow. The dependency is
40 40 # already honoured out-of-band: the forge only dispatches a job once
41 − # its needs are satisfied (Workflows.resolve_dependents), and upstream
42 − # results/outputs reach the job via the Task `needs` field.
41 + # its needs are terminal (Workflows.resolve_dependents), and upstream
42 + # results/outputs reach the job via the Task `needs` field, from which
43 + # forgejo-runner rebuilds the job's `needs:` against mock jobs
44 + # (RunnerController.build_needs/1).
43 45 definition = Map.drop(job.definition, ["needs", :needs])
44 46
45 47 run
modified lib/git_gud/workflows/workflow_job.ex
+7 −15
@@ -7,13 +7,14 @@ defmodule GitGud.Workflows.WorkflowJob do
7 7 alias GitGud.Workflows.WorkflowStep
8 8
9 9 # `blocked` = waiting on upstream `needs:`. Promoted to `queued` (or
10 − # `cancelled`) by `Workflows.resolve_dependents/1` when any sibling
11 − # job completes. `claim_next_job/1` only picks `queued`, so blocked
12 − # jobs are never accidentally claimed by a runner before their
10 + # `skipped`, or `cancelled`) by `Workflows.resolve_dependents/1` once
11 + # every need is terminal. `claim_next_job/1` only picks `queued`, so
12 + # blocked jobs are never accidentally claimed by a runner before their
13 13 # prerequisites finish.
14 − # `skipped` = upstream that the GitHub-Actions `if:` evaluated false
15 − # for. Treated as success for the purpose of resolving downstream
16 − # `needs:` (matches GitHub semantics).
14 + # `skipped` = a job whose own `if:` the runner evaluated false, or one
15 + # the forge skipped because a need didn't succeed. Not a success:
16 + # dependents are skipped in turn unless their `if:` uses a status
17 + # function (GitHub semantics).
17 18 @valid_statuses ~w(queued running success failure cancelled blocked skipped)
18 19
19 20 schema "workflow_jobs" do
@@ -89,15 +90,6 @@ defmodule GitGud.Workflows.WorkflowJob do
89 90 def terminal?(s) when s in ~w(success failure cancelled skipped), do: true
90 91 def terminal?(_), do: false
91 92
92 − @doc """
93 − Did this job's terminal state count as 'satisfying' a downstream
94 − `needs:` declaration? Mirrors GitHub Actions semantics: a need is
95 − satisfied by `success` or `skipped`, never by `failure` or
96 − `cancelled` (the downstream gets cascade-cancelled instead).
97 − """
98 − def satisfies_dependency?(s) when s in ~w(success skipped), do: true
99 − def satisfies_dependency?(_), do: false
100 −
101 93 def claim_changeset(job, runner_id) do
102 94 change(job, status: "running", runner_id: runner_id, started_at: DateTime.utc_now(:second))
103 95 end
modified lib/git_gud_web/controllers/runner_controller.ex
+35 −12
@@ -41,6 +41,7 @@ defmodule GitGudWeb.RunnerController do
41 41 alias :"Elixir.Runner.V1.RegisterRequest", as: RegisterRequest
42 42 alias :"Elixir.Runner.V1.RegisterResponse", as: RegisterResponse
43 43 alias :"Elixir.Runner.V1.Task", as: TaskMsg
44 + alias :"Elixir.Runner.V1.TaskNeed", as: TaskNeed
44 45 alias :"Elixir.Runner.V1.UpdateLogRequest", as: UpdateLogRequest
45 46 alias :"Elixir.Runner.V1.UpdateLogResponse", as: UpdateLogResponse
46 47 alias :"Elixir.Runner.V1.UpdateTaskRequest", as: UpdateTaskRequest
@@ -223,8 +224,8 @@ defmodule GitGudWeb.RunnerController do
223 224 context: build_context_struct(job, run, repo, token),
224 225 secrets: secrets,
225 226 # `needs.<upstream_id>` surfaces each declared upstream's
226 − # result + outputs to this job. Empty map when the workflow
227 − # declares no `needs:`.
227 + # result + outputs to this job. Empty map when the job declares
228 + # no `needs:`.
228 229 needs: build_needs(job),
229 230 vars: vars
230 231 }
@@ -252,17 +253,38 @@ defmodule GitGudWeb.RunnerController do
252 253 Map.put(registry, "CI_FORGE_URL", GitGudWeb.Endpoint.url())
253 254 end
254 255
255 − # Always send an EMPTY needs map. forgejo-runner's generateWorkflow
256 − # injects a MOCK job (with no `runs-on`) for every Task.needs entry and
257 − # rebuilds the target job's RawNeeds from them; act then can't resolve
258 − # the runs-on-less mock and skips the job via `if: success()`. With no
259 − # entries, no mock is injected and RawNeeds is emptied, so `success()`
260 − # passes and the job runs. Dispatch is already gated on upstream success
261 − # by Workflows.resolve_dependents/1, so the dependency stays honoured.
262 − # Trade-off: `${{ needs.<id>.result/outputs }}` isn't surfaced to the
263 − # job — none of our workflows read cross-job needs outputs.
256 + # `needs.<id>.result` and `needs.<id>.outputs`, from the snapshot
257 + # `Workflows.resolve_dependents/1` took when it queued this job.
258 + # forgejo-runner's generateWorkflow turns each entry into a mock job
259 + # (no `runs-on`, no steps, `Result` preset to the enum's name
260 + # lowercased) and points the job's RawNeeds at them; act's `success()`
261 + # / `failure()` and the `needs` context then read those results. The
262 + # mocks run as no-ops ("No steps found"), so their preset result
263 + # stands. A job with no `needs:` has an empty snapshot and gets none.
264 + #
265 + # Sent empty for a while (ed9a7cd), when every mock arrived with result
266 + # `unspecified` and act skipped the job on `if: success()` — that was
267 + # TaskNeed's field numbers being swapped (fixed in c4d7080), not the
268 + # mocks.
269 + defp build_needs(%GitGud.Workflows.WorkflowJob{needs_snapshot: %{} = snap})
270 + when map_size(snap) > 0 do
271 + Map.new(snap, fn {jid, need} ->
272 + {jid,
273 + %TaskNeed{
274 + result: status_to_result_enum(need["status"]),
275 + outputs: need["outputs"] || %{}
276 + }}
277 + end)
278 + end
279 +
264 280 defp build_needs(_job), do: %{}
265 281
282 + defp status_to_result_enum("success"), do: :RESULT_SUCCESS
283 + defp status_to_result_enum("failure"), do: :RESULT_FAILURE
284 + defp status_to_result_enum("cancelled"), do: :RESULT_CANCELLED
285 + defp status_to_result_enum("skipped"), do: :RESULT_SKIPPED
286 + defp status_to_result_enum(_), do: :RESULT_UNSPECIFIED
287 +
266 288 # Forgejo's runner proto switched `Task.context` from
267 289 # `map<string,string>` to `google.protobuf.Struct` so values can carry
268 290 # nested objects, numbers, and bools. Every key is a string except
@@ -522,12 +544,13 @@ defmodule GitGudWeb.RunnerController do
522 544 defp map_result(:RESULT_SUCCESS), do: "success"
523 545 defp map_result(:RESULT_FAILURE), do: "failure"
524 546 defp map_result(:RESULT_CANCELLED), do: "cancelled"
525 − defp map_result(:RESULT_SKIPPED), do: "success"
547 + defp map_result(:RESULT_SKIPPED), do: "skipped"
526 548 defp map_result(_), do: "running"
527 549
528 550 defp result_atom("success"), do: :RESULT_SUCCESS
529 551 defp result_atom("failure"), do: :RESULT_FAILURE
530 552 defp result_atom("cancelled"), do: :RESULT_CANCELLED
553 + defp result_atom("skipped"), do: :RESULT_SKIPPED
531 554 defp result_atom(_), do: :RESULT_UNSPECIFIED
532 555
533 556 # ── update_log ───────────────────────────────────────────────────────
added test/git_gud/workflows/job_graph_test.exs
+371 −0
@@ -0,0 +1,371 @@
1 +defmodule GitGud.Workflows.JobGraphTest do
2 + @moduledoc """
3 + What a job's `needs:` make of it once they finish, and what the run
4 + concludes — GitHub's rules: a failed, cancelled or skipped need skips
5 + its dependents unless their `if:` calls a status function, and
6 + `continue-on-error` hides a failure from both.
7 + """
8 + use GitGud.DataCase, async: false
9 +
10 + alias GitGud.Workflows
11 + alias GitGud.Workflows.WorkflowJob
12 + alias GitGud.Workflows.WorkflowRun
13 + alias GitGud.Workflows.Yaml
14 +
15 + import GitGud.ForgeFixtures
16 +
17 + setup do
18 + {_user, repo} = repository_fixture()
19 + {:ok, repo: repo}
20 + end
21 +
22 + # As in MatrixDispatchTest: a seed row plus a re-run creates the jobs
23 + # through the real path without a repository on disk.
24 + defp run_workflow(repo, yaml) do
25 + {:ok, doc} = Yaml.parse(yaml)
26 +
27 + seed =
28 + %WorkflowRun{repository_id: repo.id}
29 + |> WorkflowRun.create_changeset(%{
30 + "number" => System.unique_integer([:positive]),
31 + "workflow_path" => ".forgejo/workflows/ci.yml",
32 + "workflow_name" => doc.name,
33 + "trigger" => "push",
34 + "head_ref" => "refs/heads/main",
35 + "head_sha" => :crypto.strong_rand_bytes(20),
36 + "workflow_doc" => doc.raw
37 + })
38 + |> Repo.insert!()
39 +
40 + {:ok, run} = Workflows.rerun(seed)
41 + run
42 + end
43 +
44 + defp job(run, job_id) do
45 + Repo.one!(from j in WorkflowJob, where: j.workflow_run_id == ^run.id and j.job_id == ^job_id)
46 + end
47 +
48 + defp status(run, job_id), do: job(run, job_id).status
49 + defp run_status(run), do: Repo.get!(WorkflowRun, run.id).status
50 +
51 + describe "after a failure" do
52 + setup %{repo: repo} do
53 + run =
54 + run_workflow(repo, """
55 + name: ci
56 + on: push
57 + jobs:
58 + build:
59 + runs-on: self-hosted
60 + steps:
61 + - run: false
62 + deploy:
63 + needs: build
64 + runs-on: self-hosted
65 + steps:
66 + - run: true
67 + notify:
68 + needs: build
69 + if: always()
70 + runs-on: self-hosted
71 + steps:
72 + - run: true
73 + report:
74 + needs: build
75 + if: ${{ failure() }}
76 + runs-on: self-hosted
77 + steps:
78 + - run: true
79 + tidy:
80 + needs: build
81 + if: "!cancelled()"
82 + runs-on: self-hosted
83 + steps:
84 + - run: true
85 + """)
86 +
87 + {:ok, run: run}
88 + end
89 +
90 + test "a plain dependent is skipped, not cancelled", %{run: run} do
91 + {:ok, _} = Workflows.update_job_status(job(run, "build"), "failure")
92 +
93 + assert status(run, "deploy") == "skipped"
94 + end
95 +
96 + test "dependents whose `if:` calls a status function are queued with the real result",
97 + %{run: run} do
98 + {:ok, _} = Workflows.update_job_status(job(run, "build"), "failure")
99 +
100 + for id <- ~w(notify report tidy) do
101 + dependent = job(run, id)
102 + assert dependent.status == "queued"
103 + assert %{"build" => %{"status" => "failure"}} = dependent.needs_snapshot
104 + end
105 + end
106 +
107 + test "the run fails once the queued dependents finish", %{run: run} do
108 + {:ok, _} = Workflows.update_job_status(job(run, "build"), "failure")
109 + assert run_status(run) in ~w(queued running)
110 +
111 + for id <- ~w(notify report tidy),
112 + do: {:ok, _} = Workflows.update_job_status(job(run, id), "success")
113 +
114 + assert run_status(run) == "failure"
115 + end
116 + end
117 +
118 + test "a skipped need skips a dependent without a status function", %{repo: repo} do
119 + run =
120 + run_workflow(repo, """
121 + name: ci
122 + on: push
123 + jobs:
124 + build:
125 + runs-on: self-hosted
126 + steps:
127 + - run: true
128 + deploy:
129 + needs: build
130 + runs-on: self-hosted
131 + steps:
132 + - run: true
133 + """)
134 +
135 + {:ok, _} = Workflows.update_job_status(job(run, "build"), "skipped")
136 +
137 + assert status(run, "deploy") == "skipped"
138 + # Skipped jobs don't fail a run.
139 + assert run_status(run) == "success"
140 + end
141 +
142 + test "a failure skips a whole chain", %{repo: repo} do
143 + run =
144 + run_workflow(repo, """
145 + name: ci
146 + on: push
147 + jobs:
148 + a:
149 + runs-on: self-hosted
150 + steps:
151 + - run: false
152 + b:
153 + needs: a
154 + runs-on: self-hosted
155 + steps:
156 + - run: true
157 + c:
158 + needs: [a, b]
159 + runs-on: self-hosted
160 + steps:
161 + - run: true
162 + """)
163 +
164 + {:ok, _} = Workflows.update_job_status(job(run, "a"), "failure")
165 +
166 + assert status(run, "b") == "skipped"
167 + assert status(run, "c") == "skipped"
168 + assert run_status(run) == "failure"
169 + end
170 +
171 + test "a dependent waits for every need before it's decided", %{repo: repo} do
172 + run =
173 + run_workflow(repo, """
174 + name: ci
175 + on: push
176 + jobs:
177 + a:
178 + runs-on: self-hosted
179 + steps:
180 + - run: false
181 + b:
182 + runs-on: self-hosted
183 + steps:
184 + - run: true
185 + c:
186 + needs: [a, b]
187 + if: always()
188 + runs-on: self-hosted
189 + steps:
190 + - run: true
191 + """)
192 +
193 + {:ok, _} = Workflows.update_job_status(job(run, "a"), "failure")
194 + assert status(run, "c") == "blocked"
195 +
196 + {:ok, _} = Workflows.update_job_status(job(run, "b"), "success")
197 + assert status(run, "c") == "queued"
198 + end
199 +
200 + describe "continue-on-error" do
201 + test "a failure doesn't hold back dependents or fail the run", %{repo: repo} do
202 + run =
203 + run_workflow(repo, """
204 + name: ci
205 + on: push
206 + jobs:
207 + build:
208 + continue-on-error: true
209 + runs-on: self-hosted
210 + steps:
211 + - run: false
212 + deploy:
213 + needs: build
214 + runs-on: self-hosted
215 + steps:
216 + - run: true
217 + """)
218 +
219 + {:ok, _} = Workflows.update_job_status(job(run, "build"), "failure")
220 +
221 + deploy = job(run, "deploy")
222 + assert deploy.status == "queued"
223 + assert %{"build" => %{"status" => "success"}} = deploy.needs_snapshot
224 +
225 + {:ok, _} = Workflows.update_job_status(deploy, "success")
226 +
227 + assert status(run, "build") == "failure"
228 + assert run_status(run) == "success"
229 + end
230 +
231 + test "is read through a matrix value, and spares the leg's siblings fail-fast", %{
232 + repo: repo
233 + } do
234 + run =
235 + run_workflow(repo, """
236 + name: ci
237 + on: push
238 + jobs:
239 + build:
240 + strategy:
241 + matrix:
242 + experimental: [true, false]
243 + continue-on-error: ${{ matrix.experimental }}
244 + runs-on: self-hosted
245 + steps:
246 + - run: true
247 + """)
248 +
249 + [experimental, stable] =
250 + from(j in WorkflowJob, where: j.workflow_run_id == ^run.id, order_by: j.id)
251 + |> Repo.all()
252 + |> Enum.sort_by(&(&1.matrix["experimental"] != true))
253 +
254 + assert Workflows.effective_result(%{experimental | status: "failure"}) == "success"
255 + assert Workflows.effective_result(%{stable | status: "failure"}) == "failure"
256 +
257 + {:ok, _} = Workflows.update_job_status(experimental, "failure")
258 + assert Repo.get!(WorkflowJob, stable.id).status == "queued"
259 + end
260 + end
261 +
262 + test "a matrix need with one failed leg reports failure", %{repo: repo} do
263 + run =
264 + run_workflow(repo, """
265 + name: ci
266 + on: push
267 + jobs:
268 + build:
269 + strategy:
270 + fail-fast: false
271 + matrix:
272 + arch: [amd64, arm64]
273 + runs-on: self-hosted
274 + steps:
275 + - run: true
276 + report:
277 + needs: build
278 + if: always()
279 + runs-on: self-hosted
280 + steps:
281 + - run: true
282 + """)
283 +
284 + [amd, arm] =
285 + Repo.all(
286 + from j in WorkflowJob,
287 + where: j.workflow_run_id == ^run.id and j.job_id == "build",
288 + order_by: j.id
289 + )
290 +
291 + {:ok, _} = Workflows.update_job_status(amd, "success")
292 + {:ok, _} = Workflows.update_job_status(arm, "failure")
293 +
294 + assert %{"build" => %{"status" => "failure"}} = job(run, "report").needs_snapshot
295 + end
296 +
297 + test "re-running a skipped dependent whose need still failed skips it again", %{repo: repo} do
298 + run =
299 + run_workflow(repo, """
300 + name: ci
301 + on: push
302 + jobs:
303 + build:
304 + runs-on: self-hosted
305 + steps:
306 + - run: false
307 + deploy:
308 + needs: build
309 + runs-on: self-hosted
310 + steps:
311 + - run: true
312 + """)
313 +
314 + {:ok, _} = Workflows.update_job_status(job(run, "build"), "failure")
315 + assert {:ok, _} = Workflows.rerun_job(Repo.get!(WorkflowRun, run.id), "deploy")
316 +
317 + assert status(run, "deploy") == "skipped"
318 + assert run_status(run) == "failure"
319 + end
320 +
321 + test "a cancelled run cancels blocked jobs, `always()` or not", %{repo: repo} do
322 + run =
323 + run_workflow(repo, """
324 + name: ci
325 + on: push
326 + jobs:
327 + build:
328 + runs-on: self-hosted
329 + steps:
330 + - run: true
331 + notify:
332 + needs: build
333 + if: always()
334 + runs-on: self-hosted
335 + steps:
336 + - run: true
337 + """)
338 +
339 + {:ok, _} = Workflows.cancel(Repo.get!(WorkflowRun, run.id))
340 +
341 + assert status(run, "build") == "cancelled"
342 + assert status(run, "notify") == "cancelled"
343 + assert run_status(run) == "cancelled"
344 + end
345 +
346 + describe "status_function_if?/1" do
347 + test "spots the status functions, negated or wrapped" do
348 + for condition <- [
349 + "always()",
350 + "${{ failure() }}",
351 + "!cancelled()",
352 + "success() || failure ( )",
353 + "always() && needs.build.outputs.x == '1'"
354 + ] do
355 + assert Workflows.status_function_if?(%{"if" => condition}), condition
356 + end
357 + end
358 +
359 + test "ignores everything else" do
360 + for definition <- [
361 + %{},
362 + %{"if" => true},
363 + %{"if" => "success()"},
364 + %{"if" => "github.ref == 'refs/heads/main'"},
365 + %{"if" => "needs.always_on.result == 'success'"}
366 + ] do
367 + refute Workflows.status_function_if?(definition), inspect(definition)
368 + end
369 + end
370 + end
371 +end
modified test/git_gud/workflows/matrix_dispatch_test.exs
+2 −2
@@ -317,13 +317,13 @@ defmodule GitGud.Workflows.MatrixDispatchTest do
317 317 assert Repo.get!(WorkflowJob, release.id).status == "queued"
318 318 end
319 319
320 − test "one failing leg cancels the downstream job", %{run: run} do
320 + test "one failing leg skips the downstream job", %{run: run} do
321 321 [amd, arm, release] = jobs(run)
322 322
323 323 {:ok, _} = Workflows.update_job_status(amd, "success")
324 324 {:ok, _} = Workflows.update_job_status(arm, "failure")
325 325
326 − assert Repo.get!(WorkflowJob, release.id).status == "cancelled"
326 + assert Repo.get!(WorkflowJob, release.id).status == "skipped"
327 327 end
328 328
329 329 test "the snapshot collapses the legs into one name", %{run: run} do
modified test/git_gud_web/controllers/runner_controller_test.exs
+65 −2
@@ -435,6 +435,67 @@ defmodule GitGudWeb.RunnerControllerTest do
435 435 end
436 436 end
437 437
438 + describe "needs and results" do
439 + setup ctx do
440 + {:ok, runner, token} =
441 + GitGud.Workflows.Runners.register(%{
442 + name: "graph-runner",
443 + version: "v1",
444 + labels: %{"labels" => ["ubuntu-latest"]}
445 + })
446 +
447 + Map.merge(ctx, %{runner: runner, runner_token: token})
448 + end
449 +
450 + test "FetchTask sends each need's result and outputs", %{conn: conn, runner_token: token} do
451 + snapshot = %{
452 + "build" => %{"status" => "success", "outputs" => %{"x" => "1"}},
453 + "lint" => %{"status" => "failure", "outputs" => %{}}
454 + }
455 +
456 + {task, _run} =
457 + fetch_task(conn, token, %{trigger: "push"}, fn _ -> :ok end, needs_snapshot: snapshot)
458 +
459 + assert %Runner.V1.TaskNeed{result: :RESULT_SUCCESS, outputs: %{"x" => "1"}} =
460 + task.needs["build"]
461 +
462 + assert %Runner.V1.TaskNeed{result: :RESULT_FAILURE, outputs: outputs} = task.needs["lint"]
463 + assert outputs == %{}
464 + end
465 +
466 + test "FetchTask sends no needs for a job without any", %{conn: conn, runner_token: token} do
467 + {task, _run} = fetch_task(conn, token, %{trigger: "push"})
468 + assert task.needs == %{}
469 + end
470 +
471 + test "UpdateTask with RESULT_SKIPPED records the job as skipped", %{
472 + conn: conn,
473 + runner_token: token
474 + } do
475 + {task, _run} = fetch_task(conn, token, %{trigger: "push"})
476 +
477 + body =
478 + %Runner.V1.UpdateTaskRequest{
479 + state: %Runner.V1.TaskState{id: task.id, result: :RESULT_SKIPPED}
480 + }
481 + |> Runner.V1.UpdateTaskRequest.encode()
482 + |> IO.iodata_to_binary()
483 +
484 + resp =
485 + conn
486 + |> put_req_header("content-type", "application/protobuf")
487 + |> put_req_header("authorization", "Bearer " <> token)
488 + |> post("/api/actions/_apis/runner.v1.RunnerService/UpdateTask", body)
489 +
490 + assert resp.status == 200
491 +
492 + assert %Runner.V1.UpdateTaskResponse{state: %{result: :RESULT_SKIPPED}} =
493 + Runner.V1.UpdateTaskResponse.decode(resp.resp_body)
494 +
495 + assert GitGud.Repo.get!(GitGud.Workflows.WorkflowJob, task.id).status == "skipped"
496 + end
497 + end
498 +
438 499 # Queue one job on a run built from `run_attrs`, let the runner fetch it,
439 500 # and return the task's context as a plain map.
440 501 defp fetch_context(conn, token, run_attrs) do
@@ -443,8 +504,9 @@ defmodule GitGudWeb.RunnerControllerTest do
443 504 end
444 505
445 506 # The same, returning `{task, run}`. `prepare` gets the repository before
446 − # the fetch (to plant secrets, say).
447 − defp fetch_task(conn, token, run_attrs, prepare \\ fn _repo -> :ok end) do
507 + # the fetch (to plant secrets, say); `job_changes` are set on the queued
508 + # job (fields `create_changeset/2` doesn't cast, like `needs_snapshot`).
509 + defp fetch_task(conn, token, run_attrs, prepare \\ fn _repo -> :ok end, job_changes \\ []) do
448 510 user = GitGud.AccountsFixtures.user_fixture()
449 511
450 512 {:ok, repo} =
@@ -474,6 +536,7 @@ defmodule GitGudWeb.RunnerControllerTest do
474 536 runs_on: %{"labels" => ["ubuntu-latest"]},
475 537 definition: %{"runs-on" => "ubuntu-latest", "steps" => [%{"run" => "true"}]}
476 538 })
539 + |> Ecto.Changeset.change(job_changes)
477 540 |> GitGud.Repo.insert!()
478 541
479 542 body = FetchTaskRequest.encode(%FetchTaskRequest{tasks_version: 0}) |> IO.iodata_to_binary()
modified test/git_gud_web/protos/runner_wire_test.exs
+12 −0
@@ -79,6 +79,18 @@ defmodule GitGudWeb.Protos.RunnerWireTest do
79 79 assert bytes == <<0x10, 0x01>>
80 80 end
81 81
82 + test "TaskNeed: outputs ride on field 1 as map entries, result on field 2" do
83 + bytes =
84 + %TaskNeed{result: :RESULT_FAILURE, outputs: %{"x" => "1"}}
85 + |> TaskNeed.encode()
86 + |> IO.iodata_to_binary()
87 +
88 + entry = str(1, "x") <> str(2, "1")
89 + assert bytes == str(1, entry) <> <<0x10, 0x02>>
90 +
91 + assert %TaskNeed{result: :RESULT_FAILURE, outputs: %{"x" => "1"}} = TaskNeed.decode(bytes)
92 + end
93 +
82 94 defp decode_varint(<<0::1, n::7, rest::binary>>), do: {n, rest}
83 95
84 96 defp decode_varint(<<1::1, n::7, rest::binary>>) do

Parents: e9900fa