11.2 KiB · text History 95a46eb

Plan: give a job the context GitHub’s docs promise it

Status: ready to execute. Written to be carried out start to finish by one agent without further questions. Do it before workflow-triggers.md (whose workflow_dispatch inputs travel in the event payload built here) and before workflow-job-graph.md (which touches the same build_task/1).

Why

What a job sees is assembled in GitGudWeb.RunnerController.build_task/1, build_context_struct/4 and GitGud.Workflows.RunnerPayload.encode/1. Each of the following reads as empty or missing, without an error, in a workflow written from GitHub’s or Forgejo’s documentation:

  1. secrets.GITHUB_TOKEN is empty. build_task/1 sends GitGud.Secrets.resolve_for(repo) — the user’s secrets only. Forgejo puts the job token into the task’s secrets as GITHUB_TOKEN, GITEA_TOKEN and FORGEJO_TOKEN (forgejo-runner’s internal/app/run/runner.go reads task.Secrets["FORGEJO_TOKEN"], then GITHUB_TOKEN, to override the context token). ${{ secrets.GITHUB_TOKEN }} is the most common way workflows log in to a registry or call the API. (${{ github.token }} already works: context key token.)
  2. Workflow-level env: and defaults: are dropped. RunnerPayload.encode/1 rebuilds the document from name, a hard-coded "on" => "push", and the one job. A top-level env: {FOO: bar} or defaults: {run: {working-directory: app}} never reaches the runner, so steps run with FOO unset and in the wrong directory. The real on: is also lost, which is where act finds workflow_dispatch input defaults.
  3. github.event is an empty object. forgejo-runner reads taskContext["event"].GetStructValue().AsMap(); gitgud sends the trigger name as a string, so the map is empty. Everything under github.event.* is blank: github.event.pull_request.number, github.event.head_commit.message, github.event.before, github.event.inputs.*, and act’s inputs context for workflow_dispatch (it reads event["inputs"]).
  4. github.run_id, github.run_attempt, github.workflow_ref are empty; also absent: server_url, api_url, repository_id, actor_id, repository_owner_id, forgejo_default_actions_url. run_id is what actions/upload-artifact@v3 builds its URL from (see workflow-artifacts-and-cache.md) and what people use to make unique tags; act substitutes a default when it’s empty, so collisions are silent.

Semantics to match

Forgejo’s server is the reference, because forgejo-runner is the client: services/actions/context.go (GenerateGiteaContext) and routers/api/actions/runner/utils.go (getSecretsOfTask) in codeberg.org/forgejo/forgejo. Keys and meanings:

key value
event the event payload object (below)
event_name trigger (already sent)
run_id the run’s id (workflow_runs.id), as a string
run_number run.number (already sent)
run_attempt the job’s attempt, "1" first time, incremented by re-running the job
workflow_ref owner/repo/.forgejo/workflows/ci.yml@refs/heads/main (path + @ + the run’s ref)
server_url GitGudWeb.Endpoint.url()
api_url GitGudWeb.Endpoint.url() <> "/api/v1"
repository_id, repository_owner_id, actor_id ids as strings
forgejo_default_actions_url same value as gitea_default_actions_url
retention_days "" (Forgejo sends empty too)

Secrets: GITHUB_TOKEN, GITEA_TOKEN, FORGEJO_TOKEN = the job token, added after the user’s secrets so a user secret of the same name can’t replace it.

Event payloads — GitHub’s webhook shapes, trimmed to the fields workflows actually read. Every payload has repository (id, name, full_name = owner/name, owner: {login}, default_branch, private, html_url, clone_url) and sender (login of the triggering user, or the repo owner’s handle when unattributed — the same rule as actor).

  • push: ref, before (old sha hex, 40 zeros for a new ref), after, created, deleted (false), forced (false — not tracked; say so in a comment), base_ref (null), compare (URL of the compare page if one exists, else the commit URL), commits (up to 20: id, message, timestamp, url, author: {name, email}, committer: {name, email}), head_commit (same shape, the pushed head), pusher: {name, email}.
  • pull_request: action, number, pull_request: {number, title, body, state: "open", html_url, draft: false, merged: false, user: {login}, head: {ref, sha, label: "owner:ref", repo: <repository>}, base: {ref, sha (target head), label, repo}}.
  • workflow_dispatch: inputs (string→string), ref, workflow (the path).

Where things are

Task assembly lib/git_gud_web/controllers/runner_controller.ex: build_task/1, build_context_struct/4, wrap_as_struct/1 (all values become string_value today), pull_request_context/1
Workflow payload lib/git_gud/workflows/runner_payload.ex encode/1; the run’s whole parsed document is run.workflow_doc
Runs lib/git_gud/workflows.ex: dispatch/2, dispatch_pull_request/3, dispatch_manual/4, create_run/2, rerun/1, rerun_job/2; schema lib/git_gud/workflows/workflow_run.ex, workflow_job.ex
Push data lib/git_gud/repositories/workers/commit_backfill.ex (perform/1 has old_sha/new_sha, push_payload/5 already builds a smaller webhook payload, publish_push/4 lists commits)
Secrets lib/git_gud/secrets.ex resolve_for/1; lib/git_gud/secrets/reserved_names.ex
Tests test/git_gud_web/controllers/runner_controller_test.exs has fetch_context/3 (queues a job, fetches the task, returns its string context) — extend it to return the whole Task; test/git_gud/workflows/*_dispatch_test.exs for on-disk dispatch

Steps

1. Store the event payload on the run

Migration (mix ecto.gen.migration add_event_payload_to_workflow_runs): workflow_runs.event_payload :map (jsonb, nullable), and workflow_jobs.attempt :integer, null: false, default: 1. Run it against dev and test. Cast event_payload in WorkflowRun.create_changeset/2.

New module GitGud.Workflows.EventPayload (lib/git_gud/workflows/event_payload.ex) with pure builders: push(repo, %{ref:, before:, after:, commits:, pusher:, sender:}), pull_request(repo, pr, action, %{target_sha:, sender:}), workflow_dispatch(repo, ref, workflow_path, inputs, sender), and a shared repository(repo). Keys are strings. Shas are lowercase hex.

Wire it in:

  • Workflows.dispatch/2: build the push payload. The event gains optional :commits (list of Git.commit() maps) — compute them in dispatch/2 itself with Repositories.list_commits(repo, head_sha, limit: 20), stopping at base_sha when it is present (stop when a commit’s sha equals base; good enough — say so in a comment). The pusher is triggered_by_id (load the user once).
  • dispatch_pull_request/3: the PR payload; the target sha is what merge_base_with already resolves the target ref to — resolve it once and reuse.
  • dispatch_manual/4: gains an inputs argument (map, default %{}) — only to carry it into the payload; the triggers plan wires the API.
  • rerun/1: copy event_payload.
  • A run created before this migration has event_payload: nil: the task then sends an empty object (today’s behaviour).

2. Send typed values in the context

wrap_as_struct/1 makes every value a string_value. Add a to_value/1 that maps: binary → string_value; integer/float → number_value; boolean → bool_value; nil → null_value; map → struct_value (recursive %Google.Protobuf.Struct{}); list → list_value (%Google.Protobuf.ListValue{values: …}). Check the generated modules under lib/git_gud_web/protos / deps for the exact struct and oneof names before writing it. Every existing key stays a string; only event is a struct.

3. The context keys

In build_context_struct/4:

  • "event" => run.event_payload || %{}
  • the keys from the table above. run_attempt comes from the job (job.attempt). workflow_ref uses run.workflow_path and run.head_ref.
  • keep "event_name" and the PR overrides from pull_request_context/1.

rerun_job/2: increment the job’s attempt when it resets the job. A whole rerun/1 creates a fresh run (new run_id), so its jobs start at 1 — GitHub keeps the run id and bumps the attempt; note the difference in a comment, don’t restructure reruns here.

4. The job token as a secret

In build_task/1, after resolve_for/1: secrets = Map.merge(secrets, %{"GITHUB_TOKEN" => token, "GITEA_TOKEN" => token, "FORGEJO_TOKEN" => token}). lib/git_gud/secrets/reserved_names.ex already refuses user secrets with the GITHUB_, GITEA_ and FORGEJO_ prefixes, so no user secret should collide; merging the token last still guards rows created before that rule.

5. The whole workflow, one job

RunnerPayload.encode/1: start from run.workflow_doc, keep its top-level env, defaults, permissions, run-name and on (the real one), and replace jobs with the one job (needs still dropped, as today — the job-graph plan restores needs through Task.needs). Fallback for workflow_doc == nil (old runs): today’s document. Drop concurrency (the server decides that; see workflow-cancellation-and-concurrency.md).

Before committing, check with a real forgejo-runner if one is reachable (the .forgejo/workflows/build.yml runner): a workflow with top-level env: and defaults.run.working-directory and a step that prints them. If none is reachable, say so in the commit message.

6. Docs

README CI list: a line saying which github.* keys and payload fields a job gets, that secrets.GITHUB_TOKEN is the job token, and that forced is always false.

Tests

  • runner_controller_test.exs: for a push run with an event_payload, the task context has event as a struct whose head_commit.message, before, after, repository.full_name round-trip; run_id == to_string(run.id), run_attempt == "1", workflow_ref, server_url, api_url; task.secrets["GITHUB_TOKEN"] == context["token"]; a user secret named GITHUB_TOKEN (inserted directly, bypassing the changeset) does not win.
  • PR run: event.pull_request.number, event.pull_request.head.sha, event.action.
  • RunnerPayload.encode/1 test (new file test/git_gud/workflows/runner_payload_test.exs): a doc with top-level env, defaults and on: {workflow_dispatch: {inputs: …}} keeps them; jobs has exactly the one job; needs dropped; concurrency dropped.
  • On-disk (path_dispatch_test.exs style): a push stores an event_payload whose commits stop at the base and whose head_commit.id is the pushed sha; a new branch’s before is 40 zeros.
  • rerun_job/2 bumps attempt; rerun/1 copies event_payload.

Done when

  • mix test passes; no new compile warnings (compare mix compile --force warning lists before/after — the repo has pre-existing ones); mix format only the files you touched.
  • One commit in the repository’s message style.
  • Not pushed.