pelagia-portal/automation/README.md
Hardik 12e6d16061 feat(automation): test DB mirror + dev-server env for autofix verification
- automation/refresh-test-db.sh: daily pg_dump of prod (pelagia) into a throwaway
  mirror (pelagia_test) on pms1; cron at 03:30. ~10MB, refresh ~1s.
- Autofix clone ~/pelagia-autofix/App/.env points DATABASE_URL at pelagia_test in
  safe dev mode (no Resend/SSO secrets -> console email, local storage), port 3100.
- Fix prompt: Claude may run integration tests against the test DB and start a dev
  server on port 3100 ONLY; stop it by port (fuser -k 3100/tcp), never broad pkill
  (production also runs a next-server on 3000).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 04:49:15 +05:30

138 lines
7.6 KiB
Markdown

# Automated issue-to-deploy pipeline
End-to-end flow from a user clicking **Report Issue** in the portal to a fix
running in production:
```
Portal header (bug icon) [App/components/layout/report-issue-button.tsx]
│ server action → Forgejo API
Forgejo issue (label: portal) [git.pelagiamarine.com/shad0w/pelagia-portal]
│ polled every 10 min by Windows Scheduled Task "PelagiaClaudeIssueWatcher"
TRIAGE (watcher phase 1) [dev PC, headless Claude Code, analysis only]
│ Claude reads the issue + repo, posts a requirements-breakdown comment,
│ and routes it: adds `claude-queue` (auto-fixable) or `interactive` (human)
FIX (watcher phase 2, only for claude-queue) [headless Claude Code in C:\...\src\pelagia-autofix]
│ Claude implements + verifies fix; watcher pushes branch claude/issue-N
│ and opens a PR (label: claude-pr)
Human review: merge the PR, then create a release tag vX.Y.Z
│ tag push triggers .forgejo/workflows/deploy.yml
forgejo-runner on pms1 (pm2: forgejo-runner, label "host")
│ checks out the tag in ~/pms, pnpm install + build + prisma migrate deploy
pm2 restart ppms → live at pms.pelagiamarine.com
```
`interactive`-routed issues stop after triage for a human to pick up (run with
Claude in a steered session). The triage breakdown comment is plain (no bot
marker) so, for `claude-queue` issues, the fix stage reads it back as refined
requirements.
## Components
| Piece | Where | Notes |
|---|---|---|
| Report Issue button | `App/components/layout/report-issue-button.tsx` + `report-issue-actions.ts` | Any signed-in user; files issue with only the `portal` label (triage routes it) |
| Forgejo helper | `App/lib/forgejo.ts` | Needs `FORGEJO_URL`, `FORGEJO_REPO`, `FORGEJO_TOKEN` env (token scope: `write:issue`) |
| Issue watcher (active) | `automation/claude-issue-watcher.sh` on pms1 | Bash port; runs 24/7 via cron. Config + logs under `~/issue-watcher/` |
| Issue watcher (Windows, disabled) | `automation/claude-issue-watcher.ps1` | PowerShell original. `PelagiaClaudeIssueWatcher` task is **disabled** (pms1 is the sole worker; two pollers would race) |
| Forgejo helper | `App/lib/forgejo.ts` | Needs `FORGEJO_URL`, `FORGEJO_REPO`, `FORGEJO_TOKEN` env (token scope: `write:issue`) |
| Deploy workflow | `.forgejo/workflows/deploy.yml` | Triggers on `v*` tags; runs on the `host` runner |
| Runner | pms1 `~/forgejo-runner`, pm2 process `forgejo-runner` | Registered as `pms1-host` with labels `host`, `docker` |
## Where the watcher runs (pms1)
The watcher runs on **pms1** under cron (every 10 min), polling Forgejo over the
local loopback (`http://127.0.0.1:3001`).
- Script: `~/issue-watcher/claude-issue-watcher.sh` (source: `automation/claude-issue-watcher.sh`)
- Config: `~/issue-watcher/watcher.config.json` (gitignored; holds the token + `claudeExe` = the nvm `claude` path)
- Work clone: `~/pelagia-autofix` (separate from the deployed `~/pms`)
- Logs: `~/issue-watcher/logs/` (`watcher-<date>.log`, per-issue `claude-*.log`, `cron.log`)
- Crontab: `*/10 * * * * PATH=<nvm bin>:... ~/issue-watcher/claude-issue-watcher.sh >> ~/issue-watcher/logs/cron.log 2>&1`
**Auth:** Claude Code must be signed in on pms1 (`ssh` in, run `claude`, complete
the login → writes `~/.claude/.credentials.json`). The watcher has a preflight that
no-ops until those credentials exist, so cron can be enabled before sign-in and
activates automatically once signed in. (An `ANTHROPIC_API_KEY` env var also satisfies it.)
The Windows variant (`.ps1` + `register-watcher-task.ps1`) is the portable fallback;
re-enable its task only if pms1 is unavailable, and disable one before enabling the other.
## Test database (for autofix verification)
So the fix stage can verify against realistic data without touching production:
- **`pelagia_test`** — a PostgreSQL database on pms1, owned by `pelagia_user`, that is
a **daily mirror of production** (`pelagia`). Created once as superuser; refreshed by
`automation/refresh-test-db.sh` via cron at **03:30** (`pg_dump pelagia | psql pelagia_test`).
- The autofix clone's `~/pelagia-autofix/App/.env` points `DATABASE_URL` at `pelagia_test`
and runs in **safe dev mode** — no Resend/SSO secrets, so email is console-logged and
storage is local. `NEXTAUTH_URL`/`PORT` are set to **3100** (production app is on 3000).
- The fix prompt tells Claude it may run integration tests against this DB
(`set -a; . ./.env; set +a; pnpm test:integration`) and may start a dev server on
**port 3100 only**, stopping it by port (`fuser -k 3100/tcp`) — never a broad `pkill next`,
which would take down production (it also runs a `next-server`).
Because the test DB is refreshed daily, anything the autofix writes to it (test data,
schema experiments) is disposable. Schema-migration issues are routed to `interactive`
by triage, so the unattended fixer should not be altering the schema anyway.
## Issue label lifecycle
```
portal ──(triage)──▶ claude-queue ─▶ claude-working ─▶ claude-pr | claude-failed
└────▶ interactive (stops here — handle with Claude interactively)
```
- A `portal` issue with no decision label yet is triaged once (`maxTriagePerRun`
per run). Triage adds `claude-queue` or `interactive` and posts a breakdown.
- `claude-queue``claude-working``claude-pr` (PR opened) or `claude-failed`.
- To retry a failed issue, re-add `claude-queue`.
- To queue any manually-created issue for Claude (skipping triage), add
`claude-queue` directly. To force human handling, add `interactive`.
- Triage is skipped for issues that already carry any decision label.
## Releasing
After merging a Claude PR (or any change) on `master`:
```powershell
git pull
git tag v0.2.0 # semver: bump patch for fixes, minor for features
git push pms1 master --tags
```
The runner deploys the tag and restarts the app. Watch progress under
**Actions** on the Forgejo repo, or `pm2 logs forgejo-runner` on pms1.
## Operational notes
- The watcher runs Claude Code with `--dangerously-skip-permissions` inside the
dedicated `pelagia-autofix` clone — never point `workDir` at your main checkout.
- Watcher only works issues while this PC is on; queued issues are picked up on
the next run after boot (`-StartWhenAvailable`).
- Tokens: `portal-report-issue` (write:issue, used by the app) and
`claude-watcher` (write:issue + write:repository, used by the watcher).
Both belong to the `shad0w` Forgejo account. Rotate via
`docker exec -u 1000 forgejo forgejo admin user generate-access-token ...`.
- Server-side env for the button lives in `~/pms/App/.env` on pms1
(`FORGEJO_URL=http://127.0.0.1:3001` so it does not depend on the tunnel).
- **Known Forgejo 10 bug:** clicking *Update branch* on a PR (or pushing to its
head branch) can make the page show "This pull request is broken due to
missing fork information" even though the PR is fine (API still reports
`mergeable: true`). Fix: close and reopen the PR — via the UI, or:
```powershell
$h = @{ Authorization = "token <claude-watcher token>" }
Invoke-RestMethod -Method Patch -Headers $h -ContentType application/json `
-Uri https://git.pelagiamarine.com/api/v1/repos/shad0w/pelagia-portal/pulls/<N> -Body '{"state":"closed"}'
Invoke-RestMethod -Method Patch -Headers $h -ContentType application/json `
-Uri https://git.pelagiamarine.com/api/v1/repos/shad0w/pelagia-portal/pulls/<N> -Body '{"state":"open"}'
```
Fixed upstream in newer Gitea/Forgejo — resolves itself if Forgejo is upgraded past v10.