Yayaw
Docs

Runbooks

Operational runbooks for common maintenance tasks.

Regenerate DB Actions

bun run generate:actions

Run this after Drizzle schema changes. Commit generated files with the schema change so TypeScript and authz contract tests see the same database action surface as runtime.

Change Production Domain

Yayaw production is canonically served from https://yayaw.app.

  1. Point the apex DNS records at the self-hosted Coolify/reverse-proxy ingress and verify TLS for yayaw.app.

  2. Configure www.yayaw.app at the reverse proxy to redirect to yayaw.app with status 308.

  3. Keep retired .eu domains only as reverse-proxy redirects with status 308: route yayaw.eu and www.yayaw.eu to yayaw.app. Companion projects should follow the same pattern, for example table.yayaw.eu to table.yayaw.app.

  4. Set production environment variables:

NEXT_PUBLIC_BASE_URL=https://yayaw.app
BETTER_AUTH_TRUSTED_ORIGINS=https://*.yayaw.app

  1. Keep a durable preview Git branch, route preview.yayaw.app to its branch-backed Coolify application, and set preview NEXT_PUBLIC_BASE_URL:

NEXT_PUBLIC_BASE_URL=https://preview.yayaw.app

  1. Do not keep retired preview hosts as active preview domains after preview.yayaw.app is verified.

  2. Redeploy the latest Coolify production and preview applications so builds receive the public base URL.

  3. Update third-party callbacks and webhooks that store absolute origins, such as OAuth providers, Stripe webhooks, billing portal return URLs, email links, and analytics/site settings.

Deploy Self-Hosted Runtime

  1. Copy the example file and fill production secrets outside git:

cp .env.self-host.example .env.self-host
perl -0pi -e "s/^BETTER_AUTH_SECRET=$/BETTER_AUTH_SECRET=$(openssl rand -hex 32)/m" .env.self-host

  1. Run the schema sync and seed setup:

docker compose --env-file .env.self-host -f docker-compose.self-host.yml --profile setup run --rm migrate

  1. Start or update the runtime:

docker compose --env-file .env.self-host -f docker-compose.self-host.yml up --build -d

  1. Confirm Caddy can reach the app and that Host/X-Forwarded-* headers are preserved by opening the configured DEPLOYMENT_URL.

  2. Upload a media asset and confirm it is served from the configured STORAGE_PUBLIC_BASE_URL.

  3. Confirm bun run worker:page-ai is running through the compose worker service when PAGE_AI_QUEUE_DRIVER=db-worker.

  4. Back up Postgres and object storage together before every risky release.

Cut Over an Existing Database to Better Auth 1.7

Migration 0050_better_auth_1_7_stable is additive for rollback, but it changes account identity and adds uniqueness constraints. The migration process refuses to run it against an existing database until the operator attests that auth writes are actually paused. BETTER_AUTH_17_WRITES_PAUSED does not block a single request by itself and must never be used as a substitute for ingress or process isolation.

  1. Rehearse on a recent production restore. First expose the additive preparation columns, then run the preflight and resolve every blocker before scheduling production:

DATABASE_URL=<restored-clone-url> \
  bun src/lib/scripts/auth/better-auth-1-7-prepare.ts
DATABASE_URL=<restored-clone-url> \
  bun src/lib/scripts/auth/better-auth-1-7-preflight.ts

The preparation command is idempotent and migration 0050 repeats the same IF NOT EXISTS columns. Run it on production before the outage window so every existing client_credentials client can receive an explicitly reviewed client_credentials_scopes allowlist and, when ownership is ambiguous, an organization reference_id. Allowed machine scopes are only control-plane:read, control-plane:write, control-plane:publish, and control-plane:admin; never copy openid, profile, email, offline access, or another user-delegated scope. Existing client IDs and secrets remain unchanged. The deployment migration path reruns preparation idempotently before the blocking preflight.

  1. Immediately before cutover, take a new Postgres snapshot or pg_dump and prove that it can be restored. Record the immutable app and worker image tags used for rollback.

  2. Block every external method on /api/auth at the ingress with a maintenance response. Ensure no alternate public origin can bypass that rule. On a single-host Compose deployment, stopping the old app service is an acceptable broader outage. Do not set the attestation yet.

  3. Verify the block from outside the deployment network. For an ingress rule, this harmless invalid request must return the maintenance status, normally 503, rather than a Better Auth 2xx or validation 4xx:

curl --include --request POST \
  --header 'content-type: application/json' \
  --data '{}' \
  https://yayaw.app/api/auth/sign-in/email

  1. Only after the backup and external block are verified, set BETTER_AUTH_17_WRITES_PAUSED=true on the migration process. It is a short-lived, non-secret operator attestation:

# Self-host: set the value temporarily in the untracked .env.self-host file.
docker compose --env-file .env.self-host \
  -f docker-compose.self-host.yml --profile setup run --rm migrate

For the Coolify rolling runner, set the GitHub production environment variable BETTER_AUTH_17_WRITES_PAUSED to true, then run or rerun the deployment. The runner transfers only the normalized boolean to its private, mode-600 migration env file and does not print it. The guard distinguishes a fresh database, a database where 0050 is already applied, and an existing pending database. Both existing states execute the full Better Auth data preflight before db:migrate. The pending state requires the attestation; an applied database also requires it when the currently serving app does not advertise the stable-1.7 readiness contract, which closes a retry after a deliberate beta rollback. For the legacy docker-compose.coolify.yml resource, set the same temporary non-secret Compose variable in Coolify only after the external block is live, then remove it after the migration service completes. 6. Deploy the Better Auth server and UI together, then run the critical auth, OAuth, SCIM, Stripe, invitation, and passkey smoke tests while the external write block remains in place for normal users. The rolling script verifies the anonymous Better Auth session endpoint from inside the newly deployed app container while paused; it never weakens the public /api/auth rule for this smoke. 7. Immediately set BETTER_AUTH_17_WRITES_PAUSED=false or remove the variable from the migration environment, then remove the ingress block and reopen auth traffic. Leaving the attestation enabled is an operational error. 8. If application rollback is required, redeploy the old server and UI together without reverting migration 0050. Disable the enable-scim-plugin flag before the beta server boots: migration 0050 preserves the empty beta SCIM tables under rollback-only names, while the stable tables at the public names have an intentionally incompatible shape. Before another upgrade attempt, pause auth writes again, replay the reviewed issuer/account-key backfill from migration 0050, verify every machine OAuth client still has a user owner and organization membership reference, and rerun the preflight. The beta may have created accounts with a null issuer or OAuth clients with incomplete additive metadata during the rollback window. Every later deployment with 0050 already recorded still runs the preflight and fails closed while repair work remains.

After migration 0050, the Coolify runner refuses to automatically boot a previous image that does not advertise the stable Better Auth 1.7 readiness contract. A failed first cutover therefore stops for operator intervention; keep the external /api/auth block active. For a deliberate beta restore, set enable-scim-plugin=false in both the managed database flag and any remote flag provider, restart beta behind the ingress block, and verify from the private network that /api/auth/scim/v2/ServiceProviderConfig is unavailable before reopening traffic. Automatic rollback becomes available again once the previous live image advertises the stable-1.7 rollback contract.

Fresh databases and clean databases that already record migration 0050 while a stable-1.7 app is serving do not require the one-time write-pause attestation. An applied database behind a beta or unknown runtime requires the pause and the external ingress proof again. Migration-history drift, partial schema state, or rollback-created account repair work always fails closed and must be resolved on a restored clone first.

Update Seeded Feature Flags

  1. Edit src/lib/scripts/seed.ts.

  2. Apply the seed:

bun run seed

  1. Re-run app quality checks:

bun run check
bunx tsc --noEmit
bun run build

Retire the legacy Table documentation mirror

Public documentation is served from the bilingual documentation-page catalog at /[locale]/docs/*. The former Page-CMS pages under /table/docs are obsolete: permanent redirects preserve incoming links. Never recreate or republish this mirror with migrate-table-pages-to-cms.ts.

To retire an existing mirror safely:

  1. Inventory only global pages whose path is /table/docs or starts with /table/docs/. Keep the marketing /table and example /table/example pages.

  2. Verify every destination under /en/docs/table and /fr/docs/table is published and that each old URL redirects to its replacement.

  3. Review each page's latest and published revision IDs. A newer editorial draft needs review before retirement; do not discard it automatically.

  4. Archive the reviewed page through yayaw_pages_archive, first with dryRun, then with its exact expectedRevisionId, confirm: true, and a reason. Archiving retains the immutable revision history.

  5. Recheck redirects and the canonical documentation after archiving. Keep the archive audit receipts. Do not clear or delete canonical documentation entries.

For current documentation, use /dashboard/content/documentation to review English and French together, resolve validation errors and publish the reviewed revision. Draft status alone is not evidence that a page is obsolete.

Maintain the Yayaw Table Fumadocs Fallback Snapshot

Fumadocs renders the published bilingual Documentation catalog from Postgres at /[locale]/docs/table/*. The files in content/docs/{en,fr}/table remain only as the initial seed and explicit rollback/fallback snapshot; they are not the routine editorial source after the database cutover. Super Admins manage live English and French content from /dashboard/content/documentation.

When intentionally refreshing the fallback snapshot, update both directories, use relative links such as ./setup, and keep the English and French meta.json files in the same order. Run bun run docs:check-translations and bun run docs:check-links, then use the explicit Documentation import command only when the reviewed snapshot must replace untouched Git-imported revisions.

Validate Billing Webhooks in Staging

  1. Better Auth Stripe plugin webhook:

stripe listen --forward-to https://<staging-domain>/api/auth/stripe/webhook

  1. Custom one-time webhook:

stripe listen --forward-to https://<staging-domain>/api/billing/stripe/webhook

  1. Trigger events:

stripe trigger checkout.session.completed
stripe trigger customer.subscription.updated
stripe trigger customer.subscription.deleted
stripe trigger invoice.payment_failed
stripe trigger invoice.paid

invoice.payment_failed and invoice.paid must reach the Better Auth Stripe plugin endpoint at /api/auth/stripe/webhook. The custom one-time endpoint at /api/billing/stripe/webhook intentionally processes checkout.session.completed only.

Replay Failed One-Time Webhooks

When one-time webhook processing fails due to a transient issue, replay failed events:

bun run billing:replay-webhooks

Optional limit:

bun run billing:replay-webhooks -- --limit=20

Regenerate Fumadocs Source Artifacts

bun run docs:generate

Regenerate LLM Assistant Files

bun run docs:llm:generate

Run this after changing content/llm/llm-source.md. Do not edit AGENTS.md, GEMINI.md, or .github/copilot-instructions.md manually.

Run a Full English Docs Pass

Use this when making broad documentation changes:

bun run docs:generate
bun run docs:check-links
bun run docs:check-translations
bun run docs:llm:generate
bun run docs:llm:check
bun run check

If the docs change affects routes, imports, or MDX rendering, also run:

bunx tsc --noEmit
bun run build

Keep English docs canonical. When localized documentation is in scope, update the French mirror in the same PR and keep docs:check-translations passing.

Manage Production MCP Keys

Issue a production MCP key for a user:

bun run mcp:key -- issue --email [email protected] --permissions read,write,publish,admin

Revoke a production MCP key:

bun run mcp:key -- revoke --key-id <api-key-id>

Rotate keys by issuing the replacement first, updating the client secret, verifying yayaw_status, then revoking the old key.

Validate Documentation Integrity

bun run docs:check-links
bun run docs:check-translations
bun run docs:llm:check

If docs:llm:check fails, regenerate with bun run docs:llm:generate and review the generated assistant-file diff before committing.

Verify GitHub Code Access

  1. Confirm /dashboard/admin/billing-settings has GitHub repository access enabled.

  2. Confirm repository, GitHub App ID, and installation ID are present.

  3. Confirm BILLING_CODE_ACCESS_GITHUB_APP_PRIVATE_KEY exists in the target deployment environment.

  4. Use a paid test organization with code-access:read.

  5. Submit a GitHub username from /dashboard/organization/code-access.

  6. Confirm code_access_github_accounts records the invitation or active access state.

  7. Confirm GitHub shows the repository invitation with read-only access.

For staging without a GitHub App, use BILLING_CODE_ACCESS_GITHUB_TOKEN only as an explicit temporary fallback.

Verify Media Storage

  1. Confirm STORAGE_PROVIDER=s3 is configured with S3/MinIO keys and STORAGE_PUBLIC_BASE_URL.

  2. Upload an image from /dashboard/content/media.

  3. Confirm a media_assets row is created for the active organization.

  4. Confirm the public URL loads.

  5. Open the page editor and bind the asset to an image field.

  6. Publish the page and confirm the public page renders the selected asset.

  7. If thumbnails are enabled for the asset type, confirm a thumbnail URL is created or a non-blocking thumbnail failure is recorded.

Verify Manual Public Domain DNS

  1. Set PUBLIC_DOMAIN_PROVIDER=manual-dns.

  2. Configure PUBLIC_DOMAIN_CNAME_TARGET or PUBLIC_DOMAIN_IPV4_TARGETS.

  3. Add a domain from organization settings or yayaw_org_domain_add.

  4. Publish the TXT challenge shown in the dashboard.

  5. Point the hostname to the reverse proxy.

  6. Run the check/verify action and confirm the domain becomes verified.

  7. Open the custom host and confirm public pages render while /dashboard, /auth, /api, /docs, /o, and /ingest stay blocked.

Reset Local Database

bun run db:reset

Troubleshooting Build Issues

  1. Regenerate artifacts:

bun run generate:actions
bun run docs:generate

  1. Re-run checks:

bun run check
bunx tsc --noEmit
bun run build

Troubleshoot esbuild / Drizzle Hangs

Symptoms:

  • bun run docs:generate does not finish

  • bun run db:generate or bun run db:push stays stuck

Actions:

  1. Re-run the command once to confirm timeout output from safe scripts.

  2. Reinstall dependencies:

rm -rf node_modules
bun install

  1. Retry:

bun run docs:generate
bun run db:generate
bun run db:push

CMS Overview

How Yayaw content surfaces, data models, media, sections, pages, design tokens, and email templates fit together.

CMS Data Models

Typed, localized, publishable data models and inline variables for CMS content.