# Debugging Skill

Hard-won lessons from real failures. Follow this before spending time on dead ends.

## Rule 1: Build Locally First
Before deploying to any remote environment (Dokploy, CI, etc.), **always verify the build locally**:
- `bun run build` / `npm run build` — catches TypeScript errors, unused imports, missing modules
- `docker build .` — catches Dockerfile issues, dependency problems
- Run the test suite if one exists

**Why:** A TypeScript strict mode error (`TS6133: unused import`) caused 7 consecutive Dokploy deploy failures. A 2-second local build would have caught it instantly. Instead, 30 minutes were wasted trying to read remote logs that weren't accessible.

*Learned: 2026-01-29 — hammer-queue deploy failures*

## Rule 2: Reproduce Before Escalating
When a remote deploy/service fails:
1. **Build locally first** (Rule 1)
2. **Run locally** if possible (`bun run dev`, `docker compose up`)
3. **Check the runtime** — does the app start? Does `db:push`/migration work?
4. Only after local reproduction fails should you investigate server-side issues

**Why:** The `serial` column type worked in schema definition but broke `drizzle-kit push` on an existing table with data. Building locally passed, but running locally would have caught the db migration failure.

*Learned: 2026-01-29 — serial column broke db:push on existing table*

## Rule 3: Check the Obvious First
Before diving into API spelunking or log hunting:
- Compiler errors? (`tsc`, build output)
- Missing imports or unused imports? (strict mode)
- Schema changes compatible with existing data?
- Environment variables set correctly?
- Ports/URLs correct?

## Rule 4: When Blind to Logs, Create Your Own
If you can't access remote logs (no API endpoint, no SSH, no UI access):
- **Don't** spend more than 5 minutes hunting for log endpoints
- **Do** reproduce the issue locally where you CAN see logs
- **Do** add health check endpoints that report startup errors
- **Do** add structured error logging that surfaces in API responses

## Rule 5: Time-Box Dead Ends
If an approach isn't working after 5-10 minutes, switch strategies:
- Can't read Dokploy logs via API? → Build locally instead
- Can't SSH to server? → Use available APIs differently
- API returning opaque errors? → Test the component in isolation

## Rule 6: Schema Migration Safety
When modifying database schemas on existing tables:
- `serial` columns can't be safely added to existing tables via `drizzle-kit push` — use `integer` with app-level sequencing instead
- Always consider: "What happens to existing rows?"
- Test migrations against a database with real data, not just empty tables
- Prefer nullable columns with backfill logic over NOT NULL additions

## Deployment Checklist
Before every deploy:
```
[ ] Local build passes (frontend + backend)
[ ] TypeScript strict mode clean (no unused imports/vars)
[ ] Schema changes tested against existing data
[ ] Environment variables verified
[ ] Push to git
[ ] Deploy
[ ] Verify health endpoint after deploy
[ ] Verify API functionality
```

---
*This skill is a living document. Update it every time a new debugging lesson is learned.*