4.0 KiB
Lessons
Updated: 2026-02-24
Context / Mistake / Rule
-
Context: Documentation updates during rapid iteration Mistake: File path assumptions drifted across turns Rule: Verify target files with
findandwc -limmediately after each save operation. -
Context: Mixed policy documents (
agents.md,CLAUDE.md, project requirements) Mistake: Source-of-truth order remained implicit Rule: Anchor task execution todocs/requirements.md, then align secondary operating docs to that baseline. -
Context: AI endpoint review for MVP hardening Mistake: Utility regex and service size limits were under-enforced during fast delivery Rule: Add deterministic edge-case tests for parser heuristics and enforce file-size split before declaring MVP complete.
-
Context: Local MVP validation with UI-gated features Mistake: Test instructions skipped the exact in-app location and feature visibility conditions Rule: Document one deterministic URL path plus visibility prerequisites whenever a feature is behind settings or permissions.
-
Context: Railway deployments from local
railway.tomlMistake: Start command drifted to a non-existent runtime path and caused repeated crash loops Rule: Keeprailway.tomlstartCommandaligned with Docker runtime entrypoint and verify with deployment logs after every command change. -
Context: Quality review requests with explicit target scores Mistake: Initial assessment did not immediately convert score gaps into concrete code-level remediation tasks Rule: For any score target, map each category gap to a named patch + test gate before returning a status update.
-
Context: AI routing hardening in deterministic tool orchestration Mistake: Considered model-structured output guards before validating actual failure surface Rule: When tool routing is deterministic, prioritize planner fallback correctness and executor policy gating before adding LLM classifier layers.
-
Context: Open-source submission strategy after publish constraints Mistake: Treated npm publication as the only completion path for contribution evidence Rule: When package publication is blocked, ship the tool in-repo and open upstream PRs in high-signal repositories to preserve external contribution progress.
-
Context: AI chat UX feedback on response readability Mistake: Diagnostic metadata remained inline with assistant answers, reducing answer clarity Rule: Keep primary assistant messages user-facing only and place diagnostics (confidence, citations, verification, observability) behind an explicit info-triggered popover.
-
Context: Memory feature validation after chat/session persistence rollout Mistake: Session-scoped memory shipped without an explicit user-scoped preference path for cross-session continuity. Rule: When memory requirements mention user preferences, implement and test both session memory and user-level memory keyed independently from session IDs.
-
Context: Large table-driven Jest test expansion for policy routing and arithmetic behavior Mistake: Mixed tuple/string/object datasets under a single typed
it.eachsignature created preventable TypeScript compile failures. Rule: Keep each table shape typed independently (it.each<[...tuple]>()for positional rows and object generics only for object rows). -
Context: Ambiguous user follow-up prompts in a finance assistant ("what can i do?") Mistake: Capability-style routing captured actionable follow-up intent and bypassed tool-backed recommendation generation. Rule: Treat ambiguous action follow-ups as recommendation intent when finance context exists, and lock this with deterministic service tests.
-
Context: Recommendation replies looked short and repetitive even when tool context was available Mistake: Reliability gating accepted generic recommendation prose that lacked option sections and actionable structure. Rule: For recommendation-intent prompts, enforce sectioned output quality gates (Option 1/2 + assumptions/risk notes/next questions) and fall back to deterministic option plans when structure is missing.