topaLE
/
ghostfolio
mirror of https://github.com/ghostfolio/ghostfolio
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
Browse Source

Fix Railway config: Node.js for root, Python for agent 

Made-with: Cursor
pull/6453/head
Priyanka Punukollu 1 month ago
parent
649d12d7a8
commit
6e4e8ca412
3 changed files with 638 additions and 4 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 621
      AI.md
  2. 14
      nixpacks.toml
  3. 7
      railway.toml

621
AI.md
View File

@ -0,0 +1,621 @@
# AI.md — Ghostfolio AI Agent Codebase Analysis
## METADATA
```
project_name: ghostfolio-ai-agent
primary_language: Python
secondary_languages: TypeScript (NestJS backend, Angular frontend)
framework: LangGraph + FastAPI
llm_provider: Anthropic (Claude Sonnet 4.5)
database: SQLite (properties.db), PostgreSQL (Ghostfolio)
test_count: 183
test_pass_rate: 100%
lines_of_code_agent: ~8000 (Python)
last_analysis_date: 2026-02-28
```
---
## PURPOSE
Personal finance AI agent that unifies investment portfolio tracking (via Ghostfolio) with real estate equity tracking. Enables users to ask natural language questions about their complete financial picture including stocks, ETFs, properties, retirement readiness, job offer evaluation, and life decision analysis.
Target use case: A user with a stock portfolio in Ghostfolio and owned real estate who wants a single conversational interface to understand total net worth, run retirement projections, evaluate job offers across cities, and simulate real estate investment strategies.
---
## ARCHITECTURE OVERVIEW
### Graph Structure
```
Entry: classify_node
↓ (keyword matching → query_type string)
Route: _route_after_classify
↓ (maps query_type → executor)
Branch A: tools_node (read operations)
↓
Branch B: write_prepare → write_execute (write operations with confirmation)
↓
verify_node (confidence scoring, domain constraint check)
↓
format_node (LLM synthesis via Claude)
↓
Exit: END
```
### State Schema (AgentState)
Location: `agent/state.py`
```python
TypedDict with fields:
- user_query: str
- messages: list[BaseMessage]
- query_type: str
- portfolio_snapshot: dict
- tool_results: list[dict]
- pending_verifications: list[dict]
- confidence_score: float
- verification_outcome: str
- awaiting_confirmation: bool
- confirmation_payload: Optional[dict]
- pending_write: Optional[dict]
- bearer_token: Optional[str]
- final_response: Optional[str]
- citations: list[str]
- error: Optional[str]
- input_tokens: Optional[int]
- output_tokens: Optional[int]
```
---
## FILE STRUCTURE
```
agent/
├── main.py # FastAPI server, endpoints, auth
├── graph.py # LangGraph state machine (2963 lines)
├── state.py # AgentState TypedDict
├── chat_ui.html # Standalone chat interface
├── login.html # Auth page
├── requirements.txt # Python dependencies
├── data/
│ └── properties.db # SQLite for property tracking
├── tools/
│ ├── __init__.py # Tool registry
│ ├── portfolio.py # Ghostfolio API integration
│ ├── property_tracker.py # CRUD for owned properties
│ ├── real_estate.py # Market data (mock + live)
│ ├── wealth_bridge.py # Down payment + job offer analysis
│ ├── life_decision_advisor.py # Multi-tool orchestration
│ ├── relocation_runway.py # Financial stability timeline
│ ├── wealth_visualizer.py # Fed Reserve peer comparison
│ ├── family_planner.py # Childcare cost modeling
│ ├── realestate_strategy.py # Buy-and-rent simulation
│ ├── market_data.py # Yahoo Finance price fetching
│ ├── compliance.py # Portfolio risk rules
│ ├── tax_estimate.py # Capital gains estimation
│ ├── transactions.py # Trade history
│ ├── categorize.py # Activity pattern analysis
│ ├── write_ops.py # Buy/sell/add transaction
│ └── teleport_api.py # Global city cost-of-living
├── verification/
│ └── fact_checker.py # Tool result verification
└── evals/
├── conftest.py # Pytest fixtures
├── test_portfolio.py # 60 compliance/tax/helper tests
├── test_property_tracker.py
├── test_real_estate.py
└── ... (17 test files total)
```
---
## TOOLS REGISTRY
### Portfolio Tools
| Tool | File | Input | Output |
|------|------|-------|--------|
| portfolio_analysis | portfolio.py | date_range, token | holdings[], summary{total_cost_basis, total_current_value, gain_pct, ytd_gain_pct} |
| compliance_check | compliance.py | portfolio_data | warnings[], overall_status (CLEAR/FLAGGED) |
| tax_estimate | tax_estimate.py | activities[] | short_term_gains, long_term_gains, estimated_tax, wash_sale_warnings[] |
| transaction_query | transactions.py | symbol, limit, token | activities[] |
| transaction_categorize | categorize.py | activities[] | summary, patterns[], most_traded[] |
| market_data | market_data.py | symbol | current_price, previous_close, change_pct |
### Property Tools
| Tool | File | Input | Output |
|------|------|-------|--------|
| add_property | property_tracker.py | address, purchase_price, current_value, mortgage_balance | property record with equity calculation |
| get_properties | property_tracker.py | none | properties[], summary{total_equity, total_value} |
| update_property | property_tracker.py | property_id, current_value/mortgage_balance | updated record |
| remove_property | property_tracker.py | property_id | confirmation |
| get_total_net_worth | property_tracker.py | portfolio_value | combined net worth across asset classes |
| analyze_equity_options | property_tracker.py | property_id | 3 options: leave, cash-out refi, rental property |
### Life Decision Tools
| Tool | File | Input | Output |
|------|------|-------|--------|
| analyze_life_decision | life_decision_advisor.py | decision_type, user_context | verdict, tradeoffs[], key_numbers, next_steps |
| calculate_relocation_runway | relocation_runway.py | current_salary, offer_salary, current_city, destination_city | monthly_surplus_delta, milestones{months_to_down_payment} |
| analyze_wealth_position | wealth_visualizer.py | portfolio_value, age, annual_income | percentile_vs_peers, retirement_projection, what_if_scenarios |
| plan_family_finances | family_planner.py | current_city, annual_income, num_children | monthly_cost_breakdown, feasibility, alternatives |
| calculate_down_payment_power | wealth_bridge.py | portfolio_value, target_cities | markets[], top_recommendation |
| calculate_job_offer_affordability | wealth_bridge.py | offer_salary, offer_city, current_salary, current_city | is_real_raise, breakeven_salary_needed |
---
## QUERY CLASSIFICATION
Location: `graph.py:classify_node` (lines 454-1155)
### Classification Method
1. Keyword matching (primary, no LLM call, <10ms)
2. LLM fallback via `llm_classify_intent` when keywords fail (uses Claude Haiku)
### Query Types (61 total)
Core: `performance`, `activity`, `compliance`, `tax`, `market`, `market_overview`, `categorize`
Combined: `performance+compliance+activity`, `performance+market`, `compliance+tax`
Write: `buy`, `sell`, `dividend`, `cash`, `transaction`, `write_confirmed`, `write_cancelled`, `write_refused`
Property: `property_add`, `property_list`, `property_update`, `property_remove`, `property_net_worth`
Real Estate: `real_estate_snapshot`, `real_estate_search`, `real_estate_compare`, `real_estate_detail`, `real_estate_refused`
Wealth: `wealth_down_payment`, `wealth_job_offer`, `wealth_global_city`, `wealth_portfolio_summary`
Life: `life_decision`, `relocation_runway`, `wealth_gap`, `equity_unlock`, `family_planner`
Special: `capabilities`, `context_followup`, `unknown`
---
## MODEL SELECTION
Location: `graph.py` lines 79-103
```python
FAST_MODEL = "claude-haiku-4-5-20251001" # Classification, simple queries
SMART_MODEL = "claude-sonnet-4-20250514" # Complex queries
COMPLEX_QUERY_TYPES = {
"life_decision", "family_planner", "wealth_gap", "wealth_down_payment",
"wealth_job_offer", "wealth_global_city", "wealth_portfolio_summary",
"equity_unlock", "real_estate_detail", "real_estate_snapshot",
"real_estate_search", "real_estate_compare"
}
```
---
## DATA SOURCES
### Live APIs
1. **Ghostfolio API** (`/api/v1/portfolio/holdings`, `/api/v1/user`)
- Bearer token auth
- Returns raw holdings, activities
2. **Yahoo Finance** (`query1.finance.yahoo.com/v8/finance/chart/{symbol}`)
- No auth required
- 30-minute price cache
- Returns current price, YTD start price
3. **Teleport API** (`api.teleport.org/api/urban_areas/{slug}/scores/`)
- Cost-of-living data for global cities
- Falls back to HARDCODED_FALLBACK on failure
### Static Data
1. **ACTRIS MLS Mock Data** (`real_estate.py:_MOCK_SNAPSHOTS`)
- Austin-area market statistics (7 regions)
- January 2026 data embedded
2. **Federal Reserve SCF 2022** (`wealth_visualizer.py:FED_WEALTH_DATA`)
- Median/percentile wealth by age bracket
3. **Childcare Costs** (`family_planner.py:CHILDCARE_ANNUAL`)
- Care.com 2024 averages per city
---
## VERIFICATION LAYER
### confidence_score Calculation (main.py:108-123)
```python
base = 0.85
if tool_result is None: return 0.40
if "error" in str(tool_result).lower(): base -= 0.20
if has_verified_data_source: base += 0.10
if tool_called in ("portfolio_analysis", "property_tracker", "real_estate"): base += 0.05
return min(0.99, max(0.40, base))
```
### Domain Constraint Check (main.py:126-156)
Scans response for HIGH_RISK_PHRASES:
```python
["you should buy", "you should sell", "i recommend buying",
"guaranteed return", "will definitely", "certain to",
"risk-free", "always profitable"]
```
Passes if no phrases found OR response contains disclaimer keywords.
### verify_node (graph.py:2492-2532)
- Runs `verify_claims()` from `fact_checker.py`
- Adjusts confidence by -0.15 per failed tool
- Sets outcome: `pass` (≥0.7), `flag` (0.4-0.7), `escalate` (<0.4)
---
## HUMAN-IN-THE-LOOP
Location: `graph.py:write_prepare_node` (lines 1161-1389)
### Flow
1. User expresses write intent ("buy 10 shares of AAPL")
2. classify_node routes to `write_prepare`
3. write_prepare extracts params, fetches live price if missing, builds payload
4. Returns confirmation prompt with `awaiting_confirmation=True`
5. User replies "yes" or "confirm"
6. classify_node routes to `write_execute`
7. write_execute calls Ghostfolio API, refreshes portfolio
### Write Operations Supported
- `buy_stock(symbol, quantity, price, date_str, fee)`
- `sell_stock(symbol, quantity, price, date_str, fee)`
- `add_transaction(symbol, quantity, price, transaction_type, date_str, fee)`
- `add_cash(amount, currency)`
### Large Order Warning
Orders ≥100,000 shares display warning in confirmation prompt.
---
## API ENDPOINTS
Location: `main.py`
| Endpoint | Method | Auth | Purpose |
|----------|--------|------|---------|
| `/chat` | POST | JWT | Main chat, returns full response |
| `/chat/stream` | POST | JWT | SSE streaming response |
| `/chat/steps` | POST | JWT | SSE with graph node events |
| `/auth/login` | POST | None | Returns JWT |
| `/me` | GET | None | User profile from Ghostfolio |
| `/seed` | POST | None | Populate demo portfolio |
| `/health` | GET | None | Agent + Ghostfolio status |
| `/metrics` | GET | None | Aggregate session stats |
| `/costs` | GET | JWT | Anthropic API cost tracker |
| `/feedback` | POST | JWT | Record thumbs up/down |
| `/feedback/summary` | GET | JWT | Approval rate |
| `/real-estate/log` | GET | None | Tool invocation log |
---
## TEST COVERAGE
Location: `agent/evals/`
### Test Categories
| Category | Count | Files |
|----------|-------|-------|
| Compliance rules | 15 | test_portfolio.py |
| Tax calculation | 15 | test_portfolio.py |
| Transaction categorize | 10 | test_portfolio.py |
| Holdings consolidation | 10 | test_portfolio.py |
| Graph extraction helpers | 10 | test_portfolio.py |
| Property CRUD | 13 | test_property_tracker.py |
| Real estate feature | 8 | test_real_estate.py |
| Strategy simulation | 7 | test_realestate_strategy.py |
| Relocation runway | 5 | test_relocation_runway.py |
| Wealth bridge | 8 | test_wealth_bridge.py |
| Wealth visualizer | 6 | test_wealth_visualizer.py |
| Life decision | varies | test_life_decision_advisor.py |
| Family planner | varies | test_family_planner.py |
| Equity advisor | varies | test_equity_advisor.py |
| Property onboarding | varies | test_property_onboarding.py |
### Test Isolation
`conftest.py` patches `teleport_api._fetch_from_teleport` to return `None`, forcing all tests to use `HARDCODED_FALLBACK` data. No network calls during test runs.
---
## STRENGTHS
### Architecture
1. **Explicit state machine** — LangGraph graph with named nodes makes debugging straightforward. Each node's input/output is inspectable.
2. **Keyword-first classification** — Avoids LLM latency for 95%+ of queries. LLM fallback only for ambiguous inputs.
3. **Feature flags** — Real estate tools gated by `ENABLE_REAL_ESTATE=true`. Clean degradation when features disabled.
4. **Tool result envelope** — All tools return `{tool_name, success, tool_result_id, timestamp, result|error}`. Consistent parsing in format_node.
5. **Per-user auth** — `bearer_token` in state allows agent to operate on logged-in user's portfolio via Angular app.
### Code Quality
6. **Comprehensive docstrings** — Every tool function has Args/Returns documentation.
7. **Type hints** — TypedDict for state, Optional types for nullable fields.
8. **Error handling** — All tools return structured error dicts, never raise exceptions to caller.
9. **Test coverage** — 183 tests covering edge cases, adversarial inputs, multi-step flows.
10. **No hardcoded secrets** — All credentials via environment variables.
### Domain Logic
11. **Real ACTRIS data** — Austin market stats are actual January 2026 MLS figures, not synthetic.
12. **Federal Reserve benchmarks** — Wealth comparison uses official SCF 2022 percentiles.
13. **Tax awareness** — State income tax lookup for no-tax states (TX, WA, FL, etc.) in job offer calculations.
14. **Proper financial formulas** — Mortgage payment calculation uses standard amortization formula with 30yr/6.95% assumptions clearly documented.
---
## WEAKNESSES
### Architecture
1. **Monolithic graph.py** — 2963 lines in single file. Contains classification, extraction helpers, tool routing, format logic. Should be split into:
- `classify.py` (classification node + keyword lists)
- `routing.py` (route functions)
- `extraction.py` (ticker, price, date, property detail extraction)
- `format.py` (format_node logic)
2. **Keyword duplication** — Same city names repeated in multiple files (`_KNOWN_CITIES` in graph.py, `FALLBACK_RENTS` in relocation_runway.py, `RENT_LOOKUP` in family_planner.py). Should be single source of truth.
3. **Debug logging hardcoded** — Lines 627-648 and 1132-1152 contain debug logging to absolute file path `/Users/priyankapunukollu/...`. Should be removed or use proper logging framework.
4. **Sync/async mixing** — `life_decision_advisor.py:_run_async()` uses `asyncio.run()` from sync context with ThreadPoolExecutor fallback. Fragile pattern that can cause event loop conflicts.
### Data
5. **Mock data as production** — `real_estate.py:_MOCK_SNAPSHOTS` returns same data for every request. Users cannot distinguish mock vs live data without checking `data_source` field.
6. **Stale cache potential** — Portfolio cache TTL is 60 seconds, price cache is 30 minutes. User could see outdated data during active trading.
7. **No database migrations** — SQLite schema created inline in `_get_conn()`. Schema changes require manual migration scripts.
### Security
8. **CORS allow all** — `allow_origins=["*"]` in main.py. Should restrict to known origins in production.
9. **JWT secret validation** — If `JWT_SECRET_KEY` is missing, RuntimeError is raised at runtime, not startup. Should fail fast on app initialization.
10. **No rate limiting** — API endpoints have no throttling. Vulnerable to abuse.
### Testing
11. **No integration tests** — All tests mock external APIs. No tests verify actual Ghostfolio/Yahoo Finance integration.
12. **No LLM response tests** — Tests verify tool logic only. No assertions on format_node output quality.
---
## IMPROVEMENT OPPORTUNITIES
### High Priority
1. **Split graph.py**
- Extract to 4-5 files as described above
- Reduces cognitive load, improves testability
2. **Remove debug logging**
- Delete hardcoded file path logging
- Use `logging` module with configurable level
3. **Centralize city data**
- Create `data/cities.py` with single dict
- Import everywhere else
4. **Fix async pattern**
- Make `analyze_life_decision` fully async
- Use `await` throughout instead of `_run_async()` hack
5. **Add startup validation**
- Check all required env vars on app startup
- Fail fast with clear error messages
### Medium Priority
6. **Rate limiting**
- Add `slowapi` or similar middleware
- Limit to 10 requests/minute per IP
7. **CORS configuration**
- Move allowed origins to env var
- Default to localhost only
8. **Database migrations**
- Add Alembic or similar migration tool
- Version control schema changes
9. **LLM output testing**
- Add eval tests that check format_node responses
- Assert on citation presence, banned phrase absence
10. **Cache invalidation**
- Add endpoint to clear portfolio cache
- Reduce TTL or use webhook invalidation
### Low Priority
11. **OpenAPI documentation**
- Add Pydantic models for all request/response types
- Enable FastAPI automatic docs
12. **Health check depth**
- Check Redis, database, LLM provider connectivity
- Return structured status per dependency
13. **Structured logging**
- Use JSON log format for production
- Add request_id correlation
---
## DEPENDENCY VERSIONS
From `requirements.txt`:
```
fastapi
uvicorn[standard]
langgraph
langchain-core
langchain-anthropic
anthropic
httpx
python-dotenv
pytest
pytest-asyncio
passlib[bcrypt]
bcrypt>=3.2,<4.1
python-jose[cryptography]
```
Note: No pinned versions except bcrypt. Recommend pinning all dependencies for reproducibility.
---
## ENVIRONMENT VARIABLES
Required:
- `ANTHROPIC_API_KEY` — Claude API access
- `JWT_SECRET_KEY` — Session signing
- `ADMIN_USERNAME` — Login username
- `ADMIN_PASSWORD_HASH` — Bcrypt hash
Optional:
- `GHOSTFOLIO_BASE_URL` — Default `http://localhost:3333`
- `GHOSTFOLIO_BEARER_TOKEN` — Default portfolio token
- `ENABLE_REAL_ESTATE` — Feature flag, default `false`
- `PROPERTIES_DB_PATH` — SQLite path, default `agent/data/properties.db`
- `LANGCHAIN_TRACING_V2` — LangSmith tracing
- `LANGCHAIN_API_KEY` — LangSmith auth
- `LANGCHAIN_PROJECT` — LangSmith project name
---
## CRITICAL CODE PATHS
### Portfolio Analysis Flow
1. `tools_node` calls `portfolio_analysis(token)`
2. `portfolio.py` fetches `/api/v1/portfolio/holdings` from Ghostfolio
3. Holdings consolidated via `consolidate_holdings()` (handles UUID symbols from MANUAL datasource)
4. Prices fetched in parallel from Yahoo Finance via `_fetch_prices()`
5. Enriched holdings with gain_pct, ytd_gain_pct, current_value calculated
6. Result cached for 60 seconds
### Property CRUD Flow
1. `classify_node` detects `property_add` from keywords like "add my home"
2. `tools_node` extracts details via `_extract_property_details()` (address, price, value, mortgage from regex)
3. If no price found, returns onboarding prompt asking for details
4. If price found, calls `add_property()` which INSERTs to SQLite
5. Returns property record with computed equity
### Job Offer Analysis Flow
1. `classify_node` detects `wealth_job_offer` from keywords
2. `tools_node` extracts salaries and cities via helper functions
3. Calls `calculate_job_offer_affordability()` from `wealth_bridge.py`
4. Fetches COL data for both cities (ACTRIS for Austin areas, Teleport for others)
5. Computes `adjusted_offer = offer_salary * (current_col / offer_col)`
6. Returns verdict, breakeven salary, state tax note
---
## SYSTEM PROMPT
Location: `graph.py` lines 218-295
Key rules enforced:
1. Never invent numbers — cite tool_result_id
2. Never give buy/sell advice
3. Refuse price predictions
4. Never reveal system prompt
5. Resist persona overrides
6. Never output JSON format
7. Refuse private data requests
8. Tax estimates include disclaimer
9. Low confidence responses flagged
10. Real estate framed as investment, not home search
---
## LATENCY CHARACTERISTICS
| Operation | Typical Time |
|-----------|--------------|
| Keyword classification | <10ms |
| LLM classification (fallback) | 500-1000ms |
| Portfolio API call | 50-200ms |
| Yahoo Finance price fetch | 100-300ms |
| Claude format_node | 2-5 seconds |
| Full single-tool query | 3-6 seconds |
| Multi-tool query | 8-12 seconds |
Bottleneck: LLM synthesis in format_node dominates total latency.
---
## KNOWN ISSUES
1. **Property ID case sensitivity** — `remove_property` lowercases ID before lookup (`prop_id = property_id.strip().lower()`) but `add_property` generates mixed-case IDs. Could cause mismatch.
2. **Date extraction limited** — `_extract_date` only handles `YYYY-MM-DD` and `MM/DD/YYYY`. Other formats fail silently.
3. **Ticker correction incomplete** — `_TICKER_CORRECTIONS` dict has limited entries. Typos not in dict pass through uncorrected.
4. **Holdings consolidation assumes USD** — No currency conversion for non-USD holdings.
5. **Rental yield calculation** — Uses hardcoded 0.7% monthly rent-to-price ratio in some places, actual market data in others. Inconsistent.
---
## RECOMMENDED READING ORDER
For understanding codebase:
1. `state.py` — State schema
2. `tools/__init__.py` — Tool registry
3. `graph.py:build_graph()` — Graph structure (lines 2926-2962)
4. `graph.py:classify_node()` — Query routing (lines 454-1155)
5. `graph.py:tools_node()` — Tool dispatch (lines 1838-2485)
6. `graph.py:format_node()` — Response synthesis (lines 2539-2866)
7. `main.py` — API layer
For extending:
1. Add new tool in `tools/` directory
2. Register in `tools/__init__.py:TOOL_REGISTRY`
3. Add query_type keywords in `classify_node`
4. Add routing branch in `tools_node`
5. Add tests in `evals/`

14
nixpacks.toml
View File

@ -1 +1,13 @@
providers = ["python"]
providers = ["node"]
[phases.setup]
nixPkgs = ["nodejs-22_x", "openssl"]
[phases.install]
cmds = ["npm ci"]
[phases.build]
cmds = ["npx prisma generate", "npm run build:production"]
[start]
cmd = "npx prisma migrate deploy && npm run start"

7
railway.toml
View File

@ -1,9 +1,10 @@
[build]
builder = "nixpacks"
buildCommand = "npm ci && npx prisma generate && npm run build:production"
[deploy]
startCommand = "uvicorn main:app --host 0.0.0.0 --port $PORT"
healthcheckPath = "/health"
healthcheckTimeout = 60
startCommand = "npx prisma migrate deploy && npm run start"
healthcheckPath = "/api/v1/info"
healthcheckTimeout = 120
restartPolicyType = "ON_FAILURE"
restartPolicyMaxRetries = 3

Write Preview
Loading…
Cancel
Save