agento3/AGENTS.md

# AGENTS — AO3 Mirror v5

This file is for AI agents (Hermes, Claude Code, Codex) working on the AO3 reverse proxy mirror.

## Project Overview

AO3 Mirror v5 is a high-performance reverse proxy for archiveofourown.org, redesigned from the ground up with sticky sessions, Service Worker client-side caching, and a full 5000-proxy P2C pool — matching the go3 architecture with modern Python tooling.

- **Domain**: agento3.miscs.dev (behind Cloudflare CDN)
- **Target**: archiveofourown.org
- **Stack**: Python 3.11, FastAPI, uvicorn + uvloop + httptools, curl_cffi, Caddy
- **Proxy Pool**: 5000 WebShare proxies, P2C selection, state machine (healthy/unstable/blocked/probing)
- **Client**: Service Worker for static cache + offline fallback
- **Architecture Reference**: https://tomorin.cv/AO3-CN/go3 (go3 v2 branch)

## Architecture v5

```
User Browser
  │  ao3_sessid_proxy cookie = sticky proxy
  │
  ├─ Service Worker (sw.js)
  │   ├─ Cache API — static assets cached in browser
  │   ├─ Navigation intercept — offline mirror picker
  │   └─ Request passthrough — for non-static, non-nav
  │
  └─ HTTPS → Caddy → 2× uvicorn workers (8081-8082)
       │
       ├─ Proxy Pool (5000, P2C selection)
       │   ├─ State machine: healthy/unstable/blocked/probing
       │   ├─ Concurrency limit: asyncio.Semaphore(400)
       │   └─ Passive probing: 10s interval, 10 concurrency
       │
       ├─ Sticky Session (ao3_sessid_proxy)
       │   └─ User pinned to one proxy for session duration
       │
       ├─ Service Worker Deploy (/sw.js, /sw-YYYYMMDD.js)
       │   └─ HTML injection: auto-register SW on page load
       │
       ├─ CF Challenge → User Browser
       │   └─ All proxies blocked → rewrite challenge → browser solves
       │
       ├─ Static Cache (server-side LRU + disk MD5)
       │
       └─ Local routes: /stats, /health, /metrics
```

## Build Commands

```bash
# Start all workers + reload Caddy
cd /home/ubuntu/ao3-mirror && bash start.sh

# Stop everything
cd /home/ubuntu/ao3-mirror && bash stop.sh

# Restart daemon only
sudo systemctl restart ao3-daemon

# Restart Caddy only
sudo systemctl reload caddy

# Syntax check all Python files
python3 -m py_compile proxy_pool.py ao3_fetcher.py app.py cache.py stats.py url_rewriter.py

# Deploy Service Worker update
# 1. Edit static/sw.js
# 2. Update SERVICE_WORKER_DATE in app.py
# 3. Restart workers
```

### Core Files (v5)

| File | Purpose |
|------|---------|
| `app.py` | FastAPI backend v5 — proxy handler, sticky sessions, SW deploy, stats |
| `proxy_pool.py` | P2C proxy pool v5 — 5000 proxies, state machine, semaphore limiter |
| `ao3_fetcher.py` | Single-request fetcher v5 — no internal retry, CF detection |
| `cache.py` | Per-worker LRU cache (5000 entries, path-differentiated TTL) |
| `stats.py` | In-memory stats with batch SQLite flush every 60s |
| `url_rewriter.py` | URL/header rewriting (ao3 → mirror domain) |
| `static/sw.js` | Service Worker — client-side cache, offline fallback |
| `Caddyfile` | Caddy config — SW routes, round-robin, 60s timeouts |
| `scripts/daemon.py` | Passive worker supervision (systemd) |
| `start.sh` | Startup — kills old workers, starts new, reloads Caddy |

## Security Baseline

- No secrets in code — proxy credentials in `/home/ubuntu/proxy.txt`
- Workers bound to 127.0.0.1 only
- Cloudflare CDN terminates TLS at edge
- CORS headers restrict cross-origin access
- stats/health/metrics are read-only

## Engine Guidance

- Complex multi-file changes, architecture evolution → Hermes (here)
- Quick targeted fixes, single-file changes → Hermes
- Deploy, monitor, notify, schedule → Hermes
- Proxy pool refresh → Hermes cron (30min)
- Not sure? Start with Hermes — everything runs via Hermes

## Monitoring

- Health: https://agento3.miscs.dev/health
- Stats: https://agento3.miscs.dev/stats
- Metrics: https://agento3.miscs.dev/metrics
- SW: https://agento3.miscs.dev/sw.js → /sw-YYYYMMDD.js

## CTO Goal (Persistent Focus)

Manage akiba/agento3 as CTO. Triage issues hourly, implement top priority, get founder approval before merging. Never ship without YES.

## Commit Conventions

- One commit per meaningful change
- Python files only (no binaries, .pyc, logs, .env)
- SOUL.md updated for architectural changes
- ao3-mirror skill updated when workflows change