Files

akiba 122c408fff fix: resolve 3 security audit issues

#3: Stored XSS in stats dashboard - escape p[path] with html.escape()
#4: Caddy timeout race - increase read/write_timeout 30s -> 60s
#5: Missing CSP header - add Content-Security-Policy to Caddyfile

2026-06-30 14:03:48 +00:00

4.4 KiB

Raw Blame History

AGENTS — AO3 Mirror v5

This file is for AI agents (Hermes, Claude Code, Codex) working on the AO3 reverse proxy mirror.

Project Overview

AO3 Mirror v5 is a high-performance reverse proxy for archiveofourown.org, redesigned from the ground up with sticky sessions, Service Worker client-side caching, and a full 5000-proxy P2C pool — matching the go3 architecture with modern Python tooling.

Domain: agento3.miscs.dev (behind Cloudflare CDN)
Target: archiveofourown.org
Stack: Python 3.11, FastAPI, uvicorn + uvloop + httptools, curl_cffi, Caddy
Proxy Pool: 5000 WebShare proxies, P2C selection, state machine (healthy/unstable/blocked/probing)
Client: Service Worker for static cache + offline fallback
Architecture Reference: https://tomorin.cv/AO3-CN/go3 (go3 v2 branch)

Architecture v5

User Browser
  │  ao3_sessid_proxy cookie = sticky proxy
  │
  ├─ Service Worker (sw.js)
  │   ├─ Cache API — static assets cached in browser
  │   ├─ Navigation intercept — offline mirror picker
  │   └─ Request passthrough — for non-static, non-nav
  │
  └─ HTTPS → Caddy → 2× uvicorn workers (8081-8082)
       │
       ├─ Proxy Pool (5000, P2C selection)
       │   ├─ State machine: healthy/unstable/blocked/probing
       │   ├─ Concurrency limit: asyncio.Semaphore(400)
       │   └─ Passive probing: 10s interval, 10 concurrency
       │
       ├─ Sticky Session (ao3_sessid_proxy)
       │   └─ User pinned to one proxy for session duration
       │
       ├─ Service Worker Deploy (/sw.js, /sw-YYYYMMDD.js)
       │   └─ HTML injection: auto-register SW on page load
       │
       ├─ CF Challenge → User Browser
       │   └─ All proxies blocked → rewrite challenge → browser solves
       │
       ├─ Static Cache (server-side LRU + disk MD5)
       │
       └─ Local routes: /stats, /health, /metrics

Build Commands

# Start all workers + reload Caddy
cd /home/ubuntu/ao3-mirror && bash start.sh

# Stop everything
cd /home/ubuntu/ao3-mirror && bash stop.sh

# Restart daemon only
sudo systemctl restart ao3-daemon

# Restart Caddy only
sudo systemctl reload caddy

# Syntax check all Python files
python3 -m py_compile proxy_pool.py ao3_fetcher.py app.py cache.py stats.py url_rewriter.py

# Deploy Service Worker update
# 1. Edit static/sw.js
# 2. Update SERVICE_WORKER_DATE in app.py
# 3. Restart workers

Core Files (v5)

File	Purpose
`app.py`	FastAPI backend v5 — proxy handler, sticky sessions, SW deploy, stats
`proxy_pool.py`	P2C proxy pool v5 — 5000 proxies, state machine, semaphore limiter
`ao3_fetcher.py`	Single-request fetcher v5 — no internal retry, CF detection
`cache.py`	Per-worker LRU cache (5000 entries, path-differentiated TTL)
`stats.py`	In-memory stats with batch SQLite flush every 60s
`url_rewriter.py`	URL/header rewriting (ao3 → mirror domain)
`static/sw.js`	Service Worker — client-side cache, offline fallback
`Caddyfile`	Caddy config — SW routes, round-robin, 60s timeouts
`scripts/daemon.py`	Passive worker supervision (systemd)
`start.sh`	Startup — kills old workers, starts new, reloads Caddy

Security Baseline

No secrets in code — proxy credentials in /home/ubuntu/proxy.txt
Workers bound to 127.0.0.1 only
Cloudflare CDN terminates TLS at edge
CORS headers restrict cross-origin access
stats/health/metrics are read-only

Engine Guidance

Complex multi-file changes, architecture evolution → Hermes (here)
Quick targeted fixes, single-file changes → Hermes
Deploy, monitor, notify, schedule → Hermes
Proxy pool refresh → Hermes cron (30min)
Not sure? Start with Hermes — everything runs via Hermes

Monitoring

Health: https://agento3.miscs.dev/health
Stats: https://agento3.miscs.dev/stats
Metrics: https://agento3.miscs.dev/metrics
SW: https://agento3.miscs.dev/sw.js → /sw-YYYYMMDD.js

CTO Goal (Persistent Focus)

Manage akiba/agento3 as CTO. Triage issues hourly, implement top priority, get founder approval before merging. Never ship without YES.

Commit Conventions

One commit per meaningful change
Python files only (no binaries, .pyc, logs, .env)
SOUL.md updated for architectural changes
ao3-mirror skill updated when workflows change

4.4 KiB Raw Blame History Unescape Escape