Files
BadNote/docs/plans/2026-06-21-badnote-full-refactor.md
Akiba So 3295018ee3
All checks were successful
CI / Windows build (push) Successful in 11m34s
feat(pen): pressure-responsive width, configurable thinning, native Windows pen (tilt/buttons)
W1 — Custom pen width + pressure sensitivity (Saber-style):
- Root cause of "压感没用": perfect_freehand 1.0.4 IGNORES real stylus pressure
  (hardcodes radius=size/2 when simulatePressure=false) — width never tracked pen
  force. Upgraded perfect_freehand ^1.0.0 -> ^2.0.0 (honors real pressure); migrated
  all 5 getStroke call sites to the 2.x API (PointVector / StrokeOptions / Offset).
- De-hardcoded `thinning` into `kDefaultPenThinning` (0.85), single source shared by
  the on-screen painter and the PDF export path; exposed as PenConfig.pressureSensitivity
  with a Pressure Sensitivity slider; live-applies via a config listener.

W3 — Native Windows pen plugin (tilt + barrel/eraser buttons):
- windows/runner/pen_channel.{h,cpp}: observe WM_POINTER at the TOP of MessageHandler
  (before HandleTopLevelWindowProc, which Flutter uses to consume pen events), read
  GetPointerPenInfo penFlags + tilt, stream over EventChannel('badnote/pen'); non-consuming.
- PenInputService: single latched hardware state (no Win32-pointerId<->event.pointer
  correlation); graceful no-op off-Windows.
- pen_canvas maps barrel/inverted/eraser through PenConfig.sideButton/eraserEnd
  (eraser/undo/toggleTool/pan) and captures tilt into PenPoint.tilt -> EditorPoint.tilt.

W2 — Zoom flicker: page raster isolated in its own RepaintBoundary (safe interim);
definitive crisp-on-zoom fix gated on the on-device root-cause probe (plan M3).

Plans: ralplan-consensus plan at docs/plans/2026-06-22-badnote-pen-polish.md
(Architect APPROVE-WITH-MUST-FIX M1-M4 + Critic ITERATE->APPROVE).

Tests: 58/58 pass incl. shared-thinning invariant + thinning-affects-outline +
tilt-adapter round-trip. flutter analyze clean; linux debug build OK.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 02:10:05 +08:00

62 KiB
Raw Blame History

BadNote — Full Refactor + Feature Roadmap (own-canvas engine)

Status: PLAN (ralplan consensus — Architect APPROVE-WITH-MUST-FIX + Critic-ITERATE fixes applied 2026-06-21: MF1MF3 + P0.5 slice + SF1SF5 + cache-split / diff-write-durability / erase-is-new-behavior / render-handle-lifecycle; pending final Critic confirm) Date: 2026-06-21 Mode: DELIBERATE (high-risk: engine generalization, 60fps continuous/double-page, OCR/CAS feasibility, server/AI scope) Owner plan file: docs/plans/2026-06-21-badnote-full-refactor.md Supersedes (in part): docs/plans/2026-06-21-badnote-phase1.md

Grounding. Written after reading the live code, not from memory. Verified sources: live editor lib/editor/canvas/{pen_canvas,ink_painters,pen_stroke,pen_editor_screen}.dart, lib/editor/pdf/pen_capture_region.dart, lib/main.dart (dynamic_color + pdfrxFlutterInitialize + PenCaptureBinding), lib/services/{database_service,pdf_service,stroke_rasterizer,undo_manager}.dart, lib/services/ocr/onnx_recognition_backend.dart, lib/models/{ink_stroke,ink_point,pressure_curve}.dart, lib/providers/settings_provider.dart, lib/screens/home_screen.dart, server/badnote_server/{main,models,routers/sync_router}.py, server/README.md, pubspec.yaml, existing test/ + integration_test/ + tool/ assets.

CRITICAL DIRECTION CHANGE captured here (vs phase1 plan): the live lib/editor/canvas/ has already abandoned the phase-1 "pdfrx pageOverlaysBuilder-hosted ink + RenderProxyBox arena bypass" architecture. The live PenCanvas owns one InteractiveViewer + a single Listener that draws a PdfPageView bitmap and the ink in the same child subtree (Saber clean-room model). pdfrx is used only as a page renderer / document API (PdfDocument.openFile, PdfPageView), never for gestures. PenCaptureRegion/PenCaptureBinding are installed in main() but are NOT wired into the live canvas and are slated for retirement (§6, §9). This plan builds the full product on the own-canvas model and records that decision in the ADR (§10).


0. Vision & Scope

BadNote is a pen-first, performant note-taking app. Primary device: Windows Tablet + Surface Pen (Flutter ≥ 3.44 for WM_POINTER stylus/multitouch); desktop for review/复习/search. Differentiators the user explicitly wants to win on: (1) library-wide full-text search over PDF text + typed text + handwriting OCR with jump-to-location; (2) modern cohesive Material You UI; (3) truly book-like reading (page-flip, two-page spread, thumbnail grid, reader-vs-annotate modes). Plus a 双链 sticky-note infinite board, one-notebook-per-PDF with insertable blank pages and never-rasterized source, and a later server sync + AI refinement and ink CAS.

Existing user data MAY be reset — no migration burden; prefer the cleanest schema. Stack stays Flutter (own gesture pipeline; pdfrx only renders pages). GFW-aware CI already solved (pdfium pre-fetch + Dev Mode + vendored sqlite3 + onnxruntime pre-fetch).

0.1 The 11 feature areas → phase map (full vision; nothing dropped, everything sequenced)

# Feature Phase Risk
F1 Pen-first editor core (harden): pressure, palm rejection, perfect_freehand quality, eraser, tools, undo/redo, DB persistence of strokes per doc/page P0 LowMed
F2 Page layout modes: continuous-single/-double, paged-single/-double + switcher; lazy render @60fps P1 High
F3 Book-like reading: page-flip, two-page spread, reader vs annotate mode, quick jump P1 Med
F4 Thumbnail-grid navigation (Drawboard-style), slider, no keyboard input P1 Low
F5 Configurable pen: side-button + eraser-end mapping, pressure curve, palm sensitivity, finger-drawing toggle — Pen settings page P2 Med
F6 One-notebook-per-PDF: page-level binding, insert blank pages between PDF pages, keep PDF vector, portable bundle + relink P2 Med
F7 Infinite 便利贴 board + 双链 bidirectional links / backlinks P3 MedHigh
F8 Library-wide full-text search (PDF text + typed text + handwriting OCR) with snippets + jump P3 High (handwriting OCR)
F9 Server sync + AI refinement (llm_wiki, VLM/LLM organize) P4 High (scope)
F10 Ink CAS / formula recognition → searchable + solve toggle P5 Very High
F11 Modern UX polish: Material You, subtoolbars, drag-reorder thumbnails, hover pre-warm woven across P0P3 Low

0.2 Non-goals per phase

  • P0: no layout modes, no board, no text boxes, no search UI, no settings page beyond what exists, no server. Single-page editor + persistence + eraser/undo only.
  • P1: no 双链, no OCR-search, no server, no CAS. Layout/reading/thumbnails only.
  • P2: no server, no CAS, no handwriting OCR. Pen-config + notebook/bundle only.
  • P3: no CAS, no server-side AI. Board+双链 and local search (incl. best-effort handwriting OCR via existing text-line model; formula deferred to P5).
  • P4: sync + AI organize; CAS still out.
  • P5: formula OCR + CAS.
  • All phases: never rasterize the source PDF; no on-screen keyboard reliance (slider/grid nav).

1. Target Architecture

1.1 Layered module decomposition (target lib/editor/)

lib/editor/
├─ engine/                      # host-agnostic ink engine (pure-ish, testable)
│  ├─ coordinate_space_host.dart   # CoordinateSpaceHost + NormalizedPageHost + BoardHost
│  ├─ stroke_model.dart            # EditorStroke/EditorPoint (canonical; replaces PenStroke split — §3, §4)
│  ├─ stroke_store.dart            # StrokeStore { committed, int revision } (O(1) shouldRepaint gate)
│  ├─ stroke_geometry.dart         # buildStrokeOutline(getStroke) — single source for screen+export (kills hairline bug)
│  ├─ stroke_eraser.dart           # pure splitStroke / eraseHits (extract from pen_canvas._eraseAt)
│  └─ undo_stack.dart              # global, commit-time-ordered, host-tagged (generalize undo_manager)
├─ input/
│  ├─ input_arbiter.dart           # pure state machine: pointerCount>=2→pan/zoom, stylus→draw, palm reject
│  └─ pen_config.dart              # button/eraser-end action mapping, pressure curve, palm sensitivity (F5)
├─ render/
│  ├─ annotation_layer.dart        # Stack(StaticInkPainter, LiveInkPainter, TextBoxLayer) over a host
│  ├─ static_ink_painter.dart      # reads ink_picture_cache, keyed by revision
│  ├─ live_ink_painter.dart        # in-progress stroke only
│  └─ ink_picture_cache.dart       # LRU<revision, ui.Picture> for STATIC INK — resolution-INDEPENDENT, NO DPI bucket (vector ink re-rasterizes crisp at composite)
├─ layout/                         # F2/F3 — page layout + reading
│  ├─ page_layout.dart             # enum {continuousSingle, continuousDouble, pagedSingle, pagedDouble}
│  ├─ page_viewport.dart           # lazy windowed page hosting (visible + cache extent), recenter
│  └─ reader_controller.dart       # reader vs annotate mode, page-flip, spread, quick-jump
├─ pdf/
│  ├─ pdf_document_source.dart     # wraps PdfDocument (open/dispose/pageSize/render), notebook page-map (F6)
│  ├─ page_tile.dart               # PdfPageView host widget (bitmap) sized to host rect; drives tile DPI from transform scale (R11)
│  └─ page_tile_cache.dart         # LRU<TileKey{hostId, dpiBucket}, ui.Image> for rendered PAGE BITMAPS — DPI-bucketed (tiles blur on upscale); owns native-handle dispose (R11)
├─ board/                          # F7
│  ├─ board_host_pane.dart         # infinite InteractiveViewer board, reuses engine
│  └─ link_graph.dart              # 双链 backlink model + queries
├─ text/                           # OneNote-like editable text boxes (P3+, optional in P2 notebook)
│  ├─ text_box_model.dart
│  └─ text_box_layer.dart
├─ persistence/
│  ├─ editor_repository.dart       # batched load/save per document; maps DB ↔ engine
│  └─ save_scheduler.dart          # debounced, synchronous-snapshot-before-await
├─ search/                         # F8 (P3)
│  ├─ search_indexer.dart          # PDF text + typed text + OCR → document_fts/library index
│  └─ ocr_ingest.dart              # bridge to lib/services/ocr (text-line; formula later)
├─ sync/                           # F9 (P4)
│  └─ sync_client.dart             # talks to server/ (already has push/pull/auth)
└─ ui/
   ├─ editor_screen.dart           # top-level editor (replaces pen_editor_screen + pdf_annotator_screen)
   ├─ editor_toolbar.dart          # Material You subtoolbars (F11)
   ├─ thumbnail_grid.dart          # F4 nav
   └─ editor_shortcuts.dart        # tool numbers, Ctrl+Z/Y/F/S (mouse/desktop)

1.2 Dependency graph (acyclic)

ui/ ─────────────► layout/ ─► pdf/ ─► (pdfrx PdfDocument/PdfPageView)
  │                  │
  │                  ├──────► render/ ─► engine/ (geometry, store, hosts)
  │                  └──────► input/  ─► engine/
  ├──► board/ ─────────────► render/, engine/
  ├──► text/  ─────────────► engine/(hosts)
  ├──► search/ ────────────► persistence/, services/ocr/
  ├──► sync/   ────────────► persistence/, server API
  └──► persistence/ ───────► services/database_service.dart, engine/(stroke_model)

engine/ depends on nothing in editor/ except itself (and perfect_freehand). render/input/layout/ depend on engine. ui/ is the only place wiring Riverpod controllers. This is the seam that lets PDF-page host, infinite board, and (P5) CAS overlay all reuse one renderer.

1.3 Coordinate model — single source of truth

Truth = host content coordinates (already the live convention; generalize it):

  • NormalizedPageHost: points ∈ [0,1] of the page rect (today PenPoint(nx, ny)); width as fraction of page width (today PenStroke.width). Stored per (documentId, pageIndex).
  • BoardHost: absolute logical px on an unbounded canvas; InteractiveViewer supplies pan/zoom.

CoordinateSpaceHost exposes toContent(deviceLocal, deviceSize), toDevice(content, deviceSize), applyContentToCanvas(canvas, deviceSize). The live painters already scale nx*pageSize.width at paint time — formalize that into applyContentToCanvas (a canvas.scale) so no transformed-point copies are ever allocated. Text boxes are a sanctioned widget-space exception (Positioned widgets multiply by current zoom; stored truth stays content coords for search).

1.4 How today's lib/editor/canvas/ evolves into this (no rewrite)

Today (live) Becomes Action
pen_stroke.dart PenStroke/PenPoint (in-memory only) engine/stroke_model.dart EditorStroke/EditorPoint persistable (freezed/JSON, normalized) generalize + add JSON; converge with InkStroke (§4)
ink_painters.dart StaticInkPainter/LiveInkPainter + buildStrokePath render/static_ink_painter.dart (+ render/ink_picture_cache.dart), render/live_ink_painter.dart, engine/stroke_geometry.dart move + add ink ui.Picture cache (resolution-independent, revision-keyed) + revision gate; buildStrokePathbuildStrokeOutline shared with export
pen_canvas.dart _PenCanvasState gesture logic (_activePointers, _shouldDraw, palm reject, _eraseAt) input/input_arbiter.dart (pure SM) + render/annotation_layer.dart (widget) extract SM out of the widget; eraser→engine/stroke_eraser.dart
pen_editor_screen.dart single-page + _strokesByPage Map + _transform + slider ui/editor_screen.dart + layout/page_viewport.dart + persistence/editor_repository.dart generalize single-page → windowed multi-page; _strokesByPage → repository-backed HostStates
pen_capture_region.dart + PenCaptureBinding (unused by live canvas) retire after P0 confirms the Listener-in-shared-transform model holds on device (§6, §9)
pen_editor_screen single PdfPageView bitmap inside the InteractiveViewer (1× layout-sized bitmap, matrix-scaled by pinch) pdf/page_tile.dart multi-resolution tile whose render DPI tracks the transform scale NEW work (not relocation): own multi-resolution tiling — pinch GPU-upscales a 1× bitmap → blurry at high zoom (R11). On zoom-settle, re-instantiate the tile (or PdfPage.render()) at a DPI matching the current scale into pdf/page_tile_cache.dart (DPI-bucketed page-bitmap cache, distinct from the ink cache). Budgeted P1/P0.5.

Net: the four live files are promoted, not thrown away. P0 is mostly relocation + persistence + revision-gated Picture cache, which is low risk and immediately shippable. The one genuinely new burden the own-canvas choice imposes (beyond layout/windowing) is multi-resolution page-tile rendering — see R11 (§7) and the P0.5 gate (§5/§9): we own crisp-on-zoom re-rasterization that pdfrx's own viewer would have given for free.


2. Per-feature design (file-level)

F1 — Pen-first editor core + persistence (P0)

Add/modify:

  • engine/stroke_model.dart [ADD]EditorStroke { id, List<EditorPoint> points, EditorTool tool, int color, double width, bool filled, String? textContent, double fontSize }, EditorPoint { double x, y; double? pressure; double? tilt; int? timestamp; InputDeviceKind? pointerDeviceKind }. freezed + toJson/fromJson. (Normalized; width = fraction of page width, matching live PenStroke.) SF1 — non-lossy superset of InkStroke/InkPoint: InkPoint carries tilt/timestamp/pointerDeviceKind (database_service.dart:346362) that the live PenPoint drops; EditorPoint includes them now (nullable, cheap with freezed) so the InkStrokeEditorStroke adapter (for OCR/export reuse) is lossless in both directions. The live PenCanvas simply leaves the extra fields null at capture; existing InkStroke data round-trips intact. (Avoids a silent data-loss footgun for OCR, which keys on pointerDeviceKind/pressure.)
  • engine/stroke_store.dart [ADD]class StrokeStore { List<EditorStroke> committed; int revision; add/removeAt/replace bump revision; }. Fixes the live pen_editor_screen._commitStroke "new list identity" hack by making revision explicit.
  • engine/stroke_geometry.dart [ADD]Path buildStrokeOutline(EditorStroke, Size, {bool isComplete}) lifted verbatim from ink_painters.buildStrokePath (the proven getStroke(thinning: hl?0:0.7, smoothing:.5, streamline:.5, simulatePressure: !hasRealPressure && !hl) recipe). Single source for screen + export.
  • engine/stroke_eraser.dart [ADD] — pure eraseHits(strokes, point, radius) (extract live _eraseAt) + splitStroke for partial/segment erase. Critic note — splitStroke is NEW behavior, not an extraction: the live pen_canvas._eraseAt is whole-stroke (it removes the entire stroke on the first proximity hit and returns). There is no segment-split logic in the live canvas to port — undo_manager.removeStroke only records replacements, it does not compute them. So splitStroke (point-run splitting → 0/1/2 sub-strokes) is greenfield code; its tests (§8) exercise new functionality, not a regression port. eraseHits (whole-stroke) IS an extraction of the live behavior and stays available as the default erase mode.
  • engine/undo_stack.dart [ADD] — generalize lib/services/undo_manager.dart to host-tagged, commit-time-ordered global stack (entries carry hostId+pageIndex).
  • render/{annotation_layer,static_ink_painter,live_ink_painter,ink_picture_cache}.dart [ADD] — relocate live painters; StaticInkPainter reads ink_picture_cache (LRU<revision, ui.Picture>) keyed by revision (shouldRepaint = old.revision != revision — O(1)); per-host RepaintBoundary. ink_picture_cache is for STATIC VECTOR INK only — resolution-INDEPENDENT, NO DPI bucket (a ui.Picture of vector strokes re-rasterizes crisp at composite time at any zoom). It is a different cache from the page-bitmap page_tile_cache (§2/F2, R11), which IS DPI-bucketed because raster page tiles blur on upscale. Do not conflate the two.
  • input/input_arbiter.dart [ADD] — pure SM extracted from pen_canvas (idle→inking→erasing, pointerCount>=2→cancel+pan, palm reject = touch dropped while stylus active, eraser = kSecondaryButton || invertedStylus). Keeps the live pressure-normalization logic (_normalizedPressure).
  • persistence/editor_repository.dart [ADD]loadDocument(documentId) → Map<int,StrokeStore> (one batched query), saveHost(documentId, hostId, EditorStroke list).
    • MF3 — write contract (load-bearing for the per-row choice): saveHost MUST diff by stroke id against the rows already on disk for that host — UPSERT only changed/new rows, DELETE only removed rows. It must NOT delete-all-rows-for-host then re-insert (that is exactly what the live notes path does at database_service.dart:281306, and at 2,000 strokes it is slower than a single blob rewrite — re-inserting 2,000 rows per save). The per-stroke-row schema (§3) is justified only under this diff contract: erasing 1 of 2,000 strokes ⇒ 1 DELETE, 0 re-inserts; adding 1 stroke ⇒ 1 INSERT. The SaveScheduler hands saveHost the synchronously-captured stroke snapshot; the repository keeps a last-persisted id-set per host to compute the diff. If diffing proves fiddly under churn, the fallback is a per-page blob (NOT delete-all+reinsert) — but the diff path is the default and is what makes per-row worthwhile.
    • Durability invariant (Critic): the UPSERT(s) + DELETE(s) for one saveHost MUST run inside one sqflite transaction, and the in-memory per-host last-persisted id-set is updated ONLY after that transaction commits (in the then/post-await success path) — never optimistically before the write. A crash or interruption mid-diff must leave memory and disk consistent: either the whole diff applied (and the id-set advances) or none of it did (and the id-set is unchanged, so the next save re-derives the same diff and retries). This mirrors the existing updateNote interruption warning (database_service.dart:281306: "an interruption mid-way would permanently lose strokes, so the whole sequence must run inside one transaction").
  • persistence/save_scheduler.dart [ADD] — debounced ~800ms; serialize JSON synchronously before any await; flush on page-leave/dispose.
  • ui/editor_screen.dart, ui/editor_toolbar.dart [ADD] — port pen_editor_screen UI (floating Material You palette + page pill already good).

Public interfaces (key):

abstract class CoordinateSpaceHost {
  String get hostId; int get pageIndex;
  Offset toContent(Offset deviceLocal, Size deviceSize);
  Offset toDevice(Offset content, Size deviceSize);
  void applyContentToCanvas(Canvas c, Size deviceSize);
}
class EditorController extends ChangeNotifier {   // ui-facing, Riverpod-provided
  void beginStroke(CoordinateSpaceHost h, EditorPoint p);
  void extendStroke(EditorPoint p);
  void commitStroke();        // getStroke once → append → revision++ → undo push → scheduleSave
  void eraseAt(CoordinateSpaceHost h, Offset content, double radius);
  void undo(); void redo();
  void setTool(EditorTool t); void setColor(Color c); void setLayout(PageLayout l);
}

Acceptance: draw/erase/undo/redo on a page; close+reopen → strokes persisted (DB); StaticInkPainter.shouldRepaint==false while drawing a new stroke on a 2,000-stroke page (revision constant); no per-frame point allocation for committed strokes.

F2 — Page layout modes (P1)

  • layout/page_layout.dart [ADD]enum PageLayout { continuousSingle, continuousDouble, pagedSingle, pagedDouble }.
  • layout/page_viewport.dart [ADD] — windowed lazy hosting: only pages in [firstVisible - cacheExtent, lastVisible + cacheExtent] mount an AnnotationLayer + PageTile; others are disposed (Picture evicted). Continuous = scrollable column/two-column; paged = PageView. Recenter on layout switch.
  • pdf/pdf_document_source.dart [ADD]PdfDocument wrapper exposing pageSize(i), renderTile(...), page count; owns dispose. Replaces ad-hoc PdfDocument.openFile in pen_editor_screen.
  • pdf/page_tile.dart [ADD] — multi-resolution tile (R11 mitigation). Under one shared InteractiveViewer, a PdfPageView renders a bitmap sized to its layout constraints × devicePixelRatio and the matrix scales that 1× bitmap, so pinch-zoom GPU-upscales → blurry text/rules at high zoom (unlike pdfrx's own viewer, which re-renders crisp tiles per zoom level). page_tile watches the transform scale and on zoom-settle (debounced) re-instantiates its render at a DPI matching the current scale — via either a re-laid-out PdfPageView at the new pixel size or PdfPage.render(width/height at target DPI) into page_tile_cache. Cap retained DPI (e.g. ≤ 3× base) to bound memory; downscale path stays matrix-only (sharp enough). This is new own-canvas work, not relocation.
  • pdf/page_tile_cache.dart [ADD]LRU<TileKey{hostId, dpiBucket}, ui.Image> of rendered page bitmaps (NOT ink). Key includes the DPI bucket so a page re-rendered at higher DPI replaces (not duplicates) its lower-DPI tile. Owns the native-handle dispose lifecycle of each PdfPage.render() result (PdfImage → backing ui.Image), deferred to post-frame so the raster thread is done with an evicted tile before disposal. Distinct from render/ink_picture_cache.dart (§2/F1) which holds resolution-independent ink ui.Pictures with no DPI bucket.
  • ui/editor_toolbar.dart [MODIFY] — layout switcher control.

Two separate caches + memory budgets (Critic — do not conflate):

  • ink_picture_cache (vector ink): LRU<revision, ui.Picture>, ~812 mounted-page Pictures, dispose deferred to post-frame. Budget ~ a few MB (vector op-lists are cheap). No DPI bucket.
  • page_tile_cache (raster page bitmaps): LRU<TileKey, ui.Image>. This is the heavy one. A single A4 page rendered at 3× DPI is ~1030 MB (≈ 1785×2526 px × 4 bytes ≈ 18 MB at 3× of a 595×842 pt page @ ~2 dppt). So 812 tiles at 3× would be ~150350 MB — the old "≤64 MB / 812 pages" figure (R10) was unit-confused (it conflated ink Pictures with page bitmaps). Resolution: size the tile window to the device memory budget, not a fixed page count — e.g. keep full-DPI tiles only for the visible + ±1 pages (≈ 4 in double-page), and downgrade off-window pages to a 1× thumbnail tier (matrix-upscaled, accepted as blurry only while scrolling). The ink cache keeps its wider ~812 window (cheap). R10's ≤64 MB now applies to ink + downgraded-tier tiles; the small high-DPI tile set is budgeted separately (~64128 MB depending on device), tuned in P0.5/P1.

Data structures: PageWindow { int first, last }; Map<int, HostState> mountedHosts; TileKey { hostId, dpiBucket }. Acceptance: all four modes render; switching recenters; double-page shows two pages side-by-side; ink tracks scroll/zoom; page text/rules stay crisp at 4× zoom (no GPU-upscale blur) — DPI refreshes on zoom-settle (R11); combined cache memory stays within the device budget under a full scroll; perf gate (§7) holds.

F3 — Book-like reading (P1)

  • layout/reader_controller.dart [ADD]ReaderMode { read, annotate }; in read the arbiter never draws (pen ignored); page-flip animation for pagedSingle/Double via PageView physics; spread layout from pagedDouble.
  • ui/editor_screen.dart [MODIFY] — mode toggle in toolbar; quick-jump via slider/grid (no keyboard).

Acceptance: read mode blocks ink; annotate mode draws; page-flip animates; two-page spread aligns facing pages; quick-jump scrolls/animates to target.

F4 — Thumbnail-grid navigation (P1)

  • ui/thumbnail_grid.dart [ADD] — Drawboard-style grid; tiles render via pdf/pdf_document_source.renderTile (replaces lib/services/thumbnail_service.dart syncfusion path). Tap → jump; drag-reorder for notebook page order (F6). Slider remains the linear scrubber (already in pen_editor_screen._buildPagePill).
  • lib/services/thumbnail_service.dart [MODIFY/RETIRE] — migrate to pdfrx render; drop syncfusion_pdfviewer_platform_interface dep once ported.

Acceptance: grid shows all pages; tap jumps; drag reorders (F6); no text-field page input.

F5 — Configurable pen (P2)

  • input/pen_config.dart [ADD]PenConfig { ButtonAction sideButton; ButtonAction eraserEnd; PressureCurveType curve; double palmSensitivity; bool fingerDrawing; }; enum ButtonAction { eraser, undo, toggleTool, pan, lasso, none }. InputArbiter consults it (replaces hardcoded _isEraserSignal).
  • lib/providers/settings_provider.dart [MODIFY] — persist PenConfig (extend existing SharedPreferences notifier; it already stores pressure curve + stabilization).
  • lib/screens/settings_screen.dart [MODIFY] / [ADD] ui/pen_settings_page.dart — mapping UI (SpeedyNote-style dialog).
  • Reuse lib/models/pressure_curve.dart (already has linear/soft/hard/custom + apply).

Acceptance: remapping side-button to undo makes the barrel button undo; pressure curve changes stroke taper; palm sensitivity changes touch-cooldown; finger-drawing toggle works; all persist across restart.

F6 — One-notebook-per-PDF (P2)

  • pdf/pdf_document_source.dart [MODIFY] — a page-map: logical notebook pages → either a source-PDF page index or a synthetic blank page. Insert-blank adds a synthetic page without rasterizing or editing the source bytes (keep source vector + searchable). Ink/text bind to the logical page id (stable UUID per logical page), not raw PDF index, so inserts don't reshuffle annotations.
  • lib/services/pdf_service.dart [MODIFY] — keep headless syncfusion export/mutate; export walks the page-map. Fix the hairline bug: _renderStrokes must build a PdfPath from buildStrokeOutline points and fill it (currently strokes line-segments → hairline). Shape/line/arrow keep stroke semantics.
  • DB [MODIFY]notebook_pages(id, document_id, ordinal, source_page_index INTEGER NULL, kind); ink keyed by notebook_page_id (§4). Portable bundle = zip {source.pdf, badnote.json(strokes/text/links/page-map)} + relink-on-open (match by content hash, fall back to picker).

Acceptance: insert blank page between PDF pages → source PDF untouched (still vector/searchable in another viewer); reorder pages keeps ink attached; export .pdf shows filled ink matching screen (golden); bundle round-trips on another machine.

F7 — Infinite board + 双链 (P3)

  • board/board_host_pane.dart [ADD]BoardHost (absolute px) in a constrained-false InteractiveViewer; reuses AnnotationLayer + engine. Migrates lib/screens/split_view_screen.dart + scratchpads table.
  • board/link_graph.dart [ADD]Link { srcRef, dstRef } where a ref is (kind: notebookPage|board|note, id); backlinksOf(ref). Wiki-style [[...]] parsing in text boxes; sticky-note = a small board region or text box that can link to a page/note. Backlink panel queries links table.
  • DB [ADD]links(id, src_kind, src_id, dst_kind, dst_id, created_at) + indexes both directions; boards(id, document_id NULL, strokes_json, ...) (generalize scratchpads).

Acceptance: create a sticky linking page 3 → note X; open note X shows a backlink to page 3; board draws at perf target; link graph survives restart; deleting a target leaves a dangling-link indicator (no crash).

F8 — Library-wide full-text search (P3)

  • search/search_indexer.dart [ADD] — index three sources into a unified library index: (a) PDF embedded text (pdfrx PdfPage.loadText/charRects text API — source-pinned in P0.5 per SF4), (b) typed text boxes (content), (c) handwriting OCR (existing services/ocr text-line ONNX over rasterized strokes via StrokeRasterizer). Reuse existing document_fts FTS5 + a new library_fts spanning notebooks/boards/notes with (ref_kind, ref_id, page, snippet).
  • search/ocr_ingest.dart [ADD] — batch handwriting OCR per logical page (best-effort; set expectations: text-line only, no math). Background isolate; debounced after ink idle. The OCR backend itself no-ops when the model is unavailable (onnx_recognition_backend.dart header: "verify on-device") — search MUST never block on, nor be gated by, OCR results.
  • lib/screens/search_screen.dart [MODIFY] — unified results with snippets + jump-to-location (open editor at the page + scroll, or board at the region).

Acceptance — split into two exit tiers (SF2):

  • COMMITTED (P3 exit blocker): PDF embedded text + typed-text-box search returns results with snippets; tapping a result opens and scrolls to the page/region. This tier alone satisfies the P3 search exit (it has no unverified-recall dependency).
  • ADDITIVE (best-effort, NOT a P3 exit blocker): handwriting OCR contributes hits when the text-line model is available and confident; a word handwritten on page 5 may be returned. Expectation set: handwriting recall is limited by the text-line model and may return nothing on cursive/handwriting; math/formula is explicitly P5. P3 ships even if handwriting recall is poor, with UI copy stating the limit.

F9 — Server sync + AI refinement (P4)

  • sync/sync_client.dart [ADD] — wire the already-built FastAPI endpoints (/api/auth, /api/sync/push|pull, /api/notes, /api/documents, /api/ocr). Last-writer-wins by updated_at (server already implements this). Sync notes + (later) notebooks/boards.
  • server/badnote_server/ [MODIFY] — extend sync_router/models.py to cover notebooks/boards/links (today only notes). Add an AI-refine endpoint that runs llm_wiki/VLM/LLM over a note/page → returns organized markdown (server-side, heavy deps gated like the OCR worker).
  • lib/providers/ [ADD] sync state provider; settings page server URL/token.

Acceptance: push from device A, pull on device B reproduces notes; AI-refine returns organized text for a selected note; offline still fully functional (server optional, per README).

F10 — Ink CAS / formula (P5)

  • search/ocr_ingest.dart [MODIFY] — add a formula/math recognition backend (new model; the current PP-OCRv4 is text-line only — this is the known hard sub-problem). Behind a feature flag.
  • engine/ [ADD] cas/ — recognized formula → CAS (compute/solve) behind a toggle; renders result near the ink. Reuses the CoordinateSpaceHost seam for overlay placement.

Acceptance (stretch): a handwritten 2+3= toggled → shows 5; recognized formulas become searchable. Gated, optional, lowest priority.

F11 — Modern UX polish (woven P0P3)

  • Material You already wired (main.dart DynamicColorBuilder + harmonized schemes + Inter). Continue: subtoolbars in editor_toolbar, drag-reorder thumbnails (F4/F6), hover pre-warm (warm the next page tile + Picture on stylus hover to cut pen-down latency — reuse the live _onPointerHover seam), animated mode/layout transitions.

3. Data model & persistence (clean schema — data may reset)

Bump DB to a fresh version with _onCreate only (keep _onUpgrade harmless). Today: notes/strokes, documents/annotations(JSON-per-page), bookmarks, ocr_results, document_fts(FTS5), scratchpads (DB v5, lib/services/database_service.dart).

Target tables (additions/changes in bold):

  • documents [KEEP] (drop editor reliance on rotation; page-map owns rotation).
  • notebook_pages(id PK, document_id FK, ordinal INTEGER, source_page_index INTEGER NULL, kind TEXT, created_at) [ADD] — F6 logical pages.
  • ink(id PK, host_kind TEXT, host_id TEXT, stroke_json TEXT, ordinal INTEGER, updated_at) [ADD] — strokes addressed by host (host_kind ∈ {page, board}, host_id = notebook_page_id or board_id). Per-stroke rows (not per-page blob) so a 2,000-stroke page doesn't rewrite on every save — but ONLY valid under the MF3 diff-write contract (§2/F1 editor_repository.saveHost): UPSERT changed rows + DELETE removed rows, never delete-all+re-insert. Without diffing, per-row is worse than a blob; the contract is what makes this schema correct. Index (host_kind, host_id).
  • text_boxes(id PK, host_kind, host_id, content TEXT, rect_json, font_size, color, updated_at) [ADD] — F2-area text + F8 indexing.
  • boards(id PK, document_id FK NULL, title, created_at, updated_at) [ADD] — generalize scratchpads; board strokes live in ink with host_kind='board'.
  • links(id PK, src_kind, src_id, dst_kind, dst_id, created_at) [ADD] — 双链; indexes on (src_kind,src_id) and (dst_kind,dst_id).
  • bookmarks [KEEP].
  • ocr_results [KEEP/EXTEND] — per logical page handwriting OCR text.
  • library_fts (FTS5) [ADD](ref_kind, ref_id, page, content) unified search over PDF text + typed text + OCR. document_fts [KEEP] for PDF-page text.
  • notes/strokes/notes_fts [KEEP] (existing ink-note path; eventually folded into boards, but not deleted in P0).
  • sync_state(entity_kind, entity_id, last_pushed_at, last_pulled_at, dirty) [ADD, P4].

Coordinate semantics on disk: page ink = normalized [0,1] unrotated; board ink = absolute logical px. (Matches live conventions.)

Stroke-model convergence (important): there are currently two stroke models — PenStroke/PenPoint (live canvas, in-memory only) and InkStroke/InkPoint (freezed/JSON, DB+OCR+export). P0 introduces one canonical EditorStroke/EditorPoint (freezed/JSON, normalized) and adapters to/from InkStroke for OCR/export reuse during transition; old screens keep InkStroke until retired (§6).


4. Refactor strategy (evolve, don't rewrite)

Principle: every phase ships; the engine generalizes under load.

  1. P0 = relocation + persistence. Move the 4 live canvas files into engine/ + render/ + input/ with minimal logic change; add StrokeStore.revision + ui.Picture cache + DB persistence via editor_repository. The live single-page editor keeps working throughout. Keep PenStroke as a thin alias of EditorStroke until callers migrate.
  2. P0.5 = vertical slice (continuous-single only) — the new perf/crispness gate (see §5/§9). Stand up layout/page_viewport + pdf/page_tile rendering continuous-SINGLE only, with the rewritten perf bench (targeting ui/editor_screen, not the spike) and the zoom-DPI refresh proven crisp at 4× on the Surface. This proves the own-canvas multi-page + multi-resolution model on the real device before any double/paged/spread work.
  3. P1 = full multi-page windowing. Wrap the (now-relocated) AnnotationLayer in page_viewport; _strokesByPage Map → repository-backed mounted hosts. Single-page path stays as pagedSingle. Continuous-double / paged / spread land here, AFTER P0.5 passes.
  4. Old-code retirement (gated on parity, then delete in one step):
    • [DELETE after P1 parity] lib/screens/pdf_annotator_screen.dart, lib/widgets/pdf_annotation_layer.dart, lib/widgets/ink_canvas.dart (extract any unique draw/erase logic to engine/ first), lib/editor/pdf/spike_*.dart (throwaway M1 spike — includes the spike-based perf bench, replaced in P0.5), lib/editor/pdf/pen_capture_region.dart + PenCaptureBinding in main.dart (own-canvas Listener model won — remove the unused arena-bypass binding once P0 device-confirms).
    • [DELETE in P3, NOT P1 (SF3)] lib/screens/split_view_screen.dart — its replacement is the P3 infinite board (F7); deleting it in P1 would leave a 2-phase functionality gap (no scratchpad between P1 and P3). It stays live until the board lands.
    • [KEEP] thumbnail_service until F4 ports it; stroke_rasterizer (OCR), ctc_decoder, onnx_recognition_backend, OCR assets; pdf_service (export/mutate, with the fill fix); pptx_service/ppt_annotator_screen (PPT is separate; not in scope but not deleted).
  5. Navigation swap: home_screen _openDocument/_importPdf currently push PdfAnnotatorScreen; openM1Spike pushes PenEditorScreen. Repoint both to ui/editor_screen.dart once P1 parity passes (checklist below).
  6. Parity checklist before any delete: page rotate/delete/insert-blank/insert-image (pdf_service), bookmark add/toggle/jump, undo/redo across pages, save-on-leave, zoom in/out/fit, export-matches-screen (golden). Mirrors phase-1 §10 M3 checklist C1C11.

5. Phased delivery (each = shippable milestone with exit criteria)

Sequence de-risks: pen core + persistence first (P0), then the 60fps multi-mode layout (P1), then config/notebook (P2), then board/双链/search (P3), then sync/AI (P4), then CAS (P5).

P0 — Pen core hardened + persisted (own-canvas). Exit: draw/erase/undo/redo on a single PDF page persist to DB and reload; EditorStroke canonical model (superset of InkPoint, SF1) + revision-gated ui.Picture cache; editor_repository.saveHost honors the MF3 diff-write contract (test: erase 1 of 2,000 ⇒ 1 DELETE, 0 re-inserts); pure InputArbiter + stroke_eraser unit-tested; on-device Surface Pen confirms pressure + palm rejection + pinch-zoom in the live shared-transform model (the one device gate). Export hairline bug fixed (fill). No regression to existing screens.

P0.5 — Vertical slice gate (own-canvas multi-page + multi-resolution), continuous-SINGLE only. [NEW — SYNTHESIS] Stand up layout/page_viewport + pdf/page_tile rendering continuous-single only on the real document path (ui/editor_screen), and:

  • (a) REWRITE the perf bench (§7/§8): a new integration_test/editor_scroll_bench.dart drives ui/editor_screen + layout/page_viewport; DELETE the spike-based integration_test/perf_scroll_bench.dart (it imports spike_editor_pane.dart / pageOverlaysBuilder = the invalidated Option B; it CANNOT validate own-canvas).
  • (b) Prove crisp-on-zoom (R11): page-tile DPI refresh on zoom-settle renders crisp text/rules at 4× on the Surface (no GPU-upscale blur).
  • (c) Source-pin the pdfrx render/text APIs (SF4): confirm PdfPage.loadText/charRects/PdfPage.render()/PdfPageView signatures present in pdfrx 2.4.4; smoke-test each, so F4/F8 don't hit drift late.
  • Exit (hard gate before any double/paged/spread): continuous-single median build+raster ≤ 16.6ms, p95 ≤ 22ms on the 300-page asset via the REWRITTEN bench, AND a manual crisp-on-zoom-at-4× PASS on the Surface, AND the four pdfrx APIs source-pinned + smoke-tested. Continuous-double / paged / spread do NOT begin until P0.5 is GREEN.

P1 — Full layout modes + book-like reading + thumbnails (60fps). Precondition: P0.5 GREEN. Exit: the remaining PageLayouts (continuous-double, pagedSingle, pagedDouble); reader vs annotate; page-flip + two-page spread; thumbnail-grid jump + drag-reorder; windowed lazy hosting; perf gates met (§7) incl. continuous-double on a 300-page PDF via the rewritten bench; old PDF screens deleted after parity checklist (split_view retirement deferred to P3 per SF3); nav repointed.

P2 — Configurable pen + one-notebook-per-PDF. Exit: Pen settings page (button/eraser mapping, curve, palm sensitivity, finger toggle) persists + drives arbiter; insert-blank/reorder pages keep source vector + ink attached to logical pages; portable bundle round-trips; export walks page-map.

P3 — Infinite board + 双链 + library search. Exit: board reuses engine at perf; sticky-notes + [[links]] produce backlinks; library search COMMITTED tier (PDF embedded text + typed-text-box) returns snippets + jump-to-location (this tier is the exit blocker, SF2); handwriting-OCR tier is ADDITIVE/best-effort and does NOT block P3 exit; split_view_screen retired into board (the SF3 deletion point).

P4 — Server sync + AI refinement. Exit: push/pull notes+notebooks across two devices (LWW); AI-refine endpoint returns organized markdown; app fully functional offline.

P5 — Formula OCR + ink CAS. Exit (stretch, gated): formula recognition backend; searchable formulas; CAS solve toggle for simple expressions.

Every milestone ends with verifier/critic pass + perf-results doc updated with commit hash.


6. Refactor of input transport (record the pivot)

The live PenCanvas proves the own-canvas model: a single Listener over an InteractiveViewer whose child is Stack(PdfPageView bitmap, StaticInk, LiveInk). Because the pen, page bitmap, and ink share one transform and the Listener arbitrates by _activePointers.length + kind, there is no gesture-arena fight — pdfrx never sees gestures (it only renders). This removes the phase-1 need for PenCaptureRegion/PenCaptureBinding (RenderProxyBox arena bypass), which exist for the abandoned pageOverlaysBuilder approach. Retirement gate: delete them once P0 confirms on the physical Surface Pen that the Listener model handles stylus draw + single-finger pan + pinch-zoom + palm rejection (the live code is built for exactly this; confirm, then remove the dead binding).


7. Risks & mitigations

# Risk L I Mitigation / trigger
R1 60fps continuous + double-page on a 300-page PDF. Two columns × windowed tiles × ink Pictures may blow frame budget. Med High Windowed lazy hosting (only visible+cacheExtent mounted), bounded LRU + post-frame dispose for BOTH the ink_picture_cache and the page_tile_cache (R10), per-page RepaintBoundary, revision-gated static Picture. Gate continuous-SINGLE in P0.5, continuous-double in P1 (§5/§9). MF1 — the existing integration_test/perf_scroll_bench.dart is UNUSABLE here: it imports spike_editor_pane.dart and benchmarks pageOverlaysBuilder (Option B, the INVALIDATED architecture) — it cannot validate own-canvas. P0.5 REWRITES the bench against ui/editor_screen + layout/page_viewport and DELETES the spike-based one. Uses large_300p.pdf.
R2 Pen feel only verifiable on the user's Surface (CI has no pen). High Med P0 device gate (pressure/palm/pinch) + manual checklist in perf-results doc; synthesized-stylus widget tests as interim signal only.
R3 Handwriting/formula OCR feasibility. Current model is text-line PP-OCRv4; math is unsolved. High Med F8 ships text-line best-effort with explicit expectation-setting; formula isolated to P5 behind a flag; never block search on OCR quality.
R4 双链 graph scale (thousands of links/sticky-notes). Low Med Indexed links table (both directions), lazy backlink queries, no in-memory full graph.
R5 Server/AI scope creep. Med Med Server stays optional (README); P4 wires existing endpoints + one AI-refine route; AI heavy deps gated like OCR worker.
R6 Windows pen edge cases (no advertised pressure range; barrel-button eraser; inverted stylus). Med Med Live _normalizedPressure already handles degenerate ranges; F5 makes button/eraser mappable; test matrix in arbiter unit tests.
R7 Export fidelity (hairline bug today). High (today) Med stroke_geometry.buildStrokeOutline shared screen+export; pdf_service._renderStrokes fills a PdfPath; golden test (P0).
R8 Stroke-model convergence churn (two models today). Med Low One canonical EditorStroke + adapters; old InkStroke retained only where old screens/OCR/export still use it, deleted with them.
R9 pdfrx page-render / text API drift (thumbnails, search text, double-page sizing). Med Low Source-pin PdfDocument/PdfPageView/PdfPage.loadText/charRects/render() signatures + smoke-test in P0.5 (pulled forward, SF4) so F4/F8 don't hit drift late; keep syncfusion export until verified.
R10 Cache memory budget vs many mounted pages (TWO caches — Critic). Med Med ink_picture_cache (vector ui.Picture, no DPI bucket): ~812 mounted pages, a few MB. page_tile_cache (raster ui.Image, DPI-bucketed): the heavy store — a 3×-DPI A4 tile is ~1030 MB, so 812 high-DPI tiles would be ~150350 MB. The old "≤64MB / 812 pages" figure was unit-confused (conflated ink with page bitmaps). Fix (§2/F2): keep full-DPI tiles only for visible ±1 pages, downgrade off-window pages to a 1× thumbnail tier; ≤64MB applies to ink + downgraded tiles, the small high-DPI tile set budgeted separately (~64128MB, device-tuned). Sample BOTH caches during the scroll bench.
R11 Blurry page at high zoom under the shared transform (MF2). One InteractiveViewer matrix-scales a PdfPageView bitmap that was rendered at layout-constraint × devicePixelRatio (1×); pinch-zoom GPU-upscales it → blurry text/rules at 4×, whereas pdfrx's own viewer re-renders crisp tiles per zoom level. We own multi-resolution tiling, not just layout/windowing. Med High pdf/page_tile.dart drives render DPI from the transform scale: on zoom-settle re-instantiate the tile (re-laid-out PdfPageView at the new pixel size, or PdfPage.render() at target DPI) into the pdf/page_tile_cache.dart store (LRU<TileKey{hostId, dpiBucket}, ui.Image> — NOT the ink cache); cache key includes a DPI bucket; cap retained DPI (~3× base) to bound memory; downscale stays matrix-only. PdfPage.render() returns an async PdfImage owning a native handle → page_tile_cache owns its post-frame dispose (see Open Questions; confirm in the SF4 P0.5 source-pin). Proven crisp at 4× on the Surface in the P0.5 gate (§5). Budgeted P0.5/P1, not relocation.

Pre-mortem (DELIBERATE — 3 scenarios)

  1. "Continuous double-page janks / pages are blurry at 4× on the user's big scanned PDF." Cause: built layout modes before profiling two-column windowing, and matrix-scaled a 1× bitmap (R11). Prevention: the P0.5 vertical-slice gate profiles continuous-SINGLE first with a rewritten bench against ui/editor_screen (the spike-based perf_scroll_bench.dart is deleted — it benchmarks the invalidated Option B) AND proves crisp-on-zoom DPI refresh at 4× on the Surface; continuous-double/paged/spread merge only after P0.5 is GREEN, and the P1 double-page gate reuses the rewritten bench.
  2. "Search returns nothing for handwriting." Cause: over-promised OCR. Prevention: F8 ships PDF-text + typed-text search first (reliable), handwriting OCR as additive best-effort with UI copy stating limits; formula explicitly P5.
  3. "Insert-blank-page silently rasterized / detached annotations." Cause: editing source bytes or keying ink to raw PDF index. Prevention: logical page-map + ink keyed to notebook_page_id; golden test that the source PDF bytes are unchanged after insert and remains selectable-text in an external viewer.

8. Testing strategy

sqlite workaround: DB-touching tests run via tool/test.sh (system sqlite + LD_LIBRARY_PATH); pure-logic tests avoid the DB. Reusable assets: tool/gen_bench_pdf.dart, tool/gen_dense_strokes.dart, test/assets/large_300p.pdf, test/assets/dense_strokes.json, integration_test/coordinate_assertion_test.dart. NOT reusable: integration_test/perf_scroll_bench.dart — it imports spike_editor_pane.dart (pageOverlaysBuilder, invalidated Option B) and is DELETED + replaced by integration_test/editor_scroll_bench.dart (targets ui/editor_screen) in P0.5 (MF1).

  • Unit (no DB/widgets):
    • Coordinate transforms: NormalizedPageHost/BoardHost round-trip toContent(toDevice(x))≈x.
    • Stroke geometry: buildStrokeOutline non-empty for ≥1 point; live ⊆ committed bounds (no "pop").
    • Eraser: eraseHits whole-stroke removal (extraction of live pen_canvas._eraseAt — regression port). splitStroke segment-erase is NEW behavior, not a port (the live canvas only does whole-stroke erase; undo_manager records but never computes replacements): mid-erase ⇒ 2 segments, endpoint ⇒ 1 segment, full-erase ⇒ empty, <2-pt result dropped — these exercise greenfield code. (undo_manager_test.dart informs the replacement-bookkeeping discipline only.)
    • Revision gating: StrokeStore.add bumps revision; StaticInkPainter.shouldRepaint iff revision changed.
    • InputArbiter SM: table-driven over the device×mode matrix incl. palm rejection (touch dropped while stylus active) and pointerCount>=2→cancel.
    • Undo: global commit-time order, host-tagged reversal.
    • SaveScheduler: mutating "current host" after schedule but before write completes does not change persisted snapshot.
    • editor_repository.saveHost diff-write (MF3, fake DB counting statements): erase 1 of 2,000 strokes ⇒ exactly 1 DELETE, 0 INSERT; add 1 stroke ⇒ 1 INSERT, 0 DELETE; no-op save ⇒ 0 statements. (Guards against the live notes-path delete-all+re-insert anti-pattern at database_service.dart:281306.)
    • EditorStrokeInkStroke adapter round-trip (SF1): tilt/timestamp/pointerDeviceKind survive both directions (no lossy OCR/export conversion).
    • link_graph.backlinksOf (P3); search_indexer snippet/jump-ref mapping for the COMMITTED tier — PDF-text + typed-text (P3).
  • Widget: stylus→stroke committed, touch→not consumed (synthesized pointers); layout-mode switch recenters; reader-mode blocks ink; thumbnail tap jumps; text-box place/move/edit persists (fake DB).
  • Perf: the rewritten integration_test/editor_scroll_bench.dart (targets ui/editor_screen + layout/page_viewport; the spike-based perf_scroll_bench.dart is deleted, MF1) on large_300p.pdfcontinuous-single in P0.5 (gate), continuous-double in P1, profile mode, N≥120 frames warm, median build+raster ≤16.6ms / p95 ≤22ms; StaticInkPainter no-rebuild assertion; Picture-memory ≤64MB sample. Archived in docs/plans/full-refactor-perf-results.md with commit hash. Not a hard CI gate (no GPU) but required for milestone sign-off.
  • Manual on-device pen checklist (Surface Pen, P0 + each milestone): pressure varies width; barrel/inverted = erase; palm rest doesn't mark; single-finger scroll; two-finger pinch; page text/rules stay crisp at 4× zoom — no GPU-upscale blur, DPI refreshes on zoom-settle (R11, P0.5 gate); hover pre-warm reduces first-stroke latency; page-flip feels book-like. Recorded with device + commit.
  • Regression: keep ctc_decoder_test.dart, undo_manager_test.dart green; update widget_test.dart to boot editor_screen.
  • Export golden (P0): annotate a known page → export → image-compare filled ink matches screen (R7).
  • Server (P4): existing server/tests/ (test_sync.py etc.) green; add notebook/board sync tests.

9. Milestones / sequencing — immediate next chunk (concrete)

Next chunk = P0 (engine relocation + persistence), executable now; followed by the P0.5 vertical-slice gate (steps 1013) before any double/paged/spread layout work:

  1. [ADD] lib/editor/engine/stroke_model.dartEditorStroke/EditorPoint (freezed + JSON), normalized; fromPenStroke/toInkStroke adapters. Run build_runner.
  2. [ADD] lib/editor/engine/stroke_geometry.dart — lift buildStrokePathbuildStrokeOutline (verbatim recipe from ink_painters.dart).
  3. [ADD] lib/editor/engine/stroke_store.dart + lib/editor/render/{static_ink_painter,live_ink_painter,ink_picture_cache,annotation_layer}.dart — relocate live painters; add the resolution-independent ink ui.Picture cache keyed by revision (NO DPI bucket; the DPI-bucketed page_tile_cache is a separate P0.5 file, step 10).
  4. [ADD] lib/editor/input/input_arbiter.dart + lib/editor/engine/stroke_eraser.dart — extract from pen_canvas.dart (pure, unit-tested).
  5. [ADD] lib/editor/persistence/{editor_repository,save_scheduler}.dart + DB additions (ink, notebook_pages minimal) in database_service.dart (fresh version). saveHost implements the MF3 diff-write contract (UPSERT changed + DELETE removed, by stroke id; NO delete-all+re-insert) with a per-host last-persisted id-set.
  6. [MODIFY] lib/editor/canvas/pen_editor_screen.dart (or new ui/editor_screen.dart) to load/commit/save through the repository instead of the in-memory _strokesByPage Map.
  7. [MODIFY] lib/services/pdf_service.dart _renderStrokes → fill buildStrokeOutline path (R7) + export golden test.
  8. Tests: arbiter SM, eraser, geometry, revision-gating, save-scheduler snapshot, diff-write statement-count (MF3: erase 1/2000 ⇒ 1 DELETE 0 INSERT), EditorStroke↔InkStroke lossless round-trip (SF1), export golden. Run via tool/test.sh.
  9. Device gate: build Windows package (CI), confirm pen/palm/pinch on Surface Pen; record in perf-results doc. → unblocks P0.5.

Then P0.5 (vertical-slice gate — must pass before P1's double/paged/spread): 10. [ADD] lib/editor/layout/page_viewport.dart (continuous-single only) + lib/editor/pdf/{pdf_document_source,page_tile,page_tile_cache}.dart with zoom-settle DPI refresh (R11) + the DPI-bucketed page_tile_cache (LRU<TileKey, ui.Image>, page bitmaps; owns native-handle dispose). (The resolution-independent render/ink_picture_cache.dart for vector ink lands in P0, step 3 — it is a separate cache, no DPI bucket.) 11. [ADD] integration_test/editor_scroll_bench.dart targeting ui/editor_screen; [DELETE] integration_test/perf_scroll_bench.dart + lib/editor/pdf/spike_*.dart (the spike pane the old bench imports). Run the rewritten bench on a scanned-image 300-page asset (see Open Questions — large_300p.pdf is synthetic/vector and may not honestly stress raster re-render at 3× DPI) → continuous-single median ≤16.6ms / p95 ≤22ms; sample both caches' memory. 12. [VERIFY] source-pin pdfrx PdfPage.loadText/charRects/render()/PdfPageView in 2.4.4 (SF4) + smoke test; confirm PdfPage.render()'s PdfImage/native-handle ownership + dispose semantics so page_tile_cache can manage post-frame disposal; record signatures in perf-results doc. 13. Device gate: crisp-on-zoom at 4× PASS on the Surface (R11). → unblocks P1.

Each subsequent milestone (P1…P5) follows §5 exit criteria; verifier/critic + perf-results update per milestone.


10. RALPLAN-DR

Principles (35)

  1. Single source of truth = host content coordinates. Screen mapping is a paint-time canvas transform; never store/duplicate transformed geometry. (Already the live convention — generalize it.)
  2. One host-agnostic ink engine. PDF page, infinite board, and (P5) CAS overlay are CoordinateSpaceHosts behind one renderer — never fork the stroke pipeline.
  3. Own the gesture pipeline; pdfrx only renders. A single Listener over a shared InteractiveViewer arbitrates draw/pan/zoom/palm by pointer kind + count — no gesture-arena fights (proven live).
  4. De-risk performance AND crispness before features. A P0.5 vertical-slice gate (continuous-single, rewritten bench, crisp-at-4× on the Surface) precedes all double/paged/spread work; continuous-double is gated again in P1.
  5. Ship every phase; generalize under load. Relocation-first refactor keeps the editor working at all times; old screens deleted only after parity.

Decision Drivers (top 3)

  1. D1 — Pen feel + palm rejection + pinch on Windows Surface Pen is the make-or-break primary-device requirement (own-canvas model already targets it).
  2. D2 — 60fps across all layout modes on big PDFs with thousands of strokes.
  3. D3 — One engine reused across PDF page, infinite board, and CAS, forward-compatible with search/双链/sync.

Viable options (≥2) with bounded pros/cons

Option A — Own-canvas engine (single Listener + InteractiveViewer, pdfrx as renderer). CHOSEN (already live).

  • Pros: D1 solved structurally (one transform, no arena fight — already working in pen_canvas.dart); D3 trivial (hosts share the transform); minimal new deps; matches the proven Saber model the user cited.
  • Cons: we own page layout/windowing/tiling AND multi-resolution re-rasterization (more code than a stock viewer); double-page perf is on us (R1) and crisp-on-zoom is on us (R11 — pdfrx's own viewer re-renders crisp tiles per zoom for free; we must re-derive tile DPI from the transform); thumbnails/text-extraction still need pdfrx page APIs (R9).

Option B — pdfrx pageOverlaysBuilder-hosted ink + RenderProxyBox arena-bypass (the phase-1 plan).

  • Pros: pdfrx gives continuous scroll/tiling/text-extraction for free; ink-follows-page is structural via page overlays.
  • Cons: requires a custom PenCaptureBinding arena bypass that fights pdfrx's greedy scale recognizer (the live code already abandoned this — PenCaptureRegion is unused); pdfrx owns the transform so double-page/board reuse is awkward; D1 proven harder than Option A in practice. Invalidated — see below.

Option C — Flutter shell + Rust hot-path (rnote-style) for ink/render.

  • Pros: maximal ink perf headroom.
  • Cons: rejected in memory (badnote-flutter-344-windows-pen) — Windows pen is weaker in the Rust/GTK stack; huge FFI surface; contradicts "stay Flutter." Invalidated.

Invalidation rationale

  • B invalidated (as the INPUT model): the live codebase already moved off it; pageOverlaysBuilder+arena-bypass made stylus/touch arbitration fight pdfrx's recognizer, whereas Option A's single-Listener-over-shared-transform sidesteps the arena entirely and is already drawing with pressure/pinch/palm. Correction (MF2): B had TWO edges, not one — (1) free continuous scroll/windowing, AND (2) free per-zoom crisp re-rasterization (pdfrx re-renders tiles at each zoom level). Edge (1) is recoverable in A via windowed hosting (needed for double-page anyway); edge (2) is NOT free in A — Option A must own multi-resolution tiling (R11), which is the genuinely new cost of this choice. We accept that cost (gated in P0.5) because A's structural D1/D3 wins outweigh it. B retained only as a fallback page-render strategy, not the input model.
  • C invalidated: documented Windows-pen regression in the Rust/GTK path + "stay Flutter" hard constraint; the 3.44 WM_POINTER fix already unblocked Flutter pen, removing C's motivation.
  • Net: A chosen; B retained as a partial fallback (page rendering only); C rejected.

ADR

  • Status: Architect APPROVE-WITH-MUST-FIX applied (2026-06-21): MF1 (rewrite perf bench off the invalidated spike), MF2 (R11 zoom re-rasterization + multi-resolution tiling), MF3 (diff-write contract); P0.5 vertical-slice synthesis gate; SF1SF5. Pending Critic. Records the live pivot from the superseded phase-1 input architecture.
  • Decision: Build the full BadNote vision on an own-canvas, host-agnostic ink engine (single Listener + shared InteractiveViewer; pdfrx as page renderer/text source). Generalize the live lib/editor/canvas/ into engine/render/input/layout/...; persist a single canonical EditorStroke; phase features P0→P5 with a P0.5 vertical-slice gate.
  • Drivers: D1 Surface-Pen feel, D2 60fps multi-mode, D3 one reusable engine.
  • Alternatives considered: B (pdfrx-overlay + arena bypass — invalidated as input model, kept as render fallback; had two free edges — scroll AND per-zoom crispness — the latter is the new cost we take on), C (Flutter+Rust — rejected).
  • Why chosen: A is already proven live for pen/palm/pinch and gives D1+D3 structurally; the remaining risks (D2 multi-mode perf AND R11 crisp-on-zoom) are gated up front in P0.5 then re-gated for double-page in P1.
  • Consequences: we own layout/windowing/tiling AND multi-resolution page-tile re-rasterization (R11 — not just layout, MF2); PenCaptureRegion/PenCaptureBinding retired (dead under own-canvas); the spike-based perf bench is deleted and rewritten against ui/editor_screen (MF1); per-stroke-row persistence is valid ONLY under the diff-write contract (MF3); EditorPoint is a non-lossy superset of InkPoint (SF1); export must fill (not stroke) ink; handwriting OCR is additive/non-blocking and formula is P5 (SF2); split_view_screen deletion deferred to P3 (SF3); server stays optional.
  • Follow-ups: P0 device gate confirms pen model; P0.5 gate proves continuous-single perf + crisp-at-4× + source-pinned pdfrx APIs (SF4) before any double/paged/spread; resolve open questions below.

Open Questions (persist to .omc/plans/open-questions.md)

  • Continuous-double-page on a real 300-page scanned PDF — does windowed two-column hosting hold 60fps, or do we need tile pre-rasterization? — P0.5 gates continuous-single first; P1 perf gate decides double; affects R1/R10.
  • How is page-tile DPI refreshed on zoom under the shared transform (SF5)? — re-laid-out PdfPageView at the new pixel size, or PdfPage.render() at target DPI into page_tile_cache? What scale-change threshold + debounce triggers a refresh, and what is the retained-DPI cap? — Load-bearing for the 60fps + crisp-at-4× goal; resolved in P0.5 (R11).
  • PdfPage.render() native-handle lifecycle (Critic 4a): render() returns an async PdfImage owning a native handle backing a ui.Imagepage_tile_cache must own its dispose (post-frame, after the raster thread is done) on eviction. Confirm exact ownership + dispose API in the SF4 P0.5 source-pin; affects page_tile_cache + R10/R11.
  • Honest R11 raster stress asset (Critic 4b): does the synthetic/vector large_300p.pdf actually force RASTER re-render cost at 3× DPI, or does a scanned-image asset better exercise the crispness + tile-memory gate? — Add a scanned 300-page asset (tool/gen_bench_pdf.dart image mode or a real scan) for the P0.5/P1 perf+crispness gates; affects §8 / R11 honesty.
  • Stroke-model convergence: fold notes/InkStroke ink-note path into boards/EditorStroke, or keep notes separate long-term? — Affects §3/§4 churn.
  • Handwriting OCR: is the existing PP-OCRv4 text-line model usable on cursive/handwriting at all, or do we need a handwriting-specific model even for non-formula text? — Affects F8 expectation-setting / R3.
  • Portable bundle relink: match source PDF by content-hash only, or also store original path + size? — Affects F6.
  • Server AI-refine: run llm_wiki/VLM server-side only, or allow a local LLM path for offline users? — Affects F9 scope / R5.
  • Keep pptx_service/ppt_annotator_screen in the new engine, or freeze PPT support? — Out of the 11 features; decide before P1 nav swap.