W1 — Custom pen width + pressure sensitivity (Saber-style):
- Root cause of "压感没用": perfect_freehand 1.0.4 IGNORES real stylus pressure
(hardcodes radius=size/2 when simulatePressure=false) — width never tracked pen
force. Upgraded perfect_freehand ^1.0.0 -> ^2.0.0 (honors real pressure); migrated
all 5 getStroke call sites to the 2.x API (PointVector / StrokeOptions / Offset).
- De-hardcoded `thinning` into `kDefaultPenThinning` (0.85), single source shared by
the on-screen painter and the PDF export path; exposed as PenConfig.pressureSensitivity
with a Pressure Sensitivity slider; live-applies via a config listener.
W3 — Native Windows pen plugin (tilt + barrel/eraser buttons):
- windows/runner/pen_channel.{h,cpp}: observe WM_POINTER at the TOP of MessageHandler
(before HandleTopLevelWindowProc, which Flutter uses to consume pen events), read
GetPointerPenInfo penFlags + tilt, stream over EventChannel('badnote/pen'); non-consuming.
- PenInputService: single latched hardware state (no Win32-pointerId<->event.pointer
correlation); graceful no-op off-Windows.
- pen_canvas maps barrel/inverted/eraser through PenConfig.sideButton/eraserEnd
(eraser/undo/toggleTool/pan) and captures tilt into PenPoint.tilt -> EditorPoint.tilt.
W2 — Zoom flicker: page raster isolated in its own RepaintBoundary (safe interim);
definitive crisp-on-zoom fix gated on the on-device root-cause probe (plan M3).
Plans: ralplan-consensus plan at docs/plans/2026-06-22-badnote-pen-polish.md
(Architect APPROVE-WITH-MUST-FIX M1-M4 + Critic ITERATE->APPROVE).
Tests: 58/58 pass incl. shared-thinning invariant + thinning-affects-outline +
tilt-adapter round-trip. flutter analyze clean; linux debug build OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
62 KiB
BadNote — Full Refactor + Feature Roadmap (own-canvas engine)
Status: PLAN (ralplan consensus — Architect APPROVE-WITH-MUST-FIX + Critic-ITERATE fixes applied 2026-06-21: MF1–MF3 + P0.5 slice + SF1–SF5 + cache-split / diff-write-durability / erase-is-new-behavior / render-handle-lifecycle; pending final Critic confirm)
Date: 2026-06-21
Mode: DELIBERATE (high-risk: engine generalization, 60fps continuous/double-page, OCR/CAS feasibility, server/AI scope)
Owner plan file: docs/plans/2026-06-21-badnote-full-refactor.md
Supersedes (in part): docs/plans/2026-06-21-badnote-phase1.md
Grounding. Written after reading the live code, not from memory. Verified sources: live editor
lib/editor/canvas/{pen_canvas,ink_painters,pen_stroke,pen_editor_screen}.dart,lib/editor/pdf/pen_capture_region.dart,lib/main.dart(dynamic_color +pdfrxFlutterInitialize+PenCaptureBinding),lib/services/{database_service,pdf_service,stroke_rasterizer,undo_manager}.dart,lib/services/ocr/onnx_recognition_backend.dart,lib/models/{ink_stroke,ink_point,pressure_curve}.dart,lib/providers/settings_provider.dart,lib/screens/home_screen.dart,server/badnote_server/{main,models,routers/sync_router}.py,server/README.md,pubspec.yaml, existingtest/+integration_test/+tool/assets.CRITICAL DIRECTION CHANGE captured here (vs phase1 plan): the live
lib/editor/canvas/has already abandoned the phase-1 "pdfrxpageOverlaysBuilder-hosted ink + RenderProxyBox arena bypass" architecture. The livePenCanvasowns oneInteractiveViewer+ a singleListenerthat draws aPdfPageViewbitmap and the ink in the same child subtree (Saber clean-room model). pdfrx is used only as a page renderer / document API (PdfDocument.openFile,PdfPageView), never for gestures.PenCaptureRegion/PenCaptureBindingare installed inmain()but are NOT wired into the live canvas and are slated for retirement (§6, §9). This plan builds the full product on the own-canvas model and records that decision in the ADR (§10).
0. Vision & Scope
BadNote is a pen-first, performant note-taking app. Primary device: Windows Tablet + Surface Pen (Flutter ≥ 3.44 for WM_POINTER stylus/multitouch); desktop for review/复习/search. Differentiators the user explicitly wants to win on: (1) library-wide full-text search over PDF text + typed text + handwriting OCR with jump-to-location; (2) modern cohesive Material You UI; (3) truly book-like reading (page-flip, two-page spread, thumbnail grid, reader-vs-annotate modes). Plus a 双链 sticky-note infinite board, one-notebook-per-PDF with insertable blank pages and never-rasterized source, and a later server sync + AI refinement and ink CAS.
Existing user data MAY be reset — no migration burden; prefer the cleanest schema. Stack stays Flutter (own gesture pipeline; pdfrx only renders pages). GFW-aware CI already solved (pdfium pre-fetch + Dev Mode + vendored sqlite3 + onnxruntime pre-fetch).
0.1 The 11 feature areas → phase map (full vision; nothing dropped, everything sequenced)
| # | Feature | Phase | Risk |
|---|---|---|---|
| F1 | Pen-first editor core (harden): pressure, palm rejection, perfect_freehand quality, eraser, tools, undo/redo, DB persistence of strokes per doc/page | P0 | Low–Med |
| F2 | Page layout modes: continuous-single/-double, paged-single/-double + switcher; lazy render @60fps | P1 | High |
| F3 | Book-like reading: page-flip, two-page spread, reader vs annotate mode, quick jump | P1 | Med |
| F4 | Thumbnail-grid navigation (Drawboard-style), slider, no keyboard input | P1 | Low |
| F5 | Configurable pen: side-button + eraser-end mapping, pressure curve, palm sensitivity, finger-drawing toggle — Pen settings page | P2 | Med |
| F6 | One-notebook-per-PDF: page-level binding, insert blank pages between PDF pages, keep PDF vector, portable bundle + relink | P2 | Med |
| F7 | Infinite 便利贴 board + 双链 bidirectional links / backlinks | P3 | Med–High |
| F8 | Library-wide full-text search (PDF text + typed text + handwriting OCR) with snippets + jump | P3 | High (handwriting OCR) |
| F9 | Server sync + AI refinement (llm_wiki, VLM/LLM organize) | P4 | High (scope) |
| F10 | Ink CAS / formula recognition → searchable + solve toggle | P5 | Very High |
| F11 | Modern UX polish: Material You, subtoolbars, drag-reorder thumbnails, hover pre-warm | woven across P0–P3 | Low |
0.2 Non-goals per phase
- P0: no layout modes, no board, no text boxes, no search UI, no settings page beyond what exists, no server. Single-page editor + persistence + eraser/undo only.
- P1: no 双链, no OCR-search, no server, no CAS. Layout/reading/thumbnails only.
- P2: no server, no CAS, no handwriting OCR. Pen-config + notebook/bundle only.
- P3: no CAS, no server-side AI. Board+双链 and local search (incl. best-effort handwriting OCR via existing text-line model; formula deferred to P5).
- P4: sync + AI organize; CAS still out.
- P5: formula OCR + CAS.
- All phases: never rasterize the source PDF; no on-screen keyboard reliance (slider/grid nav).
1. Target Architecture
1.1 Layered module decomposition (target lib/editor/)
lib/editor/
├─ engine/ # host-agnostic ink engine (pure-ish, testable)
│ ├─ coordinate_space_host.dart # CoordinateSpaceHost + NormalizedPageHost + BoardHost
│ ├─ stroke_model.dart # EditorStroke/EditorPoint (canonical; replaces PenStroke split — §3, §4)
│ ├─ stroke_store.dart # StrokeStore { committed, int revision } (O(1) shouldRepaint gate)
│ ├─ stroke_geometry.dart # buildStrokeOutline(getStroke) — single source for screen+export (kills hairline bug)
│ ├─ stroke_eraser.dart # pure splitStroke / eraseHits (extract from pen_canvas._eraseAt)
│ └─ undo_stack.dart # global, commit-time-ordered, host-tagged (generalize undo_manager)
├─ input/
│ ├─ input_arbiter.dart # pure state machine: pointerCount>=2→pan/zoom, stylus→draw, palm reject
│ └─ pen_config.dart # button/eraser-end action mapping, pressure curve, palm sensitivity (F5)
├─ render/
│ ├─ annotation_layer.dart # Stack(StaticInkPainter, LiveInkPainter, TextBoxLayer) over a host
│ ├─ static_ink_painter.dart # reads ink_picture_cache, keyed by revision
│ ├─ live_ink_painter.dart # in-progress stroke only
│ └─ ink_picture_cache.dart # LRU<revision, ui.Picture> for STATIC INK — resolution-INDEPENDENT, NO DPI bucket (vector ink re-rasterizes crisp at composite)
├─ layout/ # F2/F3 — page layout + reading
│ ├─ page_layout.dart # enum {continuousSingle, continuousDouble, pagedSingle, pagedDouble}
│ ├─ page_viewport.dart # lazy windowed page hosting (visible + cache extent), recenter
│ └─ reader_controller.dart # reader vs annotate mode, page-flip, spread, quick-jump
├─ pdf/
│ ├─ pdf_document_source.dart # wraps PdfDocument (open/dispose/pageSize/render), notebook page-map (F6)
│ ├─ page_tile.dart # PdfPageView host widget (bitmap) sized to host rect; drives tile DPI from transform scale (R11)
│ └─ page_tile_cache.dart # LRU<TileKey{hostId, dpiBucket}, ui.Image> for rendered PAGE BITMAPS — DPI-bucketed (tiles blur on upscale); owns native-handle dispose (R11)
├─ board/ # F7
│ ├─ board_host_pane.dart # infinite InteractiveViewer board, reuses engine
│ └─ link_graph.dart # 双链 backlink model + queries
├─ text/ # OneNote-like editable text boxes (P3+, optional in P2 notebook)
│ ├─ text_box_model.dart
│ └─ text_box_layer.dart
├─ persistence/
│ ├─ editor_repository.dart # batched load/save per document; maps DB ↔ engine
│ └─ save_scheduler.dart # debounced, synchronous-snapshot-before-await
├─ search/ # F8 (P3)
│ ├─ search_indexer.dart # PDF text + typed text + OCR → document_fts/library index
│ └─ ocr_ingest.dart # bridge to lib/services/ocr (text-line; formula later)
├─ sync/ # F9 (P4)
│ └─ sync_client.dart # talks to server/ (already has push/pull/auth)
└─ ui/
├─ editor_screen.dart # top-level editor (replaces pen_editor_screen + pdf_annotator_screen)
├─ editor_toolbar.dart # Material You subtoolbars (F11)
├─ thumbnail_grid.dart # F4 nav
└─ editor_shortcuts.dart # tool numbers, Ctrl+Z/Y/F/S (mouse/desktop)
1.2 Dependency graph (acyclic)
ui/ ─────────────► layout/ ─► pdf/ ─► (pdfrx PdfDocument/PdfPageView)
│ │
│ ├──────► render/ ─► engine/ (geometry, store, hosts)
│ └──────► input/ ─► engine/
├──► board/ ─────────────► render/, engine/
├──► text/ ─────────────► engine/(hosts)
├──► search/ ────────────► persistence/, services/ocr/
├──► sync/ ────────────► persistence/, server API
└──► persistence/ ───────► services/database_service.dart, engine/(stroke_model)
engine/ depends on nothing in editor/ except itself (and perfect_freehand). render/input/layout/ depend on engine. ui/ is the only place wiring Riverpod controllers. This is the seam that lets PDF-page host, infinite board, and (P5) CAS overlay all reuse one renderer.
1.3 Coordinate model — single source of truth
Truth = host content coordinates (already the live convention; generalize it):
- NormalizedPageHost: points ∈
[0,1]of the page rect (todayPenPoint(nx, ny)); width as fraction of page width (todayPenStroke.width). Stored per(documentId, pageIndex). - BoardHost: absolute logical px on an unbounded canvas;
InteractiveViewersupplies pan/zoom.
CoordinateSpaceHost exposes toContent(deviceLocal, deviceSize), toDevice(content, deviceSize), applyContentToCanvas(canvas, deviceSize). The live painters already scale nx*pageSize.width at paint time — formalize that into applyContentToCanvas (a canvas.scale) so no transformed-point copies are ever allocated. Text boxes are a sanctioned widget-space exception (Positioned widgets multiply by current zoom; stored truth stays content coords for search).
1.4 How today's lib/editor/canvas/ evolves into this (no rewrite)
| Today (live) | Becomes | Action |
|---|---|---|
pen_stroke.dart PenStroke/PenPoint (in-memory only) |
engine/stroke_model.dart EditorStroke/EditorPoint persistable (freezed/JSON, normalized) |
generalize + add JSON; converge with InkStroke (§4) |
ink_painters.dart StaticInkPainter/LiveInkPainter + buildStrokePath |
render/static_ink_painter.dart (+ render/ink_picture_cache.dart), render/live_ink_painter.dart, engine/stroke_geometry.dart |
move + add ink ui.Picture cache (resolution-independent, revision-keyed) + revision gate; buildStrokePath→buildStrokeOutline shared with export |
pen_canvas.dart _PenCanvasState gesture logic (_activePointers, _shouldDraw, palm reject, _eraseAt) |
input/input_arbiter.dart (pure SM) + render/annotation_layer.dart (widget) |
extract SM out of the widget; eraser→engine/stroke_eraser.dart |
pen_editor_screen.dart single-page + _strokesByPage Map + _transform + slider |
ui/editor_screen.dart + layout/page_viewport.dart + persistence/editor_repository.dart |
generalize single-page → windowed multi-page; _strokesByPage → repository-backed HostStates |
pen_capture_region.dart + PenCaptureBinding (unused by live canvas) |
— | retire after P0 confirms the Listener-in-shared-transform model holds on device (§6, §9) |
pen_editor_screen single PdfPageView bitmap inside the InteractiveViewer (1× layout-sized bitmap, matrix-scaled by pinch) |
pdf/page_tile.dart multi-resolution tile whose render DPI tracks the transform scale |
NEW work (not relocation): own multi-resolution tiling — pinch GPU-upscales a 1× bitmap → blurry at high zoom (R11). On zoom-settle, re-instantiate the tile (or PdfPage.render()) at a DPI matching the current scale into pdf/page_tile_cache.dart (DPI-bucketed page-bitmap cache, distinct from the ink cache). Budgeted P1/P0.5. |
Net: the four live files are promoted, not thrown away. P0 is mostly relocation + persistence + revision-gated Picture cache, which is low risk and immediately shippable. The one genuinely new burden the own-canvas choice imposes (beyond layout/windowing) is multi-resolution page-tile rendering — see R11 (§7) and the P0.5 gate (§5/§9): we own crisp-on-zoom re-rasterization that pdfrx's own viewer would have given for free.
2. Per-feature design (file-level)
F1 — Pen-first editor core + persistence (P0)
Add/modify:
engine/stroke_model.dart[ADD] —EditorStroke { id, List<EditorPoint> points, EditorTool tool, int color, double width, bool filled, String? textContent, double fontSize },EditorPoint { double x, y; double? pressure; double? tilt; int? timestamp; InputDeviceKind? pointerDeviceKind }. freezed +toJson/fromJson. (Normalized;width= fraction of page width, matching livePenStroke.) SF1 — non-lossy superset ofInkStroke/InkPoint:InkPointcarriestilt/timestamp/pointerDeviceKind(database_service.dart:346–362) that the livePenPointdrops;EditorPointincludes them now (nullable, cheap with freezed) so theInkStroke↔EditorStrokeadapter (for OCR/export reuse) is lossless in both directions. The livePenCanvassimply leaves the extra fields null at capture; existingInkStrokedata round-trips intact. (Avoids a silent data-loss footgun for OCR, which keys onpointerDeviceKind/pressure.)engine/stroke_store.dart[ADD] —class StrokeStore { List<EditorStroke> committed; int revision; add/removeAt/replace bump revision; }. Fixes the livepen_editor_screen._commitStroke"new list identity" hack by making revision explicit.engine/stroke_geometry.dart[ADD] —Path buildStrokeOutline(EditorStroke, Size, {bool isComplete})lifted verbatim fromink_painters.buildStrokePath(the provengetStroke(thinning: hl?0:0.7, smoothing:.5, streamline:.5, simulatePressure: !hasRealPressure && !hl)recipe). Single source for screen + export.engine/stroke_eraser.dart[ADD] — pureeraseHits(strokes, point, radius)(extract live_eraseAt) +splitStrokefor partial/segment erase. Critic note —splitStrokeis NEW behavior, not an extraction: the livepen_canvas._eraseAtis whole-stroke (it removes the entire stroke on the first proximity hit and returns). There is no segment-split logic in the live canvas to port —undo_manager.removeStrokeonly records replacements, it does not compute them. SosplitStroke(point-run splitting → 0/1/2 sub-strokes) is greenfield code; its tests (§8) exercise new functionality, not a regression port.eraseHits(whole-stroke) IS an extraction of the live behavior and stays available as the default erase mode.engine/undo_stack.dart[ADD] — generalizelib/services/undo_manager.dartto host-tagged, commit-time-ordered global stack (entries carryhostId+pageIndex).render/{annotation_layer,static_ink_painter,live_ink_painter,ink_picture_cache}.dart[ADD] — relocate live painters;StaticInkPainterreadsink_picture_cache(LRU<revision, ui.Picture>) keyed byrevision(shouldRepaint = old.revision != revision— O(1)); per-hostRepaintBoundary.ink_picture_cacheis for STATIC VECTOR INK only — resolution-INDEPENDENT, NO DPI bucket (aui.Pictureof vector strokes re-rasterizes crisp at composite time at any zoom). It is a different cache from the page-bitmappage_tile_cache(§2/F2, R11), which IS DPI-bucketed because raster page tiles blur on upscale. Do not conflate the two.input/input_arbiter.dart[ADD] — pure SM extracted frompen_canvas(idle→inking→erasing,pointerCount>=2→cancel+pan, palm reject = touch dropped while stylus active, eraser =kSecondaryButton || invertedStylus). Keeps the live pressure-normalization logic (_normalizedPressure).persistence/editor_repository.dart[ADD] —loadDocument(documentId) → Map<int,StrokeStore>(one batched query),saveHost(documentId, hostId, EditorStroke list).- MF3 — write contract (load-bearing for the per-row choice):
saveHostMUST diff by stroke id against the rows already on disk for that host — UPSERT only changed/new rows, DELETE only removed rows. It must NOT delete-all-rows-for-host then re-insert (that is exactly what the live notes path does at database_service.dart:281–306, and at 2,000 strokes it is slower than a single blob rewrite — re-inserting 2,000 rows per save). The per-stroke-row schema (§3) is justified only under this diff contract: erasing 1 of 2,000 strokes ⇒ 1 DELETE, 0 re-inserts; adding 1 stroke ⇒ 1 INSERT. TheSaveSchedulerhandssaveHostthe synchronously-captured stroke snapshot; the repository keeps a last-persisted id-set per host to compute the diff. If diffing proves fiddly under churn, the fallback is a per-page blob (NOT delete-all+reinsert) — but the diff path is the default and is what makes per-row worthwhile. - Durability invariant (Critic): the UPSERT(s) + DELETE(s) for one
saveHostMUST run inside onesqflitetransaction, and the in-memory per-host last-persisted id-set is updated ONLY after that transaction commits (in thethen/post-await success path) — never optimistically before the write. A crash or interruption mid-diff must leave memory and disk consistent: either the whole diff applied (and the id-set advances) or none of it did (and the id-set is unchanged, so the next save re-derives the same diff and retries). This mirrors the existingupdateNoteinterruption warning (database_service.dart:281–306: "an interruption mid-way would permanently lose strokes, so the whole sequence must run inside one transaction").
- MF3 — write contract (load-bearing for the per-row choice):
persistence/save_scheduler.dart[ADD] — debounced ~800ms; serialize JSON synchronously before any await; flush on page-leave/dispose.ui/editor_screen.dart,ui/editor_toolbar.dart[ADD] — portpen_editor_screenUI (floating Material You palette + page pill already good).
Public interfaces (key):
abstract class CoordinateSpaceHost {
String get hostId; int get pageIndex;
Offset toContent(Offset deviceLocal, Size deviceSize);
Offset toDevice(Offset content, Size deviceSize);
void applyContentToCanvas(Canvas c, Size deviceSize);
}
class EditorController extends ChangeNotifier { // ui-facing, Riverpod-provided
void beginStroke(CoordinateSpaceHost h, EditorPoint p);
void extendStroke(EditorPoint p);
void commitStroke(); // getStroke once → append → revision++ → undo push → scheduleSave
void eraseAt(CoordinateSpaceHost h, Offset content, double radius);
void undo(); void redo();
void setTool(EditorTool t); void setColor(Color c); void setLayout(PageLayout l);
}
Acceptance: draw/erase/undo/redo on a page; close+reopen → strokes persisted (DB); StaticInkPainter.shouldRepaint==false while drawing a new stroke on a 2,000-stroke page (revision constant); no per-frame point allocation for committed strokes.
F2 — Page layout modes (P1)
layout/page_layout.dart[ADD] —enum PageLayout { continuousSingle, continuousDouble, pagedSingle, pagedDouble }.layout/page_viewport.dart[ADD] — windowed lazy hosting: only pages in[firstVisible - cacheExtent, lastVisible + cacheExtent]mount anAnnotationLayer+PageTile; others are disposed (Picture evicted). Continuous = scrollable column/two-column; paged =PageView. Recenter on layout switch.pdf/pdf_document_source.dart[ADD] —PdfDocumentwrapper exposingpageSize(i),renderTile(...), page count; owns dispose. Replaces ad-hocPdfDocument.openFileinpen_editor_screen.pdf/page_tile.dart[ADD] — multi-resolution tile (R11 mitigation). Under one sharedInteractiveViewer, aPdfPageViewrenders a bitmap sized to its layout constraints × devicePixelRatio and the matrix scales that 1× bitmap, so pinch-zoom GPU-upscales → blurry text/rules at high zoom (unlike pdfrx's own viewer, which re-renders crisp tiles per zoom level).page_tilewatches the transform scale and on zoom-settle (debounced) re-instantiates its render at a DPI matching the current scale — via either a re-laid-outPdfPageViewat the new pixel size orPdfPage.render(width/height at target DPI)intopage_tile_cache. Cap retained DPI (e.g. ≤ 3× base) to bound memory; downscale path stays matrix-only (sharp enough). This is new own-canvas work, not relocation.pdf/page_tile_cache.dart[ADD] —LRU<TileKey{hostId, dpiBucket}, ui.Image>of rendered page bitmaps (NOT ink). Key includes the DPI bucket so a page re-rendered at higher DPI replaces (not duplicates) its lower-DPI tile. Owns the native-handle dispose lifecycle of eachPdfPage.render()result (PdfImage→ backingui.Image), deferred to post-frame so the raster thread is done with an evicted tile before disposal. Distinct fromrender/ink_picture_cache.dart(§2/F1) which holds resolution-independent inkui.Pictures with no DPI bucket.ui/editor_toolbar.dart[MODIFY] — layout switcher control.
Two separate caches + memory budgets (Critic — do not conflate):
ink_picture_cache(vector ink):LRU<revision, ui.Picture>, ~8–12 mounted-page Pictures, dispose deferred to post-frame. Budget ~ a few MB (vector op-lists are cheap). No DPI bucket.page_tile_cache(raster page bitmaps):LRU<TileKey, ui.Image>. This is the heavy one. A single A4 page rendered at 3× DPI is ~10–30 MB (≈ 1785×2526 px × 4 bytes ≈ 18 MB at 3× of a 595×842 pt page @ ~2 dppt). So 8–12 tiles at 3× would be ~150–350 MB — the old "≤64 MB / 8–12 pages" figure (R10) was unit-confused (it conflated ink Pictures with page bitmaps). Resolution: size the tile window to the device memory budget, not a fixed page count — e.g. keep full-DPI tiles only for the visible + ±1 pages (≈ 4 in double-page), and downgrade off-window pages to a 1× thumbnail tier (matrix-upscaled, accepted as blurry only while scrolling). The ink cache keeps its wider ~8–12 window (cheap). R10's ≤64 MB now applies to ink + downgraded-tier tiles; the small high-DPI tile set is budgeted separately (~64–128 MB depending on device), tuned in P0.5/P1.
Data structures: PageWindow { int first, last }; Map<int, HostState> mountedHosts; TileKey { hostId, dpiBucket }. Acceptance: all four modes render; switching recenters; double-page shows two pages side-by-side; ink tracks scroll/zoom; page text/rules stay crisp at 4× zoom (no GPU-upscale blur) — DPI refreshes on zoom-settle (R11); combined cache memory stays within the device budget under a full scroll; perf gate (§7) holds.
F3 — Book-like reading (P1)
layout/reader_controller.dart[ADD] —ReaderMode { read, annotate }; inreadthe arbiter never draws (pen ignored); page-flip animation forpagedSingle/DoubleviaPageViewphysics; spread layout frompagedDouble.ui/editor_screen.dart[MODIFY] — mode toggle in toolbar; quick-jump via slider/grid (no keyboard).
Acceptance: read mode blocks ink; annotate mode draws; page-flip animates; two-page spread aligns facing pages; quick-jump scrolls/animates to target.
F4 — Thumbnail-grid navigation (P1)
ui/thumbnail_grid.dart[ADD] — Drawboard-style grid; tiles render viapdf/pdf_document_source.renderTile(replaceslib/services/thumbnail_service.dartsyncfusion path). Tap → jump; drag-reorder for notebook page order (F6). Slider remains the linear scrubber (already inpen_editor_screen._buildPagePill).lib/services/thumbnail_service.dart[MODIFY/RETIRE] — migrate to pdfrx render; dropsyncfusion_pdfviewer_platform_interfacedep once ported.
Acceptance: grid shows all pages; tap jumps; drag reorders (F6); no text-field page input.
F5 — Configurable pen (P2)
input/pen_config.dart[ADD] —PenConfig { ButtonAction sideButton; ButtonAction eraserEnd; PressureCurveType curve; double palmSensitivity; bool fingerDrawing; };enum ButtonAction { eraser, undo, toggleTool, pan, lasso, none }.InputArbiterconsults it (replaces hardcoded_isEraserSignal).lib/providers/settings_provider.dart[MODIFY] — persistPenConfig(extend existingSharedPreferencesnotifier; it already stores pressure curve + stabilization).lib/screens/settings_screen.dart[MODIFY] / [ADD]ui/pen_settings_page.dart— mapping UI (SpeedyNote-style dialog).- Reuse
lib/models/pressure_curve.dart(already has linear/soft/hard/custom +apply).
Acceptance: remapping side-button to undo makes the barrel button undo; pressure curve changes stroke taper; palm sensitivity changes touch-cooldown; finger-drawing toggle works; all persist across restart.
F6 — One-notebook-per-PDF (P2)
pdf/pdf_document_source.dart[MODIFY] — a page-map: logical notebook pages → either a source-PDF page index or a synthetic blank page. Insert-blank adds a synthetic page without rasterizing or editing the source bytes (keep source vector + searchable). Ink/text bind to the logical page id (stable UUID per logical page), not raw PDF index, so inserts don't reshuffle annotations.lib/services/pdf_service.dart[MODIFY] — keep headless syncfusion export/mutate; export walks the page-map. Fix the hairline bug:_renderStrokesmust build aPdfPathfrombuildStrokeOutlinepoints and fill it (currently strokes line-segments → hairline). Shape/line/arrow keep stroke semantics.- DB [MODIFY] —
notebook_pages(id, document_id, ordinal, source_page_index INTEGER NULL, kind); ink keyed bynotebook_page_id(§4). Portable bundle = zip {source.pdf, badnote.json(strokes/text/links/page-map)} + relink-on-open (match by content hash, fall back to picker).
Acceptance: insert blank page between PDF pages → source PDF untouched (still vector/searchable in another viewer); reorder pages keeps ink attached; export .pdf shows filled ink matching screen (golden); bundle round-trips on another machine.
F7 — Infinite board + 双链 (P3)
board/board_host_pane.dart[ADD] —BoardHost(absolute px) in a constrained-falseInteractiveViewer; reusesAnnotationLayer+ engine. Migrateslib/screens/split_view_screen.dart+scratchpadstable.board/link_graph.dart[ADD] —Link { srcRef, dstRef }where a ref is(kind: notebookPage|board|note, id);backlinksOf(ref). Wiki-style[[...]]parsing in text boxes; sticky-note = a small board region or text box that can link to a page/note. Backlink panel querieslinkstable.- DB [ADD] —
links(id, src_kind, src_id, dst_kind, dst_id, created_at)+ indexes both directions;boards(id, document_id NULL, strokes_json, ...)(generalizescratchpads).
Acceptance: create a sticky linking page 3 → note X; open note X shows a backlink to page 3; board draws at perf target; link graph survives restart; deleting a target leaves a dangling-link indicator (no crash).
F8 — Library-wide full-text search (P3)
search/search_indexer.dart[ADD] — index three sources into a unified library index: (a) PDF embedded text (pdfrxPdfPage.loadText/charRectstext API — source-pinned in P0.5 per SF4), (b) typed text boxes (content), (c) handwriting OCR (existingservices/ocrtext-line ONNX over rasterized strokes viaStrokeRasterizer). Reuse existingdocument_ftsFTS5 + a newlibrary_ftsspanning notebooks/boards/notes with(ref_kind, ref_id, page, snippet).search/ocr_ingest.dart[ADD] — batch handwriting OCR per logical page (best-effort; set expectations: text-line only, no math). Background isolate; debounced after ink idle. The OCR backend itself no-ops when the model is unavailable (onnx_recognition_backend.dart header: "verify on-device") — search MUST never block on, nor be gated by, OCR results.lib/screens/search_screen.dart[MODIFY] — unified results with snippets + jump-to-location (open editor at the page + scroll, or board at the region).
Acceptance — split into two exit tiers (SF2):
- COMMITTED (P3 exit blocker): PDF embedded text + typed-text-box search returns results with snippets; tapping a result opens and scrolls to the page/region. This tier alone satisfies the P3 search exit (it has no unverified-recall dependency).
- ADDITIVE (best-effort, NOT a P3 exit blocker): handwriting OCR contributes hits when the text-line model is available and confident; a word handwritten on page 5 may be returned. Expectation set: handwriting recall is limited by the text-line model and may return nothing on cursive/handwriting; math/formula is explicitly P5. P3 ships even if handwriting recall is poor, with UI copy stating the limit.
F9 — Server sync + AI refinement (P4)
sync/sync_client.dart[ADD] — wire the already-built FastAPI endpoints (/api/auth,/api/sync/push|pull,/api/notes,/api/documents,/api/ocr). Last-writer-wins byupdated_at(server already implements this). Sync notes + (later) notebooks/boards.server/badnote_server/[MODIFY] — extendsync_router/models.pyto cover notebooks/boards/links (today only notes). Add an AI-refine endpoint that runs llm_wiki/VLM/LLM over a note/page → returns organized markdown (server-side, heavy deps gated like the OCR worker).lib/providers/[ADD] sync state provider; settings page server URL/token.
Acceptance: push from device A, pull on device B reproduces notes; AI-refine returns organized text for a selected note; offline still fully functional (server optional, per README).
F10 — Ink CAS / formula (P5)
search/ocr_ingest.dart[MODIFY] — add a formula/math recognition backend (new model; the current PP-OCRv4 is text-line only — this is the known hard sub-problem). Behind a feature flag.engine/[ADD]cas/— recognized formula → CAS (compute/solve) behind a toggle; renders result near the ink. Reuses theCoordinateSpaceHostseam for overlay placement.
Acceptance (stretch): a handwritten 2+3= toggled → shows 5; recognized formulas become searchable. Gated, optional, lowest priority.
F11 — Modern UX polish (woven P0–P3)
- Material You already wired (
main.dartDynamicColorBuilder+ harmonized schemes + Inter). Continue: subtoolbars ineditor_toolbar, drag-reorder thumbnails (F4/F6), hover pre-warm (warm the next page tile + Picture on stylus hover to cut pen-down latency — reuse the live_onPointerHoverseam), animated mode/layout transitions.
3. Data model & persistence (clean schema — data may reset)
Bump DB to a fresh version with _onCreate only (keep _onUpgrade harmless). Today: notes/strokes, documents/annotations(JSON-per-page), bookmarks, ocr_results, document_fts(FTS5), scratchpads (DB v5, lib/services/database_service.dart).
Target tables (additions/changes in bold):
documents[KEEP] (drop editor reliance onrotation; page-map owns rotation).notebook_pages(id PK, document_id FK, ordinal INTEGER, source_page_index INTEGER NULL, kind TEXT, created_at)[ADD] — F6 logical pages.ink(id PK, host_kind TEXT, host_id TEXT, stroke_json TEXT, ordinal INTEGER, updated_at)[ADD] — strokes addressed by host (host_kind ∈ {page, board},host_id=notebook_page_idorboard_id). Per-stroke rows (not per-page blob) so a 2,000-stroke page doesn't rewrite on every save — but ONLY valid under the MF3 diff-write contract (§2/F1editor_repository.saveHost): UPSERT changed rows + DELETE removed rows, never delete-all+re-insert. Without diffing, per-row is worse than a blob; the contract is what makes this schema correct. Index(host_kind, host_id).text_boxes(id PK, host_kind, host_id, content TEXT, rect_json, font_size, color, updated_at)[ADD] — F2-area text + F8 indexing.boards(id PK, document_id FK NULL, title, created_at, updated_at)[ADD] — generalizescratchpads; board strokes live ininkwithhost_kind='board'.links(id PK, src_kind, src_id, dst_kind, dst_id, created_at)[ADD] — 双链; indexes on(src_kind,src_id)and(dst_kind,dst_id).bookmarks[KEEP].ocr_results[KEEP/EXTEND] — per logical page handwriting OCR text.library_fts(FTS5) [ADD] —(ref_kind, ref_id, page, content)unified search over PDF text + typed text + OCR.document_fts[KEEP] for PDF-page text.notes/strokes/notes_fts[KEEP] (existing ink-note path; eventually folded into boards, but not deleted in P0).sync_state(entity_kind, entity_id, last_pushed_at, last_pulled_at, dirty)[ADD, P4].
Coordinate semantics on disk: page ink = normalized [0,1] unrotated; board ink = absolute logical px. (Matches live conventions.)
Stroke-model convergence (important): there are currently two stroke models — PenStroke/PenPoint (live canvas, in-memory only) and InkStroke/InkPoint (freezed/JSON, DB+OCR+export). P0 introduces one canonical EditorStroke/EditorPoint (freezed/JSON, normalized) and adapters to/from InkStroke for OCR/export reuse during transition; old screens keep InkStroke until retired (§6).
4. Refactor strategy (evolve, don't rewrite)
Principle: every phase ships; the engine generalizes under load.
- P0 = relocation + persistence. Move the 4 live canvas files into
engine/+render/+input/with minimal logic change; addStrokeStore.revision+ui.Picturecache + DB persistence viaeditor_repository. The live single-page editor keeps working throughout. KeepPenStrokeas a thin alias ofEditorStrokeuntil callers migrate. - P0.5 = vertical slice (continuous-single only) — the new perf/crispness gate (see §5/§9). Stand up
layout/page_viewport+pdf/page_tilerendering continuous-SINGLE only, with the rewritten perf bench (targetingui/editor_screen, not the spike) and the zoom-DPI refresh proven crisp at 4× on the Surface. This proves the own-canvas multi-page + multi-resolution model on the real device before any double/paged/spread work. - P1 = full multi-page windowing. Wrap the (now-relocated)
AnnotationLayerinpage_viewport;_strokesByPageMap → repository-backed mounted hosts. Single-page path stays aspagedSingle. Continuous-double / paged / spread land here, AFTER P0.5 passes. - Old-code retirement (gated on parity, then delete in one step):
- [DELETE after P1 parity]
lib/screens/pdf_annotator_screen.dart,lib/widgets/pdf_annotation_layer.dart,lib/widgets/ink_canvas.dart(extract any unique draw/erase logic toengine/first),lib/editor/pdf/spike_*.dart(throwaway M1 spike — includes the spike-based perf bench, replaced in P0.5),lib/editor/pdf/pen_capture_region.dart+PenCaptureBindinginmain.dart(own-canvasListenermodel won — remove the unused arena-bypass binding once P0 device-confirms). - [DELETE in P3, NOT P1 (SF3)]
lib/screens/split_view_screen.dart— its replacement is the P3 infinite board (F7); deleting it in P1 would leave a 2-phase functionality gap (no scratchpad between P1 and P3). It stays live until the board lands. - [KEEP]
thumbnail_serviceuntil F4 ports it;stroke_rasterizer(OCR),ctc_decoder,onnx_recognition_backend, OCR assets;pdf_service(export/mutate, with the fill fix);pptx_service/ppt_annotator_screen(PPT is separate; not in scope but not deleted).
- [DELETE after P1 parity]
- Navigation swap:
home_screen_openDocument/_importPdfcurrently pushPdfAnnotatorScreen;openM1SpikepushesPenEditorScreen. Repoint both toui/editor_screen.dartonce P1 parity passes (checklist below). - Parity checklist before any delete: page rotate/delete/insert-blank/insert-image (
pdf_service), bookmark add/toggle/jump, undo/redo across pages, save-on-leave, zoom in/out/fit, export-matches-screen (golden). Mirrors phase-1 §10 M3 checklist C1–C11.
5. Phased delivery (each = shippable milestone with exit criteria)
Sequence de-risks: pen core + persistence first (P0), then the 60fps multi-mode layout (P1), then config/notebook (P2), then board/双链/search (P3), then sync/AI (P4), then CAS (P5).
P0 — Pen core hardened + persisted (own-canvas). Exit: draw/erase/undo/redo on a single PDF page persist to DB and reload; EditorStroke canonical model (superset of InkPoint, SF1) + revision-gated ui.Picture cache; editor_repository.saveHost honors the MF3 diff-write contract (test: erase 1 of 2,000 ⇒ 1 DELETE, 0 re-inserts); pure InputArbiter + stroke_eraser unit-tested; on-device Surface Pen confirms pressure + palm rejection + pinch-zoom in the live shared-transform model (the one device gate). Export hairline bug fixed (fill). No regression to existing screens.
P0.5 — Vertical slice gate (own-canvas multi-page + multi-resolution), continuous-SINGLE only. [NEW — SYNTHESIS] Stand up layout/page_viewport + pdf/page_tile rendering continuous-single only on the real document path (ui/editor_screen), and:
- (a) REWRITE the perf bench (§7/§8): a new
integration_test/editor_scroll_bench.dartdrivesui/editor_screen+layout/page_viewport; DELETE the spike-basedintegration_test/perf_scroll_bench.dart(it importsspike_editor_pane.dart/pageOverlaysBuilder= the invalidated Option B; it CANNOT validate own-canvas). - (b) Prove crisp-on-zoom (R11): page-tile DPI refresh on zoom-settle renders crisp text/rules at 4× on the Surface (no GPU-upscale blur).
- (c) Source-pin the pdfrx render/text APIs (SF4): confirm
PdfPage.loadText/charRects/PdfPage.render()/PdfPageViewsignatures present in pdfrx 2.4.4; smoke-test each, so F4/F8 don't hit drift late. - Exit (hard gate before any double/paged/spread): continuous-single median build+raster ≤ 16.6ms, p95 ≤ 22ms on the 300-page asset via the REWRITTEN bench, AND a manual crisp-on-zoom-at-4× PASS on the Surface, AND the four pdfrx APIs source-pinned + smoke-tested. Continuous-double / paged / spread do NOT begin until P0.5 is GREEN.
P1 — Full layout modes + book-like reading + thumbnails (60fps). Precondition: P0.5 GREEN. Exit: the remaining PageLayouts (continuous-double, pagedSingle, pagedDouble); reader vs annotate; page-flip + two-page spread; thumbnail-grid jump + drag-reorder; windowed lazy hosting; perf gates met (§7) incl. continuous-double on a 300-page PDF via the rewritten bench; old PDF screens deleted after parity checklist (split_view retirement deferred to P3 per SF3); nav repointed.
P2 — Configurable pen + one-notebook-per-PDF. Exit: Pen settings page (button/eraser mapping, curve, palm sensitivity, finger toggle) persists + drives arbiter; insert-blank/reorder pages keep source vector + ink attached to logical pages; portable bundle round-trips; export walks page-map.
P3 — Infinite board + 双链 + library search. Exit: board reuses engine at perf; sticky-notes + [[links]] produce backlinks; library search COMMITTED tier (PDF embedded text + typed-text-box) returns snippets + jump-to-location (this tier is the exit blocker, SF2); handwriting-OCR tier is ADDITIVE/best-effort and does NOT block P3 exit; split_view_screen retired into board (the SF3 deletion point).
P4 — Server sync + AI refinement. Exit: push/pull notes+notebooks across two devices (LWW); AI-refine endpoint returns organized markdown; app fully functional offline.
P5 — Formula OCR + ink CAS. Exit (stretch, gated): formula recognition backend; searchable formulas; CAS solve toggle for simple expressions.
Every milestone ends with verifier/critic pass + perf-results doc updated with commit hash.
6. Refactor of input transport (record the pivot)
The live PenCanvas proves the own-canvas model: a single Listener over an InteractiveViewer whose child is Stack(PdfPageView bitmap, StaticInk, LiveInk). Because the pen, page bitmap, and ink share one transform and the Listener arbitrates by _activePointers.length + kind, there is no gesture-arena fight — pdfrx never sees gestures (it only renders). This removes the phase-1 need for PenCaptureRegion/PenCaptureBinding (RenderProxyBox arena bypass), which exist for the abandoned pageOverlaysBuilder approach. Retirement gate: delete them once P0 confirms on the physical Surface Pen that the Listener model handles stylus draw + single-finger pan + pinch-zoom + palm rejection (the live code is built for exactly this; confirm, then remove the dead binding).
7. Risks & mitigations
| # | Risk | L | I | Mitigation / trigger |
|---|---|---|---|---|
| R1 | 60fps continuous + double-page on a 300-page PDF. Two columns × windowed tiles × ink Pictures may blow frame budget. | Med | High | Windowed lazy hosting (only visible+cacheExtent mounted), bounded LRU + post-frame dispose for BOTH the ink_picture_cache and the page_tile_cache (R10), per-page RepaintBoundary, revision-gated static Picture. Gate continuous-SINGLE in P0.5, continuous-double in P1 (§5/§9). MF1 — the existing integration_test/perf_scroll_bench.dart is UNUSABLE here: it imports spike_editor_pane.dart and benchmarks pageOverlaysBuilder (Option B, the INVALIDATED architecture) — it cannot validate own-canvas. P0.5 REWRITES the bench against ui/editor_screen + layout/page_viewport and DELETES the spike-based one. Uses large_300p.pdf. |
| R2 | Pen feel only verifiable on the user's Surface (CI has no pen). | High | Med | P0 device gate (pressure/palm/pinch) + manual checklist in perf-results doc; synthesized-stylus widget tests as interim signal only. |
| R3 | Handwriting/formula OCR feasibility. Current model is text-line PP-OCRv4; math is unsolved. | High | Med | F8 ships text-line best-effort with explicit expectation-setting; formula isolated to P5 behind a flag; never block search on OCR quality. |
| R4 | 双链 graph scale (thousands of links/sticky-notes). | Low | Med | Indexed links table (both directions), lazy backlink queries, no in-memory full graph. |
| R5 | Server/AI scope creep. | Med | Med | Server stays optional (README); P4 wires existing endpoints + one AI-refine route; AI heavy deps gated like OCR worker. |
| R6 | Windows pen edge cases (no advertised pressure range; barrel-button eraser; inverted stylus). | Med | Med | Live _normalizedPressure already handles degenerate ranges; F5 makes button/eraser mappable; test matrix in arbiter unit tests. |
| R7 | Export fidelity (hairline bug today). | High (today) | Med | stroke_geometry.buildStrokeOutline shared screen+export; pdf_service._renderStrokes fills a PdfPath; golden test (P0). |
| R8 | Stroke-model convergence churn (two models today). | Med | Low | One canonical EditorStroke + adapters; old InkStroke retained only where old screens/OCR/export still use it, deleted with them. |
| R9 | pdfrx page-render / text API drift (thumbnails, search text, double-page sizing). | Med | Low | Source-pin PdfDocument/PdfPageView/PdfPage.loadText/charRects/render() signatures + smoke-test in P0.5 (pulled forward, SF4) so F4/F8 don't hit drift late; keep syncfusion export until verified. |
| R10 | Cache memory budget vs many mounted pages (TWO caches — Critic). | Med | Med | ink_picture_cache (vector ui.Picture, no DPI bucket): ~8–12 mounted pages, a few MB. page_tile_cache (raster ui.Image, DPI-bucketed): the heavy store — a 3×-DPI A4 tile is ~10–30 MB, so 8–12 high-DPI tiles would be ~150–350 MB. The old "≤64MB / 8–12 pages" figure was unit-confused (conflated ink with page bitmaps). Fix (§2/F2): keep full-DPI tiles only for visible ±1 pages, downgrade off-window pages to a 1× thumbnail tier; ≤64MB applies to ink + downgraded tiles, the small high-DPI tile set budgeted separately (~64–128MB, device-tuned). Sample BOTH caches during the scroll bench. |
| R11 | Blurry page at high zoom under the shared transform (MF2). One InteractiveViewer matrix-scales a PdfPageView bitmap that was rendered at layout-constraint × devicePixelRatio (1×); pinch-zoom GPU-upscales it → blurry text/rules at 4×, whereas pdfrx's own viewer re-renders crisp tiles per zoom level. We own multi-resolution tiling, not just layout/windowing. |
Med | High | pdf/page_tile.dart drives render DPI from the transform scale: on zoom-settle re-instantiate the tile (re-laid-out PdfPageView at the new pixel size, or PdfPage.render() at target DPI) into the pdf/page_tile_cache.dart store (LRU<TileKey{hostId, dpiBucket}, ui.Image> — NOT the ink cache); cache key includes a DPI bucket; cap retained DPI (~3× base) to bound memory; downscale stays matrix-only. PdfPage.render() returns an async PdfImage owning a native handle → page_tile_cache owns its post-frame dispose (see Open Questions; confirm in the SF4 P0.5 source-pin). Proven crisp at 4× on the Surface in the P0.5 gate (§5). Budgeted P0.5/P1, not relocation. |
Pre-mortem (DELIBERATE — 3 scenarios)
- "Continuous double-page janks / pages are blurry at 4× on the user's big scanned PDF." Cause: built layout modes before profiling two-column windowing, and matrix-scaled a 1× bitmap (R11). Prevention: the P0.5 vertical-slice gate profiles continuous-SINGLE first with a rewritten bench against
ui/editor_screen(the spike-basedperf_scroll_bench.dartis deleted — it benchmarks the invalidated Option B) AND proves crisp-on-zoom DPI refresh at 4× on the Surface; continuous-double/paged/spread merge only after P0.5 is GREEN, and the P1 double-page gate reuses the rewritten bench. - "Search returns nothing for handwriting." Cause: over-promised OCR. Prevention: F8 ships PDF-text + typed-text search first (reliable), handwriting OCR as additive best-effort with UI copy stating limits; formula explicitly P5.
- "Insert-blank-page silently rasterized / detached annotations." Cause: editing source bytes or keying ink to raw PDF index. Prevention: logical page-map + ink keyed to
notebook_page_id; golden test that the source PDF bytes are unchanged after insert and remains selectable-text in an external viewer.
8. Testing strategy
sqlite workaround: DB-touching tests run via
tool/test.sh(system sqlite +LD_LIBRARY_PATH); pure-logic tests avoid the DB. Reusable assets:tool/gen_bench_pdf.dart,tool/gen_dense_strokes.dart,test/assets/large_300p.pdf,test/assets/dense_strokes.json,integration_test/coordinate_assertion_test.dart. NOT reusable:integration_test/perf_scroll_bench.dart— it importsspike_editor_pane.dart(pageOverlaysBuilder, invalidated Option B) and is DELETED + replaced byintegration_test/editor_scroll_bench.dart(targetsui/editor_screen) in P0.5 (MF1).
- Unit (no DB/widgets):
- Coordinate transforms:
NormalizedPageHost/BoardHostround-triptoContent(toDevice(x))≈x. - Stroke geometry:
buildStrokeOutlinenon-empty for ≥1 point; live ⊆ committed bounds (no "pop"). - Eraser:
eraseHitswhole-stroke removal (extraction of livepen_canvas._eraseAt— regression port).splitStrokesegment-erase is NEW behavior, not a port (the live canvas only does whole-stroke erase;undo_managerrecords but never computes replacements): mid-erase ⇒ 2 segments, endpoint ⇒ 1 segment, full-erase ⇒ empty, <2-pt result dropped — these exercise greenfield code. (undo_manager_test.dartinforms the replacement-bookkeeping discipline only.) - Revision gating:
StrokeStore.addbumps revision;StaticInkPainter.shouldRepaintiff revision changed. InputArbiterSM: table-driven over the device×mode matrix incl. palm rejection (touch dropped while stylus active) andpointerCount>=2→cancel.- Undo: global commit-time order, host-tagged reversal.
SaveScheduler: mutating "current host" after schedule but before write completes does not change persisted snapshot.editor_repository.saveHostdiff-write (MF3, fake DB counting statements): erase 1 of 2,000 strokes ⇒ exactly 1 DELETE, 0 INSERT; add 1 stroke ⇒ 1 INSERT, 0 DELETE; no-op save ⇒ 0 statements. (Guards against the live notes-path delete-all+re-insert anti-pattern at database_service.dart:281–306.)EditorStroke↔InkStrokeadapter round-trip (SF1):tilt/timestamp/pointerDeviceKindsurvive both directions (no lossy OCR/export conversion).link_graph.backlinksOf(P3);search_indexersnippet/jump-ref mapping for the COMMITTED tier — PDF-text + typed-text (P3).
- Coordinate transforms:
- Widget: stylus→stroke committed, touch→not consumed (synthesized pointers); layout-mode switch recenters; reader-mode blocks ink; thumbnail tap jumps; text-box place/move/edit persists (fake DB).
- Perf: the rewritten
integration_test/editor_scroll_bench.dart(targetsui/editor_screen+layout/page_viewport; the spike-basedperf_scroll_bench.dartis deleted, MF1) onlarge_300p.pdf— continuous-single in P0.5 (gate), continuous-double in P1, profile mode, N≥120 frames warm, median build+raster ≤16.6ms / p95 ≤22ms;StaticInkPainterno-rebuild assertion; Picture-memory ≤64MB sample. Archived indocs/plans/full-refactor-perf-results.mdwith commit hash. Not a hard CI gate (no GPU) but required for milestone sign-off. - Manual on-device pen checklist (Surface Pen, P0 + each milestone): pressure varies width; barrel/inverted = erase; palm rest doesn't mark; single-finger scroll; two-finger pinch; page text/rules stay crisp at 4× zoom — no GPU-upscale blur, DPI refreshes on zoom-settle (R11, P0.5 gate); hover pre-warm reduces first-stroke latency; page-flip feels book-like. Recorded with device + commit.
- Regression: keep
ctc_decoder_test.dart,undo_manager_test.dartgreen; updatewidget_test.dartto booteditor_screen. - Export golden (P0): annotate a known page → export → image-compare filled ink matches screen (R7).
- Server (P4): existing
server/tests/(test_sync.pyetc.) green; add notebook/board sync tests.
9. Milestones / sequencing — immediate next chunk (concrete)
Next chunk = P0 (engine relocation + persistence), executable now; followed by the P0.5 vertical-slice gate (steps 10–13) before any double/paged/spread layout work:
- [ADD]
lib/editor/engine/stroke_model.dart—EditorStroke/EditorPoint(freezed + JSON), normalized;fromPenStroke/toInkStrokeadapters. Runbuild_runner. - [ADD]
lib/editor/engine/stroke_geometry.dart— liftbuildStrokePath→buildStrokeOutline(verbatim recipe fromink_painters.dart). - [ADD]
lib/editor/engine/stroke_store.dart+lib/editor/render/{static_ink_painter,live_ink_painter,ink_picture_cache,annotation_layer}.dart— relocate live painters; add the resolution-independent inkui.Picturecache keyed byrevision(NO DPI bucket; the DPI-bucketedpage_tile_cacheis a separate P0.5 file, step 10). - [ADD]
lib/editor/input/input_arbiter.dart+lib/editor/engine/stroke_eraser.dart— extract frompen_canvas.dart(pure, unit-tested). - [ADD]
lib/editor/persistence/{editor_repository,save_scheduler}.dart+ DB additions (ink,notebook_pagesminimal) indatabase_service.dart(fresh version).saveHostimplements the MF3 diff-write contract (UPSERT changed + DELETE removed, by stroke id; NO delete-all+re-insert) with a per-host last-persisted id-set. - [MODIFY]
lib/editor/canvas/pen_editor_screen.dart(or newui/editor_screen.dart) to load/commit/save through the repository instead of the in-memory_strokesByPageMap. - [MODIFY]
lib/services/pdf_service.dart_renderStrokes→ fillbuildStrokeOutlinepath (R7) + export golden test. - Tests: arbiter SM, eraser, geometry, revision-gating, save-scheduler snapshot, diff-write statement-count (MF3: erase 1/2000 ⇒ 1 DELETE 0 INSERT),
EditorStroke↔InkStrokelossless round-trip (SF1), export golden. Run viatool/test.sh. - Device gate: build Windows package (CI), confirm pen/palm/pinch on Surface Pen; record in perf-results doc. → unblocks P0.5.
Then P0.5 (vertical-slice gate — must pass before P1's double/paged/spread):
10. [ADD] lib/editor/layout/page_viewport.dart (continuous-single only) + lib/editor/pdf/{pdf_document_source,page_tile,page_tile_cache}.dart with zoom-settle DPI refresh (R11) + the DPI-bucketed page_tile_cache (LRU<TileKey, ui.Image>, page bitmaps; owns native-handle dispose). (The resolution-independent render/ink_picture_cache.dart for vector ink lands in P0, step 3 — it is a separate cache, no DPI bucket.)
11. [ADD] integration_test/editor_scroll_bench.dart targeting ui/editor_screen; [DELETE] integration_test/perf_scroll_bench.dart + lib/editor/pdf/spike_*.dart (the spike pane the old bench imports). Run the rewritten bench on a scanned-image 300-page asset (see Open Questions — large_300p.pdf is synthetic/vector and may not honestly stress raster re-render at 3× DPI) → continuous-single median ≤16.6ms / p95 ≤22ms; sample both caches' memory.
12. [VERIFY] source-pin pdfrx PdfPage.loadText/charRects/render()/PdfPageView in 2.4.4 (SF4) + smoke test; confirm PdfPage.render()'s PdfImage/native-handle ownership + dispose semantics so page_tile_cache can manage post-frame disposal; record signatures in perf-results doc.
13. Device gate: crisp-on-zoom at 4× PASS on the Surface (R11). → unblocks P1.
Each subsequent milestone (P1…P5) follows §5 exit criteria; verifier/critic + perf-results update per milestone.
10. RALPLAN-DR
Principles (3–5)
- Single source of truth = host content coordinates. Screen mapping is a paint-time
canvastransform; never store/duplicate transformed geometry. (Already the live convention — generalize it.) - One host-agnostic ink engine. PDF page, infinite board, and (P5) CAS overlay are
CoordinateSpaceHosts behind one renderer — never fork the stroke pipeline. - Own the gesture pipeline; pdfrx only renders. A single
Listenerover a sharedInteractiveViewerarbitrates draw/pan/zoom/palm by pointer kind + count — no gesture-arena fights (proven live). - De-risk performance AND crispness before features. A P0.5 vertical-slice gate (continuous-single, rewritten bench, crisp-at-4× on the Surface) precedes all double/paged/spread work; continuous-double is gated again in P1.
- Ship every phase; generalize under load. Relocation-first refactor keeps the editor working at all times; old screens deleted only after parity.
Decision Drivers (top 3)
- D1 — Pen feel + palm rejection + pinch on Windows Surface Pen is the make-or-break primary-device requirement (own-canvas model already targets it).
- D2 — 60fps across all layout modes on big PDFs with thousands of strokes.
- D3 — One engine reused across PDF page, infinite board, and CAS, forward-compatible with search/双链/sync.
Viable options (≥2) with bounded pros/cons
Option A — Own-canvas engine (single Listener + InteractiveViewer, pdfrx as renderer). CHOSEN (already live).
- Pros: D1 solved structurally (one transform, no arena fight — already working in
pen_canvas.dart); D3 trivial (hosts share the transform); minimal new deps; matches the proven Saber model the user cited. - Cons: we own page layout/windowing/tiling AND multi-resolution re-rasterization (more code than a stock viewer); double-page perf is on us (R1) and crisp-on-zoom is on us (R11 — pdfrx's own viewer re-renders crisp tiles per zoom for free; we must re-derive tile DPI from the transform); thumbnails/text-extraction still need pdfrx page APIs (R9).
Option B — pdfrx pageOverlaysBuilder-hosted ink + RenderProxyBox arena-bypass (the phase-1 plan).
- Pros: pdfrx gives continuous scroll/tiling/text-extraction for free; ink-follows-page is structural via page overlays.
- Cons: requires a custom
PenCaptureBindingarena bypass that fights pdfrx's greedy scale recognizer (the live code already abandoned this —PenCaptureRegionis unused); pdfrx owns the transform so double-page/board reuse is awkward; D1 proven harder than Option A in practice. Invalidated — see below.
Option C — Flutter shell + Rust hot-path (rnote-style) for ink/render.
- Pros: maximal ink perf headroom.
- Cons: rejected in memory (
badnote-flutter-344-windows-pen) — Windows pen is weaker in the Rust/GTK stack; huge FFI surface; contradicts "stay Flutter." Invalidated.
Invalidation rationale
- B invalidated (as the INPUT model): the live codebase already moved off it;
pageOverlaysBuilder+arena-bypass made stylus/touch arbitration fight pdfrx's recognizer, whereas Option A's single-Listener-over-shared-transform sidesteps the arena entirely and is already drawing with pressure/pinch/palm. Correction (MF2): B had TWO edges, not one — (1) free continuous scroll/windowing, AND (2) free per-zoom crisp re-rasterization (pdfrx re-renders tiles at each zoom level). Edge (1) is recoverable in A via windowed hosting (needed for double-page anyway); edge (2) is NOT free in A — Option A must own multi-resolution tiling (R11), which is the genuinely new cost of this choice. We accept that cost (gated in P0.5) because A's structural D1/D3 wins outweigh it. B retained only as a fallback page-render strategy, not the input model. - C invalidated: documented Windows-pen regression in the Rust/GTK path + "stay Flutter" hard constraint; the 3.44 WM_POINTER fix already unblocked Flutter pen, removing C's motivation.
- Net: A chosen; B retained as a partial fallback (page rendering only); C rejected.
ADR
- Status: Architect APPROVE-WITH-MUST-FIX applied (2026-06-21): MF1 (rewrite perf bench off the invalidated spike), MF2 (R11 zoom re-rasterization + multi-resolution tiling), MF3 (diff-write contract); P0.5 vertical-slice synthesis gate; SF1–SF5. Pending Critic. Records the live pivot from the superseded phase-1 input architecture.
- Decision: Build the full BadNote vision on an own-canvas, host-agnostic ink engine (single
Listener+ sharedInteractiveViewer; pdfrx as page renderer/text source). Generalize the livelib/editor/canvas/intoengine/render/input/layout/...; persist a single canonicalEditorStroke; phase features P0→P5 with a P0.5 vertical-slice gate. - Drivers: D1 Surface-Pen feel, D2 60fps multi-mode, D3 one reusable engine.
- Alternatives considered: B (pdfrx-overlay + arena bypass — invalidated as input model, kept as render fallback; had two free edges — scroll AND per-zoom crispness — the latter is the new cost we take on), C (Flutter+Rust — rejected).
- Why chosen: A is already proven live for pen/palm/pinch and gives D1+D3 structurally; the remaining risks (D2 multi-mode perf AND R11 crisp-on-zoom) are gated up front in P0.5 then re-gated for double-page in P1.
- Consequences: we own layout/windowing/tiling AND multi-resolution page-tile re-rasterization (R11 — not just layout, MF2);
PenCaptureRegion/PenCaptureBindingretired (dead under own-canvas); the spike-based perf bench is deleted and rewritten againstui/editor_screen(MF1); per-stroke-row persistence is valid ONLY under the diff-write contract (MF3);EditorPointis a non-lossy superset ofInkPoint(SF1); export must fill (not stroke) ink; handwriting OCR is additive/non-blocking and formula is P5 (SF2);split_view_screendeletion deferred to P3 (SF3); server stays optional. - Follow-ups: P0 device gate confirms pen model; P0.5 gate proves continuous-single perf + crisp-at-4× + source-pinned pdfrx APIs (SF4) before any double/paged/spread; resolve open questions below.
Open Questions (persist to .omc/plans/open-questions.md)
- Continuous-double-page on a real 300-page scanned PDF — does windowed two-column hosting hold 60fps, or do we need tile pre-rasterization? — P0.5 gates continuous-single first; P1 perf gate decides double; affects R1/R10.
- How is page-tile DPI refreshed on zoom under the shared transform (SF5)? — re-laid-out
PdfPageViewat the new pixel size, orPdfPage.render()at target DPI intopage_tile_cache? What scale-change threshold + debounce triggers a refresh, and what is the retained-DPI cap? — Load-bearing for the 60fps + crisp-at-4× goal; resolved in P0.5 (R11). PdfPage.render()native-handle lifecycle (Critic 4a):render()returns an asyncPdfImageowning a native handle backing aui.Image—page_tile_cachemust own its dispose (post-frame, after the raster thread is done) on eviction. Confirm exact ownership + dispose API in the SF4 P0.5 source-pin; affects page_tile_cache + R10/R11.- Honest R11 raster stress asset (Critic 4b): does the synthetic/vector
large_300p.pdfactually force RASTER re-render cost at 3× DPI, or does a scanned-image asset better exercise the crispness + tile-memory gate? — Add a scanned 300-page asset (tool/gen_bench_pdf.dartimage mode or a real scan) for the P0.5/P1 perf+crispness gates; affects §8 / R11 honesty. - Stroke-model convergence: fold
notes/InkStrokeink-note path intoboards/EditorStroke, or keep notes separate long-term? — Affects §3/§4 churn. - Handwriting OCR: is the existing PP-OCRv4 text-line model usable on cursive/handwriting at all, or do we need a handwriting-specific model even for non-formula text? — Affects F8 expectation-setting / R3.
- Portable bundle relink: match source PDF by content-hash only, or also store original path + size? — Affects F6.
- Server AI-refine: run llm_wiki/VLM server-side only, or allow a local LLM path for offline users? — Affects F9 scope / R5.
- Keep
pptx_service/ppt_annotator_screenin the new engine, or freeze PPT support? — Out of the 11 features; decide before P1 nav swap.