diff --git a/docs/plans/2026-06-21-badnote-full-refactor.md b/docs/plans/2026-06-21-badnote-full-refactor.md new file mode 100644 index 0000000..546ccb9 --- /dev/null +++ b/docs/plans/2026-06-21-badnote-full-refactor.md @@ -0,0 +1,446 @@ +# BadNote — Full Refactor + Feature Roadmap (own-canvas engine) + +**Status:** PLAN (ralplan consensus — Architect APPROVE-WITH-MUST-FIX + Critic-ITERATE fixes applied 2026-06-21: MF1–MF3 + P0.5 slice + SF1–SF5 + cache-split / diff-write-durability / erase-is-new-behavior / render-handle-lifecycle; pending final Critic confirm) +**Date:** 2026-06-21 +**Mode:** DELIBERATE (high-risk: engine generalization, 60fps continuous/double-page, OCR/CAS feasibility, server/AI scope) +**Owner plan file:** `docs/plans/2026-06-21-badnote-full-refactor.md` +**Supersedes (in part):** `docs/plans/2026-06-21-badnote-phase1.md` + +> **Grounding.** Written after reading the live code, not from memory. Verified sources: +> live editor `lib/editor/canvas/{pen_canvas,ink_painters,pen_stroke,pen_editor_screen}.dart`, +> `lib/editor/pdf/pen_capture_region.dart`, `lib/main.dart` (dynamic_color + `pdfrxFlutterInitialize` + `PenCaptureBinding`), +> `lib/services/{database_service,pdf_service,stroke_rasterizer,undo_manager}.dart`, +> `lib/services/ocr/onnx_recognition_backend.dart`, `lib/models/{ink_stroke,ink_point,pressure_curve}.dart`, +> `lib/providers/settings_provider.dart`, `lib/screens/home_screen.dart`, +> `server/badnote_server/{main,models,routers/sync_router}.py`, `server/README.md`, +> `pubspec.yaml`, existing `test/` + `integration_test/` + `tool/` assets. +> +> **CRITICAL DIRECTION CHANGE captured here (vs phase1 plan):** the live `lib/editor/canvas/` has **already abandoned** the phase-1 "pdfrx `pageOverlaysBuilder`-hosted ink + RenderProxyBox arena bypass" architecture. The live `PenCanvas` owns **one `InteractiveViewer` + a single `Listener`** that draws a `PdfPageView` *bitmap* and the ink in the **same** child subtree (Saber clean-room model). pdfrx is used only as a **page renderer / document API** (`PdfDocument.openFile`, `PdfPageView`), never for gestures. `PenCaptureRegion`/`PenCaptureBinding` are installed in `main()` but are **NOT wired into the live canvas** and are slated for retirement (§6, §9). This plan builds the full product on the **own-canvas** model and records that decision in the ADR (§10). + +--- + +## 0. Vision & Scope + +**BadNote** is a pen-first, performant note-taking app. Primary device: **Windows Tablet + Surface Pen** (Flutter ≥ 3.44 for WM_POINTER stylus/multitouch); desktop for review/复习/search. Differentiators the user explicitly wants to win on: **(1)** library-wide full-text search over PDF text + typed text + handwriting OCR with jump-to-location; **(2)** modern cohesive **Material You** UI; **(3)** truly **book-like reading** (page-flip, two-page spread, thumbnail grid, reader-vs-annotate modes). Plus a **双链 sticky-note infinite board**, **one-notebook-per-PDF** with insertable blank pages and never-rasterized source, and a later **server sync + AI refinement** and **ink CAS**. + +**Existing user data MAY be reset** — no migration burden; prefer the cleanest schema. **Stack stays Flutter** (own gesture pipeline; pdfrx only renders pages). GFW-aware CI already solved (pdfium pre-fetch + Dev Mode + vendored sqlite3 + onnxruntime pre-fetch). + +### 0.1 The 11 feature areas → phase map (full vision; nothing dropped, everything sequenced) + +| # | Feature | Phase | Risk | +| --- | --- | --- | --- | +| F1 | Pen-first editor core (harden): pressure, palm rejection, perfect_freehand quality, eraser, tools, undo/redo, **DB persistence of strokes per doc/page** | **P0** | Low–Med | +| F2 | Page layout modes: continuous-single/-double, paged-single/-double + switcher; lazy render @60fps | **P1** | **High** | +| F3 | Book-like reading: page-flip, two-page spread, reader vs annotate mode, quick jump | **P1** | Med | +| F4 | Thumbnail-grid navigation (Drawboard-style), slider, no keyboard input | **P1** | Low | +| F5 | Configurable pen: side-button + eraser-end mapping, pressure curve, palm sensitivity, finger-drawing toggle — Pen settings page | **P2** | Med | +| F6 | One-notebook-per-PDF: page-level binding, insert blank pages between PDF pages, keep PDF vector, portable bundle + relink | **P2** | Med | +| F7 | Infinite 便利贴 board + **双链** bidirectional links / backlinks | **P3** | Med–High | +| F8 | Library-wide full-text search (PDF text + typed text + handwriting OCR) with snippets + jump | **P3** | **High (handwriting OCR)** | +| F9 | Server sync + AI refinement (llm_wiki, VLM/LLM organize) | **P4** | High (scope) | +| F10 | Ink CAS / formula recognition → searchable + solve toggle | **P5** | **Very High** | +| F11 | Modern UX polish: Material You, subtoolbars, drag-reorder thumbnails, hover pre-warm | **woven across P0–P3** | Low | + +### 0.2 Non-goals per phase +- **P0:** no layout modes, no board, no text boxes, no search UI, no settings page beyond what exists, no server. Single-page editor + persistence + eraser/undo only. +- **P1:** no 双链, no OCR-search, no server, no CAS. Layout/reading/thumbnails only. +- **P2:** no server, no CAS, no handwriting OCR. Pen-config + notebook/bundle only. +- **P3:** no CAS, no server-side AI. Board+双链 and **local** search (incl. best-effort handwriting OCR via existing text-line model; formula deferred to P5). +- **P4:** sync + AI organize; CAS still out. +- **P5:** formula OCR + CAS. +- **All phases:** never rasterize the source PDF; no on-screen keyboard reliance (slider/grid nav). + +--- + +## 1. Target Architecture + +### 1.1 Layered module decomposition (target `lib/editor/`) + +``` +lib/editor/ +├─ engine/ # host-agnostic ink engine (pure-ish, testable) +│ ├─ coordinate_space_host.dart # CoordinateSpaceHost + NormalizedPageHost + BoardHost +│ ├─ stroke_model.dart # EditorStroke/EditorPoint (canonical; replaces PenStroke split — §3, §4) +│ ├─ stroke_store.dart # StrokeStore { committed, int revision } (O(1) shouldRepaint gate) +│ ├─ stroke_geometry.dart # buildStrokeOutline(getStroke) — single source for screen+export (kills hairline bug) +│ ├─ stroke_eraser.dart # pure splitStroke / eraseHits (extract from pen_canvas._eraseAt) +│ └─ undo_stack.dart # global, commit-time-ordered, host-tagged (generalize undo_manager) +├─ input/ +│ ├─ input_arbiter.dart # pure state machine: pointerCount>=2→pan/zoom, stylus→draw, palm reject +│ └─ pen_config.dart # button/eraser-end action mapping, pressure curve, palm sensitivity (F5) +├─ render/ +│ ├─ annotation_layer.dart # Stack(StaticInkPainter, LiveInkPainter, TextBoxLayer) over a host +│ ├─ static_ink_painter.dart # reads ink_picture_cache, keyed by revision +│ ├─ live_ink_painter.dart # in-progress stroke only +│ └─ ink_picture_cache.dart # LRU for STATIC INK — resolution-INDEPENDENT, NO DPI bucket (vector ink re-rasterizes crisp at composite) +├─ layout/ # F2/F3 — page layout + reading +│ ├─ page_layout.dart # enum {continuousSingle, continuousDouble, pagedSingle, pagedDouble} +│ ├─ page_viewport.dart # lazy windowed page hosting (visible + cache extent), recenter +│ └─ reader_controller.dart # reader vs annotate mode, page-flip, spread, quick-jump +├─ pdf/ +│ ├─ pdf_document_source.dart # wraps PdfDocument (open/dispose/pageSize/render), notebook page-map (F6) +│ ├─ page_tile.dart # PdfPageView host widget (bitmap) sized to host rect; drives tile DPI from transform scale (R11) +│ └─ page_tile_cache.dart # LRU for rendered PAGE BITMAPS — DPI-bucketed (tiles blur on upscale); owns native-handle dispose (R11) +├─ board/ # F7 +│ ├─ board_host_pane.dart # infinite InteractiveViewer board, reuses engine +│ └─ link_graph.dart # 双链 backlink model + queries +├─ text/ # OneNote-like editable text boxes (P3+, optional in P2 notebook) +│ ├─ text_box_model.dart +│ └─ text_box_layer.dart +├─ persistence/ +│ ├─ editor_repository.dart # batched load/save per document; maps DB ↔ engine +│ └─ save_scheduler.dart # debounced, synchronous-snapshot-before-await +├─ search/ # F8 (P3) +│ ├─ search_indexer.dart # PDF text + typed text + OCR → document_fts/library index +│ └─ ocr_ingest.dart # bridge to lib/services/ocr (text-line; formula later) +├─ sync/ # F9 (P4) +│ └─ sync_client.dart # talks to server/ (already has push/pull/auth) +└─ ui/ + ├─ editor_screen.dart # top-level editor (replaces pen_editor_screen + pdf_annotator_screen) + ├─ editor_toolbar.dart # Material You subtoolbars (F11) + ├─ thumbnail_grid.dart # F4 nav + └─ editor_shortcuts.dart # tool numbers, Ctrl+Z/Y/F/S (mouse/desktop) +``` + +### 1.2 Dependency graph (acyclic) + +``` +ui/ ─────────────► layout/ ─► pdf/ ─► (pdfrx PdfDocument/PdfPageView) + │ │ + │ ├──────► render/ ─► engine/ (geometry, store, hosts) + │ └──────► input/ ─► engine/ + ├──► board/ ─────────────► render/, engine/ + ├──► text/ ─────────────► engine/(hosts) + ├──► search/ ────────────► persistence/, services/ocr/ + ├──► sync/ ────────────► persistence/, server API + └──► persistence/ ───────► services/database_service.dart, engine/(stroke_model) +``` +**engine/** depends on nothing in editor/ except itself (and `perfect_freehand`). **render/input/layout/** depend on engine. **ui/** is the only place wiring Riverpod controllers. This is the seam that lets PDF-page host, infinite board, and (P5) CAS overlay all reuse one renderer. + +### 1.3 Coordinate model — single source of truth +**Truth = host content coordinates** (already the live convention; generalize it): +- **NormalizedPageHost:** points ∈ `[0,1]` of the page rect (today `PenPoint(nx, ny)`); width as fraction of page width (today `PenStroke.width`). Stored per `(documentId, pageIndex)`. +- **BoardHost:** absolute logical px on an unbounded canvas; `InteractiveViewer` supplies pan/zoom. + +`CoordinateSpaceHost` exposes `toContent(deviceLocal, deviceSize)`, `toDevice(content, deviceSize)`, `applyContentToCanvas(canvas, deviceSize)`. The live painters already scale `nx*pageSize.width` at paint time — formalize that into `applyContentToCanvas` (a `canvas.scale`) so no transformed-point copies are ever allocated. Text boxes are a **sanctioned widget-space exception** (Positioned widgets multiply by current zoom; stored truth stays content coords for search). + +### 1.4 How today's `lib/editor/canvas/` evolves into this (no rewrite) + +| Today (live) | Becomes | Action | +| --- | --- | --- | +| `pen_stroke.dart` `PenStroke`/`PenPoint` (in-memory only) | `engine/stroke_model.dart` `EditorStroke`/`EditorPoint` **persistable** (freezed/JSON, normalized) | **generalize + add JSON**; converge with `InkStroke` (§4) | +| `ink_painters.dart` `StaticInkPainter`/`LiveInkPainter` + `buildStrokePath` | `render/static_ink_painter.dart` (+ `render/ink_picture_cache.dart`), `render/live_ink_painter.dart`, `engine/stroke_geometry.dart` | **move + add ink `ui.Picture` cache (resolution-independent, revision-keyed) + revision gate**; `buildStrokePath`→`buildStrokeOutline` shared with export | +| `pen_canvas.dart` `_PenCanvasState` gesture logic (`_activePointers`, `_shouldDraw`, palm reject, `_eraseAt`) | `input/input_arbiter.dart` (pure SM) + `render/annotation_layer.dart` (widget) | **extract** SM out of the widget; eraser→`engine/stroke_eraser.dart` | +| `pen_editor_screen.dart` single-page + `_strokesByPage` Map + `_transform` + slider | `ui/editor_screen.dart` + `layout/page_viewport.dart` + `persistence/editor_repository.dart` | **generalize** single-page → windowed multi-page; `_strokesByPage` → repository-backed `HostState`s | +| `pen_capture_region.dart` + `PenCaptureBinding` (unused by live canvas) | — | **retire** after P0 confirms the `Listener`-in-shared-transform model holds on device (§6, §9) | +| `pen_editor_screen` single `PdfPageView` bitmap inside the `InteractiveViewer` (1× layout-sized bitmap, matrix-scaled by pinch) | `pdf/page_tile.dart` **multi-resolution tile** whose render DPI tracks the transform scale | **NEW work (not relocation): own multi-resolution tiling** — pinch GPU-upscales a 1× bitmap → blurry at high zoom (R11). On zoom-settle, re-instantiate the tile (or `PdfPage.render()`) at a DPI matching the current scale into `pdf/page_tile_cache.dart` (DPI-bucketed page-bitmap cache, distinct from the ink cache). Budgeted P1/P0.5. | + +Net: the four live files are **promoted, not thrown away**. P0 is mostly *relocation + persistence + revision-gated Picture cache*, which is low risk and immediately shippable. The one genuinely **new** burden the own-canvas choice imposes (beyond layout/windowing) is **multi-resolution page-tile rendering** — see R11 (§7) and the P0.5 gate (§5/§9): we own crisp-on-zoom re-rasterization that pdfrx's own viewer would have given for free. + +--- + +## 2. Per-feature design (file-level) + +### F1 — Pen-first editor core + persistence (P0) +**Add/modify:** +- `engine/stroke_model.dart` **[ADD]** — `EditorStroke { id, List points, EditorTool tool, int color, double width, bool filled, String? textContent, double fontSize }`, `EditorPoint { double x, y; double? pressure; double? tilt; int? timestamp; InputDeviceKind? pointerDeviceKind }`. freezed + `toJson/fromJson`. (Normalized; `width` = fraction of page width, matching live `PenStroke`.) **SF1 — non-lossy superset of `InkStroke`/`InkPoint`:** `InkPoint` carries `tilt`/`timestamp`/`pointerDeviceKind` (database_service.dart:346–362) that the live `PenPoint` drops; `EditorPoint` includes them now (nullable, cheap with freezed) so the `InkStroke`↔`EditorStroke` adapter (for OCR/export reuse) is **lossless in both directions**. The live `PenCanvas` simply leaves the extra fields null at capture; existing `InkStroke` data round-trips intact. (Avoids a silent data-loss footgun for OCR, which keys on `pointerDeviceKind`/`pressure`.) +- `engine/stroke_store.dart` **[ADD]** — `class StrokeStore { List committed; int revision; add/removeAt/replace bump revision; }`. Fixes the live `pen_editor_screen._commitStroke` "new list identity" hack by making revision explicit. +- `engine/stroke_geometry.dart` **[ADD]** — `Path buildStrokeOutline(EditorStroke, Size, {bool isComplete})` lifted verbatim from `ink_painters.buildStrokePath` (the proven `getStroke(thinning: hl?0:0.7, smoothing:.5, streamline:.5, simulatePressure: !hasRealPressure && !hl)` recipe). **Single source** for screen + export. +- `engine/stroke_eraser.dart` **[ADD]** — pure `eraseHits(strokes, point, radius)` (extract live `_eraseAt`) + `splitStroke` for partial/segment erase. **Critic note — `splitStroke` is NEW behavior, not an extraction:** the live `pen_canvas._eraseAt` is **whole-stroke** (it removes the entire stroke on the first proximity hit and returns). There is no segment-split logic in the live canvas to port — `undo_manager.removeStroke` only *records* replacements, it does not compute them. So `splitStroke` (point-run splitting → 0/1/2 sub-strokes) is greenfield code; its tests (§8) exercise new functionality, not a regression port. `eraseHits` (whole-stroke) IS an extraction of the live behavior and stays available as the default erase mode. +- `engine/undo_stack.dart` **[ADD]** — generalize `lib/services/undo_manager.dart` to host-tagged, commit-time-ordered global stack (entries carry `hostId`+`pageIndex`). +- `render/{annotation_layer,static_ink_painter,live_ink_painter,ink_picture_cache}.dart` **[ADD]** — relocate live painters; `StaticInkPainter` reads `ink_picture_cache` (`LRU`) keyed by `revision` (`shouldRepaint = old.revision != revision` — O(1)); per-host `RepaintBoundary`. **`ink_picture_cache` is for STATIC VECTOR INK only — resolution-INDEPENDENT, NO DPI bucket** (a `ui.Picture` of vector strokes re-rasterizes crisp at composite time at any zoom). It is a **different cache from the page-bitmap `page_tile_cache`** (§2/F2, R11), which IS DPI-bucketed because raster page tiles blur on upscale. Do not conflate the two. +- `input/input_arbiter.dart` **[ADD]** — pure SM extracted from `pen_canvas` (`idle→inking→erasing`, `pointerCount>=2`→cancel+pan, palm reject = touch dropped while stylus active, eraser = `kSecondaryButton || invertedStylus`). Keeps the live pressure-normalization logic (`_normalizedPressure`). +- `persistence/editor_repository.dart` **[ADD]** — `loadDocument(documentId) → Map` (one batched query), `saveHost(documentId, hostId, EditorStroke list)`. + - **MF3 — write contract (load-bearing for the per-row choice):** `saveHost` MUST **diff by stroke id** against the rows already on disk for that host — **UPSERT only changed/new rows, DELETE only removed rows**. It must NOT delete-all-rows-for-host then re-insert (that is exactly what the live notes path does at database_service.dart:281–306, and at 2,000 strokes it is **slower** than a single blob rewrite — re-inserting 2,000 rows per save). The per-stroke-row schema (§3) is justified **only** under this diff contract: erasing 1 of 2,000 strokes ⇒ 1 DELETE, 0 re-inserts; adding 1 stroke ⇒ 1 INSERT. The `SaveScheduler` hands `saveHost` the synchronously-captured stroke snapshot; the repository keeps a last-persisted id-set per host to compute the diff. If diffing proves fiddly under churn, the fallback is a per-page blob (NOT delete-all+reinsert) — but the diff path is the default and is what makes per-row worthwhile. + - **Durability invariant (Critic):** the UPSERT(s) + DELETE(s) for one `saveHost` MUST run inside **one `sqflite` transaction**, and the in-memory **per-host last-persisted id-set is updated ONLY after that transaction commits** (in the `then`/post-await success path) — never optimistically before the write. A crash or interruption mid-diff must leave memory and disk consistent: either the whole diff applied (and the id-set advances) or none of it did (and the id-set is unchanged, so the next save re-derives the same diff and retries). This mirrors the existing `updateNote` interruption warning (database_service.dart:281–306: "an interruption mid-way would permanently lose strokes, so the whole sequence must run inside one transaction"). +- `persistence/save_scheduler.dart` **[ADD]** — debounced ~800ms; **serialize JSON synchronously before any await**; flush on page-leave/dispose. +- `ui/editor_screen.dart`, `ui/editor_toolbar.dart` **[ADD]** — port `pen_editor_screen` UI (floating Material You palette + page pill already good). + +**Public interfaces (key):** +```dart +abstract class CoordinateSpaceHost { + String get hostId; int get pageIndex; + Offset toContent(Offset deviceLocal, Size deviceSize); + Offset toDevice(Offset content, Size deviceSize); + void applyContentToCanvas(Canvas c, Size deviceSize); +} +class EditorController extends ChangeNotifier { // ui-facing, Riverpod-provided + void beginStroke(CoordinateSpaceHost h, EditorPoint p); + void extendStroke(EditorPoint p); + void commitStroke(); // getStroke once → append → revision++ → undo push → scheduleSave + void eraseAt(CoordinateSpaceHost h, Offset content, double radius); + void undo(); void redo(); + void setTool(EditorTool t); void setColor(Color c); void setLayout(PageLayout l); +} +``` +**Acceptance:** draw/erase/undo/redo on a page; close+reopen → strokes persisted (DB); `StaticInkPainter.shouldRepaint==false` while drawing a new stroke on a 2,000-stroke page (revision constant); no per-frame point allocation for committed strokes. + +### F2 — Page layout modes (P1) +- `layout/page_layout.dart` **[ADD]** — `enum PageLayout { continuousSingle, continuousDouble, pagedSingle, pagedDouble }`. +- `layout/page_viewport.dart` **[ADD]** — windowed lazy hosting: only pages in `[firstVisible - cacheExtent, lastVisible + cacheExtent]` mount an `AnnotationLayer` + `PageTile`; others are disposed (Picture evicted). Continuous = scrollable column/two-column; paged = `PageView`. Recenter on layout switch. +- `pdf/pdf_document_source.dart` **[ADD]** — `PdfDocument` wrapper exposing `pageSize(i)`, `renderTile(...)`, page count; owns dispose. Replaces ad-hoc `PdfDocument.openFile` in `pen_editor_screen`. +- `pdf/page_tile.dart` **[ADD] — multi-resolution tile (R11 mitigation).** Under one shared `InteractiveViewer`, a `PdfPageView` renders a bitmap sized to its **layout constraints × devicePixelRatio** and the matrix scales that 1× bitmap, so pinch-zoom GPU-**upscales** → blurry text/rules at high zoom (unlike pdfrx's own viewer, which re-renders crisp tiles per zoom level). `page_tile` watches the transform scale and on **zoom-settle** (debounced) re-instantiates its render at a DPI matching the current scale — via either a re-laid-out `PdfPageView` at the new pixel size or `PdfPage.render(width/height at target DPI)` into `page_tile_cache`. **Cap retained DPI** (e.g. ≤ 3× base) to bound memory; downscale path stays matrix-only (sharp enough). This is **new own-canvas work**, not relocation. +- `pdf/page_tile_cache.dart` **[ADD]** — `LRU` of rendered **page bitmaps** (NOT ink). Key includes the **DPI bucket** so a page re-rendered at higher DPI replaces (not duplicates) its lower-DPI tile. **Owns the native-handle dispose lifecycle** of each `PdfPage.render()` result (`PdfImage` → backing `ui.Image`), deferred to post-frame so the raster thread is done with an evicted tile before disposal. **Distinct from `render/ink_picture_cache.dart`** (§2/F1) which holds resolution-independent ink `ui.Picture`s with no DPI bucket. +- `ui/editor_toolbar.dart` **[MODIFY]** — layout switcher control. + +**Two separate caches + memory budgets (Critic — do not conflate):** +- **`ink_picture_cache` (vector ink):** `LRU`, ~8–12 mounted-page Pictures, dispose deferred to post-frame. Budget ~ a few MB (vector op-lists are cheap). No DPI bucket. +- **`page_tile_cache` (raster page bitmaps):** `LRU`. **This is the heavy one.** A single A4 page rendered at 3× DPI is ~**10–30 MB** (≈ 1785×2526 px × 4 bytes ≈ 18 MB at 3× of a 595×842 pt page @ ~2 dppt). So **8–12 tiles at 3× would be ~150–350 MB — the old "≤64 MB / 8–12 pages" figure (R10) was unit-confused** (it conflated ink Pictures with page bitmaps). **Resolution:** size the *tile* window to the device memory budget, not a fixed page count — e.g. keep **full-DPI tiles only for the visible + ±1 pages (≈ 4 in double-page), and downgrade off-window pages to a 1× thumbnail tier** (matrix-upscaled, accepted as blurry only while scrolling). The ink cache keeps its wider ~8–12 window (cheap). R10's ≤64 MB now applies to **ink + downgraded-tier tiles**; the small high-DPI tile set is budgeted separately (~64–128 MB depending on device), tuned in P0.5/P1. + +**Data structures:** `PageWindow { int first, last }`; `Map mountedHosts`; `TileKey { hostId, dpiBucket }`. **Acceptance:** all four modes render; switching recenters; double-page shows two pages side-by-side; ink tracks scroll/zoom; **page text/rules stay crisp at 4× zoom (no GPU-upscale blur) — DPI refreshes on zoom-settle** (R11); combined cache memory stays within the device budget under a full scroll; perf gate (§7) holds. + +### F3 — Book-like reading (P1) +- `layout/reader_controller.dart` **[ADD]** — `ReaderMode { read, annotate }`; in `read` the arbiter never draws (pen ignored); page-flip animation for `pagedSingle/Double` via `PageView` physics; spread layout from `pagedDouble`. +- `ui/editor_screen.dart` **[MODIFY]** — mode toggle in toolbar; quick-jump via slider/grid (no keyboard). + +**Acceptance:** read mode blocks ink; annotate mode draws; page-flip animates; two-page spread aligns facing pages; quick-jump scrolls/animates to target. + +### F4 — Thumbnail-grid navigation (P1) +- `ui/thumbnail_grid.dart` **[ADD]** — Drawboard-style grid; tiles render via `pdf/pdf_document_source.renderTile` (replaces `lib/services/thumbnail_service.dart` syncfusion path). Tap → jump; **drag-reorder** for notebook page order (F6). Slider remains the linear scrubber (already in `pen_editor_screen._buildPagePill`). +- `lib/services/thumbnail_service.dart` **[MODIFY/RETIRE]** — migrate to pdfrx render; drop `syncfusion_pdfviewer_platform_interface` dep once ported. + +**Acceptance:** grid shows all pages; tap jumps; drag reorders (F6); no text-field page input. + +### F5 — Configurable pen (P2) +- `input/pen_config.dart` **[ADD]** — `PenConfig { ButtonAction sideButton; ButtonAction eraserEnd; PressureCurveType curve; double palmSensitivity; bool fingerDrawing; }`; `enum ButtonAction { eraser, undo, toggleTool, pan, lasso, none }`. `InputArbiter` consults it (replaces hardcoded `_isEraserSignal`). +- `lib/providers/settings_provider.dart` **[MODIFY]** — persist `PenConfig` (extend existing `SharedPreferences` notifier; it already stores pressure curve + stabilization). +- `lib/screens/settings_screen.dart` **[MODIFY]** / **[ADD]** `ui/pen_settings_page.dart` — mapping UI (SpeedyNote-style dialog). +- Reuse `lib/models/pressure_curve.dart` (already has linear/soft/hard/custom + `apply`). + +**Acceptance:** remapping side-button to undo makes the barrel button undo; pressure curve changes stroke taper; palm sensitivity changes touch-cooldown; finger-drawing toggle works; all persist across restart. + +### F6 — One-notebook-per-PDF (P2) +- `pdf/pdf_document_source.dart` **[MODIFY]** — a **page-map**: logical notebook pages → either a source-PDF page index or a synthetic blank page. Insert-blank adds a synthetic page **without rasterizing or editing the source bytes** (keep source vector + searchable). Ink/text bind to the **logical** page id (stable UUID per logical page), not raw PDF index, so inserts don't reshuffle annotations. +- `lib/services/pdf_service.dart` **[MODIFY]** — keep headless syncfusion export/mutate; export walks the page-map. **Fix the hairline bug**: `_renderStrokes` must build a `PdfPath` from `buildStrokeOutline` points and **fill** it (currently strokes line-segments → hairline). Shape/line/arrow keep stroke semantics. +- DB **[MODIFY]** — `notebook_pages(id, document_id, ordinal, source_page_index INTEGER NULL, kind)`; ink keyed by `notebook_page_id` (§4). Portable bundle = zip {source.pdf, badnote.json(strokes/text/links/page-map)} + relink-on-open (match by content hash, fall back to picker). + +**Acceptance:** insert blank page between PDF pages → source PDF untouched (still vector/searchable in another viewer); reorder pages keeps ink attached; export `.pdf` shows filled ink matching screen (golden); bundle round-trips on another machine. + +### F7 — Infinite board + 双链 (P3) +- `board/board_host_pane.dart` **[ADD]** — `BoardHost` (absolute px) in a constrained-false `InteractiveViewer`; reuses `AnnotationLayer` + engine. Migrates `lib/screens/split_view_screen.dart` + `scratchpads` table. +- `board/link_graph.dart` **[ADD]** — `Link { srcRef, dstRef }` where a ref is `(kind: notebookPage|board|note, id)`; `backlinksOf(ref)`. Wiki-style `[[...]]` parsing in text boxes; sticky-note = a small board region or text box that can link to a page/note. Backlink panel queries `links` table. +- DB **[ADD]** — `links(id, src_kind, src_id, dst_kind, dst_id, created_at)` + indexes both directions; `boards(id, document_id NULL, strokes_json, ...)` (generalize `scratchpads`). + +**Acceptance:** create a sticky linking page 3 → note X; open note X shows a backlink to page 3; board draws at perf target; link graph survives restart; deleting a target leaves a dangling-link indicator (no crash). + +### F8 — Library-wide full-text search (P3) +- `search/search_indexer.dart` **[ADD]** — index three sources into a unified library index: **(a)** PDF embedded text (pdfrx `PdfPage.loadText`/`charRects` text API — source-pinned in P0.5 per SF4), **(b)** typed text boxes (`content`), **(c)** handwriting OCR (existing `services/ocr` text-line ONNX over rasterized strokes via `StrokeRasterizer`). Reuse existing `document_fts` FTS5 + a new `library_fts` spanning notebooks/boards/notes with `(ref_kind, ref_id, page, snippet)`. +- `search/ocr_ingest.dart` **[ADD]** — batch handwriting OCR per logical page (best-effort; set expectations: text-line only, no math). Background isolate; debounced after ink idle. **The OCR backend itself no-ops when the model is unavailable (onnx_recognition_backend.dart header: "verify on-device") — search MUST never block on, nor be gated by, OCR results.** +- `lib/screens/search_screen.dart` **[MODIFY]** — unified results with snippets + **jump-to-location** (open editor at the page + scroll, or board at the region). + +**Acceptance — split into two exit tiers (SF2):** +- **COMMITTED (P3 exit blocker):** PDF embedded text + typed-text-box search returns results with snippets; tapping a result opens and scrolls to the page/region. This tier alone satisfies the P3 search exit (it has no unverified-recall dependency). +- **ADDITIVE (best-effort, NOT a P3 exit blocker):** handwriting OCR contributes hits when the text-line model is available and confident; a word handwritten on page 5 may be returned. **Expectation set:** handwriting recall is limited by the text-line model and may return nothing on cursive/handwriting; math/formula is explicitly P5. P3 ships even if handwriting recall is poor, with UI copy stating the limit. + +### F9 — Server sync + AI refinement (P4) +- `sync/sync_client.dart` **[ADD]** — wire the **already-built** FastAPI endpoints (`/api/auth`, `/api/sync/push|pull`, `/api/notes`, `/api/documents`, `/api/ocr`). Last-writer-wins by `updated_at` (server already implements this). Sync notes + (later) notebooks/boards. +- `server/badnote_server/` **[MODIFY]** — extend `sync_router`/`models.py` to cover notebooks/boards/links (today only notes). Add an **AI-refine** endpoint that runs llm_wiki/VLM/LLM over a note/page → returns organized markdown (server-side, heavy deps gated like the OCR worker). +- `lib/providers/` **[ADD]** sync state provider; settings page server URL/token. + +**Acceptance:** push from device A, pull on device B reproduces notes; AI-refine returns organized text for a selected note; offline still fully functional (server optional, per README). + +### F10 — Ink CAS / formula (P5) +- `search/ocr_ingest.dart` **[MODIFY]** — add a **formula/math recognition** backend (new model; the current PP-OCRv4 is text-line only — this is the known hard sub-problem). Behind a feature flag. +- `engine/` **[ADD]** `cas/` — recognized formula → CAS (compute/solve) behind a toggle; renders result near the ink. Reuses the `CoordinateSpaceHost` seam for overlay placement. + +**Acceptance (stretch):** a handwritten `2+3=` toggled → shows `5`; recognized formulas become searchable. Gated, optional, lowest priority. + +### F11 — Modern UX polish (woven P0–P3) +- Material You already wired (`main.dart` `DynamicColorBuilder` + harmonized schemes + Inter). Continue: subtoolbars in `editor_toolbar`, drag-reorder thumbnails (F4/F6), **hover pre-warm** (warm the next page tile + Picture on stylus hover to cut pen-down latency — reuse the live `_onPointerHover` seam), animated mode/layout transitions. + +--- + +## 3. Data model & persistence (clean schema — data may reset) + +Bump DB to a fresh version with `_onCreate` only (keep `_onUpgrade` harmless). Today: notes/strokes, documents/annotations(JSON-per-page), bookmarks, ocr_results, document_fts(FTS5), scratchpads (DB v5, `lib/services/database_service.dart`). + +**Target tables (additions/changes in bold):** +- `documents` **[KEEP]** (drop editor reliance on `rotation`; page-map owns rotation). +- **`notebook_pages(id PK, document_id FK, ordinal INTEGER, source_page_index INTEGER NULL, kind TEXT, created_at)`** **[ADD]** — F6 logical pages. +- **`ink(id PK, host_kind TEXT, host_id TEXT, stroke_json TEXT, ordinal INTEGER, updated_at)`** **[ADD]** — strokes addressed by host (`host_kind ∈ {page, board}`, `host_id` = `notebook_page_id` or `board_id`). **Per-stroke rows** (not per-page blob) so a 2,000-stroke page doesn't rewrite on every save — **but ONLY valid under the MF3 diff-write contract (§2/F1 `editor_repository.saveHost`): UPSERT changed rows + DELETE removed rows, never delete-all+re-insert.** Without diffing, per-row is *worse* than a blob; the contract is what makes this schema correct. Index `(host_kind, host_id)`. +- **`text_boxes(id PK, host_kind, host_id, content TEXT, rect_json, font_size, color, updated_at)`** **[ADD]** — F2-area text + F8 indexing. +- **`boards(id PK, document_id FK NULL, title, created_at, updated_at)`** **[ADD]** — generalize `scratchpads`; board strokes live in `ink` with `host_kind='board'`. +- **`links(id PK, src_kind, src_id, dst_kind, dst_id, created_at)`** **[ADD]** — 双链; indexes on `(src_kind,src_id)` and `(dst_kind,dst_id)`. +- `bookmarks` **[KEEP]**. +- `ocr_results` **[KEEP/EXTEND]** — per logical page handwriting OCR text. +- **`library_fts` (FTS5)** **[ADD]** — `(ref_kind, ref_id, page, content)` unified search over PDF text + typed text + OCR. `document_fts` **[KEEP]** for PDF-page text. +- `notes`/`strokes`/`notes_fts` **[KEEP]** (existing ink-note path; eventually folded into boards, but not deleted in P0). +- **`sync_state(entity_kind, entity_id, last_pushed_at, last_pulled_at, dirty)`** **[ADD, P4]**. + +**Coordinate semantics on disk:** page ink = normalized `[0,1]` unrotated; board ink = absolute logical px. (Matches live conventions.) + +**Stroke-model convergence (important):** there are currently **two** stroke models — `PenStroke/PenPoint` (live canvas, in-memory only) and `InkStroke/InkPoint` (freezed/JSON, DB+OCR+export). P0 introduces **one** canonical `EditorStroke/EditorPoint` (freezed/JSON, normalized) and adapters to/from `InkStroke` for OCR/export reuse during transition; old screens keep `InkStroke` until retired (§6). + +--- + +## 4. Refactor strategy (evolve, don't rewrite) + +**Principle: every phase ships; the engine generalizes under load.** + +1. **P0 = relocation + persistence.** Move the 4 live canvas files into `engine/` + `render/` + `input/` with minimal logic change; add `StrokeStore.revision` + `ui.Picture` cache + DB persistence via `editor_repository`. The live single-page editor keeps working throughout. **Keep** `PenStroke` as a thin alias of `EditorStroke` until callers migrate. +2. **P0.5 = vertical slice (continuous-single only) — the new perf/crispness gate (see §5/§9).** Stand up `layout/page_viewport` + `pdf/page_tile` rendering continuous-SINGLE only, with the **rewritten** perf bench (targeting `ui/editor_screen`, not the spike) and the zoom-DPI refresh proven crisp at 4× on the Surface. This proves the own-canvas multi-page + multi-resolution model on the real device before any double/paged/spread work. +3. **P1 = full multi-page windowing.** Wrap the (now-relocated) `AnnotationLayer` in `page_viewport`; `_strokesByPage` Map → repository-backed mounted hosts. Single-page path stays as `pagedSingle`. Continuous-double / paged / spread land here, AFTER P0.5 passes. +4. **Old-code retirement (gated on parity, then delete in one step):** + - **[DELETE after P1 parity]** `lib/screens/pdf_annotator_screen.dart`, `lib/widgets/pdf_annotation_layer.dart`, `lib/widgets/ink_canvas.dart` (extract any unique draw/erase logic to `engine/` first), `lib/editor/pdf/spike_*.dart` (throwaway M1 spike — includes the spike-based perf bench, replaced in P0.5), `lib/editor/pdf/pen_capture_region.dart` + `PenCaptureBinding` in `main.dart` (own-canvas `Listener` model won — remove the unused arena-bypass binding once P0 device-confirms). + - **[DELETE in P3, NOT P1 (SF3)]** `lib/screens/split_view_screen.dart` — its replacement is the **P3** infinite board (F7); deleting it in P1 would leave a 2-phase functionality gap (no scratchpad between P1 and P3). It stays live until the board lands. + - **[KEEP]** `thumbnail_service` until F4 ports it; `stroke_rasterizer` (OCR), `ctc_decoder`, `onnx_recognition_backend`, OCR assets; `pdf_service` (export/mutate, with the fill fix); `pptx_service`/`ppt_annotator_screen` (PPT is separate; not in scope but not deleted). +5. **Navigation swap:** `home_screen` `_openDocument`/`_importPdf` currently push `PdfAnnotatorScreen`; `openM1Spike` pushes `PenEditorScreen`. Repoint both to `ui/editor_screen.dart` once P1 parity passes (checklist below). +6. **Parity checklist before any delete:** page rotate/delete/insert-blank/insert-image (`pdf_service`), bookmark add/toggle/jump, undo/redo across pages, save-on-leave, zoom in/out/fit, export-matches-screen (golden). Mirrors phase-1 §10 M3 checklist C1–C11. + +--- + +## 5. Phased delivery (each = shippable milestone with exit criteria) + +> Sequence de-risks: pen core + persistence first (P0), then the 60fps multi-mode layout (P1), then config/notebook (P2), then board/双链/search (P3), then sync/AI (P4), then CAS (P5). + +**P0 — Pen core hardened + persisted (own-canvas).** *Exit:* draw/erase/undo/redo on a single PDF page persist to DB and reload; `EditorStroke` canonical model (superset of `InkPoint`, SF1) + revision-gated `ui.Picture` cache; `editor_repository.saveHost` honors the MF3 diff-write contract (test: erase 1 of 2,000 ⇒ 1 DELETE, 0 re-inserts); pure `InputArbiter` + `stroke_eraser` unit-tested; **on-device Surface Pen confirms** pressure + palm rejection + pinch-zoom in the live shared-transform model (the one device gate). Export hairline bug fixed (fill). No regression to existing screens. + +**P0.5 — Vertical slice gate (own-canvas multi-page + multi-resolution), continuous-SINGLE only. [NEW — SYNTHESIS]** Stand up `layout/page_viewport` + `pdf/page_tile` rendering **continuous-single only** on the real document path (`ui/editor_screen`), and: +- (a) **REWRITE the perf bench** (§7/§8): a new `integration_test/editor_scroll_bench.dart` drives `ui/editor_screen` + `layout/page_viewport`; **DELETE the spike-based `integration_test/perf_scroll_bench.dart`** (it imports `spike_editor_pane.dart` / `pageOverlaysBuilder` = the invalidated Option B; it CANNOT validate own-canvas). +- (b) **Prove crisp-on-zoom (R11):** page-tile DPI refresh on zoom-settle renders crisp text/rules at 4× on the Surface (no GPU-upscale blur). +- (c) **Source-pin the pdfrx render/text APIs (SF4):** confirm `PdfPage.loadText`/`charRects`/`PdfPage.render()`/`PdfPageView` signatures present in pdfrx 2.4.4; smoke-test each, so F4/F8 don't hit drift late. +- *Exit (hard gate before any double/paged/spread):* **continuous-single median build+raster ≤ 16.6ms, p95 ≤ 22ms** on the 300-page asset via the REWRITTEN bench, **AND** a manual crisp-on-zoom-at-4× PASS on the Surface, **AND** the four pdfrx APIs source-pinned + smoke-tested. Continuous-double / paged / spread do NOT begin until P0.5 is GREEN. + +**P1 — Full layout modes + book-like reading + thumbnails (60fps). Precondition: P0.5 GREEN.** *Exit:* the remaining `PageLayout`s (continuous-double, pagedSingle, pagedDouble); reader vs annotate; page-flip + two-page spread; thumbnail-grid jump + drag-reorder; windowed lazy hosting; **perf gates met** (§7) incl. continuous-double on a 300-page PDF via the rewritten bench; old PDF screens deleted after parity checklist (split_view retirement deferred to P3 per SF3); nav repointed. + +**P2 — Configurable pen + one-notebook-per-PDF.** *Exit:* Pen settings page (button/eraser mapping, curve, palm sensitivity, finger toggle) persists + drives arbiter; insert-blank/reorder pages keep source vector + ink attached to logical pages; portable bundle round-trips; export walks page-map. + +**P3 — Infinite board + 双链 + library search.** *Exit:* board reuses engine at perf; sticky-notes + `[[links]]` produce backlinks; **library search COMMITTED tier** (PDF embedded text + typed-text-box) returns snippets + jump-to-location (this tier is the exit blocker, SF2); **handwriting-OCR tier is ADDITIVE/best-effort and does NOT block P3 exit**; `split_view_screen` retired into board (the SF3 deletion point). + +**P4 — Server sync + AI refinement.** *Exit:* push/pull notes+notebooks across two devices (LWW); AI-refine endpoint returns organized markdown; app fully functional offline. + +**P5 — Formula OCR + ink CAS.** *Exit (stretch, gated):* formula recognition backend; searchable formulas; CAS solve toggle for simple expressions. + +Every milestone ends with verifier/critic pass + perf-results doc updated with commit hash. + +--- + +## 6. Refactor of input transport (record the pivot) +The live `PenCanvas` proves the **own-canvas** model: a single `Listener` over an `InteractiveViewer` whose child is `Stack(PdfPageView bitmap, StaticInk, LiveInk)`. Because the pen, page bitmap, and ink share **one** transform and the `Listener` arbitrates by `_activePointers.length` + `kind`, there is **no gesture-arena fight** — pdfrx never sees gestures (it only renders). This **removes** the phase-1 need for `PenCaptureRegion`/`PenCaptureBinding` (RenderProxyBox arena bypass), which exist for the abandoned `pageOverlaysBuilder` approach. **Retirement gate:** delete them once P0 confirms on the physical Surface Pen that the `Listener` model handles stylus draw + single-finger pan + pinch-zoom + palm rejection (the live code is built for exactly this; confirm, then remove the dead binding). + +--- + +## 7. Risks & mitigations + +| # | Risk | L | I | Mitigation / trigger | +| --- | --- | --- | --- | --- | +| R1 | **60fps continuous + double-page on a 300-page PDF.** Two columns × windowed tiles × ink Pictures may blow frame budget. | Med | High | Windowed lazy hosting (only visible+cacheExtent mounted), bounded LRU + post-frame dispose for BOTH the `ink_picture_cache` and the `page_tile_cache` (R10), per-page `RepaintBoundary`, revision-gated static Picture. **Gate continuous-SINGLE in P0.5, continuous-double in P1** (§5/§9). **MF1 — the existing `integration_test/perf_scroll_bench.dart` is UNUSABLE here: it imports `spike_editor_pane.dart` and benchmarks `pageOverlaysBuilder` (Option B, the INVALIDATED architecture) — it cannot validate own-canvas. P0.5 REWRITES the bench against `ui/editor_screen` + `layout/page_viewport` and DELETES the spike-based one.** Uses `large_300p.pdf`. | +| R2 | **Pen feel only verifiable on the user's Surface** (CI has no pen). | High | Med | P0 device gate (pressure/palm/pinch) + manual checklist in perf-results doc; synthesized-stylus widget tests as interim signal only. | +| R3 | **Handwriting/formula OCR feasibility.** Current model is text-line PP-OCRv4; math is unsolved. | High | Med | F8 ships text-line best-effort with **explicit expectation-setting**; formula isolated to P5 behind a flag; never block search on OCR quality. | +| R4 | **双链 graph scale** (thousands of links/sticky-notes). | Low | Med | Indexed `links` table (both directions), lazy backlink queries, no in-memory full graph. | +| R5 | **Server/AI scope creep.** | Med | Med | Server stays optional (README); P4 wires existing endpoints + one AI-refine route; AI heavy deps gated like OCR worker. | +| R6 | **Windows pen edge cases** (no advertised pressure range; barrel-button eraser; inverted stylus). | Med | Med | Live `_normalizedPressure` already handles degenerate ranges; F5 makes button/eraser mappable; test matrix in arbiter unit tests. | +| R7 | **Export fidelity** (hairline bug today). | High (today) | Med | `stroke_geometry.buildStrokeOutline` shared screen+export; `pdf_service._renderStrokes` **fills** a `PdfPath`; golden test (P0). | +| R8 | **Stroke-model convergence churn** (two models today). | Med | Low | One canonical `EditorStroke` + adapters; old `InkStroke` retained only where old screens/OCR/export still use it, deleted with them. | +| R9 | **pdfrx page-render / text API drift** (thumbnails, search text, double-page sizing). | Med | Low | **Source-pin `PdfDocument`/`PdfPageView`/`PdfPage.loadText`/`charRects`/`render()` signatures + smoke-test in P0.5 (pulled forward, SF4)** so F4/F8 don't hit drift late; keep syncfusion export until verified. | +| R10 | **Cache memory budget vs many mounted pages (TWO caches — Critic).** | Med | Med | **`ink_picture_cache`** (vector `ui.Picture`, no DPI bucket): ~8–12 mounted pages, a few MB. **`page_tile_cache`** (raster `ui.Image`, DPI-bucketed): the heavy store — a 3×-DPI A4 tile is ~10–30 MB, so 8–12 high-DPI tiles would be ~150–350 MB. **The old "≤64MB / 8–12 pages" figure was unit-confused (conflated ink with page bitmaps).** Fix (§2/F2): keep full-DPI tiles only for visible ±1 pages, downgrade off-window pages to a 1× thumbnail tier; ≤64MB applies to ink + downgraded tiles, the small high-DPI tile set budgeted separately (~64–128MB, device-tuned). Sample BOTH caches during the scroll bench. | +| R11 | **Blurry page at high zoom under the shared transform (MF2).** One `InteractiveViewer` matrix-scales a `PdfPageView` bitmap that was rendered at **layout-constraint × devicePixelRatio** (1×); pinch-zoom GPU-**upscales** it → blurry text/rules at 4×, whereas pdfrx's own viewer re-renders crisp tiles per zoom level. We **own multi-resolution tiling**, not just layout/windowing. | Med | High | `pdf/page_tile.dart` drives render DPI from the transform scale: on zoom-settle re-instantiate the tile (re-laid-out `PdfPageView` at the new pixel size, or `PdfPage.render()` at target DPI) into the **`pdf/page_tile_cache.dart`** store (`LRU` — NOT the ink cache); cache key includes a DPI bucket; **cap retained DPI** (~3× base) to bound memory; downscale stays matrix-only. **`PdfPage.render()` returns an async `PdfImage` owning a native handle → `page_tile_cache` owns its post-frame dispose** (see Open Questions; confirm in the SF4 P0.5 source-pin). **Proven crisp at 4× on the Surface in the P0.5 gate** (§5). Budgeted P0.5/P1, not relocation. | + +### Pre-mortem (DELIBERATE — 3 scenarios) +1. **"Continuous double-page janks / pages are blurry at 4× on the user's big scanned PDF."** Cause: built layout modes before profiling two-column windowing, and matrix-scaled a 1× bitmap (R11). *Prevention:* the **P0.5 vertical-slice gate** profiles continuous-SINGLE first with a **rewritten** bench against `ui/editor_screen` (the spike-based `perf_scroll_bench.dart` is deleted — it benchmarks the invalidated Option B) AND proves crisp-on-zoom DPI refresh at 4× on the Surface; continuous-double/paged/spread merge only after P0.5 is GREEN, and the P1 double-page gate reuses the rewritten bench. +2. **"Search returns nothing for handwriting."** Cause: over-promised OCR. *Prevention:* F8 ships PDF-text + typed-text search first (reliable), handwriting OCR as additive best-effort with UI copy stating limits; formula explicitly P5. +3. **"Insert-blank-page silently rasterized / detached annotations."** Cause: editing source bytes or keying ink to raw PDF index. *Prevention:* logical page-map + ink keyed to `notebook_page_id`; golden test that the source PDF bytes are unchanged after insert and remains selectable-text in an external viewer. + +--- + +## 8. Testing strategy + +> **sqlite workaround:** DB-touching tests run via `tool/test.sh` (system sqlite + `LD_LIBRARY_PATH`); pure-logic tests avoid the DB. Reusable assets: `tool/gen_bench_pdf.dart`, `tool/gen_dense_strokes.dart`, `test/assets/large_300p.pdf`, `test/assets/dense_strokes.json`, `integration_test/coordinate_assertion_test.dart`. **NOT reusable: `integration_test/perf_scroll_bench.dart` — it imports `spike_editor_pane.dart` (`pageOverlaysBuilder`, invalidated Option B) and is DELETED + replaced by `integration_test/editor_scroll_bench.dart` (targets `ui/editor_screen`) in P0.5 (MF1).** + +- **Unit (no DB/widgets):** + - Coordinate transforms: `NormalizedPageHost`/`BoardHost` round-trip `toContent(toDevice(x))≈x`. + - Stroke geometry: `buildStrokeOutline` non-empty for ≥1 point; live ⊆ committed bounds (no "pop"). + - Eraser: `eraseHits` whole-stroke removal (extraction of live `pen_canvas._eraseAt` — regression port). **`splitStroke` segment-erase is NEW behavior, not a port** (the live canvas only does whole-stroke erase; `undo_manager` records but never computes replacements): mid-erase ⇒ 2 segments, endpoint ⇒ 1 segment, full-erase ⇒ empty, <2-pt result dropped — these exercise greenfield code. (`undo_manager_test.dart` informs the replacement-bookkeeping discipline only.) + - Revision gating: `StrokeStore.add` bumps revision; `StaticInkPainter.shouldRepaint` iff revision changed. + - `InputArbiter` SM: table-driven over the device×mode matrix incl. palm rejection (touch dropped while stylus active) and `pointerCount>=2`→cancel. + - Undo: global commit-time order, host-tagged reversal. + - `SaveScheduler`: mutating "current host" after schedule but before write completes does not change persisted snapshot. + - **`editor_repository.saveHost` diff-write (MF3, fake DB counting statements):** erase 1 of 2,000 strokes ⇒ exactly **1 DELETE, 0 INSERT**; add 1 stroke ⇒ **1 INSERT, 0 DELETE**; no-op save ⇒ 0 statements. (Guards against the live notes-path delete-all+re-insert anti-pattern at database_service.dart:281–306.) + - `EditorStroke`↔`InkStroke` adapter round-trip (SF1): `tilt`/`timestamp`/`pointerDeviceKind` survive both directions (no lossy OCR/export conversion). + - `link_graph.backlinksOf` (P3); `search_indexer` snippet/jump-ref mapping for the COMMITTED tier — PDF-text + typed-text (P3). +- **Widget:** stylus→stroke committed, touch→not consumed (synthesized pointers); layout-mode switch recenters; reader-mode blocks ink; thumbnail tap jumps; text-box place/move/edit persists (fake DB). +- **Perf:** the **rewritten** `integration_test/editor_scroll_bench.dart` (targets `ui/editor_screen` + `layout/page_viewport`; the spike-based `perf_scroll_bench.dart` is deleted, MF1) on `large_300p.pdf` — **continuous-single in P0.5 (gate), continuous-double in P1**, profile mode, N≥120 frames warm, median build+raster ≤16.6ms / p95 ≤22ms; `StaticInkPainter` no-rebuild assertion; Picture-memory ≤64MB sample. Archived in `docs/plans/full-refactor-perf-results.md` with commit hash. Not a hard CI gate (no GPU) but required for milestone sign-off. +- **Manual on-device pen checklist (Surface Pen, P0 + each milestone):** pressure varies width; barrel/inverted = erase; palm rest doesn't mark; single-finger scroll; two-finger pinch; **page text/rules stay crisp at 4× zoom — no GPU-upscale blur, DPI refreshes on zoom-settle (R11, P0.5 gate)**; hover pre-warm reduces first-stroke latency; page-flip feels book-like. Recorded with device + commit. +- **Regression:** keep `ctc_decoder_test.dart`, `undo_manager_test.dart` green; update `widget_test.dart` to boot `editor_screen`. +- **Export golden (P0):** annotate a known page → export → image-compare filled ink matches screen (R7). +- **Server (P4):** existing `server/tests/` (`test_sync.py` etc.) green; add notebook/board sync tests. + +--- + +## 9. Milestones / sequencing — immediate next chunk (concrete) + +**Next chunk = P0 (engine relocation + persistence), executable now; followed by the P0.5 vertical-slice gate (steps 10–13) before any double/paged/spread layout work:** +1. **[ADD]** `lib/editor/engine/stroke_model.dart` — `EditorStroke`/`EditorPoint` (freezed + JSON), normalized; `fromPenStroke`/`toInkStroke` adapters. Run `build_runner`. +2. **[ADD]** `lib/editor/engine/stroke_geometry.dart` — lift `buildStrokePath` → `buildStrokeOutline` (verbatim recipe from `ink_painters.dart`). +3. **[ADD]** `lib/editor/engine/stroke_store.dart` + `lib/editor/render/{static_ink_painter,live_ink_painter,ink_picture_cache,annotation_layer}.dart` — relocate live painters; add the resolution-independent ink `ui.Picture` cache keyed by `revision` (NO DPI bucket; the DPI-bucketed `page_tile_cache` is a separate P0.5 file, step 10). +4. **[ADD]** `lib/editor/input/input_arbiter.dart` + `lib/editor/engine/stroke_eraser.dart` — extract from `pen_canvas.dart` (pure, unit-tested). +5. **[ADD]** `lib/editor/persistence/{editor_repository,save_scheduler}.dart` + DB additions (`ink`, `notebook_pages` minimal) in `database_service.dart` (fresh version). `saveHost` implements the **MF3 diff-write contract** (UPSERT changed + DELETE removed, by stroke id; NO delete-all+re-insert) with a per-host last-persisted id-set. +6. **[MODIFY]** `lib/editor/canvas/pen_editor_screen.dart` (or new `ui/editor_screen.dart`) to load/commit/save through the repository instead of the in-memory `_strokesByPage` Map. +7. **[MODIFY]** `lib/services/pdf_service.dart` `_renderStrokes` → fill `buildStrokeOutline` path (R7) + export golden test. +8. **Tests:** arbiter SM, eraser, geometry, revision-gating, save-scheduler snapshot, **diff-write statement-count (MF3: erase 1/2000 ⇒ 1 DELETE 0 INSERT)**, `EditorStroke↔InkStroke` lossless round-trip (SF1), export golden. Run via `tool/test.sh`. +9. **Device gate:** build Windows package (CI), confirm pen/palm/pinch on Surface Pen; record in perf-results doc. → unblocks P0.5. + +**Then P0.5 (vertical-slice gate — must pass before P1's double/paged/spread):** +10. **[ADD]** `lib/editor/layout/page_viewport.dart` (continuous-single only) + `lib/editor/pdf/{pdf_document_source,page_tile,page_tile_cache}.dart` with zoom-settle DPI refresh (R11) + the **DPI-bucketed `page_tile_cache`** (`LRU`, page bitmaps; owns native-handle dispose). (The resolution-independent `render/ink_picture_cache.dart` for vector ink lands in P0, step 3 — it is a separate cache, no DPI bucket.) +11. **[ADD]** `integration_test/editor_scroll_bench.dart` targeting `ui/editor_screen`; **[DELETE]** `integration_test/perf_scroll_bench.dart` + `lib/editor/pdf/spike_*.dart` (the spike pane the old bench imports). Run the rewritten bench on a **scanned-image** 300-page asset (see Open Questions — `large_300p.pdf` is synthetic/vector and may not honestly stress raster re-render at 3× DPI) → continuous-single median ≤16.6ms / p95 ≤22ms; sample both caches' memory. +12. **[VERIFY]** source-pin pdfrx `PdfPage.loadText`/`charRects`/`render()`/`PdfPageView` in 2.4.4 (SF4) + smoke test; **confirm `PdfPage.render()`'s `PdfImage`/native-handle ownership + dispose semantics** so `page_tile_cache` can manage post-frame disposal; record signatures in perf-results doc. +13. **Device gate:** crisp-on-zoom at 4× PASS on the Surface (R11). → unblocks P1. + +Each subsequent milestone (P1…P5) follows §5 exit criteria; verifier/critic + perf-results update per milestone. + +--- + +## 10. RALPLAN-DR + +### Principles (3–5) +1. **Single source of truth = host content coordinates.** Screen mapping is a paint-time `canvas` transform; never store/duplicate transformed geometry. (Already the live convention — generalize it.) +2. **One host-agnostic ink engine.** PDF page, infinite board, and (P5) CAS overlay are `CoordinateSpaceHost`s behind one renderer — never fork the stroke pipeline. +3. **Own the gesture pipeline; pdfrx only renders.** A single `Listener` over a shared `InteractiveViewer` arbitrates draw/pan/zoom/palm by pointer kind + count — no gesture-arena fights (proven live). +4. **De-risk performance AND crispness before features.** A **P0.5 vertical-slice gate** (continuous-single, rewritten bench, crisp-at-4× on the Surface) precedes all double/paged/spread work; continuous-double is gated again in P1. +5. **Ship every phase; generalize under load.** Relocation-first refactor keeps the editor working at all times; old screens deleted only after parity. + +### Decision Drivers (top 3) +1. **D1 — Pen feel + palm rejection + pinch on Windows Surface Pen** is the make-or-break primary-device requirement (own-canvas model already targets it). +2. **D2 — 60fps across all layout modes** on big PDFs with thousands of strokes. +3. **D3 — One engine reused across PDF page, infinite board, and CAS**, forward-compatible with search/双链/sync. + +### Viable options (≥2) with bounded pros/cons + +**Option A — Own-canvas engine (single `Listener` + `InteractiveViewer`, pdfrx as renderer). CHOSEN (already live).** +- Pros: D1 solved structurally (one transform, no arena fight — already working in `pen_canvas.dart`); D3 trivial (hosts share the transform); minimal new deps; matches the proven Saber model the user cited. +- Cons: we own page layout/windowing/tiling **AND multi-resolution re-rasterization** (more code than a stock viewer); double-page perf is on us (R1) and crisp-on-zoom is on us (R11 — pdfrx's own viewer re-renders crisp tiles per zoom for free; we must re-derive tile DPI from the transform); thumbnails/text-extraction still need pdfrx page APIs (R9). + +**Option B — pdfrx `pageOverlaysBuilder`-hosted ink + RenderProxyBox arena-bypass (the phase-1 plan).** +- Pros: pdfrx gives continuous scroll/tiling/text-extraction for free; ink-follows-page is structural via page overlays. +- Cons: requires a custom `PenCaptureBinding` arena bypass that fights pdfrx's greedy scale recognizer (the live code already **abandoned** this — `PenCaptureRegion` is unused); pdfrx owns the transform so double-page/board reuse is awkward; D1 proven harder than Option A in practice. **Invalidated** — see below. + +**Option C — Flutter shell + Rust hot-path (rnote-style) for ink/render.** +- Pros: maximal ink perf headroom. +- Cons: rejected in memory (`badnote-flutter-344-windows-pen`) — Windows pen is weaker in the Rust/GTK stack; huge FFI surface; contradicts "stay Flutter." **Invalidated.** + +### Invalidation rationale +- **B invalidated (as the INPUT model):** the live codebase already moved off it; `pageOverlaysBuilder`+arena-bypass made stylus/touch arbitration fight pdfrx's recognizer, whereas Option A's single-`Listener`-over-shared-transform sidesteps the arena entirely and is already drawing with pressure/pinch/palm. **Correction (MF2): B had TWO edges, not one** — (1) free continuous scroll/windowing, AND (2) free **per-zoom crisp re-rasterization** (pdfrx re-renders tiles at each zoom level). Edge (1) is recoverable in A via windowed hosting (needed for double-page anyway); edge (2) is **NOT free in A** — Option A must own multi-resolution tiling (R11), which is the genuinely new cost of this choice. We accept that cost (gated in P0.5) because A's structural D1/D3 wins outweigh it. B retained only as a fallback page-**render** strategy, not the input model. +- **C invalidated:** documented Windows-pen regression in the Rust/GTK path + "stay Flutter" hard constraint; the 3.44 WM_POINTER fix already unblocked Flutter pen, removing C's motivation. +- Net: **A chosen; B retained as a partial fallback (page rendering only); C rejected.** + +### ADR +- **Status:** Architect APPROVE-WITH-MUST-FIX applied (2026-06-21): MF1 (rewrite perf bench off the invalidated spike), MF2 (R11 zoom re-rasterization + multi-resolution tiling), MF3 (diff-write contract); P0.5 vertical-slice synthesis gate; SF1–SF5. Pending Critic. Records the live pivot from the superseded phase-1 input architecture. +- **Decision:** Build the full BadNote vision on an **own-canvas, host-agnostic ink engine** (single `Listener` + shared `InteractiveViewer`; pdfrx as page renderer/text source). Generalize the live `lib/editor/canvas/` into `engine/render/input/layout/...`; persist a single canonical `EditorStroke`; phase features P0→P5 with a **P0.5 vertical-slice gate**. +- **Drivers:** D1 Surface-Pen feel, D2 60fps multi-mode, D3 one reusable engine. +- **Alternatives considered:** B (pdfrx-overlay + arena bypass — invalidated as input model, kept as render fallback; had two free edges — scroll AND per-zoom crispness — the latter is the new cost we take on), C (Flutter+Rust — rejected). +- **Why chosen:** A is already proven live for pen/palm/pinch and gives D1+D3 structurally; the remaining risks (D2 multi-mode perf AND R11 crisp-on-zoom) are gated up front in **P0.5** then re-gated for double-page in P1. +- **Consequences:** we own layout/windowing/tiling **AND multi-resolution page-tile re-rasterization** (R11 — not just layout, MF2); `PenCaptureRegion`/`PenCaptureBinding` retired (dead under own-canvas); the spike-based perf bench is deleted and rewritten against `ui/editor_screen` (MF1); per-stroke-row persistence is valid ONLY under the diff-write contract (MF3); `EditorPoint` is a non-lossy superset of `InkPoint` (SF1); export must fill (not stroke) ink; handwriting OCR is additive/non-blocking and formula is P5 (SF2); `split_view_screen` deletion deferred to P3 (SF3); server stays optional. +- **Follow-ups:** P0 device gate confirms pen model; **P0.5 gate** proves continuous-single perf + crisp-at-4× + source-pinned pdfrx APIs (SF4) before any double/paged/spread; resolve open questions below. + +--- + +## Open Questions (persist to `.omc/plans/open-questions.md`) +- [ ] Continuous-double-page on a real 300-page scanned PDF — does windowed two-column hosting hold 60fps, or do we need tile pre-rasterization? — *P0.5 gates continuous-single first; P1 perf gate decides double; affects R1/R10.* +- [ ] **How is page-tile DPI refreshed on zoom under the shared transform (SF5)?** — re-laid-out `PdfPageView` at the new pixel size, or `PdfPage.render()` at target DPI into `page_tile_cache`? What scale-change threshold + debounce triggers a refresh, and what is the retained-DPI cap? — *Load-bearing for the 60fps + crisp-at-4× goal; resolved in P0.5 (R11).* +- [ ] **`PdfPage.render()` native-handle lifecycle (Critic 4a):** `render()` returns an async `PdfImage` owning a native handle backing a `ui.Image` — `page_tile_cache` must own its dispose (post-frame, after the raster thread is done) on eviction. *Confirm exact ownership + dispose API in the SF4 P0.5 source-pin; affects page_tile_cache + R10/R11.* +- [ ] **Honest R11 raster stress asset (Critic 4b):** does the synthetic/vector `large_300p.pdf` actually force RASTER re-render cost at 3× DPI, or does a **scanned-image** asset better exercise the crispness + tile-memory gate? — *Add a scanned 300-page asset (`tool/gen_bench_pdf.dart` image mode or a real scan) for the P0.5/P1 perf+crispness gates; affects §8 / R11 honesty.* +- [ ] Stroke-model convergence: fold `notes`/`InkStroke` ink-note path into `boards`/`EditorStroke`, or keep notes separate long-term? — *Affects §3/§4 churn.* +- [ ] Handwriting OCR: is the existing PP-OCRv4 text-line model usable on cursive/handwriting at all, or do we need a handwriting-specific model even for non-formula text? — *Affects F8 expectation-setting / R3.* +- [ ] Portable bundle relink: match source PDF by content-hash only, or also store original path + size? — *Affects F6.* +- [ ] Server AI-refine: run llm_wiki/VLM server-side only, or allow a local LLM path for offline users? — *Affects F9 scope / R5.* +- [ ] Keep `pptx_service`/`ppt_annotator_screen` in the new engine, or freeze PPT support? — *Out of the 11 features; decide before P1 nav swap.* diff --git a/docs/plans/2026-06-22-badnote-pen-polish.md b/docs/plans/2026-06-22-badnote-pen-polish.md new file mode 100644 index 0000000..8192c6d --- /dev/null +++ b/docs/plans/2026-06-22-badnote-pen-polish.md @@ -0,0 +1,134 @@ +# BadNote — Pen-Polish + Native-Pen Addendum (ralplan consensus) + +**Status:** APPROVED (ralplan consensus 2026-06-22 — Architect APPROVE-WITH-MUST-FIX M1–M4 applied; Critic ITERATE→APPROVE after C1 PenPoint.tilt wrong-model fix + C2 eraser-race re-grounded on M1 native ordering + no-hover-down test) +**Date:** 2026-06-22 +**Mode:** DELIBERATE (native Windows plugin = new platform code; can only be device-verified) +**Owner plan file:** `docs/plans/2026-06-22-badnote-pen-polish.md` +**Extends (does NOT supersede):** `docs/plans/2026-06-21-badnote-full-refactor.md` + +> **Grounding (read live code 2026-06-22):** `lib/editor/canvas/{pen_canvas,pen_editor_screen,ink_painters,pen_stroke}.dart`, +> `lib/editor/engine/stroke_geometry.dart`, `lib/editor/input/pen_config.dart`, +> `windows/runner/{ocr_channel.cpp,flutter_window.cpp,win32_window.cpp,main.cpp}`, +> `windows/flutter/generated_plugin_registrant.cc`. + +--- + +## 0. Scope (4 user asks, mapped to the roadmap) + +| # | User ask (verbatim intent) | Root cause (verified) | Roadmap fit | +| --- | --- | --- | --- | +| **W1** | 添加自定义笔粗; `thinning` 肯定不能写死, 学习 Saber | `penWidth`/`highlighterWidth` already read from `PenConfig` (pen_editor_screen.dart:456–459); **but `thinning` is a hardcoded literal `0.85`** in `ink_painters.buildStrokePath` (line 38) AND `stroke_geometry.buildStrokeOutline` (line 52). | Pull **F5** (configurable pen) width+pressure slice forward to **now** | +| **W2** | 缩放的时候会闪一下 (zoom flickers once) | `PdfPageView` lives inside the `InteractiveViewer` child subtree (pen_canvas.dart:354); pdfrx re-rasterizes its page bitmap when the effective scale changes, showing a **one-frame white gap** during the async re-render = the flicker. | This IS **R11 / `pdf/page_tile`** (P0.5); add a **minimal double-buffer fix now**, full fix in P0.5 | +| **W3** | 修好 tilt 和笔按键映射 (fix tilt + pen button mapping) | **No native pen plugin exists** (only `badnote/ocr` MethodChannel). Flutter 3.44 Windows delivers `pressure` but NOT barrel→`buttons`, NOT eraser/inverted→`invertedStylus`, NOT `tilt`. So `_isEraserSignal` (pen_canvas.dart:134) never fires; `event.tilt` is always `0.0`. | **NEW work** the full-refactor plan did not budget: a native Windows pen plugin. Gates F5 button-mapping. | +| **W4** | 继续推进整体重构, 完成我所有的需求 | Roadmap exists & is approved; task #7 (P0 engine+persistence) `in_progress`. | Resume `2026-06-21-badnote-full-refactor.md` P0 → P0.5 → … after W1–W3. | + +**Non-goals here:** no new layout modes, board, search, server, or CAS (those stay in the parent roadmap's P1–P5). This addendum is only the pen-feel polish + the native-pen unblock that the user is blocked on *today*, sequenced so it feeds the parent plan's engine (`stroke_geometry`, `input_arbiter`, `pen_config`) rather than the throwaway live widgets. + +--- + +## 1. RALPLAN-DR + +### Principles +1. **Touch the canonical engine, not the live widgets.** Width/thinning changes land in `engine/stroke_geometry.dart` (the single source for screen+export, §2/F1 of the parent plan), so the fix survives the P0 relocation and exports match the screen. Do not fork logic into `ink_painters.dart` only. +2. **No hardcoded feel constants.** `thinning`, `size`, taper, and pressure-sensitivity are `PenConfig` fields with sane defaults — mirroring Saber's `StrokeOptions`-per-pen model. +3. **Native pen is additive + degrades gracefully.** The plugin enriches pointer events with barrel/inverted/tilt; if it is absent or returns nothing, the canvas behaves exactly as today (Flutter pressure still works). Never make drawing depend on the plugin. +4. **Device-gate the un-CI-testable.** Tilt/buttons/flicker can only be confirmed on the Surface; each ships behind a CI package + a manual checklist, never claimed "done" from a green analyze. + +### Decision Drivers +1. **D1 — Unblock the user's primary device today** (eraser-end + side-button + tilt are dead; pen feel needs a real size/pressure control). +2. **D2 — Don't derail the approved refactor** — every change feeds `engine/`+`input/`+`pen_config`, not the soon-retired widgets. +3. **D3 — Crisp, flicker-free zoom** without prematurely building the whole multi-resolution tiler (that's P0.5). + +### Viable options + +**W1 — configurable thinning/size** +- **Option A (CHOSEN): thread `thinning`/`size` as parameters from `PenConfig` through `buildStrokeOutline`/`buildStrokePath`; add `pressureSensitivity` (→thinning) + reuse existing `penWidth`/`highlighterWidth` (→size); Pen-settings sliders.** Pros: matches Saber (`StrokeOptions(size, thinning, …)` per pen); one source of truth; tiny diff. Cons: 2 signatures change + every caller. +- Option B: keep literals, expose only `penWidth`. Rejected — user explicitly says thinning 不能写死. + +**W2 — zoom flicker** +- **Option A (CHOSEN, now): double-buffer the page bitmap** — keep the last successfully-rendered page image painted underneath `PdfPageView` (or wrap in a tiny `RawImage` cache) so the async re-render never exposes a white frame; **then** the full R11 `page_tile` in P0.5 replaces it. Pros: kills the visible flicker immediately with a small, localized change; forward-compatible (becomes `page_tile`'s double-buffer). Cons: a stopgop that P0.5 supersedes. +- Option B: jump straight to the full `pdf/page_tile` + `page_tile_cache` now. Rejected for *now* — it's the P0.5 gate; pulling all of it forward derails P0. (We DO confirm the flicker root cause via systematic-debugging before coding either.) + +**W3 — native pen (tilt + buttons)** +- **Option A (CHOSEN): in-app Windows plugin in `windows/runner/` (pen_channel.cpp) subclassing the window proc to handle `WM_POINTER*`, call `GetPointerPenInfo`/`GetPointerPenInfoHistory` for `penFlags` (BARREL/INVERTED/ERASER) + `tiltX/tiltY`, key state by `pointerId`, forward to Dart via an `EventChannel('badnote/pen')`.** Dart `PenInputService` exposes the latest per-pointer pen state; `InputArbiter`/canvas reads it to set eraser + tilt. Pros: smallest footprint (mirrors existing `ocr_channel.cpp` pattern); no new pub package; full control of WM_POINTER. Cons: native C++ to maintain; CI-build-only, device-verify-only. +- Option B: standalone federated Flutter plugin package. Rejected — heavier scaffolding for a single-platform need; `ocr_channel.cpp` proves the in-runner pattern works here. +- Option C: wait for Flutter engine to deliver penFlags/tilt upstream. Rejected — unbounded; user blocked now. + +### Pre-mortem (3 scenarios) +1. **"Tilt/buttons still dead after the plugin ships."** Cause: WM_POINTER not reaching our handler, or Flutter's own `FlutterWindow` consumes the message first. *Prevention:* before writing the EventChannel, add a WM_POINTER **logging probe** in the window proc and confirm on-device that `GetPointerPenInfo` returns non-zero `penFlags`/tilt (systematic-debugging Phase-1 evidence at the component boundary). Only then wire the channel. +2. **"Zoom flicker fix made scrolling janky / doubled memory."** Cause: keeping a full-res second bitmap per page. *Prevention:* hold exactly ONE last-good image for the *current* page only; drop it on page change; measure frame cost on the Surface before/after. +3. **"Width/thinning change broke export goldens."** Cause: only `ink_painters.dart` was updated, `stroke_geometry.dart` (export path) drifted. *Prevention:* change BOTH via the shared `kDefaultPenThinning`; default stays `0.85` (M4) so the existing golden is unchanged; add a unit test that both builders read identical thinning for the same `PenConfig`. If a golden must be regenerated, name + commit the new baseline explicitly. +4. **"Eraser end doesn't erase on a direct pen-down (no hover)."** Cause: correctness was hung on hover-precedence instead of the M1 native ordering. *Prevention:* re-grounded above on observer-before-`HandleTopLevelWindowProc`; **on-device test: tap the eraser end straight onto the page with no prior hover — first contact must erase, not draw.** + +--- + +## 2. Work items (file-level) + +### W1 — Configurable pen width + pressure sensitivity (Saber-style) +**Modify:** +- `lib/editor/input/pen_config.dart` **[MODIFY]** — add `double pressureSensitivity` (maps to perfect_freehand `thinning`; range `[0,1]`, **default `0.85` = the current live value, M4**, so existing stroke feel and the export golden are preserved; `0` = constant width). Keep `penWidth`/`highlighterWidth` as size. Add to `copyWith`/`toJson`/`fromJson`/`==`/`hashCode` + `PenConfigController.setPressureSensitivity` (clamped). Additive persisted field (default-filled on missing key — no migration). +- `lib/editor/engine/stroke_geometry.dart` **[MODIFY]** — define `const double kDefaultPenThinning = 0.85;` (M4 — NOT 0.6; preserves goldens). `buildStrokeOutline(..., {required bool isComplete, double thinning = kDefaultPenThinning})`; remove the `0.85` literal in favour of the named const. Highlighter still forces `0.0`. +- `lib/editor/canvas/ink_painters.dart` **[MODIFY]** — `buildStrokePath(..., {required bool isComplete, double thinning = kDefaultPenThinning})`; import `kDefaultPenThinning` from `stroke_geometry.dart` so screen+export share ONE default and can never diverge. +- `lib/editor/canvas/pen_canvas.dart` **[MODIFY]** — accept a `thinning` field on `PenCanvas`; pass it to the painters (the painters need the value at paint time → pass via the painter constructors `StaticInkPainter`/`LiveInkPainter`, add `shouldRepaint` check on `thinning`). +- `lib/editor/canvas/pen_editor_screen.dart` **[MODIFY]** — feed `_penConfig?.value.pressureSensitivity ?? kDefaultPenThinning` into `PenCanvas.thinning`. +- `lib/editor/ui/pen_settings_page.dart` **[MODIFY]** — add a **pen size** slider (drives `penWidth`, e.g. `0.002–0.02`), a **highlighter size** slider, and a **pressure sensitivity** slider (drives `pressureSensitivity` `0–1`). Live-preview stroke swatch optional. + +**Acceptance:** changing pressure-sensitivity to 0 yields constant-width strokes; to ~0.8 makes light/hard press sweep width visibly; pen-size slider changes base width; values persist across restart; a unit test asserts `buildStrokeOutline` and `buildStrokePath` use the same thinning for a given config; export golden still matches screen (R7). + +### W2 — Zoom flicker — **ROOT-CAUSE FIRST, fix is probe-gated (M3)** +The flicker fix is NOT pre-committed to a double-buffer. The Architect notes the page is rendered **once** and matrix-scaled (`pen_editor_screen.dart:466–474`), so the cause may be **R11-class** (a re-raster swap / matrix blur on zoom-settle) rather than an async white-gap — and the right fix differs per cause. + +**Step 1 — Phase-1 probe (systematic-debugging, MANDATORY before any fix):** instrument one zoom on the Surface and determine which of these the "闪一下" is: +- (a) **async white-gap** — `PdfPageView` blanks for a frame while it re-rasterizes at the new scale; OR +- (b) **re-raster swap** — pdfrx renders a fresh higher-DPI bitmap and swaps it in (brief tone/size pop) = R11 territory; OR +- (c) **rebuild flash** — a `setState`/`_needsCenter` post-frame callback (`pen_editor_screen.dart:441–448`) or `ValueKey(_pageIndex)` re-mounts the subtree. +Record the verdict + a frame capture in the perf-results doc. + +**Step 2 — fix chosen by cause:** +- If **(a)**: keep the last good `ui.Image` painted as an underlay for the **current page only**, swapped atomically when the new render is ready (the seed of P0.5 `page_tile`'s double-buffer — link the TODO so it's replaced, not duplicated). **Caveat (Architect): this can race pdfrx's own internal raster cache** — verify the underlay sits *below* `PdfPageView` and is only shown while the live raster is absent. +- If **(b)**: this is R11 — do NOT build a bespoke fix; the cheap interim is "keep the page painted across the swap" and the real fix is the P0.5 `page_tile` DPI-on-settle. Defer, note in the roadmap. +- If **(c)**: remove the spurious rebuild (gate `_needsCenter`, avoid re-keying on zoom) — cheapest of all. + +**Acceptance:** root cause documented; pinch-zoom on the Surface shows **no flash**; frame cost unchanged (≤16.6ms median) before/after; if deferred to P0.5, that decision is recorded with evidence (not silently dropped). + +### W3 — Native Windows pen plugin (tilt + barrel/eraser → buttons) + +**Hook point (MUST-FIX M1):** `FlutterWindow::MessageHandler` calls `flutter_controller_->HandleTopLevelWindowProc(...)` **first** and returns early when Flutter handles the message (`windows/runner/flutter_window.cpp:56–64`). Flutter 3.44 itself consumes WM_POINTER to synthesize stylus events, so a `switch` *after* that call (line 66) — and the `ocr_channel` registration site at `:29` — **never see consumed pen messages**. The observer therefore reads pen info at the **TOP of `MessageHandler`, before** `HandleTopLevelWindowProc`, observing without consuming (do not `return` a result; fall through so Flutter still processes its event). + +**Correlation (MUST-FIX M2):** do NOT build `Map` and join it to Flutter's `event.pointer` — they are **different id spaces**. Only one pen is active at a time, so latch a **single "current stylus hardware state"** natively, update it on every observed WM_POINTER (incl. hover/`WM_POINTERENTER`), and read it at the decision points in Dart. + +**Why the eraser decision is NOT racy (re-grounded per Critic #2 — do NOT rely on hover-precedence):** the guarantee is the **M1 native ordering**, not "hover precedes down." A pen that contacts the screen directly with no hover dwell delivers `WM_POINTERDOWN` as its first message. Our observer runs at the **top of `MessageHandler`** (M1), so it latches that down's `penFlags` **before** `HandleTopLevelWindowProc` synthesizes the corresponding Flutter pointer-down — therefore when Dart's `_onPointerDown` reads `PenInputService.current`, the latch already reflects this exact contact. The EventChannel push is async, but the **native latch is updated synchronously in the same window-proc pass that precedes Flutter's event**; so the Dart service must source the eraser flag from a value guaranteed fresh by that ordering (i.e. the channel delivers the down-flags before/with the Dart down event because both originate from the same WM_POINTERDOWN, observer-first). Tilt is per-point and tolerant of one-frame lag. **This must be proven on-device with a pen-down-without-hover test (see pre-mortem #4).** + +**Threading (Architect):** the observer runs on the **platform (UI) thread** inside the window proc. Do **NOT** copy `ocr_channel.cpp`'s MTA worker-thread model (`ocr_channel.cpp:73–118`) — that pattern is for the long-running OCR call, wrong for low-latency per-event pen state. Latch + post to the channel sink directly on the platform thread. + +**Add/modify:** +- `windows/runner/pen_channel.{h,cpp}` **[ADD]** — `ObservePenMessage(message, wparam, lparam)`: if `message ∈ {WM_POINTERENTER, WM_POINTERDOWN, WM_POINTERUPDATE, WM_POINTERUP}`, `GET_POINTERID_WPARAM(wparam)` → `GetPointerType` → if `PT_PEN`, `GetPointerPenInfo(id, &POINTER_PEN_INFO)`; read `penFlags` (`PEN_FLAG_BARREL`, `PEN_FLAG_INVERTED`, `PEN_FLAG_ERASER`) + `tiltX`/`tiltY`. Latch into a native singleton AND push `{flags, tiltX, tiltY}` over `EventChannel('badnote/pen')`. Returns void; never consumes. +- `windows/runner/flutter_window.cpp` **[MODIFY]** — call `ObservePenMessage(message, wparam, lparam)` at the **top of `MessageHandler` (before** the `HandleTopLevelWindowProc` block, M1); register the channel alongside `RegisterOcrChannel` in `OnCreate` (`:29`). +- `lib/editor/input/pen_input_service.dart` **[ADD]** — listens to `EventChannel('badnote/pen')`; holds the **latest single** `PenHardwareState{barrel, inverted, eraser, tiltX, tiltY}` (not keyed by pointerId, M2). No-op / empty on non-Windows or a silent channel. +- `lib/editor/canvas/pen_stroke.dart` **[MODIFY] (Critic #1 — wrong-model fix):** the live canvas captures `PenPoint{x, y, pressure}` (`pen_stroke.dart:15-21`) which has **no `tilt` field** — `EditorPoint.tilt` lives in the *separate* engine model the live widget does not use. Add `double? tilt` to `PenPoint` now, populate it at capture, and map it through `EditorStroke.fromPenStroke` → `EditorPoint.tilt` (SF1) so tilt is lossless end-to-end on the live path. (When the canvas migrates to `EditorStroke` in P0, `PenPoint` retires and this collapses to `EditorPoint.tilt` directly.) +- `lib/editor/canvas/pen_canvas.dart` **[MODIFY]** — replace the dead `_isEraserSignal` (`pen_canvas.dart:134–136`, `kSecondaryButton||invertedStylus` never fires on Windows): on hover/down read `PenInputService.current` and resolve eraser/undo/etc. through the configured `PenConfig.sideButton`/`eraserEnd` mapping; stash `tiltX/Y` into the captured `PenPoint.tilt` (added above). +- `lib/editor/input/pen_config.dart` — `sideButton`/`eraserEnd` `PenButtonAction` already exist; the canvas now actually consults them for plugin-delivered flags. + +**Phase-1 probe before wiring (pre-mortem #1):** first land ONLY a WM_POINTER logging line in the observer and confirm on the Surface that `GetPointerPenInfo` returns non-zero `penFlags`/tilt. Only then add the EventChannel + Dart wiring. + +**Acceptance (device-gated):** on the Surface, the eraser end erases; the barrel button performs its mapped `PenButtonAction`; remapping side-button→undo makes the barrel undo; the diagnostic readout shows non-zero `tilt`; with the channel silent the app still draws with pressure (graceful degradation). CI builds the Windows package green; correctness confirmed only on-device + recorded with commit hash. + +--- + +## 3. Sequencing + +1. **W1** (pure Dart, CI-testable, low risk) — ship first; unblocks "pen feel" immediately. +2. **W2 investigate → minimal fix** — systematic-debugging Phase 1 evidence, then the double-buffer. +3. **W3 native plugin** — biggest/native; (a) WM_POINTER logging probe → device-confirm `GetPointerPenInfo` returns flags+tilt; (b) EventChannel + Dart service; (c) wire eraser/tilt + `PenConfig` mappings. +4. **W4** — resume the parent roadmap P0 (task #7) → P0.5. W1's `stroke_geometry` change and W3's `pen_input_service` are written to land in `engine/`+`input/` so the P0 relocation absorbs them rather than re-doing them. + +Each of W1/W2/W3 is an independently shippable CI package; W2/W3 carry a manual Surface checklist before "done." + +--- + +## 4. ADR +- **Decision:** Pull the **width+pressure-sensitivity** slice of F5 forward (de-hardcode `thinning` in the canonical geometry), fix the zoom flicker with a **double-buffer stopgap** that seeds the P0.5 `page_tile`, and add a **native Windows pen plugin** (`pen_channel.cpp` + `EventChannel('badnote/pen')`) to recover barrel/eraser/tilt that the Flutter 3.44 engine drops — all feeding the approved `engine/`/`input/` seams, then resume the parent roadmap. +- **Drivers:** D1 unblock primary device today; D2 don't derail the refactor; D3 flicker-free zoom without prematurely building the full tiler. +- **Alternatives:** literals-only width (rejected — user), full `page_tile` now (deferred to P0.5), federated plugin package (rejected — `ocr_channel` in-runner pattern suffices), wait-for-upstream (rejected — blocked now). +- **Consequences:** one new native file (Windows-only, device-gated); a temporary page double-buffer superseded by P0.5; `PenConfig` gains `pressureSensitivity`; `thinning` defaults centralized as `kDefaultPenThinning`. +- **Follow-ups:** W2 stopgap deleted when `page_tile` lands; W3 eraser/tilt wiring moves into `input_arbiter` during P0; confirm `GetPointerPenInfo` tilt units + sign on-device. diff --git a/lib/editor/canvas/ink_painters.dart b/lib/editor/canvas/ink_painters.dart index 23b5e95..4096e01 100644 --- a/lib/editor/canvas/ink_painters.dart +++ b/lib/editor/canvas/ink_painters.dart @@ -7,13 +7,21 @@ import 'package:flutter/material.dart'; import 'package:perfect_freehand/perfect_freehand.dart' as pf; +import '../engine/stroke_geometry.dart' show kDefaultPenThinning; import 'pen_stroke.dart'; /// Builds a filled outline [Path] for one stroke (already scaled to pixels). /// /// [pageSize] maps normalized coords to pixels. [isComplete] should be false -/// for the in-progress live stroke so freehand tapers correctly. -Path buildStrokePath(PenStroke stroke, Size pageSize, {required bool isComplete}) { +/// for the in-progress live stroke so freehand tapers correctly. [thinning] is +/// the pressure→width response (shared default [kDefaultPenThinning]); the +/// SAME value drives the export path so screen and PDF never diverge. +Path buildStrokePath( + PenStroke stroke, + Size pageSize, { + required bool isComplete, + double thinning = kDefaultPenThinning, +}) { final pixelWidth = stroke.width * pageSize.width; final hasRealPressure = stroke.points.any((p) => p.pressure != null); @@ -21,7 +29,7 @@ Path buildStrokePath(PenStroke stroke, Size pageSize, {required bool isComplete} final pfPoints = stroke.points .map( - (p) => pf.Point( + (p) => pf.PointVector( p.x * pageSize.width, p.y * pageSize.height, p.pressure ?? 0.5, @@ -31,23 +39,27 @@ Path buildStrokePath(PenStroke stroke, Size pageSize, {required bool isComplete} final outline = pf.getStroke( pfPoints, - size: pixelWidth, - // Highlighter keeps a constant width (no thinning); pen thins like the - // existing ink_canvas (_drawFreehand uses 0.7). - thinning: isHighlighter ? 0.0 : 0.7, - smoothing: 0.5, - streamline: 0.5, - // Real stylus pressure → don't simulate; no pressure → let freehand fake - // it based on velocity (matches ink_canvas behavior). - simulatePressure: !hasRealPressure && !isHighlighter, - isComplete: isComplete, + options: pf.StrokeOptions( + size: pixelWidth, + // Highlighter keeps a constant width (no thinning); pen uses the + // configurable [thinning] so real Surface-Pen pressure changes width. + thinning: isHighlighter ? 0.0 : thinning, + smoothing: 0.5, + streamline: 0.5, + // Real stylus pressure → don't simulate; no pressure → let freehand fake + // it based on velocity. (perfect_freehand 2.x honors REAL pressure when + // simulatePressure is false — 1.0.4 ignored it, which made width + // unresponsive to pen force.) + simulatePressure: !hasRealPressure && !isHighlighter, + isComplete: isComplete, + ), ); final path = Path(); if (outline.isEmpty) return path; - path.moveTo(outline.first.x, outline.first.y); + path.moveTo(outline.first.dx, outline.first.dy); for (var i = 1; i < outline.length; i++) { - path.lineTo(outline[i].x, outline[i].y); + path.lineTo(outline[i].dx, outline[i].dy); } path.close(); return path; @@ -56,15 +68,23 @@ Path buildStrokePath(PenStroke stroke, Size pageSize, {required bool isComplete} /// Paints all committed strokes for the page. Repaints only when the stroke /// list identity or page size changes (kept behind a RepaintBoundary). class StaticInkPainter extends CustomPainter { - StaticInkPainter({required this.strokes, required this.pageSize}); + StaticInkPainter({ + required this.strokes, + required this.pageSize, + this.thinning = kDefaultPenThinning, + }); final List strokes; final Size pageSize; + /// Pressure→width response shared with the live/export paths. + final double thinning; + @override void paint(Canvas canvas, Size size) { for (final stroke in strokes) { - final path = buildStrokePath(stroke, pageSize, isComplete: true); + final path = + buildStrokePath(stroke, pageSize, isComplete: true, thinning: thinning); if (path.getBounds().isEmpty) continue; canvas.drawPath( path, @@ -80,23 +100,32 @@ class StaticInkPainter extends CustomPainter { bool shouldRepaint(StaticInkPainter old) => !identical(old.strokes, strokes) || old.strokes.length != strokes.length || - old.pageSize != pageSize; + old.pageSize != pageSize || + old.thinning != thinning; } /// Paints just the in-progress stroke (the live layer), kept behind its own /// RepaintBoundary so committed strokes don't repaint on every move. class LiveInkPainter extends CustomPainter { - LiveInkPainter({required this.stroke, required this.pageSize}); + LiveInkPainter({ + required this.stroke, + required this.pageSize, + this.thinning = kDefaultPenThinning, + }); /// Current in-progress stroke, or null when nothing is being drawn. final PenStroke? stroke; final Size pageSize; + /// Pressure→width response shared with the static/export paths. + final double thinning; + @override void paint(Canvas canvas, Size size) { final s = stroke; if (s == null || s.points.isEmpty) return; - final path = buildStrokePath(s, pageSize, isComplete: false); + final path = + buildStrokePath(s, pageSize, isComplete: false, thinning: thinning); if (path.getBounds().isEmpty) return; canvas.drawPath( path, @@ -109,5 +138,7 @@ class LiveInkPainter extends CustomPainter { @override bool shouldRepaint(LiveInkPainter old) => - !identical(old.stroke, stroke) || old.pageSize != pageSize; + !identical(old.stroke, stroke) || + old.pageSize != pageSize || + old.thinning != thinning; } diff --git a/lib/editor/canvas/pen_canvas.dart b/lib/editor/canvas/pen_canvas.dart index ce0d36b..592346b 100644 --- a/lib/editor/canvas/pen_canvas.dart +++ b/lib/editor/canvas/pen_canvas.dart @@ -21,6 +21,9 @@ import 'package:flutter/gestures.dart'; import 'package:flutter/material.dart'; +import '../engine/stroke_geometry.dart' show kDefaultPenThinning; +import '../input/pen_config.dart'; +import '../input/pen_input_service.dart'; import 'ink_painters.dart'; import 'pen_stroke.dart'; @@ -43,6 +46,10 @@ class PenCanvas extends StatefulWidget { this.minScale = 0.5, this.maxScale = 8.0, this.onPenDebug, + this.thinning = kDefaultPenThinning, + this.sideButtonAction = PenButtonAction.eraser, + this.eraserEndAction = PenButtonAction.eraser, + this.onPenButtonAction, }); /// Debug hook: called with a readout of the latest pen event @@ -82,6 +89,20 @@ class PenCanvas extends StatefulWidget { final double minScale; final double maxScale; + /// perfect_freehand pressure→width response, from `PenConfig.pressureSensitivity`. + final double thinning; + + /// Configured action for the pen's side barrel button (W3 — resolved against + /// the native pen plugin's flags on Windows). + final PenButtonAction sideButtonAction; + + /// Configured action for the pen's eraser/inverted end (W3). + final PenButtonAction eraserEndAction; + + /// Fired (edge-triggered) when a hardware pen button mapped to a non-eraser + /// action (undo / toggleTool) is pressed. + final void Function(PenButtonAction action)? onPenButtonAction; + @override State createState() => _PenCanvasState(); } @@ -130,15 +151,79 @@ class _PenCanvasState extends State { return null; } - /// The eraser signal: barrel/secondary button held, or an inverted stylus. - bool _isEraserSignal(PointerEvent event) => - event.buttons == kSecondaryButton || - event.kind == PointerDeviceKind.invertedStylus; + /// The eraser signal. Two sources, ORed: + /// 1. Flutter-native: secondary button held or an inverted stylus (works on + /// desktop / platforms that surface these). + /// 2. Windows native pen plugin: barrel / inverted / eraser flags that + /// Flutter 3.44 drops, mapped through the configured side-button / + /// eraser-end actions (W3). Level-triggered, so holding the button keeps + /// erasing — correct for an eraser. + bool _isEraserSignal(PointerEvent event) { + if (event.buttons == kSecondaryButton || + event.kind == PointerDeviceKind.invertedStylus) { + return true; + } + final hw = PenInputService.instance; + if (hw.isActive) { + final s = hw.current; + if ((s.inverted || s.eraser) && + widget.eraserEndAction == PenButtonAction.eraser) { + return true; + } + if (s.barrel && widget.sideButtonAction == PenButtonAction.eraser) { + return true; + } + } + return false; + } + + /// Resolve the currently-active configured action from the native pen flags + /// (eraser-end takes precedence over the side button when both are set). + PenButtonAction _activeHwAction() { + final hw = PenInputService.instance; + if (!hw.isActive) return PenButtonAction.none; + final s = hw.current; + if (s.inverted || s.eraser) return widget.eraserEndAction; + if (s.barrel) return widget.sideButtonAction; + return PenButtonAction.none; + } + + /// Last hardware action seen, for rising-edge detection of undo/toggleTool. + PenButtonAction _lastHwAction = PenButtonAction.none; + + /// Edge-triggered dispatch of non-eraser button actions (undo / toggleTool). + /// Eraser is handled level-triggered by [_isEraserSignal]; pan suppresses + /// drawing via [_shouldDraw]. + void _dispatchHwButtonActions() { + final action = _activeHwAction(); + if (action == _lastHwAction) return; + _lastHwAction = action; + if (action == PenButtonAction.undo || + action == PenButtonAction.toggleTool) { + widget.onPenButtonAction?.call(action); + } + } + + /// True while a hardware button mapped to `pan` is held (suppresses drawing + /// so the InteractiveViewer pans instead). + bool get _hwPanActive => _activeHwAction() == PenButtonAction.pan; + + /// Pen tilt magnitude (degrees) for a stylus event, or null when unavailable. + double? _tiltFor(PointerEvent event) { + if (!_isStylus(event.kind)) return null; + final hw = PenInputService.instance; + if (!hw.isActive) return null; + final t = hw.current.tiltMagnitude; + return t == 0 ? null : t; + } /// Decide whether the gesture currently forming should DRAW. /// True iff exactly one active pointer AND (stylus OR finger-drawing on). bool _shouldDraw(PointerDeviceKind kind) { if (_activePointers.length != 1) return false; + // A hardware pen button mapped to `pan` suppresses drawing so the + // InteractiveViewer pans instead. + if (_hwPanActive) return false; if (_isStylus(kind)) return true; if (kind == PointerDeviceKind.mouse) return true; if (kind == PointerDeviceKind.touch) return _fingerDrawingEnabled; @@ -149,7 +234,8 @@ class _PenCanvasState extends State { /// Map a global pointer position into normalized page coords using the /// shared transform (inverse) and this widget's geometry. - PenPoint? _toNormalized(Offset globalPosition, double? pressure) { + PenPoint? _toNormalized(Offset globalPosition, double? pressure, + {double? tilt}) { final box = context.findRenderObject() as RenderBox?; if (box == null) return null; final local = box.globalToLocal(globalPosition); @@ -159,7 +245,7 @@ class _PenCanvasState extends State { final nx = scene.dx / widget.pageSize.width; final ny = scene.dy / widget.pageSize.height; - return PenPoint(nx, ny, pressure); + return PenPoint(nx, ny, pressure, tilt: tilt); } // --- Stroke lifecycle ----------------------------------------------------- @@ -167,7 +253,8 @@ class _PenCanvasState extends State { void _startStroke(PointerDownEvent event) { _drawPointer = event.pointer; _livePoints.clear(); - final p = _toNormalized(event.position, _normalizedPressure(event)); + final p = _toNormalized(event.position, _normalizedPressure(event), + tilt: _tiltFor(event)); if (p != null) _livePoints.add(p); if (_eraserActive || widget.tool == CanvasTool.eraser) { @@ -180,7 +267,8 @@ class _PenCanvasState extends State { } void _extendStroke(PointerMoveEvent event) { - final p = _toNormalized(event.position, _normalizedPressure(event)); + final p = _toNormalized(event.position, _normalizedPressure(event), + tilt: _tiltFor(event)); if (p == null) return; if (_eraserActive || widget.tool == CanvasTool.eraser) { @@ -255,18 +343,28 @@ class _PenCanvasState extends State { // --- Listener callbacks --------------------------------------------------- + /// Highest NORMALIZED pressure seen since the diagnostic was last reset — + /// makes "does pressure actually vary?" unambiguous on the readout. + double _peakNorm = 0; + void _emitPenDebug(PointerEvent event) { final cb = widget.onPenDebug; if (cb == null) return; - cb('${event.kind.name} p=${event.pressure.toStringAsFixed(3)} ' - 'min=${event.pressureMin.toStringAsFixed(2)} ' - 'max=${event.pressureMax.toStringAsFixed(2)} ' - 'tilt=${event.tilt.toStringAsFixed(2)}'); + final norm = _normalizedPressure(event); + if (norm != null && norm > _peakNorm) _peakNorm = norm; + cb('${event.kind.name} raw=${event.pressure.toStringAsFixed(1)}' + '/${event.pressureMax.toStringAsFixed(0)} ' + 'norm=${norm?.toStringAsFixed(3) ?? "null"} ' + 'peak=${_peakNorm.toStringAsFixed(3)} ' + 'btn=${event.buttons} tilt=${event.tilt.toStringAsFixed(2)}'); } void _onPointerHover(PointerHoverEvent event) { if (_isStylus(event.kind)) { _emitPenDebug(event); + // Fire edge-triggered button actions (undo / toggleTool) on hover so a + // mapped barrel press works without first touching down. + _dispatchHwButtonActions(); // Detect eraser (barrel button / inverted) while hovering. _eraserActive = _isEraserSignal(event); } @@ -274,7 +372,13 @@ class _PenCanvasState extends State { void _onPointerDown(PointerDownEvent event) { if (event.kind == PointerDeviceKind.trackpad) return; - if (_isStylus(event.kind)) _emitPenDebug(event); + if (_isStylus(event.kind)) { + _emitPenDebug(event); + // Fire edge-triggered button actions for a direct pen-down (no prior + // hover); the native observer latched this contact's flags before Flutter + // synthesized this event (plan M1/M2). + _dispatchHwButtonActions(); + } _activePointers[event.pointer] = event.kind; @@ -343,8 +447,15 @@ class _PenCanvasState extends State { height: widget.pageSize.height, child: Stack( children: [ - // PDF page bitmap. - Positioned.fill(child: widget.pageWidget), + // PDF page bitmap. Wrapped in its own RepaintBoundary (W2) so the + // per-move live-ink repaints and the static-ink repaints never + // mark the page's raster layer dirty — isolating it from + // ink-driven repaints. (The definitive crisp-on-zoom / no-flash + // fix is the P0.5 page_tile DPI-on-settle double-buffer; this + // boundary is the safe, non-regressive interim per plan M3.) + Positioned.fill( + child: RepaintBoundary(child: widget.pageWidget), + ), // Committed ink (static layer, isolated repaint). Positioned.fill( child: RepaintBoundary( @@ -352,6 +463,7 @@ class _PenCanvasState extends State { painter: StaticInkPainter( strokes: widget.strokes, pageSize: widget.pageSize, + thinning: widget.thinning, ), ), ), @@ -363,6 +475,7 @@ class _PenCanvasState extends State { painter: LiveInkPainter( stroke: _liveStroke, pageSize: widget.pageSize, + thinning: widget.thinning, ), ), ), diff --git a/lib/editor/canvas/pen_editor_screen.dart b/lib/editor/canvas/pen_editor_screen.dart index 42d2838..764580f 100644 --- a/lib/editor/canvas/pen_editor_screen.dart +++ b/lib/editor/canvas/pen_editor_screen.dart @@ -9,9 +9,11 @@ import 'package:flutter/material.dart'; import 'package:pdfrx/pdfrx.dart'; import '../../services/database_service.dart'; +import '../engine/stroke_geometry.dart' show kDefaultPenThinning; import '../engine/stroke_model.dart'; import '../engine/undo_stack.dart'; import '../input/pen_config.dart'; +import '../input/pen_input_service.dart'; import '../persistence/editor_repository.dart'; import '../persistence/save_scheduler.dart'; import '../ui/pen_settings_page.dart'; @@ -98,8 +100,8 @@ class _PenEditorScreenState extends State { Color _color = Colors.black; bool _allowFingerDrawing = false; - /// Pen width as a fraction of page width. - static const double _penWidthFraction = 0.004; + /// Pen width as a fraction of page width (base; pressure thins it down). + static const double _penWidthFraction = 0.006; static const double _highlighterWidthFraction = 0.02; static const List _palette = [ @@ -114,6 +116,9 @@ class _PenEditorScreenState extends State { void initState() { super.initState(); _documentId = _documentIdFromPath(widget.pdfPath); + // Begin listening to the native Windows pen plugin (barrel/eraser/tilt). + // No-op on platforms without the plugin (W3). + PenInputService.instance.start(); _initPersistence(); _initPenConfig(); _open(); @@ -125,6 +130,9 @@ class _PenEditorScreenState extends State { controller.dispose(); return; } + // Rebuild the editor when pen settings change (width, pressure + // sensitivity, button mappings) so the live canvas reflects them. + controller.addListener(_onPenConfigChanged); setState(() { _penConfig = controller; // Adopt the persisted finger-drawing preference as the initial local @@ -134,6 +142,10 @@ class _PenEditorScreenState extends State { }); } + void _onPenConfigChanged() { + if (mounted) setState(() {}); + } + Future _initPersistence() async { final service = await DatabaseService.getInstance(); if (!mounted) return; @@ -159,7 +171,7 @@ class _PenEditorScreenState extends State { loaded[pageIndex] = entry.value .map((es) => PenStroke( points: es.points - .map((ep) => PenPoint(ep.x, ep.y, ep.pressure)) + .map((ep) => PenPoint(ep.x, ep.y, ep.pressure, tilt: ep.tilt)) .toList(), color: es.color, width: es.width, @@ -211,6 +223,7 @@ class _PenEditorScreenState extends State { _document?.dispose(); _transform.dispose(); _penConfig?.dispose(); + PenInputService.instance.stop(); super.dispose(); } @@ -308,6 +321,33 @@ class _PenEditorScreenState extends State { _schedulePageSave(_pageIndex, List.of(snapshot)); } + /// Cycle pen → highlighter → eraser → pen (for the toggleTool button action). + void _cycleTool() { + setState(() { + _tool = switch (_tool) { + CanvasTool.pen => CanvasTool.highlighter, + CanvasTool.highlighter => CanvasTool.eraser, + CanvasTool.eraser => CanvasTool.pen, + }; + }); + } + + /// Handle a hardware pen-button action delivered by [PenCanvas] (W3). + /// `eraser` and `pan` are handled inside the canvas; here we map the + /// edge-triggered ones. + void _handlePenButtonAction(PenButtonAction action) { + switch (action) { + case PenButtonAction.undo: + _performUndo(); + case PenButtonAction.toggleTool: + _cycleTool(); + case PenButtonAction.eraser: + case PenButtonAction.pan: + case PenButtonAction.none: + break; + } + } + /// Toggle finger-drawing, keeping the local state and the persisted config /// (when loaded) in sync. void _toggleFingerDrawing() { @@ -457,6 +497,13 @@ class _PenEditorScreenState extends State { ? (_penConfig?.value.highlighterWidth ?? _highlighterWidthFraction) : (_penConfig?.value.penWidth ?? _penWidthFraction), + thinning: + _penConfig?.value.pressureSensitivity ?? kDefaultPenThinning, + sideButtonAction: + _penConfig?.value.sideButton ?? PenButtonAction.eraser, + eraserEndAction: + _penConfig?.value.eraserEnd ?? PenButtonAction.eraser, + onPenButtonAction: _handlePenButtonAction, allowFingerDrawing: _allowFingerDrawing, onPenDebug: _showPenDebug ? (s) => setState(() => _penDebug = s) diff --git a/lib/editor/canvas/pen_stroke.dart b/lib/editor/canvas/pen_stroke.dart index 358d9a2..319bdb6 100644 --- a/lib/editor/canvas/pen_stroke.dart +++ b/lib/editor/canvas/pen_stroke.dart @@ -11,13 +11,17 @@ import 'package:flutter/foundation.dart'; /// [x]/[y] are normalized to the page rectangle ([0,1]). /// [pressure] is the normalized stylus pressure ([0,1]) or null when the /// device reported no usable pressure (perfect_freehand then simulates it). +/// [tilt] is the pen tilt magnitude in degrees (0 = perpendicular), or null +/// when unavailable. On Windows it is sourced from the native pen plugin +/// (`badnote/pen`) since Flutter 3.44 does not surface tilt itself. @immutable class PenPoint { - const PenPoint(this.x, this.y, this.pressure); + const PenPoint(this.x, this.y, this.pressure, {this.tilt}); final double x; final double y; final double? pressure; + final double? tilt; } /// Which kind of mark a stroke is. diff --git a/lib/editor/engine/stroke_geometry.dart b/lib/editor/engine/stroke_geometry.dart index bc1b0ad..13351d2 100644 --- a/lib/editor/engine/stroke_geometry.dart +++ b/lib/editor/engine/stroke_geometry.dart @@ -13,18 +13,30 @@ import 'package:perfect_freehand/perfect_freehand.dart' as pf; import 'stroke_model.dart'; +/// Canonical default for perfect_freehand's `thinning` (how strongly pressure +/// modulates stroke width). The SINGLE source of truth shared by the on-screen +/// painter ([buildStrokeOutline] here and `ink_painters.buildStrokePath`) and +/// the PDF export path, so screen and export can never diverge. `0.85` = +/// pressure visibly sweeps width; preserves the existing feel + export golden. +/// Overridable per-stroke via [PenConfig.pressureSensitivity]. +const double kDefaultPenThinning = 0.85; + /// Builds a closed, fillable outline [Path] for one [stroke], scaled into the /// pixel space of [pageSize] (which maps normalized [0,1] coords to pixels). /// /// [isComplete] should be false for the in-progress live stroke so freehand /// tapers the trailing end correctly, and true for committed strokes. /// +/// [thinning] is perfect_freehand's pressure→width response (see +/// [kDefaultPenThinning]); highlighter always forces `0.0` (constant width). +/// /// Returns an empty [Path] when the stroke has no points (or freehand produces /// no outline). Path buildStrokeOutline( EditorStroke stroke, Size pageSize, { required bool isComplete, + double thinning = kDefaultPenThinning, }) { final path = Path(); if (stroke.points.isEmpty) return path; @@ -36,7 +48,7 @@ Path buildStrokeOutline( final pfPoints = stroke.points .map( - (p) => pf.Point( + (p) => pf.PointVector( p.x * pageSize.width, p.y * pageSize.height, p.pressure ?? 0.5, @@ -46,22 +58,25 @@ Path buildStrokeOutline( final outline = pf.getStroke( pfPoints, - size: pixelWidth, - // Highlighter keeps a constant width (no thinning); pen thins (0.7), - // matching the live recipe. - thinning: isHighlighter ? 0.0 : 0.7, - smoothing: 0.5, - streamline: 0.5, - // Real stylus pressure -> don't simulate; no pressure -> let freehand fake - // it based on velocity (highlighter never simulates). - simulatePressure: !hasRealPressure && !isHighlighter, - isComplete: isComplete, + options: pf.StrokeOptions( + size: pixelWidth, + // Highlighter keeps a constant width (no thinning); pen uses the + // configurable [thinning] so Surface-Pen pressure changes width. + thinning: isHighlighter ? 0.0 : thinning, + smoothing: 0.5, + streamline: 0.5, + // Real stylus pressure -> don't simulate; no pressure -> let freehand + // fake it based on velocity (highlighter never simulates). perfect_freehand + // 2.x honors real pressure when simulatePressure is false. + simulatePressure: !hasRealPressure && !isHighlighter, + isComplete: isComplete, + ), ); if (outline.isEmpty) return path; - path.moveTo(outline.first.x, outline.first.y); + path.moveTo(outline.first.dx, outline.first.dy); for (var i = 1; i < outline.length; i++) { - path.lineTo(outline[i].x, outline[i].y); + path.lineTo(outline[i].dx, outline[i].dy); } path.close(); return path; diff --git a/lib/editor/engine/stroke_model.dart b/lib/editor/engine/stroke_model.dart index 3ffb722..e456055 100644 --- a/lib/editor/engine/stroke_model.dart +++ b/lib/editor/engine/stroke_model.dart @@ -103,12 +103,14 @@ abstract class EditorStroke with _$EditorStroke { // ---- Adapters ----------------------------------------------------------- - /// Adapts an in-memory live [PenStroke] (normalized, no tilt/timestamp/kind). + /// Adapts an in-memory live [PenStroke] (normalized; carries tilt when the + /// native pen plugin supplied it, else null; no timestamp/pointerDeviceKind). factory EditorStroke.fromPenStroke(PenStroke stroke, {String? id}) => EditorStroke( id: id ?? _uuid.v4(), points: stroke.points - .map((p) => EditorPoint(x: p.x, y: p.y, pressure: p.pressure)) + .map((p) => + EditorPoint(x: p.x, y: p.y, pressure: p.pressure, tilt: p.tilt)) .toList(), tool: switch (stroke.kind) { PenStrokeKind.pen => EditorTool.pen, diff --git a/lib/editor/input/pen_config.dart b/lib/editor/input/pen_config.dart index 88e6813..b514487 100644 --- a/lib/editor/input/pen_config.dart +++ b/lib/editor/input/pen_config.dart @@ -4,6 +4,8 @@ import 'dart:math'; import 'package:flutter/foundation.dart'; import 'package:shared_preferences/shared_preferences.dart'; +import '../engine/stroke_geometry.dart' show kDefaultPenThinning; + /// Action that can be triggered by a hardware pen button or the eraser end. enum PenButtonAction { none, @@ -25,10 +27,13 @@ class PenConfig { this.fingerDrawing = false, this.penWidth = 0.004, this.highlighterWidth = 0.02, + this.pressureSensitivity = kDefaultPenThinning, }) : assert(pressureGamma >= 0.3 && pressureGamma <= 3.0, 'pressureGamma must be in [0.3, 3.0]'), assert(palmRejectionMs >= 0.0 && palmRejectionMs <= 500.0, - 'palmRejectionMs must be in [0, 500]'); + 'palmRejectionMs must be in [0, 500]'), + assert(pressureSensitivity >= 0.0 && pressureSensitivity <= 1.0, + 'pressureSensitivity must be in [0, 1]'); /// Which action fires when the side barrel button is held. final PenButtonAction sideButton; @@ -54,6 +59,13 @@ class PenConfig { /// Highlighter stroke width as a fraction of the canvas width. final double highlighterWidth; + /// How strongly stylus pressure modulates stroke width — maps directly to + /// perfect_freehand's `thinning`. Range [0,1]; `0` = constant width, + /// higher = pressure sweeps width more (Saber's `StrokeOptions.thinning` + /// model). Default [kDefaultPenThinning] so the out-of-box feel and the + /// export golden are unchanged. + final double pressureSensitivity; + PenConfig copyWith({ PenButtonAction? sideButton, PenButtonAction? eraserEnd, @@ -62,6 +74,7 @@ class PenConfig { bool? fingerDrawing, double? penWidth, double? highlighterWidth, + double? pressureSensitivity, }) { return PenConfig( sideButton: sideButton ?? this.sideButton, @@ -71,6 +84,7 @@ class PenConfig { fingerDrawing: fingerDrawing ?? this.fingerDrawing, penWidth: penWidth ?? this.penWidth, highlighterWidth: highlighterWidth ?? this.highlighterWidth, + pressureSensitivity: pressureSensitivity ?? this.pressureSensitivity, ); } @@ -82,6 +96,7 @@ class PenConfig { 'fingerDrawing': fingerDrawing, 'penWidth': penWidth, 'highlighterWidth': highlighterWidth, + 'pressureSensitivity': pressureSensitivity, }; factory PenConfig.fromJson(Map json) { @@ -97,6 +112,8 @@ class PenConfig { fingerDrawing: json['fingerDrawing'] as bool? ?? false, penWidth: (json['penWidth'] as num?)?.toDouble() ?? 0.004, highlighterWidth: (json['highlighterWidth'] as num?)?.toDouble() ?? 0.02, + pressureSensitivity: + (json['pressureSensitivity'] as num?)?.toDouble() ?? kDefaultPenThinning, ); } @@ -111,7 +128,8 @@ class PenConfig { palmRejectionMs == other.palmRejectionMs && fingerDrawing == other.fingerDrawing && penWidth == other.penWidth && - highlighterWidth == other.highlighterWidth; + highlighterWidth == other.highlighterWidth && + pressureSensitivity == other.pressureSensitivity; @override int get hashCode => Object.hash( @@ -122,6 +140,7 @@ class PenConfig { fingerDrawing, penWidth, highlighterWidth, + pressureSensitivity, ); } @@ -215,4 +234,11 @@ class PenConfigController extends ChangeNotifier { notifyListeners(); await _persist(); } + + /// Sets [PenConfig.pressureSensitivity]. Clamped to [0, 1]. + Future setPressureSensitivity(double sensitivity) async { + _value = _value.copyWith(pressureSensitivity: sensitivity.clamp(0.0, 1.0)); + notifyListeners(); + await _persist(); + } } diff --git a/lib/editor/input/pen_input_service.dart b/lib/editor/input/pen_input_service.dart new file mode 100644 index 0000000..c2d5218 --- /dev/null +++ b/lib/editor/input/pen_input_service.dart @@ -0,0 +1,136 @@ +// lib/editor/input/pen_input_service.dart +// +// Dart side of the native Windows pen observer (`windows/runner/pen_channel.cpp`). +// +// WHY THIS EXISTS: Flutter 3.44 on Windows delivers stylus PRESSURE but drops +// the pen's barrel button, eraser/inverted end, and tilt (it does not map +// POINTER_PEN_FLAG_* into `PointerEvent.buttons`/`invertedStylus`/`tilt`). The +// native plugin observes WM_POINTER + GetPointerPenInfo and streams the missing +// hardware state over an EventChannel; this service latches the LATEST value. +// +// CORRELATION (plan M2): we do NOT key state by Win32 pointerId joined to +// Flutter's `event.pointer` — those are different id spaces. Only one pen is +// active at a time, so a single latched "current" state is correct. The native +// observer runs at the TOP of the window proc (BEFORE Flutter synthesizes its +// pointer event, plan M1), so by the time Dart's pointer-down handler reads +// [current], the latch already reflects that exact contact — no hover required. +// +// GRACEFUL DEGRADATION: on non-Windows (or if the channel is silent) the stream +// simply never emits / errors are swallowed, and [current] stays [PenHardwareState.empty] +// so the canvas falls back to its normal Flutter-pressure drawing. + +import 'dart:async'; + +import 'package:flutter/services.dart'; + +/// Latest hardware pen state delivered by the native observer. +class PenHardwareState { + const PenHardwareState({ + this.barrel = false, + this.inverted = false, + this.eraser = false, + this.tiltX = 0.0, + this.tiltY = 0.0, + }); + + /// Side barrel button held. + final bool barrel; + + /// Pen flipped to the inverted (eraser) end. + final bool inverted; + + /// Hardware eraser flag set. + final bool eraser; + + /// Tilt in degrees along X / Y ([-90, 90]); 0 = perpendicular. + final double tiltX; + final double tiltY; + + /// Combined tilt magnitude in degrees (for [PenPoint.tilt]). + double get tiltMagnitude { + final t = tiltX * tiltX + tiltY * tiltY; + return t <= 0 ? 0.0 : _sqrt(t); + } + + static const empty = PenHardwareState(); +} + +// Avoids importing dart:math for a single call. +double _sqrt(double v) { + if (v <= 0) return 0; + var x = v; + var last = 0.0; + // Newton's method; converges fast for the small (<=~127) magnitudes here. + for (var i = 0; i < 12 && x != last; i++) { + last = x; + x = 0.5 * (x + v / x); + } + return x; +} + +/// Latches the most recent [PenHardwareState] streamed by the native pen plugin. +/// +/// Use the singleton [PenInputService.instance]. Call [start] once (e.g. in the +/// editor's `initState`) and [stop] on dispose. +class PenInputService { + PenInputService._(); + + /// Process-wide singleton (one physical pen). + static final PenInputService instance = PenInputService._(); + + /// Must match the native `EventChannel` name in `pen_channel.cpp`. + static const EventChannel _channel = EventChannel('badnote/pen'); + + StreamSubscription? _sub; + PenHardwareState _current = PenHardwareState.empty; + + /// The latest hardware pen state (or [PenHardwareState.empty] when no native + /// data has arrived — non-Windows, plugin absent, or channel silent). + PenHardwareState get current => _current; + + /// Whether the native channel has delivered at least one event (i.e. the + /// native pen plugin is present and active). Used to prefer hardware signals + /// over the Flutter fallback only when they are actually available. + bool get isActive => _active; + bool _active = false; + + /// Begins listening to the native channel. Idempotent; safe on any platform + /// (no-ops where the channel has no handler). + void start() { + if (_sub != null) return; + try { + _sub = _channel.receiveBroadcastStream().listen( + _onEvent, + onError: (Object _) { + // No native handler (e.g. Linux/macOS) or transient error — ignore + // and keep the empty fallback state. + }, + cancelOnError: false, + ); + } catch (_) { + // receiveBroadcastStream can throw synchronously if the platform side is + // unavailable; degrade silently. + } + } + + void _onEvent(dynamic event) { + if (event is! Map) return; + final flags = (event['flags'] as num?)?.toInt() ?? 0; + _current = PenHardwareState( + barrel: flags & 0x1 != 0, + inverted: flags & 0x2 != 0, + eraser: flags & 0x4 != 0, + tiltX: (event['tiltX'] as num?)?.toDouble() ?? 0.0, + tiltY: (event['tiltY'] as num?)?.toDouble() ?? 0.0, + ); + _active = true; + } + + /// Stops listening and resets state. + void stop() { + _sub?.cancel(); + _sub = null; + _active = false; + _current = PenHardwareState.empty; + } +} diff --git a/lib/editor/ui/pen_settings_page.dart b/lib/editor/ui/pen_settings_page.dart index 3dc7dee..3746454 100644 --- a/lib/editor/ui/pen_settings_page.dart +++ b/lib/editor/ui/pen_settings_page.dart @@ -93,6 +93,15 @@ class _PenSettingsSheet extends StatelessWidget { icon: Icons.compress, colorScheme: colorScheme, ), + _SliderTile( + label: 'Pressure Sensitivity', + value: config.pressureSensitivity, + min: 0.0, + max: 1.0, + divisions: 20, + formatValue: (v) => v.toStringAsFixed(2), + onChanged: controller.setPressureSensitivity, + ), _SliderTile( label: 'Pressure Gamma', value: config.pressureGamma, diff --git a/lib/services/pdf_service.dart b/lib/services/pdf_service.dart index cc6b615..54c1b0c 100644 --- a/lib/services/pdf_service.dart +++ b/lib/services/pdf_service.dart @@ -296,7 +296,7 @@ class PdfService { final pfPoints = stroke.points .map( - (pt) => pf.Point( + (pt) => pf.PointVector( pt.x * pageSize.width, pt.y * pageSize.height, pt.pressure, @@ -306,21 +306,24 @@ class PdfService { final outline = pf.getStroke( pfPoints, - size: pixelWidth, - // Highlighter keeps constant width; pen/marker taper via thinning=0.7. - thinning: isHighlighter ? 0.0 : 0.7, - smoothing: 0.5, - streamline: 0.5, - // Real stylus pressure -> don't simulate; no pressure -> let freehand - // fake it based on velocity. Highlighter never simulates. - simulatePressure: !hasRealPressure && !isHighlighter, - isComplete: true, + options: pf.StrokeOptions( + size: pixelWidth, + // Highlighter keeps constant width; pen/marker taper via thinning=0.7. + thinning: isHighlighter ? 0.0 : 0.7, + smoothing: 0.5, + streamline: 0.5, + // Real stylus pressure -> don't simulate; no pressure -> let freehand + // fake it based on velocity. Highlighter never simulates. + simulatePressure: !hasRealPressure && !isHighlighter, + isComplete: true, + ), ); if (outline.isEmpty) return null; final path = PdfPath(); - path.addPolygon(outline.map((pt) => Offset(pt.x, pt.y)).toList()); + // outline is already List in perfect_freehand 2.x. + path.addPolygon(outline.toList()); return path; } } diff --git a/lib/services/stroke_rasterizer.dart b/lib/services/stroke_rasterizer.dart index 6d6d94f..64db71f 100644 --- a/lib/services/stroke_rasterizer.dart +++ b/lib/services/stroke_rasterizer.dart @@ -139,7 +139,7 @@ class StrokeRasterizer { ) { final pfPoints = points .map( - (p) => pf.Point( + (p) => pf.PointVector( p.x, p.y, _defaultPressureCurve.apply(p.pressure).clamp(0.0, 1.0), @@ -153,18 +153,20 @@ class StrokeRasterizer { final outline = pf.getStroke( pfPoints, - size: strokeWidth, - thinning: thinning, - smoothing: 0.5, - streamline: 0.5, - simulatePressure: tool != PenTool.marker && tool != PenTool.highlighter, - isComplete: true, + options: pf.StrokeOptions( + size: strokeWidth, + thinning: thinning, + smoothing: 0.5, + streamline: 0.5, + simulatePressure: tool != PenTool.marker && tool != PenTool.highlighter, + isComplete: true, + ), ); if (outline.isEmpty) return; - final path = Path()..moveTo(outline[0].x, outline[0].y); + final path = Path()..moveTo(outline[0].dx, outline[0].dy); for (var i = 1; i < outline.length; i++) { - path.lineTo(outline[i].x, outline[i].y); + path.lineTo(outline[i].dx, outline[i].dy); } path.close(); diff --git a/lib/widgets/ink_canvas.dart b/lib/widgets/ink_canvas.dart index caa2e8a..29c9986 100644 --- a/lib/widgets/ink_canvas.dart +++ b/lib/widgets/ink_canvas.dart @@ -528,7 +528,7 @@ class _InkPainter extends CustomPainter { ) { final pfPoints = points .map( - (p) => pf.Point( + (p) => pf.PointVector( p.x, p.y, pressureCurve.apply(p.pressure).clamp(0.0, 1.0), @@ -542,25 +542,25 @@ class _InkPainter extends CustomPainter { final outlinePoints = pf.getStroke( pfPoints, - size: strokeWidth, - thinning: thinning, - smoothing: 0.5, - streamline: 0.5, - taperStart: 0.0, - taperEnd: 0.0, - capStart: true, - capEnd: true, - simulatePressure: tool != PenTool.marker && tool != PenTool.highlighter, - isComplete: isComplete, + options: pf.StrokeOptions( + size: strokeWidth, + thinning: thinning, + smoothing: 0.5, + streamline: 0.5, + // 2.x defaults: no taper + capped ends (was taperStart/End:0 + + // capStart/End:true in 1.0.4). + simulatePressure: tool != PenTool.marker && tool != PenTool.highlighter, + isComplete: isComplete, + ), ); if (outlinePoints.isEmpty) return; final path = Path(); - path.moveTo(outlinePoints[0].x, outlinePoints[0].y); + path.moveTo(outlinePoints[0].dx, outlinePoints[0].dy); for (int i = 1; i < outlinePoints.length; i++) { - path.lineTo(outlinePoints[i].x, outlinePoints[i].y); + path.lineTo(outlinePoints[i].dx, outlinePoints[i].dy); } path.close(); diff --git a/pubspec.lock b/pubspec.lock index ca81706..4fe1655 100644 --- a/pubspec.lock +++ b/pubspec.lock @@ -787,10 +787,10 @@ packages: dependency: "direct main" description: name: perfect_freehand - sha256: "77bfdd5efb223d120de5cc18c5d6e0b36a835e920521cfe67e281960bad44c9b" + sha256: f42b8164c4e7e689b278f4e2e8e8f5006e716f8c127eb44404f21bd73b1d3b56 url: "https://pub.dev" source: hosted - version: "1.0.4" + version: "2.5.2+1" petitparser: dependency: transitive description: diff --git a/pubspec.yaml b/pubspec.yaml index b96c92b..3c5b7ef 100644 --- a/pubspec.yaml +++ b/pubspec.yaml @@ -16,7 +16,7 @@ dependencies: cupertino_icons: ^1.0.8 # Pen & Ink Rendering - perfect_freehand: ^1.0.0 + perfect_freehand: ^2.0.0 # PDF syncfusion_flutter_pdfviewer: ^28.2.7 diff --git a/test/pen_config_test.dart b/test/pen_config_test.dart index 7132ab5..bd366cf 100644 --- a/test/pen_config_test.dart +++ b/test/pen_config_test.dart @@ -4,6 +4,8 @@ // applyPressureCurve. No widgets, no SharedPreferences, no Flutter framework. import 'package:flutter_test/flutter_test.dart'; +import 'package:badnote/editor/engine/stroke_geometry.dart' + show kDefaultPenThinning; import 'package:badnote/editor/input/pen_config.dart'; void main() { @@ -43,9 +45,11 @@ void main() { fingerDrawing: true, penWidth: 0.008, highlighterWidth: 0.03, + pressureSensitivity: 0.42, ); final restored = PenConfig.fromJson(original.toJson()); expect(restored, equals(original)); + expect(restored.pressureSensitivity, 0.42); }); test('fromJson falls back to defaults for missing keys', () { @@ -57,6 +61,9 @@ void main() { expect(restored.fingerDrawing, false); expect(restored.penWidth, 0.004); expect(restored.highlighterWidth, 0.02); + // Missing pressureSensitivity defaults to the shared thinning constant, + // preserving the pre-existing stroke feel + export golden (plan M4). + expect(restored.pressureSensitivity, kDefaultPenThinning); }); test('fromJson falls back to defaults for unknown enum names', () { diff --git a/test/pen_polish_test.dart b/test/pen_polish_test.dart new file mode 100644 index 0000000..5224322 --- /dev/null +++ b/test/pen_polish_test.dart @@ -0,0 +1,104 @@ +// test/pen_polish_test.dart +// +// Guards the pen-polish work (W1 configurable thinning + W3 tilt): +// - the on-screen painter (ink_painters.buildStrokePath) and the export/engine +// path (stroke_geometry.buildStrokeOutline) share ONE thinning default +// (kDefaultPenThinning) and respond to it identically — the user's core +// "thinning 写死" complaint, and the single-source-of-truth invariant the +// Critic required; +// - tilt survives the PenStroke -> EditorStroke adapter (W3 model fix). + +import 'dart:ui'; + +import 'package:flutter_test/flutter_test.dart'; + +import 'package:badnote/editor/canvas/ink_painters.dart'; +import 'package:badnote/editor/canvas/pen_stroke.dart'; +import 'package:badnote/editor/engine/stroke_geometry.dart'; +import 'package:badnote/editor/engine/stroke_model.dart'; + +PenStroke _pressuredPen() => PenStroke( + // Varied pressure, deliberately NOT reaching full force, so that + // thinning>0 (pressure-modulated, narrower) vs thinning=0 (constant full + // width) produces a measurably different bounding box. + points: const [ + PenPoint(0.10, 0.50, 0.10), + PenPoint(0.30, 0.50, 0.25), + PenPoint(0.50, 0.50, 0.40), + PenPoint(0.70, 0.50, 0.55), + ], + color: 0xFF000000, + width: 0.01, + kind: PenStrokeKind.pen, + ); + +void main() { + const size = Size(1000, 1000); + + group('W1 — thinning is configurable and single-sourced', () { + test('buildStrokePath default == explicit kDefaultPenThinning', () { + final pen = _pressuredPen(); + final byDefault = buildStrokePath(pen, size, isComplete: true); + final explicit = buildStrokePath(pen, size, + isComplete: true, thinning: kDefaultPenThinning); + expect(byDefault.getBounds(), explicit.getBounds()); + }); + + test('thinning actually affects the outline (not hardcoded/ignored)', () { + // NB: the bounding box is thinning-INVARIANT here because + // perfect_freehand's round end-caps are drawn at the full `size`; only + // the mid-section width tracks pressure×thinning. So we compare the + // outline PERIMETER (sum of contour lengths), which does reflect the + // pinched middle. + double perimeter(Path p) => + p.computeMetrics().fold(0.0, (sum, m) => sum + m.length); + final pen = _pressuredPen(); + final strong = perimeter( + buildStrokePath(pen, size, isComplete: true, thinning: 0.85)); + final none = perimeter( + buildStrokePath(pen, size, isComplete: true, thinning: 0.0)); + // Constant width (0.0) vs pressure-thinning (0.85) must differ measurably. + expect((strong - none).abs(), greaterThan(1.0), + reason: 'thinning had no effect on the outline — it is not wired'); + }); + + test('screen and export builders agree for the same stroke + thinning', () { + final pen = _pressuredPen(); + final editor = EditorStroke.fromPenStroke(pen); + for (final thinning in const [0.0, kDefaultPenThinning, 1.0]) { + final screen = + buildStrokePath(pen, size, isComplete: true, thinning: thinning) + .getBounds(); + final export = + buildStrokeOutline(editor, size, isComplete: true, thinning: thinning) + .getBounds(); + expect(screen, export, + reason: 'screen vs export diverged at thinning=$thinning'); + } + }); + }); + + group('W3 — tilt survives the PenStroke adapter', () { + test('PenPoint.tilt maps into EditorPoint.tilt', () { + const pen = PenStroke( + points: [PenPoint(0.1, 0.2, 0.5, tilt: 23.5)], + color: 0xFF112233, + width: 0.01, + kind: PenStrokeKind.pen, + ); + final editor = EditorStroke.fromPenStroke(pen); + expect(editor.points.single.tilt, 23.5); + }); + + test('null tilt stays null through the adapter', () { + const pen = PenStroke( + points: [PenPoint(0.1, 0.2, 0.5)], + color: 0xFF112233, + width: 0.01, + kind: PenStrokeKind.pen, + ); + final editor = EditorStroke.fromPenStroke(pen); + expect(editor.points.single.tilt, isNull); + }); + }); +} diff --git a/windows/runner/CMakeLists.txt b/windows/runner/CMakeLists.txt index 727ff1e..3319b72 100644 --- a/windows/runner/CMakeLists.txt +++ b/windows/runner/CMakeLists.txt @@ -10,6 +10,7 @@ add_executable(${BINARY_NAME} WIN32 "flutter_window.cpp" "main.cpp" "ocr_channel.cpp" + "pen_channel.cpp" "utils.cpp" "win32_window.cpp" "${FLUTTER_MANAGED_DIR}/generated_plugin_registrant.cc" diff --git a/windows/runner/flutter_window.cpp b/windows/runner/flutter_window.cpp index 795370a..b6cddf4 100644 --- a/windows/runner/flutter_window.cpp +++ b/windows/runner/flutter_window.cpp @@ -4,6 +4,7 @@ #include "flutter/generated_plugin_registrant.h" #include "ocr_channel.h" +#include "pen_channel.h" FlutterWindow::FlutterWindow(const flutter::DartProject& project) : project_(project) {} @@ -27,6 +28,7 @@ bool FlutterWindow::OnCreate() { } RegisterPlugins(flutter_controller_->engine()); RegisterOcrChannel(flutter_controller_->engine()); + RegisterPenChannel(flutter_controller_->engine()); SetChildContent(flutter_controller_->view()->GetNativeWindow()); flutter_controller_->engine()->SetNextFrameCallback([&]() { @@ -53,6 +55,9 @@ LRESULT FlutterWindow::MessageHandler(HWND hwnd, UINT const message, WPARAM const wparam, LPARAM const lparam) noexcept { + // Observe pen messages before Flutter consumes WM_POINTER events. + ObservePenMessage(message, wparam, lparam); + // Give Flutter, including plugins, an opportunity to handle window messages. if (flutter_controller_) { std::optional result = diff --git a/windows/runner/pen_channel.cpp b/windows/runner/pen_channel.cpp new file mode 100644 index 0000000..f6aa77f --- /dev/null +++ b/windows/runner/pen_channel.cpp @@ -0,0 +1,89 @@ +#include "pen_channel.h" + +#include +#include +#include +#include +#include + +#include + +namespace { + +std::unique_ptr> g_pen_sink; + +std::unique_ptr> g_pen_channel; + +} // namespace + +void RegisterPenChannel(flutter::FlutterEngine* engine) { + g_pen_channel = + std::make_unique>( + engine->messenger(), "badnote/pen", + &flutter::StandardMethodCodec::GetInstance()); + + auto handler = std::make_unique< + flutter::StreamHandlerFunctions>( + [](const flutter::EncodableValue* arguments, + std::unique_ptr>&& + events) + -> std::unique_ptr< + flutter::StreamHandlerError> { + g_pen_sink = std::move(events); + return nullptr; + }, + [](const flutter::EncodableValue* arguments) + -> std::unique_ptr< + flutter::StreamHandlerError> { + g_pen_sink = nullptr; + return nullptr; + }); + + g_pen_channel->SetStreamHandler(std::move(handler)); +} + +void ObservePenMessage(UINT message, WPARAM wparam, LPARAM lparam) { + if (message != WM_POINTERENTER && message != WM_POINTERDOWN && + message != WM_POINTERUPDATE && message != WM_POINTERUP) { + return; + } + + if (!g_pen_sink) { + return; + } + + UINT32 pointerId = GET_POINTERID_WPARAM(wparam); + + POINTER_INPUT_TYPE type = PT_POINTER; + if (!GetPointerType(pointerId, &type) || type != PT_PEN) { + return; + } + + POINTER_PEN_INFO ppi{}; + if (!GetPointerPenInfo(pointerId, &ppi)) { + return; + } + + int flags = 0; + if (ppi.penFlags & PEN_FLAG_BARREL) flags |= 1; + if (ppi.penFlags & PEN_FLAG_INVERTED) flags |= 2; + if (ppi.penFlags & PEN_FLAG_ERASER) flags |= 4; + + flutter::EncodableMap payload{ + {flutter::EncodableValue("flags"), flutter::EncodableValue(flags)}, + {flutter::EncodableValue("tiltX"), flutter::EncodableValue(static_cast(ppi.tiltX))}, + {flutter::EncodableValue("tiltY"), flutter::EncodableValue(static_cast(ppi.tiltY))}, + }; + + g_pen_sink->Success(flutter::EncodableValue(payload)); + + // On pointer up, send a cleared flags event to signal lift-off. + if (message == WM_POINTERUP) { + flutter::EncodableMap clear{ + {flutter::EncodableValue("flags"), flutter::EncodableValue(0)}, + {flutter::EncodableValue("tiltX"), flutter::EncodableValue(0.0)}, + {flutter::EncodableValue("tiltY"), flutter::EncodableValue(0.0)}, + }; + g_pen_sink->Success(flutter::EncodableValue(clear)); + } +} diff --git a/windows/runner/pen_channel.h b/windows/runner/pen_channel.h new file mode 100644 index 0000000..c0ba066 --- /dev/null +++ b/windows/runner/pen_channel.h @@ -0,0 +1,13 @@ +#ifndef RUNNER_PEN_CHANNEL_H_ +#define RUNNER_PEN_CHANNEL_H_ + +#include + +namespace flutter { +class FlutterEngine; +} + +void RegisterPenChannel(flutter::FlutterEngine* engine); +void ObservePenMessage(UINT message, WPARAM wparam, LPARAM lparam); + +#endif // RUNNER_PEN_CHANNEL_H_