feat(pen): pressure-responsive width, configurable thinning, native Windows pen (tilt/buttons)
All checks were successful
CI / Windows build (push) Successful in 11m34s
All checks were successful
CI / Windows build (push) Successful in 11m34s
W1 — Custom pen width + pressure sensitivity (Saber-style):
- Root cause of "压感没用": perfect_freehand 1.0.4 IGNORES real stylus pressure
(hardcodes radius=size/2 when simulatePressure=false) — width never tracked pen
force. Upgraded perfect_freehand ^1.0.0 -> ^2.0.0 (honors real pressure); migrated
all 5 getStroke call sites to the 2.x API (PointVector / StrokeOptions / Offset).
- De-hardcoded `thinning` into `kDefaultPenThinning` (0.85), single source shared by
the on-screen painter and the PDF export path; exposed as PenConfig.pressureSensitivity
with a Pressure Sensitivity slider; live-applies via a config listener.
W3 — Native Windows pen plugin (tilt + barrel/eraser buttons):
- windows/runner/pen_channel.{h,cpp}: observe WM_POINTER at the TOP of MessageHandler
(before HandleTopLevelWindowProc, which Flutter uses to consume pen events), read
GetPointerPenInfo penFlags + tilt, stream over EventChannel('badnote/pen'); non-consuming.
- PenInputService: single latched hardware state (no Win32-pointerId<->event.pointer
correlation); graceful no-op off-Windows.
- pen_canvas maps barrel/inverted/eraser through PenConfig.sideButton/eraserEnd
(eraser/undo/toggleTool/pan) and captures tilt into PenPoint.tilt -> EditorPoint.tilt.
W2 — Zoom flicker: page raster isolated in its own RepaintBoundary (safe interim);
definitive crisp-on-zoom fix gated on the on-device root-cause probe (plan M3).
Plans: ralplan-consensus plan at docs/plans/2026-06-22-badnote-pen-polish.md
(Architect APPROVE-WITH-MUST-FIX M1-M4 + Critic ITERATE->APPROVE).
Tests: 58/58 pass incl. shared-thinning invariant + thinning-affects-outline +
tilt-adapter round-trip. flutter analyze clean; linux debug build OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
446
docs/plans/2026-06-21-badnote-full-refactor.md
Normal file
446
docs/plans/2026-06-21-badnote-full-refactor.md
Normal file
@@ -0,0 +1,446 @@
|
|||||||
|
# BadNote — Full Refactor + Feature Roadmap (own-canvas engine)
|
||||||
|
|
||||||
|
**Status:** PLAN (ralplan consensus — Architect APPROVE-WITH-MUST-FIX + Critic-ITERATE fixes applied 2026-06-21: MF1–MF3 + P0.5 slice + SF1–SF5 + cache-split / diff-write-durability / erase-is-new-behavior / render-handle-lifecycle; pending final Critic confirm)
|
||||||
|
**Date:** 2026-06-21
|
||||||
|
**Mode:** DELIBERATE (high-risk: engine generalization, 60fps continuous/double-page, OCR/CAS feasibility, server/AI scope)
|
||||||
|
**Owner plan file:** `docs/plans/2026-06-21-badnote-full-refactor.md`
|
||||||
|
**Supersedes (in part):** `docs/plans/2026-06-21-badnote-phase1.md`
|
||||||
|
|
||||||
|
> **Grounding.** Written after reading the live code, not from memory. Verified sources:
|
||||||
|
> live editor `lib/editor/canvas/{pen_canvas,ink_painters,pen_stroke,pen_editor_screen}.dart`,
|
||||||
|
> `lib/editor/pdf/pen_capture_region.dart`, `lib/main.dart` (dynamic_color + `pdfrxFlutterInitialize` + `PenCaptureBinding`),
|
||||||
|
> `lib/services/{database_service,pdf_service,stroke_rasterizer,undo_manager}.dart`,
|
||||||
|
> `lib/services/ocr/onnx_recognition_backend.dart`, `lib/models/{ink_stroke,ink_point,pressure_curve}.dart`,
|
||||||
|
> `lib/providers/settings_provider.dart`, `lib/screens/home_screen.dart`,
|
||||||
|
> `server/badnote_server/{main,models,routers/sync_router}.py`, `server/README.md`,
|
||||||
|
> `pubspec.yaml`, existing `test/` + `integration_test/` + `tool/` assets.
|
||||||
|
>
|
||||||
|
> **CRITICAL DIRECTION CHANGE captured here (vs phase1 plan):** the live `lib/editor/canvas/` has **already abandoned** the phase-1 "pdfrx `pageOverlaysBuilder`-hosted ink + RenderProxyBox arena bypass" architecture. The live `PenCanvas` owns **one `InteractiveViewer` + a single `Listener`** that draws a `PdfPageView` *bitmap* and the ink in the **same** child subtree (Saber clean-room model). pdfrx is used only as a **page renderer / document API** (`PdfDocument.openFile`, `PdfPageView`), never for gestures. `PenCaptureRegion`/`PenCaptureBinding` are installed in `main()` but are **NOT wired into the live canvas** and are slated for retirement (§6, §9). This plan builds the full product on the **own-canvas** model and records that decision in the ADR (§10).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. Vision & Scope
|
||||||
|
|
||||||
|
**BadNote** is a pen-first, performant note-taking app. Primary device: **Windows Tablet + Surface Pen** (Flutter ≥ 3.44 for WM_POINTER stylus/multitouch); desktop for review/复习/search. Differentiators the user explicitly wants to win on: **(1)** library-wide full-text search over PDF text + typed text + handwriting OCR with jump-to-location; **(2)** modern cohesive **Material You** UI; **(3)** truly **book-like reading** (page-flip, two-page spread, thumbnail grid, reader-vs-annotate modes). Plus a **双链 sticky-note infinite board**, **one-notebook-per-PDF** with insertable blank pages and never-rasterized source, and a later **server sync + AI refinement** and **ink CAS**.
|
||||||
|
|
||||||
|
**Existing user data MAY be reset** — no migration burden; prefer the cleanest schema. **Stack stays Flutter** (own gesture pipeline; pdfrx only renders pages). GFW-aware CI already solved (pdfium pre-fetch + Dev Mode + vendored sqlite3 + onnxruntime pre-fetch).
|
||||||
|
|
||||||
|
### 0.1 The 11 feature areas → phase map (full vision; nothing dropped, everything sequenced)
|
||||||
|
|
||||||
|
| # | Feature | Phase | Risk |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| F1 | Pen-first editor core (harden): pressure, palm rejection, perfect_freehand quality, eraser, tools, undo/redo, **DB persistence of strokes per doc/page** | **P0** | Low–Med |
|
||||||
|
| F2 | Page layout modes: continuous-single/-double, paged-single/-double + switcher; lazy render @60fps | **P1** | **High** |
|
||||||
|
| F3 | Book-like reading: page-flip, two-page spread, reader vs annotate mode, quick jump | **P1** | Med |
|
||||||
|
| F4 | Thumbnail-grid navigation (Drawboard-style), slider, no keyboard input | **P1** | Low |
|
||||||
|
| F5 | Configurable pen: side-button + eraser-end mapping, pressure curve, palm sensitivity, finger-drawing toggle — Pen settings page | **P2** | Med |
|
||||||
|
| F6 | One-notebook-per-PDF: page-level binding, insert blank pages between PDF pages, keep PDF vector, portable bundle + relink | **P2** | Med |
|
||||||
|
| F7 | Infinite 便利贴 board + **双链** bidirectional links / backlinks | **P3** | Med–High |
|
||||||
|
| F8 | Library-wide full-text search (PDF text + typed text + handwriting OCR) with snippets + jump | **P3** | **High (handwriting OCR)** |
|
||||||
|
| F9 | Server sync + AI refinement (llm_wiki, VLM/LLM organize) | **P4** | High (scope) |
|
||||||
|
| F10 | Ink CAS / formula recognition → searchable + solve toggle | **P5** | **Very High** |
|
||||||
|
| F11 | Modern UX polish: Material You, subtoolbars, drag-reorder thumbnails, hover pre-warm | **woven across P0–P3** | Low |
|
||||||
|
|
||||||
|
### 0.2 Non-goals per phase
|
||||||
|
- **P0:** no layout modes, no board, no text boxes, no search UI, no settings page beyond what exists, no server. Single-page editor + persistence + eraser/undo only.
|
||||||
|
- **P1:** no 双链, no OCR-search, no server, no CAS. Layout/reading/thumbnails only.
|
||||||
|
- **P2:** no server, no CAS, no handwriting OCR. Pen-config + notebook/bundle only.
|
||||||
|
- **P3:** no CAS, no server-side AI. Board+双链 and **local** search (incl. best-effort handwriting OCR via existing text-line model; formula deferred to P5).
|
||||||
|
- **P4:** sync + AI organize; CAS still out.
|
||||||
|
- **P5:** formula OCR + CAS.
|
||||||
|
- **All phases:** never rasterize the source PDF; no on-screen keyboard reliance (slider/grid nav).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Target Architecture
|
||||||
|
|
||||||
|
### 1.1 Layered module decomposition (target `lib/editor/`)
|
||||||
|
|
||||||
|
```
|
||||||
|
lib/editor/
|
||||||
|
├─ engine/ # host-agnostic ink engine (pure-ish, testable)
|
||||||
|
│ ├─ coordinate_space_host.dart # CoordinateSpaceHost + NormalizedPageHost + BoardHost
|
||||||
|
│ ├─ stroke_model.dart # EditorStroke/EditorPoint (canonical; replaces PenStroke split — §3, §4)
|
||||||
|
│ ├─ stroke_store.dart # StrokeStore { committed, int revision } (O(1) shouldRepaint gate)
|
||||||
|
│ ├─ stroke_geometry.dart # buildStrokeOutline(getStroke) — single source for screen+export (kills hairline bug)
|
||||||
|
│ ├─ stroke_eraser.dart # pure splitStroke / eraseHits (extract from pen_canvas._eraseAt)
|
||||||
|
│ └─ undo_stack.dart # global, commit-time-ordered, host-tagged (generalize undo_manager)
|
||||||
|
├─ input/
|
||||||
|
│ ├─ input_arbiter.dart # pure state machine: pointerCount>=2→pan/zoom, stylus→draw, palm reject
|
||||||
|
│ └─ pen_config.dart # button/eraser-end action mapping, pressure curve, palm sensitivity (F5)
|
||||||
|
├─ render/
|
||||||
|
│ ├─ annotation_layer.dart # Stack(StaticInkPainter, LiveInkPainter, TextBoxLayer) over a host
|
||||||
|
│ ├─ static_ink_painter.dart # reads ink_picture_cache, keyed by revision
|
||||||
|
│ ├─ live_ink_painter.dart # in-progress stroke only
|
||||||
|
│ └─ ink_picture_cache.dart # LRU<revision, ui.Picture> for STATIC INK — resolution-INDEPENDENT, NO DPI bucket (vector ink re-rasterizes crisp at composite)
|
||||||
|
├─ layout/ # F2/F3 — page layout + reading
|
||||||
|
│ ├─ page_layout.dart # enum {continuousSingle, continuousDouble, pagedSingle, pagedDouble}
|
||||||
|
│ ├─ page_viewport.dart # lazy windowed page hosting (visible + cache extent), recenter
|
||||||
|
│ └─ reader_controller.dart # reader vs annotate mode, page-flip, spread, quick-jump
|
||||||
|
├─ pdf/
|
||||||
|
│ ├─ pdf_document_source.dart # wraps PdfDocument (open/dispose/pageSize/render), notebook page-map (F6)
|
||||||
|
│ ├─ page_tile.dart # PdfPageView host widget (bitmap) sized to host rect; drives tile DPI from transform scale (R11)
|
||||||
|
│ └─ page_tile_cache.dart # LRU<TileKey{hostId, dpiBucket}, ui.Image> for rendered PAGE BITMAPS — DPI-bucketed (tiles blur on upscale); owns native-handle dispose (R11)
|
||||||
|
├─ board/ # F7
|
||||||
|
│ ├─ board_host_pane.dart # infinite InteractiveViewer board, reuses engine
|
||||||
|
│ └─ link_graph.dart # 双链 backlink model + queries
|
||||||
|
├─ text/ # OneNote-like editable text boxes (P3+, optional in P2 notebook)
|
||||||
|
│ ├─ text_box_model.dart
|
||||||
|
│ └─ text_box_layer.dart
|
||||||
|
├─ persistence/
|
||||||
|
│ ├─ editor_repository.dart # batched load/save per document; maps DB ↔ engine
|
||||||
|
│ └─ save_scheduler.dart # debounced, synchronous-snapshot-before-await
|
||||||
|
├─ search/ # F8 (P3)
|
||||||
|
│ ├─ search_indexer.dart # PDF text + typed text + OCR → document_fts/library index
|
||||||
|
│ └─ ocr_ingest.dart # bridge to lib/services/ocr (text-line; formula later)
|
||||||
|
├─ sync/ # F9 (P4)
|
||||||
|
│ └─ sync_client.dart # talks to server/ (already has push/pull/auth)
|
||||||
|
└─ ui/
|
||||||
|
├─ editor_screen.dart # top-level editor (replaces pen_editor_screen + pdf_annotator_screen)
|
||||||
|
├─ editor_toolbar.dart # Material You subtoolbars (F11)
|
||||||
|
├─ thumbnail_grid.dart # F4 nav
|
||||||
|
└─ editor_shortcuts.dart # tool numbers, Ctrl+Z/Y/F/S (mouse/desktop)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 1.2 Dependency graph (acyclic)
|
||||||
|
|
||||||
|
```
|
||||||
|
ui/ ─────────────► layout/ ─► pdf/ ─► (pdfrx PdfDocument/PdfPageView)
|
||||||
|
│ │
|
||||||
|
│ ├──────► render/ ─► engine/ (geometry, store, hosts)
|
||||||
|
│ └──────► input/ ─► engine/
|
||||||
|
├──► board/ ─────────────► render/, engine/
|
||||||
|
├──► text/ ─────────────► engine/(hosts)
|
||||||
|
├──► search/ ────────────► persistence/, services/ocr/
|
||||||
|
├──► sync/ ────────────► persistence/, server API
|
||||||
|
└──► persistence/ ───────► services/database_service.dart, engine/(stroke_model)
|
||||||
|
```
|
||||||
|
**engine/** depends on nothing in editor/ except itself (and `perfect_freehand`). **render/input/layout/** depend on engine. **ui/** is the only place wiring Riverpod controllers. This is the seam that lets PDF-page host, infinite board, and (P5) CAS overlay all reuse one renderer.
|
||||||
|
|
||||||
|
### 1.3 Coordinate model — single source of truth
|
||||||
|
**Truth = host content coordinates** (already the live convention; generalize it):
|
||||||
|
- **NormalizedPageHost:** points ∈ `[0,1]` of the page rect (today `PenPoint(nx, ny)`); width as fraction of page width (today `PenStroke.width`). Stored per `(documentId, pageIndex)`.
|
||||||
|
- **BoardHost:** absolute logical px on an unbounded canvas; `InteractiveViewer` supplies pan/zoom.
|
||||||
|
|
||||||
|
`CoordinateSpaceHost` exposes `toContent(deviceLocal, deviceSize)`, `toDevice(content, deviceSize)`, `applyContentToCanvas(canvas, deviceSize)`. The live painters already scale `nx*pageSize.width` at paint time — formalize that into `applyContentToCanvas` (a `canvas.scale`) so no transformed-point copies are ever allocated. Text boxes are a **sanctioned widget-space exception** (Positioned widgets multiply by current zoom; stored truth stays content coords for search).
|
||||||
|
|
||||||
|
### 1.4 How today's `lib/editor/canvas/` evolves into this (no rewrite)
|
||||||
|
|
||||||
|
| Today (live) | Becomes | Action |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `pen_stroke.dart` `PenStroke`/`PenPoint` (in-memory only) | `engine/stroke_model.dart` `EditorStroke`/`EditorPoint` **persistable** (freezed/JSON, normalized) | **generalize + add JSON**; converge with `InkStroke` (§4) |
|
||||||
|
| `ink_painters.dart` `StaticInkPainter`/`LiveInkPainter` + `buildStrokePath` | `render/static_ink_painter.dart` (+ `render/ink_picture_cache.dart`), `render/live_ink_painter.dart`, `engine/stroke_geometry.dart` | **move + add ink `ui.Picture` cache (resolution-independent, revision-keyed) + revision gate**; `buildStrokePath`→`buildStrokeOutline` shared with export |
|
||||||
|
| `pen_canvas.dart` `_PenCanvasState` gesture logic (`_activePointers`, `_shouldDraw`, palm reject, `_eraseAt`) | `input/input_arbiter.dart` (pure SM) + `render/annotation_layer.dart` (widget) | **extract** SM out of the widget; eraser→`engine/stroke_eraser.dart` |
|
||||||
|
| `pen_editor_screen.dart` single-page + `_strokesByPage` Map + `_transform` + slider | `ui/editor_screen.dart` + `layout/page_viewport.dart` + `persistence/editor_repository.dart` | **generalize** single-page → windowed multi-page; `_strokesByPage` → repository-backed `HostState`s |
|
||||||
|
| `pen_capture_region.dart` + `PenCaptureBinding` (unused by live canvas) | — | **retire** after P0 confirms the `Listener`-in-shared-transform model holds on device (§6, §9) |
|
||||||
|
| `pen_editor_screen` single `PdfPageView` bitmap inside the `InteractiveViewer` (1× layout-sized bitmap, matrix-scaled by pinch) | `pdf/page_tile.dart` **multi-resolution tile** whose render DPI tracks the transform scale | **NEW work (not relocation): own multi-resolution tiling** — pinch GPU-upscales a 1× bitmap → blurry at high zoom (R11). On zoom-settle, re-instantiate the tile (or `PdfPage.render()`) at a DPI matching the current scale into `pdf/page_tile_cache.dart` (DPI-bucketed page-bitmap cache, distinct from the ink cache). Budgeted P1/P0.5. |
|
||||||
|
|
||||||
|
Net: the four live files are **promoted, not thrown away**. P0 is mostly *relocation + persistence + revision-gated Picture cache*, which is low risk and immediately shippable. The one genuinely **new** burden the own-canvas choice imposes (beyond layout/windowing) is **multi-resolution page-tile rendering** — see R11 (§7) and the P0.5 gate (§5/§9): we own crisp-on-zoom re-rasterization that pdfrx's own viewer would have given for free.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Per-feature design (file-level)
|
||||||
|
|
||||||
|
### F1 — Pen-first editor core + persistence (P0)
|
||||||
|
**Add/modify:**
|
||||||
|
- `engine/stroke_model.dart` **[ADD]** — `EditorStroke { id, List<EditorPoint> points, EditorTool tool, int color, double width, bool filled, String? textContent, double fontSize }`, `EditorPoint { double x, y; double? pressure; double? tilt; int? timestamp; InputDeviceKind? pointerDeviceKind }`. freezed + `toJson/fromJson`. (Normalized; `width` = fraction of page width, matching live `PenStroke`.) **SF1 — non-lossy superset of `InkStroke`/`InkPoint`:** `InkPoint` carries `tilt`/`timestamp`/`pointerDeviceKind` (database_service.dart:346–362) that the live `PenPoint` drops; `EditorPoint` includes them now (nullable, cheap with freezed) so the `InkStroke`↔`EditorStroke` adapter (for OCR/export reuse) is **lossless in both directions**. The live `PenCanvas` simply leaves the extra fields null at capture; existing `InkStroke` data round-trips intact. (Avoids a silent data-loss footgun for OCR, which keys on `pointerDeviceKind`/`pressure`.)
|
||||||
|
- `engine/stroke_store.dart` **[ADD]** — `class StrokeStore { List<EditorStroke> committed; int revision; add/removeAt/replace bump revision; }`. Fixes the live `pen_editor_screen._commitStroke` "new list identity" hack by making revision explicit.
|
||||||
|
- `engine/stroke_geometry.dart` **[ADD]** — `Path buildStrokeOutline(EditorStroke, Size, {bool isComplete})` lifted verbatim from `ink_painters.buildStrokePath` (the proven `getStroke(thinning: hl?0:0.7, smoothing:.5, streamline:.5, simulatePressure: !hasRealPressure && !hl)` recipe). **Single source** for screen + export.
|
||||||
|
- `engine/stroke_eraser.dart` **[ADD]** — pure `eraseHits(strokes, point, radius)` (extract live `_eraseAt`) + `splitStroke` for partial/segment erase. **Critic note — `splitStroke` is NEW behavior, not an extraction:** the live `pen_canvas._eraseAt` is **whole-stroke** (it removes the entire stroke on the first proximity hit and returns). There is no segment-split logic in the live canvas to port — `undo_manager.removeStroke` only *records* replacements, it does not compute them. So `splitStroke` (point-run splitting → 0/1/2 sub-strokes) is greenfield code; its tests (§8) exercise new functionality, not a regression port. `eraseHits` (whole-stroke) IS an extraction of the live behavior and stays available as the default erase mode.
|
||||||
|
- `engine/undo_stack.dart` **[ADD]** — generalize `lib/services/undo_manager.dart` to host-tagged, commit-time-ordered global stack (entries carry `hostId`+`pageIndex`).
|
||||||
|
- `render/{annotation_layer,static_ink_painter,live_ink_painter,ink_picture_cache}.dart` **[ADD]** — relocate live painters; `StaticInkPainter` reads `ink_picture_cache` (`LRU<revision, ui.Picture>`) keyed by `revision` (`shouldRepaint = old.revision != revision` — O(1)); per-host `RepaintBoundary`. **`ink_picture_cache` is for STATIC VECTOR INK only — resolution-INDEPENDENT, NO DPI bucket** (a `ui.Picture` of vector strokes re-rasterizes crisp at composite time at any zoom). It is a **different cache from the page-bitmap `page_tile_cache`** (§2/F2, R11), which IS DPI-bucketed because raster page tiles blur on upscale. Do not conflate the two.
|
||||||
|
- `input/input_arbiter.dart` **[ADD]** — pure SM extracted from `pen_canvas` (`idle→inking→erasing`, `pointerCount>=2`→cancel+pan, palm reject = touch dropped while stylus active, eraser = `kSecondaryButton || invertedStylus`). Keeps the live pressure-normalization logic (`_normalizedPressure`).
|
||||||
|
- `persistence/editor_repository.dart` **[ADD]** — `loadDocument(documentId) → Map<int,StrokeStore>` (one batched query), `saveHost(documentId, hostId, EditorStroke list)`.
|
||||||
|
- **MF3 — write contract (load-bearing for the per-row choice):** `saveHost` MUST **diff by stroke id** against the rows already on disk for that host — **UPSERT only changed/new rows, DELETE only removed rows**. It must NOT delete-all-rows-for-host then re-insert (that is exactly what the live notes path does at database_service.dart:281–306, and at 2,000 strokes it is **slower** than a single blob rewrite — re-inserting 2,000 rows per save). The per-stroke-row schema (§3) is justified **only** under this diff contract: erasing 1 of 2,000 strokes ⇒ 1 DELETE, 0 re-inserts; adding 1 stroke ⇒ 1 INSERT. The `SaveScheduler` hands `saveHost` the synchronously-captured stroke snapshot; the repository keeps a last-persisted id-set per host to compute the diff. If diffing proves fiddly under churn, the fallback is a per-page blob (NOT delete-all+reinsert) — but the diff path is the default and is what makes per-row worthwhile.
|
||||||
|
- **Durability invariant (Critic):** the UPSERT(s) + DELETE(s) for one `saveHost` MUST run inside **one `sqflite` transaction**, and the in-memory **per-host last-persisted id-set is updated ONLY after that transaction commits** (in the `then`/post-await success path) — never optimistically before the write. A crash or interruption mid-diff must leave memory and disk consistent: either the whole diff applied (and the id-set advances) or none of it did (and the id-set is unchanged, so the next save re-derives the same diff and retries). This mirrors the existing `updateNote` interruption warning (database_service.dart:281–306: "an interruption mid-way would permanently lose strokes, so the whole sequence must run inside one transaction").
|
||||||
|
- `persistence/save_scheduler.dart` **[ADD]** — debounced ~800ms; **serialize JSON synchronously before any await**; flush on page-leave/dispose.
|
||||||
|
- `ui/editor_screen.dart`, `ui/editor_toolbar.dart` **[ADD]** — port `pen_editor_screen` UI (floating Material You palette + page pill already good).
|
||||||
|
|
||||||
|
**Public interfaces (key):**
|
||||||
|
```dart
|
||||||
|
abstract class CoordinateSpaceHost {
|
||||||
|
String get hostId; int get pageIndex;
|
||||||
|
Offset toContent(Offset deviceLocal, Size deviceSize);
|
||||||
|
Offset toDevice(Offset content, Size deviceSize);
|
||||||
|
void applyContentToCanvas(Canvas c, Size deviceSize);
|
||||||
|
}
|
||||||
|
class EditorController extends ChangeNotifier { // ui-facing, Riverpod-provided
|
||||||
|
void beginStroke(CoordinateSpaceHost h, EditorPoint p);
|
||||||
|
void extendStroke(EditorPoint p);
|
||||||
|
void commitStroke(); // getStroke once → append → revision++ → undo push → scheduleSave
|
||||||
|
void eraseAt(CoordinateSpaceHost h, Offset content, double radius);
|
||||||
|
void undo(); void redo();
|
||||||
|
void setTool(EditorTool t); void setColor(Color c); void setLayout(PageLayout l);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
**Acceptance:** draw/erase/undo/redo on a page; close+reopen → strokes persisted (DB); `StaticInkPainter.shouldRepaint==false` while drawing a new stroke on a 2,000-stroke page (revision constant); no per-frame point allocation for committed strokes.
|
||||||
|
|
||||||
|
### F2 — Page layout modes (P1)
|
||||||
|
- `layout/page_layout.dart` **[ADD]** — `enum PageLayout { continuousSingle, continuousDouble, pagedSingle, pagedDouble }`.
|
||||||
|
- `layout/page_viewport.dart` **[ADD]** — windowed lazy hosting: only pages in `[firstVisible - cacheExtent, lastVisible + cacheExtent]` mount an `AnnotationLayer` + `PageTile`; others are disposed (Picture evicted). Continuous = scrollable column/two-column; paged = `PageView`. Recenter on layout switch.
|
||||||
|
- `pdf/pdf_document_source.dart` **[ADD]** — `PdfDocument` wrapper exposing `pageSize(i)`, `renderTile(...)`, page count; owns dispose. Replaces ad-hoc `PdfDocument.openFile` in `pen_editor_screen`.
|
||||||
|
- `pdf/page_tile.dart` **[ADD] — multi-resolution tile (R11 mitigation).** Under one shared `InteractiveViewer`, a `PdfPageView` renders a bitmap sized to its **layout constraints × devicePixelRatio** and the matrix scales that 1× bitmap, so pinch-zoom GPU-**upscales** → blurry text/rules at high zoom (unlike pdfrx's own viewer, which re-renders crisp tiles per zoom level). `page_tile` watches the transform scale and on **zoom-settle** (debounced) re-instantiates its render at a DPI matching the current scale — via either a re-laid-out `PdfPageView` at the new pixel size or `PdfPage.render(width/height at target DPI)` into `page_tile_cache`. **Cap retained DPI** (e.g. ≤ 3× base) to bound memory; downscale path stays matrix-only (sharp enough). This is **new own-canvas work**, not relocation.
|
||||||
|
- `pdf/page_tile_cache.dart` **[ADD]** — `LRU<TileKey{hostId, dpiBucket}, ui.Image>` of rendered **page bitmaps** (NOT ink). Key includes the **DPI bucket** so a page re-rendered at higher DPI replaces (not duplicates) its lower-DPI tile. **Owns the native-handle dispose lifecycle** of each `PdfPage.render()` result (`PdfImage` → backing `ui.Image`), deferred to post-frame so the raster thread is done with an evicted tile before disposal. **Distinct from `render/ink_picture_cache.dart`** (§2/F1) which holds resolution-independent ink `ui.Picture`s with no DPI bucket.
|
||||||
|
- `ui/editor_toolbar.dart` **[MODIFY]** — layout switcher control.
|
||||||
|
|
||||||
|
**Two separate caches + memory budgets (Critic — do not conflate):**
|
||||||
|
- **`ink_picture_cache` (vector ink):** `LRU<revision, ui.Picture>`, ~8–12 mounted-page Pictures, dispose deferred to post-frame. Budget ~ a few MB (vector op-lists are cheap). No DPI bucket.
|
||||||
|
- **`page_tile_cache` (raster page bitmaps):** `LRU<TileKey, ui.Image>`. **This is the heavy one.** A single A4 page rendered at 3× DPI is ~**10–30 MB** (≈ 1785×2526 px × 4 bytes ≈ 18 MB at 3× of a 595×842 pt page @ ~2 dppt). So **8–12 tiles at 3× would be ~150–350 MB — the old "≤64 MB / 8–12 pages" figure (R10) was unit-confused** (it conflated ink Pictures with page bitmaps). **Resolution:** size the *tile* window to the device memory budget, not a fixed page count — e.g. keep **full-DPI tiles only for the visible + ±1 pages (≈ 4 in double-page), and downgrade off-window pages to a 1× thumbnail tier** (matrix-upscaled, accepted as blurry only while scrolling). The ink cache keeps its wider ~8–12 window (cheap). R10's ≤64 MB now applies to **ink + downgraded-tier tiles**; the small high-DPI tile set is budgeted separately (~64–128 MB depending on device), tuned in P0.5/P1.
|
||||||
|
|
||||||
|
**Data structures:** `PageWindow { int first, last }`; `Map<int, HostState> mountedHosts`; `TileKey { hostId, dpiBucket }`. **Acceptance:** all four modes render; switching recenters; double-page shows two pages side-by-side; ink tracks scroll/zoom; **page text/rules stay crisp at 4× zoom (no GPU-upscale blur) — DPI refreshes on zoom-settle** (R11); combined cache memory stays within the device budget under a full scroll; perf gate (§7) holds.
|
||||||
|
|
||||||
|
### F3 — Book-like reading (P1)
|
||||||
|
- `layout/reader_controller.dart` **[ADD]** — `ReaderMode { read, annotate }`; in `read` the arbiter never draws (pen ignored); page-flip animation for `pagedSingle/Double` via `PageView` physics; spread layout from `pagedDouble`.
|
||||||
|
- `ui/editor_screen.dart` **[MODIFY]** — mode toggle in toolbar; quick-jump via slider/grid (no keyboard).
|
||||||
|
|
||||||
|
**Acceptance:** read mode blocks ink; annotate mode draws; page-flip animates; two-page spread aligns facing pages; quick-jump scrolls/animates to target.
|
||||||
|
|
||||||
|
### F4 — Thumbnail-grid navigation (P1)
|
||||||
|
- `ui/thumbnail_grid.dart` **[ADD]** — Drawboard-style grid; tiles render via `pdf/pdf_document_source.renderTile` (replaces `lib/services/thumbnail_service.dart` syncfusion path). Tap → jump; **drag-reorder** for notebook page order (F6). Slider remains the linear scrubber (already in `pen_editor_screen._buildPagePill`).
|
||||||
|
- `lib/services/thumbnail_service.dart` **[MODIFY/RETIRE]** — migrate to pdfrx render; drop `syncfusion_pdfviewer_platform_interface` dep once ported.
|
||||||
|
|
||||||
|
**Acceptance:** grid shows all pages; tap jumps; drag reorders (F6); no text-field page input.
|
||||||
|
|
||||||
|
### F5 — Configurable pen (P2)
|
||||||
|
- `input/pen_config.dart` **[ADD]** — `PenConfig { ButtonAction sideButton; ButtonAction eraserEnd; PressureCurveType curve; double palmSensitivity; bool fingerDrawing; }`; `enum ButtonAction { eraser, undo, toggleTool, pan, lasso, none }`. `InputArbiter` consults it (replaces hardcoded `_isEraserSignal`).
|
||||||
|
- `lib/providers/settings_provider.dart` **[MODIFY]** — persist `PenConfig` (extend existing `SharedPreferences` notifier; it already stores pressure curve + stabilization).
|
||||||
|
- `lib/screens/settings_screen.dart` **[MODIFY]** / **[ADD]** `ui/pen_settings_page.dart` — mapping UI (SpeedyNote-style dialog).
|
||||||
|
- Reuse `lib/models/pressure_curve.dart` (already has linear/soft/hard/custom + `apply`).
|
||||||
|
|
||||||
|
**Acceptance:** remapping side-button to undo makes the barrel button undo; pressure curve changes stroke taper; palm sensitivity changes touch-cooldown; finger-drawing toggle works; all persist across restart.
|
||||||
|
|
||||||
|
### F6 — One-notebook-per-PDF (P2)
|
||||||
|
- `pdf/pdf_document_source.dart` **[MODIFY]** — a **page-map**: logical notebook pages → either a source-PDF page index or a synthetic blank page. Insert-blank adds a synthetic page **without rasterizing or editing the source bytes** (keep source vector + searchable). Ink/text bind to the **logical** page id (stable UUID per logical page), not raw PDF index, so inserts don't reshuffle annotations.
|
||||||
|
- `lib/services/pdf_service.dart` **[MODIFY]** — keep headless syncfusion export/mutate; export walks the page-map. **Fix the hairline bug**: `_renderStrokes` must build a `PdfPath` from `buildStrokeOutline` points and **fill** it (currently strokes line-segments → hairline). Shape/line/arrow keep stroke semantics.
|
||||||
|
- DB **[MODIFY]** — `notebook_pages(id, document_id, ordinal, source_page_index INTEGER NULL, kind)`; ink keyed by `notebook_page_id` (§4). Portable bundle = zip {source.pdf, badnote.json(strokes/text/links/page-map)} + relink-on-open (match by content hash, fall back to picker).
|
||||||
|
|
||||||
|
**Acceptance:** insert blank page between PDF pages → source PDF untouched (still vector/searchable in another viewer); reorder pages keeps ink attached; export `.pdf` shows filled ink matching screen (golden); bundle round-trips on another machine.
|
||||||
|
|
||||||
|
### F7 — Infinite board + 双链 (P3)
|
||||||
|
- `board/board_host_pane.dart` **[ADD]** — `BoardHost` (absolute px) in a constrained-false `InteractiveViewer`; reuses `AnnotationLayer` + engine. Migrates `lib/screens/split_view_screen.dart` + `scratchpads` table.
|
||||||
|
- `board/link_graph.dart` **[ADD]** — `Link { srcRef, dstRef }` where a ref is `(kind: notebookPage|board|note, id)`; `backlinksOf(ref)`. Wiki-style `[[...]]` parsing in text boxes; sticky-note = a small board region or text box that can link to a page/note. Backlink panel queries `links` table.
|
||||||
|
- DB **[ADD]** — `links(id, src_kind, src_id, dst_kind, dst_id, created_at)` + indexes both directions; `boards(id, document_id NULL, strokes_json, ...)` (generalize `scratchpads`).
|
||||||
|
|
||||||
|
**Acceptance:** create a sticky linking page 3 → note X; open note X shows a backlink to page 3; board draws at perf target; link graph survives restart; deleting a target leaves a dangling-link indicator (no crash).
|
||||||
|
|
||||||
|
### F8 — Library-wide full-text search (P3)
|
||||||
|
- `search/search_indexer.dart` **[ADD]** — index three sources into a unified library index: **(a)** PDF embedded text (pdfrx `PdfPage.loadText`/`charRects` text API — source-pinned in P0.5 per SF4), **(b)** typed text boxes (`content`), **(c)** handwriting OCR (existing `services/ocr` text-line ONNX over rasterized strokes via `StrokeRasterizer`). Reuse existing `document_fts` FTS5 + a new `library_fts` spanning notebooks/boards/notes with `(ref_kind, ref_id, page, snippet)`.
|
||||||
|
- `search/ocr_ingest.dart` **[ADD]** — batch handwriting OCR per logical page (best-effort; set expectations: text-line only, no math). Background isolate; debounced after ink idle. **The OCR backend itself no-ops when the model is unavailable (onnx_recognition_backend.dart header: "verify on-device") — search MUST never block on, nor be gated by, OCR results.**
|
||||||
|
- `lib/screens/search_screen.dart` **[MODIFY]** — unified results with snippets + **jump-to-location** (open editor at the page + scroll, or board at the region).
|
||||||
|
|
||||||
|
**Acceptance — split into two exit tiers (SF2):**
|
||||||
|
- **COMMITTED (P3 exit blocker):** PDF embedded text + typed-text-box search returns results with snippets; tapping a result opens and scrolls to the page/region. This tier alone satisfies the P3 search exit (it has no unverified-recall dependency).
|
||||||
|
- **ADDITIVE (best-effort, NOT a P3 exit blocker):** handwriting OCR contributes hits when the text-line model is available and confident; a word handwritten on page 5 may be returned. **Expectation set:** handwriting recall is limited by the text-line model and may return nothing on cursive/handwriting; math/formula is explicitly P5. P3 ships even if handwriting recall is poor, with UI copy stating the limit.
|
||||||
|
|
||||||
|
### F9 — Server sync + AI refinement (P4)
|
||||||
|
- `sync/sync_client.dart` **[ADD]** — wire the **already-built** FastAPI endpoints (`/api/auth`, `/api/sync/push|pull`, `/api/notes`, `/api/documents`, `/api/ocr`). Last-writer-wins by `updated_at` (server already implements this). Sync notes + (later) notebooks/boards.
|
||||||
|
- `server/badnote_server/` **[MODIFY]** — extend `sync_router`/`models.py` to cover notebooks/boards/links (today only notes). Add an **AI-refine** endpoint that runs llm_wiki/VLM/LLM over a note/page → returns organized markdown (server-side, heavy deps gated like the OCR worker).
|
||||||
|
- `lib/providers/` **[ADD]** sync state provider; settings page server URL/token.
|
||||||
|
|
||||||
|
**Acceptance:** push from device A, pull on device B reproduces notes; AI-refine returns organized text for a selected note; offline still fully functional (server optional, per README).
|
||||||
|
|
||||||
|
### F10 — Ink CAS / formula (P5)
|
||||||
|
- `search/ocr_ingest.dart` **[MODIFY]** — add a **formula/math recognition** backend (new model; the current PP-OCRv4 is text-line only — this is the known hard sub-problem). Behind a feature flag.
|
||||||
|
- `engine/` **[ADD]** `cas/` — recognized formula → CAS (compute/solve) behind a toggle; renders result near the ink. Reuses the `CoordinateSpaceHost` seam for overlay placement.
|
||||||
|
|
||||||
|
**Acceptance (stretch):** a handwritten `2+3=` toggled → shows `5`; recognized formulas become searchable. Gated, optional, lowest priority.
|
||||||
|
|
||||||
|
### F11 — Modern UX polish (woven P0–P3)
|
||||||
|
- Material You already wired (`main.dart` `DynamicColorBuilder` + harmonized schemes + Inter). Continue: subtoolbars in `editor_toolbar`, drag-reorder thumbnails (F4/F6), **hover pre-warm** (warm the next page tile + Picture on stylus hover to cut pen-down latency — reuse the live `_onPointerHover` seam), animated mode/layout transitions.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Data model & persistence (clean schema — data may reset)
|
||||||
|
|
||||||
|
Bump DB to a fresh version with `_onCreate` only (keep `_onUpgrade` harmless). Today: notes/strokes, documents/annotations(JSON-per-page), bookmarks, ocr_results, document_fts(FTS5), scratchpads (DB v5, `lib/services/database_service.dart`).
|
||||||
|
|
||||||
|
**Target tables (additions/changes in bold):**
|
||||||
|
- `documents` **[KEEP]** (drop editor reliance on `rotation`; page-map owns rotation).
|
||||||
|
- **`notebook_pages(id PK, document_id FK, ordinal INTEGER, source_page_index INTEGER NULL, kind TEXT, created_at)`** **[ADD]** — F6 logical pages.
|
||||||
|
- **`ink(id PK, host_kind TEXT, host_id TEXT, stroke_json TEXT, ordinal INTEGER, updated_at)`** **[ADD]** — strokes addressed by host (`host_kind ∈ {page, board}`, `host_id` = `notebook_page_id` or `board_id`). **Per-stroke rows** (not per-page blob) so a 2,000-stroke page doesn't rewrite on every save — **but ONLY valid under the MF3 diff-write contract (§2/F1 `editor_repository.saveHost`): UPSERT changed rows + DELETE removed rows, never delete-all+re-insert.** Without diffing, per-row is *worse* than a blob; the contract is what makes this schema correct. Index `(host_kind, host_id)`.
|
||||||
|
- **`text_boxes(id PK, host_kind, host_id, content TEXT, rect_json, font_size, color, updated_at)`** **[ADD]** — F2-area text + F8 indexing.
|
||||||
|
- **`boards(id PK, document_id FK NULL, title, created_at, updated_at)`** **[ADD]** — generalize `scratchpads`; board strokes live in `ink` with `host_kind='board'`.
|
||||||
|
- **`links(id PK, src_kind, src_id, dst_kind, dst_id, created_at)`** **[ADD]** — 双链; indexes on `(src_kind,src_id)` and `(dst_kind,dst_id)`.
|
||||||
|
- `bookmarks` **[KEEP]**.
|
||||||
|
- `ocr_results` **[KEEP/EXTEND]** — per logical page handwriting OCR text.
|
||||||
|
- **`library_fts` (FTS5)** **[ADD]** — `(ref_kind, ref_id, page, content)` unified search over PDF text + typed text + OCR. `document_fts` **[KEEP]** for PDF-page text.
|
||||||
|
- `notes`/`strokes`/`notes_fts` **[KEEP]** (existing ink-note path; eventually folded into boards, but not deleted in P0).
|
||||||
|
- **`sync_state(entity_kind, entity_id, last_pushed_at, last_pulled_at, dirty)`** **[ADD, P4]**.
|
||||||
|
|
||||||
|
**Coordinate semantics on disk:** page ink = normalized `[0,1]` unrotated; board ink = absolute logical px. (Matches live conventions.)
|
||||||
|
|
||||||
|
**Stroke-model convergence (important):** there are currently **two** stroke models — `PenStroke/PenPoint` (live canvas, in-memory only) and `InkStroke/InkPoint` (freezed/JSON, DB+OCR+export). P0 introduces **one** canonical `EditorStroke/EditorPoint` (freezed/JSON, normalized) and adapters to/from `InkStroke` for OCR/export reuse during transition; old screens keep `InkStroke` until retired (§6).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Refactor strategy (evolve, don't rewrite)
|
||||||
|
|
||||||
|
**Principle: every phase ships; the engine generalizes under load.**
|
||||||
|
|
||||||
|
1. **P0 = relocation + persistence.** Move the 4 live canvas files into `engine/` + `render/` + `input/` with minimal logic change; add `StrokeStore.revision` + `ui.Picture` cache + DB persistence via `editor_repository`. The live single-page editor keeps working throughout. **Keep** `PenStroke` as a thin alias of `EditorStroke` until callers migrate.
|
||||||
|
2. **P0.5 = vertical slice (continuous-single only) — the new perf/crispness gate (see §5/§9).** Stand up `layout/page_viewport` + `pdf/page_tile` rendering continuous-SINGLE only, with the **rewritten** perf bench (targeting `ui/editor_screen`, not the spike) and the zoom-DPI refresh proven crisp at 4× on the Surface. This proves the own-canvas multi-page + multi-resolution model on the real device before any double/paged/spread work.
|
||||||
|
3. **P1 = full multi-page windowing.** Wrap the (now-relocated) `AnnotationLayer` in `page_viewport`; `_strokesByPage` Map → repository-backed mounted hosts. Single-page path stays as `pagedSingle`. Continuous-double / paged / spread land here, AFTER P0.5 passes.
|
||||||
|
4. **Old-code retirement (gated on parity, then delete in one step):**
|
||||||
|
- **[DELETE after P1 parity]** `lib/screens/pdf_annotator_screen.dart`, `lib/widgets/pdf_annotation_layer.dart`, `lib/widgets/ink_canvas.dart` (extract any unique draw/erase logic to `engine/` first), `lib/editor/pdf/spike_*.dart` (throwaway M1 spike — includes the spike-based perf bench, replaced in P0.5), `lib/editor/pdf/pen_capture_region.dart` + `PenCaptureBinding` in `main.dart` (own-canvas `Listener` model won — remove the unused arena-bypass binding once P0 device-confirms).
|
||||||
|
- **[DELETE in P3, NOT P1 (SF3)]** `lib/screens/split_view_screen.dart` — its replacement is the **P3** infinite board (F7); deleting it in P1 would leave a 2-phase functionality gap (no scratchpad between P1 and P3). It stays live until the board lands.
|
||||||
|
- **[KEEP]** `thumbnail_service` until F4 ports it; `stroke_rasterizer` (OCR), `ctc_decoder`, `onnx_recognition_backend`, OCR assets; `pdf_service` (export/mutate, with the fill fix); `pptx_service`/`ppt_annotator_screen` (PPT is separate; not in scope but not deleted).
|
||||||
|
5. **Navigation swap:** `home_screen` `_openDocument`/`_importPdf` currently push `PdfAnnotatorScreen`; `openM1Spike` pushes `PenEditorScreen`. Repoint both to `ui/editor_screen.dart` once P1 parity passes (checklist below).
|
||||||
|
6. **Parity checklist before any delete:** page rotate/delete/insert-blank/insert-image (`pdf_service`), bookmark add/toggle/jump, undo/redo across pages, save-on-leave, zoom in/out/fit, export-matches-screen (golden). Mirrors phase-1 §10 M3 checklist C1–C11.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Phased delivery (each = shippable milestone with exit criteria)
|
||||||
|
|
||||||
|
> Sequence de-risks: pen core + persistence first (P0), then the 60fps multi-mode layout (P1), then config/notebook (P2), then board/双链/search (P3), then sync/AI (P4), then CAS (P5).
|
||||||
|
|
||||||
|
**P0 — Pen core hardened + persisted (own-canvas).** *Exit:* draw/erase/undo/redo on a single PDF page persist to DB and reload; `EditorStroke` canonical model (superset of `InkPoint`, SF1) + revision-gated `ui.Picture` cache; `editor_repository.saveHost` honors the MF3 diff-write contract (test: erase 1 of 2,000 ⇒ 1 DELETE, 0 re-inserts); pure `InputArbiter` + `stroke_eraser` unit-tested; **on-device Surface Pen confirms** pressure + palm rejection + pinch-zoom in the live shared-transform model (the one device gate). Export hairline bug fixed (fill). No regression to existing screens.
|
||||||
|
|
||||||
|
**P0.5 — Vertical slice gate (own-canvas multi-page + multi-resolution), continuous-SINGLE only. [NEW — SYNTHESIS]** Stand up `layout/page_viewport` + `pdf/page_tile` rendering **continuous-single only** on the real document path (`ui/editor_screen`), and:
|
||||||
|
- (a) **REWRITE the perf bench** (§7/§8): a new `integration_test/editor_scroll_bench.dart` drives `ui/editor_screen` + `layout/page_viewport`; **DELETE the spike-based `integration_test/perf_scroll_bench.dart`** (it imports `spike_editor_pane.dart` / `pageOverlaysBuilder` = the invalidated Option B; it CANNOT validate own-canvas).
|
||||||
|
- (b) **Prove crisp-on-zoom (R11):** page-tile DPI refresh on zoom-settle renders crisp text/rules at 4× on the Surface (no GPU-upscale blur).
|
||||||
|
- (c) **Source-pin the pdfrx render/text APIs (SF4):** confirm `PdfPage.loadText`/`charRects`/`PdfPage.render()`/`PdfPageView` signatures present in pdfrx 2.4.4; smoke-test each, so F4/F8 don't hit drift late.
|
||||||
|
- *Exit (hard gate before any double/paged/spread):* **continuous-single median build+raster ≤ 16.6ms, p95 ≤ 22ms** on the 300-page asset via the REWRITTEN bench, **AND** a manual crisp-on-zoom-at-4× PASS on the Surface, **AND** the four pdfrx APIs source-pinned + smoke-tested. Continuous-double / paged / spread do NOT begin until P0.5 is GREEN.
|
||||||
|
|
||||||
|
**P1 — Full layout modes + book-like reading + thumbnails (60fps). Precondition: P0.5 GREEN.** *Exit:* the remaining `PageLayout`s (continuous-double, pagedSingle, pagedDouble); reader vs annotate; page-flip + two-page spread; thumbnail-grid jump + drag-reorder; windowed lazy hosting; **perf gates met** (§7) incl. continuous-double on a 300-page PDF via the rewritten bench; old PDF screens deleted after parity checklist (split_view retirement deferred to P3 per SF3); nav repointed.
|
||||||
|
|
||||||
|
**P2 — Configurable pen + one-notebook-per-PDF.** *Exit:* Pen settings page (button/eraser mapping, curve, palm sensitivity, finger toggle) persists + drives arbiter; insert-blank/reorder pages keep source vector + ink attached to logical pages; portable bundle round-trips; export walks page-map.
|
||||||
|
|
||||||
|
**P3 — Infinite board + 双链 + library search.** *Exit:* board reuses engine at perf; sticky-notes + `[[links]]` produce backlinks; **library search COMMITTED tier** (PDF embedded text + typed-text-box) returns snippets + jump-to-location (this tier is the exit blocker, SF2); **handwriting-OCR tier is ADDITIVE/best-effort and does NOT block P3 exit**; `split_view_screen` retired into board (the SF3 deletion point).
|
||||||
|
|
||||||
|
**P4 — Server sync + AI refinement.** *Exit:* push/pull notes+notebooks across two devices (LWW); AI-refine endpoint returns organized markdown; app fully functional offline.
|
||||||
|
|
||||||
|
**P5 — Formula OCR + ink CAS.** *Exit (stretch, gated):* formula recognition backend; searchable formulas; CAS solve toggle for simple expressions.
|
||||||
|
|
||||||
|
Every milestone ends with verifier/critic pass + perf-results doc updated with commit hash.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Refactor of input transport (record the pivot)
|
||||||
|
The live `PenCanvas` proves the **own-canvas** model: a single `Listener` over an `InteractiveViewer` whose child is `Stack(PdfPageView bitmap, StaticInk, LiveInk)`. Because the pen, page bitmap, and ink share **one** transform and the `Listener` arbitrates by `_activePointers.length` + `kind`, there is **no gesture-arena fight** — pdfrx never sees gestures (it only renders). This **removes** the phase-1 need for `PenCaptureRegion`/`PenCaptureBinding` (RenderProxyBox arena bypass), which exist for the abandoned `pageOverlaysBuilder` approach. **Retirement gate:** delete them once P0 confirms on the physical Surface Pen that the `Listener` model handles stylus draw + single-finger pan + pinch-zoom + palm rejection (the live code is built for exactly this; confirm, then remove the dead binding).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Risks & mitigations
|
||||||
|
|
||||||
|
| # | Risk | L | I | Mitigation / trigger |
|
||||||
|
| --- | --- | --- | --- | --- |
|
||||||
|
| R1 | **60fps continuous + double-page on a 300-page PDF.** Two columns × windowed tiles × ink Pictures may blow frame budget. | Med | High | Windowed lazy hosting (only visible+cacheExtent mounted), bounded LRU + post-frame dispose for BOTH the `ink_picture_cache` and the `page_tile_cache` (R10), per-page `RepaintBoundary`, revision-gated static Picture. **Gate continuous-SINGLE in P0.5, continuous-double in P1** (§5/§9). **MF1 — the existing `integration_test/perf_scroll_bench.dart` is UNUSABLE here: it imports `spike_editor_pane.dart` and benchmarks `pageOverlaysBuilder` (Option B, the INVALIDATED architecture) — it cannot validate own-canvas. P0.5 REWRITES the bench against `ui/editor_screen` + `layout/page_viewport` and DELETES the spike-based one.** Uses `large_300p.pdf`. |
|
||||||
|
| R2 | **Pen feel only verifiable on the user's Surface** (CI has no pen). | High | Med | P0 device gate (pressure/palm/pinch) + manual checklist in perf-results doc; synthesized-stylus widget tests as interim signal only. |
|
||||||
|
| R3 | **Handwriting/formula OCR feasibility.** Current model is text-line PP-OCRv4; math is unsolved. | High | Med | F8 ships text-line best-effort with **explicit expectation-setting**; formula isolated to P5 behind a flag; never block search on OCR quality. |
|
||||||
|
| R4 | **双链 graph scale** (thousands of links/sticky-notes). | Low | Med | Indexed `links` table (both directions), lazy backlink queries, no in-memory full graph. |
|
||||||
|
| R5 | **Server/AI scope creep.** | Med | Med | Server stays optional (README); P4 wires existing endpoints + one AI-refine route; AI heavy deps gated like OCR worker. |
|
||||||
|
| R6 | **Windows pen edge cases** (no advertised pressure range; barrel-button eraser; inverted stylus). | Med | Med | Live `_normalizedPressure` already handles degenerate ranges; F5 makes button/eraser mappable; test matrix in arbiter unit tests. |
|
||||||
|
| R7 | **Export fidelity** (hairline bug today). | High (today) | Med | `stroke_geometry.buildStrokeOutline` shared screen+export; `pdf_service._renderStrokes` **fills** a `PdfPath`; golden test (P0). |
|
||||||
|
| R8 | **Stroke-model convergence churn** (two models today). | Med | Low | One canonical `EditorStroke` + adapters; old `InkStroke` retained only where old screens/OCR/export still use it, deleted with them. |
|
||||||
|
| R9 | **pdfrx page-render / text API drift** (thumbnails, search text, double-page sizing). | Med | Low | **Source-pin `PdfDocument`/`PdfPageView`/`PdfPage.loadText`/`charRects`/`render()` signatures + smoke-test in P0.5 (pulled forward, SF4)** so F4/F8 don't hit drift late; keep syncfusion export until verified. |
|
||||||
|
| R10 | **Cache memory budget vs many mounted pages (TWO caches — Critic).** | Med | Med | **`ink_picture_cache`** (vector `ui.Picture`, no DPI bucket): ~8–12 mounted pages, a few MB. **`page_tile_cache`** (raster `ui.Image`, DPI-bucketed): the heavy store — a 3×-DPI A4 tile is ~10–30 MB, so 8–12 high-DPI tiles would be ~150–350 MB. **The old "≤64MB / 8–12 pages" figure was unit-confused (conflated ink with page bitmaps).** Fix (§2/F2): keep full-DPI tiles only for visible ±1 pages, downgrade off-window pages to a 1× thumbnail tier; ≤64MB applies to ink + downgraded tiles, the small high-DPI tile set budgeted separately (~64–128MB, device-tuned). Sample BOTH caches during the scroll bench. |
|
||||||
|
| R11 | **Blurry page at high zoom under the shared transform (MF2).** One `InteractiveViewer` matrix-scales a `PdfPageView` bitmap that was rendered at **layout-constraint × devicePixelRatio** (1×); pinch-zoom GPU-**upscales** it → blurry text/rules at 4×, whereas pdfrx's own viewer re-renders crisp tiles per zoom level. We **own multi-resolution tiling**, not just layout/windowing. | Med | High | `pdf/page_tile.dart` drives render DPI from the transform scale: on zoom-settle re-instantiate the tile (re-laid-out `PdfPageView` at the new pixel size, or `PdfPage.render()` at target DPI) into the **`pdf/page_tile_cache.dart`** store (`LRU<TileKey{hostId, dpiBucket}, ui.Image>` — NOT the ink cache); cache key includes a DPI bucket; **cap retained DPI** (~3× base) to bound memory; downscale stays matrix-only. **`PdfPage.render()` returns an async `PdfImage` owning a native handle → `page_tile_cache` owns its post-frame dispose** (see Open Questions; confirm in the SF4 P0.5 source-pin). **Proven crisp at 4× on the Surface in the P0.5 gate** (§5). Budgeted P0.5/P1, not relocation. |
|
||||||
|
|
||||||
|
### Pre-mortem (DELIBERATE — 3 scenarios)
|
||||||
|
1. **"Continuous double-page janks / pages are blurry at 4× on the user's big scanned PDF."** Cause: built layout modes before profiling two-column windowing, and matrix-scaled a 1× bitmap (R11). *Prevention:* the **P0.5 vertical-slice gate** profiles continuous-SINGLE first with a **rewritten** bench against `ui/editor_screen` (the spike-based `perf_scroll_bench.dart` is deleted — it benchmarks the invalidated Option B) AND proves crisp-on-zoom DPI refresh at 4× on the Surface; continuous-double/paged/spread merge only after P0.5 is GREEN, and the P1 double-page gate reuses the rewritten bench.
|
||||||
|
2. **"Search returns nothing for handwriting."** Cause: over-promised OCR. *Prevention:* F8 ships PDF-text + typed-text search first (reliable), handwriting OCR as additive best-effort with UI copy stating limits; formula explicitly P5.
|
||||||
|
3. **"Insert-blank-page silently rasterized / detached annotations."** Cause: editing source bytes or keying ink to raw PDF index. *Prevention:* logical page-map + ink keyed to `notebook_page_id`; golden test that the source PDF bytes are unchanged after insert and remains selectable-text in an external viewer.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. Testing strategy
|
||||||
|
|
||||||
|
> **sqlite workaround:** DB-touching tests run via `tool/test.sh` (system sqlite + `LD_LIBRARY_PATH`); pure-logic tests avoid the DB. Reusable assets: `tool/gen_bench_pdf.dart`, `tool/gen_dense_strokes.dart`, `test/assets/large_300p.pdf`, `test/assets/dense_strokes.json`, `integration_test/coordinate_assertion_test.dart`. **NOT reusable: `integration_test/perf_scroll_bench.dart` — it imports `spike_editor_pane.dart` (`pageOverlaysBuilder`, invalidated Option B) and is DELETED + replaced by `integration_test/editor_scroll_bench.dart` (targets `ui/editor_screen`) in P0.5 (MF1).**
|
||||||
|
|
||||||
|
- **Unit (no DB/widgets):**
|
||||||
|
- Coordinate transforms: `NormalizedPageHost`/`BoardHost` round-trip `toContent(toDevice(x))≈x`.
|
||||||
|
- Stroke geometry: `buildStrokeOutline` non-empty for ≥1 point; live ⊆ committed bounds (no "pop").
|
||||||
|
- Eraser: `eraseHits` whole-stroke removal (extraction of live `pen_canvas._eraseAt` — regression port). **`splitStroke` segment-erase is NEW behavior, not a port** (the live canvas only does whole-stroke erase; `undo_manager` records but never computes replacements): mid-erase ⇒ 2 segments, endpoint ⇒ 1 segment, full-erase ⇒ empty, <2-pt result dropped — these exercise greenfield code. (`undo_manager_test.dart` informs the replacement-bookkeeping discipline only.)
|
||||||
|
- Revision gating: `StrokeStore.add` bumps revision; `StaticInkPainter.shouldRepaint` iff revision changed.
|
||||||
|
- `InputArbiter` SM: table-driven over the device×mode matrix incl. palm rejection (touch dropped while stylus active) and `pointerCount>=2`→cancel.
|
||||||
|
- Undo: global commit-time order, host-tagged reversal.
|
||||||
|
- `SaveScheduler`: mutating "current host" after schedule but before write completes does not change persisted snapshot.
|
||||||
|
- **`editor_repository.saveHost` diff-write (MF3, fake DB counting statements):** erase 1 of 2,000 strokes ⇒ exactly **1 DELETE, 0 INSERT**; add 1 stroke ⇒ **1 INSERT, 0 DELETE**; no-op save ⇒ 0 statements. (Guards against the live notes-path delete-all+re-insert anti-pattern at database_service.dart:281–306.)
|
||||||
|
- `EditorStroke`↔`InkStroke` adapter round-trip (SF1): `tilt`/`timestamp`/`pointerDeviceKind` survive both directions (no lossy OCR/export conversion).
|
||||||
|
- `link_graph.backlinksOf` (P3); `search_indexer` snippet/jump-ref mapping for the COMMITTED tier — PDF-text + typed-text (P3).
|
||||||
|
- **Widget:** stylus→stroke committed, touch→not consumed (synthesized pointers); layout-mode switch recenters; reader-mode blocks ink; thumbnail tap jumps; text-box place/move/edit persists (fake DB).
|
||||||
|
- **Perf:** the **rewritten** `integration_test/editor_scroll_bench.dart` (targets `ui/editor_screen` + `layout/page_viewport`; the spike-based `perf_scroll_bench.dart` is deleted, MF1) on `large_300p.pdf` — **continuous-single in P0.5 (gate), continuous-double in P1**, profile mode, N≥120 frames warm, median build+raster ≤16.6ms / p95 ≤22ms; `StaticInkPainter` no-rebuild assertion; Picture-memory ≤64MB sample. Archived in `docs/plans/full-refactor-perf-results.md` with commit hash. Not a hard CI gate (no GPU) but required for milestone sign-off.
|
||||||
|
- **Manual on-device pen checklist (Surface Pen, P0 + each milestone):** pressure varies width; barrel/inverted = erase; palm rest doesn't mark; single-finger scroll; two-finger pinch; **page text/rules stay crisp at 4× zoom — no GPU-upscale blur, DPI refreshes on zoom-settle (R11, P0.5 gate)**; hover pre-warm reduces first-stroke latency; page-flip feels book-like. Recorded with device + commit.
|
||||||
|
- **Regression:** keep `ctc_decoder_test.dart`, `undo_manager_test.dart` green; update `widget_test.dart` to boot `editor_screen`.
|
||||||
|
- **Export golden (P0):** annotate a known page → export → image-compare filled ink matches screen (R7).
|
||||||
|
- **Server (P4):** existing `server/tests/` (`test_sync.py` etc.) green; add notebook/board sync tests.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Milestones / sequencing — immediate next chunk (concrete)
|
||||||
|
|
||||||
|
**Next chunk = P0 (engine relocation + persistence), executable now; followed by the P0.5 vertical-slice gate (steps 10–13) before any double/paged/spread layout work:**
|
||||||
|
1. **[ADD]** `lib/editor/engine/stroke_model.dart` — `EditorStroke`/`EditorPoint` (freezed + JSON), normalized; `fromPenStroke`/`toInkStroke` adapters. Run `build_runner`.
|
||||||
|
2. **[ADD]** `lib/editor/engine/stroke_geometry.dart` — lift `buildStrokePath` → `buildStrokeOutline` (verbatim recipe from `ink_painters.dart`).
|
||||||
|
3. **[ADD]** `lib/editor/engine/stroke_store.dart` + `lib/editor/render/{static_ink_painter,live_ink_painter,ink_picture_cache,annotation_layer}.dart` — relocate live painters; add the resolution-independent ink `ui.Picture` cache keyed by `revision` (NO DPI bucket; the DPI-bucketed `page_tile_cache` is a separate P0.5 file, step 10).
|
||||||
|
4. **[ADD]** `lib/editor/input/input_arbiter.dart` + `lib/editor/engine/stroke_eraser.dart` — extract from `pen_canvas.dart` (pure, unit-tested).
|
||||||
|
5. **[ADD]** `lib/editor/persistence/{editor_repository,save_scheduler}.dart` + DB additions (`ink`, `notebook_pages` minimal) in `database_service.dart` (fresh version). `saveHost` implements the **MF3 diff-write contract** (UPSERT changed + DELETE removed, by stroke id; NO delete-all+re-insert) with a per-host last-persisted id-set.
|
||||||
|
6. **[MODIFY]** `lib/editor/canvas/pen_editor_screen.dart` (or new `ui/editor_screen.dart`) to load/commit/save through the repository instead of the in-memory `_strokesByPage` Map.
|
||||||
|
7. **[MODIFY]** `lib/services/pdf_service.dart` `_renderStrokes` → fill `buildStrokeOutline` path (R7) + export golden test.
|
||||||
|
8. **Tests:** arbiter SM, eraser, geometry, revision-gating, save-scheduler snapshot, **diff-write statement-count (MF3: erase 1/2000 ⇒ 1 DELETE 0 INSERT)**, `EditorStroke↔InkStroke` lossless round-trip (SF1), export golden. Run via `tool/test.sh`.
|
||||||
|
9. **Device gate:** build Windows package (CI), confirm pen/palm/pinch on Surface Pen; record in perf-results doc. → unblocks P0.5.
|
||||||
|
|
||||||
|
**Then P0.5 (vertical-slice gate — must pass before P1's double/paged/spread):**
|
||||||
|
10. **[ADD]** `lib/editor/layout/page_viewport.dart` (continuous-single only) + `lib/editor/pdf/{pdf_document_source,page_tile,page_tile_cache}.dart` with zoom-settle DPI refresh (R11) + the **DPI-bucketed `page_tile_cache`** (`LRU<TileKey, ui.Image>`, page bitmaps; owns native-handle dispose). (The resolution-independent `render/ink_picture_cache.dart` for vector ink lands in P0, step 3 — it is a separate cache, no DPI bucket.)
|
||||||
|
11. **[ADD]** `integration_test/editor_scroll_bench.dart` targeting `ui/editor_screen`; **[DELETE]** `integration_test/perf_scroll_bench.dart` + `lib/editor/pdf/spike_*.dart` (the spike pane the old bench imports). Run the rewritten bench on a **scanned-image** 300-page asset (see Open Questions — `large_300p.pdf` is synthetic/vector and may not honestly stress raster re-render at 3× DPI) → continuous-single median ≤16.6ms / p95 ≤22ms; sample both caches' memory.
|
||||||
|
12. **[VERIFY]** source-pin pdfrx `PdfPage.loadText`/`charRects`/`render()`/`PdfPageView` in 2.4.4 (SF4) + smoke test; **confirm `PdfPage.render()`'s `PdfImage`/native-handle ownership + dispose semantics** so `page_tile_cache` can manage post-frame disposal; record signatures in perf-results doc.
|
||||||
|
13. **Device gate:** crisp-on-zoom at 4× PASS on the Surface (R11). → unblocks P1.
|
||||||
|
|
||||||
|
Each subsequent milestone (P1…P5) follows §5 exit criteria; verifier/critic + perf-results update per milestone.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. RALPLAN-DR
|
||||||
|
|
||||||
|
### Principles (3–5)
|
||||||
|
1. **Single source of truth = host content coordinates.** Screen mapping is a paint-time `canvas` transform; never store/duplicate transformed geometry. (Already the live convention — generalize it.)
|
||||||
|
2. **One host-agnostic ink engine.** PDF page, infinite board, and (P5) CAS overlay are `CoordinateSpaceHost`s behind one renderer — never fork the stroke pipeline.
|
||||||
|
3. **Own the gesture pipeline; pdfrx only renders.** A single `Listener` over a shared `InteractiveViewer` arbitrates draw/pan/zoom/palm by pointer kind + count — no gesture-arena fights (proven live).
|
||||||
|
4. **De-risk performance AND crispness before features.** A **P0.5 vertical-slice gate** (continuous-single, rewritten bench, crisp-at-4× on the Surface) precedes all double/paged/spread work; continuous-double is gated again in P1.
|
||||||
|
5. **Ship every phase; generalize under load.** Relocation-first refactor keeps the editor working at all times; old screens deleted only after parity.
|
||||||
|
|
||||||
|
### Decision Drivers (top 3)
|
||||||
|
1. **D1 — Pen feel + palm rejection + pinch on Windows Surface Pen** is the make-or-break primary-device requirement (own-canvas model already targets it).
|
||||||
|
2. **D2 — 60fps across all layout modes** on big PDFs with thousands of strokes.
|
||||||
|
3. **D3 — One engine reused across PDF page, infinite board, and CAS**, forward-compatible with search/双链/sync.
|
||||||
|
|
||||||
|
### Viable options (≥2) with bounded pros/cons
|
||||||
|
|
||||||
|
**Option A — Own-canvas engine (single `Listener` + `InteractiveViewer`, pdfrx as renderer). CHOSEN (already live).**
|
||||||
|
- Pros: D1 solved structurally (one transform, no arena fight — already working in `pen_canvas.dart`); D3 trivial (hosts share the transform); minimal new deps; matches the proven Saber model the user cited.
|
||||||
|
- Cons: we own page layout/windowing/tiling **AND multi-resolution re-rasterization** (more code than a stock viewer); double-page perf is on us (R1) and crisp-on-zoom is on us (R11 — pdfrx's own viewer re-renders crisp tiles per zoom for free; we must re-derive tile DPI from the transform); thumbnails/text-extraction still need pdfrx page APIs (R9).
|
||||||
|
|
||||||
|
**Option B — pdfrx `pageOverlaysBuilder`-hosted ink + RenderProxyBox arena-bypass (the phase-1 plan).**
|
||||||
|
- Pros: pdfrx gives continuous scroll/tiling/text-extraction for free; ink-follows-page is structural via page overlays.
|
||||||
|
- Cons: requires a custom `PenCaptureBinding` arena bypass that fights pdfrx's greedy scale recognizer (the live code already **abandoned** this — `PenCaptureRegion` is unused); pdfrx owns the transform so double-page/board reuse is awkward; D1 proven harder than Option A in practice. **Invalidated** — see below.
|
||||||
|
|
||||||
|
**Option C — Flutter shell + Rust hot-path (rnote-style) for ink/render.**
|
||||||
|
- Pros: maximal ink perf headroom.
|
||||||
|
- Cons: rejected in memory (`badnote-flutter-344-windows-pen`) — Windows pen is weaker in the Rust/GTK stack; huge FFI surface; contradicts "stay Flutter." **Invalidated.**
|
||||||
|
|
||||||
|
### Invalidation rationale
|
||||||
|
- **B invalidated (as the INPUT model):** the live codebase already moved off it; `pageOverlaysBuilder`+arena-bypass made stylus/touch arbitration fight pdfrx's recognizer, whereas Option A's single-`Listener`-over-shared-transform sidesteps the arena entirely and is already drawing with pressure/pinch/palm. **Correction (MF2): B had TWO edges, not one** — (1) free continuous scroll/windowing, AND (2) free **per-zoom crisp re-rasterization** (pdfrx re-renders tiles at each zoom level). Edge (1) is recoverable in A via windowed hosting (needed for double-page anyway); edge (2) is **NOT free in A** — Option A must own multi-resolution tiling (R11), which is the genuinely new cost of this choice. We accept that cost (gated in P0.5) because A's structural D1/D3 wins outweigh it. B retained only as a fallback page-**render** strategy, not the input model.
|
||||||
|
- **C invalidated:** documented Windows-pen regression in the Rust/GTK path + "stay Flutter" hard constraint; the 3.44 WM_POINTER fix already unblocked Flutter pen, removing C's motivation.
|
||||||
|
- Net: **A chosen; B retained as a partial fallback (page rendering only); C rejected.**
|
||||||
|
|
||||||
|
### ADR
|
||||||
|
- **Status:** Architect APPROVE-WITH-MUST-FIX applied (2026-06-21): MF1 (rewrite perf bench off the invalidated spike), MF2 (R11 zoom re-rasterization + multi-resolution tiling), MF3 (diff-write contract); P0.5 vertical-slice synthesis gate; SF1–SF5. Pending Critic. Records the live pivot from the superseded phase-1 input architecture.
|
||||||
|
- **Decision:** Build the full BadNote vision on an **own-canvas, host-agnostic ink engine** (single `Listener` + shared `InteractiveViewer`; pdfrx as page renderer/text source). Generalize the live `lib/editor/canvas/` into `engine/render/input/layout/...`; persist a single canonical `EditorStroke`; phase features P0→P5 with a **P0.5 vertical-slice gate**.
|
||||||
|
- **Drivers:** D1 Surface-Pen feel, D2 60fps multi-mode, D3 one reusable engine.
|
||||||
|
- **Alternatives considered:** B (pdfrx-overlay + arena bypass — invalidated as input model, kept as render fallback; had two free edges — scroll AND per-zoom crispness — the latter is the new cost we take on), C (Flutter+Rust — rejected).
|
||||||
|
- **Why chosen:** A is already proven live for pen/palm/pinch and gives D1+D3 structurally; the remaining risks (D2 multi-mode perf AND R11 crisp-on-zoom) are gated up front in **P0.5** then re-gated for double-page in P1.
|
||||||
|
- **Consequences:** we own layout/windowing/tiling **AND multi-resolution page-tile re-rasterization** (R11 — not just layout, MF2); `PenCaptureRegion`/`PenCaptureBinding` retired (dead under own-canvas); the spike-based perf bench is deleted and rewritten against `ui/editor_screen` (MF1); per-stroke-row persistence is valid ONLY under the diff-write contract (MF3); `EditorPoint` is a non-lossy superset of `InkPoint` (SF1); export must fill (not stroke) ink; handwriting OCR is additive/non-blocking and formula is P5 (SF2); `split_view_screen` deletion deferred to P3 (SF3); server stays optional.
|
||||||
|
- **Follow-ups:** P0 device gate confirms pen model; **P0.5 gate** proves continuous-single perf + crisp-at-4× + source-pinned pdfrx APIs (SF4) before any double/paged/spread; resolve open questions below.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Open Questions (persist to `.omc/plans/open-questions.md`)
|
||||||
|
- [ ] Continuous-double-page on a real 300-page scanned PDF — does windowed two-column hosting hold 60fps, or do we need tile pre-rasterization? — *P0.5 gates continuous-single first; P1 perf gate decides double; affects R1/R10.*
|
||||||
|
- [ ] **How is page-tile DPI refreshed on zoom under the shared transform (SF5)?** — re-laid-out `PdfPageView` at the new pixel size, or `PdfPage.render()` at target DPI into `page_tile_cache`? What scale-change threshold + debounce triggers a refresh, and what is the retained-DPI cap? — *Load-bearing for the 60fps + crisp-at-4× goal; resolved in P0.5 (R11).*
|
||||||
|
- [ ] **`PdfPage.render()` native-handle lifecycle (Critic 4a):** `render()` returns an async `PdfImage` owning a native handle backing a `ui.Image` — `page_tile_cache` must own its dispose (post-frame, after the raster thread is done) on eviction. *Confirm exact ownership + dispose API in the SF4 P0.5 source-pin; affects page_tile_cache + R10/R11.*
|
||||||
|
- [ ] **Honest R11 raster stress asset (Critic 4b):** does the synthetic/vector `large_300p.pdf` actually force RASTER re-render cost at 3× DPI, or does a **scanned-image** asset better exercise the crispness + tile-memory gate? — *Add a scanned 300-page asset (`tool/gen_bench_pdf.dart` image mode or a real scan) for the P0.5/P1 perf+crispness gates; affects §8 / R11 honesty.*
|
||||||
|
- [ ] Stroke-model convergence: fold `notes`/`InkStroke` ink-note path into `boards`/`EditorStroke`, or keep notes separate long-term? — *Affects §3/§4 churn.*
|
||||||
|
- [ ] Handwriting OCR: is the existing PP-OCRv4 text-line model usable on cursive/handwriting at all, or do we need a handwriting-specific model even for non-formula text? — *Affects F8 expectation-setting / R3.*
|
||||||
|
- [ ] Portable bundle relink: match source PDF by content-hash only, or also store original path + size? — *Affects F6.*
|
||||||
|
- [ ] Server AI-refine: run llm_wiki/VLM server-side only, or allow a local LLM path for offline users? — *Affects F9 scope / R5.*
|
||||||
|
- [ ] Keep `pptx_service`/`ppt_annotator_screen` in the new engine, or freeze PPT support? — *Out of the 11 features; decide before P1 nav swap.*
|
||||||
134
docs/plans/2026-06-22-badnote-pen-polish.md
Normal file
134
docs/plans/2026-06-22-badnote-pen-polish.md
Normal file
@@ -0,0 +1,134 @@
|
|||||||
|
# BadNote — Pen-Polish + Native-Pen Addendum (ralplan consensus)
|
||||||
|
|
||||||
|
**Status:** APPROVED (ralplan consensus 2026-06-22 — Architect APPROVE-WITH-MUST-FIX M1–M4 applied; Critic ITERATE→APPROVE after C1 PenPoint.tilt wrong-model fix + C2 eraser-race re-grounded on M1 native ordering + no-hover-down test)
|
||||||
|
**Date:** 2026-06-22
|
||||||
|
**Mode:** DELIBERATE (native Windows plugin = new platform code; can only be device-verified)
|
||||||
|
**Owner plan file:** `docs/plans/2026-06-22-badnote-pen-polish.md`
|
||||||
|
**Extends (does NOT supersede):** `docs/plans/2026-06-21-badnote-full-refactor.md`
|
||||||
|
|
||||||
|
> **Grounding (read live code 2026-06-22):** `lib/editor/canvas/{pen_canvas,pen_editor_screen,ink_painters,pen_stroke}.dart`,
|
||||||
|
> `lib/editor/engine/stroke_geometry.dart`, `lib/editor/input/pen_config.dart`,
|
||||||
|
> `windows/runner/{ocr_channel.cpp,flutter_window.cpp,win32_window.cpp,main.cpp}`,
|
||||||
|
> `windows/flutter/generated_plugin_registrant.cc`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 0. Scope (4 user asks, mapped to the roadmap)
|
||||||
|
|
||||||
|
| # | User ask (verbatim intent) | Root cause (verified) | Roadmap fit |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| **W1** | 添加自定义笔粗; `thinning` 肯定不能写死, 学习 Saber | `penWidth`/`highlighterWidth` already read from `PenConfig` (pen_editor_screen.dart:456–459); **but `thinning` is a hardcoded literal `0.85`** in `ink_painters.buildStrokePath` (line 38) AND `stroke_geometry.buildStrokeOutline` (line 52). | Pull **F5** (configurable pen) width+pressure slice forward to **now** |
|
||||||
|
| **W2** | 缩放的时候会闪一下 (zoom flickers once) | `PdfPageView` lives inside the `InteractiveViewer` child subtree (pen_canvas.dart:354); pdfrx re-rasterizes its page bitmap when the effective scale changes, showing a **one-frame white gap** during the async re-render = the flicker. | This IS **R11 / `pdf/page_tile`** (P0.5); add a **minimal double-buffer fix now**, full fix in P0.5 |
|
||||||
|
| **W3** | 修好 tilt 和笔按键映射 (fix tilt + pen button mapping) | **No native pen plugin exists** (only `badnote/ocr` MethodChannel). Flutter 3.44 Windows delivers `pressure` but NOT barrel→`buttons`, NOT eraser/inverted→`invertedStylus`, NOT `tilt`. So `_isEraserSignal` (pen_canvas.dart:134) never fires; `event.tilt` is always `0.0`. | **NEW work** the full-refactor plan did not budget: a native Windows pen plugin. Gates F5 button-mapping. |
|
||||||
|
| **W4** | 继续推进整体重构, 完成我所有的需求 | Roadmap exists & is approved; task #7 (P0 engine+persistence) `in_progress`. | Resume `2026-06-21-badnote-full-refactor.md` P0 → P0.5 → … after W1–W3. |
|
||||||
|
|
||||||
|
**Non-goals here:** no new layout modes, board, search, server, or CAS (those stay in the parent roadmap's P1–P5). This addendum is only the pen-feel polish + the native-pen unblock that the user is blocked on *today*, sequenced so it feeds the parent plan's engine (`stroke_geometry`, `input_arbiter`, `pen_config`) rather than the throwaway live widgets.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. RALPLAN-DR
|
||||||
|
|
||||||
|
### Principles
|
||||||
|
1. **Touch the canonical engine, not the live widgets.** Width/thinning changes land in `engine/stroke_geometry.dart` (the single source for screen+export, §2/F1 of the parent plan), so the fix survives the P0 relocation and exports match the screen. Do not fork logic into `ink_painters.dart` only.
|
||||||
|
2. **No hardcoded feel constants.** `thinning`, `size`, taper, and pressure-sensitivity are `PenConfig` fields with sane defaults — mirroring Saber's `StrokeOptions`-per-pen model.
|
||||||
|
3. **Native pen is additive + degrades gracefully.** The plugin enriches pointer events with barrel/inverted/tilt; if it is absent or returns nothing, the canvas behaves exactly as today (Flutter pressure still works). Never make drawing depend on the plugin.
|
||||||
|
4. **Device-gate the un-CI-testable.** Tilt/buttons/flicker can only be confirmed on the Surface; each ships behind a CI package + a manual checklist, never claimed "done" from a green analyze.
|
||||||
|
|
||||||
|
### Decision Drivers
|
||||||
|
1. **D1 — Unblock the user's primary device today** (eraser-end + side-button + tilt are dead; pen feel needs a real size/pressure control).
|
||||||
|
2. **D2 — Don't derail the approved refactor** — every change feeds `engine/`+`input/`+`pen_config`, not the soon-retired widgets.
|
||||||
|
3. **D3 — Crisp, flicker-free zoom** without prematurely building the whole multi-resolution tiler (that's P0.5).
|
||||||
|
|
||||||
|
### Viable options
|
||||||
|
|
||||||
|
**W1 — configurable thinning/size**
|
||||||
|
- **Option A (CHOSEN): thread `thinning`/`size` as parameters from `PenConfig` through `buildStrokeOutline`/`buildStrokePath`; add `pressureSensitivity` (→thinning) + reuse existing `penWidth`/`highlighterWidth` (→size); Pen-settings sliders.** Pros: matches Saber (`StrokeOptions(size, thinning, …)` per pen); one source of truth; tiny diff. Cons: 2 signatures change + every caller.
|
||||||
|
- Option B: keep literals, expose only `penWidth`. Rejected — user explicitly says thinning 不能写死.
|
||||||
|
|
||||||
|
**W2 — zoom flicker**
|
||||||
|
- **Option A (CHOSEN, now): double-buffer the page bitmap** — keep the last successfully-rendered page image painted underneath `PdfPageView` (or wrap in a tiny `RawImage` cache) so the async re-render never exposes a white frame; **then** the full R11 `page_tile` in P0.5 replaces it. Pros: kills the visible flicker immediately with a small, localized change; forward-compatible (becomes `page_tile`'s double-buffer). Cons: a stopgop that P0.5 supersedes.
|
||||||
|
- Option B: jump straight to the full `pdf/page_tile` + `page_tile_cache` now. Rejected for *now* — it's the P0.5 gate; pulling all of it forward derails P0. (We DO confirm the flicker root cause via systematic-debugging before coding either.)
|
||||||
|
|
||||||
|
**W3 — native pen (tilt + buttons)**
|
||||||
|
- **Option A (CHOSEN): in-app Windows plugin in `windows/runner/` (pen_channel.cpp) subclassing the window proc to handle `WM_POINTER*`, call `GetPointerPenInfo`/`GetPointerPenInfoHistory` for `penFlags` (BARREL/INVERTED/ERASER) + `tiltX/tiltY`, key state by `pointerId`, forward to Dart via an `EventChannel('badnote/pen')`.** Dart `PenInputService` exposes the latest per-pointer pen state; `InputArbiter`/canvas reads it to set eraser + tilt. Pros: smallest footprint (mirrors existing `ocr_channel.cpp` pattern); no new pub package; full control of WM_POINTER. Cons: native C++ to maintain; CI-build-only, device-verify-only.
|
||||||
|
- Option B: standalone federated Flutter plugin package. Rejected — heavier scaffolding for a single-platform need; `ocr_channel.cpp` proves the in-runner pattern works here.
|
||||||
|
- Option C: wait for Flutter engine to deliver penFlags/tilt upstream. Rejected — unbounded; user blocked now.
|
||||||
|
|
||||||
|
### Pre-mortem (3 scenarios)
|
||||||
|
1. **"Tilt/buttons still dead after the plugin ships."** Cause: WM_POINTER not reaching our handler, or Flutter's own `FlutterWindow` consumes the message first. *Prevention:* before writing the EventChannel, add a WM_POINTER **logging probe** in the window proc and confirm on-device that `GetPointerPenInfo` returns non-zero `penFlags`/tilt (systematic-debugging Phase-1 evidence at the component boundary). Only then wire the channel.
|
||||||
|
2. **"Zoom flicker fix made scrolling janky / doubled memory."** Cause: keeping a full-res second bitmap per page. *Prevention:* hold exactly ONE last-good image for the *current* page only; drop it on page change; measure frame cost on the Surface before/after.
|
||||||
|
3. **"Width/thinning change broke export goldens."** Cause: only `ink_painters.dart` was updated, `stroke_geometry.dart` (export path) drifted. *Prevention:* change BOTH via the shared `kDefaultPenThinning`; default stays `0.85` (M4) so the existing golden is unchanged; add a unit test that both builders read identical thinning for the same `PenConfig`. If a golden must be regenerated, name + commit the new baseline explicitly.
|
||||||
|
4. **"Eraser end doesn't erase on a direct pen-down (no hover)."** Cause: correctness was hung on hover-precedence instead of the M1 native ordering. *Prevention:* re-grounded above on observer-before-`HandleTopLevelWindowProc`; **on-device test: tap the eraser end straight onto the page with no prior hover — first contact must erase, not draw.**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Work items (file-level)
|
||||||
|
|
||||||
|
### W1 — Configurable pen width + pressure sensitivity (Saber-style)
|
||||||
|
**Modify:**
|
||||||
|
- `lib/editor/input/pen_config.dart` **[MODIFY]** — add `double pressureSensitivity` (maps to perfect_freehand `thinning`; range `[0,1]`, **default `0.85` = the current live value, M4**, so existing stroke feel and the export golden are preserved; `0` = constant width). Keep `penWidth`/`highlighterWidth` as size. Add to `copyWith`/`toJson`/`fromJson`/`==`/`hashCode` + `PenConfigController.setPressureSensitivity` (clamped). Additive persisted field (default-filled on missing key — no migration).
|
||||||
|
- `lib/editor/engine/stroke_geometry.dart` **[MODIFY]** — define `const double kDefaultPenThinning = 0.85;` (M4 — NOT 0.6; preserves goldens). `buildStrokeOutline(..., {required bool isComplete, double thinning = kDefaultPenThinning})`; remove the `0.85` literal in favour of the named const. Highlighter still forces `0.0`.
|
||||||
|
- `lib/editor/canvas/ink_painters.dart` **[MODIFY]** — `buildStrokePath(..., {required bool isComplete, double thinning = kDefaultPenThinning})`; import `kDefaultPenThinning` from `stroke_geometry.dart` so screen+export share ONE default and can never diverge.
|
||||||
|
- `lib/editor/canvas/pen_canvas.dart` **[MODIFY]** — accept a `thinning` field on `PenCanvas`; pass it to the painters (the painters need the value at paint time → pass via the painter constructors `StaticInkPainter`/`LiveInkPainter`, add `shouldRepaint` check on `thinning`).
|
||||||
|
- `lib/editor/canvas/pen_editor_screen.dart` **[MODIFY]** — feed `_penConfig?.value.pressureSensitivity ?? kDefaultPenThinning` into `PenCanvas.thinning`.
|
||||||
|
- `lib/editor/ui/pen_settings_page.dart` **[MODIFY]** — add a **pen size** slider (drives `penWidth`, e.g. `0.002–0.02`), a **highlighter size** slider, and a **pressure sensitivity** slider (drives `pressureSensitivity` `0–1`). Live-preview stroke swatch optional.
|
||||||
|
|
||||||
|
**Acceptance:** changing pressure-sensitivity to 0 yields constant-width strokes; to ~0.8 makes light/hard press sweep width visibly; pen-size slider changes base width; values persist across restart; a unit test asserts `buildStrokeOutline` and `buildStrokePath` use the same thinning for a given config; export golden still matches screen (R7).
|
||||||
|
|
||||||
|
### W2 — Zoom flicker — **ROOT-CAUSE FIRST, fix is probe-gated (M3)**
|
||||||
|
The flicker fix is NOT pre-committed to a double-buffer. The Architect notes the page is rendered **once** and matrix-scaled (`pen_editor_screen.dart:466–474`), so the cause may be **R11-class** (a re-raster swap / matrix blur on zoom-settle) rather than an async white-gap — and the right fix differs per cause.
|
||||||
|
|
||||||
|
**Step 1 — Phase-1 probe (systematic-debugging, MANDATORY before any fix):** instrument one zoom on the Surface and determine which of these the "闪一下" is:
|
||||||
|
- (a) **async white-gap** — `PdfPageView` blanks for a frame while it re-rasterizes at the new scale; OR
|
||||||
|
- (b) **re-raster swap** — pdfrx renders a fresh higher-DPI bitmap and swaps it in (brief tone/size pop) = R11 territory; OR
|
||||||
|
- (c) **rebuild flash** — a `setState`/`_needsCenter` post-frame callback (`pen_editor_screen.dart:441–448`) or `ValueKey(_pageIndex)` re-mounts the subtree.
|
||||||
|
Record the verdict + a frame capture in the perf-results doc.
|
||||||
|
|
||||||
|
**Step 2 — fix chosen by cause:**
|
||||||
|
- If **(a)**: keep the last good `ui.Image` painted as an underlay for the **current page only**, swapped atomically when the new render is ready (the seed of P0.5 `page_tile`'s double-buffer — link the TODO so it's replaced, not duplicated). **Caveat (Architect): this can race pdfrx's own internal raster cache** — verify the underlay sits *below* `PdfPageView` and is only shown while the live raster is absent.
|
||||||
|
- If **(b)**: this is R11 — do NOT build a bespoke fix; the cheap interim is "keep the page painted across the swap" and the real fix is the P0.5 `page_tile` DPI-on-settle. Defer, note in the roadmap.
|
||||||
|
- If **(c)**: remove the spurious rebuild (gate `_needsCenter`, avoid re-keying on zoom) — cheapest of all.
|
||||||
|
|
||||||
|
**Acceptance:** root cause documented; pinch-zoom on the Surface shows **no flash**; frame cost unchanged (≤16.6ms median) before/after; if deferred to P0.5, that decision is recorded with evidence (not silently dropped).
|
||||||
|
|
||||||
|
### W3 — Native Windows pen plugin (tilt + barrel/eraser → buttons)
|
||||||
|
|
||||||
|
**Hook point (MUST-FIX M1):** `FlutterWindow::MessageHandler` calls `flutter_controller_->HandleTopLevelWindowProc(...)` **first** and returns early when Flutter handles the message (`windows/runner/flutter_window.cpp:56–64`). Flutter 3.44 itself consumes WM_POINTER to synthesize stylus events, so a `switch` *after* that call (line 66) — and the `ocr_channel` registration site at `:29` — **never see consumed pen messages**. The observer therefore reads pen info at the **TOP of `MessageHandler`, before** `HandleTopLevelWindowProc`, observing without consuming (do not `return` a result; fall through so Flutter still processes its event).
|
||||||
|
|
||||||
|
**Correlation (MUST-FIX M2):** do NOT build `Map<Win32 pointerId → state>` and join it to Flutter's `event.pointer` — they are **different id spaces**. Only one pen is active at a time, so latch a **single "current stylus hardware state"** natively, update it on every observed WM_POINTER (incl. hover/`WM_POINTERENTER`), and read it at the decision points in Dart.
|
||||||
|
|
||||||
|
**Why the eraser decision is NOT racy (re-grounded per Critic #2 — do NOT rely on hover-precedence):** the guarantee is the **M1 native ordering**, not "hover precedes down." A pen that contacts the screen directly with no hover dwell delivers `WM_POINTERDOWN` as its first message. Our observer runs at the **top of `MessageHandler`** (M1), so it latches that down's `penFlags` **before** `HandleTopLevelWindowProc` synthesizes the corresponding Flutter pointer-down — therefore when Dart's `_onPointerDown` reads `PenInputService.current`, the latch already reflects this exact contact. The EventChannel push is async, but the **native latch is updated synchronously in the same window-proc pass that precedes Flutter's event**; so the Dart service must source the eraser flag from a value guaranteed fresh by that ordering (i.e. the channel delivers the down-flags before/with the Dart down event because both originate from the same WM_POINTERDOWN, observer-first). Tilt is per-point and tolerant of one-frame lag. **This must be proven on-device with a pen-down-without-hover test (see pre-mortem #4).**
|
||||||
|
|
||||||
|
**Threading (Architect):** the observer runs on the **platform (UI) thread** inside the window proc. Do **NOT** copy `ocr_channel.cpp`'s MTA worker-thread model (`ocr_channel.cpp:73–118`) — that pattern is for the long-running OCR call, wrong for low-latency per-event pen state. Latch + post to the channel sink directly on the platform thread.
|
||||||
|
|
||||||
|
**Add/modify:**
|
||||||
|
- `windows/runner/pen_channel.{h,cpp}` **[ADD]** — `ObservePenMessage(message, wparam, lparam)`: if `message ∈ {WM_POINTERENTER, WM_POINTERDOWN, WM_POINTERUPDATE, WM_POINTERUP}`, `GET_POINTERID_WPARAM(wparam)` → `GetPointerType` → if `PT_PEN`, `GetPointerPenInfo(id, &POINTER_PEN_INFO)`; read `penFlags` (`PEN_FLAG_BARREL`, `PEN_FLAG_INVERTED`, `PEN_FLAG_ERASER`) + `tiltX`/`tiltY`. Latch into a native singleton AND push `{flags, tiltX, tiltY}` over `EventChannel('badnote/pen')`. Returns void; never consumes.
|
||||||
|
- `windows/runner/flutter_window.cpp` **[MODIFY]** — call `ObservePenMessage(message, wparam, lparam)` at the **top of `MessageHandler` (before** the `HandleTopLevelWindowProc` block, M1); register the channel alongside `RegisterOcrChannel` in `OnCreate` (`:29`).
|
||||||
|
- `lib/editor/input/pen_input_service.dart` **[ADD]** — listens to `EventChannel('badnote/pen')`; holds the **latest single** `PenHardwareState{barrel, inverted, eraser, tiltX, tiltY}` (not keyed by pointerId, M2). No-op / empty on non-Windows or a silent channel.
|
||||||
|
- `lib/editor/canvas/pen_stroke.dart` **[MODIFY] (Critic #1 — wrong-model fix):** the live canvas captures `PenPoint{x, y, pressure}` (`pen_stroke.dart:15-21`) which has **no `tilt` field** — `EditorPoint.tilt` lives in the *separate* engine model the live widget does not use. Add `double? tilt` to `PenPoint` now, populate it at capture, and map it through `EditorStroke.fromPenStroke` → `EditorPoint.tilt` (SF1) so tilt is lossless end-to-end on the live path. (When the canvas migrates to `EditorStroke` in P0, `PenPoint` retires and this collapses to `EditorPoint.tilt` directly.)
|
||||||
|
- `lib/editor/canvas/pen_canvas.dart` **[MODIFY]** — replace the dead `_isEraserSignal` (`pen_canvas.dart:134–136`, `kSecondaryButton||invertedStylus` never fires on Windows): on hover/down read `PenInputService.current` and resolve eraser/undo/etc. through the configured `PenConfig.sideButton`/`eraserEnd` mapping; stash `tiltX/Y` into the captured `PenPoint.tilt` (added above).
|
||||||
|
- `lib/editor/input/pen_config.dart` — `sideButton`/`eraserEnd` `PenButtonAction` already exist; the canvas now actually consults them for plugin-delivered flags.
|
||||||
|
|
||||||
|
**Phase-1 probe before wiring (pre-mortem #1):** first land ONLY a WM_POINTER logging line in the observer and confirm on the Surface that `GetPointerPenInfo` returns non-zero `penFlags`/tilt. Only then add the EventChannel + Dart wiring.
|
||||||
|
|
||||||
|
**Acceptance (device-gated):** on the Surface, the eraser end erases; the barrel button performs its mapped `PenButtonAction`; remapping side-button→undo makes the barrel undo; the diagnostic readout shows non-zero `tilt`; with the channel silent the app still draws with pressure (graceful degradation). CI builds the Windows package green; correctness confirmed only on-device + recorded with commit hash.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Sequencing
|
||||||
|
|
||||||
|
1. **W1** (pure Dart, CI-testable, low risk) — ship first; unblocks "pen feel" immediately.
|
||||||
|
2. **W2 investigate → minimal fix** — systematic-debugging Phase 1 evidence, then the double-buffer.
|
||||||
|
3. **W3 native plugin** — biggest/native; (a) WM_POINTER logging probe → device-confirm `GetPointerPenInfo` returns flags+tilt; (b) EventChannel + Dart service; (c) wire eraser/tilt + `PenConfig` mappings.
|
||||||
|
4. **W4** — resume the parent roadmap P0 (task #7) → P0.5. W1's `stroke_geometry` change and W3's `pen_input_service` are written to land in `engine/`+`input/` so the P0 relocation absorbs them rather than re-doing them.
|
||||||
|
|
||||||
|
Each of W1/W2/W3 is an independently shippable CI package; W2/W3 carry a manual Surface checklist before "done."
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. ADR
|
||||||
|
- **Decision:** Pull the **width+pressure-sensitivity** slice of F5 forward (de-hardcode `thinning` in the canonical geometry), fix the zoom flicker with a **double-buffer stopgap** that seeds the P0.5 `page_tile`, and add a **native Windows pen plugin** (`pen_channel.cpp` + `EventChannel('badnote/pen')`) to recover barrel/eraser/tilt that the Flutter 3.44 engine drops — all feeding the approved `engine/`/`input/` seams, then resume the parent roadmap.
|
||||||
|
- **Drivers:** D1 unblock primary device today; D2 don't derail the refactor; D3 flicker-free zoom without prematurely building the full tiler.
|
||||||
|
- **Alternatives:** literals-only width (rejected — user), full `page_tile` now (deferred to P0.5), federated plugin package (rejected — `ocr_channel` in-runner pattern suffices), wait-for-upstream (rejected — blocked now).
|
||||||
|
- **Consequences:** one new native file (Windows-only, device-gated); a temporary page double-buffer superseded by P0.5; `PenConfig` gains `pressureSensitivity`; `thinning` defaults centralized as `kDefaultPenThinning`.
|
||||||
|
- **Follow-ups:** W2 stopgap deleted when `page_tile` lands; W3 eraser/tilt wiring moves into `input_arbiter` during P0; confirm `GetPointerPenInfo` tilt units + sign on-device.
|
||||||
@@ -7,13 +7,21 @@
|
|||||||
import 'package:flutter/material.dart';
|
import 'package:flutter/material.dart';
|
||||||
import 'package:perfect_freehand/perfect_freehand.dart' as pf;
|
import 'package:perfect_freehand/perfect_freehand.dart' as pf;
|
||||||
|
|
||||||
|
import '../engine/stroke_geometry.dart' show kDefaultPenThinning;
|
||||||
import 'pen_stroke.dart';
|
import 'pen_stroke.dart';
|
||||||
|
|
||||||
/// Builds a filled outline [Path] for one stroke (already scaled to pixels).
|
/// Builds a filled outline [Path] for one stroke (already scaled to pixels).
|
||||||
///
|
///
|
||||||
/// [pageSize] maps normalized coords to pixels. [isComplete] should be false
|
/// [pageSize] maps normalized coords to pixels. [isComplete] should be false
|
||||||
/// for the in-progress live stroke so freehand tapers correctly.
|
/// for the in-progress live stroke so freehand tapers correctly. [thinning] is
|
||||||
Path buildStrokePath(PenStroke stroke, Size pageSize, {required bool isComplete}) {
|
/// the pressure→width response (shared default [kDefaultPenThinning]); the
|
||||||
|
/// SAME value drives the export path so screen and PDF never diverge.
|
||||||
|
Path buildStrokePath(
|
||||||
|
PenStroke stroke,
|
||||||
|
Size pageSize, {
|
||||||
|
required bool isComplete,
|
||||||
|
double thinning = kDefaultPenThinning,
|
||||||
|
}) {
|
||||||
final pixelWidth = stroke.width * pageSize.width;
|
final pixelWidth = stroke.width * pageSize.width;
|
||||||
|
|
||||||
final hasRealPressure = stroke.points.any((p) => p.pressure != null);
|
final hasRealPressure = stroke.points.any((p) => p.pressure != null);
|
||||||
@@ -21,7 +29,7 @@ Path buildStrokePath(PenStroke stroke, Size pageSize, {required bool isComplete}
|
|||||||
|
|
||||||
final pfPoints = stroke.points
|
final pfPoints = stroke.points
|
||||||
.map(
|
.map(
|
||||||
(p) => pf.Point(
|
(p) => pf.PointVector(
|
||||||
p.x * pageSize.width,
|
p.x * pageSize.width,
|
||||||
p.y * pageSize.height,
|
p.y * pageSize.height,
|
||||||
p.pressure ?? 0.5,
|
p.pressure ?? 0.5,
|
||||||
@@ -31,23 +39,27 @@ Path buildStrokePath(PenStroke stroke, Size pageSize, {required bool isComplete}
|
|||||||
|
|
||||||
final outline = pf.getStroke(
|
final outline = pf.getStroke(
|
||||||
pfPoints,
|
pfPoints,
|
||||||
size: pixelWidth,
|
options: pf.StrokeOptions(
|
||||||
// Highlighter keeps a constant width (no thinning); pen thins like the
|
size: pixelWidth,
|
||||||
// existing ink_canvas (_drawFreehand uses 0.7).
|
// Highlighter keeps a constant width (no thinning); pen uses the
|
||||||
thinning: isHighlighter ? 0.0 : 0.7,
|
// configurable [thinning] so real Surface-Pen pressure changes width.
|
||||||
smoothing: 0.5,
|
thinning: isHighlighter ? 0.0 : thinning,
|
||||||
streamline: 0.5,
|
smoothing: 0.5,
|
||||||
// Real stylus pressure → don't simulate; no pressure → let freehand fake
|
streamline: 0.5,
|
||||||
// it based on velocity (matches ink_canvas behavior).
|
// Real stylus pressure → don't simulate; no pressure → let freehand fake
|
||||||
simulatePressure: !hasRealPressure && !isHighlighter,
|
// it based on velocity. (perfect_freehand 2.x honors REAL pressure when
|
||||||
isComplete: isComplete,
|
// simulatePressure is false — 1.0.4 ignored it, which made width
|
||||||
|
// unresponsive to pen force.)
|
||||||
|
simulatePressure: !hasRealPressure && !isHighlighter,
|
||||||
|
isComplete: isComplete,
|
||||||
|
),
|
||||||
);
|
);
|
||||||
|
|
||||||
final path = Path();
|
final path = Path();
|
||||||
if (outline.isEmpty) return path;
|
if (outline.isEmpty) return path;
|
||||||
path.moveTo(outline.first.x, outline.first.y);
|
path.moveTo(outline.first.dx, outline.first.dy);
|
||||||
for (var i = 1; i < outline.length; i++) {
|
for (var i = 1; i < outline.length; i++) {
|
||||||
path.lineTo(outline[i].x, outline[i].y);
|
path.lineTo(outline[i].dx, outline[i].dy);
|
||||||
}
|
}
|
||||||
path.close();
|
path.close();
|
||||||
return path;
|
return path;
|
||||||
@@ -56,15 +68,23 @@ Path buildStrokePath(PenStroke stroke, Size pageSize, {required bool isComplete}
|
|||||||
/// Paints all committed strokes for the page. Repaints only when the stroke
|
/// Paints all committed strokes for the page. Repaints only when the stroke
|
||||||
/// list identity or page size changes (kept behind a RepaintBoundary).
|
/// list identity or page size changes (kept behind a RepaintBoundary).
|
||||||
class StaticInkPainter extends CustomPainter {
|
class StaticInkPainter extends CustomPainter {
|
||||||
StaticInkPainter({required this.strokes, required this.pageSize});
|
StaticInkPainter({
|
||||||
|
required this.strokes,
|
||||||
|
required this.pageSize,
|
||||||
|
this.thinning = kDefaultPenThinning,
|
||||||
|
});
|
||||||
|
|
||||||
final List<PenStroke> strokes;
|
final List<PenStroke> strokes;
|
||||||
final Size pageSize;
|
final Size pageSize;
|
||||||
|
|
||||||
|
/// Pressure→width response shared with the live/export paths.
|
||||||
|
final double thinning;
|
||||||
|
|
||||||
@override
|
@override
|
||||||
void paint(Canvas canvas, Size size) {
|
void paint(Canvas canvas, Size size) {
|
||||||
for (final stroke in strokes) {
|
for (final stroke in strokes) {
|
||||||
final path = buildStrokePath(stroke, pageSize, isComplete: true);
|
final path =
|
||||||
|
buildStrokePath(stroke, pageSize, isComplete: true, thinning: thinning);
|
||||||
if (path.getBounds().isEmpty) continue;
|
if (path.getBounds().isEmpty) continue;
|
||||||
canvas.drawPath(
|
canvas.drawPath(
|
||||||
path,
|
path,
|
||||||
@@ -80,23 +100,32 @@ class StaticInkPainter extends CustomPainter {
|
|||||||
bool shouldRepaint(StaticInkPainter old) =>
|
bool shouldRepaint(StaticInkPainter old) =>
|
||||||
!identical(old.strokes, strokes) ||
|
!identical(old.strokes, strokes) ||
|
||||||
old.strokes.length != strokes.length ||
|
old.strokes.length != strokes.length ||
|
||||||
old.pageSize != pageSize;
|
old.pageSize != pageSize ||
|
||||||
|
old.thinning != thinning;
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Paints just the in-progress stroke (the live layer), kept behind its own
|
/// Paints just the in-progress stroke (the live layer), kept behind its own
|
||||||
/// RepaintBoundary so committed strokes don't repaint on every move.
|
/// RepaintBoundary so committed strokes don't repaint on every move.
|
||||||
class LiveInkPainter extends CustomPainter {
|
class LiveInkPainter extends CustomPainter {
|
||||||
LiveInkPainter({required this.stroke, required this.pageSize});
|
LiveInkPainter({
|
||||||
|
required this.stroke,
|
||||||
|
required this.pageSize,
|
||||||
|
this.thinning = kDefaultPenThinning,
|
||||||
|
});
|
||||||
|
|
||||||
/// Current in-progress stroke, or null when nothing is being drawn.
|
/// Current in-progress stroke, or null when nothing is being drawn.
|
||||||
final PenStroke? stroke;
|
final PenStroke? stroke;
|
||||||
final Size pageSize;
|
final Size pageSize;
|
||||||
|
|
||||||
|
/// Pressure→width response shared with the static/export paths.
|
||||||
|
final double thinning;
|
||||||
|
|
||||||
@override
|
@override
|
||||||
void paint(Canvas canvas, Size size) {
|
void paint(Canvas canvas, Size size) {
|
||||||
final s = stroke;
|
final s = stroke;
|
||||||
if (s == null || s.points.isEmpty) return;
|
if (s == null || s.points.isEmpty) return;
|
||||||
final path = buildStrokePath(s, pageSize, isComplete: false);
|
final path =
|
||||||
|
buildStrokePath(s, pageSize, isComplete: false, thinning: thinning);
|
||||||
if (path.getBounds().isEmpty) return;
|
if (path.getBounds().isEmpty) return;
|
||||||
canvas.drawPath(
|
canvas.drawPath(
|
||||||
path,
|
path,
|
||||||
@@ -109,5 +138,7 @@ class LiveInkPainter extends CustomPainter {
|
|||||||
|
|
||||||
@override
|
@override
|
||||||
bool shouldRepaint(LiveInkPainter old) =>
|
bool shouldRepaint(LiveInkPainter old) =>
|
||||||
!identical(old.stroke, stroke) || old.pageSize != pageSize;
|
!identical(old.stroke, stroke) ||
|
||||||
|
old.pageSize != pageSize ||
|
||||||
|
old.thinning != thinning;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -21,6 +21,9 @@
|
|||||||
import 'package:flutter/gestures.dart';
|
import 'package:flutter/gestures.dart';
|
||||||
import 'package:flutter/material.dart';
|
import 'package:flutter/material.dart';
|
||||||
|
|
||||||
|
import '../engine/stroke_geometry.dart' show kDefaultPenThinning;
|
||||||
|
import '../input/pen_config.dart';
|
||||||
|
import '../input/pen_input_service.dart';
|
||||||
import 'ink_painters.dart';
|
import 'ink_painters.dart';
|
||||||
import 'pen_stroke.dart';
|
import 'pen_stroke.dart';
|
||||||
|
|
||||||
@@ -43,6 +46,10 @@ class PenCanvas extends StatefulWidget {
|
|||||||
this.minScale = 0.5,
|
this.minScale = 0.5,
|
||||||
this.maxScale = 8.0,
|
this.maxScale = 8.0,
|
||||||
this.onPenDebug,
|
this.onPenDebug,
|
||||||
|
this.thinning = kDefaultPenThinning,
|
||||||
|
this.sideButtonAction = PenButtonAction.eraser,
|
||||||
|
this.eraserEndAction = PenButtonAction.eraser,
|
||||||
|
this.onPenButtonAction,
|
||||||
});
|
});
|
||||||
|
|
||||||
/// Debug hook: called with a readout of the latest pen event
|
/// Debug hook: called with a readout of the latest pen event
|
||||||
@@ -82,6 +89,20 @@ class PenCanvas extends StatefulWidget {
|
|||||||
final double minScale;
|
final double minScale;
|
||||||
final double maxScale;
|
final double maxScale;
|
||||||
|
|
||||||
|
/// perfect_freehand pressure→width response, from `PenConfig.pressureSensitivity`.
|
||||||
|
final double thinning;
|
||||||
|
|
||||||
|
/// Configured action for the pen's side barrel button (W3 — resolved against
|
||||||
|
/// the native pen plugin's flags on Windows).
|
||||||
|
final PenButtonAction sideButtonAction;
|
||||||
|
|
||||||
|
/// Configured action for the pen's eraser/inverted end (W3).
|
||||||
|
final PenButtonAction eraserEndAction;
|
||||||
|
|
||||||
|
/// Fired (edge-triggered) when a hardware pen button mapped to a non-eraser
|
||||||
|
/// action (undo / toggleTool) is pressed.
|
||||||
|
final void Function(PenButtonAction action)? onPenButtonAction;
|
||||||
|
|
||||||
@override
|
@override
|
||||||
State<PenCanvas> createState() => _PenCanvasState();
|
State<PenCanvas> createState() => _PenCanvasState();
|
||||||
}
|
}
|
||||||
@@ -130,15 +151,79 @@ class _PenCanvasState extends State<PenCanvas> {
|
|||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The eraser signal: barrel/secondary button held, or an inverted stylus.
|
/// The eraser signal. Two sources, ORed:
|
||||||
bool _isEraserSignal(PointerEvent event) =>
|
/// 1. Flutter-native: secondary button held or an inverted stylus (works on
|
||||||
event.buttons == kSecondaryButton ||
|
/// desktop / platforms that surface these).
|
||||||
event.kind == PointerDeviceKind.invertedStylus;
|
/// 2. Windows native pen plugin: barrel / inverted / eraser flags that
|
||||||
|
/// Flutter 3.44 drops, mapped through the configured side-button /
|
||||||
|
/// eraser-end actions (W3). Level-triggered, so holding the button keeps
|
||||||
|
/// erasing — correct for an eraser.
|
||||||
|
bool _isEraserSignal(PointerEvent event) {
|
||||||
|
if (event.buttons == kSecondaryButton ||
|
||||||
|
event.kind == PointerDeviceKind.invertedStylus) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
final hw = PenInputService.instance;
|
||||||
|
if (hw.isActive) {
|
||||||
|
final s = hw.current;
|
||||||
|
if ((s.inverted || s.eraser) &&
|
||||||
|
widget.eraserEndAction == PenButtonAction.eraser) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (s.barrel && widget.sideButtonAction == PenButtonAction.eraser) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Resolve the currently-active configured action from the native pen flags
|
||||||
|
/// (eraser-end takes precedence over the side button when both are set).
|
||||||
|
PenButtonAction _activeHwAction() {
|
||||||
|
final hw = PenInputService.instance;
|
||||||
|
if (!hw.isActive) return PenButtonAction.none;
|
||||||
|
final s = hw.current;
|
||||||
|
if (s.inverted || s.eraser) return widget.eraserEndAction;
|
||||||
|
if (s.barrel) return widget.sideButtonAction;
|
||||||
|
return PenButtonAction.none;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Last hardware action seen, for rising-edge detection of undo/toggleTool.
|
||||||
|
PenButtonAction _lastHwAction = PenButtonAction.none;
|
||||||
|
|
||||||
|
/// Edge-triggered dispatch of non-eraser button actions (undo / toggleTool).
|
||||||
|
/// Eraser is handled level-triggered by [_isEraserSignal]; pan suppresses
|
||||||
|
/// drawing via [_shouldDraw].
|
||||||
|
void _dispatchHwButtonActions() {
|
||||||
|
final action = _activeHwAction();
|
||||||
|
if (action == _lastHwAction) return;
|
||||||
|
_lastHwAction = action;
|
||||||
|
if (action == PenButtonAction.undo ||
|
||||||
|
action == PenButtonAction.toggleTool) {
|
||||||
|
widget.onPenButtonAction?.call(action);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// True while a hardware button mapped to `pan` is held (suppresses drawing
|
||||||
|
/// so the InteractiveViewer pans instead).
|
||||||
|
bool get _hwPanActive => _activeHwAction() == PenButtonAction.pan;
|
||||||
|
|
||||||
|
/// Pen tilt magnitude (degrees) for a stylus event, or null when unavailable.
|
||||||
|
double? _tiltFor(PointerEvent event) {
|
||||||
|
if (!_isStylus(event.kind)) return null;
|
||||||
|
final hw = PenInputService.instance;
|
||||||
|
if (!hw.isActive) return null;
|
||||||
|
final t = hw.current.tiltMagnitude;
|
||||||
|
return t == 0 ? null : t;
|
||||||
|
}
|
||||||
|
|
||||||
/// Decide whether the gesture currently forming should DRAW.
|
/// Decide whether the gesture currently forming should DRAW.
|
||||||
/// True iff exactly one active pointer AND (stylus OR finger-drawing on).
|
/// True iff exactly one active pointer AND (stylus OR finger-drawing on).
|
||||||
bool _shouldDraw(PointerDeviceKind kind) {
|
bool _shouldDraw(PointerDeviceKind kind) {
|
||||||
if (_activePointers.length != 1) return false;
|
if (_activePointers.length != 1) return false;
|
||||||
|
// A hardware pen button mapped to `pan` suppresses drawing so the
|
||||||
|
// InteractiveViewer pans instead.
|
||||||
|
if (_hwPanActive) return false;
|
||||||
if (_isStylus(kind)) return true;
|
if (_isStylus(kind)) return true;
|
||||||
if (kind == PointerDeviceKind.mouse) return true;
|
if (kind == PointerDeviceKind.mouse) return true;
|
||||||
if (kind == PointerDeviceKind.touch) return _fingerDrawingEnabled;
|
if (kind == PointerDeviceKind.touch) return _fingerDrawingEnabled;
|
||||||
@@ -149,7 +234,8 @@ class _PenCanvasState extends State<PenCanvas> {
|
|||||||
|
|
||||||
/// Map a global pointer position into normalized page coords using the
|
/// Map a global pointer position into normalized page coords using the
|
||||||
/// shared transform (inverse) and this widget's geometry.
|
/// shared transform (inverse) and this widget's geometry.
|
||||||
PenPoint? _toNormalized(Offset globalPosition, double? pressure) {
|
PenPoint? _toNormalized(Offset globalPosition, double? pressure,
|
||||||
|
{double? tilt}) {
|
||||||
final box = context.findRenderObject() as RenderBox?;
|
final box = context.findRenderObject() as RenderBox?;
|
||||||
if (box == null) return null;
|
if (box == null) return null;
|
||||||
final local = box.globalToLocal(globalPosition);
|
final local = box.globalToLocal(globalPosition);
|
||||||
@@ -159,7 +245,7 @@ class _PenCanvasState extends State<PenCanvas> {
|
|||||||
|
|
||||||
final nx = scene.dx / widget.pageSize.width;
|
final nx = scene.dx / widget.pageSize.width;
|
||||||
final ny = scene.dy / widget.pageSize.height;
|
final ny = scene.dy / widget.pageSize.height;
|
||||||
return PenPoint(nx, ny, pressure);
|
return PenPoint(nx, ny, pressure, tilt: tilt);
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- Stroke lifecycle -----------------------------------------------------
|
// --- Stroke lifecycle -----------------------------------------------------
|
||||||
@@ -167,7 +253,8 @@ class _PenCanvasState extends State<PenCanvas> {
|
|||||||
void _startStroke(PointerDownEvent event) {
|
void _startStroke(PointerDownEvent event) {
|
||||||
_drawPointer = event.pointer;
|
_drawPointer = event.pointer;
|
||||||
_livePoints.clear();
|
_livePoints.clear();
|
||||||
final p = _toNormalized(event.position, _normalizedPressure(event));
|
final p = _toNormalized(event.position, _normalizedPressure(event),
|
||||||
|
tilt: _tiltFor(event));
|
||||||
if (p != null) _livePoints.add(p);
|
if (p != null) _livePoints.add(p);
|
||||||
|
|
||||||
if (_eraserActive || widget.tool == CanvasTool.eraser) {
|
if (_eraserActive || widget.tool == CanvasTool.eraser) {
|
||||||
@@ -180,7 +267,8 @@ class _PenCanvasState extends State<PenCanvas> {
|
|||||||
}
|
}
|
||||||
|
|
||||||
void _extendStroke(PointerMoveEvent event) {
|
void _extendStroke(PointerMoveEvent event) {
|
||||||
final p = _toNormalized(event.position, _normalizedPressure(event));
|
final p = _toNormalized(event.position, _normalizedPressure(event),
|
||||||
|
tilt: _tiltFor(event));
|
||||||
if (p == null) return;
|
if (p == null) return;
|
||||||
|
|
||||||
if (_eraserActive || widget.tool == CanvasTool.eraser) {
|
if (_eraserActive || widget.tool == CanvasTool.eraser) {
|
||||||
@@ -255,18 +343,28 @@ class _PenCanvasState extends State<PenCanvas> {
|
|||||||
|
|
||||||
// --- Listener callbacks ---------------------------------------------------
|
// --- Listener callbacks ---------------------------------------------------
|
||||||
|
|
||||||
|
/// Highest NORMALIZED pressure seen since the diagnostic was last reset —
|
||||||
|
/// makes "does pressure actually vary?" unambiguous on the readout.
|
||||||
|
double _peakNorm = 0;
|
||||||
|
|
||||||
void _emitPenDebug(PointerEvent event) {
|
void _emitPenDebug(PointerEvent event) {
|
||||||
final cb = widget.onPenDebug;
|
final cb = widget.onPenDebug;
|
||||||
if (cb == null) return;
|
if (cb == null) return;
|
||||||
cb('${event.kind.name} p=${event.pressure.toStringAsFixed(3)} '
|
final norm = _normalizedPressure(event);
|
||||||
'min=${event.pressureMin.toStringAsFixed(2)} '
|
if (norm != null && norm > _peakNorm) _peakNorm = norm;
|
||||||
'max=${event.pressureMax.toStringAsFixed(2)} '
|
cb('${event.kind.name} raw=${event.pressure.toStringAsFixed(1)}'
|
||||||
'tilt=${event.tilt.toStringAsFixed(2)}');
|
'/${event.pressureMax.toStringAsFixed(0)} '
|
||||||
|
'norm=${norm?.toStringAsFixed(3) ?? "null"} '
|
||||||
|
'peak=${_peakNorm.toStringAsFixed(3)} '
|
||||||
|
'btn=${event.buttons} tilt=${event.tilt.toStringAsFixed(2)}');
|
||||||
}
|
}
|
||||||
|
|
||||||
void _onPointerHover(PointerHoverEvent event) {
|
void _onPointerHover(PointerHoverEvent event) {
|
||||||
if (_isStylus(event.kind)) {
|
if (_isStylus(event.kind)) {
|
||||||
_emitPenDebug(event);
|
_emitPenDebug(event);
|
||||||
|
// Fire edge-triggered button actions (undo / toggleTool) on hover so a
|
||||||
|
// mapped barrel press works without first touching down.
|
||||||
|
_dispatchHwButtonActions();
|
||||||
// Detect eraser (barrel button / inverted) while hovering.
|
// Detect eraser (barrel button / inverted) while hovering.
|
||||||
_eraserActive = _isEraserSignal(event);
|
_eraserActive = _isEraserSignal(event);
|
||||||
}
|
}
|
||||||
@@ -274,7 +372,13 @@ class _PenCanvasState extends State<PenCanvas> {
|
|||||||
|
|
||||||
void _onPointerDown(PointerDownEvent event) {
|
void _onPointerDown(PointerDownEvent event) {
|
||||||
if (event.kind == PointerDeviceKind.trackpad) return;
|
if (event.kind == PointerDeviceKind.trackpad) return;
|
||||||
if (_isStylus(event.kind)) _emitPenDebug(event);
|
if (_isStylus(event.kind)) {
|
||||||
|
_emitPenDebug(event);
|
||||||
|
// Fire edge-triggered button actions for a direct pen-down (no prior
|
||||||
|
// hover); the native observer latched this contact's flags before Flutter
|
||||||
|
// synthesized this event (plan M1/M2).
|
||||||
|
_dispatchHwButtonActions();
|
||||||
|
}
|
||||||
|
|
||||||
_activePointers[event.pointer] = event.kind;
|
_activePointers[event.pointer] = event.kind;
|
||||||
|
|
||||||
@@ -343,8 +447,15 @@ class _PenCanvasState extends State<PenCanvas> {
|
|||||||
height: widget.pageSize.height,
|
height: widget.pageSize.height,
|
||||||
child: Stack(
|
child: Stack(
|
||||||
children: [
|
children: [
|
||||||
// PDF page bitmap.
|
// PDF page bitmap. Wrapped in its own RepaintBoundary (W2) so the
|
||||||
Positioned.fill(child: widget.pageWidget),
|
// per-move live-ink repaints and the static-ink repaints never
|
||||||
|
// mark the page's raster layer dirty — isolating it from
|
||||||
|
// ink-driven repaints. (The definitive crisp-on-zoom / no-flash
|
||||||
|
// fix is the P0.5 page_tile DPI-on-settle double-buffer; this
|
||||||
|
// boundary is the safe, non-regressive interim per plan M3.)
|
||||||
|
Positioned.fill(
|
||||||
|
child: RepaintBoundary(child: widget.pageWidget),
|
||||||
|
),
|
||||||
// Committed ink (static layer, isolated repaint).
|
// Committed ink (static layer, isolated repaint).
|
||||||
Positioned.fill(
|
Positioned.fill(
|
||||||
child: RepaintBoundary(
|
child: RepaintBoundary(
|
||||||
@@ -352,6 +463,7 @@ class _PenCanvasState extends State<PenCanvas> {
|
|||||||
painter: StaticInkPainter(
|
painter: StaticInkPainter(
|
||||||
strokes: widget.strokes,
|
strokes: widget.strokes,
|
||||||
pageSize: widget.pageSize,
|
pageSize: widget.pageSize,
|
||||||
|
thinning: widget.thinning,
|
||||||
),
|
),
|
||||||
),
|
),
|
||||||
),
|
),
|
||||||
@@ -363,6 +475,7 @@ class _PenCanvasState extends State<PenCanvas> {
|
|||||||
painter: LiveInkPainter(
|
painter: LiveInkPainter(
|
||||||
stroke: _liveStroke,
|
stroke: _liveStroke,
|
||||||
pageSize: widget.pageSize,
|
pageSize: widget.pageSize,
|
||||||
|
thinning: widget.thinning,
|
||||||
),
|
),
|
||||||
),
|
),
|
||||||
),
|
),
|
||||||
|
|||||||
@@ -9,9 +9,11 @@ import 'package:flutter/material.dart';
|
|||||||
import 'package:pdfrx/pdfrx.dart';
|
import 'package:pdfrx/pdfrx.dart';
|
||||||
|
|
||||||
import '../../services/database_service.dart';
|
import '../../services/database_service.dart';
|
||||||
|
import '../engine/stroke_geometry.dart' show kDefaultPenThinning;
|
||||||
import '../engine/stroke_model.dart';
|
import '../engine/stroke_model.dart';
|
||||||
import '../engine/undo_stack.dart';
|
import '../engine/undo_stack.dart';
|
||||||
import '../input/pen_config.dart';
|
import '../input/pen_config.dart';
|
||||||
|
import '../input/pen_input_service.dart';
|
||||||
import '../persistence/editor_repository.dart';
|
import '../persistence/editor_repository.dart';
|
||||||
import '../persistence/save_scheduler.dart';
|
import '../persistence/save_scheduler.dart';
|
||||||
import '../ui/pen_settings_page.dart';
|
import '../ui/pen_settings_page.dart';
|
||||||
@@ -98,8 +100,8 @@ class _PenEditorScreenState extends State<PenEditorScreen> {
|
|||||||
Color _color = Colors.black;
|
Color _color = Colors.black;
|
||||||
bool _allowFingerDrawing = false;
|
bool _allowFingerDrawing = false;
|
||||||
|
|
||||||
/// Pen width as a fraction of page width.
|
/// Pen width as a fraction of page width (base; pressure thins it down).
|
||||||
static const double _penWidthFraction = 0.004;
|
static const double _penWidthFraction = 0.006;
|
||||||
static const double _highlighterWidthFraction = 0.02;
|
static const double _highlighterWidthFraction = 0.02;
|
||||||
|
|
||||||
static const List<Color> _palette = [
|
static const List<Color> _palette = [
|
||||||
@@ -114,6 +116,9 @@ class _PenEditorScreenState extends State<PenEditorScreen> {
|
|||||||
void initState() {
|
void initState() {
|
||||||
super.initState();
|
super.initState();
|
||||||
_documentId = _documentIdFromPath(widget.pdfPath);
|
_documentId = _documentIdFromPath(widget.pdfPath);
|
||||||
|
// Begin listening to the native Windows pen plugin (barrel/eraser/tilt).
|
||||||
|
// No-op on platforms without the plugin (W3).
|
||||||
|
PenInputService.instance.start();
|
||||||
_initPersistence();
|
_initPersistence();
|
||||||
_initPenConfig();
|
_initPenConfig();
|
||||||
_open();
|
_open();
|
||||||
@@ -125,6 +130,9 @@ class _PenEditorScreenState extends State<PenEditorScreen> {
|
|||||||
controller.dispose();
|
controller.dispose();
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
// Rebuild the editor when pen settings change (width, pressure
|
||||||
|
// sensitivity, button mappings) so the live canvas reflects them.
|
||||||
|
controller.addListener(_onPenConfigChanged);
|
||||||
setState(() {
|
setState(() {
|
||||||
_penConfig = controller;
|
_penConfig = controller;
|
||||||
// Adopt the persisted finger-drawing preference as the initial local
|
// Adopt the persisted finger-drawing preference as the initial local
|
||||||
@@ -134,6 +142,10 @@ class _PenEditorScreenState extends State<PenEditorScreen> {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
void _onPenConfigChanged() {
|
||||||
|
if (mounted) setState(() {});
|
||||||
|
}
|
||||||
|
|
||||||
Future<void> _initPersistence() async {
|
Future<void> _initPersistence() async {
|
||||||
final service = await DatabaseService.getInstance();
|
final service = await DatabaseService.getInstance();
|
||||||
if (!mounted) return;
|
if (!mounted) return;
|
||||||
@@ -159,7 +171,7 @@ class _PenEditorScreenState extends State<PenEditorScreen> {
|
|||||||
loaded[pageIndex] = entry.value
|
loaded[pageIndex] = entry.value
|
||||||
.map((es) => PenStroke(
|
.map((es) => PenStroke(
|
||||||
points: es.points
|
points: es.points
|
||||||
.map((ep) => PenPoint(ep.x, ep.y, ep.pressure))
|
.map((ep) => PenPoint(ep.x, ep.y, ep.pressure, tilt: ep.tilt))
|
||||||
.toList(),
|
.toList(),
|
||||||
color: es.color,
|
color: es.color,
|
||||||
width: es.width,
|
width: es.width,
|
||||||
@@ -211,6 +223,7 @@ class _PenEditorScreenState extends State<PenEditorScreen> {
|
|||||||
_document?.dispose();
|
_document?.dispose();
|
||||||
_transform.dispose();
|
_transform.dispose();
|
||||||
_penConfig?.dispose();
|
_penConfig?.dispose();
|
||||||
|
PenInputService.instance.stop();
|
||||||
super.dispose();
|
super.dispose();
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -308,6 +321,33 @@ class _PenEditorScreenState extends State<PenEditorScreen> {
|
|||||||
_schedulePageSave(_pageIndex, List<PenStroke>.of(snapshot));
|
_schedulePageSave(_pageIndex, List<PenStroke>.of(snapshot));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Cycle pen → highlighter → eraser → pen (for the toggleTool button action).
|
||||||
|
void _cycleTool() {
|
||||||
|
setState(() {
|
||||||
|
_tool = switch (_tool) {
|
||||||
|
CanvasTool.pen => CanvasTool.highlighter,
|
||||||
|
CanvasTool.highlighter => CanvasTool.eraser,
|
||||||
|
CanvasTool.eraser => CanvasTool.pen,
|
||||||
|
};
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Handle a hardware pen-button action delivered by [PenCanvas] (W3).
|
||||||
|
/// `eraser` and `pan` are handled inside the canvas; here we map the
|
||||||
|
/// edge-triggered ones.
|
||||||
|
void _handlePenButtonAction(PenButtonAction action) {
|
||||||
|
switch (action) {
|
||||||
|
case PenButtonAction.undo:
|
||||||
|
_performUndo();
|
||||||
|
case PenButtonAction.toggleTool:
|
||||||
|
_cycleTool();
|
||||||
|
case PenButtonAction.eraser:
|
||||||
|
case PenButtonAction.pan:
|
||||||
|
case PenButtonAction.none:
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Toggle finger-drawing, keeping the local state and the persisted config
|
/// Toggle finger-drawing, keeping the local state and the persisted config
|
||||||
/// (when loaded) in sync.
|
/// (when loaded) in sync.
|
||||||
void _toggleFingerDrawing() {
|
void _toggleFingerDrawing() {
|
||||||
@@ -457,6 +497,13 @@ class _PenEditorScreenState extends State<PenEditorScreen> {
|
|||||||
? (_penConfig?.value.highlighterWidth ??
|
? (_penConfig?.value.highlighterWidth ??
|
||||||
_highlighterWidthFraction)
|
_highlighterWidthFraction)
|
||||||
: (_penConfig?.value.penWidth ?? _penWidthFraction),
|
: (_penConfig?.value.penWidth ?? _penWidthFraction),
|
||||||
|
thinning:
|
||||||
|
_penConfig?.value.pressureSensitivity ?? kDefaultPenThinning,
|
||||||
|
sideButtonAction:
|
||||||
|
_penConfig?.value.sideButton ?? PenButtonAction.eraser,
|
||||||
|
eraserEndAction:
|
||||||
|
_penConfig?.value.eraserEnd ?? PenButtonAction.eraser,
|
||||||
|
onPenButtonAction: _handlePenButtonAction,
|
||||||
allowFingerDrawing: _allowFingerDrawing,
|
allowFingerDrawing: _allowFingerDrawing,
|
||||||
onPenDebug: _showPenDebug
|
onPenDebug: _showPenDebug
|
||||||
? (s) => setState(() => _penDebug = s)
|
? (s) => setState(() => _penDebug = s)
|
||||||
|
|||||||
@@ -11,13 +11,17 @@ import 'package:flutter/foundation.dart';
|
|||||||
/// [x]/[y] are normalized to the page rectangle ([0,1]).
|
/// [x]/[y] are normalized to the page rectangle ([0,1]).
|
||||||
/// [pressure] is the normalized stylus pressure ([0,1]) or null when the
|
/// [pressure] is the normalized stylus pressure ([0,1]) or null when the
|
||||||
/// device reported no usable pressure (perfect_freehand then simulates it).
|
/// device reported no usable pressure (perfect_freehand then simulates it).
|
||||||
|
/// [tilt] is the pen tilt magnitude in degrees (0 = perpendicular), or null
|
||||||
|
/// when unavailable. On Windows it is sourced from the native pen plugin
|
||||||
|
/// (`badnote/pen`) since Flutter 3.44 does not surface tilt itself.
|
||||||
@immutable
|
@immutable
|
||||||
class PenPoint {
|
class PenPoint {
|
||||||
const PenPoint(this.x, this.y, this.pressure);
|
const PenPoint(this.x, this.y, this.pressure, {this.tilt});
|
||||||
|
|
||||||
final double x;
|
final double x;
|
||||||
final double y;
|
final double y;
|
||||||
final double? pressure;
|
final double? pressure;
|
||||||
|
final double? tilt;
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Which kind of mark a stroke is.
|
/// Which kind of mark a stroke is.
|
||||||
|
|||||||
@@ -13,18 +13,30 @@ import 'package:perfect_freehand/perfect_freehand.dart' as pf;
|
|||||||
|
|
||||||
import 'stroke_model.dart';
|
import 'stroke_model.dart';
|
||||||
|
|
||||||
|
/// Canonical default for perfect_freehand's `thinning` (how strongly pressure
|
||||||
|
/// modulates stroke width). The SINGLE source of truth shared by the on-screen
|
||||||
|
/// painter ([buildStrokeOutline] here and `ink_painters.buildStrokePath`) and
|
||||||
|
/// the PDF export path, so screen and export can never diverge. `0.85` =
|
||||||
|
/// pressure visibly sweeps width; preserves the existing feel + export golden.
|
||||||
|
/// Overridable per-stroke via [PenConfig.pressureSensitivity].
|
||||||
|
const double kDefaultPenThinning = 0.85;
|
||||||
|
|
||||||
/// Builds a closed, fillable outline [Path] for one [stroke], scaled into the
|
/// Builds a closed, fillable outline [Path] for one [stroke], scaled into the
|
||||||
/// pixel space of [pageSize] (which maps normalized [0,1] coords to pixels).
|
/// pixel space of [pageSize] (which maps normalized [0,1] coords to pixels).
|
||||||
///
|
///
|
||||||
/// [isComplete] should be false for the in-progress live stroke so freehand
|
/// [isComplete] should be false for the in-progress live stroke so freehand
|
||||||
/// tapers the trailing end correctly, and true for committed strokes.
|
/// tapers the trailing end correctly, and true for committed strokes.
|
||||||
///
|
///
|
||||||
|
/// [thinning] is perfect_freehand's pressure→width response (see
|
||||||
|
/// [kDefaultPenThinning]); highlighter always forces `0.0` (constant width).
|
||||||
|
///
|
||||||
/// Returns an empty [Path] when the stroke has no points (or freehand produces
|
/// Returns an empty [Path] when the stroke has no points (or freehand produces
|
||||||
/// no outline).
|
/// no outline).
|
||||||
Path buildStrokeOutline(
|
Path buildStrokeOutline(
|
||||||
EditorStroke stroke,
|
EditorStroke stroke,
|
||||||
Size pageSize, {
|
Size pageSize, {
|
||||||
required bool isComplete,
|
required bool isComplete,
|
||||||
|
double thinning = kDefaultPenThinning,
|
||||||
}) {
|
}) {
|
||||||
final path = Path();
|
final path = Path();
|
||||||
if (stroke.points.isEmpty) return path;
|
if (stroke.points.isEmpty) return path;
|
||||||
@@ -36,7 +48,7 @@ Path buildStrokeOutline(
|
|||||||
|
|
||||||
final pfPoints = stroke.points
|
final pfPoints = stroke.points
|
||||||
.map(
|
.map(
|
||||||
(p) => pf.Point(
|
(p) => pf.PointVector(
|
||||||
p.x * pageSize.width,
|
p.x * pageSize.width,
|
||||||
p.y * pageSize.height,
|
p.y * pageSize.height,
|
||||||
p.pressure ?? 0.5,
|
p.pressure ?? 0.5,
|
||||||
@@ -46,22 +58,25 @@ Path buildStrokeOutline(
|
|||||||
|
|
||||||
final outline = pf.getStroke(
|
final outline = pf.getStroke(
|
||||||
pfPoints,
|
pfPoints,
|
||||||
size: pixelWidth,
|
options: pf.StrokeOptions(
|
||||||
// Highlighter keeps a constant width (no thinning); pen thins (0.7),
|
size: pixelWidth,
|
||||||
// matching the live recipe.
|
// Highlighter keeps a constant width (no thinning); pen uses the
|
||||||
thinning: isHighlighter ? 0.0 : 0.7,
|
// configurable [thinning] so Surface-Pen pressure changes width.
|
||||||
smoothing: 0.5,
|
thinning: isHighlighter ? 0.0 : thinning,
|
||||||
streamline: 0.5,
|
smoothing: 0.5,
|
||||||
// Real stylus pressure -> don't simulate; no pressure -> let freehand fake
|
streamline: 0.5,
|
||||||
// it based on velocity (highlighter never simulates).
|
// Real stylus pressure -> don't simulate; no pressure -> let freehand
|
||||||
simulatePressure: !hasRealPressure && !isHighlighter,
|
// fake it based on velocity (highlighter never simulates). perfect_freehand
|
||||||
isComplete: isComplete,
|
// 2.x honors real pressure when simulatePressure is false.
|
||||||
|
simulatePressure: !hasRealPressure && !isHighlighter,
|
||||||
|
isComplete: isComplete,
|
||||||
|
),
|
||||||
);
|
);
|
||||||
|
|
||||||
if (outline.isEmpty) return path;
|
if (outline.isEmpty) return path;
|
||||||
path.moveTo(outline.first.x, outline.first.y);
|
path.moveTo(outline.first.dx, outline.first.dy);
|
||||||
for (var i = 1; i < outline.length; i++) {
|
for (var i = 1; i < outline.length; i++) {
|
||||||
path.lineTo(outline[i].x, outline[i].y);
|
path.lineTo(outline[i].dx, outline[i].dy);
|
||||||
}
|
}
|
||||||
path.close();
|
path.close();
|
||||||
return path;
|
return path;
|
||||||
|
|||||||
@@ -103,12 +103,14 @@ abstract class EditorStroke with _$EditorStroke {
|
|||||||
|
|
||||||
// ---- Adapters -----------------------------------------------------------
|
// ---- Adapters -----------------------------------------------------------
|
||||||
|
|
||||||
/// Adapts an in-memory live [PenStroke] (normalized, no tilt/timestamp/kind).
|
/// Adapts an in-memory live [PenStroke] (normalized; carries tilt when the
|
||||||
|
/// native pen plugin supplied it, else null; no timestamp/pointerDeviceKind).
|
||||||
factory EditorStroke.fromPenStroke(PenStroke stroke, {String? id}) =>
|
factory EditorStroke.fromPenStroke(PenStroke stroke, {String? id}) =>
|
||||||
EditorStroke(
|
EditorStroke(
|
||||||
id: id ?? _uuid.v4(),
|
id: id ?? _uuid.v4(),
|
||||||
points: stroke.points
|
points: stroke.points
|
||||||
.map((p) => EditorPoint(x: p.x, y: p.y, pressure: p.pressure))
|
.map((p) =>
|
||||||
|
EditorPoint(x: p.x, y: p.y, pressure: p.pressure, tilt: p.tilt))
|
||||||
.toList(),
|
.toList(),
|
||||||
tool: switch (stroke.kind) {
|
tool: switch (stroke.kind) {
|
||||||
PenStrokeKind.pen => EditorTool.pen,
|
PenStrokeKind.pen => EditorTool.pen,
|
||||||
|
|||||||
@@ -4,6 +4,8 @@ import 'dart:math';
|
|||||||
import 'package:flutter/foundation.dart';
|
import 'package:flutter/foundation.dart';
|
||||||
import 'package:shared_preferences/shared_preferences.dart';
|
import 'package:shared_preferences/shared_preferences.dart';
|
||||||
|
|
||||||
|
import '../engine/stroke_geometry.dart' show kDefaultPenThinning;
|
||||||
|
|
||||||
/// Action that can be triggered by a hardware pen button or the eraser end.
|
/// Action that can be triggered by a hardware pen button or the eraser end.
|
||||||
enum PenButtonAction {
|
enum PenButtonAction {
|
||||||
none,
|
none,
|
||||||
@@ -25,10 +27,13 @@ class PenConfig {
|
|||||||
this.fingerDrawing = false,
|
this.fingerDrawing = false,
|
||||||
this.penWidth = 0.004,
|
this.penWidth = 0.004,
|
||||||
this.highlighterWidth = 0.02,
|
this.highlighterWidth = 0.02,
|
||||||
|
this.pressureSensitivity = kDefaultPenThinning,
|
||||||
}) : assert(pressureGamma >= 0.3 && pressureGamma <= 3.0,
|
}) : assert(pressureGamma >= 0.3 && pressureGamma <= 3.0,
|
||||||
'pressureGamma must be in [0.3, 3.0]'),
|
'pressureGamma must be in [0.3, 3.0]'),
|
||||||
assert(palmRejectionMs >= 0.0 && palmRejectionMs <= 500.0,
|
assert(palmRejectionMs >= 0.0 && palmRejectionMs <= 500.0,
|
||||||
'palmRejectionMs must be in [0, 500]');
|
'palmRejectionMs must be in [0, 500]'),
|
||||||
|
assert(pressureSensitivity >= 0.0 && pressureSensitivity <= 1.0,
|
||||||
|
'pressureSensitivity must be in [0, 1]');
|
||||||
|
|
||||||
/// Which action fires when the side barrel button is held.
|
/// Which action fires when the side barrel button is held.
|
||||||
final PenButtonAction sideButton;
|
final PenButtonAction sideButton;
|
||||||
@@ -54,6 +59,13 @@ class PenConfig {
|
|||||||
/// Highlighter stroke width as a fraction of the canvas width.
|
/// Highlighter stroke width as a fraction of the canvas width.
|
||||||
final double highlighterWidth;
|
final double highlighterWidth;
|
||||||
|
|
||||||
|
/// How strongly stylus pressure modulates stroke width — maps directly to
|
||||||
|
/// perfect_freehand's `thinning`. Range [0,1]; `0` = constant width,
|
||||||
|
/// higher = pressure sweeps width more (Saber's `StrokeOptions.thinning`
|
||||||
|
/// model). Default [kDefaultPenThinning] so the out-of-box feel and the
|
||||||
|
/// export golden are unchanged.
|
||||||
|
final double pressureSensitivity;
|
||||||
|
|
||||||
PenConfig copyWith({
|
PenConfig copyWith({
|
||||||
PenButtonAction? sideButton,
|
PenButtonAction? sideButton,
|
||||||
PenButtonAction? eraserEnd,
|
PenButtonAction? eraserEnd,
|
||||||
@@ -62,6 +74,7 @@ class PenConfig {
|
|||||||
bool? fingerDrawing,
|
bool? fingerDrawing,
|
||||||
double? penWidth,
|
double? penWidth,
|
||||||
double? highlighterWidth,
|
double? highlighterWidth,
|
||||||
|
double? pressureSensitivity,
|
||||||
}) {
|
}) {
|
||||||
return PenConfig(
|
return PenConfig(
|
||||||
sideButton: sideButton ?? this.sideButton,
|
sideButton: sideButton ?? this.sideButton,
|
||||||
@@ -71,6 +84,7 @@ class PenConfig {
|
|||||||
fingerDrawing: fingerDrawing ?? this.fingerDrawing,
|
fingerDrawing: fingerDrawing ?? this.fingerDrawing,
|
||||||
penWidth: penWidth ?? this.penWidth,
|
penWidth: penWidth ?? this.penWidth,
|
||||||
highlighterWidth: highlighterWidth ?? this.highlighterWidth,
|
highlighterWidth: highlighterWidth ?? this.highlighterWidth,
|
||||||
|
pressureSensitivity: pressureSensitivity ?? this.pressureSensitivity,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -82,6 +96,7 @@ class PenConfig {
|
|||||||
'fingerDrawing': fingerDrawing,
|
'fingerDrawing': fingerDrawing,
|
||||||
'penWidth': penWidth,
|
'penWidth': penWidth,
|
||||||
'highlighterWidth': highlighterWidth,
|
'highlighterWidth': highlighterWidth,
|
||||||
|
'pressureSensitivity': pressureSensitivity,
|
||||||
};
|
};
|
||||||
|
|
||||||
factory PenConfig.fromJson(Map<String, dynamic> json) {
|
factory PenConfig.fromJson(Map<String, dynamic> json) {
|
||||||
@@ -97,6 +112,8 @@ class PenConfig {
|
|||||||
fingerDrawing: json['fingerDrawing'] as bool? ?? false,
|
fingerDrawing: json['fingerDrawing'] as bool? ?? false,
|
||||||
penWidth: (json['penWidth'] as num?)?.toDouble() ?? 0.004,
|
penWidth: (json['penWidth'] as num?)?.toDouble() ?? 0.004,
|
||||||
highlighterWidth: (json['highlighterWidth'] as num?)?.toDouble() ?? 0.02,
|
highlighterWidth: (json['highlighterWidth'] as num?)?.toDouble() ?? 0.02,
|
||||||
|
pressureSensitivity:
|
||||||
|
(json['pressureSensitivity'] as num?)?.toDouble() ?? kDefaultPenThinning,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -111,7 +128,8 @@ class PenConfig {
|
|||||||
palmRejectionMs == other.palmRejectionMs &&
|
palmRejectionMs == other.palmRejectionMs &&
|
||||||
fingerDrawing == other.fingerDrawing &&
|
fingerDrawing == other.fingerDrawing &&
|
||||||
penWidth == other.penWidth &&
|
penWidth == other.penWidth &&
|
||||||
highlighterWidth == other.highlighterWidth;
|
highlighterWidth == other.highlighterWidth &&
|
||||||
|
pressureSensitivity == other.pressureSensitivity;
|
||||||
|
|
||||||
@override
|
@override
|
||||||
int get hashCode => Object.hash(
|
int get hashCode => Object.hash(
|
||||||
@@ -122,6 +140,7 @@ class PenConfig {
|
|||||||
fingerDrawing,
|
fingerDrawing,
|
||||||
penWidth,
|
penWidth,
|
||||||
highlighterWidth,
|
highlighterWidth,
|
||||||
|
pressureSensitivity,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -215,4 +234,11 @@ class PenConfigController extends ChangeNotifier {
|
|||||||
notifyListeners();
|
notifyListeners();
|
||||||
await _persist();
|
await _persist();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Sets [PenConfig.pressureSensitivity]. Clamped to [0, 1].
|
||||||
|
Future<void> setPressureSensitivity(double sensitivity) async {
|
||||||
|
_value = _value.copyWith(pressureSensitivity: sensitivity.clamp(0.0, 1.0));
|
||||||
|
notifyListeners();
|
||||||
|
await _persist();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
136
lib/editor/input/pen_input_service.dart
Normal file
136
lib/editor/input/pen_input_service.dart
Normal file
@@ -0,0 +1,136 @@
|
|||||||
|
// lib/editor/input/pen_input_service.dart
|
||||||
|
//
|
||||||
|
// Dart side of the native Windows pen observer (`windows/runner/pen_channel.cpp`).
|
||||||
|
//
|
||||||
|
// WHY THIS EXISTS: Flutter 3.44 on Windows delivers stylus PRESSURE but drops
|
||||||
|
// the pen's barrel button, eraser/inverted end, and tilt (it does not map
|
||||||
|
// POINTER_PEN_FLAG_* into `PointerEvent.buttons`/`invertedStylus`/`tilt`). The
|
||||||
|
// native plugin observes WM_POINTER + GetPointerPenInfo and streams the missing
|
||||||
|
// hardware state over an EventChannel; this service latches the LATEST value.
|
||||||
|
//
|
||||||
|
// CORRELATION (plan M2): we do NOT key state by Win32 pointerId joined to
|
||||||
|
// Flutter's `event.pointer` — those are different id spaces. Only one pen is
|
||||||
|
// active at a time, so a single latched "current" state is correct. The native
|
||||||
|
// observer runs at the TOP of the window proc (BEFORE Flutter synthesizes its
|
||||||
|
// pointer event, plan M1), so by the time Dart's pointer-down handler reads
|
||||||
|
// [current], the latch already reflects that exact contact — no hover required.
|
||||||
|
//
|
||||||
|
// GRACEFUL DEGRADATION: on non-Windows (or if the channel is silent) the stream
|
||||||
|
// simply never emits / errors are swallowed, and [current] stays [PenHardwareState.empty]
|
||||||
|
// so the canvas falls back to its normal Flutter-pressure drawing.
|
||||||
|
|
||||||
|
import 'dart:async';
|
||||||
|
|
||||||
|
import 'package:flutter/services.dart';
|
||||||
|
|
||||||
|
/// Latest hardware pen state delivered by the native observer.
|
||||||
|
class PenHardwareState {
|
||||||
|
const PenHardwareState({
|
||||||
|
this.barrel = false,
|
||||||
|
this.inverted = false,
|
||||||
|
this.eraser = false,
|
||||||
|
this.tiltX = 0.0,
|
||||||
|
this.tiltY = 0.0,
|
||||||
|
});
|
||||||
|
|
||||||
|
/// Side barrel button held.
|
||||||
|
final bool barrel;
|
||||||
|
|
||||||
|
/// Pen flipped to the inverted (eraser) end.
|
||||||
|
final bool inverted;
|
||||||
|
|
||||||
|
/// Hardware eraser flag set.
|
||||||
|
final bool eraser;
|
||||||
|
|
||||||
|
/// Tilt in degrees along X / Y ([-90, 90]); 0 = perpendicular.
|
||||||
|
final double tiltX;
|
||||||
|
final double tiltY;
|
||||||
|
|
||||||
|
/// Combined tilt magnitude in degrees (for [PenPoint.tilt]).
|
||||||
|
double get tiltMagnitude {
|
||||||
|
final t = tiltX * tiltX + tiltY * tiltY;
|
||||||
|
return t <= 0 ? 0.0 : _sqrt(t);
|
||||||
|
}
|
||||||
|
|
||||||
|
static const empty = PenHardwareState();
|
||||||
|
}
|
||||||
|
|
||||||
|
// Avoids importing dart:math for a single call.
|
||||||
|
double _sqrt(double v) {
|
||||||
|
if (v <= 0) return 0;
|
||||||
|
var x = v;
|
||||||
|
var last = 0.0;
|
||||||
|
// Newton's method; converges fast for the small (<=~127) magnitudes here.
|
||||||
|
for (var i = 0; i < 12 && x != last; i++) {
|
||||||
|
last = x;
|
||||||
|
x = 0.5 * (x + v / x);
|
||||||
|
}
|
||||||
|
return x;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Latches the most recent [PenHardwareState] streamed by the native pen plugin.
|
||||||
|
///
|
||||||
|
/// Use the singleton [PenInputService.instance]. Call [start] once (e.g. in the
|
||||||
|
/// editor's `initState`) and [stop] on dispose.
|
||||||
|
class PenInputService {
|
||||||
|
PenInputService._();
|
||||||
|
|
||||||
|
/// Process-wide singleton (one physical pen).
|
||||||
|
static final PenInputService instance = PenInputService._();
|
||||||
|
|
||||||
|
/// Must match the native `EventChannel` name in `pen_channel.cpp`.
|
||||||
|
static const EventChannel _channel = EventChannel('badnote/pen');
|
||||||
|
|
||||||
|
StreamSubscription<dynamic>? _sub;
|
||||||
|
PenHardwareState _current = PenHardwareState.empty;
|
||||||
|
|
||||||
|
/// The latest hardware pen state (or [PenHardwareState.empty] when no native
|
||||||
|
/// data has arrived — non-Windows, plugin absent, or channel silent).
|
||||||
|
PenHardwareState get current => _current;
|
||||||
|
|
||||||
|
/// Whether the native channel has delivered at least one event (i.e. the
|
||||||
|
/// native pen plugin is present and active). Used to prefer hardware signals
|
||||||
|
/// over the Flutter fallback only when they are actually available.
|
||||||
|
bool get isActive => _active;
|
||||||
|
bool _active = false;
|
||||||
|
|
||||||
|
/// Begins listening to the native channel. Idempotent; safe on any platform
|
||||||
|
/// (no-ops where the channel has no handler).
|
||||||
|
void start() {
|
||||||
|
if (_sub != null) return;
|
||||||
|
try {
|
||||||
|
_sub = _channel.receiveBroadcastStream().listen(
|
||||||
|
_onEvent,
|
||||||
|
onError: (Object _) {
|
||||||
|
// No native handler (e.g. Linux/macOS) or transient error — ignore
|
||||||
|
// and keep the empty fallback state.
|
||||||
|
},
|
||||||
|
cancelOnError: false,
|
||||||
|
);
|
||||||
|
} catch (_) {
|
||||||
|
// receiveBroadcastStream can throw synchronously if the platform side is
|
||||||
|
// unavailable; degrade silently.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void _onEvent(dynamic event) {
|
||||||
|
if (event is! Map) return;
|
||||||
|
final flags = (event['flags'] as num?)?.toInt() ?? 0;
|
||||||
|
_current = PenHardwareState(
|
||||||
|
barrel: flags & 0x1 != 0,
|
||||||
|
inverted: flags & 0x2 != 0,
|
||||||
|
eraser: flags & 0x4 != 0,
|
||||||
|
tiltX: (event['tiltX'] as num?)?.toDouble() ?? 0.0,
|
||||||
|
tiltY: (event['tiltY'] as num?)?.toDouble() ?? 0.0,
|
||||||
|
);
|
||||||
|
_active = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Stops listening and resets state.
|
||||||
|
void stop() {
|
||||||
|
_sub?.cancel();
|
||||||
|
_sub = null;
|
||||||
|
_active = false;
|
||||||
|
_current = PenHardwareState.empty;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -93,6 +93,15 @@ class _PenSettingsSheet extends StatelessWidget {
|
|||||||
icon: Icons.compress,
|
icon: Icons.compress,
|
||||||
colorScheme: colorScheme,
|
colorScheme: colorScheme,
|
||||||
),
|
),
|
||||||
|
_SliderTile(
|
||||||
|
label: 'Pressure Sensitivity',
|
||||||
|
value: config.pressureSensitivity,
|
||||||
|
min: 0.0,
|
||||||
|
max: 1.0,
|
||||||
|
divisions: 20,
|
||||||
|
formatValue: (v) => v.toStringAsFixed(2),
|
||||||
|
onChanged: controller.setPressureSensitivity,
|
||||||
|
),
|
||||||
_SliderTile(
|
_SliderTile(
|
||||||
label: 'Pressure Gamma',
|
label: 'Pressure Gamma',
|
||||||
value: config.pressureGamma,
|
value: config.pressureGamma,
|
||||||
|
|||||||
@@ -296,7 +296,7 @@ class PdfService {
|
|||||||
|
|
||||||
final pfPoints = stroke.points
|
final pfPoints = stroke.points
|
||||||
.map(
|
.map(
|
||||||
(pt) => pf.Point(
|
(pt) => pf.PointVector(
|
||||||
pt.x * pageSize.width,
|
pt.x * pageSize.width,
|
||||||
pt.y * pageSize.height,
|
pt.y * pageSize.height,
|
||||||
pt.pressure,
|
pt.pressure,
|
||||||
@@ -306,21 +306,24 @@ class PdfService {
|
|||||||
|
|
||||||
final outline = pf.getStroke(
|
final outline = pf.getStroke(
|
||||||
pfPoints,
|
pfPoints,
|
||||||
size: pixelWidth,
|
options: pf.StrokeOptions(
|
||||||
// Highlighter keeps constant width; pen/marker taper via thinning=0.7.
|
size: pixelWidth,
|
||||||
thinning: isHighlighter ? 0.0 : 0.7,
|
// Highlighter keeps constant width; pen/marker taper via thinning=0.7.
|
||||||
smoothing: 0.5,
|
thinning: isHighlighter ? 0.0 : 0.7,
|
||||||
streamline: 0.5,
|
smoothing: 0.5,
|
||||||
// Real stylus pressure -> don't simulate; no pressure -> let freehand
|
streamline: 0.5,
|
||||||
// fake it based on velocity. Highlighter never simulates.
|
// Real stylus pressure -> don't simulate; no pressure -> let freehand
|
||||||
simulatePressure: !hasRealPressure && !isHighlighter,
|
// fake it based on velocity. Highlighter never simulates.
|
||||||
isComplete: true,
|
simulatePressure: !hasRealPressure && !isHighlighter,
|
||||||
|
isComplete: true,
|
||||||
|
),
|
||||||
);
|
);
|
||||||
|
|
||||||
if (outline.isEmpty) return null;
|
if (outline.isEmpty) return null;
|
||||||
|
|
||||||
final path = PdfPath();
|
final path = PdfPath();
|
||||||
path.addPolygon(outline.map((pt) => Offset(pt.x, pt.y)).toList());
|
// outline is already List<Offset> in perfect_freehand 2.x.
|
||||||
|
path.addPolygon(outline.toList());
|
||||||
return path;
|
return path;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -139,7 +139,7 @@ class StrokeRasterizer {
|
|||||||
) {
|
) {
|
||||||
final pfPoints = points
|
final pfPoints = points
|
||||||
.map(
|
.map(
|
||||||
(p) => pf.Point(
|
(p) => pf.PointVector(
|
||||||
p.x,
|
p.x,
|
||||||
p.y,
|
p.y,
|
||||||
_defaultPressureCurve.apply(p.pressure).clamp(0.0, 1.0),
|
_defaultPressureCurve.apply(p.pressure).clamp(0.0, 1.0),
|
||||||
@@ -153,18 +153,20 @@ class StrokeRasterizer {
|
|||||||
|
|
||||||
final outline = pf.getStroke(
|
final outline = pf.getStroke(
|
||||||
pfPoints,
|
pfPoints,
|
||||||
size: strokeWidth,
|
options: pf.StrokeOptions(
|
||||||
thinning: thinning,
|
size: strokeWidth,
|
||||||
smoothing: 0.5,
|
thinning: thinning,
|
||||||
streamline: 0.5,
|
smoothing: 0.5,
|
||||||
simulatePressure: tool != PenTool.marker && tool != PenTool.highlighter,
|
streamline: 0.5,
|
||||||
isComplete: true,
|
simulatePressure: tool != PenTool.marker && tool != PenTool.highlighter,
|
||||||
|
isComplete: true,
|
||||||
|
),
|
||||||
);
|
);
|
||||||
if (outline.isEmpty) return;
|
if (outline.isEmpty) return;
|
||||||
|
|
||||||
final path = Path()..moveTo(outline[0].x, outline[0].y);
|
final path = Path()..moveTo(outline[0].dx, outline[0].dy);
|
||||||
for (var i = 1; i < outline.length; i++) {
|
for (var i = 1; i < outline.length; i++) {
|
||||||
path.lineTo(outline[i].x, outline[i].y);
|
path.lineTo(outline[i].dx, outline[i].dy);
|
||||||
}
|
}
|
||||||
path.close();
|
path.close();
|
||||||
|
|
||||||
|
|||||||
@@ -528,7 +528,7 @@ class _InkPainter extends CustomPainter {
|
|||||||
) {
|
) {
|
||||||
final pfPoints = points
|
final pfPoints = points
|
||||||
.map(
|
.map(
|
||||||
(p) => pf.Point(
|
(p) => pf.PointVector(
|
||||||
p.x,
|
p.x,
|
||||||
p.y,
|
p.y,
|
||||||
pressureCurve.apply(p.pressure).clamp(0.0, 1.0),
|
pressureCurve.apply(p.pressure).clamp(0.0, 1.0),
|
||||||
@@ -542,25 +542,25 @@ class _InkPainter extends CustomPainter {
|
|||||||
|
|
||||||
final outlinePoints = pf.getStroke(
|
final outlinePoints = pf.getStroke(
|
||||||
pfPoints,
|
pfPoints,
|
||||||
size: strokeWidth,
|
options: pf.StrokeOptions(
|
||||||
thinning: thinning,
|
size: strokeWidth,
|
||||||
smoothing: 0.5,
|
thinning: thinning,
|
||||||
streamline: 0.5,
|
smoothing: 0.5,
|
||||||
taperStart: 0.0,
|
streamline: 0.5,
|
||||||
taperEnd: 0.0,
|
// 2.x defaults: no taper + capped ends (was taperStart/End:0 +
|
||||||
capStart: true,
|
// capStart/End:true in 1.0.4).
|
||||||
capEnd: true,
|
simulatePressure: tool != PenTool.marker && tool != PenTool.highlighter,
|
||||||
simulatePressure: tool != PenTool.marker && tool != PenTool.highlighter,
|
isComplete: isComplete,
|
||||||
isComplete: isComplete,
|
),
|
||||||
);
|
);
|
||||||
|
|
||||||
if (outlinePoints.isEmpty) return;
|
if (outlinePoints.isEmpty) return;
|
||||||
|
|
||||||
final path = Path();
|
final path = Path();
|
||||||
path.moveTo(outlinePoints[0].x, outlinePoints[0].y);
|
path.moveTo(outlinePoints[0].dx, outlinePoints[0].dy);
|
||||||
|
|
||||||
for (int i = 1; i < outlinePoints.length; i++) {
|
for (int i = 1; i < outlinePoints.length; i++) {
|
||||||
path.lineTo(outlinePoints[i].x, outlinePoints[i].y);
|
path.lineTo(outlinePoints[i].dx, outlinePoints[i].dy);
|
||||||
}
|
}
|
||||||
path.close();
|
path.close();
|
||||||
|
|
||||||
|
|||||||
@@ -787,10 +787,10 @@ packages:
|
|||||||
dependency: "direct main"
|
dependency: "direct main"
|
||||||
description:
|
description:
|
||||||
name: perfect_freehand
|
name: perfect_freehand
|
||||||
sha256: "77bfdd5efb223d120de5cc18c5d6e0b36a835e920521cfe67e281960bad44c9b"
|
sha256: f42b8164c4e7e689b278f4e2e8e8f5006e716f8c127eb44404f21bd73b1d3b56
|
||||||
url: "https://pub.dev"
|
url: "https://pub.dev"
|
||||||
source: hosted
|
source: hosted
|
||||||
version: "1.0.4"
|
version: "2.5.2+1"
|
||||||
petitparser:
|
petitparser:
|
||||||
dependency: transitive
|
dependency: transitive
|
||||||
description:
|
description:
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ dependencies:
|
|||||||
cupertino_icons: ^1.0.8
|
cupertino_icons: ^1.0.8
|
||||||
|
|
||||||
# Pen & Ink Rendering
|
# Pen & Ink Rendering
|
||||||
perfect_freehand: ^1.0.0
|
perfect_freehand: ^2.0.0
|
||||||
|
|
||||||
# PDF
|
# PDF
|
||||||
syncfusion_flutter_pdfviewer: ^28.2.7
|
syncfusion_flutter_pdfviewer: ^28.2.7
|
||||||
|
|||||||
@@ -4,6 +4,8 @@
|
|||||||
// applyPressureCurve. No widgets, no SharedPreferences, no Flutter framework.
|
// applyPressureCurve. No widgets, no SharedPreferences, no Flutter framework.
|
||||||
|
|
||||||
import 'package:flutter_test/flutter_test.dart';
|
import 'package:flutter_test/flutter_test.dart';
|
||||||
|
import 'package:badnote/editor/engine/stroke_geometry.dart'
|
||||||
|
show kDefaultPenThinning;
|
||||||
import 'package:badnote/editor/input/pen_config.dart';
|
import 'package:badnote/editor/input/pen_config.dart';
|
||||||
|
|
||||||
void main() {
|
void main() {
|
||||||
@@ -43,9 +45,11 @@ void main() {
|
|||||||
fingerDrawing: true,
|
fingerDrawing: true,
|
||||||
penWidth: 0.008,
|
penWidth: 0.008,
|
||||||
highlighterWidth: 0.03,
|
highlighterWidth: 0.03,
|
||||||
|
pressureSensitivity: 0.42,
|
||||||
);
|
);
|
||||||
final restored = PenConfig.fromJson(original.toJson());
|
final restored = PenConfig.fromJson(original.toJson());
|
||||||
expect(restored, equals(original));
|
expect(restored, equals(original));
|
||||||
|
expect(restored.pressureSensitivity, 0.42);
|
||||||
});
|
});
|
||||||
|
|
||||||
test('fromJson falls back to defaults for missing keys', () {
|
test('fromJson falls back to defaults for missing keys', () {
|
||||||
@@ -57,6 +61,9 @@ void main() {
|
|||||||
expect(restored.fingerDrawing, false);
|
expect(restored.fingerDrawing, false);
|
||||||
expect(restored.penWidth, 0.004);
|
expect(restored.penWidth, 0.004);
|
||||||
expect(restored.highlighterWidth, 0.02);
|
expect(restored.highlighterWidth, 0.02);
|
||||||
|
// Missing pressureSensitivity defaults to the shared thinning constant,
|
||||||
|
// preserving the pre-existing stroke feel + export golden (plan M4).
|
||||||
|
expect(restored.pressureSensitivity, kDefaultPenThinning);
|
||||||
});
|
});
|
||||||
|
|
||||||
test('fromJson falls back to defaults for unknown enum names', () {
|
test('fromJson falls back to defaults for unknown enum names', () {
|
||||||
|
|||||||
104
test/pen_polish_test.dart
Normal file
104
test/pen_polish_test.dart
Normal file
@@ -0,0 +1,104 @@
|
|||||||
|
// test/pen_polish_test.dart
|
||||||
|
//
|
||||||
|
// Guards the pen-polish work (W1 configurable thinning + W3 tilt):
|
||||||
|
// - the on-screen painter (ink_painters.buildStrokePath) and the export/engine
|
||||||
|
// path (stroke_geometry.buildStrokeOutline) share ONE thinning default
|
||||||
|
// (kDefaultPenThinning) and respond to it identically — the user's core
|
||||||
|
// "thinning 写死" complaint, and the single-source-of-truth invariant the
|
||||||
|
// Critic required;
|
||||||
|
// - tilt survives the PenStroke -> EditorStroke adapter (W3 model fix).
|
||||||
|
|
||||||
|
import 'dart:ui';
|
||||||
|
|
||||||
|
import 'package:flutter_test/flutter_test.dart';
|
||||||
|
|
||||||
|
import 'package:badnote/editor/canvas/ink_painters.dart';
|
||||||
|
import 'package:badnote/editor/canvas/pen_stroke.dart';
|
||||||
|
import 'package:badnote/editor/engine/stroke_geometry.dart';
|
||||||
|
import 'package:badnote/editor/engine/stroke_model.dart';
|
||||||
|
|
||||||
|
PenStroke _pressuredPen() => PenStroke(
|
||||||
|
// Varied pressure, deliberately NOT reaching full force, so that
|
||||||
|
// thinning>0 (pressure-modulated, narrower) vs thinning=0 (constant full
|
||||||
|
// width) produces a measurably different bounding box.
|
||||||
|
points: const [
|
||||||
|
PenPoint(0.10, 0.50, 0.10),
|
||||||
|
PenPoint(0.30, 0.50, 0.25),
|
||||||
|
PenPoint(0.50, 0.50, 0.40),
|
||||||
|
PenPoint(0.70, 0.50, 0.55),
|
||||||
|
],
|
||||||
|
color: 0xFF000000,
|
||||||
|
width: 0.01,
|
||||||
|
kind: PenStrokeKind.pen,
|
||||||
|
);
|
||||||
|
|
||||||
|
void main() {
|
||||||
|
const size = Size(1000, 1000);
|
||||||
|
|
||||||
|
group('W1 — thinning is configurable and single-sourced', () {
|
||||||
|
test('buildStrokePath default == explicit kDefaultPenThinning', () {
|
||||||
|
final pen = _pressuredPen();
|
||||||
|
final byDefault = buildStrokePath(pen, size, isComplete: true);
|
||||||
|
final explicit = buildStrokePath(pen, size,
|
||||||
|
isComplete: true, thinning: kDefaultPenThinning);
|
||||||
|
expect(byDefault.getBounds(), explicit.getBounds());
|
||||||
|
});
|
||||||
|
|
||||||
|
test('thinning actually affects the outline (not hardcoded/ignored)', () {
|
||||||
|
// NB: the bounding box is thinning-INVARIANT here because
|
||||||
|
// perfect_freehand's round end-caps are drawn at the full `size`; only
|
||||||
|
// the mid-section width tracks pressure×thinning. So we compare the
|
||||||
|
// outline PERIMETER (sum of contour lengths), which does reflect the
|
||||||
|
// pinched middle.
|
||||||
|
double perimeter(Path p) =>
|
||||||
|
p.computeMetrics().fold(0.0, (sum, m) => sum + m.length);
|
||||||
|
final pen = _pressuredPen();
|
||||||
|
final strong = perimeter(
|
||||||
|
buildStrokePath(pen, size, isComplete: true, thinning: 0.85));
|
||||||
|
final none = perimeter(
|
||||||
|
buildStrokePath(pen, size, isComplete: true, thinning: 0.0));
|
||||||
|
// Constant width (0.0) vs pressure-thinning (0.85) must differ measurably.
|
||||||
|
expect((strong - none).abs(), greaterThan(1.0),
|
||||||
|
reason: 'thinning had no effect on the outline — it is not wired');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('screen and export builders agree for the same stroke + thinning', () {
|
||||||
|
final pen = _pressuredPen();
|
||||||
|
final editor = EditorStroke.fromPenStroke(pen);
|
||||||
|
for (final thinning in const [0.0, kDefaultPenThinning, 1.0]) {
|
||||||
|
final screen =
|
||||||
|
buildStrokePath(pen, size, isComplete: true, thinning: thinning)
|
||||||
|
.getBounds();
|
||||||
|
final export =
|
||||||
|
buildStrokeOutline(editor, size, isComplete: true, thinning: thinning)
|
||||||
|
.getBounds();
|
||||||
|
expect(screen, export,
|
||||||
|
reason: 'screen vs export diverged at thinning=$thinning');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
group('W3 — tilt survives the PenStroke adapter', () {
|
||||||
|
test('PenPoint.tilt maps into EditorPoint.tilt', () {
|
||||||
|
const pen = PenStroke(
|
||||||
|
points: [PenPoint(0.1, 0.2, 0.5, tilt: 23.5)],
|
||||||
|
color: 0xFF112233,
|
||||||
|
width: 0.01,
|
||||||
|
kind: PenStrokeKind.pen,
|
||||||
|
);
|
||||||
|
final editor = EditorStroke.fromPenStroke(pen);
|
||||||
|
expect(editor.points.single.tilt, 23.5);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('null tilt stays null through the adapter', () {
|
||||||
|
const pen = PenStroke(
|
||||||
|
points: [PenPoint(0.1, 0.2, 0.5)],
|
||||||
|
color: 0xFF112233,
|
||||||
|
width: 0.01,
|
||||||
|
kind: PenStrokeKind.pen,
|
||||||
|
);
|
||||||
|
final editor = EditorStroke.fromPenStroke(pen);
|
||||||
|
expect(editor.points.single.tilt, isNull);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -10,6 +10,7 @@ add_executable(${BINARY_NAME} WIN32
|
|||||||
"flutter_window.cpp"
|
"flutter_window.cpp"
|
||||||
"main.cpp"
|
"main.cpp"
|
||||||
"ocr_channel.cpp"
|
"ocr_channel.cpp"
|
||||||
|
"pen_channel.cpp"
|
||||||
"utils.cpp"
|
"utils.cpp"
|
||||||
"win32_window.cpp"
|
"win32_window.cpp"
|
||||||
"${FLUTTER_MANAGED_DIR}/generated_plugin_registrant.cc"
|
"${FLUTTER_MANAGED_DIR}/generated_plugin_registrant.cc"
|
||||||
|
|||||||
@@ -4,6 +4,7 @@
|
|||||||
|
|
||||||
#include "flutter/generated_plugin_registrant.h"
|
#include "flutter/generated_plugin_registrant.h"
|
||||||
#include "ocr_channel.h"
|
#include "ocr_channel.h"
|
||||||
|
#include "pen_channel.h"
|
||||||
|
|
||||||
FlutterWindow::FlutterWindow(const flutter::DartProject& project)
|
FlutterWindow::FlutterWindow(const flutter::DartProject& project)
|
||||||
: project_(project) {}
|
: project_(project) {}
|
||||||
@@ -27,6 +28,7 @@ bool FlutterWindow::OnCreate() {
|
|||||||
}
|
}
|
||||||
RegisterPlugins(flutter_controller_->engine());
|
RegisterPlugins(flutter_controller_->engine());
|
||||||
RegisterOcrChannel(flutter_controller_->engine());
|
RegisterOcrChannel(flutter_controller_->engine());
|
||||||
|
RegisterPenChannel(flutter_controller_->engine());
|
||||||
SetChildContent(flutter_controller_->view()->GetNativeWindow());
|
SetChildContent(flutter_controller_->view()->GetNativeWindow());
|
||||||
|
|
||||||
flutter_controller_->engine()->SetNextFrameCallback([&]() {
|
flutter_controller_->engine()->SetNextFrameCallback([&]() {
|
||||||
@@ -53,6 +55,9 @@ LRESULT
|
|||||||
FlutterWindow::MessageHandler(HWND hwnd, UINT const message,
|
FlutterWindow::MessageHandler(HWND hwnd, UINT const message,
|
||||||
WPARAM const wparam,
|
WPARAM const wparam,
|
||||||
LPARAM const lparam) noexcept {
|
LPARAM const lparam) noexcept {
|
||||||
|
// Observe pen messages before Flutter consumes WM_POINTER events.
|
||||||
|
ObservePenMessage(message, wparam, lparam);
|
||||||
|
|
||||||
// Give Flutter, including plugins, an opportunity to handle window messages.
|
// Give Flutter, including plugins, an opportunity to handle window messages.
|
||||||
if (flutter_controller_) {
|
if (flutter_controller_) {
|
||||||
std::optional<LRESULT> result =
|
std::optional<LRESULT> result =
|
||||||
|
|||||||
89
windows/runner/pen_channel.cpp
Normal file
89
windows/runner/pen_channel.cpp
Normal file
@@ -0,0 +1,89 @@
|
|||||||
|
#include "pen_channel.h"
|
||||||
|
|
||||||
|
#include <flutter/encodable_value.h>
|
||||||
|
#include <flutter/event_channel.h>
|
||||||
|
#include <flutter/event_stream_handler_functions.h>
|
||||||
|
#include <flutter/flutter_engine.h>
|
||||||
|
#include <flutter/standard_method_codec.h>
|
||||||
|
|
||||||
|
#include <memory>
|
||||||
|
|
||||||
|
namespace {
|
||||||
|
|
||||||
|
std::unique_ptr<flutter::EventSink<flutter::EncodableValue>> g_pen_sink;
|
||||||
|
|
||||||
|
std::unique_ptr<flutter::EventChannel<flutter::EncodableValue>> g_pen_channel;
|
||||||
|
|
||||||
|
} // namespace
|
||||||
|
|
||||||
|
void RegisterPenChannel(flutter::FlutterEngine* engine) {
|
||||||
|
g_pen_channel =
|
||||||
|
std::make_unique<flutter::EventChannel<flutter::EncodableValue>>(
|
||||||
|
engine->messenger(), "badnote/pen",
|
||||||
|
&flutter::StandardMethodCodec::GetInstance());
|
||||||
|
|
||||||
|
auto handler = std::make_unique<
|
||||||
|
flutter::StreamHandlerFunctions<flutter::EncodableValue>>(
|
||||||
|
[](const flutter::EncodableValue* arguments,
|
||||||
|
std::unique_ptr<flutter::EventSink<flutter::EncodableValue>>&&
|
||||||
|
events)
|
||||||
|
-> std::unique_ptr<
|
||||||
|
flutter::StreamHandlerError<flutter::EncodableValue>> {
|
||||||
|
g_pen_sink = std::move(events);
|
||||||
|
return nullptr;
|
||||||
|
},
|
||||||
|
[](const flutter::EncodableValue* arguments)
|
||||||
|
-> std::unique_ptr<
|
||||||
|
flutter::StreamHandlerError<flutter::EncodableValue>> {
|
||||||
|
g_pen_sink = nullptr;
|
||||||
|
return nullptr;
|
||||||
|
});
|
||||||
|
|
||||||
|
g_pen_channel->SetStreamHandler(std::move(handler));
|
||||||
|
}
|
||||||
|
|
||||||
|
void ObservePenMessage(UINT message, WPARAM wparam, LPARAM lparam) {
|
||||||
|
if (message != WM_POINTERENTER && message != WM_POINTERDOWN &&
|
||||||
|
message != WM_POINTERUPDATE && message != WM_POINTERUP) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!g_pen_sink) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
UINT32 pointerId = GET_POINTERID_WPARAM(wparam);
|
||||||
|
|
||||||
|
POINTER_INPUT_TYPE type = PT_POINTER;
|
||||||
|
if (!GetPointerType(pointerId, &type) || type != PT_PEN) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
POINTER_PEN_INFO ppi{};
|
||||||
|
if (!GetPointerPenInfo(pointerId, &ppi)) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
int flags = 0;
|
||||||
|
if (ppi.penFlags & PEN_FLAG_BARREL) flags |= 1;
|
||||||
|
if (ppi.penFlags & PEN_FLAG_INVERTED) flags |= 2;
|
||||||
|
if (ppi.penFlags & PEN_FLAG_ERASER) flags |= 4;
|
||||||
|
|
||||||
|
flutter::EncodableMap payload{
|
||||||
|
{flutter::EncodableValue("flags"), flutter::EncodableValue(flags)},
|
||||||
|
{flutter::EncodableValue("tiltX"), flutter::EncodableValue(static_cast<double>(ppi.tiltX))},
|
||||||
|
{flutter::EncodableValue("tiltY"), flutter::EncodableValue(static_cast<double>(ppi.tiltY))},
|
||||||
|
};
|
||||||
|
|
||||||
|
g_pen_sink->Success(flutter::EncodableValue(payload));
|
||||||
|
|
||||||
|
// On pointer up, send a cleared flags event to signal lift-off.
|
||||||
|
if (message == WM_POINTERUP) {
|
||||||
|
flutter::EncodableMap clear{
|
||||||
|
{flutter::EncodableValue("flags"), flutter::EncodableValue(0)},
|
||||||
|
{flutter::EncodableValue("tiltX"), flutter::EncodableValue(0.0)},
|
||||||
|
{flutter::EncodableValue("tiltY"), flutter::EncodableValue(0.0)},
|
||||||
|
};
|
||||||
|
g_pen_sink->Success(flutter::EncodableValue(clear));
|
||||||
|
}
|
||||||
|
}
|
||||||
13
windows/runner/pen_channel.h
Normal file
13
windows/runner/pen_channel.h
Normal file
@@ -0,0 +1,13 @@
|
|||||||
|
#ifndef RUNNER_PEN_CHANNEL_H_
|
||||||
|
#define RUNNER_PEN_CHANNEL_H_
|
||||||
|
|
||||||
|
#include <windows.h>
|
||||||
|
|
||||||
|
namespace flutter {
|
||||||
|
class FlutterEngine;
|
||||||
|
}
|
||||||
|
|
||||||
|
void RegisterPenChannel(flutter::FlutterEngine* engine);
|
||||||
|
void ObservePenMessage(UINT message, WPARAM wparam, LPARAM lparam);
|
||||||
|
|
||||||
|
#endif // RUNNER_PEN_CHANNEL_H_
|
||||||
Reference in New Issue
Block a user