Zoom flicker root cause (from the on-device badnote_input_log.txt): the pinch
computed its per-frame scale change as desiredScale / getMaxScaleOnAxis(), i.e.
it fed the LIVE matrix back into its own update. Consecutive frames in the log
show `cur` (the live read) dropping to 0.75-0.89 for a single frame while the
result track stayed smooth, so the code demanded a 1.3-1.4x correction that
popped the zoom bigger/smaller and snapped back. The >1.4 glitch guard missed it
because the spikes sat at 1.31-1.40.
Fix: the scale branch of PenInteractiveViewer now drives the transform
ABSOLUTELY from a gesture-start snapshot (_scaleStart, _referenceFocalPoint) plus
the recognizer's clean, monotonic cumulative details.scale. Each frame is fully
re-derived in closed form (pure scale+translate, no matrix inversion, no live
read-back), so a transient mis-read or interleaved write cannot survive into the
next frame. The per-frame glitch guard now keys on the recognizer's own
scale-ratio (the true finger motion) instead of the corrupted live read. 2-finger
pan still falls out of the same focal-anchor formula.
Pen feel: lower perfect_freehand streamline 0.5 -> 0.32 (new shared constants
kPenStreamline/kPenSmoothing, single-sourced across screen + export so the
parity test still holds). At 0.5 a quick flick lagged so far behind the pen that
short fast strokes collapsed toward their start and rendered as a dot
("写字识别成单击"); 0.32 tracks the real path for a crisper, lower-latency feel.
flutter analyze lib/editor clean; 66/66 tests pass (incl. screen==export parity).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Buttons (likely fix + ground truth): device diag showed ptr=12577 pen=10056 —
WM_POINTER reaches the observer and GetPointerPenInfo succeeds, so the buttons
were just read from the wrong field. Native now resolves the barrel from BOTH
penFlags(PEN_FLAG_BARREL) AND pointerInfo.pointerFlags(POINTER_FLAG_SECONDBUTTON)
— many pens use the latter. It also emits the full raw set (pointerFlags,
penFlags, penMask, ButtonChangeType, tilt) plus OR-accumulated flags so a single
session reveals exactly which field each button sets.
Comprehensive logging (per user request "用好用的log库 / 我手动开启日志再记录"):
new DiagnosticLogger emits through dart:developer log(name 'badnote.input') —
capturable via `flutter run` / DevTools / `flutter logs` — AND mirrors to a file
(path shown in the overlay) for the packaged GUI build that has no console.
Manually enabled by the toolbar diagnostic toggle; off by default. PEN lines log
on raw-field change; ZOOM lines log every scale frame + rebaselines.
Zoom: scale-only glitch rejection didn't stop the jumping, so add focal/position
glitch rejection — drop a 2-finger frame whose focal jumps >250px (a touch
misread). The full per-frame trace (raw scale, pointerCount, applied change,
focal jump, drops) is now logged so the residual cause is unambiguous.
InputDiagnostics singleton accumulates the stats; the overlay shows summary +
last trace lines + log path + reset. Removed the ad-hoc inline zoom min/max.
Dart: analyze clean, 66/66 tests, linux build green. Native compiles on CI.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Eraser (regression from the preview I added):
- LAG: the preview did setState on every hover/erase-move (rebuilding the whole
canvas) and recomputed perfect_freehand getStroke per overlapped stroke per
frame. Now the cursor is a ValueNotifier driving the preview layer's repaint
directly (no canvas rebuild), and the highlight is a plain polyline of the
point-runs inside the radius (no getStroke).
- STUCK RED ("一直红着"): the cursor was never cleared. Preview is now
active-erase-only and cleared on pen up/cancel.
- "选中了的笔画也不见得能删掉": radius was strokeWidth*2 (tiny) so a pass removed
~2 points and the stroke survived. Now a decisive fixed 0.02 (page-width
fraction). The highlight traces exactly the point-run that splitStrokeByCircle
removes, so what turns red is what gets deleted.
Zoom: replace the per-frame scale CLAMP with glitch REJECTION — drop a frame
demanding an implausible per-frame scale jump (>1.4x or <0.71x; a real pinch is
≲1.15x/frame). A dropped frame catches up the next frame (absolute tracking), so
no lag, but the Windows multi-touch spike never shows. Pairs with the existing
pointer-count re-baseline.
Native diagnostic: ObservePenMessage now counts WM_POINTER* / PT_PEN / legacy
mouse messages it sees and emits them on the channel; PenInputService exposes
`debugSummary` and the overlay shows `native ptr=… pen=… mouse=… msg=0x…`. This
will tell us on-device whether WM_POINTER ever reaches the observer (→ buttons
recoverable) or Flutter is on a non-pointer path (→ not).
Dart: analyze clean, 66/66 tests, linux build green. Native compiles on CI.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Zoom jumping (device: min 0.5 / max 2.47 while zooming near 1): the per-frame
scale clamp limited single-frame spikes but not multi-frame runs. Root cause is
pointer-count transitions — on Windows touch the two fingers land/lift at
different times and digitizers drop/re-acquire touches, and stock InteractiveViewer
keeps _scaleStart/_referenceFocalPoint from the OLD finger set, so the next frame
jumps. PenInteractiveViewer now re-baselines (and skips the transitional frame)
whenever details.pointerCount changes. The per-frame clamp stays as a secondary
guard.
Buttons (device evidence: btn=1 for tip-down, side-button, AND inverted; kind
never becomes invertedStylus): Flutter does NOT surface the barrel/eraser/inverted
state at all — unreachable from Dart. The only path is the native badnote/pen
plugin, which was SILENT because WM_POINTER is delivered to the Flutter CHILD
view window, not the top-level FlutterWindow where ObservePenMessage was hooked.
Fix: subclass the child HWND (SetWindowSubclass + comctl32) and observe its
WM_POINTER messages, passing every message through unchanged via DefSubclassProc
(observation-only, input behavior preserved). This is what should finally feed
GetPointerPenInfo penFlags + tilt to the channel — to be confirmed on-device with
the diagnostic (btn / kind / tilt readout).
Dart: analyze clean, 66/66 tests, linux build green. Native compiles on Windows CI.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace stock InteractiveViewer with PenInteractiveViewer, a focused fork of
Flutter 3.44's InteractiveViewer for our config (constrained=false, infinite
boundary, no rotation — that machinery dropped as a no-op here). Two deliberate
changes, grounded in the Rnote/Saber research:
1. The pan/zoom ScaleGestureRecognizer excludes stylus/invertedStylus via
`supportedDevices`. The pen never reaches it, so a stylus stroke can no longer
be stolen as a pan on its first frame (the "写字识别成单击" feel bug, caused by
stock IV's panEnabled updating a frame after the stroke began). Drawing is
owned solely by the canvas Listener; no arena fight, no panEnabled lag. The
prior _lastStylus hover hack is removed (superseded).
2. Per-frame scale change is clamped (×0.74..×1.35). Stock IV already damps focal
jitter and guards the pan branch, but a single-frame multi-touch glitch could
still spike details.scale, popping the zoom bigger/smaller and snapping back
(the reported pinch flicker). Clamping swallows the spike; a real (gradual)
pinch is unaffected since scale tracks absolutely from gesture start. Cap is
far above any real pinch (~1.1-1.2x/frame), so no felt lag.
Everything else (scale-about-focal, pan, fling inertia, mouse-wheel zoom) is
Flutter's proven logic verbatim.
Also add an on-device input diagnostic (bug-report toggle): the existing pen
readout already prints kind/pressure/buttons; now it also shows live
zoom=now/min/max so the next device test captures (a) whether the side/eraser
button arrives as buttons/invertedStylus, and (b) the value any residual pinch
flash jumps to. 66/66 tests, analyze clean, linux build green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Eraser preview (user request, "加一个淡一点的描边"): new EraserPreviewPainter
shows the eraser circle and a faint red outline over the committed strokes the
eraser currently overlaps, so you can see what is about to be deleted. Mounted
only in eraser mode (tool or barrel/inverted signal) with a live cursor that
follows the hovering/erasing pen; shares _eraserRadius/_pageAspect with the live
erase so preview and action always agree. Kept in its own RepaintBoundary.
Pen-feel fix (the "写字识别成单击" pan-steal): panEnabled now also requires the
last pointer to not be a stylus. Windows fires stylus HOVER before contact, so
_lastStylus is already true when the pen touches down -> the InteractiveViewer's
pan is disabled BEFORE the stroke's first move, instead of one frame late. A
2+ pointer pinch still always pans (focal translation); a finger/mouse down
flips _lastStylus back so finger-pan keeps working.
Grounded in Rnote + Saber research: Saber uses the same button detection we have
(buttons==kSecondaryButton || invertedStylus); the deeper zoom-flash / draw-vs-
pan robustness wants a Saber-style forked InteractiveViewer (single recognizer,
decide-at-start) -- scoped as the next step, not done here. 66/66 tests, linux
build green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
W4/P0 engine: add engine/stroke_eraser.dart (pure, aspect-corrected) with
whole-stroke `strokeHit` + partial `splitStrokeByCircle`. Grazing a long
stroke now CUTS it into surviving pieces instead of deleting it whole.
Wired through PenCanvas.onEraseStroke (now (index, replacements)) →
pen_editor_screen._eraseStroke (replaceRange); undo/persistence unchanged
(whole-page snapshot). 8 new unit tests; 66/66 pass.
Fix side-button (侧键): _isEraserSignal used `buttons == kSecondaryButton`,
but tip-down + barrel = kStylusContact|kPrimaryStylusButton = 0x03, so the
side button only registered on hover, never while drawing. Now a bitmask
test. (Eraser-end/tilt remain blocked on the silent native badnote/pen
channel — needs on-device native logging.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
W1 — Custom pen width + pressure sensitivity (Saber-style):
- Root cause of "压感没用": perfect_freehand 1.0.4 IGNORES real stylus pressure
(hardcodes radius=size/2 when simulatePressure=false) — width never tracked pen
force. Upgraded perfect_freehand ^1.0.0 -> ^2.0.0 (honors real pressure); migrated
all 5 getStroke call sites to the 2.x API (PointVector / StrokeOptions / Offset).
- De-hardcoded `thinning` into `kDefaultPenThinning` (0.85), single source shared by
the on-screen painter and the PDF export path; exposed as PenConfig.pressureSensitivity
with a Pressure Sensitivity slider; live-applies via a config listener.
W3 — Native Windows pen plugin (tilt + barrel/eraser buttons):
- windows/runner/pen_channel.{h,cpp}: observe WM_POINTER at the TOP of MessageHandler
(before HandleTopLevelWindowProc, which Flutter uses to consume pen events), read
GetPointerPenInfo penFlags + tilt, stream over EventChannel('badnote/pen'); non-consuming.
- PenInputService: single latched hardware state (no Win32-pointerId<->event.pointer
correlation); graceful no-op off-Windows.
- pen_canvas maps barrel/inverted/eraser through PenConfig.sideButton/eraserEnd
(eraser/undo/toggleTool/pan) and captures tilt into PenPoint.tilt -> EditorPoint.tilt.
W2 — Zoom flicker: page raster isolated in its own RepaintBoundary (safe interim);
definitive crisp-on-zoom fix gated on the on-device root-cause probe (plan M3).
Plans: ralplan-consensus plan at docs/plans/2026-06-22-badnote-pen-polish.md
(Architect APPROVE-WITH-MUST-FIX M1-M4 + Critic ITERATE->APPROVE).
Tests: 58/58 pass incl. shared-thinning invariant + thinning-affects-outline +
tilt-adapter round-trip. flutter analyze clean; linux debug build OK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire the new EditorRepository + SaveScheduler into PenEditorScreen:
load strokes on open (keyed by a stable djb2 doc-id from the path),
save per page on commit/erase via the debounced diff-write scheduler
(synchronous snapshot before await), flush on dispose. Strokes now
survive close/reopen. PenStroke<->EditorStroke conversion at the
boundary.
Page slider is no longer persistent: a compact prev/'n/total'/next pill;
tapping the label reveals the slider (collapses again), so it stops
blocking the page. Add a pen-pressure diagnostic toggle (bug icon) that
shows the live kind/pressure/min/max Windows delivers — to pin down why
pressure reads flat on the Surface Pen.
Strokes vanished on pen-up: StaticInkPainter aliased the same mutable
list so shouldRepaint saw no change. Commit/erase now replace the list.
Finger-drawing toggle wins over palm-rejection; pressure surfaces even
when the pen reports no min/max range; pages recenter after a flip; the
keyboard page-jump (unreliable on Windows) is now a drag slider. Also
register the dynamic_color plugin in generated registrants.
Clean-room reimplementation of Saber's input model: we own the gesture
pipeline so pen draws with real pressure (perfect_freehand), two-finger
pinch zooms/pans, and palm is rejected (stylus-priority, 2nd-pointer
cancels stroke). pdfrx renders one page at a time (PdfPageView, no
gestures) under a shared transform; page-based nav. Material You theme
via dynamic_color (system accent + seed fallback) and a floating tonal
tool palette + page pill. Old pdfrx-overlay spike no longer wired.
Surface Pen reports as PointerDeviceKind.stylus and multitouch is
delivered only on Flutter 3.44+ (WM_POINTER migration, PR #165323).
3.41.4 dropped pen events, so our stylus-kind capture got nothing and
pinch-zoom did not work. Pin CI to Flutter 3.44.2 (shadowing any cached
older SDK on the runner) and require flutter >=3.44.0 in pubspec.
pdfrx requires Windows Developer Mode (its build uses symbolic links);
set AllowDevelopmentWithoutDevLicense on the runner before building so
flutter build windows does not fail the symlink/dev-mode check.
pdfium_dart's native-assets hook downloads pdfium.dll from github via
package:http, which ignores HTTP_PROXY, so it fails behind the GFW.
Pre-fetch pdfium.dll through a gh-proxy mirror and pre-place it at the
hook's deterministic shared-output path (output.exists short-circuits
the download); also rewrite the hook URL to the mirror as a fallback.
Fix the build-step proxy address to the LAN proxy.
Add pdfrx 2.4.4. PenCaptureRegion routes stylus to ink (arena-bypass
via PenCaptureBinding) while touch falls through to pdfrx scroll/zoom.
Spike pane hosts PdfViewer with page-overlay ink at normalized coords +
frame-time HUD; reachable from home screen for on-device testing.
Bench/coordinate harness under integration_test. Generated assets are
gitignored (regenerate via tool/gen_*.dart).
pdfrx render core, static/live ink layers, RenderProxyBox pen/touch
arbitration, infinite board, text boxes. Planner-Architect-Critic
consensus APPROVE. MUST #1 (API source-pin) and MUST #2 (coordinate
assertion) verified on pdfium; MUST #3/#4/#5 pending on Surface Pen.
The app showed no window because RegisterOcrChannel ran
winrt::init_apartment() (defaults to MTA) on the Flutter platform
thread, which main.cpp already put in an STA via CoInitializeEx. That
throws RPC_E_CHANGED_MODE and kills the app before the window appears.
Remove that call; run the WinRT OCR on a dedicated MTA worker thread
per recognize call instead, joining before returning the result.
ocr_channel.cpp calls engine->messenger() but only had a forward
declaration of flutter::FlutterEngine. Include the engine header so
the type is complete (fixes C2027 + the cascading make_unique error
that broke the Windows compile at ocr_channel.cpp:76).
Run 11 reached the build and revealed two issues:
- Runner re-resolved sqlite3 to 3.3.3 (cn mirror), mismatching the
vendored 3.3.2 binary hash. Pin it via dependency_overrides.
- flutter_onnxruntime's github download of ONNX Runtime failed.
Pre-fetch it via a China-accessible proxy into the plugin's
expected build path so it skips the download and bundles the dll.
The concurrency block caused Gitea to not create a run for the push.
Gitea already auto-cancels the prior run on a new push, so it is not
needed. Keep the faster curl.exe SDK download.
Run 9's Invoke-WebRequest download of the ~1GB SDK was very slow. Use
curl.exe (bundled on Windows 10+) which streams to disk, with an IWR
fallback. Add a concurrency group so a new push cancels an in-progress
run instead of queuing behind it.
Run 8: Flutter is not on the runner (PATH, FLUTTER_ROOT and common
dirs are all empty). Make the workflow self-contained: if Flutter is
not found, download the pinned 3.41.4 SDK from storage.flutter-io.cn
into a persistent C:\flutter-ci dir (reused on later runs) and add
it to PATH.
Run 7 failed with 'Cannot find: pwsh in PATH'. The runner has Windows
PowerShell 5.1, not PowerShell 7. Switch explicit-shell steps to
'powershell'; all commands are 5.1-compatible.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
No workflow runs were being created for recent pushes (total_count frozen),
while the prior commit without these expressions did create runs. Gitea's
workflow parser appears to reject the `${{ vars.* }}` / `${{ secrets.* || ... }}`
expressions, so no run is triggered at all.
Make the workflow fully static: hardcode the local proxy for the ONNX Runtime
download, drop the vars.FLUTTER_ROOT reference (env FLUTTER_ROOT + common paths
still work), and remove the commented fallback block. README documents the
runner-side options.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of the failing Windows runs (seen in the runner logs): the Actions
shell does not inherit Flutter on PATH, so `flutter --version` failed in ~30s.
Checkout via the gitea.com mirror works fine.
Add a "Locate Flutter" step that checks PATH, the FLUTTER_ROOT repo variable,
and common Windows install locations, then prepends it via GITHUB_PATH so later
steps find flutter. Fails with a clear message if not found.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the Gitea CI maximally likely to produce a usable Windows .exe on a
self-hosted runner behind the GFW, since compilation must go through CI:
- Build is the priority: format/analyze/test now run with continue-on-error so
a toolchain-version lint difference can never block the actual compile.
- ONNX Runtime download: default HTTP(S)_PROXY to the local proxy
(http://127.0.0.1:7890, overridable via repo secrets) so CMake's
file(DOWNLOAD) can fetch it; documented system-install alternative.
- Checkout stays on the gitea.com mirror, with a commented manual-checkout
fallback (clones from the local Gitea) if gitea.com is unreachable.
- Artifact upload is best-effort; an explicit step prints the Release output
path so the binary is findable even if upload fails.
- Dropped the optional server job to keep the Windows build focused.
- README: documented the runner prerequisites (Flutter on PATH, VS C++ build
tools, proxy, gitea.com, host-mode runner) that the workflow can't set.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make on-device OCR a pluggable local service so it runs locally on every
platform (not just Windows), aimed at GoodNotes/Notability-class handwriting on
low-power hardware (e.g. Zen2 APU, CPU/iGPU).
- New OcrBackend abstraction (lib/services/ocr/): selector prefers an embedded
ONNX recognition backend, falling back to the OS-native backend (Windows
WinRT), and to a clean no-op when neither is available.
- OnnxRecognitionBackend: flutter_onnxruntime session from a bundled asset,
dart:ui preprocessing (resize to 48px, CHW float32, normalized), pure-Dart CTC
greedy decode. Fully guarded — absent model/dict is a no-op; never throws.
- ocr_engine.dart kept as a thin facade (recognizeImage) delegating to the
selector, so ocr_service.dart is unchanged.
- CtcDecoder unit-tested (6 tests). flutter analyze clean; all tests pass.
- Model is not committed; tool/fetch_ocr_model.sh + assets/models/ocr/README.md
document fetching PP-OCRv4 rec + dict on the dev machine.
- CI: forward HTTPS_PROXY to the Windows build so CMake can fetch the ONNX
Runtime native lib behind the GFW; README documents the system-install
alternative. PP-OCR geometry/blank assumptions documented for on-device tuning.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The self-hosted Windows runner can't reach github.com (GFW), so the previous
workflow failed cloning actions/checkout and would also fail downloading the
Flutter SDK and the sqlite3 native binary.
- Vendor the official, SHA-256-verified sqlite3 binaries under vendor/sqlite3/
and select them via pubspec hooks.user_defines (source: test-sqlite3). Builds
and tests now run fully offline — no GitHub download, no proxy, no
LD_LIBRARY_PATH hack (removed .local-sqlite/).
- Consolidate CI into one Windows workflow: fetch actions from the gitea.com
mirror, use the runner's pre-installed Flutter (no SDK download), and use the
flutter-io.cn pub/Flutter mirrors. Server tests use the Tsinghua PyPI mirror.
- Document the offline build + China mirrors in README.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bug fixes (Flutter):
- Wrap multi-statement DB writes (insert/update/delete note, deleteDocument,
deletePageData, OCR FTS merge, migrations) in transactions to prevent data
loss on interruption and a read-modify-write FTS race.
- Fix PdfDocument leaks on exception (try/finally dispose) and preserve image
aspect ratio when stamping images onto PDF pages.
- Guard file-picker against empty selection (was .single -> crash).
- Fix eraser ConcurrentModificationError and unmodifiable-list crash on PDF
pages; capture page synchronously on save to stop wrong-page data loss.
- Fix Riverpod DB-not-ready races, broken pull-to-refresh, settings load race,
and search N+1; transform stored annotations on PDF page rotation.
- Normalize pen pressure for devices without a pressure range.
- PPT: single source of truth for slide strokes so ink displays and exports.
UI/UX:
- Material 3 typography, theme-aware colors (dark-mode fixes), hover cursors
and right-click/visible actions on desktop, keyboard shortcuts (undo/redo/
save/find), toolbar overflow handling, friendlier empty states, semantic OCR
status badges, relative timestamps, 1-based page indicators, large-deck PPT
navigation, and a scratchpad-scope label in split view.
Server (optional backend):
- Persist JWT secret (was per-process random), block path traversal in storage,
fix CORS '*'+credentials, add OCR job ownership checks, last-writer-wins sync
guard, constant-time login, and split out heavy OCR deps so the API/tests run
without them.
CI: Gitea workflows for format+analyze+test (Linux, system sqlite) and a
Windows release build; pristine `flutter analyze`, all Flutter and server tests
green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
