feat(search): index PDF text, OCR scanned PDFs on import

Search now covers handwriting, the PDF text layer, AND scanned (rasterized) PDFs. - PdfTextIndexer runs at import: sums the embedded text layer across pages; if present it stores that as the document body, otherwise the PDF is rasterized and its rendered pages are OCR'd in the background. The result lands in the sidecar `pageText` field (distinct from `ocrText`, the handwriting OCR). Idempotent (skips a sidecar that already has pageText); degrades gracefully with no OCR engine. - pdfrx_page_text_source abstracts text/render so it's testable. - VaultSearchIndex now harvests title + typed text + handwriting OCR + PDF pageText, so search finds notes, typed PDFs and scanned PDFs. analyze clean, 409 tests green.
2026-06-25 00:23:19 +08:00
parent 20add27a30
commit e939759458
10 changed files with 668 additions and 16 deletions
--- a/lib/services/pdf_text_indexer.dart
+++ b/lib/services/pdf_text_indexer.dart
@@ -0,0 +1,198 @@
+// lib/services/pdf_text_indexer.dart
+//
+// Import-time document-body text indexing for file-backed PDF notebooks, so the
+// vault-scan search index (VaultSearchIndex) covers the underlying document —
+// not just the user's annotations. Three text sources now feed search:
+//   (a) handwriting        → OCR'd into the sidecar's `ocrText` (OcrService),
+//   (b) a PDF text layer   → its embedded printed text, captured here,
+//   (c) a RASTERIZED PDF   → background OCR of the rendered pages, captured here.
+//
+// Flow (kicked off after VaultService.createNotebook, fire-and-forget):
+//   1. IDEMPOTENCY: if the sidecar already carries `pageText`, do nothing.
+//   2. EXTRACT the embedded text layer per page (pdfrx `loadText`).
+//   3. DECIDE text-layer vs rasterized: sum the embedded text length across all
+//      pages; if it clears [textLayerThreshold] the PDF has a usable text layer
+//      and we persist that. Otherwise the PDF is rasterized (scanned image, no
+//      text) and we OCR each rendered page.
+//   4. PERSIST the per-page text (joined by form-feed) into the sidecar's
+//      `pageText` field — writing THROUGH an open SidecarRepository when the
+//      editor already has the doc open (no race), else a transient handle.
+//
+// GRACEFUL DEGRADATION / HONESTY:
+//   * The OCR engine is whatever OcrService/OcrEngine resolves to: the bundled
+//     ONNX PP-OCR recognizer if present, else the native Windows OCR
+//     MethodChannel, else NONE. On a platform/CI without any backend, OCR
+//     returns null and a rasterized PDF simply gets no `pageText` — no crash,
+//     and the embedded-text path still works.
+//   * pdfrx render + native OCR can only be exercised on-device. The pdfrx-
+//     backed loaders are injected through [PdfTextIndexer] so unit tests fake
+//     them; the production wiring lives in [PdfrxPageTextSource].
+
+import 'dart:async';
+import 'dart:io';
+
+import '../editor/persistence/sidecar_repository.dart';
+import '../storage/badnote_sidecar.dart';
+import '../storage/sidecar_store.dart';
+
+/// Extracts a PDF's embedded text layer, one entry per page (page order). An
+/// empty list (or all-empty entries) means "no usable text layer".
+typedef PdfEmbeddedTextLoader = Future<List<String>> Function(String pdfPath);
+
+/// OCRs the rendered pages of a rasterized PDF, returning one entry per page
+/// (page order). Entries may be empty where a page yielded nothing. Returns an
+/// empty list when no OCR backend is available (a clean no-op).
+typedef PdfPageOcrRunner = Future<List<String>> Function(String pdfPath);
+
+/// Indexes a PDF's document body into its sidecar's `pageText` at import time.
+///
+/// Stateless apart from the two injected text sources; safe to construct per
+/// import. All disk/native work is awaited internally — call [indexPdf] without
+/// awaiting (fire-and-forget) from the import handler to keep import snappy.
+class PdfTextIndexer {
+  PdfTextIndexer({
+    required PdfEmbeddedTextLoader loadEmbeddedText,
+    required PdfPageOcrRunner ocrPages,
+    this.textLayerThreshold = 16,
+  })  : _loadEmbeddedText = loadEmbeddedText,
+        _ocrPages = ocrPages;
+
+  final PdfEmbeddedTextLoader _loadEmbeddedText;
+  final PdfPageOcrRunner _ocrPages;
+
+  /// Minimum total embedded-text length (across all pages, after trimming) for a
+  /// PDF to count as having a usable text layer. Below this it is treated as
+  /// rasterized and routed to OCR. Small on purpose: a scanned PDF typically
+  /// yields zero or a few stray ligature chars, while any real text page clears
+  /// it easily.
+  final int textLayerThreshold;
+
+  /// The page separator stored inside `pageText` (form feed). The search index
+  /// treats `pageText` as a flat blob, so this is purely cosmetic / future-proof.
+  static const String pageSeparator = '\f';
+
+  /// Index the PDF at [pdfPath] (an in-vault copy) into its sidecar's `pageText`.
+  ///
+  /// Idempotent: returns immediately if the sidecar already has non-empty
+  /// `pageText`. Never throws — any failure (unreadable PDF, missing OCR backend)
+  /// degrades to leaving `pageText` unset. Returns the text it persisted (for
+  /// tests), or null when nothing was indexed.
+  Future<String?> indexPdf(String pdfPath) async {
+    try {
+      // 1. Idempotency: skip a doc whose body has already been indexed.
+      final existing = await _currentPageText(pdfPath);
+      if (existing != null && existing.trim().isNotEmpty) return null;
+
+      // 2. Embedded text layer.
+      final embedded = await _loadEmbeddedText(pdfPath);
+      final embeddedLen = embedded.fold<int>(
+        0,
+        (sum, page) => sum + page.trim().length,
+      );
+
+      // 3. Text-layer vs rasterized decision.
+      List<String> pages;
+      if (embeddedLen >= textLayerThreshold) {
+        pages = embedded;
+      } else {
+        // Rasterized (scanned, no text layer) → background OCR.
+        pages = await _ocrPages(pdfPath);
+      }
+
+      final joined = _joinPages(pages);
+      if (joined.isEmpty) return null;
+
+      // 4. Persist into the sidecar (through an open repo if the editor holds it).
+      await _persistPageText(pdfPath, joined);
+      return joined;
+    } catch (_) {
+      // Background indexing must never surface an error to the import flow.
+      return null;
+    }
+  }
+
+  /// Whether the embedded text in [embedded] clears [textLayerThreshold], i.e.
+  /// the PDF has a usable text layer (false → rasterized, needs OCR). Exposed for
+  /// unit-testing the decision in isolation.
+  bool hasUsableTextLayer(List<String> embedded) {
+    final len = embedded.fold<int>(0, (sum, p) => sum + p.trim().length);
+    return len >= textLayerThreshold;
+  }
+
+  static String _joinPages(List<String> pages) {
+    final nonEmpty = pages.map((p) => p.trim()).where((p) => p.isNotEmpty);
+    return nonEmpty.join(pageSeparator).trim();
+  }
+
+  /// Read the sidecar's current `pageText` (for the idempotency check), from the
+  /// open repo if present else from disk. Null when no sidecar exists yet.
+  Future<String?> _currentPageText(String pdfPath) async {
+    final open = SidecarRepositoryRegistry.forPath(pdfPath);
+    if (open != null) return open.loadedPageText;
+    final sidecar = await SidecarStore.read(
+      File('$pdfPath$kSidecarSuffix'),
+    );
+    return sidecar?.pageText;
+  }
+
+  /// Write [pageText] into the sidecar. Prefer the editor's already-open repo
+  /// (same in-memory sidecar — no race); otherwise merge into the on-disk
+  /// sidecar (creating one if the editor hasn't yet).
+  Future<void> _persistPageText(String pdfPath, String pageText) async {
+    final open = SidecarRepositoryRegistry.forPath(pdfPath);
+    if (open != null) {
+      open.schedulePageTextSave(pageText);
+      await open.flush();
+      return;
+    }
+
+    final file = File('$pdfPath$kSidecarSuffix');
+    final current = await SidecarStore.read(file);
+    final merged = _withPageText(current, pdfPath, pageText);
+    await SidecarStore.writeAtomic(file, merged);
+  }
+
+  /// Build a sidecar carrying [pageText], preserving every other field of
+  /// [current] (or a fresh minimal sidecar when none exists yet).
+  static BadnoteSidecar _withPageText(
+    BadnoteSidecar? current,
+    String pdfPath,
+    String pageText,
+  ) {
+    if (current == null) {
+      return BadnoteSidecar(
+        sourceFile: _basename(pdfPath),
+        docType: 'pdf',
+        createdAt: DateTime.now().toUtc(),
+        updatedAt: DateTime.now().toUtc(),
+        pageText: pageText,
+      );
+    }
+    return BadnoteSidecar(
+      version: current.version,
+      sourceFile: current.sourceFile,
+      docType: current.docType,
+      title: current.title,
+      pageCount: current.pageCount,
+      rotation: current.rotation,
+      createdAt: current.createdAt,
+      updatedAt: DateTime.now().toUtc(),
+      strokes: current.strokes,
+      highlights: current.highlights,
+      texts: current.texts,
+      bookmarks: current.bookmarks,
+      scratchLinks: current.scratchLinks,
+      legacyAnnotations: current.legacyAnnotations,
+      ocrText: current.ocrText,
+      pageText: pageText,
+      legacyId: current.legacyId,
+      background: current.background,
+    );
+  }
+
+  static String _basename(String path) {
+    final norm = path.replaceAll('\\', '/');
+    final i = norm.lastIndexOf('/');
+    return i == -1 ? norm : norm.substring(i + 1);
+  }
+}
--- a/lib/services/pdfrx_page_text_source.dart
+++ b/lib/services/pdfrx_page_text_source.dart
@@ -0,0 +1,111 @@
+// lib/services/pdfrx_page_text_source.dart
+//
+// Production wiring for [PdfTextIndexer]'s two injected text sources, backed by
+// pdfrx (the same engine the editor renders with). Kept SEPARATE from
+// PdfTextIndexer so the indexer's logic (the threshold decision, idempotency,
+// sidecar persistence) is unit-testable without the native pdfium/OCR stack —
+// only this file touches pdfrx, dart:ui, and the OCR engine, and it is exercised
+// on-device, not in CI.
+
+import 'dart:async';
+import 'dart:typed_data';
+import 'dart:ui' as ui;
+
+import 'package:pdfrx/pdfrx.dart';
+
+import 'ocr_engine.dart';
+
+/// pdfrx-backed loaders for [PdfTextIndexer].
+class PdfrxPageTextSource {
+  const PdfrxPageTextSource._();
+
+  /// Load the embedded text layer of every page (page order). Each entry is a
+  /// page's raw text (possibly empty). Returns an empty list on any failure, so
+  /// the indexer treats the PDF as having no text layer (→ OCR fallback).
+  static Future<List<String>> loadEmbeddedText(String pdfPath) async {
+    PdfDocument? doc;
+    try {
+      doc = await PdfDocument.openFile(pdfPath);
+      final out = <String>[];
+      for (final page in doc.pages) {
+        final raw = await page.loadText();
+        out.add(raw?.fullText ?? '');
+      }
+      return out;
+    } catch (_) {
+      return const [];
+    } finally {
+      await doc?.dispose();
+    }
+  }
+
+  /// Render each page and OCR it (page order). Returns one entry per page
+  /// (empty where nothing was recognized), or an empty list when the PDF can't
+  /// be opened. Honours the OCR engine's own graceful no-op: when no backend is
+  /// available every page comes back empty.
+  ///
+  /// Rendering is done at [renderScale]× the page's native 72-dpi size to give
+  /// the recognizer enough resolution on scanned scans without exploding memory.
+  static Future<List<String>> ocrPages(
+    String pdfPath, {
+    double renderScale = 2.0,
+  }) async {
+    PdfDocument? doc;
+    try {
+      doc = await PdfDocument.openFile(pdfPath);
+      final out = <String>[];
+      for (final page in doc.pages) {
+        final text = await _ocrOnePage(page, renderScale);
+        out.add(text ?? '');
+      }
+      return out;
+    } catch (_) {
+      return const [];
+    } finally {
+      await doc?.dispose();
+    }
+  }
+
+  static Future<String?> _ocrOnePage(PdfPage page, double renderScale) async {
+    PdfImage? image;
+    try {
+      final fullWidth = page.width * renderScale;
+      final fullHeight = page.height * renderScale;
+      image = await page.render(
+        fullWidth: fullWidth,
+        fullHeight: fullHeight,
+      );
+      if (image == null) return null;
+      final png = await _bgraToPng(image.pixels, image.width, image.height);
+      if (png == null) return null;
+      return OcrEngine.recognizeImage(png);
+    } catch (_) {
+      return null;
+    } finally {
+      image?.dispose();
+    }
+  }
+
+  /// Encode pdfrx's BGRA8888 raw pixels as PNG (the format [OcrEngine] expects).
+  static Future<Uint8List?> _bgraToPng(
+    Uint8List bgra,
+    int width,
+    int height,
+  ) async {
+    final completer = Completer<ui.Image>();
+    ui.decodeImageFromPixels(
+      bgra,
+      width,
+      height,
+      ui.PixelFormat.bgra8888,
+      completer.complete,
+    );
+    final image = await completer.future;
+    try {
+      final data = await image.toByteData(format: ui.ImageByteFormat.png);
+      return data?.buffer.asUint8List();
+    } finally {
+      image.dispose();
+    }
+  }
+}
--- a/lib/services/vault_search_index.dart
+++ b/lib/services/vault_search_index.dart
@@ -10,9 +10,12 @@
 //                              pages,
 //   * the handwriting OCR text persisted in the sidecar's `ocrText` field.
 //
-// It does NOT (yet) index a PDF's embedded text layer — that is a known gap (see
-// the REPORT in the task / the class doc below). Matching uses the existing pure
-// search primitives (normalize / rank / snippet), so CJK substring search works.
+// It ALSO indexes a file-backed PDF's document body text, captured once at
+// import into the sidecar's `pageText` field by [PdfTextIndexer]: the embedded
+// (printed) text layer for a normal PDF, or background OCR of the rendered pages
+// for a RASTERIZED / scanned PDF that has no text layer. Matching uses the
+// existing pure search primitives (normalize / rank / snippet), so CJK substring
+// search works.

 import 'dart:io';

@@ -71,13 +74,14 @@ class VaultSearchHit {
 /// source of truth).
 ///
 /// HONEST SCOPE (what search covers / does NOT):
-///   * COVERS: note/doc titles, typed text boxes, and handwriting OCR text that
-///     has been persisted into a sidecar's `ocrText` field.
-///   * DOES NOT cover: a PDF's embedded (printed) text layer — only the user's
-///     annotations are indexed, not the underlying document body. Indexing the
-///     PDF text layer would require rendering each page through pdfrx at scan
-///     time; deferred. OCR is only present where it has already been run and
-///     written back to the sidecar.
+///   * COVERS: note/doc titles, typed text boxes, handwriting OCR text persisted
+///     into a sidecar's `ocrText` field, AND a PDF's document body text persisted
+///     into `pageText` at import — the embedded text layer, or background OCR of
+///     a rasterized/scanned PDF (see [PdfTextIndexer]).
+///   * CAVEAT: `pageText` is only present once import-time indexing has run and
+///     written it back to the sidecar. A PDF imported before this feature (or
+///     whose OCR backend was unavailable) has no `pageText`, so only its
+///     annotations are searchable until it is re-indexed.
 class VaultSearchIndex {
  VaultSearchIndex(this._vault);

@@ -174,6 +178,11 @@ class VaultSearchIndex {
      }
      final ocr = sidecar.ocrText;
      if (ocr != null && ocr.trim().isNotEmpty) parts.add(ocr.trim());
+      // Document body text captured at import: the PDF's embedded text layer,
+      // or background OCR of a rasterized/scanned PDF. Covers the underlying
+      // document, not just the user's annotations.
+      final body = sidecar.pageText;
+      if (body != null && body.trim().isNotEmpty) parts.add(body.trim());
    }
    return parts.join('\n');
  }