Files
BadNote/lib/services/ocr/ctc_decoder.dart
Akiba So 99b98b96b0
Some checks failed
CI / Flutter (analyze, test, Windows build) (push) Failing after 30s
CI / Server tests (optional) (push) Failing after 29s
OCR: embedded, cross-platform ONNX backend with pluggable fallback
Make on-device OCR a pluggable local service so it runs locally on every
platform (not just Windows), aimed at GoodNotes/Notability-class handwriting on
low-power hardware (e.g. Zen2 APU, CPU/iGPU).

- New OcrBackend abstraction (lib/services/ocr/): selector prefers an embedded
  ONNX recognition backend, falling back to the OS-native backend (Windows
  WinRT), and to a clean no-op when neither is available.
- OnnxRecognitionBackend: flutter_onnxruntime session from a bundled asset,
  dart:ui preprocessing (resize to 48px, CHW float32, normalized), pure-Dart CTC
  greedy decode. Fully guarded — absent model/dict is a no-op; never throws.
- ocr_engine.dart kept as a thin facade (recognizeImage) delegating to the
  selector, so ocr_service.dart is unchanged.
- CtcDecoder unit-tested (6 tests). flutter analyze clean; all tests pass.
- Model is not committed; tool/fetch_ocr_model.sh + assets/models/ocr/README.md
  document fetching PP-OCRv4 rec + dict on the dev machine.
- CI: forward HTTPS_PROXY to the Windows build so CMake can fetch the ONNX
  Runtime native lib behind the GFW; README documents the system-install
  alternative. PP-OCR geometry/blank assumptions documented for on-device tuning.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-21 03:51:54 +08:00

93 lines
3.1 KiB
Dart

/// Pure-Dart CTC (Connectionist Temporal Classification) greedy decoder.
///
/// Decodes per-timestep class logits into a string by taking the argmax at
/// each timestep, collapsing consecutive duplicate classes, dropping the
/// blank class, and mapping the remaining class indices to characters.
///
/// Index mapping note (PaddleOCR PP-OCR rec convention with [blankIndex] == 0):
/// the CTC blank occupies class index 0, so the character dictionary is
/// shifted by one. The character for class index `k` (k >= 1) is
/// `charset[k - 1]`. If [blankIndex] != 0, this exact shift may not apply and
/// the mapping should be reviewed for the specific exported model.
class CtcDecoder {
CtcDecoder(this.charset, {this.blankIndex = 0});
/// The character dictionary (without the blank entry).
final List<String> charset;
/// The class index reserved for the CTC blank symbol.
final int blankIndex;
/// Decode `[T][C]` logits into a string.
///
/// For each timestep the argmax over the `C` classes is taken; consecutive
/// duplicate indices are collapsed and the blank index is dropped. Remaining
/// indices are mapped to characters via the dictionary shift described in the
/// class docs. Out-of-range indices are skipped.
String decode(List<List<double>> logits) {
final buffer = StringBuffer();
var previousIndex = -1;
for (final row in logits) {
if (row.isEmpty) {
previousIndex = -1;
continue;
}
// argmax over the classes of this timestep.
var bestIndex = 0;
var bestValue = row[0];
for (var c = 1; c < row.length; c++) {
if (row[c] > bestValue) {
bestValue = row[c];
bestIndex = c;
}
}
// Collapse consecutive duplicates.
if (bestIndex == previousIndex) {
continue;
}
previousIndex = bestIndex;
// Drop the blank class.
if (bestIndex == blankIndex) {
continue;
}
final ch = _charForIndex(bestIndex);
if (ch != null) {
buffer.write(ch);
}
}
return buffer.toString();
}
/// Reshape a flat row-major `[T*C]` list into `[T][C]` and decode it.
String decodeFlat(List<double> flat, int timeSteps, int numClasses) {
if (timeSteps <= 0 || numClasses <= 0) return '';
final logits = <List<double>>[];
for (var t = 0; t < timeSteps; t++) {
final start = t * numClasses;
final end = start + numClasses;
if (end > flat.length) break;
logits.add(flat.sublist(start, end));
}
return decode(logits);
}
/// Map a class index to its character, applying the blank shift. Returns null
/// for the blank index or out-of-range indices.
String? _charForIndex(int index) {
if (index == blankIndex) return null;
// With blankIndex == 0 the dictionary is shifted by one: class index k
// maps to charset[k - 1]. For other blank positions we fall back to a
// direct index, which may need adjustment per the exported model.
final mapped = blankIndex == 0 ? index - 1 : index;
if (mapped < 0 || mapped >= charset.length) return null;
return charset[mapped];
}
}