# 3-Tier OCR输油管
2026-01-07T18:00
[影像情报部分](/blog/constrained-fuzzy-image-intelligence) 此文章在图像合成器波结构中引入了 imageSummarizer Wave 架构和范围更广的图案@ . @ this art deep *-}%dives into the **OCR 子系统**“—”三层文字提取@,}智能路由 @, 和胶片优化,
**为何另篇文章?** OCR输油管从“"”Tesseract with Vision LLM fallback " 演变成一个复杂的三Q-梯级系统,由基于 ML-的 OCR,}Openç-Frame投票=, text_- 仅限条状提取@,和成本$-Awarre routing @.}It{'}它很复杂足以证明自己有详细的解货'.〞
**相关条款**:
- [缩放模糊模式](/blog/constrained-fuzziness-pattern) *-* 基础模式
- [缩放的模糊分子MOM( MM)](/blog/constrained-mom-mixture-of-models) - 多模式协调
- [内幕拖拉](/blog/constrained-fuzzy-context-dragging) -记忆和时间
- [影像情报部分](/blog/constrained-fuzzy-image-intelligence) @-_# 全波结构概览
- [DocSummamer 缩写器](/blog/building-a-document-summarizer-with-rag) - 具有类似模式的文件分析
- [数据合成器](/blog/datasummarizer-how-it-works) - 确定论数据剖析=-}第一种方法
[TOC]
---
## 问题=:}文字解析是困难的
真实的OCR @-#world 图像在可预测的方式上失败@:+%
- **立体字体**: 有关标准字体的实验性培训 *\ MPK1 在装饰文字上失败
- **吵闹的GIFs**: 框架压缩文物@, jitter}, 字幕更改
- **低对比度**@: 暗背景的黑暗文字
- **旋转的文本**: non- 横向文本角度
- **混合内容**: 带多个文本区域的截图
- **APPI费用**@: 视觉LLM电话费用昂贵 每张图像$($0.001-0.01}#)
如果传统方法不使用 Vision LLMQ", 则其传统方法@: @ @ @ "_Run Tesseract @,}
**问题**@ :_ 这要么错过了拼写文本@ MS K1_ Tesseract 失败} 要么花费太高 : @ MPK3Qalways use Vision LLM=.}
**解决方案**: 添加一个中层 @(Florence}-2ONNX) 处理本地Styl化字体的“, 升至视觉LM ” 只有当两种当地方法都失败时, “.
---
## 3-Tier OCR建筑
系统以优先顺序运行波浪, @ ( @ higher number @ MPK1_ 稍后执行@ MS K2#
```
Wave Priority Order:
40: TextLikelinessWave → Heuristic text detection
50: OcrWave → Tesseract OCR (if text-likely)
51: MlOcrWave → Florence-2 ML OCR (if Tesseract low confidence)
55: Florence2Wave → Florence-2 captions (optional)
80: VisionLlmWave → Vision LLM (escalation)
```
### 1: ( 传统 OCRMS K2
-=YTET -伊甸园字幕组=- 翻译:
|----------|-------|-------|----------|------------|
| **50** |{~50}|Free|干净的文字,高对比度MS K5标准字@|Stylized字体MSQK7低品质MSKO8旋轉的文字□|
**发送信号@: @%**
- `ocr.text` -摘录文本
- `ocr.confidence` - 宇宙魔方表示信心得分
### -=YTET -伊甸园字幕组=- 翻译:
-=YTET -伊甸园字幕组=- 翻译:
|----------|-------|-------|----------|------------|
| **51** |{~200}| 免费MS K3 标准字体 , memes, 装饰性文字|复杂图表~, 旋转文本□|
**发送信号@: @%**
- `ocr.ml.text` -=YTET -伊甸园字幕组=- 翻译:
- `ocr.ml.multiframe_text` - 多-Frame GIF 文字@(Precious 用于动画 @)}
- `ocr.ml.confidence` -模擬信任分數
### 3:VisionLLM(Cloud 秋季=)
-=YTET -伊甸园字幕组=- 翻译:
|----------|-------|-------|----------|------------|
| **80** 尤其复杂的场景必须尊重确定性信号 @|#
**发送信号@: @%**
- `ocr.vision.text` -Vision LLM OCR 文本提取
- `ocr.vision.confidence` *-* LLM 信任 *(* 一般来说是0.95)
- `caption.text` @-_objective 描述性字幕@(}和 OCR)Q分开
---
## ONNX Arsenal :当地ML模型
在潜入 OCR 三层之前 **确定性ML模型** 所有模型都通过 ONNX Runtime@— @no API 调用\ ,}没有云依赖性 *,}无成本=.}
### 为什么是ONX ?
- **本地运行**无 API 键@, @无网络长期性 @,}没有经常性费用
- **确定性**:}相同的输入@=}同样的输出 @(#no 采样 */* 温度随机性=)*
- **快速快**:#硬件 @-#加速@(_CPU}/$GPU),}优化推论
- **便携式**“:”在 Windows @,}Linux, macOS上的工作
- **自动下載@ - @ {}**@: 第一次运行自动下载模型
\* *微小警告:GPU执行提供商可以引入可忽略的浮动MS K1点非确定性 .信号合同 @(}信任阈值{, 路由逻辑})仍然完全确定性}.#*
### 五种ONNX模型
> **注注注注注释注注说明附注注注,注注 注注的注 注 注 注注说明注注附注注 注的注注 注注说注注的附注注说明,注 注,注说明的注说明说明注 注说明注说明 注注 注附注注的注释注 注 注 注注释注说明注释注的说明注,说明注的 注 注说明说明说明 注 注的说明说明,说明说明的附注 注注附注说明注附注的注附注 注 注注释说明注注释的注注释 注注注释注释注,的注, 注注,注释注附注附注注注释说明的说明 注说明 注的注释 注 注附注说明说明注释说明说明附注 注说明的注释注释注释说明 注注释注释 注说明注释注释的说明注释 注注释 注的附注注释注注释,注注释附注注附注注释的 注注脚注注注**: 大小是近似值,并因变体而不同 : @/Qquantization@. 典型下载尺寸如下*.
|模范 @|Approx.大小 @MS K3目的 @|速度@|型样_|
|-------|--------------|---------|-------|------------|
| **东部** @| ~100MB| 场景文字探测 {|} MS K4ms *|文本检测 ~|#
| **花生鼠** @| ~150MB| 字符-区域文本检测 *#| NSK5ms\MS K6文本检测 <_|*
| **佛罗伦萨-2** @| ~250MB| OCRMS K3字幕| NSK5ms MS K6Vision-语言MSKO8
| **真实的#-ESRGAN** @| @ @ ~60MB @ I| @ @ *4×# 超级-# 分辨率提升@|} @ MPK6 @ ms @ MASSK7_ 图像增强@MS K8#
| **CLIP** |+~100QAMs @|+多模式嵌入@|#
**磁盘空间总数**取决于所选择的模型变体
---
### -=YTET -伊甸园字幕组=- 翻译:
**高效和准确的现场文本检测器** @-在自然场景中发现文字区域@.
```csharp
// EAST detects text bounding boxes with confidence scores
var result = await textDetector.RunEastDetectionAsync(imagePath);
// Output: List of BoundingBox with coordinates + confidence
// Example: [BoundingBox(x1:50, y1:100, x2:300, y2:150, confidence:0.92)]
```
**如何运作**:
- 就现场文本数据集培训的深深学习模式
- 输出得分映射 * @(_Acustify_)}+_几何地图*#(_Box座标=)}
- 处理旋转的文本@ , moux @ MS K1_ scale text
- 使用非{-}*马希姆禁止 *(}NAMS)}以合并重叠框
**为何有决定权?**
- 在推断中没有随机性 {(}fn微软雅黑frozen weights} )
- 相同的图像@→ @% 相同捆绑框
- 信任得分是可以复制的
- 升降阈值为 config-}驱动的 @ (eMS K2g{.,} `< 0.5 → escalate`)
**技术细节**:
```csharp
// EAST preprocessing (from implementation)
- Input size: 320×320 (must be multiple of 32)
- Format: BGR with mean subtraction [123.68, 116.78, 103.94]
- Output stride: 4 (downsampled 4×)
- Score threshold: 0.5
- NMS IoU threshold: 0.4
```
**实例产出**:
```
Input: meme.png (800×600)
EAST detection: 15 text regions found
Region 1: (50, 480, 750, 580) - confidence 0.87 [bottom subtitle area]
Region 2: (100, 50, 300, 90) - confidence 0.62 [top text]
Region 3: ...
Route decision: ANIMATED (subtitle pattern in bottom 30%)
```
---
### 2.CRAFT:字符区域认识
**字符QM- 级别文本检测** “-”优异的艺术作品“,”和Stylized text@.
```csharp
// CRAFT finds character-level regions, then groups into words
var result = await textDetector.RunCraftDetectionAsync(imagePath);
// Better than EAST for: decorative fonts, curved text, logos
```
**如何运作**:
- 检测单个字符区域 @(}(比原颗粒更多)
- 将字符组合成单词
- Flood- 填充算法找到连接的文本组件
- 处理东帝汶错过的曲线文字
**使用CPFT时**:
1. 东帝汶没有或失败
2. 图像中包含艺术字体@/ @ 装饰字体 @ (auto @ MPK2_ 检测@ MS K3}
3. 用户明确选择 CRAFT 检测器
**技术细节**:
```csharp
// CRAFT preprocessing
- Max dimension: 1280px (maintains aspect ratio)
- Format: RGB normalized with ImageNet stats
- Mean: [0.485, 0.456, 0.406]
- Std: [0.229, 0.224, 0.225]
- Output stride: 2 (downsampled 2×)
- Threshold: 0.4 for character regions
```
**EAST与CRAFT比较**:
【|】 特写 @|}东区 @MS K2#CRAFT *|*
|---------|------|-------|
-=YTET -伊甸园字幕组=- 翻译:
@|}速度: | @ @ MS K2 @ ms @ I| @ @ *~30#% 速率:
@|_Best for {|}标准文本=,}字幕:|} 装饰性字体@,}徽标=#|}
|Curved text *|有限公司*|精良 *|
@| 型号: | MS K2MB _| NSK4 MB |#
---
### -=YTET -伊甸园字幕组=- 翻译:
**在 OCR 前增强低-}质量图像** @ - @ @ I4× @ 升格为模糊@ MS K2 @ 小文本 @ MPK3}
```csharp
// Upscale low-quality image before running OCR
if (quality.Sharpness < 30) // Laplacian variance threshold
{
var upscaled = await esrganService.UpscaleAsync(imagePath, scale: 4);
// Now run OCR on the enhanced image
}
```
**当它使用 ' @%s**:
- 图像清晰度 @< @ @ 30} @ I( @ Laplacian different *)#
- 检测到文本区域,但非常小
- OCR信心低,但有文本区域
- 用户明确要求升级
**示例实例**:
```
Input: 100×75 screenshot with tiny text
Laplacian variance: 18 (very blurry)
ESRGAN: Upscale to 400×300 (~500ms)
New Laplacian variance: 87 (sharp)
OCR: Tesseract confidence: 0.92 (vs 0.42 before upscaling)
Text: "Click here to continue" (vs garbled before)
```
**技术细节**:
```csharp
// Real-ESRGAN processing
- Input: Any size (processed in 128×128 tiles if large)
- Output: 4× scaled (200×150 → 800×600)
- Model: x4plus variant (general photos)
- Processing: ~500ms for 800×600 image
- Memory: ~2GB peak (tiles reduce this)
```
**现代经济学**:
```
Scenario: Screenshot with tiny text
Option 1: Send low-res to Vision LLM
Image: 100×75 = ~20 tokens
LLM can't read tiny text → fails
Cost: $0.0002 (wasted)
Option 2: Upscale with ESRGAN, use Tesseract
ESRGAN: Free (local), 500ms
Tesseract: Free (local), 50ms
Success: 92% confidence
Cost: $0
Result: ESRGAN + local OCR beats Vision LLM for low-res images
```
---
### -=YTET -伊甸园字幕组=- 翻译:
**用于语义图像搜索的多式嵌入** - 将图像和文字投放到共享的矢量空间@.
```csharp
// Generate embedding for semantic search
var embedding = await clipService.GenerateEmbeddingAsync(imagePath);
// Returns: float[512] vector
// Later: semantic search across thousands of images
var similarImages = await vectorDb.SearchAsync(queryEmbedding, topK: 10);
```
**如何运作**:
- CLIP VIT -B/32 视觉编码器
- 工程图像到 @ 512-维矢量的图像
- 培训,以与文本说明保持一致
- 启用没有关键字的“% "” 定位图像
**使用案例**:
- RAG 系统中的语义图像搜索
- 重复检测@( @ 即便编辑@ /_ ropped)}
- 基于内容的群集@- @% 1
- 类似形象建议
**技术细节**:
```csharp
// CLIP visual encoder
- Model: ViT-B/32 (Vision Transformer)
- Input: 224×224 RGB (center crop + resize)
- Output: 512-dimensional embedding
- Normalized: L2 norm = 1.0
- Speed: ~100ms per image
```
**示例实例**:
```
Input images:
cat_on_couch.jpg → [0.23, -0.51, 0.88, ...]
dog_on_couch.jpg → [0.19, -0.48, 0.91, ...]
car_photo.jpg → [-0.67, 0.33, -0.12, ...]
Query: "animals on furniture"
Text embedding → [0.21, -0.50, 0.89, ...]
Cosine similarity:
cat_on_couch: 0.94 (very similar!)
dog_on_couch: 0.91 (similar)
car_photo: 0.12 (not similar)
Result: Returns cat and dog images
```
---
### @(_Country@2)_BARBAR_Vision_-_Language 模型 -2:_CAR_在Tier@MSK4#中被覆盖
有关佛罗伦萨--2ONNX OCR和字幕的详情,请参看“.”部分。
---
### 自动-下载系统
首先使用时自动下载所有模型@: @%
```bash
$ imagesummarizer image.png --pipeline auto
[First run]
Downloading EAST scene text detector (~100MB)...
Progress: ████████████████████ 100% (102.4 MB)
Downloading Florence-2 base model (~250MB)...
Progress: ████████████████████ 100% (248.7 MB)
Downloading CLIP ViT-B/32 visual (~350MB)...
Progress: ████████████████████ 100% (347.2 MB)
Models saved to: ~/.mostlylucid/models/
Total disk space: 1.16 GB
[Subsequent runs]
All models cached, analysis starts immediately
```
**优减降解**:
```csharp
// If ONNX model download fails, system falls back gracefully
EAST unavailable → Try CRAFT → Fall back to Tesseract PSM
Real-ESRGAN unavailable → Skip upscaling, use original image
CLIP unavailable → Skip embeddings, OCR still works
Florence-2 unavailable → Use Tesseract → Vision LLM escalation
```
每个 ONNX 模型故障都记录为备份路径@,}确保系统不会因为缺少模式而崩溃 @.#
---
### 为何如此重要
> **定价注释**以下成本示例使用说明性定价@(~$0.005/ @image for Vision LLM).}实际API费用因提供者和型号而异 .}核心洞察力=—}本地处理消除了大部分API呼叫的
**没有ONNX模型** (+Baseline+):+
```
Every image → Send to Vision LLM
Cost: ~$0.005/image (example pricing)
Time: ~2s network + inference
100 images = ~$0.50, ~200s
```
**使用ONNX模型** 先是): -=YTET 翻译:
```
85 images → EAST + Florence-2 (local)
Cost: $0
Time: ~200ms
10 images → EAST + Tesseract (local)
Cost: $0
Time: ~50ms
5 images → EAST + Vision LLM (escalation)
Cost: ~$0.025 (5 × $0.005)
Time: ~2s each
100 images = ~$0.025, ~30s total
```
**节节余**@: @ @ ~95% @ 降低成本 @ I, @ @ *~85%# 更快@ MS K4} @ **确定性路线**.
ONNX模型将系统从“"”概率率一路转换到 “"”和“"”的确定性基础,
---
## 宇宙魔方 OCR
基线@. fast,确定性MS K2对干净文字效果很好 @.
```csharp
public class OcrWave : IAnalysisWave
{
public string Name => "OcrWave";
public int Priority => 60; // After color/identity
public async Task> AnalyzeAsync(
string imagePath,
AnalysisContext context,
CancellationToken ct)
{
var signals = new List();
// Get preprocessed image from cache
var image = context.GetCached>("image");
// Run Tesseract OCR
using var engine = new TesseractEngine(@"./tessdata", "eng", EngineMode.Default);
using var page = engine.Process(image);
var text = page.GetText();
var confidence = page.GetMeanConfidence();
signals.Add(new Signal
{
Key = "ocr.text", // Tesseract OCR result
Value = text,
Confidence = confidence,
Source = Name,
Tags = new List { "ocr", "text" },
Metadata = new Dictionary
{
["engine"] = "tesseract",
["mean_confidence"] = confidence,
["word_count"] = text.Split(' ').Length
}
});
signals.Add(new Signal
{
Key = "ocr.confidence",
Value = confidence,
Confidence = 1.0,
Source = Name
});
return signals;
}
}
```
**关键信号**:
- `ocr.full_text` “- 摘录文本”
- `ocr.early_exit` - 信号跳过 Tier 2/3, 如果信任度高的话
- 信任得分驱动升级决定
---
## M2:佛罗伦萨-2ONNX
微软's Florence @-2 是一个视觉=-}语言模型,非常精通密集字幕和OCR. ONNX 版本本地运行,无API$@.}
### 为什么佛罗伦萨? -2?
- **比星际字体的宇宙魔方好**@ :_ 手持装饰文字的文本@ MS K1_ memes,_ 徽标
- **比视野LLM快**@: @ @ ~200# ms 与@1-5 @%s
- **自由**: 运行本地端@,}不需要 API 密钥
- **多式联运理解**@: 能够在上下文中提取文字@(e.gMS K3 语音泡泡+)
### 执行 执行情况
```csharp
public class MlOcrWave : IAnalysisWave
{
private readonly Florence2OnnxModel _model;
public string Name => "MlOcrWave";
public int Priority => 51; // Runs AFTER Tesseract (priority 50)
public async Task> AnalyzeAsync(
string imagePath,
AnalysisContext context,
CancellationToken ct)
{
var signals = new List();
// Check if Tesseract already succeeded with high confidence
var tesseractConfidence = context.GetValue("ocr.confidence");
if (tesseractConfidence >= 0.95)
{
signals.Add(new Signal
{
Key = "ocr.ml.skipped", // Consistent namespace: ocr.ml.*
Value = true,
Confidence = 1.0,
Source = Name,
Metadata = new Dictionary
{
["reason"] = "tesseract_high_confidence",
["tesseract_confidence"] = tesseractConfidence
}
});
return signals;
}
// Run Florence-2 OCR
var result = await _model.ExtractTextAsync(imagePath, ct);
signals.Add(new Signal
{
Key = "ocr.ml.text", // Florence-2 ML OCR text
Value = result.Text,
Confidence = result.Confidence,
Source = Name,
Tags = new List { "ocr", "text", "ml" },
Metadata = new Dictionary
{
["model"] = "florence2-base",
["inference_time_ms"] = result.InferenceTime,
["token_count"] = result.TokenCount
}
});
// For animated GIFs, extract all unique frames
if (context.GetValue("identity.frame_count") > 1)
{
var frameResults = await ExtractMultiFrameTextAsync(
imagePath,
maxFrames: 10,
ct);
signals.Add(new Signal
{
Key = "ocr.ml.multiframe_text",
Value = frameResults.CombinedText,
Confidence = frameResults.AverageConfidence,
Source = Name,
Metadata = new Dictionary
{
["frames_processed"] = frameResults.FrameCount,
["unique_text_segments"] = frameResults.UniqueSegments,
["deduplication_method"] = "levenshtein_85"
}
});
}
return signals;
}
}
```
### MOUM-Frame GIF 处理
用于动态 GIFs @,}佛罗伦萨•-2}进程,直到在平行的 : 中“10 ” 取样框
```csharp
private async Task ExtractMultiFrameTextAsync(
string imagePath,
int maxFrames,
CancellationToken ct)
{
// Load GIF and extract frames
using var image = await Image.LoadAsync(imagePath, ct);
var frames = new List>();
int frameCount = image.Frames.Count;
int step = Math.Max(1, frameCount / maxFrames);
for (int i = 0; i < frameCount; i += step)
{
frames.Add(image.Frames.CloneFrame(i));
}
// Process all frames in parallel (bounded concurrency to avoid thrashing)
var semaphore = new SemaphoreSlim(4); // Max 4 concurrent inferences
var tasks = frames.Select(async frame =>
{
await semaphore.WaitAsync(ct);
try
{
var result = await _model.ExtractTextAsync(frame, ct);
return result;
}
finally
{
semaphore.Release();
}
});
var results = await Task.WhenAll(tasks);
semaphore.Dispose();
// Deduplicate using Levenshtein distance
var uniqueTexts = DeduplicateByLevenshtein(
results.Select(r => r.Text).ToList(),
threshold: 0.85);
return new MultiFrameResult
{
CombinedText = string.Join("\n", uniqueTexts),
FrameCount = frames.Count,
UniqueSegments = uniqueTexts.Count,
AverageConfidence = results.Average(r => r.Confidence)
};
}
private List DeduplicateByLevenshtein(
List texts,
double threshold)
{
var unique = new List();
foreach (var text in texts)
{
bool isDuplicate = false;
foreach (var existing in unique)
{
var distance = LevenshteinDistance(text, existing);
var maxLen = Math.Max(text.Length, existing.Length);
var similarity = 1.0 - (distance / (double)maxLen);
if (similarity >= threshold)
{
isDuplicate = true;
break;
}
}
if (!isDuplicate)
{
unique.Add(text);
}
}
return unique;
}
```
**示例实例**:{93-_Frame GIF →}10 樣本框 → MPK5 獨特的文字結果
```
Frame 1-45: "I'm not even mad."
Frame 46-93: "That's amazing."
```
---
## B. 分流决定
OpenCV 文本检测@(~5-20 @ms @)}决定选择哪条路径@ :
```csharp
public class TextDetectionService
{
public TextDetectionResult DetectText(Image image)
{
// Use OpenCV EAST text detector
var (regions, confidence) = RunEastDetector(image);
return new TextDetectionResult
{
HasText = regions.Count > 0,
RegionCount = regions.Count,
Confidence = confidence,
Route = SelectRoute(regions, confidence, image)
};
}
private ProcessingRoute SelectRoute(
List regions,
double confidence,
Image image)
{
// No text detected
if (regions.Count == 0)
return ProcessingRoute.NoOcr;
// Animated GIF with subtitle pattern
if (image.Frames.Count > 1 && HasSubtitlePattern(regions))
return ProcessingRoute.AnimatedFilmstrip;
// High confidence, standard text
if (confidence >= 0.8 && HasStandardTextCharacteristics(regions))
return ProcessingRoute.Fast; // Florence-2 only
// Moderate confidence
if (confidence >= 0.5)
return ProcessingRoute.Balanced; // Florence-2 + Tesseract voting
// Low confidence, complex image
return ProcessingRoute.Quality; // Full pipeline + Vision LLM
}
private bool HasSubtitlePattern(List regions)
{
// Subtitles are typically in bottom 30% of frame
var bottomRegions = regions.Where(r =>
r.BoundingBox.Y > r.ImageHeight * 0.7);
return bottomRegions.Count() >= regions.Count * 0.5;
}
}
```
### 路线绩效
“|”路线“|”触发器,当 | 处理“MS K3”时(时间):“|”成本“|”
|-------|---------------|------------|------|------|
| **飞快** 高置信度 @(>0.8),标准文本@|Florence{-2}只|Q~100ms @MS K6#free_|*
| **平衡** @|# 中度信心 @MS K1 | @Florence{-2} @+ @Tesseract 投票 @|#~300 @ms @MSC7_Free□| @
| **平等** @| @ 低可信度 @ (<0.5), @ 复合( MSSK2 @ 多式) @ I-_ 框架@ MS K4# 视觉LLM @ *|# # ~1-5# @ s|# @ @ _$0.001-0.01} @ MPK9#
| **度量** @|_GIF 字幕模式@|_Text_-}仅限胶片*|}{~2-3|}{$0.002-0.005} *#MSKK7}
---
## 文本@- @ 仅条形采掘
GIF 字幕“:”的突破优化摘要 **仅文本区域**-=YTET -伊甸园字幕组=- 翻译:
### 问题
“93-_Frame GIF”的传统方法,配有字幕@:}
```
Option 1: Process every frame
93 frames × 300×185 × ~150 tokens/frame = 13,950 tokens
Cost: ~$0.14 @ $0.01/1K tokens
Time: ~27 seconds
Option 2: Sample 10 frames
10 frames × 300×185 × ~150 tokens/frame = 1,500 tokens
Cost: ~$0.015
Time: ~3 seconds
Problem: Might miss subtitle changes
```
### 解答@ : @ text-}唯一的条形
仅解压缩文本约束框@,删除背景像素 @:
```
2 text regions × 250×50 × ~25 tokens/region = 50 tokens
Cost: ~$0.0005
Time: ~2 seconds
Token reduction: 30×
```
### 执行 执行情况
```csharp
public class FilmstripService
{
public async Task CreateTextOnlyStripAsync(
string imagePath,
CancellationToken ct)
{
using var gif = await Image.LoadAsync(imagePath, ct);
// 1. Detect subtitle region (bottom 30% of frames)
var subtitleRegion = DetectSubtitleRegion(gif);
// 2. Extract frames with text changes
var uniqueFrames = ExtractUniqueTextFrames(gif, subtitleRegion);
// 3. Extract tight bounding boxes around text
var textRegions = ExtractTextBoundingBoxes(uniqueFrames);
// 4. Create horizontal strip of text-only regions
var strip = CreateHorizontalStrip(textRegions);
return new TextOnlyStrip
{
Image = strip,
RegionCount = textRegions.Count,
TotalTokens = EstimateTokens(strip),
OriginalTokens = EstimateTokens(gif),
Reduction = CalculateReduction(strip, gif)
};
}
private Rectangle DetectSubtitleRegion(Image gif)
{
// Analyze bottom 30% of frame for text patterns
int subtitleHeight = (int)(gif.Height * 0.3);
int subtitleY = gif.Height - subtitleHeight;
return new Rectangle(0, subtitleY, gif.Width, subtitleHeight);
}
private List> ExtractUniqueTextFrames(
Image gif,
Rectangle subtitleRegion)
{
var uniqueFrames = new List>();
Image? previousFrame = null;
for (int i = 0; i < gif.Frames.Count; i++)
{
var frame = gif.Frames.CloneFrame(i);
var subtitleCrop = frame.Clone(ctx =>
ctx.Crop(subtitleRegion));
// Compare with previous frame
if (previousFrame == null ||
HasTextChanged(subtitleCrop, previousFrame, threshold: 0.05))
{
uniqueFrames.Add(subtitleCrop);
previousFrame = subtitleCrop;
}
}
return uniqueFrames;
}
private bool HasTextChanged(
Image current,
Image previous,
double threshold)
{
// Threshold bright pixels (white/yellow text on dark background)
var currentBright = CountBrightPixels(current);
var previousBright = CountBrightPixels(previous);
// Calculate Jaccard similarity of bright pixels
var intersection = currentBright.Intersect(previousBright).Count();
var union = currentBright.Union(previousBright).Count();
var similarity = union > 0 ? intersection / (double)union : 1.0;
// Text changed if similarity drops below threshold
return similarity < (1.0 - threshold);
}
// Helper type for bounding box + crop
private record TextCrop
{
public required Image CroppedImage { get; init; }
public required Rectangle Bounds { get; init; }
}
private List ExtractTextBoundingBoxes(
List> frames)
{
var textCrops = new List();
foreach (var frame in frames)
{
// Threshold to get text mask
var mask = ThresholdBrightPixels(frame, minValue: 200);
// Find connected components (text regions)
var components = FindConnectedComponents(mask);
// Get tight bounding box around all components
var bbox = GetTightBoundingBox(components);
// Add padding
bbox.Inflate(5, 5);
// Clone the region (dispose properly in production!)
var cropped = frame.Clone(ctx => ctx.Crop(bbox));
textCrops.Add(new TextCrop
{
CroppedImage = cropped,
Bounds = bbox
});
}
return textCrops;
}
private Image CreateHorizontalStrip(
List textCrops)
{
// Calculate strip dimensions
int totalWidth = textCrops.Sum(c => c.Bounds.Width);
int maxHeight = textCrops.Max(c => c.Bounds.Height);
// Create blank canvas
var strip = new Image(totalWidth, maxHeight);
// Paste text regions horizontally
int xOffset = 0;
foreach (var crop in textCrops)
{
strip.Mutate(ctx => ctx.DrawImage(
crop.CroppedImage,
new Point(xOffset, 0),
opacity: 1.0f));
xOffset += crop.Bounds.Width;
// Dispose crop after use (important!)
crop.CroppedImage.Dispose();
}
return strip;
}
}
```
### 可见示例
**输入投入**: anchorman-not-even-mad.gif (93 frames, 300×185)
**处理处理**:
```
1. Detect subtitle region: bottom 30% (300×55)
2. Extract unique frames: 93 frames → 2 text changes
3. Extract tight bounding boxes:
- Frame 1-45: "I'm not even mad." → 252×49 bbox
- Frame 46-93: "That's amazing." → 198×49 bbox
4. Create horizontal strip: 450×49 total
```
**产出产出产出 产出 输出产出产出**@ : @ text@ MS K1 @ 只带条 @ (450×49) @
**调时经济学**:
- 完整框架 @ 10 @ 樣本@ MS K1} @ I300×185 @ *×* * 10 * * = * @ MPK6 符號
- @2} @ ~300_matters
- **文本@ - @ 仅条**@: @ @ 450×49 @ @ *=} @ I~50# 神令
**30× 减少** 保存所有字幕文本@. @%
---
## 视觉LLM 升级
当宇宙魔方和佛罗伦萨 -2 失败或产生低端的- 信任结果, 升级为幻觉LLM @ (GPTMS K4o}, Claude *3.5 Sonnet, Gemini Pro VisionMSKO8 或者像迷你式的Ollama模型
### 质量门
```csharp
public class OcrQualityWave : IAnalysisWave
{
private readonly SpellChecker _spellChecker;
public string Name => "OcrQualityWave";
public int Priority => 58; // After Florence-2 and Tesseract
public async Task> AnalyzeAsync(
string imagePath,
AnalysisContext context,
CancellationToken ct)
{
var signals = new List();
// Get best OCR result from earlier waves (priority order)
string? ocrText =
context.GetValue("ocr.ml.text") ?? // Florence-2 (priority 51)
context.GetValue("ocr.text"); // Tesseract (priority 50)
if (string.IsNullOrWhiteSpace(ocrText))
{
signals.Add(new Signal
{
Key = "ocr.quality.no_text",
Value = true,
Confidence = 1.0,
Source = Name
});
return signals;
}
// Run spell check (deterministic quality assessment)
var spellResult = _spellChecker.CheckTextQuality(ocrText);
// Additional quality signals to avoid false positives
var alphanumRatio = CalculateAlphanumericRatio(ocrText); // Letters/digits vs junk
var avgTokenLength = CalculateAverageTokenLength(ocrText);
signals.Add(new Signal
{
Key = "ocr.quality.spell_check_score",
Value = spellResult.CorrectWordsRatio,
Confidence = 1.0,
Source = Name,
Metadata = new Dictionary
{
["total_words"] = spellResult.TotalWords,
["correct_words"] = spellResult.CorrectWords,
["garbled_words"] = spellResult.GarbledWords,
["alphanum_ratio"] = alphanumRatio,
["avg_token_length"] = avgTokenLength
}
});
// Deterministic escalation threshold
// NOTE: Spellcheck alone can false-trigger on proper nouns, memes, brand names.
// Use additional signals (alphanum ratio, token length) to reduce false escalations.
bool isGarbled = spellResult.CorrectWordsRatio < 0.5 &&
alphanumRatio > 0.7; // Mostly valid characters, just not in dictionary
signals.Add(new Signal
{
Key = "ocr.quality.is_garbled",
Value = isGarbled,
Confidence = 1.0,
Source = Name
});
// Signal Vision LLM escalation
if (isGarbled)
{
signals.Add(new Signal
{
Key = "ocr.quality.escalation_required",
Value = true,
Confidence = 1.0,
Source = Name,
Tags = new List { "action_required", "escalation" },
Metadata = new Dictionary
{
["reason"] = "spell_check_below_threshold",
["quality_score"] = spellResult.CorrectWordsRatio,
["threshold"] = 0.5,
["target_tier"] = "vision_llm"
}
});
// Cache garbled text for Vision LLM to access
context.SetCached("ocr.garbled_text", ocrText);
}
return signals;
}
}
```
**升级是决定性的**-=YTET -伊甸园字幕组=- 翻译:
### 带电影片的视觉LLM
当动画 GIFs @ , 触发升级时, 请使用文本@ MS K1 @ 仅条字符@ MPK2}
```csharp
public class VisionLlmWave : IAnalysisWave
{
private readonly IVisionLlmClient _client;
public string Name => "VisionLlmWave";
public int Priority => 50;
public async Task> AnalyzeAsync(
string imagePath,
AnalysisContext context,
CancellationToken ct)
{
var signals = new List();
// Check if escalation is required
var escalationRequired = context.GetValue(
"ocr.quality.escalation_required");
if (!escalationRequired)
{
signals.Add(new Signal
{
Key = "vision.llm.skipped",
Value = true,
Confidence = 1.0,
Source = Name,
Metadata = new Dictionary
{
["reason"] = "no_escalation_required"
}
});
return signals;
}
// For animated GIFs, use text-only strip
string imageToProcess = imagePath;
bool usedFilmstrip = false;
if (context.GetValue("identity.frame_count") > 1)
{
var filmstrip = await CreateTextOnlyStripAsync(imagePath, ct);
imageToProcess = filmstrip.Path;
usedFilmstrip = true;
signals.Add(new Signal
{
Key = "vision.filmstrip.created",
Value = true,
Confidence = 1.0,
Source = Name,
Metadata = new Dictionary
{
["mode"] = "text_only",
["region_count"] = filmstrip.RegionCount,
["token_reduction"] = filmstrip.Reduction,
["original_tokens"] = filmstrip.OriginalTokens,
["final_tokens"] = filmstrip.TotalTokens
}
});
}
// Build constrained prompt
var prompt = BuildConstrainedPrompt(context);
// Call Vision LLM
var result = await _client.ExtractTextAsync(
imageToProcess,
prompt,
ct);
// Emit OCR text signal (Vision LLM tier)
signals.Add(new Signal
{
Key = "ocr.vision.text", // Vision LLM OCR result
Value = result.Text,
Confidence = 0.95, // High but not 1.0 - still probabilistic
Source = Name,
Tags = new List { "ocr", "vision", "llm" },
Metadata = new Dictionary
{
["model"] = result.Model,
["used_filmstrip"] = usedFilmstrip,
["inference_time_ms"] = result.InferenceTime,
["token_count"] = result.TokenCount,
["cost_usd"] = result.Cost
}
});
// Optionally emit caption if requested (separate from OCR)
if (result.Caption != null)
{
signals.Add(new Signal
{
Key = "caption.text", // Descriptive caption, not OCR
Value = result.Caption,
Confidence = 0.90,
Source = Name,
Tags = new List { "caption", "description" }
});
}
return signals;
}
private string BuildConstrainedPrompt(AnalysisContext context)
{
var sb = new StringBuilder();
sb.AppendLine("Extract all text from this image.");
sb.AppendLine();
sb.AppendLine("CONSTRAINTS:");
sb.AppendLine("- Only extract text that is actually visible");
sb.AppendLine("- Preserve formatting and line breaks");
sb.AppendLine("- If no text is present, return empty string");
sb.AppendLine();
// Add context from earlier waves
var garbledText = context.GetCached("ocr.garbled_text");
if (!string.IsNullOrEmpty(garbledText))
{
sb.AppendLine("CONTEXT:");
sb.AppendLine("Traditional OCR detected garbled text:");
sb.AppendLine($" \"{garbledText}\"");
sb.AppendLine("Use this as a hint for stylized or unusual fonts.");
sb.AppendLine();
}
sb.AppendLine("Return only the extracted text, no commentary.");
return sb.ToString();
}
}
```
---
## 优先链
当所有级别完成\,}最后文本选择使用严格的优先排序@ :
```csharp
public static string? GetFinalText(DynamicImageProfile profile)
{
// Priority chain (highest to lowest quality)
// NOTE: This selects ONE source, but the ledger exposes ALL sources
// with confidence scores for downstream inspection
// 1. Vision LLM OCR (best for complex/garbled text)
var visionText = profile.GetValue("ocr.vision.text");
if (!string.IsNullOrEmpty(visionText))
return visionText;
// 2. Florence-2 multi-frame GIF OCR (best for animations)
var florenceMultiText = profile.GetValue("ocr.ml.multiframe_text");
if (!string.IsNullOrEmpty(florenceMultiText))
return florenceMultiText;
// 3. Florence-2 single-frame ML OCR (good for stylized fonts)
var florenceText = profile.GetValue("ocr.ml.text");
if (!string.IsNullOrEmpty(florenceText))
return florenceText;
// 4. Tesseract OCR (reliable for clean standard text)
var tesseractText = profile.GetValue("ocr.text");
if (!string.IsNullOrEmpty(tesseractText))
return tesseractText;
// 5. Fallback (empty)
return string.Empty;
}
```
**每一级各有已知特点**:
|Q来源@|Q信号密钥 @|Q最佳信任 #|#
|--------|------------|----------|------------|------|-------|
* 幻影传奇* `ocr.vision.text` |#复杂图表 @,_旋转文本@,}拼凑的字母 @MS K3 #0.95}{|#$0.001-0.01#|
~ 佛罗伦萨・-2~ (GIF})| `ocr.ml.multiframe_text` | @MSK0#动画的GIFs,配有字幕: @|}@0.85-0.92 @ @MS K3}免费的|}(~200%s) *|*
@ |佛罗伦萨-2 MS K2single _)|# `ocr.ml.text` |+Stylized fonts@,+Memes @,+装饰文字 @MS K3_Q#0.85-0.90} @|+Free|_Z~200+Msms *|*
@| 宇宙魔方#| `ocr.text` -=YTET -伊甸园字幕组=- 翻译:
---
## 成本分析
### 在三-Tier系统之前
@100# 图像@,}所有使用视野LLM=:
```
100 images × $0.005/image = $0.50
Total time: 100 × 2s = 200 seconds
```
### 3-Tier系统后
路线分布@(}%smission * ):*
- @ 60 @ 图像 @ MPK1 @ FAST 路由 @ I(Florence@ MS K3} 只,free @ MCK5 @ @ @ ~100ms @) @
- @25@ 图像 @ → @ balanceD 路线 @ I( @ Florence_ MSSK3} @ TMK4 宇宙魔方=, @ freeQ,# * MS K7 @ ms@) @
- @10 图像@→ 平等路线 @MS K2 视野LM, $0.005,#~2s{)
- @5 @ 图像 @ →@ NAMATIONed 路由 @ I( @ filmstrip,} @ MPK4 @ @ @ MCK5 @sQK6#
```
Cost:
60 × $0 = $0
25 × $0 = $0
10 × $0.005 = $0.05
5 × $0.002 = $0.01
Total: $0.06
Time:
60 × 0.1s = 6s
25 × 0.3s = 7.5s
10 × 2s = 20s
5 × 2.5s = 12.5s
Total: 46 seconds
Savings:
Cost: 88% reduction ($0.50 → $0.06)
Time: 77% reduction (200s → 46s)
```
**中层 @( @Florence@-2)}以零成本处理图像的\ 85%}{.}**
---
## 团结在一起
这里的'是配有字幕的Meme GIF 的完整流源
```
1. Load image: anchorman-not-even-mad.gif (93 frames)
2. IdentityWave (priority 10):
→ identity.frame_count = 93
→ identity.format = "gif"
→ identity.is_animated = true
3. TextLikelinessWave (priority 40, ~10ms):
→ Heuristic text detection: 15 regions in bottom 30%
→ Subtitle pattern: DETECTED
→ text.likeliness = 0.85
4. OcrWave (priority 50, ~60ms):
→ Run Tesseract OCR on first frame
→ ocr.text = "I'm not emn mad." (garbled)
→ ocr.confidence = 0.62
5. MlOcrWave (priority 51, ~180ms):
→ Tesseract confidence < 0.95, run Florence-2
→ Sample 10 frames (animated GIF)
→ Run Florence-2 on each frame (parallel)
→ Deduplicate: 10 results → 2 unique texts
→ ocr.ml.multiframe_text = "I'm not even mad.\nThat's amazing."
→ ocr.ml.confidence = 0.91
6. OcrQualityWave (priority 58, ~5ms):
→ Check Florence-2 result
→ Spell check: 6/6 words correct (100%)
→ ocr.quality.is_garbled = false
→ ocr.quality.escalation_required = false
7. VisionLlmWave (priority 80, SKIPPED):
→ No escalation required (Florence-2 succeeded)
Final output:
Text: "I'm not even mad.\nThat's amazing."
Source: ocr.ml.multiframe_text
Confidence: 0.91
Cost: $0 (local processing)
Time: ~250ms total (Tesseract + Florence-2)
```
如果佛罗伦萨*-2失信,
```
6. OcrQualityWave:
→ Spell check: 2/6 words correct (33%)
→ ocr.quality.is_garbled = true
→ ocr.quality.escalation_required = true
7. VisionLlmWave:
→ Create text-only filmstrip (2 regions, 450×49)
→ Send to Vision LLM: "Extract all text from this strip"
→ vision.llm.text = "I'm not even mad.\nThat's amazing."
→ Confidence: 0.95
→ Cost: ~$0.002 (30× token reduction vs full frames)
→ Time: ~2.3s
```
---
## 配置配置
三个“-”系统是完全可配置的 “:”
```json
{
"DocSummarizer": {
"Ocr": {
"Tesseract": {
"Enabled": true,
"DataPath": "/usr/share/tesseract-ocr/4.00/tessdata",
"Languages": ["eng"],
"EarlyExitThreshold": 0.95
},
"Florence2": {
"Enabled": true,
"ModelPath": "models/florence2-base",
"ConfidenceThreshold": 0.85,
"MaxFrames": 10,
"DeduplicationMethod": "levenshtein",
"LevenshteinThreshold": 0.85
},
"Quality": {
"SpellCheckThreshold": 0.5,
"EscalationEnabled": true
}
},
"VisionLlm": {
"Enabled": true,
"Provider": "ollama",
"OllamaUrl": "http://localhost:11434",
"Model": "minicpm-v:8b",
"MaxRetries": 3,
"TimeoutSeconds": 30
},
"Filmstrip": {
"TextOnlyMode": true,
"SubtitleRegionPercent": 0.3,
"BrightPixelThreshold": 200,
"TextChangeThreshold": 0.05
},
"Routing": {
"FastRouteConfidence": 0.8,
"BalancedRouteConfidence": 0.5,
"TextDetectionEnabled": true
}
}
}
```
---
## 失败模式
@| 失灵 #| 侦测#|回应 #|#
|---------|-----------|----------|
| **宇宙魔方失败** 或拼写检查 @< @ 0.5}{|_ZARBAR_ 和佛罗伦萨的连结
| **佛罗伦萨-2失敗** @0.5}@| Q|* *Escratiate to Vision LLM * MASK6 * 或拼写检查 * < * * @ MS K4 * * * NSK5 * 已升格为幻觉LLM=MSK-6*
| **愿景LLM超时** 请求超过 30%s *|%s*}返回现有最佳OCR结果
| **所有级别失败** @ |_ 所有结果都是空的或乱七八糟的 @ I|} 返回空白字符串并充满信心 *\ MS K2} @ MPK3}
| **APPI费用限额** @|_每日预算超过#|_禁用视觉LLM%,}使用佛罗伦萨=-2}仅使用@|
| **型号不详** @ |_ Skipp Terrole@,}继续到下一站 @|#
每个失败都是决定性的 并用完整的出处记录下来
---
## 与其他方法的比较
### 传统:Tesseract MS K1手动后退
```
For each image:
1. Run Tesseract
2. If looks wrong, manually fix or skip
Problems:
- No middle tier (binary: works or doesn't)
- Manual intervention required
- No cost optimization
```
### Cloud M-First: 總是使用視覺LLM
```
For each image:
1. Send to GPT-4o/Claude
2. Pay $0.005-0.01 per image
Problems:
- Expensive (85% of images could be free)
- Slow (network latency)
- Still hallucinates without constraints
```
### 3⁄4 ̄ ̧漯B
```
For each image:
1. OpenCV text detection (5-20ms, free)
2. Route to appropriate tier
3. Florence-2 handles 85% locally (200ms, free)
4. Vision LLM only for complex cases (2-5s, $0.001-0.01)
Benefits:
- 88% cost reduction
- 77% faster (most images process locally)
- Deterministic escalation (auditable)
- Filmstrip optimization (30× token reduction)
- Constrained by deterministic signals
```
---
## 结论结论
3-第OCR输油管证明 **{\fn华文楷体\fs16\1cHE0E0E0}费用是多少?** 和 **本地端@- @% 1 第一处理** 能在不牺牲质量的前提下,大大改善业绩和经济学。
密钥洞见@: @%
1. **佛罗伦萨・-2ONNX是甜蜜的地方**“:”比“视觉光学LM”更快、更便宜的Stencils Stylized 字体的宇宙法更好
2. **文本@ - @ $% 1 仅实现 & MS K1 pass reduction**@ :_ 提取捆绑框@,}未完整框
3. **出行是决定性的**: OpenCV 检测 *+信任阈值*, 无猜測
4. **升级是可审计的**“: ” 每一级都以源头发出信号。
5. **失败是优雅的***:* 优先链条确保向现有最佳来源回流
缩放比例表@: **本地定点分析 @ →当地 ML 模型 @ I→云层升级**每个级别都有已知的特点和成本交易
这是用于 OCR:}确定性信号的受控制模糊度 {(}_spell check\ ,} 文本检测}{MS K3 限制概率模型 {(}{Florence-2,}Vision LLM),和最终产出总合源,按质量计算{MSC7
---
## 资源资源资源 资源和资源资源资源
### LucidRAG 文档
- **[图像成像器库](https://github.com/scottgal/lucidrag/tree/main/src/Mostlylucid.DocSummarizer.Images)** - 源代码
- **[OCR 愿景一体化](https://github.com/scottgal/lucidrag/blob/main/src/Mostlylucid.DocSummarizer.Images/docs/vision-ocr-integration.md)** -Routing,电影剧MS K2象征性经济学
- **[《建筑结构指南》](https://github.com/scottgal/lucidrag/blob/main/src/Mostlylucid.DocSummarizer.Images/docs/architecture.md)** -=YTET -伊甸园字幕组=- 翻译:
- **[管道文件](https://github.com/scottgal/lucidrag/blob/main/src/Mostlylucid.DocSummarizer.Images/docs/pipelines.md)** @ -_ Auto @ MASK1}平衡 @ I,优质路线
- **[信号参考](https://github.com/scottgal/lucidrag/blob/main/src/Mostlylucid.DocSummarizer.Images/docs/signals.md)** 完整的信号目录
### CLI 工具
- **[图像放大器 CLI](https://github.com/scottgal/lucidrag/tree/main/src/Mostlylucid.ImageSummarizer.Cli)** - 命令 @-线性工具
- **[CLI 阅读](https://github.com/scottgal/lucidrag/blob/main/src/Mostlylucid.ImageSummarizer.Cli/README.md)** - 用法和配置
- **[演示图像](https://github.com/scottgal/lucidrag/tree/main/src/Mostlylucid.ImageSummarizer.Cli/demo-images)** -样本 GIF 和框架条
### 研究文件
- **[佛罗伦萨-2文件](https://arxiv.org/abs/2311.06242)** - 微软' 視覺-語言模型
- **[EST 文本检测器](https://arxiv.org/abs/1704.03155)** 高效的现场文字探测
- **[CRAFT 文件](https://arxiv.org/abs/1904.01941)** -字符区域意识
- **[真实的#-ESRGAN](https://arxiv.org/abs/2107.10833)** “-实用超级”“-分辨率”
- **[CLIP 文件](https://arxiv.org/abs/2103.00020)** 学习可转移视觉模型
### 相关条款
- **[影像情报部分](/blog/constrained-fuzzy-image-intelligence)** - 波浪结构概览
- **[DocSummamer 缩写器](/blog/building-a-document-summarizer-with-rag)** -文件分析管道
- **[数据合成器](/blog/datasummarizer-how-it-works)** - 数据分析方法
---
## 系列丛书
@ | @ part * MPK1_ 模式 @ I|} 聚焦@ MSC3
|------|---------|-------|
| 1 | [受限制的模糊](/blog/constrained-fuzziness-pattern) @|单元件 @ |#
| 2 | [限制的模糊MOM](/blog/constrained-mom-mixture-of-models) | 多元件|
| 3 | [环境拖累](/blog/constrained-fuzzy-context-dragging) *|Q时间* */Q内存*
| 4 | [图像情报](/blog/constrained-fuzzy-image-intelligence) |#波形结构_,}图案_|
| **4.1** | **3-Tier OCR 管道 @(}这篇文章=)** | **OCR,}ONNX 模型 , 胶片** |
**下一頁**@ :_ part *\ MS K1} 将展示图像的缩写@ ,} [DocSummamer 缩写器](/blog/building-a-document-summarizer-with-rag),和 [数据合成器](/blog/datasummarizer-how-it-works) 以 LucidRAG\ . 组成多-modal 图形 RAG
所有部件都沿着相同的变数 : **建议的概率元件**.