Ever needed to pull text from an image or a PDF? Let’s be real—it’s usually a time sink. Enter Ollama-OCR, a slick Optical Character Recognition (OCR) package built to streamline extraction across formats by tapping into cutting-edge Vision Language Models (VLMs) running locally via Ollama.

Ollama-OCR ships as both a Python package and a Streamlit web app. That means flexibility: drop it into your codebase or spin up a visual UI for non-technical stakeholders.

Why Ollama-OCR Deserves a Spot in Your Toolkit

This isn't just another Tesseract wrapper. It’s built on modern VLMs, unlocking capabilities traditional OCR can't touch. Here’s the rundown:

• Unified PDF & Image Support: Stop juggling converters. Feed it PDFs or images directly.
• Model Agnostic (BYO VLM): Swap models based on the job—speed vs. accuracy, local vs. heavy lifting.
• Flexible Output Formats: Get raw text, clean Markdown, structured JSON, Key-Value pairs, or extracted Tables.
• Native Batch Processing: Crunch directories of files in parallel with progress tracking built-in.
• Custom Prompting: This is the killer feature. Instruct the model what to extract (e.g., "Only grab invoice totals and dates").
• Explicit Language Support: Hint the model for better accuracy on non-English docs.

Under the Hood: The Vision Model Lineup

The magic happens because Ollama-OCR abstracts the model layer. You pick the engine; it handles the inference plumbing. Here’s the current roster:

Vision Model	Core Strength	Best For
LLaVA	Efficient, real-time processing	Speed-critical pipelines; acceptable accuracy trade-offs
Llama 3.2 Vision	SOTA accuracy for complex document reasoning	Default recommendation for precision extraction
Granite 3.2 Vision	Compact, enterprise-grade; excels at structured visual data (tables, charts, infographics, plots, diagrams)	Structured document analysis (financial reports, scientific papers)
Moondream	Tiny footprint (~1.6B/2B params), optimized for edge/on-device	Resource-constrained environments; offline mobile/edge deployments
MiniCPM-V	Handles arbitrary aspect ratios & high res (up to 1.8MP / 1344x1344) without tiling artifacts	High-res scans, odd-sized receipts, wide tables

Output Formats: Pick Your Poison

Post-extraction, you dictate the schema:

• Markdown: Preserves headers, lists, code blocks—ready for RAG ingestion or static site generators.
• Text: Clean, raw string. No markup noise.
• JSON: Single object wrapper for programmatic parsing.
• Structured: Organized object hierarchy for downstream processing.
• Key-Value Pairs: Dictionary output for form/invoice field extraction.
• Table Format: Dedicated extraction for tabular data (CSV/Markdown table ready).

Getting Started: The Python SDK

Prereqs: Ollama installed and running locally. Pull your vision models first:

bash
ollama pull llama3.2-vision:11b
ollama pull granite3.2-vision
ollama pull moondream
ollama pull minicpm-v

Install the package:

bash
pip install ollama-ocr

Single File Inference

Simple, synchronous extraction with full control over prompt engineering.

python
from ollama_ocr import OCRProcessor

# Init processor - point to your Ollama endpoint
ocr = OCRProcessor(
    model_name='llama3.2-vision:11b',
    base_url="http://host.docker.internal:11434/api/generate" # Adjust if remote/containerized
)

# Process image or PDF
result = ocr.process_image(
    image_path="path/to/your/image.png", # Accepts PDF paths too
    format_type="markdown",               # Options: markdown, text, json, structured, key_value
    custom_prompt="Extract all text, focusing on dates and names.", # Prompt engineering hook
    language="English"                    # Language hint for the VLM
)
print(result)

Batch Processing (Parallelized)

Built for throughput. Uses ThreadPoolExecutor under the hood (max_workers).

python
from ollama_ocr import OCRProcessor

# Init with parallel workers
ocr = OCRProcessor(
    model_name='llama3.2-vision:11b',
    max_workers=4 # Tune based on VRAM/RAM & CPU cores
)

# Process a directory recursively
batch_results = ocr.process_batch(
    input_path="path/to/images/folder",  # Dir path or list of file paths
    format_type="markdown",
    recursive=True,                       # Walk subdirectories
    preprocess=True,                      # Enable image preprocessing (deskew, denoise, etc.)
    custom_prompt="Extract all text, focusing on dates and names.",
    language="English"
)

# Iterate results
for file_path, text in batch_results['results'].items():
    print(f"\nFile: {file_path}")
    print(f"Extracted Text: {text}")

# Built-in stats
print("\nProcessing Statistics:")
print(f"Total images: {batch_results['statistics']['total']}")
print(f"Successfully processed: {batch_results['statistics']['successful']}")
print(f"Failed: {batch_results['statistics']['failed']}")

The No-Code Route: Streamlit Web UI

Prefer a GUI? The repo includes a full-featured Streamlit app. Drag-and-drop, live preview, real-time streaming output, and one-click download for batch jobs.

Spin it up:

1. Clone:

   bash
   git clone https://github.com/imanoop7/Ollama-OCR.git

2. Enter Dir:

   bash
   cd Ollama-OCR

3. Install Deps:

   bash
   pip install -r requirements.txt

4. Navigate to App Source:

   bash
   cd src/ollama_ocr

5. Launch:

   bash
   streamlit run app.py

Ecosystem & Extensibility

The project includes a Google Colab notebook for zero-setup demos and showcases integrations with AutoGen and LangGraph—signaling it’s ready for agentic RAG pipelines and complex document understanding workflows. Licensed MIT, so fork away, contribute back, or embed in commercial products.

Reference

https://github.com/imanoop7/Ollama-OCR