Mike Higgins - Digital Consultant

Optical Character Recognition (OCR) has long been treated as a server-side problem. The assumption has always been that the computational weight of reading pixels and translating them into text belongs behind an API endpoint, not in a user's browser tab. Tesseract.js challenges that assumption entirely. It brings one of the most battle-tested OCR engines in existence directly into the browser, with no server required, no API keys, and no data leaving the user's machine.

This post is a thorough introduction to Tesseract.js: what it is, how it came to be, how it works under the hood, and how to use it effectively in your own projects.

A Brief History of Tesseract

To understand Tesseract.js, you first need to understand Tesseract itself.

Tesseract began its life at Hewlett-Packard Laboratories in Bristol and Greeley, Colorado, between 1985 and 1994. It was developed initially as a PhD research project aimed at improving the accuracy of OCR for HP's flatbed scanners. By 1995, it had become one of the highest-performing OCR engines in the UNLV Annual Test of OCR Accuracy, an independent benchmark that was taken seriously in the field. Despite this, it never became a commercial product, and development quietly wound down.

The engine might have faded into obscurity had Google not become involved. In 2005, HP released Tesseract as an open-source project. In 2006, Google adopted it, and original developer Ray Smith led ongoing development there until 2017. It is now maintained by the open-source community on GitHub.

The most significant architectural shift came with Tesseract version 4.0, which introduced a new recognition engine powered by Long Short-Term Memory (LSTM) neural networks. Rather than recognising individual characters in isolation, the LSTM engine processes sequences of text, meaning surrounding characters provide context that dramatically improves accuracy. This is the engine that now powers Tesseract.js under the surface.

What Is Tesseract.js?

Tesseract.js is a JavaScript library that wraps the Tesseract OCR engine. Crucially, it is not a JavaScript rewrite of Tesseract. Instead, the original C++ source code of the engine has been compiled into WebAssembly (WASM) using the Emscripten toolchain. Tesseract.js is the glue layer: a JavaScript API that initialises the WASM module, manages memory, handles language data, and exposes a clean, Promise-based interface for your application to consume.

The key benefits of this architecture are:

Client-side processing: OCR runs entirely in the user's browser. Sensitive documents, invoices, or passports never leave the device.
No infrastructure required: There is no server to provision, no API rate limit to worry about, and no cost-per-page billing model.
Near-native performance: Because the engine runs as WebAssembly rather than interpreted JavaScript, it executes at speeds much closer to a native application.

The library is maintained on GitHub at naptha/tesseract.js and is available via npm.

The Technical Architecture in Depth

Understanding what actually happens when you call Tesseract.recognize() is worthwhile, because it shapes how you use the library correctly.

1. WebAssembly and Emscripten

Emscripten is a toolchain that cross-compiles C and C++ code into WebAssembly. The result is a .wasm file: a compact binary format that browsers can parse and execute in a sandboxed environment. WebAssembly runs at near-native speed because it is much closer to machine code than JavaScript, and the browser JIT-compiles it rather than interpreting it.

When Tesseract.js initialises, it fetches the .wasm binary (typically from a CDN or your own server) and loads it into the browser's memory. From that point on, the full OCR engine is running locally.

Tesseract.js actually ships multiple WASM builds:

Standard: The baseline build compatible with all modern browsers.
SIMD (Single Instruction, Multiple Data): A hardware-accelerated variant for browsers that support WebAssembly SIMD operations. This can yield significant performance improvements on image preprocessing tasks.

The library detects which build to use automatically at runtime.

2. Web Workers: Keeping the UI Responsive

OCR is a long-running, CPU-intensive task. If it ran on the browser's main thread, the entire user interface would freeze for the duration of the recognition. Tesseract.js avoids this by automatically spawning a Web Worker to handle OCR work.

A Web Worker is a background thread with no access to the DOM. It communicates with the main thread exclusively by passing messages. This means your application's UI, including animations, input handling, and rendering, remains fully responsive while Tesseract is reading an image in the background.

This is transparent to you as a developer. When you call worker.recognize(image), Tesseract.js internally posts a message to the worker thread, receives progress updates and the final result through a message channel, and resolves the returned Promise on the main thread when the job is complete.

3. Language Data Files (`.traineddata`)

Tesseract cannot recognise text without language model files. These .traineddata files contain:

The LSTM neural network weights for the target language.
A unicharset: the set of characters the model can recognise.
A recoder: a mapping between the character set and the neural network's internal representations.
Optional dictionary data, including word lists and patterns that help the engine prefer real words when resolving ambiguous characters.

When you specify a language such as 'eng' when creating a worker, Tesseract.js fetches the corresponding .traineddata file. By default, this happens via a CDN. For production applications, you should host these files yourself to avoid network dependency and reduce latency.

Google publishes three families of trained data:

| Family | Optimised For | Notes | |---|---|---| | tessdata_fast | Speed | Uses integer-quantised LSTM models. Smaller files, faster inference. | | tessdata_best | Accuracy | Uses full-precision float models. Slower but more accurate. | | tessdata | Compatibility | Contains legacy models alongside updated LSTM versions for compatibility with older workflows. |

For browser-based use, tessdata_fast is almost always the right choice. The accuracy difference between fast and best is modest for clean printed text, and the reduced file size has a meaningful impact on how quickly a first-time user gets a result.

4. The OCR Pipeline

Once the engine and language data are ready, the recognition pipeline for a single page proceeds roughly as follows:

Image ingestion: The input can be a URL, a File, a Blob, a base64 string, a HTMLImageElement, a HTMLCanvasElement, or a raw ImageData object. Tesseract.js normalises all of these into image data that the WASM engine can process.
Preprocessing: The engine analyses the image's binarisation threshold, converting the colour image into black and white. This makes character edges cleaner and improves recognition accuracy.
Layout analysis: Tesseract segments the image into regions, columns, paragraphs, lines, and finally individual word bounding boxes. This stage is called "page segmentation mode" (PSM), and it is configurable.
LSTM recognition: The LSTM network processes each text line as a sequence. It assigns probability distributions over possible characters for each position, then uses a language model (when available) to decode the most likely string.
Result construction: Tesseract assembles the final output, including the plain text, confidence scores per word, and bounding box coordinates for every character, word, line, and block. These are returned to your application as a structured result object.

Getting Started

Installation

bash

npm install tesseract.js

Tesseract.js works in the browser and in Node.js. The API is identical in both environments.

Basic Recognition

javascript

import { createWorker } from 'tesseract.js';

const worker = await createWorker('eng');
const { data: { text } } = await worker.recognize('image.png');
console.log(text);
await worker.terminate();

That is the entirety of a basic integration. createWorker('eng') is the v5 API, which handles loading the language model in a single call, removing the need for the separate loadLanguage() and initialize() calls required in older versions.

Reading a Canvas Element

One of the most powerful use cases in browser applications is reading text from a <canvas>. This is exactly what we do in this document converter when native PDF text extraction fails.

javascript

const canvas = document.getElementById('my-canvas');
const { data: { text } } = await worker.recognize(canvas);

This allows you to combine Tesseract.js with other rendering libraries. For example, you can use pdf.js to render each page of a PDF onto a canvas at high resolution, then pass that canvas to Tesseract.js for text extraction. This is precisely the pattern used by this converter's OCR fallback.

Accessing Rich Result Data

Tesseract.js returns far more than just the plain text string. The full result object contains bounding box coordinates and confidence scores at multiple levels of granularity.

javascript

const { data } = await worker.recognize('document.jpg');

// Plain text
console.log(data.text);

// Per-word confidence and position
data.words.forEach(word => {
  console.log(`"${word.text}" - confidence: ${word.confidence}%`);
  console.log('Bounding box:', word.bbox); // { x0, y0, x1, y1 }
});

// Access at block, paragraph, line, or character level
console.log(data.blocks);
console.log(data.lines);
console.log(data.symbols); // individual characters

This granularity is extremely useful for applications that need to reconstruct a document's visual layout, highlight recognised text on top of the original image, or validate individual field values from a form.

Monitoring Progress

Because OCR can take several seconds, tracking progress and communicating it to the user is important.

javascript

const worker = await createWorker('eng', 1, {
  logger: (m) => {
    if (m.status === 'recognizing text') {
      console.log(`Progress: ${Math.round(m.progress * 100)}%`);
    }
  }
});

The logger callback receives frequent progress updates with a status string and a progress float between 0 and 1. This is straightforward to connect to a progress bar or status indicator in your UI.

Page Segmentation Modes

Tesseract's page segmentation mode (PSM) tells the engine what kind of layout to expect. Choosing the correct PSM for your input can significantly affect accuracy.

| PSM | Value | Use Case | |---|---|---| | OSD only | 0 | Orientation and script detection only | | Auto OSD | 1 | Fully automatic segmentation with OSD | | Auto (no OSD) | 3 | Automatic page segmentation (default) | | Single column | 4 | Variable-size column of text | | Single block of text | 6 | Assumes a uniform block. Good for paragraphs. | | Single line | 7 | Treats the entire image as a single line | | Single word | 8 | Treats the entire image as a single word | | Circle word | 9 | Treats the image as a single word in a circle | | Single character | 10 | Single character recognition | | Sparse text | 11 | Find as much text as possible, in no particular order |

javascript

import { createWorker, PSM } from 'tesseract.js';

const worker = await createWorker('eng');
await worker.setParameters({
  tessedit_pageseg_mode: PSM.SINGLE_COLUMN,
});

For typical document pages with mixed headings, paragraphs, and columns, the default (PSM 3) works well. For UI screenshots or images containing a single label or form field, PSM 7 or PSM 8 will give you a much cleaner result.

Processing Multiple Images with a Scheduler

When you need to process many images concurrently, the Scheduler distributes jobs across a pool of workers in parallel, leveraging multi-core CPUs for significant throughput improvements.

javascript

import { createWorker, createScheduler } from 'tesseract.js';

const scheduler = createScheduler();

// Create a pool of workers
const workers = await Promise.all(
  Array.from({ length: 4 }, () => createWorker('eng'))
);
workers.forEach(w => scheduler.addWorker(w));

// Add all jobs. The scheduler distributes them across workers automatically.
const results = await Promise.all(
  imageList.map(img => scheduler.addJob('recognize', img))
);

await scheduler.terminate(); // Terminates all workers in the pool

All workers in a scheduler should be configured with the same language and parameters. The scheduler queues incoming jobs and dispatches each one to the next available worker, so as one worker finishes a page it immediately picks up the next pending job.

Image Quality and Preprocessing

Tesseract is powerful, but it is not magic. Its accuracy is directly dependent on image quality. A few practical guidelines:

Scale up low-resolution sources. Tesseract performs best when text height is at least 20 to 30 pixels. When rendering PDFs with pdf.js, always use a scale factor of at least 2.0. This doubles the resolution and gives the LSTM engine more pixel information to work with.

Convert to greyscale. Colour information is noise for OCR. Converting to greyscale before recognition reduces the data the engine has to process and often improves accuracy.

Apply thresholding for low-contrast images. Scanned documents sometimes have grey backgrounds, shadows, or coffee stains. Increasing contrast or applying a binary threshold (making pixels either black or white) can dramatically improve recognition.

You can do all of this preprocessing in the browser using the Canvas API:

javascript

function preprocessCanvas(sourceCanvas) {
  const { width, height } = sourceCanvas;
  const outputCanvas = document.createElement('canvas');
  outputCanvas.width = width;
  outputCanvas.height = height;

  const ctx = outputCanvas.getContext('2d');
  ctx.drawImage(sourceCanvas, 0, 0);

  // Apply greyscale filter
  ctx.filter = 'grayscale(1) contrast(1.5)';
  ctx.drawImage(outputCanvas, 0, 0);

  return outputCanvas;
}

For more sophisticated preprocessing, the jimp library (browser-compatible) offers thresholding, dilation, and noise removal operations.

Limitations and When Not to Use It

Tesseract.js is an excellent tool, but it has real limitations that are worth understanding.

It does not understand handwriting. Tesseract's LSTM engine is trained on printed text. Handwritten notes, cursive scripts, and sketched annotations will typically produce poor results. Dedicated handwriting recognition models (often powered by specialised neural networks) are needed for this use case.

Complex table structures are unreliable. Tesseract will extract text from within table cells, but it does not understand the relational structure of rows and columns. If you need to extract structured data from tables, you will need post-processing logic to reconstruct the layout from the bounding box coordinates.

Multi-language documents require care. If a single document contains text in multiple languages, you can load multiple language models and pass them as a +-delimited string: createWorker('eng+fra+deu'). However, this increases loading time and memory usage. It also increases the chances of misclassification when the engine encounters an ambiguous character that exists in multiple alphabets.

Performance varies by device. Tesseract.js runs on the end user's hardware. A high-resolution, multi-page scanned document processed on a low-powered mobile phone will take significantly longer than on a desktop machine. Always set user expectations appropriately and provide meaningful progress feedback.

First-run latency. The WASM binary and the language data files must be downloaded on the first use. For the English fast model, this is roughly 4 to 6 megabytes. Subsequent runs benefit from browser caching, but the first experience can feel slow if you do not prepare the user for it. Pre-loading the worker in the background while the user is filling in a form or reading instructions is a good pattern.

Privacy Considerations

One of Tesseract.js's most compelling properties for certain applications is privacy. Because OCR runs entirely client-side:

The contents of uploaded documents are never transmitted to a server.
There is no third-party processing of sensitive data.
The application works offline once the WASM and language data files have been cached by the browser.

This makes it a strong fit for tools that handle financial documents, medical records, identity documents, or any other content where users should reasonably expect their data to remain on their own device.

Real-World Use Cases

To make this concrete, here are some application patterns where Tesseract.js is a natural fit:

Document digitisation tools: Convert scanned PDFs or photographed pages into searchable, editable text. This is the use case covered in this document converter's OCR fallback.

Receipt and invoice parsing: Extract totals, line items, and dates from photographed receipts. Pair the bounding box output with layout logic to identify specific fields.

Accessibility tools: Read aloud text from images on web pages for users who rely on screen readers, without requiring server-side infrastructure.

Proof of identity workflows: Allow users to photograph and extract their name and address from utility bills or other documents locally, without uploading the document.

Form data extraction: Pre-fill form fields by reading a photographed paper form, reducing manual data entry.

Putting It All Together: A Worked Example

Here is a self-contained example that renders a PDF page to a canvas and extracts text using Tesseract.js, closely mirroring the pattern used in this project's OCR fallback:

javascript

import { createWorker } from 'tesseract.js';
import * as pdfjs from 'pdfjs-dist';

async function extractTextFromPdf(pdfFile) {
  // Set up pdfjs worker
  const workerUrl = new URL('pdfjs-dist/build/pdf.worker.min.js', import.meta.url);
  pdfjs.GlobalWorkerOptions.workerSrc = workerUrl.toString();

  const arrayBuffer = await pdfFile.arrayBuffer();
  const pdf = await pdfjs.getDocument({ data: arrayBuffer }).promise;

  // Initialise Tesseract worker once, outside the page loop
  const ocrWorker = await createWorker('eng');
  const pages = [];

  for (let pageNum = 1; pageNum <= pdf.numPages; pageNum++) {
    const page = await pdf.getPage(pageNum);

    // Render at 2x scale for better OCR accuracy
    const viewport = page.getViewport({ scale: 2.0 });
    const canvas = document.createElement('canvas');
    canvas.width = viewport.width;
    canvas.height = viewport.height;

    const context = canvas.getContext('2d');
    await page.render({ canvasContext: context, viewport }).promise;

    const { data: { text } } = await ocrWorker.recognize(canvas);
    pages.push(text.trim());
  }

  await ocrWorker.terminate();
  return pages.join('\n\n');
}

Notice that the worker is created once before the loop and terminated once after. Creating a new worker for every page would incur repeated WASM loading and language model fetching, which would be both slow and memory-intensive.

Summary

Tesseract.js is a remarkable piece of engineering. It takes 40 years of OCR research, compiles it to WebAssembly, and delivers it to a browser tab in a handful of kilobytes of JavaScript glue code. For the right use case, it eliminates an entire category of server infrastructure while simultaneously improving user privacy.

The key things to carry away from this post:

Create workers once and reuse them. Worker creation is expensive; recognition calls are cheap by comparison.
Match your scale factor to your accuracy requirements. Rendering at 2x or higher resolution is nearly always worth it.
Prefer tessdata_fast for browser use. The accuracy trade-off versus tessdata_best is small; the file size difference is not.
Use a Scheduler for batch jobs. It parallelises work across multiple workers automatically.
Set user expectations. OCR takes time. Progress indicators and informative toast messages make that time feel acceptable.
Consider privacy as a feature. Client-side OCR is a genuine selling point for applications that handle sensitive documents.

If you are building a tool that needs to extract text from images or scanned documents, Tesseract.js deserves serious consideration before you reach for a cloud API.

Tesseract.js is open source and available at github.com/naptha/tesseract.js. The Tesseract OCR engine is available at github.com/tesseract-ocr/tesseract.

A Brief History of Tesseract

What Is Tesseract.js?

The Technical Architecture in Depth

1. WebAssembly and Emscripten

2. Web Workers: Keeping the UI Responsive

3. Language Data Files (.traineddata)

4. The OCR Pipeline

Getting Started

Installation

Basic Recognition

Reading a Canvas Element

Accessing Rich Result Data

Monitoring Progress

Page Segmentation Modes

Processing Multiple Images with a Scheduler

Image Quality and Preprocessing

Limitations and When Not to Use It

Privacy Considerations

Real-World Use Cases

Putting It All Together: A Worked Example

Summary

3. Language Data Files (`.traineddata`)