Mistral AI: Convert PDFs to Markdown with New API

Mistral Launches New API for PDF Processing

On Thursday, Mistral, a French developer of large language models (LLMs), introduced a new API designed for developers working with intricate PDF documents. Mistral OCR is an optical character recognition (OCR) API capable of converting any PDF into a text file, streamlining the process for AI model ingestion.

The Importance of Clean Data for LLMs

LLMs, the foundation of popular generative AI (GenAI) tools like OpenAI’s ChatGPT, function optimally with raw text. Consequently, organizations aiming to establish their own AI workflows recognize the increasing importance of storing and indexing data in a pristine format for effective AI processing and reuse.

Multimodal Capabilities of Mistral OCR

Distinguishing itself from many OCR APIs, Mistral OCR is a multimodal API. This means it can identify and differentiate between illustrations, photographs, and textual content within a document.

The API generates bounding boxes around these graphical elements, incorporating them into the output data.

Markdown Formatting for Enhanced Usability

Mistral OCR doesn’t simply produce a continuous block of text. Instead, the output is structured in Markdown, a formatting syntax commonly used by developers to incorporate links, headings, and other formatting elements into plain text files.

Markdown's Role in the GenAI Landscape

LLMs heavily utilize Markdown for their training datasets. Furthermore, AI assistants, such as Mistral’s Le Chat and OpenAI’s ChatGPT, frequently generate Markdown to create bulleted lists, embed links, or emphasize text with bold formatting.

Assistant applications seamlessly translate Markdown output into a visually rich text format. This explains the growing significance of both raw text and Markdown as GenAI technology continues to expand.

Mistral's Vision for AI Accessibility

“Organizations have amassed a vast collection of documents, often in PDF or slide formats, which are inaccessible to LLMs, particularly Retrieval-Augmented Generation (RAG) systems,” stated Guillaume Lample, Mistral co-founder and chief science officer.

He continued, “With Mistral OCR, our customers can now transform complex and rich documents into readable content across all languages.”

Lample emphasized that this advancement represents a vital step towards broader adoption of AI assistants within companies seeking to simplify access to their extensive internal documentation.

Availability and Deployment Options

Mistral OCR is accessible through Mistral’s dedicated API platform and via its cloud partners, including AWS, Azure, Google Cloud Vertex, and others.

For organizations handling classified or sensitive data, Mistral provides the option of on-premise deployment.

Performance Benchmarks

According to Mistral, its OCR API surpasses the performance of APIs offered by Google, Microsoft, and OpenAI.

The company’s testing involved complex documents containing mathematical expressions (using LaTeX formatting), intricate layouts, and tables. It also demonstrated superior performance with non-English documents.

Speed and Efficiency

Given its specialized focus, Mistral OCR is designed for speed. This is expected when compared to multimodal LLMs like GPT-4o, which incorporate OCR capabilities alongside a multitude of other features.

Integration with Mistral Le Chat

Mistral is currently leveraging Mistral OCR within its own AI assistant, Le Chat. When a user uploads a PDF file, the system utilizes Mistral OCR to analyze the document’s content before processing the text.

Applications with RAG Systems

Companies and developers are likely to integrate Mistral OCR with a RAG (Retrieval-Augmented Generation) system to utilize multimodal documents as input for LLMs.

Potential use cases are numerous, with law firms being a prime example, potentially using the API to efficiently process large volumes of legal documentation.

Understanding RAG

RAG is a technique employed to retrieve data and incorporate it as contextual information within a generative AI model.