Handling Long Documents

For long documents, use page ranges and document segmentation to limit extraction to the content you need. Before you begin, make sure you have:

A Datalab account with an API key (new accounts include $5 in free credits)
Python 3.10+ installed
The Datalab SDK: pip install datalab-python-sdk
Your DATALAB_API_KEY environment variable set

Restrict to Specific Pages

If you know which pages contain the data you need, use page_range:

from datalab_sdk import DatalabClient, ExtractOptions
import json

client = DatalabClient()

schema = {
    "type": "object",
    "properties": {
        "executive_summary": {"type": "string", "description": "Executive summary text"}
    }
}

# Only process pages 0-5 (first 6 pages)
options = ExtractOptions(
    page_schema=json.dumps(schema),
    page_range="0-5",
    mode="balanced"
)

result = client.extract("long_document.pdf", options=options)

Page-based charges use the selected pages. Complex extraction requests may also use compute-based pricing; see Billing.

Segment and Chain Extractions

For documents with distinct sections (like financial reports or contracts), extract the table of contents first, then process each section separately.

Step 1: Extract Table of Contents

import json
from datalab_sdk import DatalabClient, ExtractOptions

client = DatalabClient()

toc_schema = {
    "type": "object",
    "properties": {
        "table_of_contents": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "section_name": {"type": "string"},
                    "page_number": {"type": "integer", "description": "Section start page number"}
                }
            }
        }
    }
}

# Extract TOC from first few pages
options = ExtractOptions(
    page_schema=json.dumps(toc_schema),
    page_range="0-5",
    mode="balanced"
)

result = client.extract("report.pdf", options=options)
toc = json.loads(result.extraction_schema_json)

print("Sections found:")
for item in toc["table_of_contents"]:
    print(f"  {item['section_name']}: page {item['page_number']}")

Step 2: Extract Each Section

page_range uses original, 0-indexed PDF pages. Printed table-of-contents numbers can have a different offset or use Roman numerals. Verify and map each extracted page_number to a PDF page index before running these examples. Set total_pages to the actual PDF page count so the final section includes all remaining pages.

# Define schemas for different sections
section_schemas = {
    "Financial Highlights": {
        "type": "object",
        "properties": {
            "revenue": {"type": "number"},
            "net_income": {"type": "number"},
            "eps": {"type": "number"}
        }
    },
    "Risk Factors": {
        "type": "object",
        "properties": {
            "risks": {
                "type": "array",
                "items": {"type": "string"}
            }
        }
    }
}

# Build page ranges from the verified TOC
total_pages = 150  # Replace with the actual number of pages in report.pdf.
sections = toc["table_of_contents"]
results = {}

for i, section in enumerate(sections):
    section_name = section["section_name"]
    start_page = int(section["page_number"])

    # End page is start of next section (or end of document)
    end_page = int(sections[i + 1]["page_number"]) - 1 if i + 1 < len(sections) else total_pages - 1
    if not 0 <= start_page <= end_page < total_pages:
        raise ValueError("Review the TOC page mapping and total_pages before extraction")

    # Get schema for this section if we have one
    schema = section_schemas.get(section_name)
    if schema:
        page_range = f"{start_page}-{end_page}"

        options = ExtractOptions(
            page_schema=json.dumps(schema),
            page_range=page_range,
            mode="balanced"
        )

        result = client.extract("report.pdf", options=options)
        results[section_name] = json.loads(result.extraction_schema_json)

print(results)

Use Document Segmentation

For documents without a clear table of contents, use Document Segmentation to locate named sections in the parsed document.

import json
from datalab_sdk import DatalabClient, SegmentOptions

client = DatalabClient()
segmentation_schema = {
    "segmentation_strategy": "custom",
    "segments": [
        {"name": "introduction", "description": "Introduction and background"},
        {"name": "methods", "description": "Methods and approach"},
        {"name": "results", "description": "Findings and results"},
        {"name": "conclusion", "description": "Summary and conclusions"},
    ],
}
options = SegmentOptions(segmentation_schema=json.dumps(segmentation_schema), mode="balanced")
result = client.segment("paper.pdf", options=options)
for segment in result.segmentation_results["segments"]:
    print(segment["name"], segment["pages"])

curl -X POST https://www.datalab.to/api/v1/segment \
  -H "X-API-Key: $DATALAB_API_KEY" \
  -F "file=@paper.pdf" \
  -F "mode=balanced" \
  -F 'segmentation_schema={"segmentation_strategy":"custom","segments":[{"name":"introduction","description":"Introduction and background"},{"name":"methods","description":"Methods and approach"},{"name":"results","description":"Findings and results"},{"name":"conclusion","description":"Summary and conclusions"}]}'

# Poll request_check_url from the response until status is complete or failed.

Full Example

Process sections after verifying the table of contents and mapping its page numbers to PDF page indices:

import json
from datalab_sdk import DatalabClient, ExtractOptions

client = DatalabClient()


def extract_sections(pdf_path: str, sections: list[dict], section_schemas: dict, total_pages: int) -> dict:
    """Extract from reviewed sections with original, 0-indexed PDF page numbers."""

    # Extract each reviewed section with its schema
    results = {}

    for i, section in enumerate(sections):
        section_name = section["section_name"]
        start_page = int(section["page_number"])

        # Calculate page range
        if i + 1 < len(sections):
            end_page = int(sections[i + 1]["page_number"]) - 1
            page_range = f"{start_page}-{end_page}"
        else:
            end_page = total_pages - 1
            page_range = f"{start_page}-{end_page}"
        if not 0 <= start_page <= end_page < total_pages:
            raise ValueError("Review the TOC page mapping and total_pages before extraction")

        # Check if we have a schema for this section
        schema = section_schemas.get(section_name)
        if not schema:
            continue

        options = ExtractOptions(
            page_schema=json.dumps(schema),
            page_range=page_range,
            mode="balanced"
        )

        try:
            result = client.extract(pdf_path, options=options)
            results[section_name] = json.loads(result.extraction_schema_json)
            print(f"Extracted: {section_name}")
        except Exception as e:
            print(f"Error extracting {section_name}: {e}")

    return results


# Define schemas for sections you care about
schemas = {
    "Financial Highlights": {
        "type": "object",
        "properties": {
            "total_revenue": {"type": "number", "description": "Total revenue"},
            "net_income": {"type": "number", "description": "Net income"},
            "year": {"type": "string", "description": "Fiscal year"}
        }
    },
    "Business Overview": {
        "type": "object",
        "properties": {
            "description": {"type": "string", "description": "Business description"},
            "products": {"type": "array", "items": {"type": "string"}}
        }
    }
}

# Replace with the reviewed PDF page indices and actual total page count.
sections = [
    {"section_name": "Financial Highlights", "page_number": 12},
    {"section_name": "Business Overview", "page_number": 30},
]
results = extract_sections("annual_report.pdf", sections, schemas, total_pages=150)
print(json.dumps(results, indent=2))

Tips

Process pages you need - Use page_range to avoid processing unnecessary pages
Extract TOC first - Build page ranges dynamically from the document structure
Choose extraction and parsing independently - Set extraction_mode for extraction quality and mode for parsing quality; see Extraction Modes
Handle errors - Some sections may not match your schema exactly

Next Steps

Structured Extraction

Learn the full structured extraction API and schema options.

Document Segmentation

Automatically split documents by section headers.

Batch Processing

Process multiple long documents efficiently in parallel.

Pipelines

Chain processors into versioned, reusable pipelines.

General

Document Conversion

Structured Extraction

Document Segmentation

Form Filling

File Management

Pipelines

Create Document

Track Changes

Forge Evals

Restrict to Specific Pages

Segment and Chain Extractions

Step 1: Extract Table of Contents

Step 2: Extract Each Section

Use Document Segmentation

Full Example

Tips

Next Steps

Structured Extraction

Document Segmentation

Batch Processing

Pipelines

​Restrict to Specific Pages

​Segment and Chain Extractions

​Step 1: Extract Table of Contents

​Step 2: Extract Each Section

​Use Document Segmentation

​Full Example

​Tips

​Next Steps

Structured Extraction

Document Segmentation

Batch Processing

Pipelines

Restrict to Specific Pages

Segment and Chain Extractions

Step 1: Extract Table of Contents

Step 2: Extract Each Section

Use Document Segmentation

Full Example

Tips

Next Steps