Last month, I found myself staring at a 15,000-character video meeting transcript, trying to extract structured data for a sales pipeline. My usual approach — calling LLM APIs directly — was giving me inconsistent results. Same input, different outputs. Production nightmares waiting to happen.
Any data scientist will tell you the same joke: that they spend 80% of the time cleaning data and 20% complaining about cleaning data. Project time estimate: 1 week. Actual time spent cleaning unstructured data and making it usable: 3 months.
The below XKCD comic (character: Cueball) perfectly captures this reality — most of the work isn’t the fancy ML algorithms, it’s the unglamorous data wrangling that makes those algorithms work.
That’s when I stumbled upon Google’s LangExtract library. Another AI tool promising to solve all our unstructured data problems? I was skeptical, but desperate enough to try anything.
What Makes LangExtract Special
1. Exact Source Grounding — No More “Trust Me Bro”
Remember the last time you used an LLM API and wondered, “Where exactly did this information come from in the original text?” LangExtract solves this by mapping every extracted entity back to its precise location in the source document.
2. Smart Chunking — Because Size Does Matter
The library handles the “needle-in-a-haystack” problem that plagues long documents. Instead of throwing everything at the LLM and hoping for the best, LangExtract uses intelligent chunking strategies. It respects sentence boundaries, paragraph breaks, and natural text flow resulting in chunks that actually make sense to the LLM, leading to better extraction quality.
3. Parallel Processing — Speed Without Sacrifice
LangExtract processes multiple chunks simultaneously using the max_workers parameter. You can have up to 10 chunks being processed in parallel, dramatically reducing overall latency without compromising quality.
4. Multiple Extraction Passes — The “Second Opinion” Approach
The library runs multiple extraction passes independently, relying on the LLM’s stochastic nature to catch entities that might be missed in a single run. After all passes complete, results are merged using a “first-pass-wins” rule for conflicts. Perfect for critical applications where accuracy matters more than cost.
5. Interactive Visualization — See What You’re Getting
Instead of staring at raw JSON output, you get an interactive view that shows extracted entities in their original context. It’s like having a highlighter that shows you exactly what was extracted and where.
The API: How It Actually Works
I wanted to extract structured output from an unstructured 15,000 character video meeting transcript which was messy — filled with transcription errors, filler words, and conversational tangents. But buried within were crucial details: product interests, budget constraints, decision timelines, and competitor mentions.
My goal was pretty simple — transform this unstructured mess into a clean JSON structure to find valuable utterances such as — product names, business terms, and key discussion points, all while maintaining traceability back to the original text.
Here’s how I went about it.
First, I installed the library. This was pretty straight-forward:
pip install langextract
Then I described what I wanted LangExtract to do in prompt_description field i.e to extract the exact products, key business terms from the transcript:
# Define what you want to extract
prompt_description = "Extract product names and key business terms from the text"
Then provide few examples to LangExtract on how does a successful extraction looks like by providing a sample text phrase and identifying your specific entity, in my case, products and business terms (very similar concept of a training set for your model):
# Provide examples to guide the extraction
examples = [
data.ExampleData(
text="We discussed the new CRM platform and its AI features.",
extractions=[
data.Extraction(
extraction_class="product_name",
extraction_text="CRM platform",
char_interval=data.CharInterval(start_pos=18, end_pos=30)
),
data.Extraction(
extraction_class="business_term",
extraction_text="AI features",
char_interval=data.CharInterval(start_pos=39, end_pos=49)
)
])
]
Final step is to call the LangExtract API to do the job:
# Extract entities using LangExtract
result = lx.extract(
text_or_documents=transcript_text,
prompt_description=prompt_description,
examples=examples,
model_id="gemini-2.5-flash",
extraction_passes=1, # Just 1 pass for speed
max_workers=4,
max_char_buffer=5000, # Smaller buffer for faster processing
debug=False
)
Here’s the complete code in one go:
import lang_extract as lx
from lang_extract import data
# Define what you want to extract
prompt_description = "Extract product names and key business terms from the text"
# Provide examples to guide the extraction
examples = [
data.ExampleData(
text="We discussed the new CRM platform and its AI features.",
extractions=[
data.Extraction(
extraction_class="product_name",
extraction_text="CRM platform",
char_interval=data.CharInterval(start_pos=18, end_pos=30)
),
data.Extraction(
extraction_class="business_term",
extraction_text="AI features",
char_interval=data.CharInterval(start_pos=39, end_pos=49)
)
]
)
]
# Extract entities using LangExtract
result = lx.extract(
text_or_documents=transcript_text,
prompt_description=prompt_description,
examples=examples,
model_id="gemini-2.5-flash",
extraction_passes=1, # Reduced to 1 pass for speed
max_workers=4, # Reduced workers
max_char_buffer=5000, # Smaller buffer for faster processing
debug=False
)
When I ran this extraction for the video meeting transcript in text format, the result was a JSONL file with list of extractions matching the schema / structured output format I had specified in the examples.
The JSONL output revealed that LangExtract successfully extracted 67 distinct entities from the 45-minute conversation, ranging from company names like “TechFlow Solutions” to specific business terms like “B2B SaaS company” and “case studies”.
{
"extractions": [
{
"extraction_class": "company_name",
"extraction_text": "TechFlow Solutions",
"char_interval": {
"start_pos": 52,
"end_pos": 70
},
"alignment_status": "match_exact",
"extraction_index": 2,
"group_index": 1,
"description": null,
"attributes": {}
},
{
"extraction_class": "business_term",
"extraction_text": "B2B SaaS company",
"char_interval": {
"start_pos": 451,
"end_pos": 467
},
"alignment_status": "match_exact",
"extraction_index": 3,
"group_index": 2,
"description": null,
"attributes": {}
},
........
{
"extraction_class": "business_term",
"extraction_text": "case studies",
"char_interval": {
"start_pos": 9426,
"end_pos": 9438
},
"alignment_status": "match_exact",
"extraction_index": 7,
"group_index": 6,
"description": null,
"attributes": {}
}
],
"text": "Google Meet Transcript\nMeeting: CoolForce Demo with TechFlow Solutions\nTranscript generated by Google Meet\nConfidence level: 95%\nWords transcribed: 1,247\nSpeakers detected: 2\nRecording quality: Good\n",
"document_id": "doc_4ddcbb4e"
}
Lets look at a specific snippet of input text and compare it with the extracted JSON structured output below.
Snippet of the text input:
[00:00:05] Sarah Johnson: Hi Mike, thanks for joining us today. I'm Sarah Johnson, Senior Account Executive here at CoolForce. How are you doing?
[00:00:12] Mike Chen: Hey Sarah, doing great thanks. Yeah, I'm Mike Chen, I'm the VP of Sales over at TechFlow Solutions. We're a B2B SaaS company, about one fifty employees, been around for about 8 years now.
Snippet of the corresponding JSON output:
{
"extraction_class": "company_name",
"extraction_text": "CoolForce",
"char_interval": {
"start_pos": 32,
"end_pos": 41
},
"alignment_status": "match_exact",
"extraction_index": 1,
"group_index": 0,
"description": null,
"attributes": {}
},
{
"extraction_class": "company_name",
"extraction_text": "TechFlow Solutions",
"char_interval": {
"start_pos": 52,
"end_pos": 70
},
"alignment_status": "match_exact",
"extraction_index": 2,
"group_index": 1,
"description": null,
"attributes": {}
}
Here’s my favorite feature of LangExtract: interactive visualization of output. The API to generate visualization — visualize() is also pretty easy to use, just pass in the JSONL output file:
# Save the extraction results for visualization
lx.io.save_annotated_documents([result], output_name="full_entity_extractions.jsonl", output_dir=".")
# Generate interactive HTML visualization
html_content = lx.visualize("full_entity_extractions.jsonl")
with open("full_entity_visualization.html", "w") as f:
f.write(html_content)
… and viola!
So, what does LangExtract do well?
1. Exact Source Grounding
This is where LangExtract shines. Every extracted entity comes with exact character positions in the source text, as promised. The beauty is in the char_interval (refer code above for reference) — every extraction comes with exact character positions. For example, when I needed to trace back a particular entity to verify it wasn’t a transcription error, I could pinpoint it to character position 1,247. That level of traceability is gold for production systems! LangExtract eliminates the hallucination problem that plagues direct LLM API calls.
However, I also noticed some interesting patterns: the library excelled at identifying product names and established business terminology, but occasionally struggled with industry-specific acronyms like “S D R” (Sales Development Representative) and “C P Q” (Configure, Price, Quote). The extraction quality varied noticeably depending on the context — terms mentioned in clear, structured sentences were captured perfectly, while those buried in conversational tangents or transcription errors sometimes produced “match_fuzzy” results instead of “match_exact.”
{
"extraction_class": "product_name",
"extraction_text": "HubSpot",
"char_interval": {
"start_pos": 6653,
"end_pos": 6660
},
"alignment_status": "match_fuzzy",
"extraction_index": 1,
"group_index": 0,
"description": null,
"attributes": {}
},
{
"extraction_class": "product_name",
"extraction_text": "HubSpot",
"char_interval": {
"start_pos": 6653,
"end_pos": 6660
},
"alignment_status": "match_fuzzy",
"extraction_index": 1,
"group_index": 0,
"description": null,
"attributes": {}
},
2. Consistent Schema Output
Direct LLM API calls are like playing Russian roulette with your data structure. LangExtract enforces the schema religiously. Define a JSON structure once, and you get that exact structure every time. No more parsing variations of “sometimes it’s a list, sometimes it’s a dict.” In the above example, every 67 extractions returned the same JSON output structure matching the schema I had defined. This makes the downstream data processing predictable and reliable in production systems.
3. Multiple Extraction Passes
LangExtract lets you control extraction passes through the extraction_passes parameter. Want to be thorough? Increase the passes. Need speed? Reduce them. This level of control over recall vs. speed is not trivial to implement when LLM APIs are used directly.
# High recall, slower processing
result_high_recall = lx.extract(
text_or_documents=transcript_text,
prompt_description=prompt_description,
examples=examples,
model_id="gemini-2.5-flash",
extraction_passes=3, # Multiple passes for thorough extraction
max_workers=8,
max_char_buffer=2000
)
# Fast processing, lower recall
result_fast = lx.extract(
text_or_documents=transcript_text,
prompt_description=prompt_description,
examples=examples,
model_id="gemini-2.5-flash",
extraction_passes=1, # Single pass for speed
max_workers=8,
max_char_buffer=2000
)
4. Interactive Visualization
This feature caught me off guard (pleasant surprise!) as its perfect for debugging and validation. I could instantly see which parts of my transcript were being processed and what was being extracted. No more black-box explanations. Another feather in the cap for easy human evaluation.
5. Open Source Advantage
Unlike many Google tools that are locked behind enterprise walls, LangExtract is open source. You can inspect the code, contribute improvements, and understand exactly what’s happening under the hood. This transparency builds trust, especially when dealing with sensitive data.
The Verdict: Should You Use It?
For production systems that need reliable, traceable, unstructured to structured extraction? Absolutely. LangExtract solves extraction problems that direct API calls don’t address well.
LangExtract is the most production-ready unstructured to structured extraction library I’ve used. It treats extraction as an engineering problem, not just an AI problem. That mindset shift alone makes it valuable.
The library forces you to think about data quality, traceability, and consistency — things that matter in production but are often overlooked in AI demos. For that reason alone, it’s earned a permanent place in my toolkit!
—
Disclaimer: The content and opinions expressed in this blog post are entirely my own and are presented for informational purposes only. The project described herein was undertaken independently and does not reflect, represent, or relate to any work, initiatives, products, or strategies of my current or past employers. No portion of this post should be construed as being affiliated with, endorsed by, or a part of my professional responsibilities or organizational activities in any capacity.