Entity Extraction with spaCy

You've got a messy pile of text — emails, support tickets, legal docs — and you need to pull out the people, places, and organizations mentioned in it. Doing that by hand is soul-crushing and slow. That's the pain this lesson kills: entity extraction with spaCy lets you automate named-entity recognition (NER) in a few lines of Python, turning raw text into structured data you can filter, search, or feed into other AI pipelines. By the end of this lesson, you'll not only understand how spaCy's NER engine works, but you'll have written a working extraction script and know exactly when to reach for it versus other approaches.

The problem this lesson solves

When your app processes free-form text, you're drowning in unstructured data. A support ticket contains a customer name, a product ID, a date, and maybe an urgent issue — but they're all glued together in prose. Manually parsing that is impractical at scale, and even simple regex rules fail when language gets creative: "Apple" could be a company or a fruit; "Paris" could be a city or a person's name.

That's where entity extraction (also called named-entity recognition or NER) steps in. NER is a subtask of natural language processing (NLP) that identifies spans of text that refer to real-world entities and classifies them into categories like PERSON, GPE (geopolitical entity, e.g., countries/cities), ORG (organizations), DATE, MONEY, and more. spaCy is a popular Python NLP library that ships with pre-trained models capable of doing this out of the box, with a clean and fast API.

This lesson directly supports the Applied AI engineering track. Before this, you've likely worked with text preprocessing and basic NLP concepts; after this, you'll be ready to integrate entity extraction into retrieval pipelines, chatbots, or data processing systems.

Core concept / mental model

Think of spaCy's NER as a highlighter for meaning. You feed it a sentence, and it paints each recognized entity with a color-coded category. The model doesn't use rule-based patterns (unless you add them); it learns from thousands of labeled examples to predict which tokens form an entity and what type it is.

Here's the mental model to hold onto:

• Entities are spans — contiguous sequences of tokens that together form a single named entity. For example, in "New York City", the three tokens belong to one entity of type GPE.
• NER is a sequence labeling task — spaCy processes text token-by-token and decides for each token whether it starts, continues, or is outside an entity.
• ​Models are language-specific and pipeline-based — spaCy's en_core_web_sm is a compact English model that processes text in a pipeline: sentence-boundary detection, part-of-speech tagging, dependency parsing, and finally NER.

Pro tip: NER in spaCy is not magic — it's a trained statistical model. The quality depends on the training data and the model size. For domain-specific text (like medical records or financial filings), you'll likely need to fine-tune or add custom rules.

Another way to visualize it: a text document is a map, and entities are the landmarks. spaCy's NER tags each landmark with a label — "this is a city", "this is a person" — so you can navigate the text without reading every word.

How it works step by step

Let's break down the process of using spaCy for entity extraction into clear steps:

1. Install spaCy and download a model — You need the library and a language pipeline. The smallest English model is en_core_web_sm, but for better accuracy you might use en_core_web_md or en_core_web_lg (the larger, the more accurate but slower).
2. Load the pipeline — nlp = spacy.load("en_core_web_sm") gives you an nlp object that processes raw text.
3. Process the text — Call doc = nlp("text here"). This runs the entire pipeline and produces a Doc object that holds the tokens, their linguistic features, and the entities.
4. Access entities — Iterate over doc.ents to get each entity span. Each entity has .text (string), .label_ (like "PERSON"), and .start_char / .end_char (character offsets in the original text).
5. Format or store results — Convert entities into a list of dictionaries, JSON, or a Pandas DataFrame for further use.

Understanding the pipeline

The en_core_web_sm pipeline is a series of components: a tok2vec layer that turns words into vectors, a tagger for part-of-speech, a parser for dependencies, and an NER component that identifies entities. When you call nlp(text), all components run in sequence and enrich the Doc.

You can also disable components you don't need to speed things up:

nlp = spacy.load("en_core_web_sm", disable=["parser", "tagger"])

But note: the NER component may rely on features from earlier components for best accuracy, and disabling them can hurt performance. For most tasks, keep the default pipeline.

Entity types you'll see

spaCy's standard model includes a range of entity types, but you'll commonly encounter these:

Hands-on walkthrough

Now let's put theory into practice. You'll write a script that extracts entities from a sample text and outputs them in a clean format.

Step 1: Install and load the model

First, ensure you have spaCy installed and the model downloaded:

pip install spacy
python -m spacy download en_core_web_sm

If you're in a Jupyter notebook, you can also run those with ! prefix.

Step 2: Extract entities from a sample text

Create a Python script or notebook cell:

import spacy

# Load the small English pipeline
nlp = spacy.load("en_core_web_sm")

text = """Microsoft Corporation announced today that CEO Satya Nadella will visit Paris next Monday. The company plans to invest €1.5 million in a new AI research lab."""

doc = nlp(text)

# Iterate over detected entities
for ent in doc.ents:
    print(f"{ent.text:<30} {ent.label_:<10} {ent.start_char:>3}–{ent.end_char:<3}")

Expected output

Microsoft Corporation        ORG         0–21
Satya Nadella                PERSON     45–58
Paris                        GPE        73–78
next Monday                  DATE       87–98
€1.5 million                 MONEY      149–160

Notice how "next Monday" is treated as a single DATE entity, and "€1.5 million" as a MONEY entity. The character offsets let you locate the entity in the original string.

Step 3: Convert to structured data

For real-world use, you'll want a dictionary or JSON output. Here's how to build a list of entity dictionaries:

entities = [
    {"text": ent.text, "label": ent.label_, "start": ent.start_char, "end": ent.end_char}
    for ent in doc.ents
]

import json
print(json.dumps(entities, indent=2))

Expected output

[
  {
    "text": "Microsoft Corporation",
    "label": "ORG",
    "start": 0,
    "end": 21
  },
  {
    "text": "Satya Nadella",
    "label": "PERSON",
    "start": 45,
    "end": 58
  },
  ...
]

Step 4: Visualize entities (optional but useful)

spaCy ships with displacy, a visualizer that renders entities in a browser:

from spacy import displacy

displacy.render(doc, style="ent", jupyter=True)  # in Jupyter
# Or save to HTML: displacy.serve(doc, style="ent")

This is a great debugging aid to see what the model caught and missed.

Compare options / when to choose what

spaCy's built-in NER is not the only way to extract entities. Depending on your use case, you might consider alternatives:

Deciding factor

For most pipeline starters, spaCy's pre-trained NER is the sweet spot: it's fast, deterministic, and good enough to bootstrap. If you need higher accuracy on specialized text, you can either fine-tune spaCy or pair it with an LLM for verification.

Pro tip: Start with spaCy's default model, evaluate its output on a sample of your real data, and only then invest in fine-tuning or an LLM. Better to measure than assume.

Troubleshooting & edge cases

Even seasoned developers hit snags. Here are common problems and how to fix them:

1. Model not found error — If you get OSError: [E050] Can't find model 'en_core_web_sm', it means the model isn't downloaded. Run python -m spacy download en_core_web_sm and make sure you're in the same Python environment.
2. Entities are merged or split incorrectly — The model may group "Paris Hilton" as a PERSON instead of GPE + PERSON, or fail to split "New York" from "New York Times". This is a model limitation. You can fix it by adding rules with nlp.add_pipe("entity_ruler", before="ner") to override patterns for your domain.
3. Misclassification due to ambiguous words — "Apple" is often labeled ORG, but in a fruit context it might be a PRODUCT or just not an entity. The model can't reliably disambiguate without context. Consider using a larger model (en_core_web_lg) or a custom pipeline.
4. Entities are missing entirely — If you expect entities and none appear, the text might be too short or the model might confuse the language. Ensure you're using the right language model (e.g., en_core_web_sm for English). Also check that your input isn't truncated; entities long spans need enough context.
5. Performance issues with large texts — NER on a 10MB document can be slow. Split the text into chunks (paragraphs) and process each with nlp.pipe(texts, batch_size=..) to speed up with parallel processing.

Example: Adding a custom rule to fix a recurring error

Suppose your data has many mentions of "PythonSkillset" and the model doesn't catch it as an entity. Here's how to add a rule:

import spacy
from spacy.pipeline import EntityRuler

nlp = spacy.load("en_core_web_sm")
ruler = nlp.add_pipe("entity_ruler", before="ner")
patterns = [{"label": "ORG", "pattern": "PythonSkillset"}]
ruler.add_patterns(patterns)

doc = nlp("PythonSkillset is the best learning platform.")
for ent in doc.ents:
    print(ent.text, ent.label_)  # Output: PythonSkillset ORG

Now the rule overrides the model and reliably tags PythonSkillset as an ORG.

What you learned & what's next

You've now understand entity extraction with spaCy from the ground up: you know the problem it solves, you have a mental model of how NER works inside spaCy's pipeline, you walked through hands-on extraction steps, compared it to alternatives, and learned how to fix common pitfalls. You can explain the core idea behind entity extraction with spaCy and you've completed a practical exercise that extracts entities and converts them to structured data.

Your next step in the Applied AI engineering track is to integrate entity extraction into a larger application. For example, you might combine it with retrieval-augmented generation where you extract entities from queries to filter your knowledge base, or use it in a data processing pipeline to automatically tag documents. The natural next lesson in this track is likely "Building an entity-based search" or "Fine-tuning spaCy NER for custom domains" — either way, you now have the core skill to make that leap.

Keep practicing by running the script on your own texts, and don't forget to check the official spaCy documentation for more advanced options like training your own model.