Entity resolution is the process of identifying, linking, and unifying records that refer to the same real-world entity across disparate data sources — without a shared primary key. A six-stage pipeline of profiling, standardisation, blocking, fuzzy scoring, entity clustering, and survivorship produces a single authoritative golden record from every fragmented duplicate. When implemented correctly, enterprise teams achieve 93–97% precision on datasets of 10 million records or more.
If your team is merging CRM and ERP data, running KYC checks, or consolidating post-acquisition databases, entity resolution is the foundation every downstream system depends on. Book a demo to see how Match Data Pro resolves entities at scale.
What Entity Resolution Actually Solves
Most enterprise databases contain the same real-world person, company, or address stored as multiple distinct records. A customer named “Robert Johnson” in Salesforce appears as “Bob Johnson” in a legacy ERP. The same company, “Acme Corp.”, is stored as “ACME Corporation” in one system and “Acme Corp” in another. Without entity resolution, every downstream report double-counts, every mailing list fires duplicates, and every 360-degree customer view is incomplete.
The problem compounds across data quality dimensions. Typos, abbreviated names, transposed address numbers, missing suite fields, and inconsistent date formats each create artificial record splits. A single customer can accumulate six or more fragmented records across CRM, ERP, billing, and marketing systems over a typical five-year data lifecycle. Multiply that across 500,000 customers and you have millions of dirty records feeding your analytics stack.
Entity resolution addresses three specific failure modes that exact matching cannot fix:
- Name variants and nicknames: “Jim” vs “James”, “IBM” vs “International Business Machines”
- Address inconsistency: “123 Main St Ste 4” vs “123 Main Street Suite Four”
- Data entry errors: Transposed digits in phone numbers, swapped first/last name fields
The Six-Stage Entity Resolution Pipeline
Every production-grade entity resolution system runs through the same core stages. Skipping any one of them causes precision loss or recall failure at scale.
Stage 1: Data Profiling
Data profiling is the mandatory first step. The platform scans every field in every source dataset and produces a quality report: null rates, cardinality, format violations, value distributions, and cross-field consistency checks. A good profiling pass surfaces problems you did not know existed — a phone field that is 40% null, an email field where 15% of values contain spaces, a company name field where 22% of entries are all-caps. You cannot tune your matching pipeline without this baseline.
Stage 2: Standardisation
Data standardisation normalises every field to a consistent format before any matching logic runs. Name fields get title-cased and common abbreviations expanded. Address fields get parsed into components (street number, street name, unit type, unit number, city, state, postal code) and verified against the CASS-certified postal database. Phone numbers get stripped to ten digits. Company suffixes (“LLC”, “Ltd”, “Incorporated”) get normalised to a canonical form. This stage alone eliminates 30–40% of false negatives that would otherwise appear as non-matches.
Stage 3: Blocking
Comparing every record to every other record produces an O(n²) operation. On a dataset of one million records, that is one trillion comparisons. Blocking reduces this to a manageable candidate set by grouping records that share at least one indexed key — a soundex code, a postal code prefix, or a partial email domain. Only records within the same block are compared. A well-designed blocking strategy retains 99%+ of true matches while cutting the comparison space by 99.9%.
Stage 4: Fuzzy Scoring
Within each block, fuzzy matching algorithms score each candidate pair on a field-by-field basis. A weighted composite score determines whether the pair is a match, a non-match, or a review candidate. Production pipelines combine multiple algorithms:
| Field Type | Algorithm | Match Threshold |
|---|---|---|
| Person name | Jaro-Winkler | ≥ 0.88 |
| Company name | Token Sort Ratio | ≥ 0.82 |
| Address | Levenshtein (normalised) | ≥ 0.90 |
| Phone number | Exact / partial | ≥ 0.95 |
| Exact (domain-split) | 1.00 / 0.80 |
See our guide on deterministic vs probabilistic matching to understand how threshold strategy changes at each confidence tier.
Stage 5: Entity Clustering
High-scoring pairs do not always form clean one-to-one matches. Record A matches B, and B matches C, but A and C score below the threshold. Graph-based entity clustering resolves this transitivity problem. Each record is a node; each match is an edge with a weight equal to the match score. A connected component algorithm groups all nodes in a cluster into a single entity. Senzing entity resolution, integrated into Match Data Pro, applies this graph-based approach natively — resolving identities without hand-crafted rules and updating clusters in real time as new records arrive.
Stage 6: Survivorship and the Golden Record
Once a cluster is identified, survivorship rules select the best field value from each contributing record to build the golden record. Common survivorship strategies include most-recent (the value from the most recently updated record wins), most-frequent (the value appearing across the most source records wins), and source-priority (a CRM field beats an ERP field for customer data). The golden record is then written back to the target system — CRM, data warehouse, or API endpoint — as the single authoritative representation of that entity.
Deterministic vs. Probabilistic Entity Resolution
Every entity resolution system sits on a spectrum between deterministic and probabilistic matching. Understanding the tradeoff is critical to configuration decisions.
Deterministic matching applies rule-based logic: if the email address is identical, the records are the same entity. Rules are fast to execute, fully auditable, and produce zero false positives when the matching field is reliable. The weakness is brittleness. A single typo breaks the match. Rules require manual maintenance as data patterns evolve.
Probabilistic matching assigns a weighted confidence score across multiple fields. No single field is required to match exactly. A 0.92 name score plus a 0.95 address score plus a 0.88 phone score produces a composite that exceeds the match threshold even if no individual field is a perfect string match. This approach captures the matches that deterministic rules miss, at the cost of some false positives that require threshold tuning.
Production entity resolution platforms use both: deterministic rules as a fast pre-pass for high-confidence anchors, then probabilistic scoring for the remainder. Match Data Pro’s configurable entity resolution engine supports both modes, with per-field weight sliders and threshold controls that data engineers can tune without writing code.
Entity Resolution vs. Deduplication: What Is the Difference?
The terms are related but not interchangeable. Deduplication finds and collapses exact or near-exact duplicate records within a single dataset. Entity resolution links records across multiple datasets that may not be duplicates of each other — they are independent records from independent systems that refer to the same underlying entity.
A practical example: your CRM has one record for “Acme Corp.” Your billing system has one record for “ACME Corporation.” Neither system has a duplicate internally. But both records refer to the same customer. Deduplication alone will not surface this. Entity resolution will.
| Capability | Deduplication | Entity Resolution |
|---|---|---|
| Finds duplicates within one dataset | Yes | Yes |
| Links records across multiple systems | No | Yes |
| Handles no shared identifier | Partial | Yes |
| Builds a cross-system golden record | No | Yes |
| Graph-based cluster resolution | No | Yes |
Common Entity Resolution Failure Patterns and How to Fix Them
Most entity resolution failures fall into five categories, each with a known fix.
1. Skipping standardisation before scoring
Comparing “123 Main St” to “123 Main Street” without standardisation produces a Levenshtein score of 0.72 — below a typical 0.85 threshold. After standardisation expands “St” to “Street”, the score is 1.0. Always run standardisation before scoring. Never score raw input.
2. Using only one algorithm per field
Levenshtein handles character transpositions well but fails on token reordering (“John Smith” vs “Smith, John”). Jaro-Winkler handles prefix variation but misses phonetic equivalents (“Katherine” vs “Catherine”). Multi-algorithm scoring — running two or three algorithms and taking the maximum or a weighted average — captures the matches that any single algorithm would miss.
3. Setting thresholds too tight or too loose
A threshold of 0.95 on a name field misses legitimate matches. A threshold of 0.70 produces false positives at scale. Threshold calibration requires a labelled sample: manually reviewed match/non-match pairs drawn from your actual data. Target precision above 90% and recall above 85% before moving to production.
4. Ignoring transitive match clusters
Pairwise matching misses chains: A matches B, B matches C, but A does not directly match C. Without graph-based clustering, all three remain separate entities. The fix is connected-component analysis on the match graph — grouping all transitively linked pairs into a single entity cluster.
5. Not automating re-resolution as data changes
Entity resolution is not a one-off project. New records arrive daily. Old records are updated. An address that previously failed to match may now match after a user corrects a typo. Match Data Pro’s job automation runs re-resolution on a configurable schedule — hourly, nightly, or triggered by incoming data — so your golden records stay current without manual intervention.
Ready to put a production-grade pipeline in place? Start a free trial of Match Data Pro — no contract, no infrastructure to provision, results in hours.
Frequently Asked Questions
What is entity resolution in simple terms?
Entity resolution determines that two or more records in different systems refer to the same real-world person, company, or object — even when the records contain spelling differences, abbreviations, or missing fields. The process links those records and builds a single authoritative golden record that represents the entity across all systems.
How is entity resolution different from record linkage?
Record linkage and entity resolution describe the same core process. Record linkage is the statistical term used in academic literature; entity resolution is the term more commonly used in enterprise software. Both involve finding corresponding records across datasets without a shared identifier, using probabilistic scoring and clustering algorithms to group matching records.
What accuracy can I expect from entity resolution?
Well-configured pipelines on clean, standardised data achieve 93–97% precision and 88–93% recall. Precision measures what fraction of identified matches are correct. Recall measures what fraction of true matches were found. Accuracy depends heavily on data quality going in — profiling and standardisation before matching are the single biggest levers for improving both metrics.
Can entity resolution run in real time or only in batch?
Both modes are possible. Batch resolution processes an entire dataset in a single run — suitable for initial data migration or nightly refresh. Real-time entity resolution uses a live API: each new record is matched against the existing entity graph at point of entry and either assigned to an existing entity or used to create a new one. Match Data Pro’s live fuzzy search API supports real-time entity matching at sub-100ms latency.
Do I need a shared identifier to perform entity resolution?
No. Entity resolution is specifically designed for the case where no shared identifier exists. The pipeline uses composite fuzzy scores across name, address, phone, email, and other available fields to determine match probability. Shared identifiers (such as a national ID or tax number) are used as high-weight anchor fields when available, but they are not required for the system to function.