Fuzzy matching algorithms score the similarity between two strings on a scale of 0 to 100, then route records to auto-match, review, or reject based on configurable thresholds. Choosing the right algorithm for each field type — names, addresses, codes, phone numbers — determines whether your deduplication pipeline achieves 90%+ precision or generates a flood of false positives that erode trust in the output.
This article explains the five core fuzzy matching techniques, how to configure scoring thresholds, and how to combine them into a weighted pipeline that handles real-world data variability at enterprise scale.
Schedule a demo to see how Match Data Pro applies these techniques to your actual data.
Why Algorithm Choice Drives Match Accuracy
No single fuzzy matching algorithm works equally well on every field type. Levenshtein edit distance performs well on short codes and IDs. Jaro-Winkler was designed for personal names, where prefix characters carry more weight. Token-based ratios handle company names and addresses where word order varies. Phonetic encoding catches spelling variations that look different but sound the same.
Applying the wrong algorithm to a field produces one of two failures:
- False positives: Records that share edit-distance closeness but represent different entities. Example: “Smith, J.” matches “Smith, K.” at 92 on Levenshtein score.
- False negatives: Real matches that score below threshold because the algorithm does not normalise for the variation present. Example: “International Business Machines” and “IBM” score 18 on Levenshtein but 94 on token ratio after abbreviation expansion.
The fix is field-type routing: each field class gets the algorithm best suited to its variation patterns, with a composite weighted score deciding the final outcome. Understanding how each fuzzy matching algorithm works is the foundation of building a reliable pipeline.
The Five Core Fuzzy Matching Techniques
1. Levenshtein Edit Distance
Levenshtein counts the minimum number of single-character edits — insertions, deletions, substitutions — needed to transform one string into another. Normalised score: 1 – (edit_distance / max(len_a, len_b)) × 100.
Best for: Product codes, tax IDs, postal codes, short alphanumeric identifiers.
| Registro A | Disco B | Levenshtein Score |
|---|---|---|
| AB123-XY7 | AB123-XZ7 | 89 |
| TX-882-001 | TX-882-011 | 89 |
| PROD-4421 | PROD-442 | 89 |
Weakness: Sensitive to token reordering. “John Michael Smith” scores poorly against “Smith, John Michael” despite being the same person.
2. Similitud de Jaro-Winkler
Jaro-Winkler extends the Jaro metric by adding a prefix bonus: strings that share the same leading characters receive a score boost. This models how people tend to remember and type names — getting the start right, making errors toward the end.
Best for: First names, last names, single-token text fields.
| Registro A | Disco B | Jaro-Winkler Score |
|---|---|---|
| Jonathan | Jonathon | 97 |
| Catherine | Katharine | 84 |
| Stephens | Stevens | 93 |
Weakness: Performs poorly on multi-token strings. “Robert James Brown” vs “Brown, Robert J.” — Jaro-Winkler scores low due to ordering difference.
3. Token Sort and Token Set Ratio
Token-based algorithms split strings into individual tokens (words), sort or set-compare them, and score based on the best token alignment. This handles reordering, missing words, and partial matches that destroy character-level algorithms.
Best for: Company names, full addresses, multi-word product descriptions.
| Registro A | Disco B | Token Set Score |
|---|---|---|
| Acme Corporation Ltd | Acme Corp | 91 |
| General Electric Co | GE Company | 72 |
| Smith, John Michael | John Michael Smith | 100 |
Weakness: Common tokens inflate scores. “The University of Texas” vs “The University of Georgia” scores high because “The University of” matches.
4. Phonetic Encoding
Phonetic algorithms — Soundex, Metaphone, Double Metaphone — convert a string to a code representing how it sounds. Two strings that sound alike produce the same or similar code. This catches spelling variants that look different but are acoustically equivalent.
Best for: Names with common spelling variants, international name transliterations.
| Registro A | Soundex | Disco B | Soundex | Match? |
|---|---|---|---|---|
| Smyth | S530 | Smith | S530 | Sí |
| Fischer | F260 | Fisher | F260 | Sí |
| Knievel | K140 | Neville | N140 | No |
Weakness: Soundex collapses too aggressively for surnames. Double Metaphone is preferred for people data. Phonetics alone produces too many false positives to use as a primary scoring method.
5. N-gram Similarity
N-gram similarity breaks strings into overlapping character substrings of length n (typically 2 or 3) and measures the proportion of shared n-grams. It is robust to arbitrary substring matches — useful when partial abbreviations appear.
Best for: Free-text fields, partial address matching, search-as-you-type applications.
| String A | String B | Shared Bigrams | Puntuación |
|---|---|---|---|
| matchdata | matcdata | ma, at, tc, da, at, ta | 86 |
| address | adress | ad, dr, re, es, ss | 91 |
Match Data Pro’s live fuzzy search API uses n-gram indexing to return ranked candidates in under 50ms on datasets of millions of records.
The Fuzzy Matching Algorithm Pipeline
The diagram below shows how field-type routing, per-field scoring, and threshold-based routing combine in a production pipeline:
Configuring Scoring Thresholds
Thresholds determine how the pipeline routes a record pair after scoring. Getting them wrong is the most common cause of deduplication failure. Too high: real duplicates escape. Too low: unrelated records get merged, destroying data integrity.
Umbrales iniciales recomendados según el caso de uso
| Caso de uso | Auto-Match Threshold | Banda de reseñas | Reject Below |
|---|---|---|---|
| Customer deduplication (CRM) | 88 | 65 to 87 | 65 |
| Vendor / supplier matching | 85 | 60 to 84 | 60 |
| Patient record linkage | 92 | 75 to 91 | 75 |
| Product catalogue matching | 90 | 70 to 89 | 70 |
| Financial entity matching | 93 | 78 to 92 | 78 |
These are starting points, not universal rules. Calibrate thresholds using a sample of known matches and known non-matches from your actual data. Measure precision and recall at several threshold values, then choose the point that best fits your acceptable false-positive and false-negative rate. Understanding the difference between deterministic and probabilistic matching helps set realistic expectations for threshold performance.
The Review Band
Record pairs that fall in the review band — typically 60 to 87 depending on the domain — go to a human review queue. This band is not a failure state; it is a deliberate routing decision. The goal is to send the minimum number of pairs to review while catching the cases where automated confidence is genuinely ambiguous.
Narrowing the review band reduces human workload. Widening it reduces risk of automated errors. Most production pipelines target a review rate of 5 to 15% of candidate pairs.
Building a Weighted Composite Scoring Pipeline
A single-algorithm score on a single field is almost never sufficient for production deduplication. The standard approach is a weighted composite: assign different algorithms to different fields, score each field independently, then combine scores using field weights that reflect each field’s discriminative power.
Example Composite Score Configuration
| Campo | Algoritmo | Peso | Puntuación | Weighted Contribution |
|---|---|---|---|---|
| Apellido | Jaro-Winkler | 30% | 97 | 29.1 |
| Nombre | Jaro-Winkler | 20% | 88 | 17.6 |
| Correo electrónico | Exact + Levenshtein fallback | 25% | 100 | 25.0 |
| Línea 1 de la dirección | Proporción del conjunto de fichas | 15% | 84 | 12.6 |
| Teléfono | Exact + digit-only Levenshtein | 10% | 72 | 7.2 |
| Composite Score | 91.5 |
A composite of 91.5 on a customer deduplication task with an auto-match threshold of 88 triggers an automatic merge with survivorship rules applied. This record pair would have scored below threshold on any single algorithm alone — the composite is what elevates confidence.
Match Data Pro lets you configure field weights, algorithm selection, and thresholds for every matching rule set via its data matching platform, without writing code. AI-powered match suggestions review your data sample and recommend starting weights based on observed field distributions.
Blocking: Scaling Without Brute-Force Comparison
Comparing every record against every other record (full Cartesian product) is O(n²). On a dataset of 1 million records that is 500 billion comparisons. Blocking solves this by partitioning records into candidate buckets based on shared attributes — only records in the same bucket are compared.
Common Blocking Strategies
- Phonetic blocking: Bucket records by Soundex or Double Metaphone of the last name. All “Smith/Smyth/Smithe” variants land in the same bucket.
- First-N character blocking: Bucket by the first 3 characters of a postcode or company name. Fast and effective for well-formatted data.
- Multi-pass blocking: Run multiple blocking keys and union the candidate sets. Increases recall while keeping comparison count manageable.
- Canopy clustering: Use a fast, loose similarity measure (such as n-gram Jaccard) to form canopies, then apply expensive algorithms only within each canopy.
Blocking typically reduces the comparison space by 95 to 99%, enabling fuzzy matching on tens of millions of records within practical run times. Data deduplication in practice explains how blocking fits into a full end-to-end pipeline.
Survivorship Rules: What Happens After a Match
Scoring determines whether two records are the same entity. Survivorship determines which field values to keep in the merged golden record. The two decisions are independent but equally important.
Normas generales sobre la supervivencia:
- Most recent: Keep the value from the record with the latest modified date.
- Most complete: Keep the value from the record with fewer null fields.
- Source priority: CRM beats marketing list; ERP beats spreadsheet. Source trust hierarchy is defined per field.
- Longest non-null: For address fields, the more complete value wins.
- Consensus: Where three or more sources agree, use the consensus value.
Survivorship rules interact with data standardisation — values should be normalised before survivorship runs so that “ST” and “Street” are not treated as different values when choosing which address to retain.
Address Matching: A Special Case
Addresses require preprocessing before any similarity algorithm runs. Raw address fields contain unit suffixes, directional prefixes, abbreviations, and postal code variants that inflate edit distance and suppress true similarity scores.
The correct pipeline for address matching is:
- Parse address into components (street number, street name, suffix, unit, city, state, ZIP).
- Run CASS verification against the USPS database to standardise and append ZIP+4.
- Apply fuzzy scoring (token set ratio on street name, exact on ZIP) on the standardised components.
Match Data Pro’s integrated CASS address verification runs as a preprocessing step before fuzzy scoring, so the algorithms receive clean, normalised components rather than raw free text. This eliminates an entire class of false negatives caused by address formatting inconsistency.
Practical Examples: Before and After
The following examples show how the same record pair scores under a single algorithm vs a weighted composite with preprocessing.
Example 1: Customer Deduplication
| Campo | Registro A | Disco B |
|---|---|---|
| Nombre | Robert J. Williams | Bob Williams |
| Correo electrónico | rwilliams@acme.com | rwilliams@acme.com |
| DIRECCIÓN | 123 Main St, Austin TX 78701 | 123 Main Street, Austin, TX 78701-2211 |
| Teléfono | 512-555-0192 | (512) 555.0192 |
Levenshtein on name alone: 44 (Robert J. Williams vs Bob Williams — large edit distance). Composite with nickname expansion, email exact match, CASS-standardised address token match, and digit-normalised phone: 96.2. Automatic merge triggered.
Example 2: Vendor Matching
| Campo | Registro A | Disco B |
|---|---|---|
| Nombre | International Business Machines Corp | IBM Corporation |
| Tax ID | 13-0871985 | 13-0871985 |
| DIRECCIÓN | 1 New Orchard Rd, Armonk NY 10504 | New Orchard Road, Armonk, New York |
Levenshtein on name: 18. Token set after abbreviation expansion: 89. Tax ID exact match: 100. CASS-standardised address token: 94. Composite: 93.7. Automatic merge triggered. Multi-algorithm weighted scoring is what makes this resolution possible.
Ready to apply these techniques to your data? Start a free trial of Match Data Pro — no contract, no setup fees, and AI-powered match suggestions that recommend algorithm weights based on your actual field distributions.
Preguntas frecuentes
What is a fuzzy matching technique?
A fuzzy matching technique is an algorithm that scores the similarity between two strings on a numeric scale rather than requiring an exact character match. Techniques include Levenshtein edit distance, Jaro-Winkler similarity, token-based ratios, phonetic encoding, and n-gram similarity. Each is suited to different field types and variation patterns.
What threshold should I use for fuzzy matching?
Starting thresholds depend on the domain: 88 for CRM deduplication, 92 for patient records, 85 for vendor matching. Calibrate using a sample of known matches and non-matches from your actual data. Measure precision and recall at several threshold values, then select the point that best balances false positives against false negatives for your use case.
What is the difference between Levenshtein and Jaro-Winkler?
Levenshtein counts the total character edits between two strings — insertions, deletions, and substitutions. Jaro-Winkler extends Jaro similarity with a prefix bonus, making it better for names where the start of the string is more reliable than the end. Use Levenshtein for short codes and IDs; use Jaro-Winkler for personal names and other single-token fields.
Why do I need multiple algorithms in one pipeline?
No single algorithm handles every field type accurately. Personal names need Jaro-Winkler. Company names need token ratio. Codes need Levenshtein. Addresses need CASS preprocessing plus token matching. A weighted composite assigns the right algorithm to each field and combines the per-field scores into a single confidence value. This is the only approach that consistently achieves 90%+ precision on enterprise data.
How does blocking reduce fuzzy matching run time?
Blocking partitions records into candidate buckets based on shared attributes — such as Soundex of the last name or first 3 characters of a postcode — so that expensive similarity algorithms only run on records in the same bucket. This reduces the comparison space by 95 to 99%, cutting run time from days to minutes on large datasets without meaningful recall loss.