Database matching connects records that describe the same real-world entity across two or more separate databases — without a shared primary key. The process uses fuzzy scoring, blocking, standardisation, and survivorship rules to produce a single authoritative record per entity. When done correctly, it eliminates duplicates, resolves conflicts, and gives every downstream system a clean, unified view of customers, vendors, or products.
Running four databases without matching them is the norm in most organisations. CRM holds one version of a customer. The ERP holds another. Marketing ops adds a third. Support ticketing adds a fourth. None of them agree on the company name, address format, or even the contact’s name. The result: fragmented analytics, misdirected outreach, and operational errors that compound over time.
Try Match Data Pro free — no contract, no setup fees. Connect your databases and run your first match job in minutes.
Why Exact Joins Fail Across Databases
SQL joins on customer ID or email work when both systems share the same identifier. In the real world, they almost never do. Consider these two records for the same company pulled from a CRM and an ERP:
| Campo | CRM Record | ERP Record |
|---|---|---|
| Empresa | Acme Corp. | ACME Corporation |
| DIRECCIÓN | Calle Main, 123, local 4 | Calle Main, 123, oficina 4 |
| Teléfono | 800-555-0100 | (800) 555 0100 |
| Contacto | J. Smith | John Smith |
An exact join returns zero matches. A fuzzy matching engine scores each field pair and returns a composite score of 0.91 — well above the 0.85 auto-match threshold. The records merge into a single golden entity.
The core problem with exact joins is that real-world data has four failure modes across databases: format variation (St vs Street), case variation (ACME vs Acme), abbreviation (Corp vs Corporation), and transcription errors (Smtih vs Smith). None of those fail fuzzy matching when the algorithms and weights are configured correctly.
The Six-Stage Database Matching Pipeline
A production-grade database matching pipeline runs through six defined stages. Skipping any one of them increases false positives, false negatives, or both.
Fase 1: Perfilado de datos
Before matching, profile every source database. Measure null rates per field, format distributions, and uniqueness scores. A field with 40% nulls cannot anchor a match — it needs lower weight or a fallback algorithm. AI-powered data profiling in Match Data Pro automates this scan and surfaces issues before the first match job runs.
Fase 2: Normalización
Normalise every field that will participate in matching. Company names: strip legal suffixes (LLC, Inc, Ltd), uppercase, remove punctuation. Addresses: parse into components (street number, street name, unit, city, state, ZIP). Phones: strip formatting to E.164. Dates: ISO 8601. Data standardisation at this stage reduces match error rates by 30 to 50 percent before any algorithm runs.
Etapa 3: Bloqueo
Comparing every record in database A against every record in database B is O(n²). At one million records per side, that is one trillion comparisons — computationally infeasible. Blocking reduces candidate pairs by grouping records that share a key token: the first three characters of a standardised company name, the ZIP code, or the email domain. Match Data Pro’s blocking engine typically reduces candidate pairs by 95 to 99 percent while retaining 99.5 percent of true matches.
Etapa 4: Puntuación difusa
Each candidate pair receives a composite score across multiple fields and algorithms. A typical configuration for company records:
| Campo | Algoritmo | Peso |
|---|---|---|
| Nombre de empresa | Jaro-Winkler + token sort | 0.35 |
| Línea 1 de la dirección | Normalizado según Levenshtein | 0.25 |
| City + ZIP | Exact / phonetic fallback | 0.20 |
| Teléfono | Exacto (normalizado) | 0.12 |
| Contact name | Jaro-Winkler | 0.08 |
The resulting composite score routes each pair: 0.85 and above auto-matches, 0.65 to 0.84 enters a human review queue, below 0.65 stays as distinct records. These thresholds are configurable per project. Learn more about fuzzy matching algorithms and how they combine into composite scores.
Stage 5: Survivorship
When two records match, which field values survive into the golden record? Survivorship rules define this explicitly. Common strategies: most-complete (longest non-null value wins), most-recent (latest modification date wins), most-trusted (CRM values override ERP for customer fields, ERP wins for financial fields). Match Data Pro supports configurable field-level survivorship across any number of source systems. See survivorship rules and golden records for implementation patterns.
Etapa 6: Producción del «Disco de Oro»
The surviving field values merge into a single authoritative record — the golden record. Every source record that contributed to it retains a link back, providing a full audit trail of which source supplied which field value. This lineage is critical for regulatory compliance and data governance. Learn how golden records in MDM maintain persistent identity across systems.
Common Failure Patterns in Multi-Database Matching
Matching Before Standardising
Running fuzzy algorithms on raw data is the single most common cause of low match rates. “123 Main St Ste 4” and “123 Main Street Suite 4” score 0.61 without normalisation — below the auto-match threshold. After parsing and normalising, they score 0.98. Always standardise first.
Ignoring Blocking Key Design
A poorly designed blocking key creates either too many or too few candidate pairs. Using only ZIP code as the key misses records where a ZIP was entered incorrectly. Using full company name misses abbreviations. The fix: use multiple overlapping blocking keys and take the union of candidate sets. This is sometimes called “multipass blocking.”
Single-Algorithm Scoring
Levenshtein alone scores “Robert Smith” vs “Bob Smith” at 0.38 — a miss. Adding a phonetic algorithm and a nickname expansion table brings the composite to 0.79. Multi-algorithm weighted scoring is not optional in production database matching — it is a baseline requirement.
No Address Verification Step
Two records can share the same customer but have different addresses — one correct, one stale. Without CASS address verification, neither address can be trusted as the canonical value. Match Data Pro’s CASS-certified address verification resolves addresses to the USPS standardised form before any matching runs.
Database Matching at Scale: Architecture Decisions
Batch vs Real-Time
Initial database consolidation projects run as batch jobs: ingest all records, profile, standardise, block, score, merge, and output. Ongoing synchronisation — as new records arrive from any source — requires a real-time or near-real-time layer. Match Data Pro’s live fuzzy search API handles point-of-entry matching so new records are checked against the master database before they create duplicates.
Cross-System Entity Resolution
When matching spans more than two databases — CRM, ERP, marketing automation, support, and a third-party data provider — graph-based entity resolution becomes necessary. Each record becomes a node. Each confirmed match becomes an edge. Connected components in the graph represent the same real-world entity. Match Data Pro integrates Senzing’s graph-based engine for exactly this scenario. See entity resolution and data linkage for the architecture.
Automatización de trabajos
One-off database matching projects deliver short-term value. Scheduled, automated match jobs deliver sustained data quality. Match Data Pro’s job automation layer runs matching pipelines on defined schedules — nightly, weekly, or triggered by new data ingestion — without manual intervention. Full run logs and match statistics are available for every job.
What to Measure After Database Matching
Three metrics define whether a database matching project succeeded:
- Precision: Of all auto-matched pairs, what percentage are correct? Target 95%+. Lower precision means over-matching — distinct entities merged incorrectly.
- Recall: Of all true duplicate pairs, what percentage did the system find? Target 90%+. Lower recall means under-matching — real duplicates left unresolved.
- Review queue rate: What percentage of pairs land in the human review queue? Above 15% usually means the thresholds or algorithms need tuning.
After a Match Data Pro implementation, most teams see precision above 96% and recall above 91% on well-standardised data. Review queue rates typically settle between 3 and 8 percent.
Ready to connect your siloed databases and build a clean master record? Start your free trial or book a demo to see the full pipeline on your own data.
Preguntas frecuentes
What is database matching?
Database matching is the process of identifying records across two or more databases that refer to the same real-world entity — a customer, vendor, or product — and linking or merging them into a single authoritative record. It uses fuzzy scoring algorithms, blocking, and survivorship rules to handle the format inconsistencies that prevent exact joins from working.
How is database matching different from a SQL JOIN?
A SQL JOIN requires a shared exact identifier — a primary key or a perfectly formatted field. Database matching works without a shared key by scoring similarity across multiple fields using algorithms like Levenshtein, Jaro-Winkler, and phonetic matching. It can link “Acme Corp.” with “ACME Corporation” where a JOIN would return zero matches.
How long does a database matching project take?
An initial batch match across two databases of up to 500,000 records typically completes in one to three days: one day for profiling and standardisation, one day for algorithm configuration and test runs, and one day for survivorship rule setup and golden record output. Match Data Pro’s automation layer can compress this to hours on clean data.
What match rate should I expect across siloed databases?
On well-standardised data, a configured multi-algorithm pipeline typically achieves 90 to 96 percent recall — finding 9 out of every 10 true duplicates — with precision above 95 percent. Raw, unstandardised data will score lower until the standardisation stage is tuned. Running a data quality profile before matching is the fastest way to identify and fix the fields holding match rates down.
Can database matching work on international data?
Yes. International records introduce additional challenges: character sets (accented characters, transliteration), address formats that differ by country, and name ordering conventions. Match Data Pro handles Unicode normalisation, international address parsing, and configurable blocking keys that work across different country formats. CASS address verification applies to US addresses; international standardisation rules apply globally.