Hands-On Entity Resolution: A Practical Guide to Data Matching With Python 9781098148485

Table of contents :
Preface
Who Should Read This Book
Why I Wrote This Book
Navigating This Book
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments
1. Introduction to Entity Resolution
What Is Entity Resolution?
Why Is Entity Resolution Needed?
Main Challenges of Entity Resolution
Lack of Unique Names
Inconsistent Naming Conventions
Data Capture Inconsistencies
Worked Example
Deliberate Obfuscation
Match Permutations
Blind Matching?
The Entity Resolution Process
Data Standardization
Record Blocking
Attribute Comparison
Match Classification
Clustering
Canonicalization
Worked Example
Measuring Performance
Getting Started
2. Data Standardization
Sample Problem
Environment Setup
Acquiring Data
Wikipedia Data
TheyWorkForYou Data
Adding Facebook links
Cleansing Data
Wikipedia
TheyWorkForYou
Attribute Comparison
Constituency
Measuring Performance
Sample Calculation
Summary
3. Text Matching
Edit Distance Matching
Levenshtein Distance
Jaro Similarity
Jaro-Winkler Similarity
Phonetic Matching
Metaphone
Match Rating Approach
Comparing the Techniques
Sample Problem
Full Similarity Comparison
Measuring Performance
Summary
4. Probabilistic Matching
Sample Problem
Single Attribute Match Probability
First Name Match Probability
Last Name Match Probability
Multiple Attribute Match Probability
Probabilistic Models
Bayes’ Theorem
m Value
u Value
Lambda ( λ ) Value
Bayes Factor
Fellegi-Sunter Model
Match Weight
Expectation-Maximization Algorithm
Iteration 1
Iteration 2
Iteration 3
Introducing Splink
Configuring Splink
Splink Performance
Summary
5. Record Blocking
Sample Problem
Data Acquisition
Wikipedia Data
UK Companies House Data
Data Standardization
Wikipedia Data
UK Companies House Data
Record Blocking and Attribute Comparison
Record Blocking with Splink
Attribute Comparison
Match Classification
Measuring Performance
Summary
6. Company Matching
Sample Problem
Data Acquisition
Data Standardization
Companies House Data
Maritime and Coastguard Agency Data
Record Blocking and Attribute Comparison
Match Classification
Measuring Performance
Matching New Entities
Summary
7. Clustering
Simple Exact Match Clustering
Approximate Match Clustering
Sample Problem
Data Acquisition
Data Standardization
Record Blocking and Attribute Comparison
Data Analysis
Expectation-Maximization Blocking Rules
Match Classification and Clustering
Cluster Visualization
Cluster Analysis
Summary
8. Scaling Up on Google Cloud
Google Cloud Setup
Setting Up Project Storage
Creating a Dataproc Cluster
Configuring a Dataproc Cluster
Entity Resolution on Spark
Measuring Performance
Tidy Up!
Summary
9. Cloud Entity Resolution Services
Introduction to BigQuery
Enterprise Knowledge Graph API
Schema Mapping
Reconciliation Job
Result Processing
Entity Reconciliation Python Client
Measuring Performance
Summary
10. Privacy-Preserving Record Linkage
An Introduction to Private Set Intersection
How PSI Works
PSI Protocol Based on ECDH
Bloom Filters
Bloom filter example
Golomb-Coded Sets
GCS example
Example: Using the PSI Process
Environment Setup
Google Cloud setup
Option 1: Prebuilt PSI package
Option 2: Build PSI package
Server install
Server Code
Client Code
Using raw encrypted server values
Using Bloom filter–encoded encrypted server values
Using GCS-encoded encrypted server values
Full MCA and Companies House Sample Example
Summary
11. Further Considerations
Data Considerations
Unstructured Data
Data Quality
Temporal Equivalence
Attribute Comparison
Set Matching
Geocoding Location Matching
Aggregating Comparisons
Post Processing
Graphical Representation
Real-Time Considerations
Performance Evaluation
Pairwise Approach
Cluster-Based Approach
Future of Entity Resolution
Index

Polecaj historie

Practical Data Analysis with Python

4,538 712 2MB Read more

Modern Data Architectures with Python: A practical guide to building and deploying data pipelines, data warehouses & data lakes 9781801070492

Build scalable and reliable data ecosystems using Data Mesh, Databricks Spark, and Kafka Key Features Develop modern da

747 273 9MB Read more

Data Ingestion with Python Cookbook: A practical guide to ingesting, monitoring 9781837632602

Deploy your data ingestion pipeline, orchestrate, and monitor efficiently to prevent loss of data and quality Key Featu

1,623 124 31MB Read more

Python For Data Analysis: A Beginner’s Guide to Learn Data Analysis with Python Programming.

4,363 942 3MB Read more

Practical Data Science with Python 3

4,715 809 14MB Read more

Mastering Machine Learning with Python in Six Steps: A Practical Implementation Guide to Predictive Data Analytics Using Python 9781484228661, 1484228669

Master machine learning with Python in six steps and explore fundamental to advanced topics, all designed to make you a

2,024 240 5MB Read more

Mastering Machine Learning with Python in Six Steps: A Practical Implementation Guide to Predictive Data Analytics Using Python 9781484249475

Explore fundamental to advanced Python 3 topics in six steps, all designed to make you a worthy practitioner. This updat

1,217 142 13MB Read more

Python for Data Science : Clear and Complete Guide to Data Science and Analysis with Python

Are you interested in learning data science with Python? Do you want to know what you need to get started? Then you have

3,661 917 2MB Read more

Python For Beginners: A Practical and Step-by-Step Guide to Programming with Python

The book covers all the basic and fundamental elements of programming in Python, including topics such as computers, pro

606 265 83MB Read more

Python for Data Science: 2 Books in 1. A Practical Beginner’s Guide to learn Python Programming, introducing into Data Analytics, Machine Learning, Web Development, with Hands-on Projects

2,250 303 2MB Read more