Fuzzy Data Matching with SQL: Enhancing Data Quality and Query Performance 9781098152277

Table of contents :
Preface
What Problems Are We Trying to Solve?
What Will We Cover?
Part I: Review
Part II: Various Data Problems
Part III: Bringing It Together
Appendix
Who Is This Book For?
Why SQL?
Warning! Opinions Ahead!
Typographical Conventions Used in This Book
Additional Information on the Book’s Conventions
The Data “Model”
Environment Layout
Customer Table
“Normalized” View
Meet the Snedleys
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments
I. Review
1. A SELECT Review
Simple SELECT Statements
Common Table Expressions
In CASE of Emergency
Joins
A Diversion into NULL Values
OUTER JOINs
Finding the Most Current Value
Final Thoughts on SELECT
2. Function Junction
Aggregate Functions
MAX
MIN
COUNT
SUM
AVG
Conversion Functions
CAST and CONVERT
COALESCE
TRY_CONVERT
Cryptographic Functions: HASHBYTES
Date and Time Functions
GETDATE
DATEADD
DATEDIFF
DATEPART
ISDATE
Logical Functions: IIF
String Functions
CHARINDEX and PATINDEX
LEN
LEFT, RIGHT, and SUBSTRING
LTRIM, RTRIM, and TRIM
LOWER and UPPER
REPLACE and TRANSLATE
REVERSE
STRING_AGG
System Functions
ISNULL
ISNUMERIC
Final Thoughts on Functions
II. Various Data Problems
3. Names, Names, Names
What’s in a Name?
Last Names
Punctuation
Suffixes
First Names
Middle Name
Nicknames
Company Name
Full Name
“Person-Like Entities”
Final Thoughts on Names
4. Location, Location, Location
What Makes an Address?
Street Address
Box, Suite, Lot, or Apartment Number
Don’t Overdo It!
City
County
State or State Abbreviation
ZIP or Postal Code
Country
Final Thoughts on Locations
5. Dates, Dates, Dates
Time Is Relative
Final Thoughts on Dates
6. Email
What Makes a Valid Email Address?
Final Thoughts on Email
7. Phone Numbers
What Makes a “Phone Number”?
One Final Note on Tax IDs
Final Thoughts on Phone Numbers (and Tax IDs)
8. Bad Characters
Data Representations
Invisible Whitespace
COLLATE
Cleaning Up the Input Data
Final Thoughts on Bad Characters
9. Orthogonal Data
A Common Problem, A Common Solution, A New Common Problem
Lather, Rinse, Repeat
Final Thoughts on Orthogonal Data
III. Bringing It Together
10. The Big Score
What Will We Want?
Tuning Scores
Eliminating Duplicates
Duplicate Data
Duplicated Data
Final Thoughts on Scoring
11. Data Quality, or GIGO
Sneaking Data Quality In
Impossible Data
Simply Wrong
Semantically Wrong
ETL Your Way to Success
Final Thoughts on Data Quality
12. Tying It All Together
Approach
What’s the Score?
First Pass: Naive Matching
Second Pass: Normalizing Relations
Impossible Data
Now Let’s Normalize
Third Pass: Score!
What About Tuning?
Final Thoughts on Practical Matters
13. Code Is Data, Too!
Working with XML Data
Working with JSON Data
Extracting Data from HTML
Code-Generating Code
Impact Analysis: The Second Case Study
Gather Together Every Code “Artifact” You Can
Import Artifacts into SQL
And Now, for My Next Trick
Final Thoughts on Code As Data
Final Thoughts on All of It
A. The Data “Model”
Customer Table
NormalizedCustomer View
PotentialMatches Table
CustomerCountByState View
PostalAbbreviations Table
Glossary
Index

Polecaj historie

Fuzzy Data Matching with SQL: Enhancing Data Quality and Query Performance [1 ed.] 1098152271, 9781098152277

If you were handed two different but related sets of data, what tools would you use to find the matches? What if all you

217 86 2MB Read more

Fuzzy Data Matching with SQL: Enhancing Data Quality and Query Performance 9781098152277, 1098152271

If you were handed two different but related sets of data, what tools would you use to find the matches? What if all you

154 69 942KB Read more

SQL Server 2017 Query Performance Tuning: Troubleshoot and Optimize Query Performance 9781484238882, 1484238885

Identify and fix causes of poor performance. You will learn Query Store, adaptive execution plans, and automated tuning

1,241 222 33MB Read more

Getting Started with SQL and Databases : Managing and Manipulating Data with SQL 9781484294932, 9781484294925

The book starts with a general introduction to writing SQL and covers the basic concepts. Author Mark Simon then covers

117 86 1016KB Read more

Getting Started with SQL and Databases: Managing and Manipulating Data with SQL 9781484294925, 1484294920

Learn the basics of writing SQL scripts. Using Standard SQL as the starting point, this book teaches writing SQL in vari

110 32 3MB Read more

SQL server 2012 query performance tuning [3rd ed] 9781430242031, 9781430242048

Queries not running fast enough? Tired of the phone calls from frustrated users? Grant Fritchey's book SQL Server 2

132 74 18MB Read more

SQL for Data Analytics: Perform fast and efficient data analysis with the power of SQL 1789807352, 9781789807356

Take your first steps to become a fully qualified data analyst by learning how to explore large relational datasets. Key

2,053 381 22MB Read more

Data Wrangling with SQL: A hands-on guide to manipulating, wrangling, and engineering data using SQL [1 ed.] 9781837630028

Become a data wrangling expert and make well-informed decisions by effectively utilizing and analyzing raw unstructured

2,348 141 4MB Read more

SQL Server 2017 Query Performance Tuning: troubleshoot and optimize query performance [Fifth edition] 9781484238875, 9781484238882, 1484238877, 1484238885

Identify and fix causes of poor performance. You will learn Query Store, adaptive execution plans, and automated tuning

1,001 182 32MB Read more

SQL for Data Analytics: Perform Fast and Efficient Data Analysis with the Power of SQL 1789807352, 9781789807356

Take your first steps to become a fully qualified data analyst by learning how to explore large relational datasets. Key

4,538 1,107 22MB Read more