Programming Microsoft SQL Server 2012 9780735658226, 0735658226

Table of contents :
Introduction......Page 23
Acknowledgements......Page 39
Part I: Core SQL Server Development......Page 41
Chapter 1: Introducing SQL Server Data Tools......Page 43
Database Tooling Designed for Developers......Page 44
Declarative, Model-Based Development......Page 45
Connected Development......Page 46
Disconnected Development......Page 47
Versioning and Snapshots......Page 48
Working with SSDT......Page 49
Connecting with SQL Server Object Explorer......Page 50
Gathering New Requirements......Page 56
Using the Table Designer (Connected)......Page 57
Working Offline with a SQL Server Database Project......Page 62
Using the Table Designer (Offline Database Project)......Page 65
Introducing LocalDB......Page 67
Refactoring the Database......Page 71
Testing and Debugging......Page 73
Comparing Schemas......Page 75
Publishing to SQL Azure......Page 79
Adopting SSDT......Page 82
Summary......Page 83
Chapter 2: T-SQL Enhancements......Page 85
More Than Just Another Temporary Table Solution......Page 86
Submitting Orders......Page 87
Using TVPs for Bulk Inserts and Updates......Page 89
Passing TVPs Using ADO.NET......Page 92
Passing Collections to TVPs Using Custom Iterators......Page 94
TVP Limitations......Page 97
More Portable Dates and Times......Page 98
Time Zone Awareness......Page 99
Date and Time Accuracy, Storage, and Format......Page 100
Date and Time Functions......Page 102
The MERGE Statement......Page 105
Defining the Merge Source and Target......Page 107
The WHEN MATCHED Clause......Page 108
The WHEN NOT MATCHED BY TARGET Clause......Page 109
Using MERGE for Table Replication......Page 110
The WHEN NOT MATCHED BY SOURCE Clause......Page 111
MERGE Output......Page 113
Choosing a Join Method......Page 114
MERGE DML Behavior......Page 115
The INSERT OVER DML Syntax......Page 116
A Filterable Alternative to OUTPUT…INTO......Page 117
Consuming CHANGES......Page 120
The GROUPING SETS Operator......Page 123
Rolling Up by Level......Page 125
Rolling Up All Level Combinations......Page 126
Returning Just the Top Level......Page 128
Mixing and Matching......Page 129
Handling NULL Values......Page 130
Windowing (OVER Clause) Enhancements......Page 133
Sliding Aggregations......Page 136
New T-SQL Functions in SQL Server 2012......Page 137
New Analytic Functions......Page 138
New Conversion Functions......Page 143
New Date and Time Functions......Page 144
New Logical Functions......Page 146
New String Functions......Page 147
The THROW Statement......Page 149
Re-Throwing Exceptions......Page 150
Comparing THROW and RAISERROR......Page 151
Using ROW_NUMBER......Page 153
Using OFFSET/FETCH NEXT......Page 154
The SEQUENCE Object......Page 155
Sequence Limitations......Page 157
Metadata Discovery......Page 158
Summary......Page 162
Chapter 3: Exploring SQL CLR......Page 165
Visual Studio/SQL Server Integration......Page 166
SQL Server Database Projects in Visual Studio......Page 167
SQL CLR Code Attributes......Page 169
Your First SQL CLR Stored Procedure......Page 170
CLR Stored Procedures and Server-Side Data Access......Page 172
Piping Data with SqlDataRecord and SqlMetaData......Page 174
Deployment......Page 176
Getting Ready......Page 177
Deploying Your Assembly......Page 178
Deploying Your Stored Procedures......Page 181
Testing Your Stored Procedures......Page 182
CLR Functions......Page 183
CLR Triggers......Page 188
CLR Aggregates......Page 191
SQL CLR Types......Page 196
Security......Page 201
Examining and Managing CLR Types in a Database......Page 202
Summary......Page 208
Chapter 4: Working with Transactions......Page 209
Understanding the ACID Properties......Page 210
Local Transaction Support in SQL Server......Page 212
Explicit Transaction Mode......Page 213
Batch-Scoped Transaction Mode......Page 216
Read Uncommitted Isolation Level......Page 219
Read Committed Isolation Level......Page 221
Snapshot Isolation Level......Page 222
Read Committed Snapshot Isolation Level......Page 223
Isolation Levels in ADO.NET......Page 224
Distributed Transaction Terminology......Page 226
Rules and Methods of Enlistment......Page 227
Distributed Transactions in SQL Server......Page 229
Distributed Transactions in the .NET Framework......Page 230
Using a Resource Manager in a Successful Transaction......Page 238
Transactions in SQL CLR (CLR Integration)......Page 241
Putting It All Together......Page 244
Summary......Page 246
Chapter 5: SQL Server Security......Page 247
Secure Communications......Page 248
SQL Server Security Overview......Page 249
SQL Server Logins......Page 250
Database Users......Page 251
The guest User Account......Page 252
How Clients Establish a Connection......Page 253
Password Policies......Page 255
User-Schema Separation......Page 256
Execution Context......Page 258
Encryption Support......Page 262
Encrypting Data on the Move......Page 263
Encrypting Data at Rest......Page 264
Transparent Data Encryption......Page 269
SQL Server Audit......Page 274
Creating an Audit Object......Page 275
Auditing Options......Page 276
Recording Audits to the File System......Page 278
Auditing Server Events......Page 279
Auditing Database Events......Page 280
Viewing Audited Events......Page 282
Partially Contained Databases......Page 284
Creating a Contained User......Page 285
Other Partially Contained Database Features......Page 286
SQL Server Browser Service......Page 289
Intelligent Observation......Page 290
Summary......Page 291
Part II: Going Beyond Relational......Page 293
Chapter 6: XML and the Relational Database......Page 295
Character Data as XML......Page 296
Working with the xml Data Type as a Variable......Page 297
Working with XML in Tables......Page 298
XML Schema Definitions (XSDs)......Page 299
XML Indexes......Page 306
FOR XML Commands......Page 308
FOR XML AUTO......Page 309
FOR XML EXPLICIT......Page 311
The TYPE Option......Page 316
FOR XML PATH......Page 317
Emitting a ROOT Element......Page 320
Producing an Inline XSD Schema......Page 321
Producing Element-Based XML......Page 322
Shredding XML Using OPENXML......Page 324
Understanding XQuery Expressions and XPath......Page 325
SQL Server XQuery in Action......Page 328
XML DML......Page 336
Summary......Page 338
Chapter 7: Hierarchical Data and the Relational Database......Page 339
The hierarchyid Data Type......Page 340
Creating a Hierarchical Table......Page 341
The GetLevel Method......Page 342
The GetRoot Method......Page 343
The GetDescendant Method......Page 344
The ToString Method......Page 345
The GetAncestor Method......Page 350
Hierarchical Table Indexing Strategies......Page 353
Breadth-First Indexing......Page 354
The IsDescendantOf Method......Page 355
Reordering Nodes within the Hierarchy......Page 357
The GetReparentedValue Method......Page 358
Transplanting Subtrees......Page 359
More hierarchyid Methods......Page 361
Summary......Page 362
Traditional BLOB Strategies......Page 363
BLOBs in the File System......Page 364
Introducing FILESTREAM......Page 365
Enabling FILESTREAM for the Machine......Page 366
Enabling FILESTREAM for the Server Instance......Page 368
Creating a FILESTREAM-Enabled Database......Page 369
Creating a Table with FILESTREAM Columns......Page 370
Storing and Retrieving FILESTREAM Data......Page 371
Deleting FILESTREAM Data......Page 374
Understanding SqlFileStream......Page 375
Building the Windows Forms Client......Page 377
Programming SqlFileStream Data Access......Page 378
Creating a Streaming HTTP Service......Page 388
Building a WPF Client......Page 392
FILESTREAM Limitations and Considerations......Page 395
Introducing FileTable......Page 397
Creating a FileTable......Page 400
Manipulating a FileTable......Page 402
Searching Documents......Page 405
Summary......Page 406
SQL Server Spaces Out......Page 407
Geodetic (Ellipsoidal Sphere) Model......Page 408
Importing Well-Known Text (WKT)......Page 410
Importing WKB......Page 413
Spatial Data Types......Page 414
Working with geometry......Page 415
Working with geography......Page 428
Spatial Enhancements in SQL Server 2012......Page 440
New Spatial Data Classes......Page 441
New Spatial Methods......Page 445
Other Enhancements......Page 451
Integrating with Microsoft Bing Maps......Page 453
Summary......Page 463
Part III: Applied SQL......Page 465
.NET Data Access Evolution......Page 467
Preparing the Sample Database......Page 470
Monitoring Database Activity with SQL Server Profiler......Page 475
Using the Raw Data Access Objects......Page 476
Working with DataSets......Page 495
Language-Integrated Query (LINQ)......Page 512
LINQ to DataSet......Page 513
Object Relational Modeling (ORM) Comes to .NET......Page 517
LINQ to SQL: Then and Now......Page 519
Entity Framework: Now and in the Future......Page 522
Summary......Page 548
Defining Services......Page 549
WCF Data Access Options......Page 550
WCF Data Services......Page 551
Building a WCF Data Service......Page 552
Creating the Entity Data Model......Page 553
Testing WCF Data Services with Internet Explorer......Page 555
Building Client Applications for WCF Data Services......Page 558
Extending WCF Data Services......Page 584
WCF RIA Services......Page 588
Establishing a WCF RIA Services Link......Page 589
Creating the Entity Data Model......Page 591
Building the Domain Service and Metadata Classes......Page 592
Building the Silverlight Client......Page 601
Testing the Complete WCF RIA Services Solution......Page 609
Making the Right WCF Data Access Choice......Page 617
Summary......Page 618
Chapter 12: Moving to the Cloud with......Page 619
But What Is SQL Azure?......Page 621
Why the Limitations?......Page 622
The First One’s Free......Page 623
Getting Set Up......Page 624
Beyond the Prerequisites......Page 625
Provisioning Your Server......Page 626
Managing Your Database......Page 629
Creating Tables and Entering Data......Page 630
Index Design......Page 632
Management and Visualizations......Page 633
Connecting from Down Below......Page 636
Migrating and Syncing Between Earth and Cloud......Page 639
Extract, Deploy, Export, and Import DAC files......Page 640
Scenarios......Page 642
A SQL Azure Federations Lexicon......Page 647
Creating a Federation......Page 648
Federated Tables......Page 649
Central Tables and Reference Tables......Page 650
Federations Support in SSMS and SSDT......Page 651
SQL Azure Reporting......Page 652
Provisioning......Page 653
Report Authoring......Page 654
Deploying Reports......Page 655
Summary......Page 657
Chapter 13: SQL Azure Data Sync and......Page 659
Data Management......Page 660
Capabilities and Features......Page 661
Data Sync Terminology......Page 662
Sync Groups......Page 663
The Client Sync Agent......Page 664
SQL Azure Data Sync Considerations......Page 665
Creating an Occasionally Connected System......Page 666
Prerequisites......Page 669
Provisioning the SQL Azure Data Sync Server......Page 670
Creating the Sync Group......Page 671
About Windows Azure......Page 681
Creating the FlixPoll Solution......Page 682
Adding the FlixPoll Data Service......Page 683
Adding the Entity Data Model......Page 684
Creating the FlixPoll Client......Page 687
Consuming OData on Windows Phone......Page 702
SQL Server on the Phone......Page 706
Deploying to Windows Azure......Page 712
Summary......Page 714
Chapter 14: Pervasive Insight......Page 715
The Microsoft BI Stack: What’s It All About?......Page 716
Master Data Services......Page 717
Data Quality Services......Page 720
Integration Services......Page 721
Data Marts and Data Warehouses......Page 723
SQL Server Data Warehouse Appliances......Page 724
The Multidimensional Engine......Page 726
PowerPivot and SSAS Tabular Mode......Page 727
Data Mining......Page 730
Power View......Page 731
Reporting Services......Page 732
Alerting......Page 733
Using Excel Services......Page 734
PerformancePoint Services......Page 736
SQL Server Editions and SharePoint Version Requirements......Page 737
Summary......Page 739
Chapter 15: xVelocity In-Memory Technologies......Page 741
Column Store Databases......Page 742
Column Store Tech in the BI Industry......Page 743
What You Can’t Do......Page 744
How Columnstore Indexes Work......Page 746
xVelocity for Analysis: PowerPivot and SSAS Tabular Models......Page 749
Clearing Up the Analysis Services Vocabulary......Page 750
Friends, Countrymen, Bring Me Your Data......Page 751
Building the BISM......Page 752
Dial M for Modeling......Page 755
Modeling, Part Deux......Page 758
Querying in Excel......Page 764
PowerPivot for SharePoint......Page 766
Moving to SSAS Tabular......Page 767
Power View Here We Come......Page 772
Welcome Back to VertiPaq......Page 774
Summary......Page 775
Index......Page 777
About the Authors......Page 813

Citation preview

Programming Microsoft SQL Server 2012 ®

Leonard Lobel Andrew Brust

®

Pub shed w th the author zat on of M crosoft Corporat on by O’Re y Med a, Inc 1005 Gravenste n H ghway North Sebastopo , Ca forn a 95472 Copyr ght © 2012 by S eek Techno og es Inc , and B ue Badge Ins ghts, Inc A r ghts reserved No part of the contents of th s book may be reproduced or transm tted n any form or by any means w thout the wr tten perm ss on of the pub sher ISBN 978-0-7356-5822-6 123456789 M 765432 Pr nted and bound n the Un ted States of Amer ca M crosoft Press books are ava ab e through bookse ers and d str butors wor dw de If you need support re ated to th s book, ema M crosoft Press Book Support at msp [email protected] crosoft com P ease te us what you th nk of th s book at http //www m crosoft com/ earn ng/booksurvey M crosoft and the trademarks sted at http //www m crosoft com/about/ ega /en/us/Inte ectua Property/ Trademarks/EN-US aspx are trademarks of the M crosoft group of compan es A other marks are property of the r respect ve owners The examp e compan es, organ zat ons, products, doma n names, ema addresses, ogos, peop e, p aces, and events dep cted here n are fict t ous No assoc at on w th any rea company, organ zat on, product, doma n name, ema address, ogo, person, p ace, or event s ntended or shou d be nferred Th s book expresses the author’s v ews and op n ons The nformat on conta ned n th s book s prov ded w thout any express, statutory, or mp ed warrant es Ne ther the authors, O’Re y Med a, Inc , M crosoft Corporat on, nor ts rese ers, or d str butors w be he d ab e for any damages caused or a eged to be caused e ther d rect y or nd rect y by th s book Acquisitions Editor: Russe Jones Developmental Editor: Russe Jones Production Editor: Me an e Yarbrough Editorial Production: Chr st an Ho dener, S4Car s e Pub sh ng Serv ces Technical Reviewer: John Pau Meu er Copyeditor: Andrew Jones Indexer: WordCo Index ng Serv ces Cover Design: Tw st Creat ve • Seatt e Cover Composition: ContentWorks, Inc Illustrator: Rebecca Demarest

To my partner, Mark, and our children, Adam, Jacqueline, Joshua, and Sonny. With all my love, I thank you guys, for all of yours. —  Leonard Lobel

For my three boys: Miles, Sean, and Aidan. And for my sister, Valerie Hope. — Andrew Brust

Contents at a Glance Introduction xxi Part I

Core SQL Server Development

Chapter 1

Introducing SQL Server Data Tools

Chapter 2

T-SQL Enhancements

Chapter 3

Exploring SQL CLR

125

Chapter 4

Working with Transactions

169

Chapter 5

SQL Server Security

207

Part II

Going Beyond Relational

Chapter 6

XML and the Relational Database

255

Chapter 7

Hierarchical Data and the Relational Database

299

Chapter 8

Native File Streaming

323

Chapter 9

Geospatial Support

367

Part III

Applied SQL

Chapter 10

The Microsoft Data Access Juggernaut

427

Chapter 11

WCF Data Access Technologies

509

Chapter 12

Moving to the Cloud with SQL Azure

579

Chapter 13



SQL Azure Data Sync and Windows Phone Development

619

Chapter 14

Pervasive Insight

675

Chapter 15

xVelocity In-Memory Technologies

701

3 45

Index 737

Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi Acknowledgements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxvii

Part I

Core SQL Server Development

Chapter 1 Introducing SQL Server Data Tools Introduc ng SSDT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 4

Database Too ng Des gned for Deve opers. . . . . . . . . . . . . . . . . . . . .

4

Dec arat ve, Mode -Based Deve opment. . . . . . . . . . . . . . . . . . . . . . . .

5

Connected Deve opment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6

D sconnected Deve opment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7

Vers on ng and Snapshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

8

Target ng D fferent P atforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9

Work ng w th SSDT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9

Connect ng w th SQL Server Object Exp orer. . . . . . . . . . . . . . . . . . .

10

Gather ng New Requ rements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

16

Us ng the Tab e Des gner (Connected). . . . . . . . . . . . . . . . . . . . . . . . .

17

Work ng Offl ne w th a SQL Server Database Project. . . . . . . . . . . .

22

Tak ng a Snapshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

25

Us ng the Tab e Des gner (Offl ne Database Project). . . . . . . . . . . . .

25

Introduc ng Loca DB. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

27

Refactor ng the Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

31

Test ng and Debugg ng

33

Compar ng Schemas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

35

Pub sh ng to SQL Azure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

39

Adopt ng SSDT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

42

Summary

43

What do you think of this book? We want to hear from you! M crosoft s nterested n hear ng your feedback so we can cont nua y mprove our books and earn ng resources for you. To part c pate n a br ef on ne survey, p ease v s t:

microsoft.com/learning/booksurvey

vii

Chapter 2 T-SQL Enhancements Tab e-Va ued Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

45 46

More Than Just Another Temporary Tab e So ut on . . . . . . . . . . . . .

46

Subm tt ng Orders. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

47

Us ng TVPs for Bu k Inserts and Updates. . . . . . . . . . . . . . . . . . . . . . .

49

Pass ng TVPs Us ng ADO NET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

52

Pass ng Co ect ons to TVPs Us ng Custom Iterators. . . . . . . . . . . . .

54

TVP L m tat ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

57

Date and T me Data Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

58

Separat on of Dates and T mes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

58

More Portab e Dates and T mes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

58

T me Zone Awareness. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

59

Date and T me Accuracy, Storage, and Format. . . . . . . . . . . . . . . . . .

60

Date and T me Funct ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

62

The MERGE Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

65

Defin ng the Merge Source and Target. . . . . . . . . . . . . . . . . . . . . . . .

67

The WHEN MATCHED C ause . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

68

The WHEN NOT MATCHED BY TARGET C ause . . . . . . . . . . . . . . . . .

69

Us ng MERGE for Tab e Rep cat on . . . . . . . . . . . . . . . . . . . . . . . . . . .

70

The WHEN NOT MATCHED BY SOURCE C ause. . . . . . . . . . . . . . . . .

71

MERGE Output. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

73

Choos ng a Jo n Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

74

MERGE DML Behav or. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

75

The INSERT OVER DML Syntax. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A F terab e A ternat ve to OUTPUT…INTO. . . . . . . . . . . . . . . . . . . . .

76 77

Consum ng CHANGES. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

viii Contents

The GROUPING SETS Operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

83

Ro ng Up by Leve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

85

Ro ng Up A Leve Comb nat ons. . . . . . . . . . . . . . . . . . . . . . . . . . . .

86

Return ng Just the Top Leve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

88

M x ng and Match ng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

89

Hand ng NULL Va ues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

90

W ndow ng (OVER C ause) Enhancements. . . . . . . . . . . . . . . . . . . . . . . . . . . S d ng Aggregat ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

93 96

Us ng RANGE versus ROWS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 New T-SQL Funct ons n SQL Server 2012 . . . . . . . . . . . . . . . . . . . . . . . . . . . New Ana yt c Funct ons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

97 98

New Convers on Funct ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

103

New Date and T me Funct ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

104

New Log ca Funct ons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

106

New Str ng Funct ons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

107

Changed Mathemat ca Funct on. . . . . . . . . . . . . . . . . . . . . . . . . . . .

109

The THROW Statement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

109

Re-Throw ng Except ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

110

Compar ng THROW and RAISERROR. . . . . . . . . . . . . . . . . . . . . . . . .

111

Server-S de Pag ng. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

113

Us ng ROW NUMBER. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

113

Us ng OFFSET/FETCH NEXT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

114

The SEQUENCE Object. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sequence L m tat ons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

115 117

Metadata D scovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

118

Summary

122

Chapter 3 Exploring SQL CLR

125

Gett ng Started Enab ng CLR Integrat on. . . . . . . . . . . . . . . . . . . . . . . . . . V sua Stud o/SQL Server Integrat on . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

126 126

SQL Server Database Projects n V sua Stud o. . . . . . . . . . . . . . . . .

127

Automated Dep oyment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

129

SQL CLR Code Attr butes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

129

Your F rst SQL CLR Stored Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

130

CLR Stored Procedures and Server-S de Data Access. . . . . . . . . . . . . . . . .

132

P p ng Data w th SqlDataRecord and SqlMetaData. . . . . . . . . . . . . 134 Dep oyment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

136

Contents ix

Gett ng Ready. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

137

Dep oy ng Your Assemb y. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

138

Dep oy ng Your Stored Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . .

141

Test ng Your Stored Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

142

CLR Funct ons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

143

CLR Tr ggers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

148

CLR Aggregates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

151

SQL CLR Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

156

Secur ty. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

161

Exam n ng and Manag ng CLR Types n a Database. . . . . . . . . . . . . . . . . .

162

Best Pract ces for SQL CLR Usage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

168

Summary

168

Chapter 4 Working with Transactions What Is a Transact on?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Understand ng the ACID Propert es. . . . . . . . . . . . . . . . . . . . . . . . . . Loca Transact on Support n SQL Server. . . . . . . . . . . . . . . . . . . . . . . . . . .

170 170 172

Autocomm t Transact on Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

173

Exp c t Transact on Mode

173

Imp c t Transact on Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

176

Batch-Scoped Transact on Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . .

176

Iso at on Leve s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

179

Read Uncomm tted Iso at on Leve . . . . . . . . . . . . . . . . . . . . . . . . . . .

179

Read Comm tted Iso at on Leve . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

181

Repeatab e Read Iso at on Leve . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

182

Ser a zab e Iso at on Leve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

182

Snapshot Iso at on Leve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

182

Read Comm tted Snapshot Iso at on Leve . . . . . . . . . . . . . . . . . . . .

183

Iso at on Leve s n ADO NET. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

184

D str buted Transact ons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D str buted Transact on Term no ogy. . . . . . . . . . . . . . . . . . . . . . . . .

x Contents

169

186 186

Ru es and Methods of En stment. . . . . . . . . . . . . . . . . . . . . . . . . . . .

187

D str buted Transact ons n SQL Server . . . . . . . . . . . . . . . . . . . . . . .

189

D str buted Transact ons n the NET Framework. . . . . . . . . . . . . . .

190

Us ng a Resource Manager n a Successfu Transact on . . . . . . . . .

198

Transact ons n SQL CLR (CLR Integrat on). . . . . . . . . . . . . . . . . . . . . . . . . .

201

Putt ng It A Together. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

204

Summary

206

Chapter 5 SQL Server Security

207

Four Themes of the Secur ty Framework . . . . . . . . . . . . . . . . . . . . . . . . . . .

208

Secure by Des gn. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

208

Secure by Defau t. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

208

Secure by Dep oyment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

208

Secure Commun cat ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

208

SQL Server Secur ty Overv ew . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

209

SQL Server Log ns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

210

Database Users. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

211

The guest User Account . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

212

Authent cat on and Author zat on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

213

How C ents Estab sh a Connect on. . . . . . . . . . . . . . . . . . . . . . . . . .

213

Password Po c es. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

215

User-Schema Separat on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

216

Execut on Context. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

218

Encrypt on Support. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

222

Encrypt ng Data on the Move. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

223

Encrypt ng Data at Rest

224

Transparent Data Encrypt on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

229

SQL Server Aud t. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Creat ng an Aud t Object. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

234 235

Aud t ng Opt ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

236

Record ng Aud ts to the F e System. . . . . . . . . . . . . . . . . . . . . . . . . .

238

Record ng Aud ts to the W ndows Event Log. . . . . . . . . . . . . . . . . .

239

Aud t ng Server Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

239

Aud t ng Database Events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

240

V ew ng Aud ted Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

242

Query ng Aud t Cata og V ews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

244

Contents xi

Part a y Conta ned Databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

245

Creat ng a Conta ned User. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

245

Other Part a y Conta ned Database Features. . . . . . . . . . . . . . . . . .

246

How Hackers Attack SQL Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

249

D rect Connect on to the Internet. . . . . . . . . . . . . . . . . . . . . . . . . . . .

249

Weak System Adm n strator Account Passwords. . . . . . . . . . . . . . .

249

SQL Server Browser Serv ce. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

249

SQL Inject on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

250

Inte gent Observat on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

250

Summary

Part II

244

Creat ng a Part a y Conta ned Database. . . . . . . . . . . . . . . . . . . . . .

251

Going Beyond Relational

Chapter 6 XML and the Relational Database

255

Character Data as XML. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

256

The xml Data Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

257

Work ng w th the xml Data Type as a Var ab e. . . . . . . . . . . . . . . . .

257

Work ng w th XML n Tab es. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

258

XML Schema Defin t ons (XSDs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

259

XML Indexes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

266

FOR XML Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

268

FOR XML RAW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 FOR XML AUTO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 FOR XML EXPLICIT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 Add t ona FOR XML Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

276

The TYPE Opt on . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

276

FOR XML PATH. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 Em tt ng a ROOT E ement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Produc ng an In ne XSD Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . .

281

Produc ng E ement-Based XML. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

282

Shredd ng XML Us ng OPENXML. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

284

Query ng XML Data Us ng XQuery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

285

Understand ng XQuery Express ons and XPath. . . . . . . . . . . . . . . . xii Contents

280

285

SQL Server XQuery n Act on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

288

XML DML. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

296

Summary

Chapter 7 Hierarchical Data and the Relational Database

298

299

The hierarchyid Data Type. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

300

Creat ng a H erarch ca Tab e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

301

The GetLevel Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Popu at ng the H erarchy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

302 303

The GetRoot Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

303

The GetDescendant Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

304

The ToString Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

305

The GetAncestor Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

310

H erarch ca Tab e Index ng Strateg es . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

313

Depth-F rst Index ng. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

314

Breadth-F rst Index ng. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

314

Query ng H erarch ca Tab es. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

315

The IsDescendantOf Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

315

Reorder ng Nodes w th n the H erarchy. . . . . . . . . . . . . . . . . . . . . . . . . . . .

317

The GetReparentedValue Method. . . . . . . . . . . . . . . . . . . . . . . . . . . .

318

Transp ant ng Subtrees. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

319

More hierarchyid Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

321

Summary

322

Chapter 8 Native File Streaming Trad t ona BLOB Strateg es. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

323 323

BLOBs n the Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

324

BLOBs n the F e System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

324

Introduc ng FILESTREAM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

325

Enab ng FILESTREAM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

326

Enab ng FILESTREAM for the Mach ne . . . . . . . . . . . . . . . . . . . . . . .

326

Enab ng FILESTREAM for the Server Instance . . . . . . . . . . . . . . . . .

328

Contents xiii

Creat ng a FILESTREAM-Enab ed Database. . . . . . . . . . . . . . . . . . . . . . . . . Creat ng a Tab e w th FILESTREAM Co umns. . . . . . . . . . . . . . . . . . .

329 330

Stor ng and Retr ev ng FILESTREAM Data

331

De et ng FILESTREAM Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

334

D rect Stream ng n NET w th SqlFileStream. . . . . . . . . . . . . . . . . . . . . . . .

335

Understand ng SqlFileStream. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 Bu d ng the W ndows Forms C ent. . . . . . . . . . . . . . . . . . . . . . . . . .

337

Programm ng SqlFileStream Data Access

338

Creat ng a Stream ng HTTP Serv ce . . . . . . . . . . . . . . . . . . . . . . . . . .

348

Bu d ng a WPF C ent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

352

FILESTREAM L m tat ons and Cons derat ons. . . . . . . . . . . . . . . . . . . . . . . . Introduc ng F eTab e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

357

Creat ng a F eTab e. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

360

Man pu at ng a F eTab e. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

362

Search ng Documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

365

Summary

366

Chapter 9 Geospatial Support

367

SQL Server Spaces Out. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

367

Spat a Mode s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

368

P anar (F at-Earth) Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

368

Geodet c (E pso da Sphere) Mode. . . . . . . . . . . . . . . . . . . . . . . . . .

368

Spat a Data Standards. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

370

Import ng We -Known Text (WKT) . . . . . . . . . . . . . . . . . . . . . . . . . .

370

Import ng WKB. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

373

Import ng Geography Markup Language (GML). . . . . . . . . . . . . . .

374

Spat a Data Types. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

374

Work ng w th geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

375

Work ng w th geography

388

Spat a Enhancements n SQL Server 2012. . . . . . . . . . . . . . . . . . . . . . . . . .

xiv Contents

355

400

New Spat a Data C asses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

401

New Spat a Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

405

Other Enhancements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

411

Part III

Integrat ng w th M crosoft B ng Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

413

Summary

423

Applied SQL

Chapter 10 The Microsoft Data Access Juggernaut

427

NET Data Access Evo ut on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

427

Prepar ng the Samp e Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

430

Mon tor ng Database Act v ty w th SQL Server Profi er. . . . . . . . . . . . . . .

435

Convent ona ADO NET. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

436

Us ng the Raw Data Access Objects. . . . . . . . . . . . . . . . . . . . . . . . . .

436

Work ng w th DataSets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455 Language-Integrated Query (LINQ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LINQ to DataSet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Object Re at ona Mode ng (ORM) Comes to NET. . . . . . . . . . . . . . . . . . Mu t p e ORM Offer ngs from Redmond. . . . . . . . . . . . . . . . . . . . . .

472 473 477 479

LINQ to SQL Then and Now. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

479

Ent ty Framework Now and n the Future. . . . . . . . . . . . . . . . . . . . .

482

Summary

Chapter 11 WCF Data Access Technologies

508

509

Defin ng Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

509

WCF Data Access Opt ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

510

WCF Data Serv ces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

511

Bu d ng a WCF Data Serv ce. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

512

Creat ng the Ent ty Data Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

513

Test ng WCF Data Serv ces w th Internet Exp orer. . . . . . . . . . . . . .

515

Bu d ng C ent App cat ons for WCF Data Serv ces . . . . . . . . . . . .

518

Extend ng WCF Data Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

544

WCF RIA Serv ces

548

Estab sh ng a WCF RIA Serv ces L nk. . . . . . . . . . . . . . . . . . . . . . . . .

549

Creat ng the Ent ty Data Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

551

Bu d ng the Doma n Serv ce and Metadata C asses. . . . . . . . . . . .

552

Bu d ng the S ver ght C ent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

561

Contents xv

Inspect ng the NET Fram ng Protoco w th F dd er. . . . . . . . . . . . .

569

Test ng the Comp ete WCF RIA Serv ces So ut on . . . . . . . . . . . . . .

569

Mak ng the R ght WCF Data Access Cho ce. . . . . . . . . . . . . . . . . . . . . . . . .

577

Summary

578

Chapter 12 Moving to the Cloud with SQL Azure H story. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

581

But What Is SQL Azure?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

581

Why the L m tat ons?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

582

Pr c ng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

583

The F rst One’s Free. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

583

Gett ng Set Up. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

584

Beyond the Prerequ s tes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

585

Prov s on ng Your Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

586

Prov s on ng Your Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

589

Manag ng Your Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

589

Creat ng Tab es and Enter ng Data. . . . . . . . . . . . . . . . . . . . . . . . . . .

590

Query ng n the Browser. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

592

Index Des gn. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

592

Management and V sua zat ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

593

Connect ng from Down Be ow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

596

M grat ng and Sync ng Between Earth and C oud. . . . . . . . . . . . . . . . . . .

xvi Contents

579

599

DACPACs to the Rescue. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

600

Extract, Dep oy, Export, and Import DAC fi es. . . . . . . . . . . . . . . . .

600

Scenar os. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

602

SQL Azure Federat ons. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

607

A SQL Azure Federat ons Lex con. . . . . . . . . . . . . . . . . . . . . . . . . . . .

607

Creat ng a Federat on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

608

Federated Tab es . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

609

Us ng a Federat on Member . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

610

Sp tt ng and Dropp ng Federat on Members. . . . . . . . . . . . . . . . . .

610

Centra Tab es and Reference Tab es . . . . . . . . . . . . . . . . . . . . . . . . .

610

Fan-Out Quer es and Mu t -Tenancy . . . . . . . . . . . . . . . . . . . . . . . . .

611

Federat ons Support n SSMS and SSDT. . . . . . . . . . . . . . . . . . . . . . .

611

Federat ons Make Sense n the C oud . . . . . . . . . . . . . . . . . . . . . . . .

612

SQL Azure Report ng. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

612

Prov s on ng. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

613

Report Author ng. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

614

Dep oy ng Reports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

615

Gett ng Your Bear ngs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

617

Summary

Chapter 13 SQL Azure Data Sync and Windows Phone 7 Development Character st cs of an Occas ona y Connected System. . . . . . . . . . . . . . . . Data Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gett ng to Know SQL Azure Data Sync. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

617

619 620 620 621

Capab t es and Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

621

Data Sync Term no ogy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

622

Sync Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

623

The C ent Sync Agent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

624

SQL Azure Data Sync Cons derat ons. . . . . . . . . . . . . . . . . . . . . . . . .

625

Creat ng an Occas ona y Connected System. . . . . . . . . . . . . . . . . . . . . . . .

626

Prerequ s tes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

629

Configur ng SQL Azure Data Sync. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

630

Prov s on ng the SQL Azure Data Sync Server . . . . . . . . . . . . . . . . .

630

Creat ng the Sync Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

631

Host ng WCF Data Serv ces n W ndows Azure. . . . . . . . . . . . . . . . . . . . . . About W ndows Azure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

641 641

Creat ng the F xPo So ut on. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

642

Add ng the F xPo Data Serv ce. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

643

Add ng the Ent ty Data Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

644

Creat ng the F xPo C ent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

647

Consum ng OData on W ndows Phone . . . . . . . . . . . . . . . . . . . . . . . . . . . .

662

SQL Server on the Phone. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

666

Dep oy ng to W ndows Azure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

672

Summary

674

Contents xvii

Chapter 14 Pervasive Insight The M crosoft BI Stack What’s It A About?. . . . . . . . . . . . . . . . . . . . . . . .

676

Master Data Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

677

Data Qua ty Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

680

Integrat on Serv ces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

681

SQL Server RDBMS, Fast Track DW, and SQL Server PDW. . . . . . . . . . . . .

683

Data Marts and Data Warehouses . . . . . . . . . . . . . . . . . . . . . . . . . . .

683

The Star Schema. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

684

SQL Server Data Warehouse App ances. . . . . . . . . . . . . . . . . . . . . .

684

Ana ys s Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

686

The Mu t d mens ona Eng ne . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

686

PowerP vot and SSAS Tabu ar Mode. . . . . . . . . . . . . . . . . . . . . . . . . .

687

Data M n ng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

690

Power V ew. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

691

Report ng Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

692

Report Parts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

693

A ert ng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

693

Dashboard Components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

694

Exce and Exce Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Us ng Exce Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

694 694

PerformancePo nt Serv ces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

696

StreamIns ght. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

697

SQL Server Ed t ons and SharePo nt Vers on Requ rements. . . . . . . . . . .

697

Summary

699

Chapter 15 xVelocity In-Memory Technologies Co umn Store Databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Co umn Store Tech n the BI Industry. . . . . . . . . . . . . . . . . . . . . . . . . xVe oc ty n the RDBMS Co umnstore Indexes. . . . . . . . . . . . . . . . . . . . . .

xviii Contents

675

701 702 703 704

Bu d ng a Co umnstore Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

704

What You Can’t Do . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

704

How Co umnstore Indexes Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

706

xVe oc ty for Ana ys s PowerP vot and SSAS Tabu ar Mode s. . . . . . . . . .

709

C ear ng Up the Ana ys s Serv ces Vocabu ary . . . . . . . . . . . . . . . . .

710

The Lowdown on BISM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

711

Fr ends, Countrymen, Br ng Me Your Data. . . . . . . . . . . . . . . . . . . .

711

Bu d ng the BISM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

712

D a M for Mode ng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

715

Mode ng, Part Deux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

718

Query ng n Exce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

724

PowerP vot for SharePo nt. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

726

Mov ng to SSAS Tabu ar. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

727

Power V ew Here We Come. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

732

We come Back to Vert Paq . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

734

Summary

735

Index 737 About the Authors

773

What do you think of this book? We want to hear from you! M crosoft s nterested n hear ng your feedback so we can cont nua y mprove our books and earn ng resources for you. To part c pate n a br ef on ne survey, p ease v s t:

microsoft.com/learning/booksurvey

Contents xix

Introduction —Leonard Lobel

W

e come! Th s s a book about M crosoft SQL Server 2012 wr tten just for you, the ­deve oper Whether you are programm ng aga nst SQL Server d rect y at the ­database eve or further up the stack us ng M crosoft NET, th s book shows you the way The atest re ease of M crosoft’s flagsh p database product de vers an ­unprecedented, h gh y sca ab e data p atform capab e of hand ng the most demand ng tasks and ­work oads As w th every re ease, SQL Server 2012 adds many new features and ­enhancements for deve opers, adm n strators, and ( ncreas ng y) end users a ke Co ect ve y, these ­product enhancements re nforce—and advance—SQL Server’s pos t on as a ­prom nent contender n the ndustry As the product cont nues to evo ve, ts stack of offer ngs ­cont nues to expand And as the comp ete SQL Server stack s too arge for any one book to cover effect ve y, our emphas s n this book s on programmability Spec fica y, we exp ore the p ethora of ways n wh ch SQL Server (and ts c oud cous n, M crosoft SQL Azure) can be programmed for bu d ng custom app cat ons and serv ces

How Significant Is the SQL Server 2012 Release? SQL Server, part cu ar y ts re at ona database eng ne, matured qu te some t me ago So the “s gn ficance” of every new re ease over recent years can be v ewed— n some ways—as re at ve y nom na The ast watershed re ease of the product was actua y SQL Server 2005, wh ch was when the re at ona eng ne (that, for years, defined SQL Server) stopped occupy ng “center stage,” and nstead took ts pos t on a ongs de a set of ­serv ces that today, co ect ve y, define the product These nc ude the ­Bus ness ­Inte gence (BI) components Report ng Serv ces, Ana ys s Serv ces, and Integrat on Serv ces—features that began appear ng as ear y as 1999 but, pr or to SQL Server 2005, were ntegrated sporad ca y as a patchwork of oose y coup ed add-ons, ­w zards, and management conso es SQL Server 2005 changed a that w th a comp ete ­overhau For the first t me, the overa SQL Server product de vered a broader, r cher, and more conso dated set of features and serv ces wh ch are bu t nto—rather than bo ted onto—the p atform None of the product vers ons that have been re eased s nce that t me—SQL Server 2008, 2008 R2, and now 2012—have changed under y ng ­arch tecture th s rad ca y That sa d, each SQL Server re ease cont nues to advance tse f n v ta y s gn ficant ways SQL Server 2008 (re eased August 6, 2008) added a host of new features to the

xxi

re at ona eng ne—T-SQL enhancements, Change Data Capture (CDC), Transparent Data Encrypt on (TDE), SQL Aud t, FILESTREAM—p us powerfu BI capab t es w th Exce P votTab es, charts, and CUBE formu as SQL Server 2008 R2 (re eased Apr 21, 2010), nterna y dubbed the “BI Refresh” wh e n deve opment, added a revamped vers on of Report ng Serv ces as we as PowerP vot for Exce and SharePo nt, Master Data Serv ces, and StreamIns ght, but offered tt e more than m nor tweaks and fixes to the re at ona eng ne The newest re ease—SQL Server 2012—offic a y aunched on March 7, 2012 L ke ­every new re ease, th s vers on mproves on a of the key “ab t es” (ava ab ty, ­sca ab ty, manageab ty, programmab ty, and so on) Among the ch ef re ab ty mprovements s the new H gh Ava ab ty D saster Recovery (HADR) a ternat ve to database m rror ng HADR (a so common y known as “A ways On”) ut zes mu t p e secondary servers n an “ava ab ty group” for sca e-out read-on y operat ons (rather than forc ng them to s t d e, just wa t ng for a fa over to occur) Mu t subnet fa over c uster ng s another notab e new manageab ty feature SQL Server 2012 adds many new features to the re at ona eng ne, most of wh ch are covered n th s book There are powerfu T-SQL extens ons, most notab y the ­w ndow ng enhancements, p us 22 new T-SQL funct ons, mproved error hand ng, server-s de pag ng, sequence generators, r ch metadata d scovery techn ques, and conta ned databases There are a so remarkab e mprovements for unstructured data, such as the ­F eTab e abstract on over FILESTREAM and the W ndows fi e system API, fu -text p ­ roperty search ng, and Stat st ca Semant c Search Spat a support gets a b g boost as we , w th support for c rcu ar data, fu -g obe support, ncreased performance, and greater par ty between the geometry and geography data types And new “­co umnstore” techno ogy drast ca y ncreases performance of extreme y arge cubes (xVe oc ty for PowerP vot and Ana ys s Serv ces) and data warehouses (us ng an xVe oc ty- ke mp ementat on n the re at ona eng ne) The aforement oned re at ona eng ne features are mpress ve, but st amount to tt e more than “­ add t ves” over an a ready estab shed database p atform A new re ease needs more than just extra c ng on the cake for customers to perce ve an upgrade as ­compe ng To that end, M crosoft has nvested heav y n BI w th SQL Server 2012, and the ­effort shows The BI port on of the stack has been expanded great y, de ver ng key ­advances n “pervas ve ns ght ” Th s nc udes major updates to the product’s ­ana yt cs, data ­v sua zat on (such as se f-serv ce report ng w th Power V ew), and master data ­management capab t es, as we Data Qua ty Serv ces (DQS), a brand new data qua ty eng ne There s a so a new Bus ness Inte gence ed t on of the product that nc udes a of these capab t es w thout requ r ng a fu Enterpr se ed t on cense F na y, SQL Server Data Too s (SSDT) br ngs brand new database too ng ns de V sua Stud o SSDT xxii  Introduction

prov des a dec arat ve, mode -based des gn-t me exper ence for deve op ng databases wh e connected, offl ne, on-prem se, or n the c oud

Who Should Read This Book Th s book s ntended for deve opers who have a bas c know edge of re at ona ­database terms and pr nc p es

Assumptions In ta or ng the content of th s book, there are a few assumpt ons that we make about you F rst, we expect that you are a deve oper who s a ready know edgeab e about ­re at ona database concepts—whether that exper ence s w th SQL Server or ­non-M crosoft p atforms As such, you a ready know about tab es, v ews, pr mary and fore gn keys (re at onsh ps), stored procedures, user-defined funct ons, and tr ggers These essent a s are assumed know edge and are not covered n th s book S m ar y, we don’t exp a n proper re at ona des gn, ru es of data norma zat on, strateg c ­ndex ng pract ces, how to express bas c quer es, and other re at ona fundamenta s We a so ­assume that you have at east bas c fam ar ty w th SQL statement syntax—aga n, e ther T-SQL n SQL Server or SQL d a ects n other p atforms—and have a bas c work ng know edge of NET programm ng n C# on the c ent Hav ng sa d a that, we have a fa r y bera po cy regard ng these prerequ s tes For examp e, f you’ve on y dabb ed w th T-SQL or you’re more comfortab e w th M crosoft V sua Bas c NET than C#, that’s okay, as ong as you’re w ng to try and p ck up on th ngs as you read a ong Most of our code samp es are not that comp ex However, our exp anat ons assume some bas c know edge on your part, and you m ght need to do a tt e research f you ack the exper ence

Note  For the sake of consistency, all the .NET code in this book is written in C#. However, this book is in no way C#-oriented, and there is certainly ­nothing C#-specific in the .NET code provided. As we just stated, the code samples are not very complex, and if you are more experienced with Visual Basic .NET than you are with C#, you should have no trouble translating the C# code to Visual Basic .NET on the fly as you read it. W th that base ne estab shed, our approach has been to add va ue to the SQL Server documentat on by prov d ng a deve oper-or ented nvest gat on of ts features,



Introduction  xxiii

espec a y the new and mproved features n SQL Server 2012 We start w th the brand new database too ng, and the many r ch extens ons made to T-SQL and the ­re at ona database eng ne Then we move on to w der spaces, such as nat ve fi e stream ng, ­geospat a data, and other types of unstructured data We a so have chapters on ­secur ty, transact ons, c ent data access, secur ty, mob e/c oud deve opment, and more W th n these chapters, you w find deta ed coverage of the atest and most ­mportant SQL Server programm ng features You w atta n pract ca know edge and techn ca understand ng across the product’s numerous programmab ty po nts, empower ng you to deve op the most soph st cated database so ut ons for your end users Converse y, th s s not ntended as a resource for system adm n strators, database adm n strators, project managers, or end users Our genera ru e of thumb s that we don’t d scuss features that are not part cu ar y programmab e

Who Should Not Read This Book Th s book s not ntended for SQL Server adm n strators; t s a med square y at deve opers—and on y deve opers who have mastery of bas c database concepts

Organization of This Book The chapters of th s book are organ zed n three sect ons ■

Core SQL Server features

■

Beyond re at ona features

■

App ed SQL for bu d ng app cat ons and serv ces

By no means does th s book need to be read n any part cu ar order Read t from start to fin sh f you want, or jump r ght n to just those chapters that su t your needs or p que your nterests E ther way, you’ find the pract ca gu dance you need to get your job done The fo ow ng overv ew prov des a summary of these sect ons and the r chapters After the overv ew, you w find nformat on about the book’s compan on webs te, from wh ch you can down oad code samp es and work hands-on w th a the examp es n the book

xxiv  Introduction

Core SQL Server Development In Part I, we focus on core SQL Server features These nc ude brand new too ng (SSDT), enhancements to T-SQL, extended programmab ty w th SQL CLR code n NET ­anguages such as M crosoft V sua Bas c NET and C#, transact ons, and secur ty ■

Chapter 1  Introduc ng SQL Server Data Too s Our open ng chapter s a about SQL Server Data Too s (SSDT) W th the re ease of SQL Server 2012, SSDT now serves as your pr mary deve opment env ronment for bu d ng SQL Server app cat ons Wh e SQL Server Management Stud o (SSMS) cont nues to serve as the pr mary too for database adm n strators, SSDT represents a brand new deve oper exper ence SSDT p ugs n to ­M crosoft V sua Stud o for connected deve opment of on-prem se databases or SQL Azure ­databases runn ng n the c oud, as we as a new database project type for ­offl ne deve opment and dep oyment Us ng pract ca , rea -wor d scenar os, you w a so earn how to everage SSDT features such as code nav gat on, ­Inte Sense, refactor ng, schema compare, and more

■

Chapter 2  T-SQL Enhancements In Chapter 2, we exp ore the s gn ficant enhancements made to Transact-SQL ­( T-SQL)—wh ch st rema ns the best programm ng too for custom SQL Server deve opment We cover severa powerfu extens ons to T-SQL added n SQL Server 2008, beg nn ng w th tab e-va ued parameters (TVPs) You earn how to pass ent re sets of rows around between stored procedures and funct ons on the server, as we as between c ent and server us ng M crosoft ADO NET Date and t me features are exp ored next, nc ud ng separate date and t me data types, t me zone awareness, and mprovements n date and t me range, storage, and prec s on We then show many ways to use MERGE, a flex b e data ­man pu at on anguage (DML) statement that encapsu ates a the nd v dua ­operat ons typ ca y nvo ved n any merge scenar o From there, you earn about INSERT OVER DML for enhanced change data capture from the OUTPUT c ause of any DML statement We a so exam ne GROUPING SETS, an extens on to the trad t ona GROUP BY c ause that ncreases your opt ons for s c ng and d c ng data n aggregat on quer es We then d ve n to the new T-SQL enhancements ntroduced n SQL Server 2012, start ng w th windowing features The first w ndow ng funct ons to a ­ ppear n T-SQL date back to SQL Server 2005, w th the ntroduct on of ­severa ­rank ng funct ons W ndow ng capab t es have been qu te m ted ever s nce, but SQL Server 2012 fina y de vers some ­major ­mprovements to change a that F rst



Introduction  xxv

you w grasp ­w ndow ng c­ oncepts and the pr nc p es beh nd the OVER c ause, and then everage that ­know edge to ­ca cu ate runn ng and s d ng aggregates and perform other ana yt c ­ca cu at ons You w earn about every one of the 22 new T-SQL funct ons, ­nc ud ng 8 ana yt c w ndow ng funct ons, 3 convers on funct ons, 7 date and t me re ated funct ons, 2 og ca funct ons, and 2 str ng funct ons We a so ­exam ne mproved error hand ng w th THROW, server-s de pag ng w th ­OFFSET/FETCH NEXT, sequence generators, and r ch metadata d scovery ­techn ques We exp a n a of these new funct ons and features, and prov de c ear code samp es demonstrat ng the r use ■

Chapter 3  Exp or ng SQL CLR Chapter 3 prov des thorough coverage of SQL CLR programm ng—wh ch ets you run comp ed NET code on SQL Server—as we as gu dance on when and where you shou d put t to use We go beyond mere stored procedures, ­tr ggers, and funct ons to exp a n and demonstrate the creat on of CLR types and ­aggregates—ent t es that cannot be created at all n T-SQL We a so cover the d fferent methods of creat ng SQL CLR objects n SQL Server Database Projects n V sua Stud o and how to manage the r dep oyment, both from SSDT/V sua Stud o and from T-SQL scr pts n SQL Server Management Stud o and e sewhere

■

Chapter 4  Work ng w th Transact ons No matter how you wr te and package your code, you must keep your data cons stent to ensure ts ntegr ty The key to cons stency s transact ons, wh ch we cover n Chapter 4 Transact ons can be managed from a var ety of p aces, ke many SQL Server programmab ty features If you are wr t ng T-SQL code or c ent code us ng the ADO NET SqlClient prov der or System.Transactions, you need to be aware of the var ous transact on so at on eve s supported by SQL Server, the appropr ate scope of your transact ons, and best pract ces for wr t ng transact ona code Th s chapter gets you there

■

Chapter 5  SQL Server Secur ty Chapter 5 d scusses SQL Server secur ty at ength and exam nes your cho ces for keep ng data safe and secure from pry ng eyes and ma c ous ntent We beg n w th the bas c secur ty concepts concern ng og ns, users, ro es, ­authent cat on, and author zat on You then go on to earn about key-based encrypt on ­support, wh ch protects your data both wh e n trans t and at rest We then ­exam ne other powerfu secur ty features, nc ud ng Transparent Data ­Encrypt on (TDE) and SQL Server Aud t TDE a ows you to encrypt ent re databases n the background w thout spec a cod ng requ rements W th SQL

xxvi  Introduction

Server Aud t, v rtua y any act on taken by any user can be recorded for aud t ng n e ther the fi e system or the W ndows event og We a so show how to create contained databases, a new feature n SQL Server 2012 that e m nates host nstance dependenc es by stor ng og n credent a s d rect y n the database The chapter conc udes by prov d ng cruc a gu dance for adher ng to best pract ces and avo d ng common secur ty p tfa s

Going Beyond Relational W th the re ease of SQL Server 2012, M crosoft broadens support for sem -­structured and unstructured data n the re at ona database In Part II, we show how to ­everage the “beyond re at ona ” capab t es n SQL Server 2012—features that are ­becom ng ncreas ng y cr t ca n today’s wor d of b nary pro ferat on, and the ­emergence of h ghperformance so-ca ed “No SQL” database p atforms ■

Chapter 6  XML and the Re at ona Database SQL Server 2005 ntroduced the xml data type, and a ot of r ch XML support to go a ong w th t That nnovat on was an mmeasurab e mprovement over the use of p a n varchar or text co umns to ho d str ngs of XML (wh ch was common n ear er vers ons of SQL Server), and thus revo ut on zed the storage of XML n the re at ona database It empowers the deve opment of database ­app cat ons that work w th h erarch ca data natively—w th n the env ronment of the re at ona database system—someth ng not poss b e us ng ord nary str ng co umns In Chapter 6, we take a deep d ve nto the xml data type, XQuery extens ons to T-SQL, server-s de XML Schema Defin t on (XSD) co ect ons, XML co umn ndexng, and many more XML features

■

Chapter 7  H erarch ca Data and the Re at ona Database But XML s not your on y opt on for work ng w th h erarch ca data n the ­database In Chapter 7, we exp ore the hierarchyid data type that enab es you to cast a h erarch ca structure over any re at ona tab e Th s data type s ­mp emented as a “system CLR” type, wh ch s noth ng more rea y than a SQL CLR user-defined type (UDT), just ke the ones we show how to create on your own n Chapter 3 The va ue stored n a hierarchyid data type encodes the ­comp ete path of any g ven node n the tree structure, from the root down to the spec fic ord na pos t on among other s b ng nodes shar ng the same ­parent Us ng methods prov ded by th s new type, you can now effic ent y bu d, query, and man pu ate tree-structured data n your re at ona tab es Th s data type a so p ays an mportant ro e n SQL Server’s new F eTab e feature, as we exp a n n the next chapter on nat ve fi e stream ng



Introduction  xxvii

■

Chapter 8  Nat ve F e Stream ng In Chapter 8, you earn a about the FILESTREAM, an nnovat ve feature that ntegrates the re at ona database eng ne w th the NTFS fi e system to prov de h gh y effic ent storage and management of arge b nary objects (BLOBs)—­mages, v deos, documents, you name t Before FILESTREAM, you had to choose between stor ng BLOB data n the database us ng varbinary(max) (or the ­now-deprecated image) co umns, or outs de the database as unstructured b nary streams (typ ca y, as fi es n the fi e system) FILESTREAM prov des a powerfu abstract on ayer that ets you treat BLOB data og ca y as an ntegra part of the database, wh e SQL Server stores the BLOB data phys ca y separate from the database n the NTFS fi e system beh nd the scenes You w earn everyth ng you need to program aga nst FILESTREAM, us ng both T-SQL and the h gh-performance SqlFileStream NET c ass The wa kthroughs n th s chapter bu d W ndows, web, and W ndows Presentat on ­Foundat on (WPF) app cat ons that use FILESTREAM for BLOB data storage You w a so see how F eTab e, a new feature n SQL Server 2012, bu ds on FILESTREAM F eTab e comb nes FILESTREAM w th the hierarchyid (covered n Chapter 7) and the W ndows fi e system API, tak ng database BLOB ­management to new eve s As mp ed by the two words jo ned together n ts name, one F eTab e funct ons as two d st nct th ngs s mu taneous y a tab e and a fi e system—and you w earn how to exp o t th s new capab ty from both ang es

■

Chapter 9  Geospat a Support Chapter 9 exp ores the wor d of geospat a concepts and the r ch spat a s­ upport prov ded by the geometry and geography data types W th these system CLR types, t s very easy to ntegrate ocat on-awareness nto your ­app cat ons— at the database eve Respect ve y, geometry and geography enab e spat a ­deve opment aga nst the two bas c geospat a surface mode s p anar (flat) and geodet c ­(round-earth) W th spat a data (represented by geometr c or ­geograph c ­coord nates) stored n these data types, you can determ ne ­ntersect ons and ­ca cu ate ength, area, and d stance measurements aga nst that data The chapter first qu ck y covers the bas cs and then prov des wa kthroughs n wh ch you bu d severa geospat a database app cat ons, nc ud ng one that ntegrates mapp ng w th M crosoft B ng Maps We a so exam ne the s gn ficant spat a enhancements added n SQL Server 2012 A though ent re books have been wr tten on th s vast and ever-expand ng top c, our chapter de ves nto ­suffic ent depth so you can get busy work ng w th geospat a data r ght away

xxviii  Introduction

Applied SQL After we’ve covered so much nformat on about what you can do on the server and n the database, we move to Part III of the book, where we exp ore techno og es and ­demonstrate techn ques for bu d ng c ent/server, n-t er, and c oud so ut ons on top of your databases Whatever your scenar o, these chapters show you the most effect ve ways to extend your data’s reach We then conc ude w th coverage of SQL Azure, the BI stack, and the new co umnstore techno ogy known as xVe oc ty ■

Chapter 10  The M crosoft Data Access Juggernaut Chapter 10 covers every c ent/server data access strategy ava ab e n the NET Framework today We beg n w th ear est M crosoft ADO NET techn ques us ng raw data access objects and the DataSet abstract on, and d scuss the ­ongo ng re evance of these NET 1 0 techno og es We then exam ne ater data ­access techno og es, nc ud ng the concepts and syntax of anguage- ntegrated query (LINQ) We ook at LINQ to DataSet and LINQ to SQL, and then turn our focus heav y on the ADO NET Ent ty Framework (EF), M crosoft’s current ­recommended data access so ut on for NET You w earn Object Re at ona Mapp ng (ORM) concepts, and d scover how EF’s Ent ty Data Mode (EDM) prov des a powerfu abstract on ayer to dramat ca y stream ne the app cat on deve opment process

■

Chapter 11  WCF Data Access Techno og es After you have mastered the c ent/server techn ques taught n Chapter 10, you are ready to expose your data as serv ces to the wor d Chapter 11 prov des you w th deta ed exp anat ons and code samp es to get the job done us ng two techno og es based on W ndows Commun cat ons Foundat on (WCF) The first part of Chapter 11 covers WCF Data Serv ces, wh ch everages ­Representat ona State Transfer Protoco (REST) and Open Data Protoco (OData) to mp ement serv ces over your data source After exp a n ng these key ­concepts, you w see them put to pract ca use w th concrete examp es As you mon tor background network and database act v ty, we zone n and ock down on the cr t ca nterna s that make t a work The second part of the chapter demonstrates data access us ng WCF RIA Serv ces, a ater techno ogy that t­ argets S ver ght c ents n part cu ar (a though t can support other c ents as we ) We art cu ate the s m ar t es and d fferences between these two ­WCF-based techno og es, and arm you w th the know edge of how and when to use each one



Introduction  xxix

■

Chapter 12  Mov ng to the C oud w th SQL Azure In Chapter 12, we ook at the wor d of c oud database comput ng w th SQL Azure We exp a n what SQL Azure s a about, how t s s m ar to SQL Server and how t d ffers We ook at how SQL Azure s pr ced, how to s gn up for t, and how to prov s on SQL Azure servers and databases We exam ne the SQL Azure too ng and how to work w th SQL Azure from SSMS and SSDT We ­exp a n the many ways that Data-T er App cat ons (DACs) can be used to m grate databases between SQL Server and SQL Azure, us ng SSMS, SSDT, and the nat ve too ng of SQL Azure as we We fin sh up the chapter by exam n ng a spec a part t on ng feature ca ed SQL Azure Federat ons and we ook at SQL Azure Report ng, too

■

Chapter 13  SQL Azure Data Sync and W ndows Phone Deve opment Chapter 13 covers the broad top c of so-ca ed occas ona y connected ­systems by bu d ng out a comp ete so ut on that ncorporates SQL Azure Data Sync, W ndows Azure, and the W ndows Phone 7 deve opment p atform On the back end, an on-prem se SQL Server database s kept synchron zed w th a pub c-fac ng SQL Azure database n the c oud us ng SQL Azure Data Sync The c oud database s exposed us ng WCF Data Serv ces (a so hosted n the c oud by dep oy ng to W ndows Azure), and consumed v a OData by a mob e c ent app cat on runn ng on a W ndows Phone 7 dev ce The end-to-end so ut on deta ed n th s chapter demonstrates how these techno og es work to keep data n sync across on-prem se SQL Server, SQL Azure databases n the c oud, and oca storage on W ndows Phone 7 dev ces

■

Chapter 14  Pervas ve Ins ght In Chapter 14, we prov de an overv ew of the ent re SQL Server BI stack, ­nc ud ng SQL Server Fast Track Data Warehouse app ances, SQL Server ­Para e Data Warehouse ed t on, SQL Server Integrat on Serv ces, Ana ys s Serv ces, Master Data Serv ces, Data Qua ty Serv ces, Report ng Serv ces, Power V ew, PowerP vot, and StreamIns ght In the nterest of comp eteness, we a so prov de br ef overv ews of Exce Serv ces and PerformancePo nt Serv ces n SharePo nt and how they comp ement SQL Server We exp a n what each BI component does, and how they work together Perhaps most mportant, we show you how these techno og es from the BI arena are re evant to your work w th re at ona data, and how, n that ght, they can be qu te approachab e These techno og es shou dn’t be thought of as segregated or tangent a They are ntegra parts of SQL Server, and we seek to make them part of what you do w th the product

xxx  Introduction

■

Chapter 15  xVe oc ty In-Memory Techno og es In Chapter 15, we ook at M crosoft’s xVe oc ty co umn store techno ogy, and how to use t from the SQL Server re at ona database, as we as PowerP vot and Ana ys s Serv ces We exp a n how co umn-or ented databases work, we exam ne the new co umnstore ndexes n SQL Server 2012, and d scuss ts batch process ng mode, too We ook at how easy t s for re at ona database experts to work w th PowerP vot and SSAS Tabu ar mode, and we show how to br ng a these techno og es together w th the SQL Server Power V ew data ana ys s, d scovery, and v sua zat on too

Conventions and Features in This Book Th s book presents nformat on us ng convent ons des gned to make the nformat on readab e and easy to fo ow ■

Boxed e ements w th abe s such as “Note” prov de add t ona nformat on or a ternat ve methods for comp et ng a step successfu y

■

Text that you type (apart from code b ocks) appears n bo d

■

Code e ements n text (apart from code b ocks) appear n ta c

■

■

A p us s gn (+) between two key names means that you must press those keys at the same t me For examp e, “Press A t+Tab” means that you ho d down the A t key wh e you press the Tab key A vert ca bar between two or more menu tems (for examp e, F e C ose), means that you shou d se ect the first menu or menu tem, then the next, and so on

System Requirements To fo ow a ong w th the book’s text and run ts code samp es successfu y, we ­recommend that you nsta the Deve oper ed t on of SQL Server 2012, wh ch s ­ava ab e to a great number of deve opers through M crosoft’s MSDN Prem um subscr pt on, on your PC You w a so need V sua Stud o 2010; we recommend that you use the ­Profess ona ed t on or one of the Team ed t on re eases, each of wh ch s a so ava ab e w th the correspond ng ed t on of the MSDN Prem um subscr pt on product A the code samp es w a so work w th the upcom ng V sua Stud o 11, n beta at the t me of th s wr t ng



Introduction  xxxi

Important  To cover the widest range of features, this book is based on the Developer edition of SQL Server 2012. The Developer edition possesses the same feature set as the Enterprise edition of the product, although Developer edition licensing terms preclude production use. Both editions are high-end platforms that offer a superset of the features available in other editions (Standard, Workgroup, Web, and Express). We believe that it is in the best ­interest of developers for us to cover the full range of developer features in SQL Server 2012, including those available only in the Enterprise and Developer editions. To run these ed t ons of SQL Server and V sua Stud o, and thus the samp es n th s book, you’ need the fo ow ng 32-b t hardware and software (The 64-b t hardware and software requ rements are not sted here but are very s m ar ) ■

1 GHz or faster (2 GHz recommended) processor

■

Operat ng system, any of the fo ow ng

• M crosoft W ndows Server 2008 R2 SP1 • W ndows 7 SP1 (32- or 64-b t) • W ndows Server 2008 SP2 • W ndows V sta SP2 ■

■

■

■

xxxii  Introduction

For SQL Server 2012, 4 GB or more RAM recommended for a ed t ons (except the Express ed t on, wh ch requ res on y 1 GB) For SQL Server 2012, approx mate y 1460 MB of ava ab e hard d sk space for the recommended nsta at on Approx mate y 375 MB of add t ona ava ab e hard d sk space for SQL Server Books On ne, SQL Server Mob e Everywhere Books On ne, and samp e databases For V sua Stud o 2010, max mum of 20 GB ava ab e space requ red on ­nsta at on dr ve Note that th s figure nc udes space for nsta ng the fu set of MSDN documentat on A work ng Internet connect on (requ red to down oad the code samp es from the compan on webs te) A few of the code samp es a so requ re an Internet ­connect on to run

■

Super VGA (1024 × 768) or h gher reso ut on v deo adapter and mon tor ­recommended

■

M crosoft Mouse or compat b e po nt ng dev ce recommended

■

M crosoft Internet Exp orer 9 0 or ater recommended

Installing SQL Server Data Tools SSDT does not get nsta ed w th e ther V sua Stud o or SQL Server Instead, SSDT sh ps separate y v a the Web P atform Insta er (WebPI) Th s enab es M crosoft to d str bute t me y SSDT updates out-of-band w th (that s, w thout wa t ng for major re eases of) V sua Stud o or SQL Server Before you fo ow a ong w th the procedures n Chapter 1, down oad and nsta SSDT from http://msdn.microsoft.com/en-us/data/hh297027

Using the Book’s Companion Website V s t the book’s compan on webs te at the fo ow ng address http://go.microsoft.com/FWLink/?LinkId=252994

Code Samples A the code samp es d scussed n th s book can be down oaded from the book’s ­compan on webs te W th n the compan on mater a s parent fo der on the s te s a ch d fo der for each chapter Each ch d fo der, n turn, conta ns the samp e code for the chapter Because most of the code s exp a ned n the text, you m ght prefer to create t from scratch rather than open the fin shed vers on supp ed n the compan on samp e code ­However, the fin shed vers on w st prove usefu f you make a sma error a ong the way or f you want to run the code qu ck y before read ng through the narrat ve that descr bes t

Sample AdventureWorks Databases As of SQL Server 2005, and updated through SQL Server 2012, M crosoft prov des the popu ar AdventureWorks fam y of samp e databases Severa chapters n th s book reference the AdventureWorks2012 on ne transact on process ng (OLTP) database, and Chapter 15 references the AdventureWorksDW2012 data warehous ng database



Introduction  xxxiii

To fo ow a ong w th the procedures n these chapters, you can down oad these databases d rect y from the book’s compan on webs te The databases posted there are the exact vers ons that th s book was wr tten aga nst, or g na y obta ned from ­CodeP ex, wh ch s M crosoft’s open source webs te ( n fact, a of M crosoft’s offic a product code samp es are hosted on CodeP ex) To ensure you rece ve the same resu ts as you fo ow a ong w th certa n chapters n th s book, we recommend down oad ng the AdventureWorks2012 OLTP and AdventureWorksDW2012 data warehous ng databases from the book’s compan on webs te rather than from CodeP ex (where updated ­vers ons may cause d fferent resu ts than the or g na vers ons) You can find the d rect ons to attach (use) the samp e databases on the samp e ­database down oad page

Previous Edition Chapters In add t on to a the code samp es, the book’s compan on webs te a so conta ns severa chapters from the 2008 and 2005 ed t ons of th s book that were not updated for th s ed t on n order to accommodate coverage of new SQL Server 2012 features You can down oad SQL Server 2005 chapters that cover Serv ce Broker, nat ve XML Web Serv ces, SQL Server Management Stud o, SQL Server Express ed t on, Integrat on Serv ces, and debugg ng, as we as SQL Server 2008 chapters on data warehous ng, on ne ana yt ca process ng (OLAP), data m n ng, and Report ng Serv ces

Errata & Book Support We’ve made every effort to ensure the accuracy of th s book and ts compan on ­content Any errors that have been reported s nce th s book was pub shed are sted on our M crosoft Press s te at oreilly.com http://go.microsoft.com/FWLink/?LinkId=252995 If you find an error that s not a ready sted, you can report t to us through the same page If you need add t ona support, ema M crosoft Press Book Support at [email protected] P ease note that product support for M crosoft software s not offered through the addresses above

xxxiv  Introduction

We Want to Hear from You At M crosoft Press, your sat sfact on s our top pr or ty, and your feedback our most va uab e asset P ease te us what you th nk of th s book at http://www.microsoft.com/learning/booksurvey The survey s short, and we read every one of your comments and deas Thanks n advance for your nput!

Stay in Touch Let’s keep the conversat on go ng! We’re on Tw tter http://twitter.com/MicrosoftPress



Introduction  xxxv

Acknowledgements

I

t’s hard to be eve I first began research for th s book at an ear y Software Des gn Rev ew for SQL Server “Dena ” n Redmond back n October 2010 Th s s my second ed t on as ead author of th s book, and a though I enjoyed the work even more th s t me around, t was certa n y no eas er than the 2008 ed t on My goa —upfront—was to produce the most comprehens ve (yet approachab e) SQL Server 2012 deve oper resource that I cou d, one that best answers, “How many ways can I program SQL Server?” I cou d not have even contemp ated pursu ng that goa w thout the a d of numerous other ta ented and car ng nd v dua s—fo ks who deserve spec a recogn t on The r generous support was ent out n many d fferent yet equa y essent a ways So the order of names ment oned be ow s by no means an nd cat on of degree or proport on; s mp y put, I cou dn’t have wr tten th s book w thout everyone’s he p

W th so many peop e to thank, Cra g Brann ng (CEO of Ta an, Inc ) s at the top of my most wanted st Back n m d-2010, Cra g was qu ck to approach me about ­tak ng on th s project Next th ng I knew, I was on board and we were scarfing down unch (smooth work!) Thank you (and a the other wonderfu fo ks at Ta an) for ­gett ng th s book off the ground n the first p ace, and prov d ng a cont nuous source of ­support throughout ts product on I’m a so extreme y fortunate to have teamed up w th my co eague and fr end, ­co-author Andrew Brust (M crosoft MVP/RD) Th s s actua y Andrew’s th rd t me around contr but ng h s know edge and expert se to th s resource; he not on y ­co-authored the 2008 ed t on, but was ead author of the first ed t on for SQL Server 2005 So I thank h m once aga n for wr t ng four ste ar chapters n th s new 2012 ­ed t on And Pau De cog ano (who a so contr buted to the 2008 ed t on) d d a superb job confront ng the top c of end-to-end c oud deve opment w th SQL Azure Data Sync, W ndows Azure, and W ndows Phone 7—a n a s ng e outstand ng chapter Pau , your amb t on s adm rab e, and I thank you for your t re ess work and the great job done! Ken Jones, my pa at O’Re y Med a, gets spec a ment on for h s expert gu dance, p us h s steady pat ence through a the adm n strat ve shenan gans Thank you Ken, and to your ove y w fe, Andrea, as we , for her ns ghtfu he p w th the geospat a ­content I was a so very ucky to have worked c ose y w th Russe Jones,Me an e Yarbrough, John Mue er, and ­Chr st an Ho dener, whose superb ed tor a contr but ons, techn ca rev ew, and overa gu dance were v ta to the successfu product on of th s book The ass stance prov ded by a number of peop e from var ous M crosoft product teams he ped tack e the cha enge of wr t ng about new software as t evo ved through

xxxvii

severa beta re eases Thank you to Roger Doherty, for nv t ng me out to Redmond for the SDR n 2010, as we as for connect ng me w th the r ght peop e I needed to get my job done Gert Drapers and Adam Mahood were part cu ar y he pfu for the ­ns de scoop on SSDT as t changed from one CTP to the next Adam’s a ways d rect and ­a ways ava ab e nes of commun cat on turned an ent re chapter’s hard work nto fun work Doug Laudensch ager a so prov ded va uab e ns ght, wh ch enhanced new ­coverage of unstructured FILESTREAM data And natura y, a great b g thank you to the ent re product team for creat ng the best re ease of SQL Server ever! I’m a so part cu ar y proud of a the brand new NET data access coverage n th s book, and wou d ke to g ve spec a thanks to my pa Marce de Vr es, M crosoft MVP and RD n the Nether ands Marce s a master of d str buted arch tectures, and h s ­nva uab e ass stance great y he ped shape coverage n the WCF data access chapter Ik ben heel dankbaar voor jouw inbreng! Th s book cou d not have been wr tten, of course, w thout the ove and support of my fam y I have been consumed by th s project for much of the past e ghteen months—wh ch has at t mes transformed me nto an absentee I owe an enormous debt of grat tude to my wonderfu partner Mark, and our awesome k ds Adam, ­Jacque ne, Josh, and Sonny, for be ng so pat ent and to erant w th me And greatest thanks of a go out to my dear Mom, b ess her sou , for a ways encourag ng me to wr te w th ­“express on ” —Leonard Lobel

When you’re not a fu -t me author, there’s rea y never a “good” t me to wr te a book It’s a ways an added extra, and t typ ca y takes s gn ficant y more t me than ant c pated That creates burdens for many peop e, nc ud ng the author’s fam y, the book’s ed tors, and co-authors as we When one of the authors s start ng a new bus ness, burdens doub e, a around I’d ke to thank my fam y, the M crosoft Press team and espec a y th s book’s ead author, Lenn Lobe , for endur ng these burdens, w th flex b ty and uncommon y nfin te pat ence —Andrew Brust

xxxviii  Acknowledgements

Par t I

Core SQL Server Development chapter 1

Introduc ng SQL Server Data Too s

chapter 2

T-SQL Enhancements

chapter 3

Exp or ng SQL CLR

125

chapter 4

Work ng w th Transact ons

169

chapter 5

SQL Server Secur ty

207

3 45

1

C hapter 1

Introducing SQL Server Data Tools —Leonard Lobel

W

th the re ease of SQL Server 2012, M crosoft de vers a powerfu , new ntegrated ­deve opment env ronment (IDE) for des gn ng, test ng, and dep oy ng SQL Server databases— oca y, ­offl ne, or n the c oud—a from r ght ns de M crosoft V sua Stud o (both the 2010 vers on and the ­upcom ng V sua Stud o 11, current y n beta at the t me of th s wr t ng) Th s IDE s ca ed SQL Server Data Too s (SSDT), and t represents a major step forward from prev ous y ava ab e too ng—notab y, SQL Server Management Stud o (SSMS) and V sua Stud o Database Profess ona ed t on (DbPro) SSDT s not ntended to be a rep acement for SSMS, but nstead can be v ewed much more as a great y evo ved mp ementat on of DbPro Indeed, SSMS s a ve and we n SQL Server 2012, and t cont nues to serve as the pr mary management too for database administrators who need to ­configure and ma nta n hea thy SQL Server nsta at ons And a though DbPro was a good first step towards offl ne database deve opment, ts re at ve y m ted des gn-t me exper ence has prec uded ts w despread adopt on So for years, programmers have been pr mar y us ng SSMS to conduct ­deve opment tasks (and before SQL Server 2005, they typ ca y used two database ­adm n strator [DBA] too s—SQL Enterpr se Manager and Query Ana yzer) It’s a ways been necessary for ­programmers to use management-or ented too s (such as SSMS) rather than deve oper-focused too s (such as V sua Stud o) as a pr mary database deve opment env ronment when bu d ng database ­app cat ons—unt now The re ease of SSDT prov des a s ng e env ronment hosted n V sua Stud o, w th database ­too ng that spec fica y targets the deve opment process Thus, you can now des gn and bu d your ­databases w thout constant y togg ng between V sua Stud o and other too s In th s chapter, you’ earn how to use SSDT ns de V sua Stud o to s gn ficant y enhance your product v ty as a SQL Server ­deve oper We beg n w th an overv ew of key SSDT concepts and features, and then wa k you through ­demonstrat ons of var ous connected and d sconnected scenar os

3

Introducing SSDT Database Tooling Designed for Developers The nconven ent truth s database deve opment s hard Gett ng everyth ng done correct y s a huge cha enge—proper schema and re at ona des gn, the ntr cac es of Transact-SQL (T-SQL) as a anguage, performance tun ng, and more, are a d fficu t tasks n and of themse ves However, w th respect to the deve opment process—the way n wh ch you create and change a database—there are some part cu ar scenar os that the r ght too ng can mprove great y SSDT de vers that too ng

The SSDT Umbrella of Services and Tools SSDT encompasses more than just the new database too ng covered n th s chapter; t s actua y a packag ng of what was former y the V sua Stud o 2008–based Bus ness Inte gence Deve oper Stud o (BIDS) too SSDT supports the trad t ona BIDS project types for SQL Server Ana ys s Serv ces (SSAS), Report ng Serv ces (SSRS), and Integrat on Serv ces (SSIS), n add t on to the new database too ng So w th SSDT, M crosoft has now brought together a of the SQL Server database deve opment exper ences ns de a s ng e vers on of V sua Stud o Desp te ts broader defin t on, th s chapter uses the term SSDT spec fica y as t perta ns to the new database deve opment too s that SSDT adds to the V sua Stud o IDE

Here are some of the cha enges that deve opers face when des gn ng databases ■

■

■

■

Dependencies  By ts very nature, the database s fu of dependenc es between d fferent k nds of schema objects Th s comp cates deve opment, as even the s mp est changes can very qu ck y become very comp ex when dependenc es are nvo ved Late Error Detection  You can spend a ot of t me bu d ng comp ex scr pts, on y to find out that there are prob ems when you try to dep oy them to the database Or, your scr pt may dep oy w thout errors, but you have an ssue somewhere that doesn’t man fest tse f unt the user encounters a run-t me error “Drift“ Detection  The database s a constant y mov ng target After dep oyment, t’s fa r y common for a DBA to come a ong and tweak or patch someth ng n the product on d ­ atabase; for examp e, add ng ndexes to mprove query performance aga nst part cu ar tab es When the env ronments fa out of sync, the database s n a d fferent state than you and your ­app cat on expect t to be—and those d fferences need to be dent fied and reconc ed Versioning  Deve opers have grown so accustomed to work ng w th “change scr pts“ that t makes you wonder, where s the definition of the database? Of course you can re y on t be ng in the database, but where s t from the standpo nt of preserv ng and protect ng t? How do you ma nta n the defin t on across d fferent vers ons of your app cat on? It’s very d fficu t to revert to a po nt n t me and reca an ear er vers on of the database that matches up w th an

4  Part I  Core SQL Server Development

ear er vers on of an app cat on So you can’t eas y synchron ze vers ons and vers on h story between your database and app cat on ■

Deployment  Then there are the cha enges of target ng d fferent vers ons, nc ud ng most recent y, SQL Azure You may need to dep oy the same database out to d fferent ocat ons, and must account for vary ng compat b ty eve s when d fferent ocat ons are runn ng ­d fferent vers ons of SQL Server (such as SQL Server 2005, 2008, 2008 R2, 2012, and SQL Azure)

Many of these pa n po nts are rooted n the not on that the database s “statefu “ Every t me you bu d and run a NET app cat on n V sua Stud o, t s a ways n t a zed to the same “new“ state but as soon as the app cat on goes off to access the database, t’s the same “o d“ database w th the same schema and data n t Thus, you are forced to th nk not on y about the des gn of the database, but a so about how you mp ement that des gn—how you actua y get that des gn moved nto the ­database g ven the database’s current state If the root of these prob ems es n the database be ng statefu , then the heart of the so ut on es n work ng dec arat ve y rather than mperat ve y So rather than just work ng w th change scr pts, SSDT ets you work w th a declaration of what you believe (or want) the database to be Th s a ows you to focus on the des gn, wh e the too takes care of wr t ng the appropr ate change scr pts that w safe y app y your des gn to an actua target database SSDT takes a dec arat ve, mode -based ­approach to database des gn—and as you advance through th s chapter, you’ see how th s approach remed es the aforement oned pa n po nts

Declarative, Model-Based Development We’ve started exp a n ng that SSDT uses a dec arat ve, mode -based approach What th s means s that there s a ways an n-memory representat on of what a database ooks ke—an SSDT database ­model—and a the SSDT too s (des gners, va dat ons, Inte Sense, schema compare, and so on) operate on that mode Th s mode can be popu ated by a ve connected database (on-prem se or SQL Azure), an offl ne database project under source contro , or a po nt- n-t me snapshot taken of an offl ne database project (you w work w th snapshots n the upcom ng exerc ses) But to re terate, the too s are agnost c to the mode ’s back ng; they work exc us ve y aga nst the mode tse f Thus, you enjoy a r ch, cons stent exper ence n any scenar o—regard ess of whether you’re work ng w th ­on-prem se or c oud databases, offl ne projects, or vers oned snapshots The T-SQL representat on of any object n an SSDT mode s a ways expressed n the form of a ­CREATE statement An ALTER statement makes no sense n a dec arat ve context—a ­CREATE ­statement dec ares what an object shou d ook ke, and that’s the on y th ng that you (as a ­deve oper) need to be concerned w th Depend ng on the state of the target database, of course, a change scr pt conta n ng e ther a CREATE statement ( f the object doesn’t ex st yet) or an appropr ate ­ALTER ­statement ( f t does) w be needed to proper y dep oy the object’s defin t on Furthermore, f ­dependenc es are nvo ved (wh ch they very often are), other objects need to be dropped and ­re-created n the dep oyment process Fortunate y, you can now re y on SSDT to dent fy any changes (the “d fference“) between your mode defin t on and the actua database n order to compose the

Chapter 1  ntroduc ng SQL Server Data Too s   5

necessary change scr pt Th s keeps you focused on just the defin t on F gure 1-1 dep cts the SSDT mode -based approach to database deve opment

SQL Server Data Tools (SSDT)

Database Model SQL Server Database Project

SQL Server 2005, 2008, 2008 R2, 2012

LocalDB

SQL Azure

Database Snapshot File (.dacpac)

Figure 1-1  SSDT works w th a mode backed by connected databases (on prem se or n the c oud), offl ne ­database projects, or database snapshot fi es.

Connected Development A though SSDT p aces great emphas s on the dec arat ve mode , t n no way prevents you from ­work ng mperat ve y aga nst ve databases when you want or need to You can open query w ndows to compose and execute T-SQL statements d rect y aga nst a connected database, w th the ass stance of a debugger f des red, just as you can n SSMS The connected SSDT exper ence s dr ven off the new SQL Server Object Exp orer n V sua ­Stud o You can use th s new dockab e too w ndow to accomp sh common database deve opment tasks that former y requ red SSMS Us ng the new SQL Server Object Exp orer s str k ng y s m ar to ­work ng aga nst a connected database n SSMS’s Object Exp orer—but remember that (and we’ r sk ­overstat ng th s) the SSDT too s operate on y on a database model So when work ng n connected mode, SSDT actua y creates a mode from the rea database—on the fly—and then ets you ed t that mode Th s “buffered“ approach s a subt e, yet key, d st nct on from the way that SSMS operates When you save a schema change made w th the new tab e des gner, SSDT works out the n ­ ecessary scr pt needed to update the rea database so t reflects the change(s) made to the mode Of course, the end resu t s the same as the connected SSMS exper ence, so t sn’t str ct y necessary to ­understand th s buffered behav or that’s occurr ng beh nd the scenes But after you do grasp t, the too ’s offl ne project deve opment and snapshot vers on ng capab t es w mmed ate y seem natura and ntu t ve to you Th s s because offl ne projects and snapshots are s mp y d fferent back ngs of 6  Part I  Core SQL Server Development

the very same SSDT mode When you’re work ng w th the SQL Server Object Exp orer, the mode ’s ­back ng just happens to be a ve connected database There’s an add t ona nuance to SSDT’s buffered-wh e-connected approach to database ­deve opment that bears ment on ng There are n fact two mode s nvo ved n the process of app y ng schema changes to a database Just before SSDT attempts to app y your schema changes, t a ­ ctua y creates a new mode of the current y connected database SSDT uses th s mode as the target for a mode compar son w th the mode you’ve been ed t ng Th s dua -mode approach prov des the “dr ft detect on“ mechan sm you need to ensure that the schema compare operat on (upon wh ch SSDT w be bas ng ts change scr pt) accounts for any schema changes that may have been made by another user s nce you began ed t ng your vers on of the mode Va dat on checks w then catch any prob ems caused by the other user’s changes (wh ch wou d not have been present when you began mak ng your changes)

Disconnected Development The new SQL Server Object Exp orer ets you connect to and nteract w th any database r ght from ns de V sua Stud o But SSDT offers a great dea more than a mere rep acement for the connected SSMS exper ence It a so de vers a r ch offl ne exper ence w th the new SQL Server Database Project type and oca database runt me (Loca DB) The T-SQL scr pt fi es n a SQL Server Database Project are a dec arat ve n nature (on y CREATE statements; no ALTER statements) Th s s a rad ca y d fferent approach than you’re accustomed to when “deve op ng“ databases n SSMS (where you execute far more ALTER statements than CREATE statements) Aga n, you get to focus on defin ng “th s s how the database shou d ook,“ and et the too determ ne the appropr ate T-SQL change scr pt needed to actua y update the ve database to match your defin t on If you are fam ar w th the Database Profess ona (DbPro) ed t on of V sua Stud o, you w nstant y recogn ze the many s m ar t es between DbPro’s Database Projects and SSDT’s SQL Server Database Projects Desp te major over ap however, SSDT project types are d fferent than DbPro project types, and appear as a d st nct project temp ate n V sua Stud o’s Add New Project d a og The new tab e des gner, buffered connect on mechan sm, and other SSDT features covered n th s chapter work on y w th SSDT SQL Server Database Projects However, and as you may have guessed, t’s easy to upgrade ex st ng DbPro projects to SSDT projects Just r ght-c ck the project n So ut on Exp orer and choose Convert To SQL Server Database Project Note that th s s a one-way upgrade, and that DbPro ­art facts that are not yet supported by SSDT (such as data generat on p ans, see the fo ow ng Note) w not convert

Note  There are several important features still available only in DbPro, most notably data generation, data compare, schema view, and database unit testing. Eventually, SSDT plans on providing key elements of the DbPro feature set and will completely replace the Database Professional edition of Visual Studio. For now though, you must continue to rely on DbPro for what’s still missing in SSDT.

Chapter 1  ntroduc ng SQL Server Data Too s   7

The new SQL Server Database Project type enjoys many of the same capab t es and features as other V sua Stud o project types Th s nc udes not on y source contro for each nd v dua ­database object defin t on, but many of the common code nav gat on and refactor ng parad gms that ­deve opers have grown to expect of a modern IDE (such as Rename, Goto Defin t on, and F nd A ­References) The SSDT database mode ’s r ch metadata a so prov des for far better Inte Sense than what SSMS has been offer ng s nce SQL Server 2008, g v ng you much more of that “strong y-typed“ confidence factor as you code You can a so set breakpo nts, s ng e step through T-SQL code, and work w th the Loca s w ndow much ke you can when debugg ng NET project types W th SSDT, app cat on and database deve opment too ng has now fina y been un fied under one roof V sua Stud o A major advantage of the mode -based approach s that mode s can be generated from many d fferent sources When connected d rect y v a SQL Server Object Exp orer, SSDT generates a mode from the connected database, as we exp a ned a ready When you create a SQL Server Database Project (wh ch can be mported from any ex st ng database, scr pt, or snapshot), you are creat ng an offl ne, source-contro ed project ns de V sua Stud o that fu y descr bes a rea database But t’s ­actua y a project—not a rea database Now, SSDT generates a mode that’s backed by your SQL Server Database Project So the exper ence offl ne s just the same as when connected—the des gners, Inte Sense, va dat on checks, and a the other too s work exact y the same way As you conduct your database deve opment w th n the project, you get the same “background comp at on“ exper ence that you’re used to exper enc ng w th typ ca NET deve opment us ng C# or V sua Bas c (VB) NET For examp e, mak ng a change n the project that can’t be subm tted to the database because of dependency ssues w resu t n des gn-t me warn ngs and errors n the Error L st pane You can then c ck on the warn ngs and errors to nav gate d rect y to the var ous dependenc es so they can be dea t w th Once a the bu d errors d sappear, you’ be ab e to subm t the changes to update the database

Versioning and Snapshots A database project g ves you an offl ne defin t on of a database As w th a V sua Stud o projects, each database object (tab e, v ew, stored procedure, and every other d st nct object) ex sts as a text fi e that can be p aced under source code contro The project system comb ned w th source ­contro enab es you to secure the defin t on of a database n V sua Stud o, rather than re y ng on the ­defin t on be ng stored n the database tse f At any po nt n t me, and as often as you’d ke, you can create a database snapshot A snapshot s noth ng more than a fi e ( n the Data-t er App cat on Component Package, [dacpac] format) that ho ds the ser a zed state of a database mode , based on the current project at the t me the ­snapshot s taken It s essent a y a s ng e-fi e representat on of your ent re database schema Snapshots can ­ater be deser a zed and used w th any SSDT too (schema compare, for examp e) So you can deve op, dep oy, and synchron ze database structures across oca /c oud databases and d fferent y vers oned offl ne database projects, a w th cons stent too ng

8  Part I  Core SQL Server Development

Pause for a moment to th nk about the powerfu capab t es that snapshots prov de A snaphot encapsu ates an ent re database structure nto a s ng e .dacpac fi e that can be nstant y d ­ eser a zed back nto a mode at any t me Thus, they can serve as e ther the source or target of a schema ­compare operat on aga nst a ve database (on-prem se or SQL Azure), an offl ne SQL Server Database Project, or some other snapshot taken at any other po nt n t me Snapshots can a so be he pfu when you don’t have access to the target database, but are expected nstead to hand a change scr pt off to the DBA for execut on In add t on to the change scr pt, you can a so send the DBA a snapshot of the database project taken just before any of your offl ne work was performed That snapshot s your change scr pt’s assumpt on of what the ve database ooks ke So the DBA, n turn, can perform a schema compare between your snapshot and the ve database (th s can be done from SSDT’s command- ne too w thout V sua Stud o) The resu ts of that schema ­compare w nstant y et the DBA know f t’s safe to run your change scr pt If the resu ts revea d screpanc es between the ve database and the database snapshot upon wh ch your change scr pt s based, the DBA can reject your change scr pt and a ert you to the prob em

Targeting Different Platforms SQL Server Database Projects have a target p atform sw tch that ets you spec fy the spec fic SQL Server vers on that you ntend to dep oy the project to A the va dat on checks aga nst the ­project-backed mode are based on th s sett ng, mak ng t tr v a for you to test and dep oy your database to any part cu ar vers on of of SQL Server (2005 and ater), nc ud ng SQL Azure It’s s mp y a matter of choos ng SQL Azure as the target to ensure that your database can be dep oyed to the c oud w thout any prob ems If your database project defines someth ng that s not supported n SQL Azure (a tab e w th no c ustered ndex, for examp e), t w get flagged as an error automat ca y

Working with SSDT Our ntroduct on to the SSDT too set s comp ete, and t’s now t me to see t n act on The rest of th s chapter presents a samp e scenar o that demonstrates, step-by-step, how to use many of the SSDT features that you’ve just earned about Pract c ng a ong w th th s examp e w so d fy your know edge and understand ng of the too , and prepare you for us ng SSDT n the rea wor d w th rea database projects

Important  SSDT does not get installed with either Visual Studio or SQL Server. Instead, SSDT ships separately via the Web Platform Installer (WebPI). This enables Microsoft to ­distribute timely SSDT updates out-of-band with (that is, without waiting for major releases of) Visual Studio or SQL Server. Before proceeding, download and install SSDT from http://msdn.microsoft.com/en-us/data/hh297027.



Chapter 1  ntroduc ng SQL Server Data Too s   9

Connecting with SQL Server Object Explorer The journey w start by creat ng a database You w first use SSDT to execute a prepared scr pt that creates the database n a query w ndow connected to a SQL Server nstance Then you w start w ­ ork ng w th SSDT d rect y aga nst the connected database Th s exper ence s s m ar to us ng ­prev ous too s, so t’s the perfect p ace to start Later on, you’ sw tch to work ng d sconnected us ng an offl ne database project Launch V sua Stud o 2010, c ck the V ew menu, and choose SQL Server Object Exp orer Th s d sp ays the new SQL Server Object Exp orer n a V sua Stud o pane (docked to the eft, by defau t) Th s new too w ndow s the ma n act v ty hub for the connected deve opment exper ence n SSDT From the SQL Server Object Exp orer, you can eas y connect to any server nstance for wh ch you have credent a s In our scenar o, the localhost nstance runn ng on the deve opment mach ne s a fu SQL Server 2012 Deve oper ed t on nsta at on Th s nstance s assum ng the ro e of a ve product on database server that you can mag ne s runn ng n an on-prem se datacenter You’re now go ng to connect to that “product on“ database server R ght-c ck the SQL Server node at the top of the SQL Server Object Exp orer, choose Add SQL Server, and type n your mach ne name as the server to connect to A though you can certa n y, a ternat ve y, type localhost nstead (or even s mp y the s ng e-dot shorthand syntax for localhost), we’re d rect ng you to use the mach ne name nstead Th s s because you’ soon earn about the new oca database runt me (Loca DB) that SSDT prov des for offl ne test ng and debugg ng The Loca DB nstance always runs oca y, whereas the product on database on the other hand just ­happens to be runn ng oca y Because t can be potent a y confus ng to see both localhost and (localdb) n the SQL Server Object Exp orer, us ng the mach ne name nstead of localhost makes t c ear that one ­represents the product on database wh e the other refers to the database used for oca ­(offl ne) deve opment and test ng w th SSDT The screen snapshots for the figures n th s chapter were taken on a W ndows Server 2008 R2 mach ne named SQL2012DEV, so we’ be us ng that mach ne name throughout the chapter when referr ng to the product on database runn ng on localhost Of course, you’ need to rep ace the assumed SQL2012DEV mach ne name w th your own mach ne name ­wherever you see t ment oned If you have nsta ed SQL Server to use m xed-mode authent cat on and you are not us ng ­ ndows authent cat on, then you’ a so need to choose SQL Server authent cat on and supp y your W ­credent a s at th s po nt, before you can connect Once connected, SQL Server Object Exp orer shows the ­product on server and ets you dr down to show a the databases runn ng on t, as shown n F gure 1-2

10  Part I  Core SQL Server Development

Figure 1-2  The new SQL Server Object Exp orer n V sua Stud o expanded to show severa connected databases.

Once connected, r ght-c ck the server nstance node SQL2012DEV and choose New Query V sua Stud o opens a new T-SQL code ed tor w ndow, ke the one shown n F gure 1-3

Figure 1-3  A connected query w ndow.

Th s env ronment shou d seem very fam ar to anyone exper enced w th SSMS or Query ­Ana yzer Not ce the status bar at the bottom nd cat ng that the query w ndow s current y connected to the SQL2012DEV nstance (aga n, t w actua y read your mach ne name) The too bar at the top ­nc udes a drop-down st nd cat ng the current defau t database for the nstance you’re connected to As w th prev ous too s, th s w be the master database (as shown at the top of F gure 1-3) You must st take care to change th s sett ng (or ssue an appropr ate USE statement) so that you don’t ­nadvertent y ­access the master database when you rea y mean to access your app cat on’s database In th s ­exerc se, you’re creat ng a brand new database, so t’s fine that the current database s set to master at th s t me



Chapter 1  ntroduc ng SQL Server Data Too s   11

Tip This concern stems from the fact that, by default, the master database is established as every login’s default database. A great way to protect yourself from accidentally t­ rampling over the master database when working with SSDT (or any other tool) is to change your login’s default database to be your application’s database, which will then become the ­default database (rather than master) for every new query window that you open. This will greatly reduce the risk of unintentional data corruption in master, which can have ­disasterous consequences. When you navigate to your login from the Security node in SQL Server Object Explorer, you’ll be able to see its default database set to master in the Properties window, but you won’t be able to change it. This is a management task that is not supported in the SSDT tooling, although you can still use SSDT to execute the appropriate ALTER LOGIN statement in a query window. Alternatively, you can easily make the change in SSMS as follows. Start SSMS, connect to your server instance, and drill down to your login beneath the Security and Logins nodes in the SSMS Object Explorer. Then right-click the login and choose Properties. Click the default database drop-down list, change its value from master to your application’s database, and click OK. From then on, your database (not master) will be set as the default for every new SSDT query window that you open. Type the code shown n L st ng 1-1 nto the query w ndow (or open the scr pt fi e ava ab e n the down oadab e code on th s book’s compan on webs te; see the sect on “Code Samp es“ n the ­“Introduct on“ for deta s on how to down oad the samp e code) You m ght next be nc ned to press F5 to execute the scr pt, but that won’t work W th SSDT n V sua Stud o, press ng F5 builds and deploys a SQL Server Database Project to a debug nstance (you’ be creat ng such a project ater on, but you don’t have one yet) Th s s very d fferent to the way F5 s used n SSMS or Query Ana yzer to mmed ate y execute the current scr pt (or current y se ected scr pt) SSDT uses a d fferent keyboard shortcut for th s purpose In fact, there are two keyboard shortcuts (w th correspond ng too bar buttons and r ght-c ck context menu tems); one to execute w thout a debugger (Ctr +Sh ft+E) and one to execute us ng an attached debugger (A t+F5) You’ pract ce ­debugg ng ater on, so for now just press Ctrl+Shift+E to mmed ate y execute the scr pt and create the database (you can a so c ck the Execute Query button n the too bar, or r ght-c ck anywhere w th n the code w ndow and choose Execute from the context menu) Listing 1-1  T SQL scr pt for creat ng the SampleDb database

CREATE DATABASE SampleDb GO USE SampleDb GO -- Create the customer and order tables CREATE TABLE Customer(

12  Part I  Core SQL Server Development

CustomerId bigint NOT NULL PRIMARY KEY, FirstName varchar(50) NOT NULL, LastName varchar(50) NOT NULL, CustomerRanking varchar(50) NULL) CREATE TABLE OrderHeader( OrderHeaderId bigint NOT NULL, CustomerId bigint NOT NULL, OrderTotal money NOT NULL) -- Create the relationship ALTER TABLE OrderHeader ADD CONSTRAINT FK_OrderHeader_Customer FOREIGN KEY(CustomerId) REFERENCES Customer(CustomerId) -- Add a few customers INSERT INTO Customer (CustomerId, FirstName, LastName, CustomerRanking) VALUES (1, 'Lukas', 'Keller', NULL), (2, 'Jeff', 'Hay', 'Good'), (3, 'Keith', 'Harris', 'so-so'), (4, 'Simon', 'Pearson', 'A+'), (5, 'Matt', 'Hink', 'Stellar'), (6, 'April', 'Reagan', '') -- Add a few orders INSERT INTO OrderHeader(OrderHeaderId, CustomerId, OrderTotal) VALUES (1, 2, 28.50), (2, 2, 169.00), -- Jeff's orders (3, 3, 12.99), -- Keith's orders (4, 4, 785.75), (5, 4, 250.00), -- Simon's orders (6, 5, 6100.00), (7, 5, 4250.00), -- Matt's orders (8, 6, 18.50), (9, 6, 10.00), (10, 6, 18.00) -- April's orders GO -- Create a handy view summarizing customer orders CREATE VIEW vwCustomerOrderSummary WITH SCHEMABINDING AS SELECT c.CustomerID, c.FirstName, c.LastName, c.CustomerRanking, ISNULL(SUM(oh.OrderTotal), 0) AS OrderTotal FROM dbo.Customer AS c LEFT OUTER JOIN dbo.OrderHeader AS oh ON c.CustomerID = oh.CustomerID GROUP BY c.CustomerID, c.FirstName, c.LastName, c.CustomerRanking GO

Th s s a very s mp e scr pt that we’ d scuss n a moment But first, not ce what just happened SSDT executed the scr pt d rect y aga nst a connected SQL Server nstance, and then sp t the code w ndow hor zonta y to d sp ay the resu t ng server messages n the bottom pane The green con ­abe ed Query Executed Successfu y n the ower- eft corner offers assurance that a went we w th the scr pt execut on Because of the two mu t -row INSERT statements used to create samp e ­customers and order data, you can see the two “rows affected“ messages n the bottom Message pane, as shown n F gure 1-4 Overa , the exper ence thus far s very s m ar to prev ous too s, ensur ng a smooth ­trans t on to SSDT for deve opers a ready fam ar w th the o der too ng

Chapter 1  ntroduc ng SQL Server Data Too s   13

Figure 1-4  The query w ndow after successfu y execut ng a T SQL scr pt.

Th s s mp e scr pt created a database named SampleDb, w th the two tab es Customer and ­ rderHeader It a so defined a fore gn key on the CustomerId co umn n both tab es, wh ch estab shes O the parent-ch d (one-to-many) re at onsh p between them It then added a few customer and re ated order rows nto the r respect ve tab es F na y, t created a v ew summar z ng each customer’s orders by aggregat ng a the r order tota s Now run two quer es to v ew some data At the bottom of the code w ndow, type the fo ow ng two SELECT statements SELECT * FROM Customer SELECT * FROM vwCustomerOrderSummary

Not ce the Inte Sense as you type After you fin sh typ ng, hover the cursor over Customer, and then aga n over vwCustomerOrderSummary V sua Stud o d sp ays too t ps descr b ng those ­objects respect ve y as a tab e and a v ew Now hover the cursor over the star symbo n each ­SELECT ­statement V sua Stud o d sp ays a too t p st ng a the fie ds represented by the star symbo n each query Th s funct ona ty s prov ded by the SSDT T-SQL anguage serv ces runn ng n the background that ­cont nuous y query the database mode backed by the connected SampleDb database Now se ect just the two SELECT statements ( eave the ent re scr pt above them unse ected) and press Ctrl+Shift+E The resu t s s m ar to press ng F5 n SSMS or Query Ana yzer on y the se ected text s executed (wh ch s what you’d expect) SSDT runs both quer es and d sp ays the r resu ts, as shown n F gure 1-5 You don’t need the query w ndow any onger, so go ahead and c ose t now (you a so don’t need to save the scr pt) R ght-c ck the Databases node n SQL Server Object Exp orer and choose Refresh You’ see the new SampleDb database node appear Expand t to dr down nto the database As ­F gure 1-6 shows, the env ronment s s m ar to the Object Exp orer n SSMS, and ets you carry out most (but not a ) of the deve oper-or ented tasks that SSMS ets you perform 14  Part I  Core SQL Server Development

Figure 1-5  V ew ng the resu ts of se ected statements executed n the query w ndow.

Figure 1-6 The SampleDb database n SQL Server Object Exp orer expanded to show severa of ts objects.



Chapter 1  ntroduc ng SQL Server Data Too s   15

The database s now up and runn ng on SQL2012DEV Everyth ng s perfect—unt that ema from market ng arr ves The r team has just put together some new requ rements for you, and now there’s more work to be done

Gathering New Requirements The new requ rements perta n to the way customers are ranked n the Customer tab e Or g na y, the market ng team had requested add ng the CustomerRanking co umn as a ghtwe ght mechan sm for data entry operators to rank customer performance Th s ad-hoc rat ng was supposed to be oose y based on the customer’s tota purchases across a orders, but you can see from the CustomerRanking va ues n F gure 1-5 that users fo owed no cons stency whatsoever as they entered data (no surpr se there) They’ve typed th ngs ke A+, so-so, and Good And some customers have no mean ngfu data at a , such as empty str ngs, wh tespace, or NULL va ues To mprove the s tuat on, the market ng team wou d ke to ret re the ad-hoc data entry co umn and rep ace t w th a forma zed customer rank ng system that s more a gned w th the r or g na ntent In the r change request ema (wh ch s natura y flagged Urgent), they have attached the spreadsheet shown n F gure 1-7 conta n ng new reference data for var ous pre-defined rank ng eve s They’ve scr bb ed someth ng about dep oy ng to SQL Azure as we , and then they s gn off w th “P S , We need t by Fr day“ (and no surpr se there, e ther)

Figure 1-7  Reference data for the new customer rank ng system.

After g v ng the matter some thought, you organ ze a h gh- eve task st Your st tem zes the deve opment steps you p an on tak ng to fu fi the market ng department’s requ rements 1. Remove the CustomerRanking co umn from the Customer tab e 2. Create a new CustomerRanking tab e based on the spreadsheet n F gure 1-7, w th a pr mary

key co umn CustomerRankingId stor ng the va ues 1 through 5 3. Add a new co umn CustomerRankingId to the Customer tab e 4. Create a fore gn key on the CustomerRankingId co umn to estab sh a re at onsh p between

the Customer tab e and the new CustomerRanking tab e 16  Part I  Core SQL Server Development

5. Update the vwCustomerOrderSummary v ew to jo n on the new CustomerRanking tab e 6. Create a new uspRankCustomers stored procedure to update the Customer tab e’s new fore gn

key co umn, based on each customer’s tota order va ue 7. Va date for SQL Azure, then dep oy to the c oud

The rest of th s chapter wa ks through th s procedure n deta , step by step A ong the way, you’ earn to everage many mportant SSDT features and w ga n ns ght nto the way the new too ng works It’s t me to get started w th the first step remov ng a co umn from a tab e

Note  The scenario we’ve presented here is admittedly somewhat artificial. We are not necessarily advocating these particular steps as the best way to solve a given problem, and certainly hope you are working with better designs than this in your own database. But for the purpose of this exercise—namely, learning how to use SSDT—we ask that you go along with it. The premise may be contrived, but the steps we’ve outlined for the solution are in fact quite representative of typical recurring activities in the everyday life of an average SQL Server developer.

Using the Table Designer (Connected) In SQL Server Object Exp orer, r ght-c ck the Customer tab e and choose V ew Des gner to open the SSDT tab e des gner, as shown n F gure 1-8

Figure 1-8  The new SSDT tab e des gner.

The top- eft pane of th s des gner w ndow sts the defined co umns n a gr d just as n the SSMS tab e d ­ es gner, but the s m ar ty ends there A very d fferent mechan sm s at p ay w th the new SSDT ­des gner, one that shou d be easy to understand after a the d scuss on we’ve had around ­dec arat ve, mode -based des gn The CREATE TABLE statement n the bottom T-SQL pane g ves t away Knowng that the tab e a ready ex sts n the database, why s th s a CREATE statement? We , that’s because

Chapter 1  ntroduc ng SQL Server Data Too s   17

th s sn’t actua y T-SQL code that you ntend to execute aga nst the database as- s (wh ch wou d fa of course, because the tab e ex sts) Rather, t’s a T-SQL declaration of “how th s tab e shou d ook,“ whether t ex sts or not—and ndeed, whether t ex sts w th a different schema or not— n the target database Here’s what’s actua y happen ng The des gner s operat ng over a memory-res dent database mode ns de ts work ng env ronment Because you are connected at the moment, that mode s backed by the ve SampleDb database But when you sw tch to work ng offl ne w th a SQL Server ­Database Project (as you w n the next sect on of th s chapter), you’ nteract w th the very same tab e des gner over a mode backed by a project nstead of a rea database A mode can a so be backed by a database snapshot Because the tab e des gner just operates over a mode , the same too works cons stent y n any of these scenar os You want to remove the CustomerRanking co umn, and that can be done e ther by de et ng t from the gr d n the top pane or ed t ng t out of the dec arat ve T-SQL code n the bottom pane Both panes are mere y v ews nto the same tab e, so any changes appear b d rect ona y Throughout th s exerc se, you’ exper ment w th d fferent ed t ng techn ques n the tab e des gner, start ng w th the qu ckest method Just r ght-c ck CustomerRanking n the top gr d and choose De ete The co umn s removed from the gr d and, as you’d expect, the T-SQL code s updated to reflect the change That was a pretty easy change App y ng that change to the database shou d be easy, too Go ahead and c ck the Update button on the too bar Unfortunate y, nstead of just work ng as you’d ke, you rece ve the fo ow ng error message Update cannot proceed due to validation errors. Please correct the following errors and try again. SQL71501 :: View: [dbo].[vwCustomerOrderSummary] contains an unresolved reference to an object. Either the object does not exist or the reference is ambiguous because it could refer to any of the following objects: [dbo].[Customer].[c]::[CustomerRanking], [dbo].[Customer]. [CustomerRanking] or [dbo].[OrderHeader].[c]::[CustomerRanking]. SQL71501 :: View: [dbo].[vwCustomerOrderSummary] contains an unresolved reference to an object. Either the object does not exist or the reference is ambiguous because it could refer to any of the following objects: [dbo].[Customer].[c]::[CustomerRanking], [dbo].[Customer]. [CustomerRanking] or [dbo].[OrderHeader].[c]::[CustomerRanking]. SQL71558 :: The object reference [dbo].[Customer].[CustomerID] differs only by case from the object definition [dbo].[Customer].[CustomerId]. SQL71558 :: The object reference [dbo].[OrderHeader].[CustomerID] differs only by case from the object definition [dbo].[OrderHeader].[CustomerId].

What went wrong? Referr ng back to L st ng 1-1, not ce that the v ew defin t on for vwCustomerOrderSummary spec fies the WITH SCHEMABINDING c ause Th s means that the co umns of the v ew are bound to the under y ng tab es exposed by the v ew, wh ch protects you from “break ng“ the v ew w th schema changes—as you’ve done just now The prob em, as reported by the first two errors, s that the schema-bound v ew’s CustomerRanking co umn has sudden y been removed from the Customer tab e that under es the v ew The second two errors are actua y on y case-sens t v ty warn ngs that, on the r own, wou d not prevent the update from succeed ng We w exp a n these ­case-sens t v ty warn ngs a b t ater; for now, rema n focused on the dependency ssue that’s b ock ng the update 18  Part I  Core SQL Server Development

The nterest ng th ng worth not ng at th s po nt s that SSDT caught the cond t on before even attempting to app y your changes to the ve database (wh ch wou d certa n y have thrown an ­error) In fact, you cou d have been aware of th s ssue even before c ck ng Update f you had ­prev ous y opened the Error L st pane, because SSDT constant y va dates changes to the mode n the ­background wh e you ed t t n the des gner C ck the Cance button to d sm ss the error w ndow Then c ck the V ew menu and choose ­Error L st to open the pane Not ce how the errors and warn ngs appear, just ke comp at on errors ­appear for C# and VB NET projects And just ke those project types, you can doub e-c ck tems n the Error L st and nstant y nav gate to the offend ng code to dea w th the errors In th s case, both ­dependency errors are n vwCustomerOrderSummary, so doub e-c ck e ther one now to open a code ed tor nto the v ew, as shown n F gure 1-9

Figure 1-9  Detect ng and nav gat ng va dat on errors.

You want to rev se th s v ew to jo n aga nst the new CustomerRanking tab e, but that’s not ­com ng up unt step 4 n your task st So for now, just perform the m n mum ed ts necessary to c ear the ­va dat on errors (wh ch are dent fied by red squ gg es ke you’ve seen n other V sua Stud o code w ndows) so you can update the database and move on De ete (by comment ng out) the two ­references to c.CustomerRanking co umn from the v ew (one s n the co umn st, the other n the GROUP BY c ause) Not ce how the errors d sappear from the Error L st pane as you correct the code You’re now beg nn ng to exper ence the effects of mode -based deve opment w th SSDT n V sua Stud o W th a c ear Error L st, you know that a your changes are va d You have a tered a tab e and a v ew, but those changes have been made on y to the memory-res dent mode The changed objects are current y open n separate V sua Stud o w ndows, and both w ndows have an Update button

Chapter 1  ntroduc ng SQL Server Data Too s   19

Yet t makes no d fference wh ch of the two buttons you c ck— n e ther case, Update means that all changes that have been buffered get sent to the database So wh chever Update button you c ck, your ed ts to both the tab e and the v ew are go ng to get app ed to the database at once How s that go ng to happen? The ed ts were s mp e enough, but the T-SQL change scr pt needed to app y those ed ts s actua y a b t more comp ex And there n ay the beauty of th s new too ng—a of that scr pt ng comp ex ty s hand ed for you by SSDT The too compares the ed ted mode w th a brand-new mode based on the ve database, and then works out the change scr pt automat ca y Creat ng a fresh mode from the database at th s t me makes sure you’re work ng w th ts atest state, n case t dr fted because t was mode ed for the current ed t ng sess on Then t runs an nterna schema compare operat on between the ed ted mode (the source) and the atest mode based on the ve ­database (the target) to dent fy a the r d fferences F na y, SSDT generates a change scr pt that can be executed on the ve database to app y the changes C ck Update now to generate the change scr pt Before runn ng the change scr pt, SSDT d sp ays an nformat ve report of a the act ons that the change scr pt s go ng to take C ck Update now to d sp ay the Prev ew Database Updates w ndow, as shown n F gure 1-10

Figure 1-10  The Prev ew Database Updates w ndow.

You shou d a ways scrut n ze th s prev ew to make sure t’s cons stent w th the act ons and resu ts you wou d expect of the ed ts you’ve made In th s case, you’re be ng warned about data oss n the Customer tab e by dropp ng the CustomerRanking co umn You’re a so be ng to d that the scr pt w drop and then re-create the schema b nd ng of the vwCustomerOrderSummary v ew, before and after the tab e s a tered A of th s s expected Now you can c ck Update Database to mmed ate y execute the change scr pt, or you can c ck Generate Scr pt to oad the change scr pt nto a code ed tor so you can v ew, poss b y mod fy, and choose to e ther execute t or not In most cases, you’ fee comfortab e just c ck ng Update Database, part cu ar y f you’ve rev ewed the warn ngs and act ons reported by the database update prev ew Do ng so w mmed ate y 20  Part I  Core SQL Server Development

­execute the change scr pt to update the ve database But be ng that th s s your very first update, c ck Generate Scr pt nstead so you can exam ne the scr pt before you run t The scr pt s shown n L st ng 1-2 (to conserve space, error-check ng code has been commented out) Listing 1-2  The change scr pt for the a tered tab e and v ew automat ca y generated by SSDT.

/* Deployment script for SampleDb */ // ... :setvar DatabaseName "SampleDb" GO // ... USE [$(DatabaseName)]; GO // ... BEGIN TRANSACTION GO PRINT N'Removing schema binding from [dbo].[vwCustomerOrderSummary]...'; GO ALTER VIEW [dbo].[vwCustomerOrderSummary] AS SELECT c.CustomerID, c.FirstName, c.LastName, c.CustomerRanking, ISNULL(SUM(oh.OrderTotal), 0) AS OrderTotal FROM dbo.Customer AS c LEFT OUTER JOIN dbo.OrderHeader AS oh ON c.CustomerID = oh.CustomerID GROUP BY c.CustomerID, c.FirstName, c.LastName, c.CustomerRanking; // ... GO PRINT N'Altering [dbo].[Customer]...'; GO ALTER TABLE [dbo].[Customer] DROP COLUMN [CustomerRanking]; GO // ... PRINT N'Adding schema binding to [dbo].[vwCustomerOrderSummary]...'; GO -- Create a handy view summarizing customer orders ALTER VIEW vwCustomerOrderSummary WITH SCHEMABINDING AS SELECT c.CustomerID, c.FirstName, c.LastName, ISNULL(SUM(oh.OrderTotal), 0) AS OrderTotal FROM dbo.Customer AS c LEFT OUTER JOIN dbo.OrderHeader AS oh ON c.CustomerID = oh.CustomerID GROUP BY c.CustomerID, c.FirstName, c.LastName



Chapter 1  ntroduc ng SQL Server Data Too s   21

GO // ... IF @@TRANCOUNT>0 BEGIN PRINT N'The transacted portion of the database update succeeded.' COMMIT TRANSACTION END ELSE PRINT N'The transacted portion of the database update failed.' GO DROP TABLE #tmpErrors GO PRINT N'Update complete.' GO

It’s great that you d dn’t have to write the change scr pt, but t’s st the change scr pt Let’s ook t over qu ck y now

mportant that you understand

Us ng var ab e subst tut on, the scr pt first ssues a USE statement that sets SampleDb as the ­current database and then t beg ns a transact on The transact on w get comm tted on y f the ent re change scr pt comp etes successfu y Then the scr pt ssues an ALTER VIEW statement that removes the schema b nd ng from vwCustomerOrderSummary w thout yet chang ng ts defin t on So t st conta ns those two references to the CustomerRanking co umn that’s about to get dropped from the Customer tab e, but that w not present a prob em because WITH SCHEMABINDING has been removed from the v ew Next, the scr pt ssues the ALTER TABLE statement that actua y drops the co umn from the tab e After the co umn s dropped, another ALTER VIEW statement s ssued on vwCustomerOrderSummary w th the new vers on that no onger references the dropped co umn and s once aga n schemabound F na y, the transact on s comm tted and the update s comp ete Press Ctrl+Shift+E The scr pt s executed and output descr b ng act ons taken by the scr pt are d sp ayed n the Messages pane Removing schema binding from [dbo].[vwCustomerOrderSummary]... Altering [dbo].[Customer]... Adding schema binding to [dbo].[vwCustomerOrderSummary]... The transacted portion of the database update succeeded. Update complete.

You can c ose a open w ndows now V sua Stud o w prompt to save changes, but t’s not ­ ecessary to do so because the database has just been updated R ght-c ck on the database and n choose Refresh, and then dr down nto SampleDb n SQL Server Object Exp orer to confirm that the tab e and v ew changes have been app ed You w see that the CustomerRanking co umn has been ­removed from the database, and that comp etes step 1

Working Offline with a SQL Server Database Project W th SQL Server Database Projects, you can deve op databases w th no connect on whatsoever to a SQL Server nstance A SQL Server Database Project s a project that conta ns nd v dua , ­dec arat ve, T-SQL source code fi es These source fi es co ect ve y define the comp ete structure of a database 22  Part I  Core SQL Server Development

Because the database defin t on s ma nta ned th s way ns de a V sua Stud o project, t can be preserved and protected w th source code contro (SCC) just ke the art facts n a your other V sua Stud o project types SSDT generates a mode from the project structure beh nd the scenes, just ke t generates a mode from the ve database when work ng connected Th s ets you use the same SSDT too s whether work ng offl ne or connected You carr ed out your first task on ne wh e connected d rect y to a ve database Now you’ create a SQL Server Database Project for the database so that you can cont nue your work offl ne A though (as you’ve seen) t’s easy to use SSDT for connected deve opment, you shou d dea y deve op your databases offl ne w th SQL Server Database Projects, and pub sh to ve servers whenever you’re ready to dep oy your changes By do ng so, you w der ve the benefits of source contro , snapshot vers on ng, and ntegrat on w th the rest of your app cat on’s code through the V sua Stud o so ut on and project system There are severa ways to create a SQL Server Database Project You can start w th an empty project, des gn a database structure from the ground up, and then pub sh the ent re structure to a new database on a SQL Server nstance oca y or n the c oud on SQL Azure Or, as n th s scenar o, you have an ex st ng database on a oca SQL Server nstance from wh ch you want to generate a SQL Server Database Project And you want th s project popu ated w th a the dec arat ve T-SQL source fi es that comp ete y define the ex st ng database structure It’s easy to do th s w th the Import Database d a og R ght-c ck the SampleDb database under the SQL2012DEV nstance n SQL Server Object Exp orer and choose Create New Project to d sp ay the Import Database d a og, as shown n F gure 1-11

Figure 1-11  Creat ng a SQL Server Database Project from an ex st ng database.



Chapter 1  ntroduc ng SQL Server Data Too s   23

The source database connect on confirms that your new project w be generated from the S­ ampleDb database on SQL2012DEV Change the target project name from Database1 to SampleDb (and set the ocat on, too, f you w sh) Check the Create New So ut on checkbox, and f you have an SCC prov der for V sua Stud o, check Add To Source Contro as we Then c ck Start If you checked Add To Source Contro , you w be prompted at th s po nt to supp y credent a s and server nformat on (th s w depend on your defau t SCC prov der n V sua Stud o) It takes a few ­moments for V sua Stud o to scour the database, d scover ts schema, and generate the dec arat ve T-SQL source fi es for the new project When done, V sua Stud o d sp ays a w ndow w th ­nformat on descr b ng a the act ons t took to create the project C ck F n sh to c ose th s w ndow The new ­project s then opened n the So ut on Exp orer automat ca y (docked to the r ght, by defau t) The dbo fo der represents the dbo schema n the database Expand t, and then expand the Tab es and V ews fo ders, as shown n F gure 1-12

Figure 1-12  The source contro ed SQL Server Database Project after mport ng a database.

SSDT set up your project th s way because the Fo der Structure sett ng n the Import Database d a og (F gure 1-11) was set to Schema\Object Type Th s te s SSDT to create a fo der for each ­schema, and then a fo der for each object type (tab e, v ew, and so on) conta ned n that schema You are free to create add t ona fo ders as you extend your project Un ess you have a very spec fic or un que convent on, t s best pract ce to ma nta n a cons stent project structure based on the schema ­organ zat on n the database ke we’ve shown here

More Information  Schemas in SQL Server bear similarity to namespaces in .NET. Our simple example database has only a single namespace (dbo, short for database owner), but more complex databases typically consolidate database objects into multiple schemas. Just as namespaces are used to organize many classes in large .NET applications, schemas help manage many objects defined in large databases. Like classes and namespaces, two database objects can be assigned the same name if they are contained in two different schemas. For example, both Sales.Person and Billing. Person refer to two completely different Person tables (one in the Sales schema and one in the Billing schema). SQL Server schemas can define objects at just a single level however, whereas .NET namespaces can be nested in as many levels as desired to define an elaborate hierarchy of classes. 24  Part I  Core SQL Server Development

Taking a Snapshot Before you make any offl ne database changes, take a snapshot Th s w create a s ng e-fi e mage of the current database schema that you can refer to or revert to at any t me n the future You’ take another snapshot when you’ve comp eted a your database changes, and thereby preserve the two po nts n t me— just before, and just after, the changes are made And because they are ma nta ned as part of the project, snapshot fi es are a so protected under source contro R ght-c ck the SampleDb project n So ut on Exp orer and choose Snapshot Project After v­ a dat ng the project, V sua Stud o creates a new Snapshots fo der and, ns de the Snapshots fo der, t creates a new .dacpac fi e w th a fi ename based on the current date and t me You’ usua y want to g ve the snapshot a better name, so rename the fi e to Version1Baseline.dacpac

Using the Table Designer (Offline Database Project) W th your “base ne“ snapshot saved, you’re ready to create the new CustomerRanking tab e R ­ eca that th s s the new reference tab e based on the spreadsheet n F gure 1-7 In So ut on Exp orer, ­r ght-c ck the project’s Tab es fo der (under the dbo schema fo der) and choose Add Tab e Name the tab e CustomerRanking and c ck Add A new offl ne tab e des gner w ndow opens You’ find that t ooks and fee s exact y the same as the one n F gure 1-8 that you used when work ng on ne That’s because t is the same too , on y th s t me t’s the des gner over a mode backed by a source-contro ed project fi e (CustomerRanking.sql) rather than a mode backed by a ve tab e Because there’s no connected database, the tab e des gner has no Update button— nstead, when work ng offl ne, schema changes are saved to the project scr pt fi e for that tab e Th s n turn updates the mode , and then the same va dat on checks and ­Inte Sense you exper enced when work ng wh e connected are run aga nst the project So you can find out r ght away f and when you make a break ng change, before dep oy ng to a rea database Ear er, when you removed the CustomerRanking co umn from the Customer tab e, we ment oned that you can des gn the tab e us ng e ther the gr d n the top pane or the T-SQL code w ndow n the ­bottom pane You can a so v ew and change parts of the schema defin t on from the Propert es gr d We’ ­demonstrate a of these techn ques now as you ay out the schema of the new CustomerRanking tab e SSDT starts you off w th a new tab e that has one co umn n t an int pr mary key named Id To rename th s co umn to CustomerRankingId, se ect the co umn name Id n the top pane’s gr d, rep ace t w th CustomerRankingId, and press Enter Beneath t, add the RankName co umn, set ts data type to varchar(20), and uncheck A ow Nu s You can see that SSDT updates the T-SQL code n the bottom pane w th a CREATE TABLE statement that reflects the changes made n the top pane Add the th rd co umn by ed t ng the T-SQL code n the bottom pane Append a comma after the second co umn and type [Description] VARCHAR(200) NOT NULL As expected, the gr d n the top pane s updated to show the new Description co umn you just added n code F na y, tweak the data type us ng the Propert es gr d C ck the Description co umn n the top pane and scro to the Length property n the Propert es gr d (to d sp ay the Propert es gr d f t’s not ­current y v s b e, c ck V ew and choose Propert es W ndow) C ck the drop-down st and se ect MAX

Chapter 1  ntroduc ng SQL Server Data Too s   25

to change the data type from varchar(200) to varchar(max) When you’re done, the tab e des gner shou d ook s m ar to F gure 1-13

Figure 1-13  The tab e des gner for the new CustomerRanking tab e n an offl ne SQL Server Database Project.

Save CustomerRanking.sql and c ose the tab e des gner Th s comp etes step 2 You are now ready to add the fore gn key co umn to the Customer tab e (step 3) that jo ns to th s new C ­ ustomerRanking tab e Doub e-c ck Customer.sql n So ut on Exp orer to open the tab e des gner for the Customer tab e Use any techn que you’d ke to add a new nu ab e int co umn named CustomerRankingId ( t must be nu ab e at th s po nt, because t doesn’t have any data yet) Now you can estab sh the fore gn key re at onsh p to the CustomerRanking tab e (step 4) In the upper-r ghthand corner of the Tab e Des gner s a Context V ew area that summar zes other ­pert nent objects re ated to the tab e In the Context V ew, r ght-c ck Fore gn Keys and choose Add New ­Fore gn Key Name the new fore gn key FK Customer CustomerRanking ( t s best pract ce to ass gn fore gn key names that nd cate wh ch tab es part c pate n the re at onsh p) Then ed t the ­FOREIGN KEY temp ate code added to the T-SQL code w ndow n the bottom pane to be FOREIGN KEY (CustomerRankingID) REFERENCES CustomerRanking(CustomerRankingId) The tab e des gner w ndow shou d now ook s m ar to F gure 1-14 After rev ew ng the tab e schema, save the Customer.sql fi e and c ose the des gner

26  Part I  Core SQL Server Development

Figure 1-14  The tab e des gner for the Customer tab e after creat ng the fore gn key on CustomerRankingId.

Introducing LocalDB Your next tasks nvo ve a ter ng a v ew (step 5) and creat ng a stored procedure (step 6) It w be very he pfu to have a test SQL Server env ronment ava ab e as you mp ement these steps You don’t want to use SQL2012DEV, because that’s the “ ve“ server You need another SQL Server that can be used just for test ng offl ne Loca DB g ves you that test env ronment Th s s a new, ghtwe ght, s ng e-user nstance of SQL Server that sp ns up on demand when you bu d your project Th s s extreme y handy when ­work ng offl ne and there s no other deve opment server ava ab e to you The offic a name for th s new ­var ant of SQL Server s “SQL Express Loca DB,“ wh ch can be m s ead ng because t s d st nct from the Express ed t on of SQL Server To avo d confus on, we refer to t s mp y as “Loca DB “

Note  The new LocalDB does not support every SQL Server feature (for example, it can’t be used with FILESTREAM). However, it does support most functionality required for typical database development. Press F5 to bu d the project Th s first va dates the ent re database structure defined by the project and then dep oys t to Loca DB Note, however, that th s s just the defau t behav or; you can change the project propert es to target another ava ab e server for test ng f you requ re features not supported by Loca DB (for examp e, FILESTREAM, wh ch we cover n Chapter 8) The dep oyment s carr ed out by perform ng a schema compare between the project and L­ oca DB on the target server More prec se y, and as a ready exp a ned, mode s of the source ­project and ­target database are generated, and the schema compare actua y works on the two mode s ­Be ng your very first bu d, the database does not ex st yet on the target server, so the schema ­compare generates a scr pt that creates the who e database from scratch As you mod fy the project,

Chapter 1  ntroduc ng SQL Server Data Too s   27

s­ ubsequent bu ds w generate ncrementa change scr pts that spec fy just the act ons needed to br ng the target database back n sync w th the project Look back over n SQL Server Object Exp orer and you’ see that SSDT has started a new Loca DB nstance The host name s (localdb)\SampleDb, and t s a comp ete y separate nstance than the SQL2012DEV nstance (wh ch has not yet been updated w th the new CustomerRanking tab e and the CustomerRankingId fore gn key n the Customer tab e) F gure 1-15 shows SampleDb dep oyed to Loca DB, expanded to revea ts tab es Not ce that t does nc ude the new CustomerRanking tab e

Figure 1-15  The oca database runt me (Loca DB) after bu d ng and dep oy ng the SQL Server Database Project.

Now you have a test database to p ay around w th, but of course there’s no data n t You w add some now so that you can test the v ew you’re about to a ter and the stored procedure you’re about to create Us ng s mp e copy/paste, SSDT ets you mport sma sets of rows from any “tab e“ source ( nc ud ng M crosoft Word and Exce ) nto a database tab e that has compat b e co umns F rst, br ng n the reference data from the rank ng defin t ons prov ded by the spreadsheet n F gure 1-7 You can eas y grab the data stra ght out of the spreadsheet and dump t r ght nto the new CustomerRanking tab e Open the spreadsheet n Exce , se ect the five rows of data (comp ete 28  Part I  Core SQL Server Development

rows, not ce s or co umns), then r ght-c ck the se ect on and choose Copy Back n SQL Server Object Exp orer, r ght-c ck the CustomerRanking tab e and choose V ew Data The Ed tab e Data Gr d n SSDT opens w th a temp ate for nsert ng a new row R ght-c ck the row se ector for the new row temp ate and choose Paste (be sure to r ght-c ck the row se ector n the eft gray marg n area, and not a ce , before past ng) As shown n F gure 1-16, SSDT fa thfu y cop es the data from Exce nto the ­CustomerRanking tab e

Figure 1-16  Reference data mported nto a database tab e from Exce v a the c pboard.

You a so need a few customers to work w th Us ng the same copy/paste techn que, you w ­transfer rows from the Customer tab e n the product on database to the test database on Loca DB (for th s ­exerc se, you won’t transfer re ated order data n the OrderHeader tab e) There are on y a handfu of customers, so you’ just copy them a over Typ ca y though, you’d extract just the subset of data that prov des a representat ve samp ng good enough for test ng purposes Expand the product on server (SQL2012DEV) node n SQL Server Object Exp orer and dr down to the Customer tab e R ght-c ck the tab e and choose V ew Data Se ect a the customer rows, then r ght-c ck the se ect on and choose Copy Next, r ght-c ck the Customer tab e n the test database on (localdb)\SampleDb and choose V ew Data R ght-c ck the new row se ector and choose Paste to copy n the s x customer rows You are now at step 5, wh ch s to update the vwCustomerOrderSummary v ew Reca that th s s the same v ew you ed ted back n step 1 (wh e connected), when you removed the schema-bound reference to the o d CustomerRanking co umn that was be ng dropped from the Customer tab e W th the new reference tab e and fore gn key re at onsh p now n p ace, you w rev se the v ew once aga n (offl ne, th s t me) to jo n w th the CustomerRanking tab e on CustomerRankingId, so that t can expose the d sp ay name n the reference tab e’s RankName co umn In So ut on Exp orer, doub e-c ck vwCustomerOrderSummary.sql n the project’s V ews fo der (­under the dbo schema fo der) The v ew opens up n a new code w ndow, and your attent on may first be drawn to severa squ gg es that V sua Stud o pa nts n the v ew’s code They’re not red, ­because there s rea y noth ng s gn ficant y wrong w th the v ew, and so these are just warn ngs ­Hover the cursor over one of them to v ew the warn ng text n a too t p (you can a so see a of them sted as warn ng tems n the Error L st pane) The warn ngs nd cate that the v ew uses CustomerID (end ng n a cap ta D) to reference a co umn that s actua y defined as CustomerId (end ng n a ­owercase d) These are the same case-sens t v ty warn ngs you saw ear er when attempt ng to update the ­database w th dependency ssues Object names n SQL Server are norma y not case-sens t ve

Chapter 1  ntroduc ng SQL Server Data Too s   29

( ke VB NET), but non-defau t co at on sett ngs can change that behav or so that they are case-sens t ve ( ke C#) Th s wou d cause a prob em f you dep oyed the v ew to a SQL Server nstance configured for a ­case-sens t ve co at on of object names Add another LEFT OUTER JOIN to the v ew to add n the CustomerRanking tab e jo ned on the ­CustomerRankingId of the Customer tab e, and add RankName to the SELECT and GROUP BY ­co umn sts You want your code to be squeaky-c ean, so now s a so a good t me to reso ve those ­case-sens t v ty warn ngs Rep ace CustomerID w th CustomerId n the four p aces that t occurs (once n the SELECT co umn st, tw ce n the first JOIN, and once more n the GROUP BY co umn st) L st ng 1-3 shows the v ew defin t on after mak ng the changes Listing 1-3  The updated vwCustomerOrderSummary v ew defin t on jo n ng on the new CustomerRanking tab e.

-- Create a handy view summarizing customer orders CREATE VIEW vwCustomerOrderSummary WITH SCHEMABINDING AS SELECT c.CustomerId, c.FirstName, c.LastName, r.RankName, ISNULL(SUM(oh.OrderTotal), 0) AS OrderTotal FROM dbo.Customer AS c LEFT OUTER JOIN dbo.OrderHeader AS oh ON c.CustomerId = oh.CustomerId LEFT OUTER JOIN dbo.CustomerRanking AS r ON c.CustomerRankingId = r.CustomerRankingId GROUP BY c.CustomerId, c.FirstName, c.LastName, r.RankName

Save the vwCustomerOrderSummary.sql fi e to update the offl ne project You know that p ­ ress ng F5 now w dep oy the changed v ew to the test database on Loca DB But what f you attempt to ­execute the scr pt d rect y by press ng Ctrl+Shift+E, r ght here n the code w ndow? Go ahead and try You’ rece ve th s error message n response Msg 2714, Level 16, State 3, Procedure vwCustomerOrderSummary, Line 2 There is already an object named 'vwCustomerOrderSummary' in the database.

Here’s what happened F rst, SSDT connected the query w ndow to the (localdb)\SampleDb ­nstance Then t attempted to execute the scr pt mperat ve y aga nst the connected database, just as you’ve a ready seen w th Ctr +Sh ft+E But be ng part of an offl ne project, th s scr pt s dec arat ve and so t’s expressed as a CREATE VIEW statement The v ew a ready ex sts n the database, so the error message makes perfect sense Aga n, the proper way to update the database s to dep oy t v a an ncrementa dep oyment scr pt by debugg ng w th F5 However, you are ndeed connected to the test database on Loca DB, even though you’re ­work ng ns de the query w ndow of an offl ne project that hasn’t yet been dep oyed to Loca DB Th s means that you can actua y test the v ew before you dep oy t Se ect a the text from SELECT unt the end of the scr pt (that s, eave on y the CREATE VIEW port on of the w ndow unse ected) and press Ctrl+Shift+E aga n Th s t me, you get a much better resu t On y the chosen SELECT statement executes, wh ch s perfect y va d T-SQL for the connected database The query resu ts show you what 30  Part I  Core SQL Server Development

the v ew s go ng to return, and you got that nformat on w thout hav ng to dep oy first In th s mode, you are actua y work ng connected and offl ne s mu taneous y! You can se ect any T-SQL to nstant y execute t, test and debug stored procedures, and even get execut on p ans, a wh e “offl ne “

Refactoring the Database The v ew s ready to be dep oyed, but now you dec de to change some names first Customers are the on y th ng be ng ranked, so shorten ng the tab e name CustomerRanking to Ranking and ­co umn names CustomerRankingId to RankingId s go ng to make your T-SQL more readab e (wh ch s ­mportant!) W thout the proper too ng, t can be very ted ous to rename objects n the database But the refactor ng capab t es prov ded by SSDT make th s remarkab y easy In the new LEFT OUTER JOIN you just added, r ght-c ck on the CustomerRanking tab e reference, and then choose Refactor, Rename Type Ranking for the new tab e name and c ck OK You are presented w th a prev ew w ndow (F gure 1-17) that w appear very fam ar f you’ve ever used the refactor ng features n V sua Stud o w th ord nary NET projects

Figure 1-17  Prev ew ng changes before refactor ng s app ed to the database.

Th s d a og shows a the references to the CustomerRanking tab e that w be changed to ­Ranking when you c ck App y (not ce that checkboxes are prov ded so that you can a so choose wh ch ­references shou d get updated and wh ch shou d not) Scro through the change st to prev ew each one, and then c ck App y to mmed ate y nvoke the rename operat on Every affected project fi e s updated accord ng y, but V sua Stud o won’t actua y rename project fi es themse ves The project scr pt fi e defin ng the new y renamed Ranking tab e s st named CustomerRanking.sql R ght-c ck the fi e n So ut on Exp orer, choose Rename, and change the fi ename to Ranking.sql Now rename the pr mary key co umn n the Ranking tab e a ong w th ts correspond ng fore gn key co umn n the Customer tab e, both of wh ch are current y named CustomerRankingId The two key co umns are referenced on the same LEFT OUTER JOIN ne, so th s w be easy R ght-c ck the r.CustomerRankingId key co umn n the jo n and choose Refactor, Rename Type RankingId for the new name, c ck OK, prev ew the changes, and c ck App y to update the pr mary key co umn name

Chapter 1  ntroduc ng SQL Server Data Too s   31

n the Ranking tab e Then repeat for the c.CustomerRankingId key co umn to update the fore gn key co umn name n the Customer tab e (the actua order n wh ch you refactor the co umn names n these tab es s mmater a ) There’s one more th ng to rename, and that’s the fore gn key defin t on n the Customer tab e Th s sn’t str ct y necessary of course, but the (se f- mposed) convent on to name fore gn keys ­defin t ons after the tab es they jo n d ctates that FK Customer CustomerRanking shou d be renamed to ­FK ­Customer Ranking The Customer tab e s spec fied first n the v ew’s FROM c ause, so r ght-c ck on t now and choose Go to Defin t on Th s nav gates d rect y to the Customer tab e defin t on n a new query w ndow In the CONSTRAINT c ause, r ght-c ck FK Customer CustomerRanking and choose Refactor, Rename Type FK Customer Ranking for the new name, c ck OK, prev ew the changes (just one, n th s case), and c ck App y You’re a set to dep oy the changes w th another bu d, so press F5 once aga n After the bu d comp etes, c ck Refresh n the SQL Server Object Exp orer too bar and ook at the test database runn ng under (localdb)\SampleDb to confirm that the CustomerRanking tab e has been renamed to Ranking R ght-c ck the Ranking tab e and choose V ew Data to confirm that a the data n the renamed tab e s ntact When you rename objects n a SQL Server Database Project, SSDT generates a change scr pt w th correspond ng EXECUTE sp rename statements n t, as opposed to dropp ng one object and creat ng another (wh ch, for tab es, wou d resu t n rrevocab e data oss) So the too does the r ght th ng, re y ng u t mate y on the SQL Server sp rename stored procedure to proper y change the object’s name nterna y w th n the database It’s t me to create the stored procedure that ranks the customers F rst, create a Stored ­Procedures fo der beneath the dbo fo der n So ut on Exp orer (to do th s, r ght-c ck the dbo fo der, and choose Add New Fo der) Th s fo der wou d have a ready been created when you mported the database nto the project, had there been any stored procedures n the database at the t me Then r ght-c ck the new Stored Procedures fo der and choose Add Stored Procedure Name the stored procedure uspRankCustomers and c ck Add SSDT creates a new fi e named uspRankCustomers. sql and opens t n a new T-SQL ed tor w ndow Rep ace the temp ate code w th the scr pt shown n ­L st ng 1-4 and save t, but keep the w ndow open Now press F5 to perform another bu d and push the new stored procedure out to the test database on Loca DB Listing 1-4  The stored procedure to rank customers based on the r tota order amount.

CREATE PROCEDURE uspRankCustomers AS DECLARE @CustomerId int DECLARE @OrderTotal money DECLARE @RankingId int

DECLARE curCustomer CURSOR FOR SELECT CustomerId, OrderTotal FROM vwCustomerOrderSummary



OPEN curCustomer FETCH NEXT FROM curCustomer INTO @CustomerId, @OrderTotal

32  Part I  Core SQL Server Development

WHILE @@FETCH_STATUS = 0 BEGIN IF @OrderTotal = 0 SET @RankingId = 1 ELSE IF @OrderTotal < 100 SET @RankingId = 2 ELSE IF @OrderTotal < 1000 SET @RankingId = 3 ELSE IF @OrderTotal < 10000 SET @RankingId = 4 ELSE SET @RankingId = 5 UPDATE Customer SET RankingId = @RankingId WHERE CustomerId = @CustomerId END

FETCH NEXT FROM curCustomer INTO @CustomerId, @OrderTotal

CLOSE curCustomer DEALLOCATE curCustomer

Th s stored procedure “ranks“ the customers, exam n ng them nd v dua y and ass gn ng each a va ue based on the r order tota It does th s by open ng a cursor aga nst the order summary v ew, wh ch returns one row per customer w th the r nd v dua orders aggregated nto a s ng e order tota Based on the do ar va ue of the tota , t then updates the customer w th a rank ng va ue between one and five Then t advances to the next customer unt t reaches the end of the cursor As ment oned at the outset, th s so ut on may be a b t contr ved (and we’re sure you can th nk of a better approach), but t su ts our demonstrat on purposes here just fine

Testing and Debugging Are you n the hab t of runn ng new or untested code on ve databases? We certa n y hope not Though you cou d, you shou d not s mp y push a of the changes you’ve made n the project (steps 2 through 6) back to the ve database on SQL2012DEV, and then run the stored procedure there for the very first t me It’s much safer to test the stored procedure offl ne first w th Loca DB You w now earn how to do that us ng the ntegrated debugger n V sua Stud o Then you can confident y dep oy everyth ng back to SQL2012DEV, and fina y (step 7), to the c oud! The uspRankCustomers stored procedure s st open n the code ed tor C ck ns de the eft marg n on the OPEN curCustomer ne to set a breakpo nt just before the cursor s opened The breakpo nt appears as a red bu et n the marg n where you c cked Th s s exact y how breakpo nts are set n C# or VB NET code, and SSDT now de vers a s m ar debugg ng exper ence for T-SQL code as we In SQL Server Object Exp orer, expand the Stored Procedures node ( ocated beneath Programmab ty, just as n SSMS) for SampleDb beneath the Loca DB nstance R ght-c ck the Stored Procedures node, choose Refresh, and you w see the uspRankCustomers stored procedure you just dep oyed R ghtc ck on the stored procedure and choose Debug Procedure SSDT generates an EXEC statement to nvoke uspRankCustomers and opens t n a new query w ndow The debugger s a ready started, and s paused on the USE [SampleDb] statement above the EXEC statement



Chapter 1  ntroduc ng SQL Server Data Too s   33

More Info  The debugging session began instantly in this case because the ­uspRankCustomers stored procedure being debugged has no parameters. When stored procedure parameters are expected, SSDT will first display a dialog to solicit the parameter values, and then plug those values into the EXEC statement before starting the debugger. Press F5 to cont nue execut on The debugger reaches the EXEC statement, enters the stored procedure, and then breaks on the OPEN curCustomer statement where you prev ous y set the breakpo nt Now start s ng e stepp ng through the stored procedure’s execut on w th the debugger’s F10 keystroke Press F10 three t mes to step over the next three statements Th s opens the cursor, fetches the first customer from t, and you are now paused on the first IF statement that tests the first customer’s order tota for zero do ars Ear er, you cop ed s x customer rows from the SQL2012DEV database to Loca DB, but we s­ pec fica y nstructed you not to copy any order data So th s oop w terate each customer, and (based on an order tota of zero do ars) ass gn a rank ng va ue of 1 for every customer Rather than nterrupt ng your debugg ng sess on now to mport some samp e order data and start over, you w use the debugger’s Loca s w ndow to s mu ate non-zero order tota s for the first two customers C ck the Debug menu, and then choose W ndows Loca s In the Loca s w ndow, you can see that @CustomerId s 1 (th s s the first customer) and ­@­OrderTotal s 0 (expected, because there’s no samp e order data) @RankingId s not yet set, but f you a ow execut on to cont nue as- s, the customer w be ranked w th a 1 Doub e-c ck the 0 0000 va ue for @OrderTotal n the Loca s w ndow, type 5000 and press Enter Now the stored procedure th nks that the customer actua y has $5,000 n tota orders Press F10 to s ng e step Because @OrderTotal no onger equa s zero, execut on advances to the next IF cond t on that tests the order tota for ­be ng under $100 Press F10 aga n and execut on advances to the next IF cond t on that tests for under $1,000 Press F10 once more to reach the IF cond t on test ng for under $10,000 Th s ­cond t on y e ds true (there are $5,000 n tota orders), so press ng F10 to s ng e step once more advances to the SET statement that ass gns a rank ng va ue of 4 Th s s the correct va ue for orders n the range of $1,000 to $10,000 F gure 1-18 shows the debugg ng sess on paused at th s po nt Cont nue press ng F10 to s ng e step through the rema n ng SET, UPDATE, and FETCH NEXT s­ tatements, and then back up aga n to the first IF statement test ng the second customer’s order tota va ue for zero do ars Use the Loca s w ndow to fake another amount; th s t me change @OrderTotal to 150 S ng e step a few more t mes to make sure that th s resu ts n the stored procedure ass gn ng a rank ng va ue of 3 th s t me, wh ch s the correct va ue for orders n the range of $100 to $1,000 Now press F5 to et the stored procedure fin sh process ng the rest of the customers w th no more ­ntervent on on your part When the stored procedure comp etes execut on, r ght-c ck the Customer tab e n SQL Server Object Exp orer (be sure to p ck the Loca DB nstance and not SQL2012DEV) and choose V ew Data The tab e data confirms that the first customer’s rank ng was set to 4, the second customer’s rank ng was set to 3, and a the other customer rank ngs were set to 1 ( f you a ready have a Customer tab e

34  Part I  Core SQL Server Development

w ndow open from before, the prev ous va ues w button n the too bar to update the d sp ay)

st

be d sp ayed; you need to c ck the Refresh

Figure 1-18  T SQL debugg ng sess on of a stored procedure n V sua Stud o.

Th s was by no means exhaust ve test ng, but t w suffice for demonstrat on purposes The key po nt s that SSDT prov des an env ronment you can use for debugg ng and test ng as you deve op your database offl ne, unt you’re ready to dep oy to a ve env ronment (as you are now)

Comparing Schemas You are ready to dep oy to the database back to the ve server on SQL2012DEV As you may have correct y surm sed by now, the process s fundamenta y the same as work ng offl ne w th Loca DB each t me F5 s pressed SSDT runs a schema compare to generate a change scr pt The project ­propert es (by defau t) spec fy a connect on str ng that po nts to Loca DB So bu d ng w th F5 uses the test database as the target for the schema compare w th the project as the source, and then executes the generated change scr pt aga nst the test database on Loca DB Th s a happens as a comp ete y unattended set of operat ons every t me you press F5 Now you w carry out those very same steps once aga n, on y th s t me you’ get more nvo ved n the process In part cu ar, you w spec fy the ve SQL2012DEV nstance as the target for the schema compare, rather than Loca DB You w a so rev ew the resu ts of the schema compare, and have the chance to choose to dep oy or not dep oy spec fic detected changes F na y, you’ get the ­opportun ty to v ew, ed t, save, and execute the change scr pt after t s generated, rather than hav ng

Chapter 1  ntroduc ng SQL Server Data Too s   35

t execute automat ca y So there’s a b t more ntervent on nvo ved n the process now, but you want t that way The schema compare process tse f s the same as the F5 bu d—you just get to exerc se more contro over t to support d fferent dep oyment scenar os R ght-c ck the SampleDb project n So ut on Exp orer and choose Schema Compare to open a new schema compare w ndow You need to spec fy a source and target for any schema compare, natura y Because you aunched the w ndow from the SQL Server Database Project context menu n So ut on Exp orer, V sua Stud o sets the source to the project automat ca y, eav ng you to set just the target To set the target, c ck ts drop-down st and choose Se ect Target to d sp ay the Se ect Target Schema d a og, shown n F gure 1-19

Figure 1-19  SSDT ets you choose between projects, databases, and snapshots for schema compare operat ons.

Not ce how you can choose between three schemas for the target—a project, a database, or a data-t er app cat on fi e (snapshot) The same cho ces are a so supported for the source, a though the SQL Server Database Project was assumed as the source automat ca y n th s case Any comb nat on of source and target source schemas s supported; SSDT s mp y creates source and target mode s from your cho ce of back ngs Then, work ng off the mode s, t shows you the d fferences and generates a change scr pt for you Th s flex b ty s a major benefit of mode -based database deve opment w th SSDT The Se ect Target Schema d a og has correct y assumed that you want to use a database as the target A you need to do s choose the ve SampleDb database runn ng on SQL2012DEV C ck New Connect on to open a Connect on Propert es d a og, type your actua mach ne name for the server name (wh ch s SQL2012DEV n the current examp e), choose SampleDb from the database name drop-down st, and c ck OK (V sua Stud o w remember th s connect on for the future, and make t ava ab e for reca n the database dropdown the next t me you run a schema compare ) C ck OK once more, and then c ck the Compare button n the too bar to start the schema compare It takes just a few moments for the operat on to comp ete When t fin shes, the schema compare d sp ays a the changes you’ve made offl ne s nce creat ng the project (steps 2 through 6) The report ets you see each added, changed, and dropped object, and t can be organ zed by type (tab e, v ew, and so on), schema, or act on (change, add, or de ete) Se ect ng any object n the top pane presents ts T-SQL dec arat on n the bottom pane, s de-by-s de (source on the eft, target on the r ght), w th synchron zed scro bars The T-SQL s co or-coded to h gh ght every one of the object’s d fferences 36  Part I  Core SQL Server Development

If des red, you can exc ude spec fic objects from the change scr pt (wh ch hasn’t been generated yet) by c ear ng the r nd v dua checkboxes back up n the top pane Se ect the vwCustomerOrderSummary v ew n the top pane to see the source and target ­vers ons of the v ew n the bottom pane As shown n F gure 1-20, the r ch v sua d sp ay rendered by the schema compare too makes t easy to dent fy a the changes made to the v ew

Figure 1-20  V ew ng the schema d fferences between a SQL Server Database Project and a ve database.

As w th the tab e des gner, you can choose to update the ve database mmed ate y by ­generat ng and runn ng the change scr pt w thout prev ew ng t Or you can choose to be more caut ous, and just generate the change scr pt Then you can v ew, ed t, and u t mate y dec de whether or not to execute t Your confidence eve shou d be very h gh by now, so just c ck the Update button n the too bar (and then c ck Yes to confirm) to et t run SSDT updates the target database and d sp ays a ­comp et on message when t’s done C ck OK to d sm ss the message The d fferences from before the update are st d sp ayed n the w ndow, now d mmed n gray (you can c ck Compare aga n to confirm that there are no onger any d fferences between the project and the ve database on SQL2012DEV) In SQL Server Object Exp orer, dr down on SampleDb under SQL2012DEV (or refresh a ready dr ed down nodes) to ver fy that t reflects a the work performed n the project for steps 2 through 6 on your task st You are a most ready to run the new uspRankCustomers stored procedure and update the ve ­ ustomer tab e, but there’s one more th ng to do before that A though the dep oyment created the C schema of the Ranking tab e, t d dn’t copy ts data You need to mport the reference data from the spreadsheet n F gure 1-7 aga n, th s t me nto the ve database on SQL2012DEV You can certa n y use the same copy/paste tr ck we showed ear er when you mported the spreadsheet nto the test database on Loca DB, but we’ take th s opportun ty now to show you how to scr pt tab e data w th SSDT

Chapter 1  ntroduc ng SQL Server Data Too s   37

Note The Ranking table is a typical example of reference data. Databases often rely on r­ eference data, which are usually small sets of read-only entries, to help define their structure. Although technically not schema, it would certainly be convenient to mark the contents of the Ranking table as such, so that the five entries it contains become part of the SSDT database model and travel with the schema definition wherever you deploy it to. Unfortunately, this feature could not make it in time for the final product release, but Microsoft is evaluating plans to add this capability to a future version of SSDT. Under the (localdb)\SampleDb node (the Loca DB nstance) n SQL Server Object Exp orer, r­ ght-c ck the Ranking tab e and choose V ew Data to open a w ndow show ng the five rows n the tab e Next, c ck the Scr pt button on the too bar (the next to the ast button) SSDT generates INSERT statements for the five rows of data n the Ranking tab e, and d sp ays them n a new query w ndow You want to execute these INSERT statements n a query w ndow connected to the ve database on SQL2012DEV, so se ect a the INSERT statements and press Ctrl+C to copy them to the c pboard Then under the SQL2012DEV node n SQL Server Object Exp orer, r ght-c ck the SampleDb database and choose New Query Press Ctrl+V to paste the INSERT statements nto the new query w ndow and then press Ctrl+Shift+E to execute them The reference data has now been mported nto the ve database and you’re ready to update the customers In the same query w ndow, type EXEC uspRankCustomers, se ect the text of the statement, and press Ctrl+Shift+E The stored procedure executes and updates the customers (You can gnore the nu va ue warn ng; t refers to the SUM aggregate funct on n the v ew, wh ch does not affect the resu t ) To see the fina resu t, type SELECT * FROM vwCustomerOrderSummary, se ect t, and press Ctrl+Shift+E once aga n As shown n F gure 1-21, each customer’s rank ng s correct y ass gned based on the r tota order amount

Figure 1-21  V ew ng the fina resu ts of offl ne deve opment n the ve database.

38  Part I  Core SQL Server Development

Publishing to SQL Azure The market ng team’s ast request was that you dep oy a copy of the database to SQL Azure To ensure that the database s c oud-ready, you just need to te SSDT that you are target ng the SQL Azure p atform by chang ng a property of the project Then, f any SQL Azure compat b ty ssues are dent fied, they can be reso ved before you dep oy As you m ght expect by now, you w use the very same techn ques you’ve earned throughout th s chapter to dep oy the SQL Server Database Project to SQL Azure

Note  Our discussion in this section assumes you already have an available SQL Azure server instance that you can publish to. SQL Azure server names always begin with a unique identifier randomly assigned just to you, followed by .database.windows.net. Chapter 12 (which is dedicated to SQL Azure) explains how to use the Azure Management Portal to create your own cloud databases on SQL Azure, after setting up a Windows Azure account. R ght-c ck the SampleDb project n So ut on Exp orer and choose Propert es In the Project ­Sett ngs tab, you’ not ce that the Target P atform s current y set to SQL Server 2012 Change t to SQL Azure as shown n F gure 1-22, press Ctrl+S to save the propert es, and then c ose the propert es w ndow

Figure 1-22  Chang ng the target p atform of a SQL Server Database Project to SQL Azure.



Chapter 1  ntroduc ng SQL Server Data Too s   39

Now press F5 to once aga n bu d the project and dep oy t to Loca DB The bu d fa s, and the ­Error L st pane shows the fo ow ng error SQL71560: Table [dbo].[OrderHeader] does not have a clustered index. required for inserting data in this version of SQL Server.

Clustered indexes are

Th s error nforms you that the OrderHeader tab e s m ss ng a c ustered ndex The astute reader m ght have not ced back n L st ng 1-1 that the OrderHeaderId co umn n th s tab e does not spec fy PRIMARY KEY ( ke the Customer tab e does on ts CustomerId co umn), and so OrderHeader has no c ustered ndex Th s was an overs ght that m ght not have been caught so eas y because tab es n onprem se ed t ons of SQL Server do not requ re a c ustered ndex But SQL Azure databases ­abso ute y requ re a c ustered ndex on every tab e, so now that you’re target ng the c oud spec fica y, the probem s brought to your attent on ns de the project Th s s a qu ck and easy fix to make us ng the tab e des gner Back n the SQL Server Database ­ roject ( n So ut on Exp orer), doub e-c ck the OrderHeader.sql tab e (under the dbo and Tab es P fo ders) to open the project’s defin t on of the tab e n the des gner R ght-c ck the OrderHeaderId co umn, choose Set Pr mary Key, save, and then c ose the tab e des gner The pr mary key defin t on resu ts n the creat on of a c ustered ndex on the tab e Th s reso ves the ssue, and you’ see the error d sappear from the Error L st pane mmed ate y Now that you know the database s c oud-compat b e, you’re ready to dep oy t to SQL Azure R ght-c ck the SQL Server Database Project n So ut on Exp orer and choose Pub sh to d sp ay the Pub sh Database d a og C ck Ed t, enter the server and og n nformat on for your SQL Azure ­database, and c ck OK F gure 1-23 shows the Pub sh Database d a og w th the target connect on str ng po nt ng to a SQL Azure database

Figure 1-23  The Pub sh Database d a og set to dep oy the project to a SQL Azure target nstance.

As we’ve been not ng a a ong, you can scr pt the dep oyment w thout execut ng t by c ck ng Generate Scr pt But you’re ready to dep oy to SQL Azure r ght now C ck Pub sh, and V sua Stud o 40  Part I  Core SQL Server Development

sp ns up the same fam ar process It performs a schema compare between the source SQL Server ­ atabase Project and target SQL Azure nstance, and then generates and executes the resu t ng D change scr pt on the target As w th your very first bu d to Loca DB, the database does not ex st yet on the target, so the change scr pt creates the who e database n the c oud from scratch ­Subsequent dep oyments w generate ncrementa change scr pts that spec fy just the act ons needed to synchron ze the SQL Azure database w th the project Dur ng the dep oyment process, the Data Too s Operat ons w ndow n V sua Stud o prov des a dynam c d sp ay of what’s happen ng F gure 1-24 shows the Data Too s Operat ons w ndow after the pub sh process s comp ete

Figure 1-24  The Data Too s Operat ons pane reports a the act ons taken to dep oy to SQL Azure.

A rea y n ce feature of the Data Too s Operat ons pane s the ab ty to see the scr pts that were just executed ns de query w ndows and v ew the r execut on resu ts C ck the var ous nks (V ew Prev ew, V ew Scr pt, and V ew Resu ts) to rev ew the dep oyment you just ran After dep oy ng, SSDT automat ca y adds your SQL Azure server nstance to the SQL Server Object Exp orer, as shown n F gure 1-25 You can dr down on SQL Azure databases n SQL Server Object Exp orer and work w th them us ng the very same deve opment too s and techn ques that we’ve shown throughout th s chapter It’s exact y the same mode -based, buffered exper ence you have w th connected deve opment of on-prem se databases, on y now t’s a SQL Azure database ­back ng the mode Thus, SQL Server Object Exp orer funct ons as a s ng e access po nt for connected ­deve opment aga nst any SQL Server database, wherever t’s ocated You’ve used SSDT to successfu y mp ement a the tasks to fu fi your requ rements Before conc ud ng your work, take another snapshot R ght-c ck the project n So ut on Exp orer one ast t me and choose Snapshot Project SSDT ser a zes the database mode (based on the ­project’s current state) nto another .dacpac fi e n the Snapshots fo der, wh ch you shou d rename to ­Version1Complete.dacpac Now your project has two snapshots, Version1Baseline.dacpac and Version1Complete.dacpac, and each represents the database structure at two d fferent po nts n t me The co ect on w grow over t me as you take new snapshots dur ng future deve opment, and thus your project accumu ates an h stor ca account of ts database structure as t changes w th each new vers on And because any snapshot can serve as e ther the source or target mode of a schema compare operat on, t’s very easy



Chapter 1  ntroduc ng SQL Server Data Too s   41

to d fference between any two po nts n t me, or between any s ng e po nt n t me and e ther a ve database (on-prem se or SQL Azure) or an offl ne SQL Server Database Project

Figure 1-25  A SQL Azure database connected n SQL Server Object Exp orer.

Adopting SSDT No too s perfect, and SSDT s no except on Yet even as we ca out those areas where the too s ack ng, we’ st emphas ze what b g be evers we are n th s new techno ogy, and that we great y encourage SSDT adopt on over trad t ona database deve opment methods The SSDT team has done a fantast c job w th the mode -based des gn, and there s a ot more too ng st that can be prov ded by everag ng the mode ’s r ch metadata, such as database d agrams and query des gners that are not yet prov ded There s a so no spat a v ewer to graph ca y d sp ay spat a query resu ts, such as the one prov ded n SQL Server Management Stud o (we cover spat a quer es and the spat a v ewer n Chapter 9) A though SSDT s nt mate y aware of database schema, t does not prov de data-or ented f­ unct ona ty So t can’t generate data or compare data n the database, nor does t support ­database un t test ng These are mportant features supported by the V sua Stud o Database Profess ona ­ed t on (DbPro) that are st m ss ng from SSDT Th s means that, a though SSDT s pos t oned to ­obso esce ­DbPro, that won’t happen unt t ach eves par ty w th key components of the DbPro feature set

42  Part I  Core SQL Server Development

Summary Th s chapter began w th a h gh- eve overv ew descr b ng the many cha enges deve opers face ­work ng w th databases dur ng the deve opment process Through hands-on exerc ses, you then saw how the new SQL Server Data Too s (SSDT) prov des features that can he p you tack e those ­cha enges You worked through a number of scenar os us ng SSDT for connected deve opment, offl ne ­ eve opment w th the oca database runt me (Loca DB), source contro , debugg ng, and test ng—and d then dep oyed to a oca env ronment as we as the c oud—a from w th n V sua Stud o A ong the way, you earned how to use many mportant SSDT features, such as schema compare, refactor ng, snapshot vers on ng, and mu t -p atform target ng A though you can a ways earn more about the new too s just by us ng them, th s chapter has prepared you to use SSDT as you encounter the many cha eng ng SQL Server deve opment tasks that e ahead



Chapter 1  ntroduc ng SQL Server Data Too s   43

C hapter 2

T-SQL Enhancements —Leonard Lobel

A

s SQL Server evo ves, M crosoft cont nues to mprove and enhance Transact SQL (T-SQL)—the nat ve procedura programm ng anguage that deve opers have been us ng s nce the b rth of the product Today, T-SQL ma nta ns ts ro e as the pr mary anguage for programm ng the re at ona database eng ne, even as numerous other capab t es and programmab ty po nts are added to the SQL Server stack w th each new re ease

It s true that SQL Server Common Language Runt me ntegrat on (SQL CLR) a ows you to use C# or ­V sua Bas c (VB) NET as an a ternat ve to T-SQL for wr t ng stored procedures, tr ggers, and other ­database objects Th s mportant capab ty was added n SQL Server 2005, and Chapter 3 covers SQL CLR n deta You w see that SQL CLR s a great so ut on when the r ch, managed env ronment ­afforded by the NET Framework can be everaged advantageous y on the database server ­Interna y, SQL Server everages SQL CLR to mp ement the hierarchyid (Chapter 7) and geometry/­geography (Chapter 9) data types Pr or to SQL CLR, extended stored procedures were the pr mary means for ­programm ng funct ona ty that cou d not be ach eved us ng T-SQL a one Thus, SQL CLR stored ­procedures are certa n y recommended as a safe rep acement for extended stored procedures (wh ch have the potent a to crash SQL Server f they encounter an error) But n a most a other cases, you w st want to use T-SQL as your pr mary anguage for ­ rogramm ng the most effic ent set-based quer es and stored procedures at the database eve In p th s chapter, we’ beg n w th n-depth coverage of these T-SQL features first ntroduced n SQL Server 2008 ■

Tab e-va ued parameters (TVPs)

■

Date and t me data types

■

The MERGE statement

■

The INSERT OVER DML syntax

■

The GROUPING SETS operator

We’ then exp ore these new T-SQL features n SQL Server 2012 ■

W ndow ng enhancements w th the OVER c ause

■

New T-SQL funct ons

45

■

Error hand ng mprovements w th the THROW statement

■

Server-s de resu t set pag ng w th OFFSET and FETCH NEXT

■

Sequent a number generat on w th the SEQUENCE object

■

Metadata d scovery

Note  You can use either SQL Server Management Studio (SSMS) or SQL Server Data Tools (SSDT), which we cover in Chapter 1, to run the code in this chapter.

Table-Valued Parameters As the name mp es, a tab e-va ued parameter (TVP) ets you pass an ent re set of rows as a s ng e parameter to T-SQL stored procedures and user-defined funct ons (UDFs) Th s s extreme y usefu n and of tse f, but arguab y the most compe ng facet of TVPs s the r ab ty to marsha an ent re set of rows across the network, from your NET c ent to your SQL Server database, w th a s ng e stored procedure ca (one roundtr p) and a s ng e tab e-va ued parameter Pr or to SQL Server 2008, deve opers were forced to resort to c ever hacks n an effort to reduce mu t p e roundtr ps nto one when nsert ng mu t p e rows—us ng techn ques such as XML, de m ted or encoded text, or even (gasp) accept ng hundreds (up to 2100!) of parameters But spec a og c then needs to be mp emented for packag ng and unpackag ng the parameter va ues on both s des of the w re Worse, the code to mp ement that og c s often gnar y and unma nta nab e None of those techn ques even come c ose to the e egance and s mp c ty of us ng TVPs, wh ch offer a nat ve ­so ut on to th s prob em

More Than Just Another Temporary Table Solution A TVP s based on a user-defined table type, wh ch you create to descr be the schema for a set of rows that can be passed to stored procedures and UDFs It’s he pfu to beg n understand ng TVPs by first compar ng them to s m ar “set” constructs, such as tab e var ab es, temp tab es, and Common Tab e Express ons (CTEs) A of these prov de a source of tabu ar data that you can query and jo n aga nst, so you can treat a TVP, tab e var ab e, temporary tab e, or CTE just ke you wou d an ord nary tab e or v ew n v rtua y any scenar o CTEs and tab e var ab es store the r row data n memory—assum ng reasonab y s zed sets that don’t overflow the RAM cache a ocated for them, n wh ch case, they do push the r data nto tempdb In ­contrast, a TVP’s data s a ways stored n tempdb When you first popu ate a TVP, SQL Server creates a tab e n tempdb to back that TVP as t gets passed from one stored procedure (or UDF) to another Once the stack unw nds and the TVP fa s out of scope n your T-SQL code, SQL Server c eans up tempdb ­automat ca y You never nteract d rect y w th tempdb, because TVPs prov de a tota abstract on over t

46  Part 1  Core SQL Server Development

The true power of TVPs es n the ab ty to pass an ent re tab e (a set of rows) as a s ng e ­ arameter from c ent to server, and between your T-SQL stored procedures and user-defined p ­funct ons Tab e var ab es and temporary tab es, on the other hand, cannot be passed as parameters CTEs are m ted n scope to the statement fo ow ng the r creat on and are therefore nherent y ­ncapab e of be ng passed as parameters Reusab ty s another s de benefit of TVPs The schema of a TVP s centra y ma nta ned, wh ch s not the case w th tab e var ab es, temporary tab es, and CTEs You define the schema once by creat ng a new user-defined type (UDT) of type table, wh ch you do by app y ng the AS TABLE c ause to the CREATE TYPE statement, as shown n L st ng 2-1 Listing 2-1  Defin ng the schema for a user defined tab e type.

CREATE TYPE CustomerUdt AS TABLE (Id int, CustomerName nvarchar(50), PostalCode nvarchar(50))

Th s statement creates a new user-defined tab e type named CustomerUdt w th three co umns TVP var ab es of type CustomerUdt can then be dec ared and popu ated w th rows of data that fit th s schema, and SQL Server w store and manage those rows n tempdb beh nd the scenes These var ab es can be passed free y between stored procedures—un ke regu ar tab e var ab es, wh ch are stored n RAM beh nd the scenes and cannot be passed as parameters When TVP var ab es dec ared as CustomerUdt fa out of scope and are no onger referenced, the under y ng data n tempdb support ng the TVP s de eted automat ca y by SQL Server You can see that, n fact, a TVP s essent a y a user-defined tab e type Popu ated nstances of th s type can be passed on as parameters to stored procedures and user-defined funct ons—someth ng you st can’t do w th a regu ar tab e var ab e Once the tab e type s defined, you can create stored procedures w th parameters of that type to pass an ent re set of rows us ng TVPs TVP types are d sp ayed n V sua Stud o’s SQL Server Object Exp orer n the User-Defined ­Table Types node beneath Programmability Types, as shown n F gure 2-1 (as t does n SQL Server ­Management Stud o’s Object Exp orer) There are many pract ca app cat ons for pass ng ent re sets of data around as parameters, and we’ exp ore a number of them n the rest of th s sect on

Submitting Orders A typ ca scenar o n wh ch TVPs can be app ed s an order entry system When a customer p aces an order, a new order row and any number of new order deta rows must be created n the database Trad t ona y, th s m ght be accomp shed by creat ng two stored procedures—one for nsert ng an order row and one for nsert ng an order deta row The app cat on wou d nvoke a stored p ­ rocedure



Chapter 2  T SQL Enhancements   47

ca for each nd v dua row, so for an order w th 20 deta s, there wou d be a tota of 21 stored ­procedure ca s (1 for the order and 20 for the deta s) There cou d of course be even arger orders w th many more than 20 deta s As a resu t, numerous roundtr ps are made between the app cat on and the database, each one carry ng on y a s ng e row of data

Figure 2-1  User defined tab e types that can be used for TVPs d sp ayed n SQL Server Object Exp orer.

Enter TVPs Now you can create a s ng e stored procedure w th just two TVPs, one for the order row and one for the order deta s rows The c ent can now ssue a s ng e ca to th s stored procedure, pass ng to t the ent re order w th a ts deta s, as shown n L st ng 2-2

Note  The code in Listing 2-2 assumes that the Order and OrderDetail tables already exist, and that the OrderUdt and OrderDetailUdt table types have already been created with a column schema that matches the tables. Listing 2-2  Creat ng a stored procedure that accepts TVPs.

CREATE PROCEDURE uspInsertNewOrder (@OrderTvp AS OrderUdt READONLY, @OrderDetailsTvp AS OrderDetailUdt READONLY)

48  Part 1  Core SQL Server Development

AS INSERT INTO [Order] SELECT * FROM @OrderTvp INSERT INTO [OrderDetail] SELECT * FROM @OrderDetailsTvp

As you can see, th s code nserts nto the Order and OrderDetail tab es d rect y from the rows passed n through the two TVPs You are essent a y perform ng a bu k nsert w th a s ng e ca , rather than nd v dua nserts across mu t p e ca s wrapped n a transact on We’ now take ook at the bu k nsert poss b t es for TVPs and how to create, dec are, popu ate, and pass TVPs n T-SQL Then we’ demonstrate how to popu ate TVPs and pass them across the ­network from NET c ent app cat on code to stored procedures us ng ADO NET

Using TVPs for Bulk Inserts and Updates Here’s an examp e of a stored procedure that you can create n the AdventureWorks2012 database that accepts a TVP and nserts a of the rows that get passed n through t nto the Product.Location tab e By creat ng a user-defined tab e type named LocationUdt that descr bes the schema for each row passed to the stored procedure, any code can ca the stored procedure and pass to t a set of rows for nsert on nto Product.Location us ng a s ng e parameter typed as LocationUdt F rst, create the user-defined tab e data type LocationUdt, as shown n L st ng 2-3 Listing 2-3  Creat ng the LocationUdt tab e type to be used for bu k operat ons w th TVPs.

CREATE TYPE LocationUdt AS TABLE( LocationName varchar(50), CostRate int)

Now a TVP var ab e of th s type can be dec ared to ho d a set of rows w th the two co umns ­LocationName and CostRate These rows can be fed to a stored procedure by pass ng the TVP var ab e nto t The stored procedure can then se ect from the TVP just ke a regu ar tab e or v ew and thus use t as the source for an INSERT INTO…SELECT statement that appends each row to the Product. Location tab e Rows added to Product.Location requ re more than just the two fie ds for the ocat on name and cost rate The tab e a so needs va ues for the ava ab ty and mod fied date fie ds, wh ch you’ et the stored procedure hand e What you’re do ng here s defin ng a schema that can prov de a subset of the requ red Product.Location fie ds (Name and CostRate), for pass ng mu t p e rows of data to a stored procedure that prov des va ues for the rema n ng requ red fie ds (Availability and ­ModifiedDate) In the examp e, the stored procedure sets Availability to 0 and ModifiedDate to the



Chapter 2  T SQL Enhancements   49

GETDATE funct on on each row of data nserted from the TVP (passed n as the on y parameter) that prov des the va ues for Name and CostRate, as shown n L st ng 2-4 Listing 2-4  Creat ng a stored procedure to perform a bu k nsert us ng a TVP dec ared as the LocationUdt

tab e type.

CREATE PROCEDURE uspInsertProductionLocation (@TVP LocationUdt READONLY) AS INSERT INTO [Production].[Location] ([Name], [CostRate], [Availability], [ModifiedDate]) SELECT *, 0, GETDATE() FROM @TVP

You now have a stored procedure that can accept a TVP conta n ng a set of rows w th ocat on names and cost rates to be nserted nto the Production.Location tab e, and that sets the ava ab ty quant ty and mod fied date on each nserted row—a ach eved w th a s ng e parameter and a s ng e INSERT INTO…SELECT statement! The procedure doesn’t know or care how the ca er popu ates the TVP before t s used as the source for the INSERT INTO…SELECT statement For examp e, the ca er cou d manua y add one row at a t me, as fo ows DECLARE @LocationTvp AS LocationUdt INSERT INTO @LocationTvp VALUES('UK', 122.4) INSERT INTO @LocationTvp VALUES('Paris', 359.73) EXEC uspInsertProductionLocation @LocationTvp

Or the ca er cou d bu k nsert nto the TVP from another source tab e us ng INSERT INTO…SELECT, as n the next examp e You w fi the TVP from the ex st ng Person.StateProvince tab e us ng the tab e’s Name co umn for LocationName and the va ue 0 for CostRate Pass ng th s TVP to the stored procedure w resu t n a new set of rows added to Production.Location w th the r Name fie ds set ­accord ng to the names n the Person.StateProvince tab e, the r CostRate and Availability va ues set to 0, and the r ModifiedDate va ues set by GETDATE, as shown here DECLARE @LocationTVP AS LocationUdt INSERT INTO @LocationTVP SELECT [Name], 0.00 FROM [Person].[StateProvince] EXEC uspInsertProductionLocation @LocationTVP

The TVP cou d a so be popu ated on the c ent us ng ADO NET, as you’ earn n the next sect on Bu k updates (and de etes) us ng TVPs are poss b e as we You can create an UPDATE statement by jo n ng a TVP (wh ch you must a as) to the tab e you want to update The rows updated n the tab e are determ ned by the matches jo ned to by the TVP and can be set to new va ues that are a so conta ned n the TVP For examp e, you can pass a TVP popu ated w th category IDs and names for

50  Part 1  Core SQL Server Development

updat ng a Category tab e n the database, as shown n L st ng 2-5 By jo n ng the TVP to the Category tab e on the category ID, the uspUpdateCategories stored procedure can update a the match ng rows n the Category w th the new category names passed n the TVP Listing 2-5  Bu k updates us ng TVPs.

CREATE TABLE Category (Id int PRIMARY KEY, Name nvarchar(max), datetime2(0) DEFAULT SYSDATETIME()) CreatedAt -- Initialize with a few INSERT INTO Category(Id, INSERT INTO Category(Id, INSERT INTO Category(Id, INSERT INTO Category(Id, INSERT INTO Category(Id, INSERT INTO Category(Id,

categories Name) VALUES(1, Name) VALUES(2, Name) VALUES(3, Name) VALUES(4, Name) VALUES(5, Name) VALUES(6,

'Housewares') 'Maternity') 'Mens Apparel') 'Womens Apparel') 'Bath') 'Automotive')

-- View the list of categories SELECT * FROM Category -- Will be used by uspUpdateCategories to pass in a set of category updates CREATE TYPE EditedCategoriesUdt AS TABLE (Id int PRIMARY KEY, Name nvarchar(max)) GO -- Receive multiple rows for the change set and update the Category table CREATE PROCEDURE uspUpdateCategories(@EditedCategoriesTVP AS EditedCategoriesUdt READONLY) AS BEGIN -- Update names in the Category table by joining on the TVP by ID UPDATE Category SET Category.Name = ec.Name FROM Category INNER JOIN @EditedCategoriesTVP AS ec ON Category.Id = ec.Id END -- Load up a few changes into a new TVP instance (to categories 1 and 5) DECLARE @Edits AS EditedCategoriesUdt INSERT INTO @Edits VALUES(1, 'Gifts & Housewares') INSERT INTO @Edits VALUES(5, 'Bath & Kitchen') -- Call the stored procedure EXECUTE uspUpdateCategories @Edits -- View the updated names for categories 1 and 5 in the Category table SELECT * FROM Category



Chapter 2  T SQL Enhancements   51

Passing TVPs Using ADO.NET We’ conc ude our d scuss on of TVPs w th the new SqlDbType.Structured enumerat on n ADO NET You w earn how easy t s to marsha mu t p e rows of data from your c ent app cat on to SQL S­ erver, ­w thout requ r ng mu t p e roundtr ps or mp ement ng spec a og c on the server for p ­ rocess ng the data S mp y prepare an ord nary SqlCommand object (Chapter 10 exp a ns the SqlCommand ­object and ADO NET n deta ), set ts CommandType property to CommandType.­StoredProcedure, and popu ate ts Parameters co ect on w th SqlParameter objects These are the rout ne steps for ­sett ng up a ca to any stored procedure Then, to mark a SqlParameter as a TVP, set ts ­SqlDbType property to ­SqlDbType.Structured Th s ets you spec fy any DataTable, DbDataReader, or IEnumerable object as the parameter va ue to be passed to the stored procedure n a s ng e ca to the server In L st ng 2-6, a new customer order s stored n separate Order and OrderDetail DataTable objects w th n a DataSet The two tab es are passed to the SQL Server stored procedure you saw ear er ­(L st ng 2-2 n the sect on “Subm tt ng Orders”), wh ch accepts them as TVPs for nsert on nto the Order and OrderDetail database tab es Note that the C# code st ngs n th s chapter are fragments of a arger so ut on, abbrev ated to conserve space A fu y funct ona so ut on s supp ed w th the samp e code for th s chapter that you can down oad from the book’s compan on webs te (see the “Introduct on” for deta s) The code for L st ng 2-6 s part of the TVPsW thDataTab e project n the TSq Enhancements so ut on Listing 2-6 Pass ng DataTable objects as TVPs to a SQL Server stored procedure from ADO.NET.

// Assumes conn is an open SqlConnection object and ds is // a DataSet with an Order and OrderDetails table using(conn) { // Create the command object to call the stored procedure SqlCommand cmd = new SqlCommand("uspInsertNewOrder", conn); cmd.CommandType = CommandType.StoredProcedure; // Create the parameter for passing the Order TVP SqlParameter headerParam = cmd.Parameters.AddWithValue ("@OrderHeaderTvp", ds.Tables["OrderHeader"]); // Create the parameter for passing the OrderDetails TVP SqlParameter detailsParam = cmd.Parameters.AddWithValue ("@OrderDetailsTvp", ds.Tables["OrderDetail"]); // Set the SqlDbType of the parameters to Structured headerParam.SqlDbType = SqlDbType.Structured; detailsParam.SqlDbType = SqlDbType.Structured; // Execute the stored procedure cmd.ExecuteNonQuery(); }

52  Part 1  Core SQL Server Development

Th s code ca s a SQL Server stored procedure and passes to t an order header and the comp ete set of order deta s n a s ng e roundtr p Remarkab y, t’s just as s mp e as that You just need to ­ensure that the schema of the DataTable objects match the correspond ng TVP-tab e-type schema You can a so send a set of rows d rect y to a parameter zed SQL statement w thout creat ng a stored procedure Because the SQL statement s dynam ca y constructed on the c ent, there s no stored procedure s gnature that spec fies the name of the user-defined tab e type for the TVP ­Therefore, you need to te ADO NET what the type s by sett ng the TypeName property to the name of the tab e type defined on the server For examp e, the code n L st ng 2-7 (part of the ­T VPAdd t ona Samp es project n the TSq Enhancements so ut on for th s chapter) passes a DataTable to a parameter zed SQL statement Listing 2-7  Pass ng TVPs to a parameter zed SQL statement from ADO.NET.

// Define the INSERT INTO...SELECT statement to insert into Categories const string TSqlStatement = @" INSERT INTO Categories (CategoryID, CategoryName) SELECT nc.CategoryID, nc.CategoryName FROM @NewCategoriesTvp AS nc"; // Assumes conn is an open SqlConnection object and ds is // a DataSet with a Category table using(conn) { // Set up the command object for the statement SqlCommand cmd = new SqlCommand(TSqlStatement, conn); // Add a TVP specifying the DataTable as the parameter value SqlParameter catParam = cmd.Parameters.AddWithValue ("@NewCategoriesTvp", ds.Tables["Category"]); catParam.SqlDbType = SqlDbType.Structured; catParam.TypeName = "dbo.CategoriesUdt"; // Execute the command cmd.ExecuteNonQuery(); }

Sett ng the TypeName property to dbo.CategoriesUdt n th s code means that you have a ­ ser-defined tab e type by that name on the server, created us ng the CREATE TYPE…AS TABLE u ­statement that defines the CategoryID and CategoryName co umns You can a so use any object der ved from DbDataReader (that s, a connected data source) to stream rows of data to a TVP The examp e shown n L st ng 2-8 (part of the ­T VPAdd t ona Samp es project n the TSq Enhancements so ut on for th s chapter) ca s an Orac e stored procedure to se ect data from an Orac e database nto a connected OracleDataReader The reader object gets passed as a s ng e tab e-va ued nput parameter to a SQL Server stored



Chapter 2  T SQL Enhancements   53

­ rocedure, wh ch can then use the Orac e data n the reader as the source for add ng new rows p nto the Category tab e n the SQL Server d ­ atabase (for th s code to work, you must have the ADO NET Prov der for Orac e nsta ed and a ­project reference set to System.Data.OracleClient, as we as access to an Orac e database server) Listing 2-8  Pass ng a connected OracleDataReader source as a TVP to SQL Server.

// Set up command object to select from Oracle OracleCommand selCmd = new OracleCommand ("SELECT CategoryID, CategoryName FROM Categories;", oracleConn); // Execute the command and return the results in a connected // reader that will automatically close the connection when done OracleDataReader rdr = selCmd.ExecuteReader(CommandBehavior.CloseConnection); // Set up command object to insert to SQL Server SqlCommand insCmd = new SqlCommand("uspInsertCategories", connection); insCmd.CommandType = CommandType.StoredProcedure; // Add a TVP specifying the reader as the parameter value SqlParameter catParam = cmd.Parameters.AddWithValue("@NewCategoriesTvp", rdr); catParam.SqlDbType = SqlDbType.Structured; // Execute the stored procedure insertCommand.ExecuteNonQuery();

Passing Collections to TVPs Using Custom Iterators What f you’re work ng w th co ect ons popu ated w th bus ness objects rather than DataTable ­objects popu ated w th DataRow objects? You m ght not th nk at first that bus ness objects and TVPs cou d work together, but the fact s that they can—and qu te gracefu y, too A you need to do s ­ mp ement the IEnumerable nterface n your co ect on c ass Th s nterface requ res your co ect on c ass to supp y a custom terator method named GetEnumerator, wh ch ADO NET w ca for each object conta ned n the co ect on when you nvoke ExecuteNonQuery Let’s demonstrate w th the same order entry examp e as L st ng 2-6, on y now you’ use ord nary bus ness object co ect ons (rather than DataTable and DataRow objects) as the source for the TVPs You have OrderHeader and OrderDetail c asses w th propert es to match the correspond ng TVP-tab e-type schemas, as shown n L st ng 2-9 Of course, a rea mp ementat on cou d nc ude much more than th s s mp e data transfer object, such as encapsu ated bus ness og c The code n L st ngs 2-9 and 2-10 s part of the TVPsW thB zCo ect on project n the TSq Enhancements so ut on for th s chapter

54  Part 1  Core SQL Server Development

Listing 2-9  Defin ng c asses for pass ng bus ness objects as TVPs.

public class OrderHeader { public int OrderId { get; set; } public int CustomerId { get; set; } public DateTime OrderedAt { get; set; } } public class OrderDetail { public int OrderId { get; set; } public int LineNumber { get; set; } public int ProductId { get; set; } public int Quantity { get; set; } public decimal Price { get; set; } }

Ord nar y, work ng w th List and List objects m ght be a ­su tab e opt on for conta n ng co ect ons of OrderHeader and OrderDetail objects n your app cat on But n these co ect ons they won’t suffice on the r own as nput va ues for TVPs because List does not mp ement IEnumerable The so ut on s to create OrderHeaderCollection and ­OrderDetailCollection c asses that nher t List and List respect ve y, and then mp ement IEnumerable n order to “TVP-enab e” them as shown n L st ng 2-10 Listing 2-10  Defin ng co ect on c asses w th custom terators for pass ng bus ness objects as TVPs.

public class OrderHeaderCollection : List, IEnumerable { IEnumerator IEnumerable.GetEnumerator() { var sdr = new SqlDataRecord( new SqlMetaData("OrderId", SqlDbType.Int), new SqlMetaData("CustomerId", SqlDbType.Int), new SqlMetaData("OrderedAt", SqlDbType.Date)); foreach (OrderHeader oh in this) { sdr.SetInt32(0, oh.OrderId); sdr.SetInt32(1, oh.CustomerId); sdr.SetDateTime(2, oh.OrderedAt); yield return sdr; } } }



Chapter 2  T SQL Enhancements   55

public class OrderDetailCollection : List, IEnumerable { IEnumerator IEnumerable.GetEnumerator() { var sdr = new SqlDataRecord( new SqlMetaData("OrderId", SqlDbType.Int), new SqlMetaData("LineNumber", SqlDbType.Int), new SqlMetaData("ProductId", SqlDbType.Int), new SqlMetaData("Quantity", SqlDbType.Int), new SqlMetaData("Price", SqlDbType.Money)); foreach (OrderDetail od in this) { sdr.SetInt32(0, od.OrderId); sdr.SetInt32(1, od.LineNumber); sdr.SetInt32(2, od.ProductId); sdr.SetInt32(3, od.Quantity); sdr.SetDecimal(4, od.Price); yield return sdr; } } }

We’ on y exp a n the OrderHeaderCollection c ass; you’ be ab e to nfer how the ­ rderDetailCollection c ass—or any of your own co ect on c asses— mp ements the custom terator O needed to support TVPs F rst, aga n, t nher ts List, so an OrderHeaderCollection object s everyth ng that a List object s Th s means mp c t y, by the way, that t a so mp ements IEnumerable, wh ch s what makes any sequence “foreach-ab e” or “LINQ-ab e ” But to the heart of our d scuss on, t exp c t y mp ements IEnumerable, wh ch means t has a custom iterator method for ADO NET to consume when an nstance of th s co ect on c ass s ­ass gned to a SqlDbType.Structured parameter for p p ng over to SQL Server w th a TVP Every enumerable c ass requ res a match ng enumerator method, so not surpr s ng y ­mp ement ng IEnumerable requ res prov d ng a GetEnumerator method that returns an IEnumerator Th s method first n t a zes a new SqlDataRecord object w th a ­schema that matches the tab e-type schema that the TVPs are dec ared as It then enters a oop that ­terates a the e ements n the co ect on (poss b e because List mp c t y ­mp ements IEnumerable) On the first terat on, t sets the co umn property va ues of the ­SqlDataRecord object to the property va ues of the first OrderHeader e ement, and then ssues the mag c yield return statement By defin t on, any method ( ke th s one) wh ch returns IEnumerator and has a yield return statement n t s a custom terator method; t s expected to return a ­sequence of T objects unt the method execut on path comp etes ( n th s case, when the foreach oop fin shes)

56  Part 1  Core SQL Server Development

The crux of th s s that you are never ca ng th s method d rect y Instead, when you ­nvoke ExecuteNonQuery to run a stored procedure w th a SqlDbType.Structured parameter (that s, a TVP), ADO NET expects the co ect on passed for the parameter va ue to mp ement IEnumerable so that IEnumerable.GetEnumerator can be ca ed ­nterna y to fetch each new record for p p ng over to the server When the first e ement s fetched from the co ect on, GetEnumerator s entered, the SqlDataRecord s n t a zed, and s then ­popu ated w th va ues us ng the SetInt32 and SetDateTime methods (there’s a SetXXX method for each data type) That SqlDataRecord “row” s then pushed nto the p pe ne to the server by yield return When the next e ement s fetched from the co ect on, the GetEnumerator method resumes from the po nt that t yield returned the prev ous e ement, rather than enter ng GetEnumerator aga n from the top Th s means the SqlDataRecord gets n t a zed w th schema nformat on on y once, wh e ts popu at on w th one e ement after another s orchestrated by the contro ng ADO NET code for ­E xecuteNonQuery that actua y sh ps one SqlDataRecord after another to the server The actua code to ca the stored procedure s 100 percent dent ca to the code n L st ng 2-6 that uses a DataTable rather than a co ect on Subst tut ng a co ect on for a DataTable object requ res no code changes and works flaw ess y, prov ded the co ect on mp ements IEnumerator Th s mean the co ect on has a GetEnumerator method that feeds each object nstance to a ­SqlDataRecord and maps each object property to a co umn defined by the user-defined tab e type that the TVP s dec ared as

TVP Limitations TVPs have severa noteworthy m tat ons F rst and foremost, once TVPs are n t a y popu ated and passed, they are read-on y structures The READONLY keyword must be app ed to TVPs n the ­s gnatures of your stored procedures, or they w not comp e S m ar y, the OUTPUT keyword cannot be used You cannot update the co umn va ues n the rows of a TVP, and you cannot nsert or de ete rows If you must mod fy the data n a TVP, you need to mp ement a workaround, such as nsertng data from the TVP nto a temporary tab e or nto a tab e var ab e to wh ch you can then app y changes There s no ALTER TABLE…AS TYPE statement that supports chang ng the schema of a TVP tab e type Instead, you must first drop a stored procedures and UDFs that reference the type before you can drop the type, re-create t w th a new schema, and then re-create the stored procedures Indexng s m ted as we , w th support on y for PRIMARY KEY and UNIQUE constra nts A so, stat st cs on TVPs are not ma nta ned by SQL Server It’s a so mportant to note that the Ent ty Framework (wh ch we cover n Chapter 10) does not support TVPs However, you can eas y send any co ect on of Ent ty Framework ent t es to a stored procedure that accepts TVPs by cod ng a custom terator just ke the one you created n the ­sect on “Pass ng Co ect ons to TVPs Us ng Custom Iterators ” Th s techn que s demonstrated n the ­T VPsW thEF project of the TSq Enhancements so ut on that you can down oad from the book’s ­compan on webs te



Chapter 2  T SQL Enhancements   57

Date and Time Data Types The date, time, datetime2, and datetimeoffset types are four date and t me data types that were ntroduced n SQL Server 2008 These types shou d be used for a new database deve opment n eu of the trad t ona datetime and smalldatetime data types The newer types are a gned w th the NET Framework, M crosoft W ndows, and the SQL standard—un ke datetime and smalldatetime—and there are mprovements n range, prec s on, and storage as we

Separation of Dates and Times We’ beg n by ook ng at the separate date and time types If you need to store on y a date va ue (for examp e, a date of b rth), use the date type S m ar y, use the time type for stor ng just a t me va ue (for examp e, a da y med cat on t me), as shown here DECLARE @DOB date DECLARE @MedsAt time

The datetime and smalldatetime types, wh ch were the on y prev ous y ava ab e opt ons, each nc ude both a date and a t me port on In cases where on y the date or on y the t me s needed, the extraneous port on consumes storage need ess y, wh ch resu ts n wasted space n the database In ­add t on to sav ng storage, us ng date rather than datetime y e ds better performance for date-on y man pu at ons and ca cu at ons, because there s no t me port on to be hand ed or cons dered And the time type s prov ded for those ess common cases when you requ re a t me w thout a date; for ­examp e, feed ng t me at the zoo

More Portable Dates and Times To store both a date and a t me as a s ng e va ue, use the datetime2 data type Th s data type supports the same range of va ues as the DateTime data type n the NET Framework, so t can store dates from 1/1/0001 (DateTime.MinValue n NET) to 12/31/9999 (DateTime.MaxValue n NET) n the Gregor an ca endar Contrast th s w th the a owab e date va ues for the o d datetime type, wh ch range on y from 1/1/1753 to 12/31/9999 Th s means that dates n NET from 1/1/0001 through 12/31/1752 can’t be stored at a n SQL Server’s datetime type, a prob em so ved by us ng e ther the date, datetime2, or datetimeoffset type Because the supported range of dates s the same n both NET and SQL Server, any date can be safe y passed between these c ent and server p atforms w th no concern You are strong y encouraged to d scont nue us ng the o der datetime and smalldatetime data types and to use on y date and datetime2 types for new deve opment (or the new datetimeoffset type for t me zone awareness, wh ch s d scussed next) There has a so been a need for greater prec s on of fract ona seconds n t me va ues The ­datetime type s accurate on y w th n rough y 3 33 m seconds, whereas t me va ues n W ndows and NET have a s gn ficant y greater 100-nanosecond (10-m onth of a second) accuracy (The ­smalldatetime type doesn’t even support seconds and s accurate on y to the m nute ) Stor ng t mes n the ­database therefore resu ts n a oss of prec s on L ke the expanded range of supported dates, the new time, datetime2, and datetimeoffset types are now more a gned w th NET and other p atforms by a so 58  Part 1  Core SQL Server Development

­ rov d ng the same 100 nanosecond accuracy As a resu t, you no onger ncur any data oss of p ­fract ona second accuracy between p atforms when record ng t me va ues to the database Of course, there are storage mp cat ons that come w th greater t me prec s on, and we’ d scuss those ­momentar y

Time Zone Awareness The fourth and ast data type n th s category s datetimeoffset Th s type defines a date and t me w th the same range and prec s on that datetime2 prov des but a so nc udes an offset va ue w th a range of –14 00 to +14 00 that dent fies the t me zone In the past, the on y pract ca approach for ­g oba zat on of dates and t mes n the database has been to store them n Coord nated Un versa T me (UTC) format Do ng th s requ res back-and-forth convers on between UTC and oca t me that must be hand ed at the app cat on eve , and that means wr t ng code Us ng the datetimeoffset type, you can store va ues that represent the oca date and t me n ­ fferent reg ons of the wor d and nc ude the appropr ate t me zone offset for the reg on n each d va ue Because the t me zone offset embedded n the date and t me va ue s spec fic to a part cu ar ­oca e, SQL Server s ab e to perform date and t me compar sons between d fferent oca es w thout any c­ onvers on efforts requ red on your part A though datetimeoffset va ues appear to go n and come out as dates and t mes oca to a part cu ar reg on, they are nterna y converted, stored, and treated n UTC format for compar sons, sort ng, and ndex ng, wh e the t me zone “tags a ong ” Ca cu at ons and compar sons are therefore performed correct y and cons stent y across a dates and t mes n the database regard ess of the d fferent t me zones n var ous reg ons By s mp y ­append ng the t me zone offset to datetimeoffset va ues, SQL Server hand es the convers ons to and from UTC for you automat ca y n the background Even better, you can obta n a datetimeoffset va ue e ther as UTC or oca t me For those of you bu d ng databases that need to store var ous oca t mes (or even just dates) n d fferent reg ons of the wor d, th s s an extreme y conven ent feature The ­database hand es a the deta s, so the app cat on deve oper doesn’t have to T me zone funct ona ty s s mp y ava ab e for free r ght at the database eve For examp e, the database knows that 9 15 AM n New York s n fact ater than 10 30 AM n Los Ange es f you store the va ues n a datetimeoffset data type w th appropr ate t me zone ­offsets ­Because the New York t me spec fies a t me zone offset of –5 00 and the Los Ange es t me has an offset of –8 00, SQL Server s aware of the three-hour d fference between the two t me zones and accounts for that d fference n a date/t me man pu at ons and ca cu at ons Th s behav or s ­demonstrated by the code n L st ng 2-11 Listing 2-11  T me zone ca cu at ons us ng datetimeoffset.

DECLARE @Time1 datetimeoffset DECLARE @Time2 datetimeoffset DECLARE @MinutesDiff int



Chapter 2  T SQL Enhancements   59

SET @Time1 = '2012-02-10 09:15:00-05:00' SET @Time2 = '2012-02-10 10:30:00-08:00'

-- NY time is UTC -05:00 -- LA time is UTC -08:00

SET @MinutesDiff = DATEDIFF(minute, @Time1, @Time2) SELECT @MinutesDiff

If you run th s code, you w see that the DATEDIFF ca cu at on returns 255 Th s nd cates that SQL Server s c ear y ab e to account for the three-hour d fference n the t me zones Because 10 30 AM n Los Ange es s actua y 1 30 PM n New York, a d fference of 255 m nutes (4 hours and 15 m nutes) between that t me and 9 15 AM New York t me was ca cu ated correct y

Note  Time zone names are not supported, nor is there support for daylight savings time. Unfortunately, these features did not make it into the final release of the product, but they are on the list for the next version of SQL Server. Time zones can be expressed only by hour/minute offsets, and you must continue to handle daylight savings time considerations on your own.

Date and Time Accuracy, Storage, and Format Date va ues stored n date, datetime2, and datetimeoffset types are compacted nto a fixed storage space of 3 bytes They use an opt m zed format that s 1 byte ess than the 4 bytes consumed by the date port on of the o der datetime type (support ng a greater range n a sma er space) T me va ues stored n time, datetime2, and datetimeoffset types, by defau t, consume 5 bytes of storage to support the same 100-nanosecond accuracy as W ndows and NET However, you can spec fy a ower degree of prec s on for even more compacted storage by prov d ng an opt ona sca e parameter when dec ar ng time, datetime2, and datetimeoffset var ab es The sca e can range from 0 to 7, w th 0 ­offer ng no fract ona -second prec s on at a consum ng 3 bytes, and 7 (the defau t) offer ng the g ­ reatest fract ona -second prec s on (100 nanoseconds) consum ng 5 bytes The sca e ­essent a y d ctates the ­number of d g ts supported after the dec ma po nt of the seconds va ue, where a sca e va ue of 7 supports a fract ona prec s on of 100 nanoseconds (each 100 nanoseconds be ng 0 0000001 second) The defau t sca e s 7, wh ch offers the greatest prec s on (to 100 nanoseconds) n the argest space (5 bytes) Th s means that dec ar ng a var ab e as time, datetime2, or datetimeoffset s the same as ­dec ar ng t as time(7), datetime2(7), or datetimeoffset(7), mak ng the fo ow ng two statements equ va ent DECLARE @StartDateTime datetime2 DECLARE @StartDateTime datetime2(7)

If you don’t requ re any fract ona prec s on at a , use a sca e of 0, as n the fo ow ng statement, wh ch consumes on y 3 bytes to store a t me n @FeedingTime DECLARE @FeedingTime time(0)

60  Part 1  Core SQL Server Development

Two t me va ues w th d ffer ng sca es are perfect y compat b e w th each other for compar son SQL Server automat ca y converts the va ue w th the ower sca e to match the va ue w th the greater sca e and compares the two safe y V rtua y a ndustry-standard str ng tera formats are supported for conven ent y represent ng dates and t mes For examp e, the date May 15, 2012, can be expressed n any of the formats shown n Tab e 2-1 Table 2-1  Common va d date and t me str ng tera formats Format

Example

Numer c

5/15/2012, 15 05 2012, 05.15.2012

A phabet ca

May 15, 2012

SO8601

2012 05 15, 201205153

ODBC

{d 2012 05 15 }

W3C XML

2012 05 15Z

You have s m ar flex b ty for represent ng t mes For examp e, the same t me va ue can be ­expressed as 23 30, 23 30 00, 23 30 00 0000, or 11 30 00 PM T me zone offsets are expressed mere y by ­append ng a p us or m nus s gn fo owed by the UTC hours and m nutes for the zone—for ­examp e, +02 00 for Jerusa em

Note  There are even more possible formatting variations than those we’ve indicated here, where the purpose has been to convey how accommodating SQL Server is with respect to variations in date and time syntax, rather than to duplicate content from SQL Server Books Online (which you can easily refer to on your own for a complete specification of supported string literal formats for dates and times). In addition, SQL Server 2012 provides several new T-SQL functions (covered later in the chapter in the section "New T-SQL Functions in SQL Server 2012") for parsing and creating culture-specific and culture-neutral dates. You can use CAST or CONVERT to extract just the date or t me port ons of a datetime2 co umn for search ng When you perform such a convers on on a datetime2 co umn that s ndexed, SQL Server does not need to resort to a sequent a tab e scan and s ab e to perform the much faster ndex seek to ocate the spec fic date or t me For examp e, the fo ow ng code defines a tab e w th a datetime2 type that has a c ustered ndex Se ect ng by date or t me on y can be ach eved us ng CONVERT, wh e st us ng the c ustered ndex for effic ent search ng, as shown n L st ng 2-12 Listing 2-12 Us ng CONVERT to extract the date and t me port on from a datetime2 co umn.

CREATE TABLE DateList(MyDate datetime2) CREATE CLUSTERED INDEX idx1 ON DateList(MyDate)



Chapter 2  T SQL Enhancements   61

-- Insert some rows into DateList INSERT INTO DateList VALUES ('2011-10-10 12:15:00'), ('2012-04-07 09:00:00'), ('2012-04-07 10:00:00'), ('2011-10-10 09:00:00') SELECT MyDate FROM DateList WHERE MyDate = '2011-10-10 12:15:00' SELECT MyDate FROM DateList WHERE CONVERT(time(0), MyDate) = '09:00:00' SELECT MyDate FROM DateList WHERE CONVERT(date, MyDate) = '2012-04-07'

If you request the est mated execut on p an for th s code, you w see that the c ustered ndex s ­everaged when us ng CONVERT to query e ther the date or t me port on of the ndexed datetime2 ­co umn

Date and Time Functions A of the trad t ona date-re ated and t me-re ated funct ons, nc ud ng DATEADD, DATEDIFF, ­DATEPART, and DATENAME, of course fu y support the newer date and t me data types, and severa ­funct ons have been added as we We conc ude our d scuss on of the SQL Server 2008 date and t me data types by exp or ng the T-SQL extens ons added to support them A few add t ona handy date and t me funct ons were added n SQL Server 2012, wh ch we cover n the sect on on new SQL Server 2012 funct ons ater n the chapter The SYSDATETIME and SYSUTCDATETIME funct ons return the date and t me on the server as ­datetime2 types (w th fu seven-sca e prec s on accuracy w th n 100 nanoseconds), just as the ­GETDATE and GETUTCDATE funct ons cont nue to return the current date and t me as datetime types Another new funct on, SYSDATETIMEOFFSET, returns the date and t me on the server as a datetimeoffset type, w th a t me zone offset reflect ng the reg ona sett ngs estab shed on the server, wh ch nc udes awareness of oca day ght sav ngs t me The code n L st ng 2-13 shows the contrast between the var ous s m ar server date and t me funct ons Listing 2-13  Compar ng server date and t me funct ons.

SET NOCOUNT ON SELECT GETDATE() AS 'GETDATE() datetime' SELECT GETUTCDATE() AS 'GETUTCDATE() datetime' SELECT SYSDATETIME() AS 'SYSDATETIME() datetime2' SELECT SYSUTCDATETIME() AS 'SYSUTCDATETIME() datetime2' SELECT SYSDATETIMEOFFSET() AS 'SYSDATETIMEOFFSET() datetimeoffset'

Runn ng th s code just after 8 20 PM on February 10, 2012 n New York resu ts n the fo ow ng output GETDATE() datetime ----------------------2012-02-10 20:21:19.380

62  Part 1  Core SQL Server Development

GETUTCDATE() datetime ----------------------2012-02-11 01:21:19.380 SYSDATETIME() datetime2 --------------------------2012-02-10 20:21:19.3807984 SYSUTCDATETIME() datetime2 --------------------------2012-02-11 01:21:19.3807984 SYSDATETIMEOFFSET() datetimeoffset ---------------------------------2012-02-10 20:21:19.3807984 -05:00

The TODATETIMEOFFSET and SWITCHOFFSET funct ons a ow you to perform t me zone offset man pu at ons TODATETIMEOFFSET w convert any date or t me type (that has no t me zone offset) to a datetimeoffset type by app y ng whatever t me zone offset you prov de ­SWITCHOFFSET makes t easy to find out what the same t me s n two d fferent t me zones You prov de the ­datetimeoffset for a source ocat on and a t me zone offset for a target ocat on, and S­ WITCHOFFSET returns a ­datetimeoffset represent ng the equ va ent date and t me n the target ­ocat on, as shown n L st ng 2-14 Listing 2-14  Perform ng t me zone offset man pu at ons us ng TODATETIMEOFFSET and SWITCHOFFSET.

DECLARE @TheTime datetime2 DECLARE @TheTimeInNY datetimeoffset DECLARE @TheTimeInLA datetimeoffset -- Hold a time that doesn't specify a time zone SET @TheTime = '2012-02-10 7:35PM' -- Convert it into one that specifies time zone for New York SET @TheTimeInNY = TODATETIMEOFFSET(@TheTime, '-05:00') -- Calculate the equivalent time in Los Angeles SET @TheTimeInLA = SWITCHOFFSET(@TheTimeInNY , '-08:00') SELECT @TheTime AS 'Any Time' SELECT @TheTimeInNY AS 'NY Time' SELECT @TheTimeInLA AS 'LA Time'

Here s the output resu t Any Time --------------------------2012-02-10 19:35:00.0000000 NY Time ---------------------------------2012-02-10 19:35:00.0000000 -05:00



Chapter 2  T SQL Enhancements   63

LA Time ---------------------------------2012-02-10 16:35:00.0000000 -08:00

You can use TODATETIMEOFFSET w th INSERT INTO…SELECT to bu k- nsert date and t me va ues w th no t me zone nformat on from a source tab e nto a target tab e and to app y a t me zone offset to produce datetimeoffset va ues n the target tab e For examp e, the fo ow ng code cop es a the row va ues from the dt2 co umn n tab e test1 (of type datetime2, wh ch has no t me zone nformat on) nto the dto co umn n test2 (of type datetimeoffset) and app es a t me zone offset of –05 00 to each cop ed va ue INSERT INTO test2(dto) SELECT TODATETIMEOFFSET(dt2, '-05:00') FROM test1

The next examp e retr eves a the datetimeoffset va ues from the dto co umn n the test2 tab e, wh ch can nc ude va ues across a var ety of d fferent t me zones Us ng a SWITCHOFFSET funct on that spec fies an offset of –05 00, the va ues are automat ca y converted to New York t me from ­whatever t me zone s stored n the test2 tab e SELECT SWITCHOFFSET(dto, '-05:00') FROM test2

Last, both the ex st ng DATEPART and DATENAME funct ons have been extended to add support for m croseconds (mcs), nanoseconds (ns), and t me zone offsets (tz) n the h gher-prec s on and t me zone-aware types, as shown n L st ng 2-15 Listing 2-15  Us ng the new date port ons n SQL Server 2008 w th DATEPART and DATENAME.

SET NOCOUNT ON DECLARE @TimeInNY datetimeoffset SET @TimeInNY = SYSDATETIMEOFFSET() -- Show the current time in NY SELECT @TimeInNY AS 'Time in NY' -- DATEPART with tz gets the time zone value SELECT DATEPART(tz, @TimeInNY) AS 'NY Time Zone Value' -- DATENAME with tz gets the time zone string SELECT DATENAME(tz, @TimeInNY) AS 'NY Time Zone String' -- Both DATEPART and DATENAME with mcs gets the microseconds SELECT DATEPART(mcs, @TimeInNY) AS 'NY Time Microseconds' -- Both DATEPART and DATENAME with ns gets the nanoseconds SELECT DATEPART(ns, @TimeInNY) AS 'NY Time Nanoseconds'

Runn ng th s code returns the fo ow ng output Time in NY ---------------------------------2012-02-10 20:50:55.7851424 -05:00

64  Part 1  Core SQL Server Development

NY Time Zone Value ------------------300 NY Time Zone String ------------------------------05:00 NY Time Microseconds -------------------785142 NY Time Nanoseconds ------------------785142400

The MERGE Statement The MERGE statement does just what ts name says It comb nes the norma nsert, update, and de ete operat ons nvo ved n a typ ca merge scenar o, a ong w th the se ect operat on that prov des the source and target data for the merge Essent a y, that means t comb nes four statements nto one In fact, you can comb ne five statements nto one us ng the OUTPUT c ause, and even more than that w th INSERT OVER DML (a spec a T-SQL feature syntax, wh ch we cover next) Pr or to SQL Server 2008, separate, mu t p e statements were requ red to ach eve what can now be accomp shed w th a s ng e MERGE statement Th s statement has a flex b e syntax that a ows you to exerc se fine contro over source and target match ng, as we as the var ous set-based DML act ons carr ed out on the target The resu t s s mp er code that’s eas er to wr te and ma nta n (and a so runs faster) than the equ va ent code us ng separate statements to ach eve the same resu t Th s first examp e uses MERGE to manage stocks and trades Beg n by creat ng the two tab es to ho d stocks that you own and da y trades that you make, as shown n L st ng 2-16 Listing 2-16  Creat ng the Stock and Trade tab es.

CREATE TABLE Stock(Symbol varchar(10) PRIMARY KEY, Qty int CHECK (Qty > 0)) CREATE TABLE Trade(Symbol varchar(10) PRIMARY KEY, Delta int CHECK (Delta 0))

You start off w th 10 shares of Adventure Works stock and 5 shares of B ue Yonder A r nes stock These are stored n the Stock tab e INSERT INTO Stock VALUES ('ADVW', 10) INSERT INTO Stock VALUES ('BYA', 5)

Dur ng the day, you conduct three trades You buy 5 new shares for Adventure Works, se 5 shares of B ue Yonder A r nes, and buy 3 shares for your new nvestment n Northw nd Traders These are stored n the Trade tab e, as fo ows INSERT INTO Trade VALUES('ADVW', 5)



Chapter 2  T SQL Enhancements   65

INSERT INTO Trade VALUES('BYA', -5) INSERT INTO Trade VALUES('NWT', 3)

Here are the contents of the two tab es SELECT * FROM Stock GO Symbol ---------ADVW BYA

Qty ----------10 5

(2 row(s) affected)

SELECT * FROM Trade GO Symbol ---------ADVW BYA NWT

Delta ----------5 -5 3

(3 row(s) affected)

At the c os ng of the day, you want to update the quant t es n the Stock tab e to reflect the trades of the day you recorded n the Trade tab e Your Adventure Works stock quant ty has r sen to 15, you no onger own any B ue Yonder A r nes (hav ng so d the on y 5 shares you owned), and you now own 3 new shares of Northw nd Traders stock That’s go ng to nvo ve jo n ng the Stock and Trade tab es to detect changes n stock quant t es resu t ng from your trades, as we as nsert, update, and de ete operat ons to app y those changes back to the Stock tab e A th s og c and man pu at on can be performed w th a s ng e statement us ng MERGE, as shown n L st ng 2-17 Listing 2-17  App y ng trades to stocks us ng MERGE.

MERGE Stock USING Trade ON Stock.Symbol = Trade.Symbol WHEN MATCHED AND (Stock.Qty + Trade.Delta = 0) THEN -- delete stock if entirely sold DELETE WHEN MATCHED THEN -- update stock quantity (delete takes precedence over update) UPDATE SET Stock.Qty += Trade.Delta WHEN NOT MATCHED BY TARGET THEN -- add newly purchased stock INSERT VALUES (Trade.Symbol, Trade.Delta);

66  Part 1  Core SQL Server Development

Let’s d ssect th s statement It beg ns of course w th the MERGE keyword tse f, fo owed by ­USING and ON keywords that respect ve y dent fy the target and source of the merge operat on, and the joining keys used to re ate the source and target to each other for the merge Three merge clauses then fo ow w th the WHEN…THEN syntax, and the statement s then fina y term nated w th a ­sem co on (;)

Important  The statement-terminating semicolon (part of the SQL standard) is usually ­unnecessary in SQL Server. However, the MERGE statement absolutely requires it, and you will receive an error if you omit it.

Defining the Merge Source and Target MERGE has a very e egant mp ementat on n SQL Server At ts core, t operates on a jo n between the source and target of the merge no d fferent y than the way any standard JOIN pred cate n the FROM c ause of any SELECT statement works As you’ see short y when you exam ne SQL Server’s query p an for MERGE, the source, the target, and the jo n between them are hand ed nterna y n exact y the same manner as for a regu ar SELECT The parts of the MERGE syntax that express th s se ect ­operat on nc ude the MERGE keyword tse f, a ong w th USING and ON, wh ch respect ve y spec fy the target, source, and jo n pred cate, as shown n th s sn ppet from L st ng 2-17 MERGE Stock USING Trade ON Stock.Symbol = Trade.Symbol

The target can be any tab e or updateab e v ew and s spec fied mmed ate y fo ow ng the MERGE keyword It s the rec p ent of the changes resu t ng from the merge, wh ch can nc ude ­comb nat ons of new, changed, and de eted rows In th s examp e, the Stock tab e s the target, and t rece ves changes merged nto t from da y trade nformat on (the source) The source s the prov der of the data, wh ch s the Trade tab e n th s examp e, and s spec fied w th the USING keyword r ght after the target The source can be v rtua y anyth ng Th s ­nc udes not on y regu ar tab es and v ews, but subquer es, CTEs, tab e var ab es, remote tab es, ­tab e-va ued ­funct ons (TVFs) and TVPs a ke, and even text fi es accessed w th OPENROWSETBYTES ­Fundamenta y, anyth ng that s va d n the FROM c ause of a SELECT statement s va d as the source of a MERGE statement—noth ng more, noth ng ess The jo n pred cate spec fied by the ON keyword that fo ows defines the co umn key or keys r­ e at ng the source and target to each other, no d fferent y than a standard tab e jo n Aga n, a ­ nyth ng you can put n a SELECT jo n can be spec fied for a MERGE jo n w th ON The jo n defines wh ch records are cons dered match ng or nonmatch ng between the source and target In th s examp e, source and target tab es are re ated by the Symbol co umn The jo n pred cate te s SQL Server what stocks ex st and don’t ex st n both tab es so that you can nsert, update, and de ete data n the target tab e accord ng y The type of jo n ( nner, eft outer, r ght outer, or fu outer) s determ ned by wh ch of the var ous merge c auses are then app ed next n the MERGE statement

Chapter 2  T SQL Enhancements   67

The WHEN MATCHED Clause The prev ous examp e uses three merge c auses two WHEN MATCHED c auses and one WHEN NOT MATCHED BY TARGET c ause Let’s ook at each of them c ose y The first WHEN MATCHED c ause executes when a match ng stock symbo s found n both the Stock and the Trade tab es, as shown n th s code sn ppet WHEN MATCHED AND (Stock.Qty + Trade.Delta = 0) THEN -- delete stock if entirely sold DELETE

A match wou d norma y mean updat ng the quant ty va ue n the Stock tab e by the de ta va ue (amount bought or so d) n the Trade tab e However, n th s scenar o, you want to phys ca y de ete the row n the Stock tab e f ts updated va ue resu ts n 0, because that means you don’t rea y own that part cu ar stock at a anymore (as s the case w th B ue Yonder A r nes) You can code for that scenar o by qua fy ng the WHEN MATCHED c ause w th an add t ona pred cate that tests whether the stock quant ty resu t ng from the trade y e ds 0 Th s g ves you flex b ty to prov de your own cr ter a as pred cates to your merge c auses, and app y custom bus ness og c as fi ters to the var ous match ng cond t ons In th s part cu ar case, you want to remove a row from the Stock tab e us ng the DELETE statement rather than chang ng ts va ue to 0 The next merge c ause s another WHEN MATCHED c ause, but th s second one has no pred cate qua fy ng the match cond t on, as shown n th s code sn ppet WHEN MATCHED THEN -- update stock quantity (delete takes precedence over update) UPDATE SET Stock.Qty += Trade.Delta

Th s second c ause hand es a the other trades of preex st ng stock that have not resu ted n 0 and changes the stock quant ty accord ng y us ng the UPDATE statement (that s, the Stock.Qty va ues w go up or down depend ng on the pos t ve or negat ve number n Trade.Delta) In th s examp e, you want the Adventure Works stock quant ty to go up to 15, reflect ng the 5 shares purchased on top of the 10 you a ready owned

Note  An error would occur if you tried to sell more than you owned. In fact, an ­error would also occur if you tried to sell everything that you owned without first catching that c­ ondition by deleting the stock in the earlier merge clause. That’s because a check ­constraint on the Qty column (defined when you created the Stock table at the beginning of the example) instructs the database not to tolerate any zero or negative Qty values. SQL Server has very part cu ar ru es govern ng the use of mu t p e merge c auses You are ­ erm tted to have one or two WHEN MATCHED c auses—but no more If there are two WHEN p MATCHED c auses, the first one must be qua fied w th an AND cond t on, as th s examp e has shown

68  Part 1  Core SQL Server Development

Furthermore, one c ause must spec fy an UPDATE, and the other must spec fy a DELETE As demonstrated, MERGE w choose one of the two WHEN MATCHED c auses to execute based on whether the AND ­cond t on eva uates to true for any g ven row

The WHEN NOT MATCHED BY TARGET Clause The ast merge c ause s WHEN NOT MATCHED BY TARGET, as shown n th s sn ppet WHEN NOT MATCHED BY TARGET THEN -- add newly owned stock INSERT VALUES (Trade.Symbol, Trade.Delta);

Th s c ause hand es rows found n the source but not n the target Th s refers to stocks that are be ng traded for the first t me, wh ch s the new Northw nd Traders stock that doesn’t yet ex st n the target Stock tab e The c ause has no pred cate (a though t cou d), and so there are no add t ona ­cond t ons for the c ause Here, you s mp y add the new data found n the Trade tab e to the Stock tab e us ng the INSERT statement

Note The BY TARGET keywords are optional for this clause. WHEN NOT MATCHED is equivalent to WHEN NOT MATCHED BY TARGET. On y one WHEN NOT MATCHED BY TARGET c ause s perm tted n a s ng e MERGE statement It can be qua fied w th an AND cond t on, as you saw ear er w th WHEN MATCHED (There s no ­purpose for an AND cond t on on the WHEN NOT MATCHED BY TARGET c ause n th s examp e ) After execut ng the MERGE statement n L st ng 2-17, the Stock tab e s updated to reflect a the trades of the day merged nto t, as shown here SELECT * FROM Stock GO Symbol ---------ADVW NWT

Qty ----------15 3

(2 row(s) affected)

Just as des red and expected, B ue Yonder A r nes s gone, Northw nd Traders has been added, and Adventure Works has been updated Th s s a rather mpress ve resu t for just one statement! It took ess code to wr te and w take ess effort to ma nta n than the equ va ent operat ons wr tten as separate statements wou d, and t a so runs faster because t s comp ed and executed as a s ng e statement No add t ona overhead s ncurred for the s mp e reason that th s statement operates on the same fundamenta pr nc p es as your bas c SELECT statement’s FROM and JOIN c auses



Chapter 2  T SQL Enhancements   69

Using MERGE for Table Replication Let’s move on to another examp e that shows how MERGE can be used as a too for ach ev ng s mp e rep cat on between two tab es F rst, define the tab es Original and Replica w th dent ca schemas Then create a stored procedure w th a MERGE statement that rep cates changes made n the Original tab e over to the Replica tab e, as shown n L st ng 2-18 Listing 2-18  Creat ng two tab es and a stored procedure that uses MERGE to synchron ze them.

CREATE TABLE Original(PK int primary key, FName varchar(10), Number int) CREATE TABLE Replica(PK int primary key, FName varchar(10), Number int) GO CREATE PROCEDURE uspSyncReplica AS MERGE Replica AS R USING Original AS O ON O.PK = R.PK WHEN MATCHED AND (O.FName != R.FName OR O.Number != R.Number) THEN UPDATE SET R.FName = O.FName, R.Number = O.Number WHEN NOT MATCHED THEN INSERT VALUES(O.PK, O.FName, O.Number) WHEN NOT MATCHED BY SOURCE THEN DELETE;

The MERGE statement n th s stored procedure hand es the rep cat on task by jo n ng the two t­ ab es on the r pr mary keys (PK) and prov d ng three merge c auses The first c ause processes ­updates, as shown here WHEN MATCHED AND (O.FName != R.FName OR O.Number != R.Number) THEN UPDATE SET R.FName = O.FName, R.Number = O.Number

Here, WHEN MATCHED s used to find a the records that ex st n both the or g na and the rep ca, and then the UPDATE statement updates the match ng rows on the rep ca s de w th the atest or g na data Perform ng such an update when no data has actua y changed s wastefu , so the pred cate qua fies the merge c ause to app y on y when a row change s detected n any of the nonkey va ues between the or g na and the rep ca The second merge c ause hand es nsert ons, as fo ows WHEN NOT MATCHED THEN INSERT VALUES(O.PK, O.FName, O.Number)

As ment oned ear er, WHEN NOT MATCHED s equ va ent to WHEN NOT MATCHED BY TARGET, wh ch returns a the or g na rows not found n the rep ca tab e These records represent new rows added to the or g na tab e s nce the ast merge, wh ch are now added to the rep ca as we us ng the INSERT statement n th s c ause

70  Part 1  Core SQL Server Development

The WHEN NOT MATCHED BY SOURCE Clause The th rd, and ast, merge c ause hand es de et ons, as shown here, fo owed by the requ red ­sem co on term nator WHEN NOT MATCHED BY SOURCE THEN DELETE;

The WHEN NOT MATCHED BY SOURCE c ause serves as the exact reverse of WHEN NOT MATCHED BY TARGET and returns target (rep ca) rows not found n the source (or g na ) tab e Th s scenar o wou d occur as the resu t of remov ng rows from the or g na tab e after they have been rep cated Rows removed from the or g na tab e need to be removed from the rep ca tab e as we , wh ch s accomp shed by the s mp e DELETE statement n th s c ause Regard ng the use of mu t p e c auses, WHEN NOT MATCHED BY SOURCE has very same ru es as WHEN MATCHED You are perm tted up to two WHEN NOT MATCHED BY SOURCE c auses If you have two, the first one must be qua fied w th an AND cond t on Furthermore, one c ause must spec fy an UPDATE, and the other must spec fy a DELETE If two WHEN NOT MATCHED BY SOURCE c auses are spec fied, MERGE w choose one of them to execute for any g ven row based on whether the AND cond t on eva uates to true Let’s get started w th some data Both tab es are empty at th s po nt (wh ch does n fact mean that they are a ready n sync) Start by fi ng the or g na tab e w th a few rows, as shown here INSERT Original VALUES(1, 'Sara', 10) INSERT Original VALUES(2, 'Steven', 20)

Now ca the uspSyncReplica stored procedure to br ng the rep ca tab e up to date The first t me you ca the stored procedure, on y the WHEN NOT MATCHED c ause w execute It w execute tw ce, actua y, once for each of the two rows to be added to rep ca tab e from the or g na tab e The rows affected count d sp ayed after runn ng the stored procedure conveys th s, as shown here EXEC uspSyncReplica GO (2 row(s) affected)

Exam n ng both tab es now ver fies that the rep ca s synchron zed w th the or g na , as shown here SELECT * FROM Original SELECT * FROM Replica GO PK ----------1 2

FName ---------Sara Steven

Number ----------10 20

(2 row(s) affected)



Chapter 2  T SQL Enhancements   71

PK ----------1 2

FName ---------Sara Steven

Number ----------10 20

(2 row(s) affected)

Now perform some more changes to the or g na tab e A m x of nsert, update, and de ete ­operat ons br ngs the two tab es out of sync aga n INSERT INTO Original VALUES(3, 'Andrew', 100) UPDATE Original SET FName = 'Stephen', Number += 10 WHERE PK = 2 DELETE FROM Original WHERE PK = 1 GO SELECT * FROM Original SELECT * FROM Replica GO PK ----------2 3

FName ---------Stephen Andrew

Number ----------30 100

(2 row(s) affected) PK ----------1 2

FName ---------Sara Steven

Number ----------10 20

(2 row(s) affected)

Invok ng the stored procedure once aga n br ngs the two tab es back n sync by rep cat ng changes from the or g na tab e to the rep ca, as fo ows EXEC uspSyncReplica GO (3 row(s) affected)

Th s t me, you can see that three rows were affected as the resu t of one nsert (Andrew), one update (Stephen), and one de ete (Sara) Exam n ng both tab es ver fies that the rep ca s once aga n synchron zed w th the or g na , as shown here SELECT * FROM Original SELECT * FROM Replica GO PK ----------2 3

FName ---------Stephen Andrew

Number ----------30 100

72  Part 1  Core SQL Server Development

(2 row(s) affected) PK ----------2 3

FName ---------Stephen Andrew

Number ----------30 100

(2 row(s) affected)

MERGE Output The MERGE statement supports the same OUTPUT c ause ntroduced n SQL Server 2005 for the INSERT, UPDATE, and DELETE statements Th s c ause returns change nformat on from each row ­affected by DML operat ons n the same INSERTED and DELETED pseudo-tab es exposed by tr ggers Be ng ab e to capture th s nformat on n the OUTPUT c ause s often a better cho ce than captur ng t n tr ggers, because tr ggers ntroduce nondeterm n st c behav or—that s, you cannot guarantee that mu t p e tr ggers on the same tab e w cons stent y fire n the same order every t me (Th s s often the cause of subt e bugs that can be very d fficu t to track down, and thus tr ggers shou d genera y be avo ded when poss b e ) In add t on to the co umns of the INSERTED and DELETED pseudo-tab e, another v rtua co umn named $action s supported when OUTPUT s used w th the MERGE statement The $action ­co umn w return one of the three str ng va ues—‘INSERT’, ‘UPDATE’, or ‘DELETE’—depend ng on the ­act on taken for each row L st ng 2-19 shows a s ght y mod fied vers on of the uspSyncReplica stored ­procedure that nc udes an OUTPUT c ause on the MERGE statement The OUTPUT c ause se ects the v rtua $action co umn and a the co umns of the INSERTED and DELETED pseudo-tab es so that the MERGE statement can prov de a deta ed report of every DML operat on t performs If you have been fo ow ng a ong w th the steps for the prev ous vers on, you’ need to drop everyth ng (the tab es and the stored procedure) first Then, subst tute the mp ementat on of the stored procedure n L st ng 2-18 w th the code shown n L st ng 2-19 and repeat the same steps Listing 2-19  Us ng the OUTPUT c ause and $action v rtua co umn w th MERGE.

CREATE PROCEDURE uspSyncReplica AS MERGE Replica AS R USING Original AS O ON O.PK = R.PK WHEN MATCHED AND (O.FName != R.FName OR O.Number != R.Number) THEN UPDATE SET R.FName = O.FName, R.Number = O.Number WHEN NOT MATCHED THEN INSERT VALUES(O.PK, O.FName, O.Number) WHEN NOT MATCHED BY SOURCE THEN DELETE OUTPUT $action, INSERTED.*, DELETED.*;



Chapter 2  T SQL Enhancements   73

Runn ng th s mod fied vers on of the MERGE statement for the ast set of changes n the rep cat on examp e produces the fo ow ng output $action ------DELETE UPDATE INSERT

PK ----NULL 2 3

FName ------NULL Stephen Andrew

Number -----NULL 30 100

PK ----1 2 NULL

FName ------Sara Steven NULL

Number -----10 20 NULL

(3 row(s) affected)

Th s output shows a the act ons taken by the merge and a the before-and-after data nvo ved Of course, you can use OUTPUT…INTO nstead of just OUTPUT ke you can w th the INSERT, UPDATE, and DELETE statements to send th s nformat on to another tab e for h story ogg ng or add t ona process ng And conc ud ng our treatment of MERGE, you’ earn about the INSERT OVER DML syntax ntroduced n SQL Server 2008 that prov des even greater flex b ty

Choosing a Join Method SQL Server s very smart about exam n ng MERGE statements, ana yz ng d fferent comb nat ons of merge c auses, and automat ca y perform ng the appropr ate type of jo n to return the source and target data needed on both s des of the operat on There are four types of jo ns, and SQL Server ­c ever y p cks the r ght one for the job based on the merge c auses that t finds n the MERGE ­statement, as shown n Tab e 2-2 Table 2-2  Source-to-target tab e jo n types chosen by SQL Server based on your merge c ause(s) Merge clause

Join type

Returns

Valid actions

WHEN MATCHED

nner

Match ng data n both source and target

Update, de ete

WHEN NOT MATCHED [BY TARGET]

Left outer

Source data not found n target

nsert

WHEN NOT MATCHED BY SOURCE

R ght outer

Target data not found n source

Update, de ete

WHEN NOT MATCHED [BY TARGET] comb ned w th any other c ause

Fu outer

Source data not found n target, and other match ng or nonmatch ng target data

nsert, update, de ete

The WHEN MATCHED c ause returns rows that are found n both tab es (that s, rows n both tab es w th match ng va ues n the jo n ng key co umns defined w th ON) The query processor treats th s as an nner jo n, because you are retr ev ng on y rows that ex st on both s des E ther an UPDATE or a DELETE operat on on match ng target rows s perm tted n the WHEN MATCHED c ause WHEN NOT MATCHED [BY TARGET] s treated as a eft outer jo n, because t retr eves on y source rows that do not ex st n the target Therefore, on y an INSERT can be performed n the WHEN NOT MATCHED [BY TARGET] c ause to “fi the ho e” n the target w th the m ss ng source data

74  Part 1  Core SQL Server Development

The WHEN NOT MATCHED BY SOURCE c ause s the comp ement to WHEN NOT MATCHED [BY TARGET] and resu ts n a r ght outer jo n to retr eve on y target rows that do not ex st n the source As w th WHEN MATCHED, you can perform an UPDATE or a DELETE operat on on those target rows m ss ng from the source Last, comb n ng WHEN NOT MATCHED [BY TARGET] w th e ther or both of the other two c auses retr eves source data not found n the target as we as target data found or not found n the source In th s case, a fu outer jo n s performed between the two tab es You can perform an INSERT n the WHEN NOT MATCHED [BY TARGET] c ause to add source data m ss ng from the target and an ­UPDATE or DELETE n the other c ause(s) to mod fy or remove target data

Important  The only valid statement in WHEN MATCHED or WHEN NOT MATCHED BY SOURCE is UPDATE or DELETE, and the only valid statement in WHEN NOT MATCHED [BY TARGET] is INSERT. No other T-SQL statements (including stored procedure calls) are ­allowed in any of these merge clauses. However, the INSERT and UPDATE statements in a merge clause are permitted to reference user-defined functions.

MERGE DML Behavior As you’ve just earned, the MERGE statement comb nes the four separate DML statements (SELECT, INSERT, UPDATE, and DELETE) nvo ved n a merge operat on The actua nterna ­mp ementat on of MERGE s the same as the d st nct DML statements t encapsu ates Th s was a good ca by ­M crosoft, because t means that deve opers can beg n us ng MERGE r ght away to everage ts mproved ­performance and ower ma ntenance benefits, w thout concern for break ng any ex st ng code A AFTER and INSTEAD OF tr ggers that have a ready been defined n ex st ng tab es or ­ pdateab e v ews cont nue to fire when those tab es or updateab e v ews are des gnated as the u target of a MERGE statement For examp e, the WHEN NOT MATCHED THEN…INSERT c ause fires any nsert tr ggers defined for the target; s m ar y, the WHEN MATCHED THEN…DELETE c ause fires de ete ­tr ggers There s no concept of a “merge tr gger,” so t’s safe for you to mmed ate y start us ng MERGE n your T-SQL code Tr ggers w fire just the same as f you used separate statements nstead of MERGE Ex st ng bus ness og c, constra nts, and ru es a cont nue to funct on as they d d before The same opt ons and features you are accustomed to us ng w th the separate DML statements are a so ava ab e w th MERGE For examp e, you can use a CTE for the source of a merge by defin ng t us ng the WITH c ause before the MERGE statement and then referenc ng t w th the USING c ause The TOP c ause s s m ar y supported, as are trad t ona query h nts And of course, both source and target tab es can be a ased w th AS just ke tab es n regu ar quer es can You can see how SQL Server nterna y mp ements the MERGE statement for the stored procedure that hand es tab e rep cat on by exam n ng the database eng ne’s query execut on p an, shown n F gure 2-2



Chapter 2  T SQL Enhancements   75

Clustered Index Merge [Replica] . [PK—Replica—321507874DE... Cost: 45%

Merge Join (Full Outer Join) Cost: 25%

Clustered Index Scan (Clustered) [Original] . [PK—Original—321507874... Cost: 15%

Clustered Index Scan (Clustered) [Replica] . [PK—Replica—321507874DE... Cost: 15%

Figure 2-2  Query p an for a MERGE operat on to synchron ze a rep ca w th an or g na (screen mage has been cropped to fit the pr nted page).

The work beg ns by scann ng the source and target tab e (query p ans are read from r ght to eft) At the r ght of F gure 2-2, you can see the advantage of hav ng created ndexes on the co umns ­jo n ng the Original and Replica tab es for the merge Because the jo n pred cate uses the pr mary keys of both tab es, the query opt m zer uses a c ustered ndex scan for the best poss b e read ­performance You shou d therefore a ways create an ndex (poss b y c ustered and dea y un que) on the jo ned co umns used for merg ng two tab es The p an a so revea s that a fu outer jo n s performed between the two tab es, as shown n the center of F gure 2-2 As ment oned ear er, the type of jo n used depends on wh ch WHEN…THEN merge c auses are (or aren’t) spec fied n the MERGE statement In th s case, SQL Server performs a fu outer jo n because the MERGE statement uses a three of the poss b e merge c auses The c ustered ndex merge operat on that fo ows (at the eft of F gure 2-2) mp ements the pred cate on the first merge c ause that se ects match ng rows on y f any of the nonkey co umns have changed between the or g na and rep ca tab es The c ustered ndex merge consumes the stream de vered by the fu outer jo n and dec des on a row-by-row bas s whether the row be ng passed to th s operat on shou d be processed as an update, an nsert, or a de ete, based on the syntax of the MERGE statement and the data n the source and target tab es

The INSERT OVER DML Syntax As you just saw, the MERGE statement supports the OUTPUT c ause to access the INSERTED and DELETED pseudo-tab es, as we as a spec a $action str ng va ue that returns ‘INSERT’, ‘UPDATE’, or ‘DELETE’ accord ng to the act on performed for each row processed by the merge operat on The resu ts generated by the OUTPUT c ause can be captured n another tab e or tab e v­ ar ab e us ng OUTPUT…INTO, mak ng t poss b e to eas y ma nta n h stor ca records of changes n the database However, because us ng OUTPUT…INTO dumps every row captured by the c ause nto the dest nat on tab e or tab e var ab e, t s not poss b e to fi ter on th s data

76  Part 1  Core SQL Server Development

A Filterable Alternative to OUTPUT…INTO INSERT OVER DML refers to a spec a syntax n wh ch you wrap an INSERT INTO…SELECT statement around any data man pu at on anguage (DML) statement (INSERT, UPDATE, DELETE, or MERGE) that has an OUTPUT c ause, rather than us ng OUTPUT…INTO on the DML statement tse f The subt e but cruc a d fference here s that OUTPUT…INTO captures the changes, wh e the INSERT OVER DML s­ yntax consumes the changes captured by OUTPUT By treat ng the OUTPUT changes as the source for a standard INSERT INTO…SELECT statement, you can app y any WHERE c ause to the INSERT INTO…SELECT statement that you want Beyond that, there sn’t anyth ng more you can do w th INSERT OVER DML than what you can w th OUTPUT…INTO But to casua y over ook the s gn ficance of fi ter ng change data s to m ss the po nt of INSERT OVER DML ent re y By be ng ab e to fi ter change data w th a WHERE c ause, you can ­better contro wh ch data changes captured are sh pped to the dest nat on tab e and wh ch aren’t The ­benefits of th s capab ty are best rea zed w th an examp e In th s scenar o, you are ma nta n ng an aud t of a changes that are posted from one tab e to another You have a master st of book pr ces and she f ocat ons n the Book tab e where each book s dent fied by ts ISBN number Every week, you rece ve a new tab e named WeeklyChanges that ­conta ns updates to the book data, a so keyed by ISBN number In add t on to pr ce and she f changes for ex st ng books, the week y update tab e can a so nc ude new books You earned a ready that MERGE can hand e th s scenar o ( t was made to, n fact), and can effect ve y app y the appropr ate updates and nserts from the WeeklyChanges tab e to the Book tab e by jo n ng on ISBN and us ng the WHEN MATCHED and WHEN NOT MATCHED c auses You a so know that the OUTPUT…INTO c ause can be used to capture and store a data changes performed by the merge nto a h story tab e To work through the scenar o, start first by us ng OUTPUT…INTO to dump these changes to the ­BookHistory tab e Then you’ mod fy the approach to use INSERT OVER DML nstead, and ga n better contro of the change data arch va process Create the Book and WeeklyChange tab es as shown n L st ng 2-20 Listing 2-20  Creat ng the Book and WeeklyChange tab es.

CREATE TABLE Book( ISBN varchar(20) PRIMARY KEY, Price decimal, Shelf int) CREATE TABLE WeeklyChange( ISBN varchar(20) PRIMARY KEY, Price decimal, Shelf int)

Next, create the BookHistory tab e Th s w be the rec p ent of the changes captured by the ­OUTPUT INTO c ause of the MERGE statement, as shown n L st ng 2-21



Chapter 2  T SQL Enhancements   77

Listing 2-21  Creat ng the BookHistory tab e.

CREATE TABLE BookHistory( Action nvarchar(10), NewISBN varchar(20), NewPrice decimal, NewShelf int, OldISBN varchar(20), OldPrice decimal, OldShelf int, ArchivedAt datetime2)

Now create the uspUpdateBooks stored procedure to perform the week y update merge operat on The stored procedure records a data changes to the BookHistory tab e, as shown n L st ng 2-22 Listing 2-22 Us ng OUTPUT…INTO w th MERGE.

CREATE PROCEDURE uspUpdateBooks AS BEGIN MERGE Book AS B USING WeeklyChange AS WC ON B.ISBN = WC.ISBN WHEN MATCHED AND (B.Price WC.Price OR B.Shelf WC.Shelf) THEN UPDATE SET B.Price = WC.Price, B.Shelf = WC.Shelf WHEN NOT MATCHED THEN INSERT VALUES(WC.ISBN, WC.Price, WC.Shelf) OUTPUT $action, inserted.*, deleted.*, SYSDATETIME() INTO BookHistory; END

Th s code shou d requ re no deta ed exp anat on, as we just covered MERGE thorough y n the prev ous sect on What’s d fferent here, however, s the OUTPUT…INTO c ause Th s c ause records the act on, new va ues, o d va ues, and current server date and t me n the BookHistory tab e, from the $action v rtua co umn, INSERTED pseudo-tab e co umns, DELETED pseudo-tab e co umns, and SYSDATETIME funct on (In th s part cu ar scenar o, $action can return on y the str ng ‘INSERT’ or ­‘UPDATE’, because de et ons are not be ng processed ) Now popu ate the Book and WeeklyChange tab es to set the stage for the update, as shown here INSERT INSERT INSERT INSERT

INTO INTO INTO INTO

Book VALUES('A', 100, 1) Book VALUES('B', 200, 2) WeeklyChange VALUES('A', 101, 1) WeeklyChange VALUES('C', 300, 3)

The WeeklyChange tab e shows a change n pr ce for book A from $100 to $101 and a so adds a new book C pr ced at $300 on she f 3 When you run the uspUpdateBooks stored procedure, those 78  Part 1  Core SQL Server Development

two operat ons take p ace as you’d expect Th s s confirmed by the number of rows affected message, as shown here EXEC uspUpdateBooks GO (2 row(s) affected)

In add t on, the act ons and data changes made by those nsert and update operat ons (and the dates and t mes that they occurred) have been saved to the BookHistory tab e as the resu t of the OUTPUT…INTO c ause, as shown here SELECT * FROM BookHistory GO Action -----UPDATE INSERT

NewISBN ------A C

NewPrice -------101 300

NewShelf -------1 3

OldISBN ------A NULL

OldPrice -------100 NULL

OldShelf -------1 NULL

ArchivedAt --------------------------2012-02-25 14:47:23.9907552 2012-02-25 14:47:23.9907552

So far, so good, r ght? Unt , after further cons derat on, you rea ze that most of your operat ons w be nserts, and there w be very few updates Thus, an unacceptab e amount of storage s go ng to be requ red for captur ng the nserts that have no o d va ues and that therefore don’t prov de any mean ngfu h stor ca va ue beyond the creat on date and t me (wh ch you cou d store n the Book tab e f you wanted to) G ven that every book must be nserted but most books are not typ ca y ­updated, the resu t w be a d sproport onate amount of nserts over updates n the BookHistory tab e That’s a ot of storage to ho d dup cate data that has tt e or no va ue Unfortunate y, there’s s mp y no way to fi ter out the nserts and save just the updates us ng OUTPUT…INTO In our rev sed scenar o, you are nterested n captur ng on y updates w thout nserts Th s requ res fi ter ng, and INSERT OVER DML prov des the so ut on You w use INSERT OVER DML n the next examp e, and—just to keep th ngs nterest ng—the h stor ca data w be appended to the Book tab e tse f, rather than be ng stored to a separate ­h story tab e In the rev sed examp e, there s an ArchivedAt co umn n the Book tab e that conta ns the server date and t me for h story records added when books are updated Thus, a NULL n th s ­co umn ­dent fies the rows ho d ng the current va ues, whereas a other rows w th non-NULL va ues n the ­ArchivedAt co umn represent prev ous va ues n h story rows Start ke you d d the ast t me by creat ng the tab es, as shown n L st ng 2-23 Listing 2-23  Creat ng new vers ons of the Book and WeeklyChange tab es for use w th NSERT OVER DML.

-- Cleanup from previous example DROP TABLE Book DROP TABLE WeeklyChange DROP TABLE BookHistory DROP PROCEDURE uspUpdateBooks GO



Chapter 2  T SQL Enhancements   79

CREATE TABLE Book( ISBN varchar(20), Price decimal, Shelf int, ArchivedAt datetime2) CREATE UNIQUE CLUSTERED INDEX UI_Book ON Book(ISBN, ArchivedAt) CREATE TABLE WeeklyChange( ISBN varchar(20) PRIMARY KEY, Price decimal, Shelf int)

Not ce th s t me that a un que ndex s created on the comb ned ISBN and ArchivedAt co umns A pr mary key cannot be created on just the ISBN co umn as before, because the Book tab e w now conta n mu t p e records w th the same ISBN number (because the tab e ho ds h stor ca change data as we as current data) On y one row per ISBN number w have a NULL va ue n the ­ArchivedAt ­co umn, wh ch s the row that conta ns the current va ues for a book A other rows w th the same ISBN number w have non-NULL va ues n ArchivedAt, wh ch serves as the t me stamp of a ­prev ous update and w conta n the h stor ca va ues arch ved for that book at that t me By ­ensur ng ­un queness between both ISBN and ArchivedAt, you are guaranteed that on y one current row for each book s ever stored n the Book tab e, because the tab e w not to erate two rows w th the same ISBN numbers and NULL va ues for ArchivedAt

Consuming CHANGES You w now mp ement a mod fied vers on of the uspUpdateBooks n L st ng 2-24 that uses the new INSERT OVER DML syntax to capture and store h stor ca data for updates that are appended to the Book tab e Listing 2-24 Us ng MERGE w th NSERT OVER DML.

CREATE PROCEDURE uspUpdateBooks AS BEGIN INSERT INTO Book(ISBN, Price, Shelf, ArchivedAt) SELECT ISBN, Price, Shelf, GETDATE() FROM (MERGE Book AS B USING WeeklyChange AS WC ON B.ISBN = WC.ISBN AND B.ArchivedAt IS NULL WHEN MATCHED AND (B.Price WC.Price OR B.Shelf WC.Shelf) THEN UPDATE SET Price = WC.Price, Shelf = WC.Shelf WHEN NOT MATCHED THEN INSERT VALUES(WC.ISBN, WC.Price, WC.Shelf, NULL) OUTPUT $action, WC.ISBN, Deleted.Price, Deleted.Shelf ) CHANGES(RowAction, ISBN, Price, Shelf) WHERE RowAction = 'UPDATE'; END

80  Part 1  Core SQL Server Development

Because there s h stor ca data n the Book tab e that s rre evant for the merge operat on, c­ r ter a has been added to the jo n pred cate after the ON keyword that tests for NULL va ues n the ­ArchivedAt co umn Th s te s MERGE to cons der on y current books as match ng target rows aga nst the source of week y changes and to comp ete y gnore a the arch ved h story records The CHANGES keyword s what makes INSERT OVER DML poss b e CHANGES exposes the ­co umns of the OUTPUT c ause defined on the nner MERGE statement to the WHERE c ause of the outer INSERT INTO…SELECT statement In L st ng 2-24, th s nc udes the v rtua $action co umn, the ISBN number, and the o d va ues for pr ce and she f be ng rep aced by the update operat on By expos ng the v rtua $action through the CHANGES keyword as RowAction, the INSERT INTO… SELECT statement can app y a WHERE c ause to fi ter out the act ons of new y nserted books and ­append on y the act ons of changed books back nto the Book tab e As exp a ned, th s s someth ng that cannot be ach eved us ng the OUTPUT…INTO c ause on the MERGE statement tse f, but s ­poss b e us ng th s very spec fic INSERT OVER DML syntax The code fi ters out nsert act ons w th the s mp e cr ter a WHERE RowAction = ‘UPDATE’ You cou d of course app y even more soph st cated og c than that f you wanted to For examp e, you may w sh to save h story data for certa n users, reg ons, or any other changed data co umns returned by the OUTPUT c ause and exposed by the CHANGES keyword And that’s exact y the key to the power of INSERT OVER DML Wa k through the scenar o step by step Start w th two books (A and B) and one pr ce change (book A from $100 to $110), as shown here INSERT INTO Book VALUES('A', 100, 1, NULL) INSERT INTO Book VALUES('B', 200, 2, NULL) INSERT INTO WeeklyChange VALUES('A', 110, 1) SELECT * FROM Book SELECT * FROM WeeklyChange GO ISBN ---A B

Price ----100 200

Shelf ----1 2

ArchivedAt ----------------------NULL NULL

(2 row(s) affected) ISBN Price Shelf ---- ----- ----A 110 1 (1 row(s) affected)

When you execute uspUpdateBooks, the nner MERGE statement w update the pr ce for book A n the Book tab e and send ts or g na va ues to the OUTPUT c ause The OUTPUT c ause va ues are then consumed by the outer INSERT INTO…SELECT statement v a the CHANGES keyword The outer statement can therefore use those or g na book va ues for nsert ng h stor ca data back nto the

Chapter 2  T SQL Enhancements   81

Book tab e w th an ArchivedAt va ue set to the current server date and t me (just before 5 00 PM on 2/25/2012, n th s examp e), as fo ows EXEC uspUpdateBooks SELECT * FROM Book GO ISBN ---A A B

Price ----110 100 200

Shelf ----1 1 2

ArchivedAt --------------------------NULL 2012-02-25 16:57:19.8600000 NULL

(3 row(s) affected)

You can see the current data for book A (the row w th an ArchivedAt date of NULL) at $110, and you a so see the prev ous data for book A, wh ch was changed from $100 at about 5 00 PM on 2/25/2012 Somet me ater, you rece ve a new set of changes Th s t me, book A has changed from she f 1 to she f 6, and a new book C has been added, as shown here DELETE FROM WeeklyChange INSERT INTO WeeklyChange VALUES('A', 110, 6) INSERT INTO WeeklyChange VALUES('C', 300, 3) GO

Just ke the first t me you ran the stored procedure, the current row for book A s updated, and a snapshot of the prev ous contents of the row s added to the tab e w th the current server date and t me, as shown here EXEC uspUpdateBooks SELECT * FROM Book GO ISBN ---A A A B C

Price ----110 100 110 200 300

Shelf ----6 1 1 2 3

ArchivedAt --------------------------NULL 2012-02-25 16:57:19.8600000 2012-02-25 16:58:36.1900000 NULL NULL

(5 row(s) affected)

Now book A has two h story records show ng the va ues saved from two ear er updates ­dent fied by date and t me va ues n the ArchivedAt co umn There s a so a new row for book C, wh ch was nserted by the WHEN NOT MATCHED c ause of the MERGE statement However, not ce that there s no h story row for book C, because the nsert act on that was ­actua y captured by the MERGE statement’s OUTPUT c ause was subsequent y fi tered out n the WHERE c ause of the outer INSERT INTO…SELECT statement Had you not fi tered out nsert act ons n that WHERE c ause, another h story record wou d have a so been added to the tab e for book C w th mean ng ess NULL va ues for both Price and Shelf Therefore, by fi ter ng OUTPUT act ons by us ng 82  Part 1  Core SQL Server Development

INSERT OVER DML, you avo d the pro ferat on of h story rows that wou d get generated w th each new book You a so e m nated the extra co umns for the new va ues that were used n the ­prev ous examp e’s BookHistory tab e, as they are stored n each updated vers on arch ved n the Book tab e tse f And you e m nated the need to store an Action co umn, because on y update act ons are ­be ng captured In the end, a ot of benefit was der ved by e m nat ng need ess storage, and t was a ­accomp shed w th a s ng e INSERT OVER DML statement As you can see, the comb nat on of the MERGE and INSERT OVER DML features n SQL Server s a very powerfu add t on to T-SQL A fu y oaded MERGE (hand ng nserts, updates, and de etes) wrapped up us ng INSERT OVER DML de vers an enormous amount of funct ona ty n a s ng e, ­manageab e statement We recommend us ng these new features n eu of the trad t ona mu t -statement approaches n whatever future deve opment scenar os you find t poss b e to do so In add t on to mproved performance, you’ apprec ate the greater manageab ty that resu ts from conso dat ng mu t p e statements nto one

The GROUPING SETS Operator The GROUP BY c ause has been part of the SELECT statement syntax s nce the ear est d a ects of T-SQL You use GROUP BY to create quer es that co apse mu t p e rows be ong ng to the same group nto a s ng e summary row and perform aggregate ca cu at ons (such as SUM and AVG) across the ­nd v dua rows of each group SQL Server 6 5 ater extended the GROUP BY c ause by add ng the WITH CUBE and WITH ROLLUP operators These operators perform add t ona group ng and ­aggregat on of data n standard re at ona quer es, s m ar to what s prov ded by on ne ana yt ca process ng (OLAP) quer es that s ce and d ce your data nto Ana ys s Serv ces cubes, but w thout ever eav ng the re at ona database wor d SQL Server 2008 added the GROUPING SETS operator that further extends the capab t es of the GROUP BY c ause for summar z ng and ana yz ng your data In th s sect on, you w exam ne GROUP BY n many of ts var ant forms You’ start w th the bas c GROUP BY c ause, and then you’ earn how the WITH CUBE and WITH ROLLUP operators can be used to enhance those summary resu ts Then you’ exp ore the GROUPING SETS operator added n SQL Server 2008 Start w th a s mp e nventory tab e that conta ns quant t es for var ous tems n d verse co ors that are ava ab e at d fferent store ocat ons, as shown n L st ng 2-25 Listing 2-25  Creat ng the Inventory tab e.

CREATE TABLE Inventory( Store varchar(2), Item varchar(20), Color varchar(10), Quantity decimal)



Chapter 2  T SQL Enhancements   83

Next, add some nventory data There are 13 rows that conta n nventory for tab es, cha rs, and sofas ava ab e n b ue, red, and green, at NY, NJ, and PA ocat ons, as shown n L st ng 2-26 Listing 2-26  Popu at ng the Inventory tab e.

INSERT INSERT INSERT INSERT INSERT INSERT INSERT INSERT INSERT INSERT INSERT INSERT INSERT

INTO INTO INTO INTO INTO INTO INTO INTO INTO INTO INTO INTO INTO

Inventory Inventory Inventory Inventory Inventory Inventory Inventory Inventory Inventory Inventory Inventory Inventory Inventory

VALUES('NY', VALUES('NJ', VALUES('NY', VALUES('NJ', VALUES('PA', VALUES('NY', VALUES('PA', VALUES('NY', VALUES('NJ', VALUES('NY', VALUES('NJ', VALUES('PA', VALUES('NJ',

'Table', 'Blue', 124) 'Table', 'Blue', 100) 'Table', 'Red', 29) 'Table', 'Red', 56) 'Table', 'Red', 138) 'Table', 'Green', 229) 'Table', 'Green', 304) 'Chair', 'Blue', 101) 'Chair', 'Blue', 22) 'Chair', 'Red', 21) 'Chair', 'Red', 10) 'Chair', 'Red', 136) 'Sofa', 'Green', 2)

Now use a bas c GROUP BY c ause to query on th s data SELECT Item, Color, SUM(Quantity) AS TotalQty, COUNT(Store) AS Stores FROM Inventory GROUP BY Item, Color ORDER BY Item, Color

As mp ed by ts syntax, th s query groups a the nventory records by tem and then by co or w th n each tem The resu t set therefore nc udes one summary row for each un que comb nat on of tems and co ors The store ocat on s not nc uded n the group ng, and so the resu ts returned by the query app y to a stores Each summary row nc udes a TotalQty co umn ca cu ated by the SUM aggregate funct on as the tota quant ty for a rows of the same tem and co or across a stores The ast co umn, Stores, s ca cu ated by the COUNT aggregate funct on as the number of store ocat ons at wh ch each un que comb nat on of tems and co ors s ava ab e, as shown here Item -------------------Chair Chair Sofa Table Table Table

Color ---------Blue Red Green Blue Green Red

TotalQty -----------------------------123 167 2 224 533 223

Stores -----2 3 1 2 2 3

(6 row(s) affected)

These resu ts show that SQL Server grouped the nventory records shar ng the same tem and co or nto a s ng e summary row The store ocat on s not nc uded n the breakdown, because you d d not group by t, and so each summary row app es to a stores For each tem, the tota quant ty s ­ca cu ated as the sum of the nd v dua quant ty va ues for the tem and co or comb nat ons n each group, and the store count s ca cu ated as the number of store ocat ons at wh ch each tem and 84  Part 1  Core SQL Server Development

co or comb nat on s ava ab e W th GROUP BY, every co umn returned by the query must be e ther one of the co umns actua y be ng grouped by (such as the Store, Item, and Color co umns) or an aggregate funct on that operates across a the comb ned member rows for the group [such as the SUM(Quantity) and COUNT(Store) funct ons]

Rolling Up by Level Th s query demonstrates the most bas c app cat on of the GROUP BY c ause, wh ch s mp y groups and aggregates It answers the quest on “How many tems per co or are n each store ocat on?” by ­group ng tems and co ors The WITH ROLLUP and WITH CUBE operators (wh ch were ntroduced n SQL Server 6 5) can be used to answer more quest ons than that Each of these operators supp ements the resu ts of an ord nary GROUP BY c ause w th add t ona summary aggregat ons on the under y ng data Here s the same query you ran before, on y th s t me us ng WITH ROLLUP SELECT Item, Color, SUM(Quantity) AS TotalQty, COUNT(Store) AS Stores FROM Inventory GROUP BY Item, Color WITH ROLLUP ORDER BY Item, Color GO Item -------------------NULL Chair Chair Chair Sofa Sofa Table Table Table Table

Color ---------NULL NULL Blue Red NULL Green NULL Blue Green Red

TotalQty -----------------------------1272 290 123 167 2 2 980 224 533 223

Stores -----13 5 2 3 1 1 7 2 2 3

(10 row(s) affected)

Th s t me, you rece ve the same s x grouped resu ts as before, supp emented w th four add t ona rollup rows (the ones w th NULL va ues for Item or Color, h gh ghted here n bo d) Ro up rows ­conta n add t ona h gher- eve summary nformat on that essent a y “groups the groups” of the query resu ts Any row w th NULL va ues n t s a ro up row, and the NULL shou d be thought of as “a ­va ues” n th s context In these resu ts, the first row s the top- eve ro up, as nd cated by NULL va ues for both Item and Color Th s top- eve ro up reports a grand tota quant ty of 1,272 for the ent re set (a tems n a co ors) n a store ocat ons (where the ent re set cons sts of the 13 un que tem/co or comb nat ons across a ocat ons) The next resu t s an tem- eve ro up for cha rs It reports a tota quant ty of 290 for cha rs n a co ors across 5 store ocat ons The two resu ts that fo ow are the same summary rows for cha rs ­returned by the first “p a n” GROUP BY query and that were just ro ed up They show 123 b ue cha rs n 2 ocat ons and 167 red cha rs n 3 ocat ons The next resu t s an tem- eve ro up for sofas On y

Chapter 2  T SQL Enhancements   85

one store ocat on carr es sofas, and they’re ava ab e on y n green The sofa ro up therefore conta ns the same va ues as the one and on y summary row for 2 green sofas ava ab e n 1 ocat on The ast set of rows report on tab es n the same way that the cha r and sofa data was returned Th s nc udes an tem- eve ro up show ng 980 tab es across 7 ocat ons fo owed by the summary rows ­show ng 224 b ue tab es n 2 ocat ons, 533 green tab es n 2 ocat ons, and 223 red tab es n 3 ocat ons ­returned So by s mp y add ng WITH ROLLUP, you can answer a second quest on that the first ord nary GROUP BY query cou dn’t “How many cha rs, tab es, and sofas are n stock, regardless of co or?”

Rolling Up All Level Combinations Us ng WITH CUBE now nstead of WITH ROLLUP takes th s resu t set to the next eve , as shown here SELECT Item, Color, SUM(Quantity) AS TotalQty, COUNT(Store) AS Stores FROM Inventory GROUP BY Item, Color WITH CUBE ORDER BY Item, Color GO Item -------------------NULL NULL NULL NULL Chair Chair Chair Sofa Sofa Table Table Table Table

Color ---------NULL Blue Green Red NULL Blue Red NULL Green NULL Blue Green Red

TotalQty -----------------------------1272 347 535 390 290 123 167 2 2 980 224 533 223

Stores -----13 4 3 6 5 2 3 1 1 7 2 2 3

(13 row(s) affected)

You now have the same resu t set returned by WITH ROLLUP, on y th s t me three more ro up rows have been added (aga n, nd cated n bo d here) Let’s ook at exact y what SQL Server d d By ­app y ng WITH CUBE, you nstructed the database eng ne to create a mu t d mens ona representat on of the data on the fly, wh ch s oose y referred to as a cube The number of d mens ons n the cube s based on the number of group ng co umns Th s nventory examp e has on y two d mens ons, but a query cou d have many more d mens ons f t spec fies more group ng co umns A cube conta ns ro ups for a the poss b e permutat ons of d mens on va ues, not just the comb nat ons of one va ue within another, as per the nest ng eve s defined by group ng co umns sted n the GROUP BY c ause So WITH CUBE returns the same ro ups returned by WITH ROLLUP—wh ch nc udes a tems regard ess of co or—p us add t ona ro ups for a co ors regard ess of tem As a resu t, you can now answer a th rd quest on that the ear er GROUP BY quer es cou dn’t “How many tems of any type n 86  Part 1  Core SQL Server Development

a part cu ar co or are n stock?” That means that you can now a so see how many b ue, green, or red tems you have n nventory regard ess of whether they are cha rs, sofas, or tab es Because a cube ro s up every poss b e comb nat on of d mens on va ues ndependent of the order of eve s expressed w th GROUP BY, each add t ona group ng eve ncreases the s ze of the resu t set exponent a y For examp e, f you mod fy the query to group by store ocat on as we , SQL Server returns 44 rows nc ud ng ro ups for every poss b e comb nat on of va ues across the three group ng co umns Store, Item, and Color, as fo ows SELECT Store, Item, Color, SUM(Quantity) AS TotalQty FROM Inventory GROUP BY Store, Item, Color WITH CUBE ORDER BY Store, Item, Color GO Store ----NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL NJ NJ NJ NJ NJ NJ NJ NJ NJ NJ NJ NJ NY NY NY NY NY NY NY NY NY NY NY PA



Item -------------------NULL NULL NULL NULL Chair Chair Chair Sofa Sofa Table Table Table Table NULL NULL NULL NULL Chair Chair Chair Sofa Sofa Table Table Table NULL NULL NULL NULL Chair Chair Chair Table Table Table Table NULL

Color ---------NULL Blue Green Red NULL Blue Red NULL Green NULL Blue Green Red NULL Blue Green Red NULL Blue Red NULL Green NULL Blue Red NULL Blue Green Red NULL Blue Red NULL Blue Green Red NULL

TotalQty -----------------------------1272 347 535 390 290 123 167 2 2 980 224 533 223 190 122 2 66 32 22 10 2 2 156 100 56 504 225 229 50 122 101 21 382 124 229 29 578

Chapter 2  T SQL Enhancements   87

PA PA PA PA PA PA PA

NULL NULL Chair Chair Table Table Table

Green Red NULL Red NULL Green Red

304 274 136 136 442 304 138

(44 row(s) affected)

These resu ts can now answer nventory quest ons for every conce vab e comb nat on of group ng eve s For examp e, across a ocat ons, there are 347 b ue tems (tab es, cha rs, and sofas), 290 cha rs (a co ors), and 533 green tab es, whereas n NY spec fica y, there are 50 red tems, 382 tab es, and 124 b ue tab es, and so on Every permutat on of store ocat on, tem, and co or—and the r ro ups —are returned by th s s ng e query

Returning Just the Top Level The ast var at on on GROUP BY s the GROUPING SETS operator ntroduced n SQL Server 2008 Th s operator returns just the top- eve ro up rows for each group ng eve and does not nc ude the actua group eve summary nformat on that was returned by ear er vers ons of the query, as fo ows SELECT Store, Item, Color, SUM(Quantity) AS TotalQty FROM Inventory GROUP BY GROUPING SETS (Store, Item, Color) ORDER BY Store, Item, Color GO Store ----NULL NULL NULL NULL NULL NULL NJ NY PA

Item -------------------NULL NULL NULL Chair Sofa Table NULL NULL NULL

Color ---------Blue Green Red NULL NULL NULL NULL NULL NULL

TotalQty -----------------------------347 535 390 290 2 980 190 504 578

(9 row(s) affected)

GROUPING SETS s mere y another var at on on GROUP BY that you can use when you requ re on y top- eve ro ups for each of your group ng eve s (that s, one set of group ro ups per eve ) In th s case, you get a tota quant ty report for a co ors, a tems, and a store ocat ons w thout ­nc ud ng the summary rows for each of the comb nat ons of group ng eve s, just as ear er vers ons of the query d d

88  Part 1  Core SQL Server Development

Mixing and Matching But the GROUPING SETS story doesn’t end here, of course Un ke WITH ROLLUP and WITH CUBE— wh ch are mutua y exc us ve n the same query—ro up and cube operat ons can be used together and w th GROUPING SETS n any comb nat on Th s means that you can compose one query that returns on y top- eve ro ups for certa n group ng eve s and a so returns the ower- eve ro ups and summary rows for other group ng eve s, just ke you get us ng WITH ROLLUP or WITH CUBE n ­separate quer es To ach eve th s, SQL Server prov des an a ternat ve syntax for WITH ROLLUP and WITH CUBE that makes these operators capab e of be ng expressed w th one another n the same GROUP BY c ause Th s syntax s actua y qu te s mp e drop the WITH keyword, p ace the ROLLUP or CUBE keyword ­before the group ng co umns rather than after, and enc ose the group ng co umns n parentheses For examp e, the fo ow ng two GROUP BY c auses are nterchangeab e GROUP BY Item, Color WITH ROLLUP GROUP BY ROLLUP(Item, Color)

S m ar y, these two c auses are a so equ va ent GROUP BY Item, Color WITH CUBE GROUP BY CUBE(Item, Color)

A though the two vers ons are nterchangeab e when used on the r own, you must use the newer syntax f you want to comb ne them w th one another or w th GROUPING SETS n a s ng e query Here s another vers on of the nventory query that does just that SELECT Store, Item, Color, SUM(Quantity) AS TotalQty FROM Inventory GROUP BY GROUPING SETS(Store), CUBE(Item, Color) ORDER BY Store, Item, Color

The GROUP BY c ause n th s query nc udes both a GROUPING SETS operator on Store and a CUBE operator on Item and Color Th s te s SQL Server to return top- eve ro ups on y on the Store co umn and fu summar es w th mu t d mens ona ro ups on the Item and Color co umns Here are the resu ts Store ----NJ NJ NJ NJ NJ NJ NJ NJ NJ NJ NJ



Item -------------------NULL NULL NULL NULL Chair Chair Chair Sofa Sofa Table Table

Color ---------NULL Blue Green Red NULL Blue Red NULL Green NULL Blue

TotalQty -----------------------------190 122 2 66 32 22 10 2 2 156 100

Chapter 2  T SQL Enhancements   89

NJ NY NY NY NY NY NY NY NY NY NY NY PA PA PA PA PA PA PA PA

Table NULL NULL NULL NULL Chair Chair Chair Table Table Table Table NULL NULL NULL Chair Chair Table Table Table

Red NULL Blue Green Red NULL Blue Red NULL Blue Green Red NULL Green Red NULL Red NULL Green Red

56 504 225 229 50 122 101 21 382 124 229 29 578 304 274 136 136 442 304 138

(31 row(s) affected)

The rows w th NULL va ues for both Item and Color (h gh ghted here n bo d) are the top- eve ro ups for Store returned by the GROUPING SETS(Store) operator These rows report just the t­ota s for each store (a tems, a co ors) A of the other rows are the mu t d mens ona ro up and ­summary resu ts returned by CUBE(Item, Color) These rows report aggregat ons for every ­comb nat on of Item and Color Because Store s returned by GROUPING SETS and not by CUBE, you don’t see ­comb nat ons that nc ude a stores You can use GROUPING SETS, ROLLUP, and CUBE n any comb nat on you want w th the GROUP BY c ause As a resu t, you ga n tremendous flex b ty for group ng, aggregat ng, and ana yz ng your data just the way you need to The on y restr ct on n usage s the same one that app es when us ng GROUP BY on ts own co umns returned by the query must be spec fied e ther n the GROUP BY c ause ( n any of the GROUPING SETS, ROLLUP, or CUBE operators) or n an aggregate funct on that operates across a the comb ned rows for the group (such as SUM, COUNT, MIN, MAX, and so on)

Handling NULL Values We’ conc ude the d scuss on of GROUPING SETS by d scuss ng NULL va ues As you’ve seen, SQL Server returns NULL va ues to represent a va ues n h gh- eve ro up rows If you’re fortunate enough to be work ng w th data that s guaranteed not to conta n NULL va ues, fe s good for you But th s s far more often not the case, and thus a prob em ar ses d st ngu sh ng between “rea ” NULL va ues and the NULL va ues represent ng “a va ues” n ro up rows To demonstrate, add two more rows to the Inventory tab e for amps that have no co or ­assoc at on These rows store NULL va ues n the Color co umn, as shown n L st ng 2-27

90  Part 1  Core SQL Server Development

Listing 2-27  ntroduc ng NULL va ues nto the Inventory tab e.

INSERT INTO Inventory VALUES('NY', 'Lamp', NULL, 36) INSERT INTO Inventory VALUES('NJ', 'Lamp', NULL, 8)

Now run the exact same query you ran before SELECT Store, Item, Color, SUM(Quantity) AS TotalQty FROM Inventory GROUP BY GROUPING SETS(Store), CUBE(Item, Color) ORDER BY Store, Item, Color GO Store ----NJ NJ NJ NJ : NJ NJ NY NY NY NY : PA PA

Item -------------------NULL NULL NULL NULL

Color ---------NULL NULL Blue Green

TotalQty -----------------------------8 198 122 2

Table Table NULL NULL NULL NULL

Blue Red NULL NULL Blue Green

100 56 36 540 225 229

Table Table

Green Red

304 138

(37 row(s) affected)

These are very confus ng resu ts Because both the “a co ors” ro up co umns and the amp co umns w th “no co or” have a NULL va ue for Color, t s mposs b e to d st ngu sh between the two when ana yz ng the query resu ts For examp e, the first row returns the ro up for a tems w th no co or n NJ (that’s the 8 amps), and the second row returns the ro up for a tems n all co ors n NJ, but there s no way to d scern that d fference because NULL s used to represent both “no co or” and “a co ors ” The same prob em occurs aga n further down n the resu ts for NY, where there are a so co or ess amps n stock Once aga n, because “no co or” and “a co ors” are both represented by NULL va ues, the resu ts are noth ng short of perp ex ng The so ut on to th s prob em s to use the GROUPING funct on n your query The GROUPING funct on returns a b t va ue of 1 (true) f the co umn passed to t represents an “a va ues” ro up, and t returns 0 (fa se) otherw se It s therefore poss b e to d st ngu sh between “a va ues” ro up co umns (wh ch are a ways NULL) and regu ar data (wh ch might be NULL, as s the case for the amps, wh ch have no co or va ues) Here s a rev sed vers on of the query that uses the GROUPING funct on n conjunct on w th CASE to produce a better resu t set that c ears up the confus on between “a va ues” and “no va ue”



Chapter 2  T SQL Enhancements   91

SELECT CASE WHEN GROUPING(Store) = 1 CASE WHEN GROUPING(Item) = 1 CASE WHEN GROUPING(Color) = 1 SUM(Quantity) AS TotalQty FROM Inventory GROUP BY GROUPING SETS(Store), ORDER BY Store, Item, Color

THEN '(all)' ELSE Store END AS Store, THEN '(all)' ELSE Item END AS Item, THEN '(all)' ELSE Color END AS Color,

CUBE(Item, Color)

The CASE construct tests each group ng co umn returned by the query us ng the GROUPING funct on If t returns 1 (true), that means that the co umn represents an “a va ues” ro up In th s case, the str ng (all) s returned, rather than the NULL va ue that wou d have otherw se been returned If t returns 0 (fa se), the co umn conta ns regu ar data, wh ch m ght or m ght not be NULL A though there are NULL va ues on y n for the Color co umn for amps, app y the same CASE and GROUPING to the Store and Item co umns as we Th s s a defens ve cod ng measure aga nst the poss b ty of the Store or Item co umn a so conta n ng NULL va ues n the future Tak ng th s approach now reso ves the confus on w th respect to NULL va ues and ro ups n the query resu ts, as shown here Store ----NJ NJ NJ NJ : NJ NJ NY NY NY NY : PA PA

Item -------------------(all) (all) (all) (all)

Color ---------NULL (all) Blue Green

TotalQty -----------------------------8 198 122 2

Table Table (all) (all) (all) (all)

Blue Red NULL (all) Blue Green

100 56 36 540 225 229

Table Table

Green Red

304 138

(37 row(s) affected)

It’s perfect y understandab e now that the first row returns the ro up for a tems w th no co or n NJ, whereas the second row returns the ro up for a tems n a co ors ( nc ud ng no co or) n NJ The same s true farther down n the NY resu ts, where there are a so co or ess amps n stock Therefore, to avo d any potent a confus on concern ng NULL va ues n your group ng quer es, you shou d a ways use the ­GROUPING funct on n th s manner to trans ate the NULL va ues that mean “a va ues” for the user As ong as you’re mod fy ng the query to produce more readab e resu ts, enhance t one more t me In the same way that you trans ated the NULL for “a va ues” to the text (all), you can trans ate the NULL va ues for regu ar “m ss ng” data to (n/a) Th s s easy to do by add ng an ELSE c ause to the CASE construct that uses the ISNULL funct on on the co umn, as shown here SELECT CASE WHEN GROUPING(Store) = 1 THEN '(all)' ELSE ISNULL(Store, '(n/a)') END AS Store, CASE WHEN GROUPING(Item) = 1 THEN '(all)'

92  Part 1  Core SQL Server Development

ELSE ISNULL(Item, '(n/a)') END AS Item, CASE WHEN GROUPING(Color) = 1 THEN '(all)' ELSE ISNULL(Color, '(n/a)') END AS Color, SUM(Quantity) AS TotalQty FROM Inventory GROUP BY GROUPING SETS(Store), CUBE(Item, Color) ORDER BY Store, Item, Color

The ELSE c ause n each CASE construct runs f the GROUPING funct on returns 0 (fa se) Th s means that the co umn s not an “a va ues” ro up, but regu ar co umn data You want regu ar co umn data to be returned as s, except for NULL va ues that shou d be returned as the str ng (n/a) The ISNULL funct on tests for NULL va ues and performs the trans at on on them, as shown n the resu ts returned by the query Store ----NJ NJ NJ NJ : NJ NJ NJ NJ NJ NJ NY NY NY : NY NY NY NY NY : PA PA

Item -------------------(all) (all) (all) (all)

Color ---------(all) (n/a) Blue Green

TotalQty -----------------------------198 8 122 2

Sofa Stool Stool Table Table Table (all) (all) (all)

Green (all) (n/a) (all) Blue Red (all) (n/a) Blue

2 8 8 156 100 56 540 36 225

Chair Stool Stool Table Table

Red (all) (n/a) (all) Blue

21 36 36 382 124

Table Table

Green Red

304 138

(37 row(s) affected)

The query now returns no NULL va ues at a , wh ch s much better for your users who don’t rea y know or care exact y what NULL means anyway By trans at ng these va ues appropr ate y to (all) and (n/a), you have produced a far more usab e report for them

Windowing (OVER Clause) Enhancements The first windowing capab t es appeared n SQL Server 2005 w th the ntroduct on of the OVER c ause and a set of four rank ng funct ons ROW NUMBER, RANK, DENSE RANK, and NTILE In our ­d scuss on, the term “w ndow” refers to the scope of v s b ty from one row n a resu t set r­ e at ve

Chapter 2  T SQL Enhancements   93

to ne ghbor ng rows n the same resu t set By defau t, OVER produces a s ng e w ndow over the ­ent re resu t set, but ts assoc ated PARTITION BY c ause ets you d v de the resu t set up nto ­mu t p e groups, each conta ned ns de the r own w ndow The row sequence w th n each w ndow s ­determ ned by an assoc ated ORDER BY c ause, and based on th s sequence, the rank ng funct ons ass gn an accumu at ng va ue to the rows n the w ndow In add t on to the rank ng funct ons, the OVER c ause can be used w th the trad t ona ­aggregate funct ons SUM, COUNT, MIN, MAX, and AVG When do ng so, you do not spec fy the GROUP BY c ause that’s norma y requ red w th the aggregate funct ons Instead, each row ca cu ates an ­aggregat on based on the w ndow of rows defined w th OVER, opt ona y grouped us ng ­PARTITION BY Th s s certa n y usefu , because t a ows you to obta n aggregat ons w thout be ng forced to ­conso date (and ose) deta rows w th a GROUP BY c ause But unfortunate y (unt now), the ­aggregate funct ons cou d not a so use ORDER BY n the OVER c ause (as s requ red when us ng OVER w th the rank ng funct ons), mak ng t mposs b e to ca cu ate cumulative aggregat ons at the row eve w th n each w ndow For examp e, you cou d use AVG w th OVER (and, opt ona y PARTITION BY), but w thout an assoc ated ORDER BY, there s no des gnated sequence to the rows n each w ndow, ­mak ng t mposs b e for SQL Server to compute a runn ng average from one row to the next w th n the w ndow Thus, the best that AVG w th OVER cou d do s compute the average for all the rows n the w ndow ( ndependent of row sequence), and then return that va ue for every row SQL Server 2012 fina y addresses th s shortcom ng In the fo ow ng code samp es, you’ see how OVER/ORDER BY can now be used w th a the trad t ona aggregate funct ons to prov de running calculations w th n ordered w ndows You’ a so earn how to frame w ndows us ng the ROWS and RANGE c ause, wh ch adjusts the s ze and scope of the w ndow to enab e sliding calculations And fina y, SQL Server 2012 ntroduces e ght new ana yt c funct ons (covered n the next sect on) that are des gned spec fica y to work w th ordered (and opt ona y part t oned) w ndows us ng OVER w th ORDER BY (and opt ona y PARTITION BY)

Note  All remaining topics in this chapter are new features in SQL Server 2012. The code n L st ng 2-28 creates a tab e popu ated w th financ a transact ons from severa d fferent accounts Tangent a y, note the use of the new DATEFROMPARTS funct on (a so covered n the next sect on), wh ch s used to construct a date va ue from year, month, and day parameters Listing 2-28  Prepar ng samp e transact on data for query ng w th w ndow funct ons.

CREATE TABLE TxnData (AcctId GO INSERT INTO TxnData (AcctId, (1, DATEFROMPARTS(2012, 4, (1, DATEFROMPARTS(2012, 4, (1, DATEFROMPARTS(2012, 4, (1, DATEFROMPARTS(2012, 4, (1, DATEFROMPARTS(2012, 4,

int, TxnDate date, Amount decimal) TxnDate, Amount) VALUES 10), 500), -- 5 transactions for acct 1 22), 250), 24), 75), 26), 125), 28), 175),

94  Part 1  Core SQL Server Development

(2, (2, (2, (2, (2, (2, (2, (2, (3, (3, (3, (3,

DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012, DATEFROMPARTS(2012,

4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4,

11), 15), 22), 25), 27), 27), 29), 30), 14), 15), 22), 23),

500), -- 8 transactions for acct 2 50), 5000), 550), 105), 95), 100), 2500), 500), -- 4 transactions for acct 3 600), 25), 125)

Running Aggregations In SQL Server 2012, an ORDER BY c ause may be spec fied w th OVER to produce runn ng ­aggregat ons w th n each w ndow, as L st ng 2-29 demonstrates Listing 2-29 Us ng OVER w th ORDER BY to produce runn ng aggregat ons.

SELECT AcctId, TxnDate, Amount, RAvg = AVG(Amount) OVER (PARTITION RCnt = COUNT(*) OVER (PARTITION RMin = MIN(Amount) OVER (PARTITION RMax = MAX(Amount) OVER (PARTITION RSum = SUM(Amount) OVER (PARTITION FROM TxnData ORDER BY AcctId, TxnDate

AcctId -----1 1 1 1 1 2 2 2 :

TxnDate ---------2012-02-10 2012-02-22 2012-02-24 2012-02-26 2012-02-28 2012-02-11 2012-02-15 2012-02-22

Amount -----500 250 75 125 175 500 50 5000

RAvg ----------500.000000 375.000000 275.000000 237.500000 225.000000 500.000000 275.000000 1850.000000

BY BY BY BY BY

RCnt ---1 2 3 4 5 1 2 3

AcctId AcctId AcctId AcctId AcctId

RMin ---500 250 75 75 75 500 50 50

ORDER ORDER ORDER ORDER ORDER

RMax ---500 500 500 500 500 500 500 5000

BY BY BY BY BY

TxnDate), TxnDate), TxnDate), TxnDate), TxnDate)

RSum ---500 750 825 950 1125 500 550 5550

The resu ts of th s query are part t oned (w ndowed) by account W th n each w ndow, the ­ ccount’s runn ng averages, counts, m n mum/max mum va ues, and sums are ordered by transact on a date, show ng the chrono og ca y accumu ated va ues for each account No ROWS c ause s spec fied (we’ exp a n how to use the ROWS c ause next), so ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW s assumed by defau t Th s y e ds a w ndow frame s ze that spans from the beg nn ng

Chapter 2  T SQL Enhancements   95

of the part t on (the first row of each account) through the current row When the account ID changes, the prev ous w ndow s “c osed” and new ca cu at ons start runn ng for a new w ndow over the next account ID

Sliding Aggregations You can a so narrow each account’s w ndow by fram ng t w th a ROWS c ause n the OVER c ause Th s enab es s d ng ca cu at ons, as demonstrated n L st ng 2-30 Listing 2-30 Us ng OVER w th ORDER BY and PRECEDING to produce s d ng aggregat ons.

SELECT AcctId, TxnDate, Amount, SAvg = AVG(Amount) OVER (PARTITION BY AcctId ORDER ROWS BETWEEN 2 PRECEDING SCnt = COUNT(*) OVER (PARTITION BY AcctId ORDER SMin = MIN(Amount) OVER (PARTITION BY AcctId ORDER SMax = MAX(Amount) OVER (PARTITION BY AcctId ORDER SSum = SUM(Amount) OVER (PARTITION BY AcctId ORDER FROM TxnData ORDER BY AcctId, TxnDate

AcctId -----1 1 1 1 1 2 2 2 :

TxnDate ---------2012-02-10 2012-02-22 2012-02-24 2012-02-26 2012-02-28 2012-02-11 2012-02-15 2012-02-22

Amount -----500 250 75 125 175 500 50 5000

SAvg ----------500.000000 375.000000 275.000000 150.000000 125.000000 500.000000 275.000000 1850.000000

SCnt ---1 2 3 3 3 1 2 3

SMin ---500 250 75 75 75 500 50 50

SMax ---500 500 500 250 175 500 500 5000

BY TxnDate AND CURRENT ROW), BY TxnDate ROWS 2 BY TxnDate ROWS 2 BY TxnDate ROWS 2 BY TxnDate ROWS 2

PRECEDING), PRECEDING), PRECEDING), PRECEDING)

SSum ---500 750 825 450 375 500 550 5550

Th s s ght y mod fied vers on of the prev ous query spec fies ROWS BETWEEN 2 PRECEDING AND CURRENT ROW n the OVER c ause for the RAvg co umn, overr d ng the defau t w ndow s ze ­Spec fica y, t frames the w ndow w th n each account’s part t on to a max mum of three rows the current row, the row before t, and one more row before that one Once the w ndow expands to three rows, t stops grow ng and starts s d ng down the subsequent rows unt a new part t on (the next ­account) s encountered The BETWEEN…AND CURRENT ROW keywords that spec fy the upper bound of the w ndow are assumed defau t, so to reduce code c utter, the other co umn defin t ons spec fy just the ower bound of the w ndow w th the shorter var at on ROWS 2 PRECEDING Not ce how the w ndow “s des” w th n each account For examp e, the s d ng max mum for ­account 1 drops from 500 to 250 n the fourth row, because 250 s the argest va ue n the w ndow of three rows that beg ns two rows ear er—and the 500 from the very first row s no onger v s b e

96  Part 1  Core SQL Server Development

n that w ndow S m ar y, the s d ng sum for each account s based on the defined w ndow Thus, the s d ng sum of 375 on the ast row of account 1 s the tota sum of that row (175) p us the two ­preced ng rows (75 + 125) on y—not the tota sum for a transact ons n the ent re account, as the runn ng sum had ca cu ated

Using RANGE versus ROWS F na y, RANGE can be used nstead of ROWS to hand e “t es” w th n a w ndow A though ROWS treats each row n the w ndow d st nct y, RANGE w merge rows conta n ng dup cate ORDER BY va ues, as demonstrated by L st ng 2-31 Listing 2-31 Compar ng ROWS and RANGE for ca cu at ng w ndow funct ons.

SELECT AcctId, TxnDate, Amount, SumByRows = SUM(Amount) OVER (ORDER BY TxnDate ROWS UNBOUNDED PRECEDING), SumByRange = SUM(Amount) OVER (ORDER BY TxnDate RANGE UNBOUNDED PRECEDING) FROM TxnData WHERE AcctId = 2 ORDER BY TxnDate

AcctId -----2 2 2 2 2 2 2 2

TxnDate ---------2012-02-11 2012-02-15 2012-02-22 2012-02-25 2012-02-27 2012-02-27 2012-02-29 2012-02-30

Amount -----500 50 5000 550 105 95 100 2500

SumByRows --------500 550 5550 6100 6205 6300 6400 8900

SumByRange ---------500 550 5550 6100 6300 6300 6400 8900

In th s resu t set, ROWS and RANGE both return the same va ues, w th the except on of the fifth row Because the fifth and s xth rows are both t ed for the same date (2/27/2012), RANGE returns the comb ned runn ng sum for both rows The seventh row (for 2/29/2012) breaks the t e, and ROWS “catches up” w th RANGE to return runn ng tota s for the rest of the w ndow

New T-SQL Functions in SQL Server 2012 The atest vers on of SQL Server augments T-SQL w th many new funct ons In th s sect on, we cover the 22 new funct ons (and 1 changed funct on) ntroduced n SQL Server 2012 We’ start by cover ng the new T-SQL ana yt c funct ons, because they operate us ng the same w ndow ng pr nc p es we were just d scuss ng



Chapter 2  T SQL Enhancements   97

New Analytic Functions E ght new ana yt c funct ons have been added to T-SQL A of them work n conjunct on w th an ordered w ndow defined w th an assoc ated ORDER BY c ause that can be opt ona y part t oned w th a PARTITION BY c ause and framed w th a BETWEEN c ause The new funct ons are ■

FIRST VALUE

■

LAST VALUE

■

LAG

■

LEAD

■

PERCENT RANK

■

CUME DIST

■

PERCENTILE CONT

■

PERCENTILE DISC

In L st ng 2-32, the FIRST VALUE, LAST VALUE, LAG, and LEAD funct ons are used to ana yze a set of orders at the product eve Listing 2-32  Us ng the FIRST VALUE, LAST VALUE, LAG, and LEAD funct ons.

DECLARE @Orders AS table(OrderDate date, ProductID int, Quantity int) INSERT INTO @Orders VALUES ('2012-03-18', 142, 74), ('2012-04-11', 123, 95), ('2012-04-12', 101, 38), ('2012-05-21', 130, 12), ('2012-05-30', 101, 28), ('2012-07-25', 123, 57), ('2012-07-28', 101, 12) SELECT OrderDate, ProductID, Quantity, WorstOn = FIRST_VALUE(OrderDate) OVER(PARTITION BY ProductID ORDER BY Quantity), BestOn = LAST_VALUE(OrderDate) OVER(PARTITION BY ProductID ORDER BY Quantity ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING), PrevOn = LAG(OrderDate, 1) OVER(PARTITION BY ProductID ORDER BY OrderDate), NextOn = LEAD(OrderDate, 1) OVER(PARTITION BY ProductID ORDER BY OrderDate) FROM @Orders ORDER BY OrderDate

OrderDate ---------2012-03-18 2012-04-11

ProductID --------142 123

Quantity -------74 95

WorstOn ---------2012-03-18 2012-07-25

98  Part 1  Core SQL Server Development

BestOn ---------2012-03-18 2012-04-11

PrevOn ---------NULL NULL

NextOn ---------NULL 2012-07-25

2012-04-12 2012-05-21 2012-05-30 2012-07-25 2012-07-28

101 130 101 123 101

38 12 28 57 12

2012-07-28 2012-05-21 2012-07-28 2012-07-25 2012-07-28

2012-04-12 2012-05-21 2012-04-12 2012-04-11 2012-04-12

NULL NULL 2012-04-12 2012-04-11 2012-05-30

2012-05-30 NULL 2012-07-28 NULL NULL

In th s query, four ana yt c funct ons spec fy an OVER c ause that part t ons the resu t set by ­ProductID The product part t ons defined for FIRST VALUE and LAST VALUE are sorted by Quantity, whereas the ­product part t ons for LAG and LEAD are sorted by OrderDate The fu resu t set s sorted by OrderDate, so you need to v sua ze the sorted part t on for each of the four funct ons to understand the output—the resu t set sequence s not the same as the row sequence used for the w ndow ng funct ons The WorstOn and BestOn co umns use FIRST VALUE and LAST VALUE, respect ve y, to return the “worst” and “best” dates for the product n each part t on Performance s measured by ­quant ty, so sort ng each product’s part t on by quant ty w pos t on the worst order at the first row n the ­part t on and the best order at the ast row n the part t on FIRST VALUE and LAST VALUE can return the va ue of any co umn (OrderDate, n th s case), not just the aggregate co umn tse f For LAST VALUE, t s a so necessary to exp c t y define a w ndow over the ent re part t on w th ROWS BETWEEN ­UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING Otherw se, as exp a ned n our ­ear er d scuss on about runn ng and s d ng aggregat ons, the defau t w ndow s ROWS ­BETWEEN ­UNBOUNDED PRECEDING AND CURRENT ROW Th s defau t behav or frames (constra ns) the ­w ndow, and does not cons der the rema n ng rows n the part t on that are needed to obta n the h ghest quant ty for LAST VALUE In the output, not ce that OrderDate, LowestOn, and HighestOn for the first order (product 142) are a the same va ue (3/18) Th s s because product 142 was on y ordered once, so FIRST VALUE and LAST VALUE operate over a part t on that has on y th s one row n t, w th an OrderDate va ue of 3/18 The second row s for product 123, quant ty 95, ordered on 4/11 Four rows ahead n the resu t set (not the part t on) there s another order for product 123, quant ty 57, p aced on 7/25 Th s means that, for th s product, FIRST VALUE and LAST VALUE operate over a part t on that has these two rows n t, sorted by quant ty Th s pos t ons the 7/25 order (quant ty 57) first and the 4/11 (quant ty 95) ast w th n the part t on As a resu t, rows for product 123 report 7/25 for WorstDate and 4/11 for BestDate The next order (product 101) appears two more t mes n the resu t set, creat ng a part t on of three rows Aga n, based on the Quantity sort of the part t on, each row n the part t on reports the product’s worst and best dates (wh ch are 7/28 and 4/12, respect ve y) The PrevOn and NextOn co umns use LAG and LEAD to return the prev ous and next date that each product was ordered They spec fy an OVER c ause that part t ons by ProductId as before, but the rows n these part t ons are sorted by OrderDate Thus, the LAG and LEAD funct ons exam ne each product’s orders n chrono og ca sequence, regard ess of quant ty For each row n each part t on, LAG s ab e to access prev ous ( agg ng) rows w th n the same part t on S m ar y, LEAD can access subsequent ( ead ng) rows w th n the same part t on The first parameter to LAG and LEAD spec fies the co umn va ue to be returned from a agg ng or ead ng row, respect ve y The second ­parameter spec fies the number of rows back or forward LAG and LEAD shou d seek w th n each part t on, ­re at ve to the current row L st ng 2-32 passes OrderDate and 1 as parameters to LAG and LEAD, ­us ng



Chapter 2  T SQL Enhancements   99

product part t ons that are ordered by date Thus, the query returns the most recent past date, and nearest future date, that each product was ordered Because the first order’s product (142) was on y ordered once, ts s ng e-row part t on has no ­agg ng or ead ng rows, and so LAG and LEAD both return NULL for PrevOn and NextOn (note that both funct ons a so opt ona y accept a th rd parameter to be used nstead of NULL when no ­agg ng or ead ng row ex sts) The second order (on 4/11) s for product 123, wh ch was ordered aga n on 7/25, creat ng a part t on w th two rows sorted by OrderDate, w th the 4/11 order pos t oned first and the 7/25 order pos t oned ast w th n the part t on The first row n a mu t -row w ndow has no agg ng rows, but at east one ead ng row S m ar y, the ast order n a mu t -row w ndow has at east one ­agg ng row, but no ead ng rows As a resu t, the first order (4/11) reports NULL and 7/25 for PrevOn and NextOn, respect ve y, and the second order (7/25) reports 4/11 and NULL for PrevOn and ­NextOn, respect ve y Product 101 was ordered three t mes, wh ch creates a part t on of three rows In th s part t on, the second row has both a agg ng row and a ead ng row Thus, the three ­orders r­ eport PrevOn and NextOn va ues for product 101, respect ve y nd cat ng NULL-5/30 for the first (4/12) order, 4/12-7/28 for the second (5/30) order, and 5/30-NULL for the th rd and ast order The ast funct ons to exam ne are PERCENT RANK (rank d str but on), CUME DIST ­(cumu at ve ­d str but on, or percent e), PERCENTILE CONT (cont nuous percent e), and PERCENTILE DISC ­(d screte percent e) The quer es n L st ng 2-33 demonstrate these funct ons, wh ch are a c ose y re ated, by query ng sa es figures across each quarter of two years Listing 2-33  Us ng the PERCENT RANK, CUME DIST, PERCENTILE CONT, and PERCENTILE DISC funct ons.

DECLARE @Sales table(Yr int, Qtr int, Amount money) INSERT INTO @Sales VALUES (2010, 1, 5000), (2010, 2, 6000), (2010, 3, 7000), (2010, 4, 2000), (2011, 1, 1000), (2011, 2, 2000), (2011, 3, 3000), (2011, 4, 4000) -- Distributed across all 8 quarters SELECT Yr, Qtr, Amount, R = RANK() OVER(ORDER BY Amount), PR = PERCENT_RANK() OVER(ORDER BY Amount), CD = CUME_DIST() OVER(ORDER BY Amount) FROM @Sales ORDER BY Amount -- Distributed (partitioned) by year with percentile lookups SELECT Yr, Qtr, Amount, R = RANK() OVER(PARTITION BY Yr ORDER BY Amount), PR = PERCENT_RANK() OVER(PARTITION BY Yr ORDER BY Amount),

100  Part 1  Core SQL Server Development

CD = CUME_DIST() OVER(PARTITION BY Yr ORDER BY Amount), PD5 = PERCENTILE_DISC(.5) WITHIN GROUP (ORDER BY Amount) PD6 = PERCENTILE_DISC(.6) WITHIN GROUP (ORDER BY Amount) PC5 = PERCENTILE_CONT(.5) WITHIN GROUP (ORDER BY Amount) PC6 = PERCENTILE_CONT(.6) WITHIN GROUP (ORDER BY Amount) FROM @Sales ORDER BY Yr, Amount

Yr ---2011 2011 2010 2011 2011 2010 2010 2010

Qtr --1 2 4 3 4 1 2 3

Amount ------1000.00 2000.00 2000.00 3000.00 4000.00 5000.00 6000.00 7000.00

R 1 2 2 4 5 6 7 8

PR ----------------0 0.142857142857143 0.142857142857143 0.428571428571429 0.571428571428571 0.714285714285714 0.857142857142857 1

CD ----0.125 0.375 0.375 0.5 0.625 0.75 0.875 1

Yr ---2010 2010 2010 2010 2011 2011 2011 2011

Qtr --4 1 2 3 1 2 3 4

Amount ------2000.00 5000.00 6000.00 7000.00 1000.00 2000.00 3000.00 4000.00

R 1 2 3 4 1 2 3 4

PR ----------------0 0.333333333333333 0.666666666666667 1 0 0.333333333333333 0.666666666666667 1

CD ---0.25 0.5 0.75 1 0.25 0.5 0.75 1

PD5 ------5000.00 5000.00 5000.00 5000.00 2000.00 2000.00 2000.00 2000.00

OVER(PARTITION OVER(PARTITION OVER(PARTITION OVER(PARTITION

PD6 ------6000.00 6000.00 6000.00 6000.00 3000.00 3000.00 3000.00 3000.00

PC5 ---5500 5500 5500 5500 2500 2500 2500 2500

BY BY BY BY

Yr), Yr), Yr), Yr)

PC6 ---5800 5800 5800 5800 2800 2800 2800 2800

The new funct ons are a based on the RANK funct on ntroduced n SQL Server 2005 So both these quer es a so report on RANK, wh ch w a d both n our exp anat on and your understand ng of each of the new funct ons In the first query, PERCENT RANK and CUME DIST (a ased as PR and CD respect ve y) rank quarter y sa es across the ent re two-year per od Look at the va ue returned by RANK (a ased as R) It ranks each row n the unpart t oned w ndow (a e ght quarters) by do ar amount Both 2011Q2 and 2010Q4 are t ed for $2,000 n sa es, so RANK ass gns them the same va ue (2) The next row breaks the t e, so RANK cont nues w th 4, wh ch accounts for the “empty s ot” created by the two prev ous rows that were t ed Now exam ne the va ues returned by PERCENT RANK and CUME DIST Not ce how they reflect the same nformat on as RANK, w th dec ma va ues rang ng from 0 and 1 The on y d fference between the two s a s ght var at on n the way the r formu as are mp emented, such that PERCENT RANK a ways starts w th 0 wh e CUME DIST a ways starts w th a va ue greater than 0 Spec fica y, PERCENT RANK returns (RANK – 1) / (N – 1) for each row, where N s the tota number of rows n the w ndow Th s a ways returns 0 for the first (or on y) row n the w ndow CUME DIST returns RANK / N, wh ch a ways



Chapter 2  T SQL Enhancements   101

returns a va ue greater than 0 for the first row n the w ndow (wh ch wou d be 1, f there’s on y row) For w ndows w th two or more rows, both funct ons return 1 for the ast row n the w ndow w th dec ma va ues d str buted among a the other rows The second query exam nes the same sa es figures, on y th s t me the resu t set s part t oned by year There are no t es w th n each year, so RANK ass gns the sequent a numbers 1 through 4 to each of the quarters, for 2010 and 2011, by do ar amount You can see that PERCENT RANK and CUME DIST perform the same RANK ca cu at ons as exp a ned for the first query (on y, aga n, part t oned by year th s t me) Th s query a so demonstrates PERCENTILE DISC and PERCENTILE CONT These very s m ar f­ unct ons each accept a percent e parameter (the des red CUME DIST va ue) and “reach n” to the w ndow for the row at or near that percent e We demonstrate by ca ng both funct ons tw ce, once w th a percent e parameter va ue of 5 and once w th 6, return ng co umns a ased as PD5, PD6, PC5, and PC6 Both funct ons exam ne the CUME DIST va ue for each row n the w ndow to find the one nearest to 5 and 6 The subt e d fference between them s that PERCENTILE DISC w return a ­prec se (d screte) va ue from the row w th the match ng percent e (or greater), wh e PERCENTILE CONT nterpo ates a va ue based on a cont nuous range Spec fica y, PERCENTILE CONT returns a va ue rang ng from the row match ng the spec fied percent e—or, f there s no exact match, a ca cu ated va ue based on the spec fied percent e f there s no exact match—and the row w th the next h gher percent e n the w ndow Th s exp a ns the va ues returned by these funct ons n th s query For the year 2010, the 5 percent e (CUME DIST va ue) s ocated exact y on quarter 1, wh ch had $5,000 n sa es Thus PERCENTILE DISC( 5) returns 5000 There s no row n the w ndow w th a ­percent e of 6, so PERCENTILE DISC( 6) matches up aga nst the first row w th a percent e greater than or equa to 6, wh ch s the row for quarter 2 w th $6,000 n sa es, and thus returns 6000 In both cases, PERCENTILE DISC returns a d screte va ue from a row n the w ndow at or greater than the spec fied percent e The same ca cu at ons are performed for 2011, return ng 2000 for ­­PERCENTILE DISC( 5) and 3000 for PERCENTILE DISC( 6), correspond ng to the $2,000 n sa es for quarter 2 ­(percent e 5) and the $3,000 n sa es for quarter 3 (percent e 75) As we stated, PERCENTILE CONT s very s m ar It takes the same percent e parameter to find the row n the w ndow match ng that percent e If there s no exact match, the funct on ca cu ates a va ue based on the sca e of percent es d str buted across the ent re w ndow, rather than ook ng ahead to the row hav ng the next greater percent e va ue, as PERCENTILE DISC does Then t returns the ­med an between that va ue and the va ue found n the row w th the next greater percent e For 2010, the 5 percent e matches up w th 5000 (as before) The next percent e n the w ndow s for 75 for 6000 The med an between 5000 and 6000 s 5500 and thus, PERCENTILE CONT(.5) returns 5500 There s no row n the w ndow w th a percent e of 6, so PERCENTILE CONT(.6) ca cu ates what the va ue for 6 wou d be (somewhere between 5000 and 6000, a b t c oser to 5000) and then ca cu ates the med an between that va ue and the next percent e n the w ndow (aga n, 75 for 6000) Thus, ­PERCENTILE CONT(.6) returns 5800; s ght y h gher than the 5500 returned for PERCENTILE CONT(.5).

102  Part 1  Core SQL Server Development

More Info The PERCENTILE DISC and PERCENTILE CONT functions define their window ordering ­using ORDER BY in a WITHIN GROUP clause rather than in the OVER clause. Thus, you do not (and cannot) specify ORDER BY in the OVER clause. The OVER clause is still required, however, so OVER (with empty parentheses) must be specified even if you don’t want to partition using PARTITION BY.

New Conversion Functions These three funct ons are des gned to ass st you w th pars ng and convert ng between d fferent data types and cu ture-sens t ve str ngs ■

TRY CONVERT

■

PARSE

■

TRY PARSE

The TRY CONVERT funct on s the ong-awa ted “safe” vers on of the CONVERT funct on TRY CONVERT works exact y ke ts ub qu tous counterpart, except that t w return NULL rather than ra se an error f the supp ed data cannot be converted to the spec fied data type For examp e, ­cons der the code n L st ng 2-34 Listing 2-34 Us ng TRY CONVERT for va d and nva d convers ons.

SELECT TRY_CONVERT(money, 'test') AS BadResult SELECT TRY_CONVERT(money, '29.5') AS GoodResult

BadResult ---------------NULL GoodResult ---------------29.50

As you can see, the TRY CONVERT funct on has the same syntax and usage as CONVERT, except that t w never ra se a convers on error The first attempt to convert the str ng ‘test’ to a money data type s nva d of course, but TRY CONVERT s mp y returns NULL n that case and a ows execut on to cont nue The second attempt to convert the str ng ’29.5’ to a money data type s perfect y va d, and performs just as CONVERT wou d The new PARSE funct on understands d fferent date, t me, and currency formats that are ­compat b e w th the var ous cu tures supported by the NET Framework Th s funct on s pa red w th



Chapter 2  T SQL Enhancements   103

ts counterpart TRY PARSE that returns NULL f the spec fied str ng cannot be parsed as requested (rather than throw ng an error as PARSE wou d) L st ng 2-35 demonstrates Listing 2-35 Us ng PARSE to ana yze and convert cu ture spec fic date, t me, and currency str ngs.

SELECT PARSE('Monday, 13 December 2010' AS datetime2 USING 'en-US') AS USResult SELECT PARSE('€345,98' AS money USING 'nl-NL') AS NLResult

USResult -----------------------2010-12-13 00:00:00.000 NLResult -----------------------345.98

The USING c ause te s SQL Server how t shou d nterpret the nput str ng w th respect to cu ture You don’t need to spec fy a cu ture w th PARSE; the cu ture w be assumed by defau t based on the current anguage f USING s om tted (the current anguage can be set us ng the SET LANGUAGE statement) The first statement parses an en-US (U S Eng sh) date-formatted str ng nto the datetime2 data type, and the second statement parses an nl-NL (Nether ands) Euro-formatted currency str ng nto the money data type Because both str ngs conta n punctuat on appropr ate for the spec fied cu tures, the PARSE funct on correct y nterprets the nput str ng n both cases However, the fo ow ng statement ra ses an error SELECT PARSE('$345,98' AS money USING 'nl-NL') AS NLResult

Because the do ar s gn symbo s not va d for a Nether ands-based currency str ng that expects and accepts the euro symbo (€), the PARSE funct on fa s Msg 9819, Level 16, State 1, Line 39 Error converting string value '$345,98' into data type money using culture 'nl-NL'.

To hand e cases where the nput str ng hasn’t been va dated for the spec fied cu ture, use TRY PARSE nstead SELECT TRY_PARSE('$345,98' AS money USING 'nl-NL') AS NLResult

Th s statement returns NULL rather than ra s ng an error Thus, you can use TRY PARSE to va date, and a you need to do s test ts resu t for NULL to determ ne f the va dat on s successfu

New Date and Time Functions A set of new funct ons et you construct dates and t mes based on d screte va ues you supp y for the var ous parts of the overa va ue There s a vers on of th s funct on for each of the supported date and t me data types n SQL Server ( nc ud ng the o der datetime and smalldatetime, a though th s shou d not encourage the r cont nued use) 104  Part 1  Core SQL Server Development

■

DATEFROMPARTS

■

TIMEFROMPARTS

■

DATETIME2FROMPARTS

■

DATETIMEOFFSETFROMPARTS

■

DATETIMEFROMPARTS

■

SMALLDATETIMEFROMPARTS

There s a so a new funct on to return the number of days n the month of a spec fied date ■

EOMONTH

The xxxFROMPARTS funct ons a work the same way; they each expect parameters that spec fy each part of the va ue to be generated Us ng these funct ons, you can create exp c t date and t me va ues w thout the formatt ng and cu ture amb gu ty concerns so common w th str ngs L st ng 2-36 demonstrates these date and t me construct on funct ons Listing 2-36  Construct ng date and t me va ues us ng the new xxxFROMPARTS funct ons.

SELECT SELECT SELECT SELECT SELECT SELECT

DATEFROMPARTS(2010, 12, 31) AS ADate TIMEFROMPARTS(23, 59, 59, 1234567, 7) AS ATime DATETIME2FROMPARTS(2010, 12, 31, 23, 59, 59, 1234567, 7) AS ADateTime2 DATETIMEOFFSETFROMPARTS(2010, 12, 31, 14, 23, 36, 5, 12, 0, 1) ADateTimeOff DATETIMEFROMPARTS(2010, 12, 31, 23, 59, 59, 123) AS ADateTime SMALLDATETIMEFROMPARTS(2010, 12, 31, 23, 59) AS ASmallDateTime

ADate ----------------------2010-12-31 ATime ----------------------23:59:59.1234567 ADateTime2 --------------------------2010-12-31 23:59:59.1234567 ADateTimeOff ---------------------------------2010-12-31 14:23:36.5 +12:00 ADateTime ----------------------2010-12-31 23:59:59.123 ASmallDateTime ----------------------2010-12-31 23:59:00



Chapter 2  T SQL Enhancements   105

The EOMONTH funct on s a handy way to determ ne the ast day of the month of any g ven date As you’d expect, eap years are accounted for as we , as L st ng 2-37 demonstrates Listing 2-37  Us ng the EOMONTH funct on to obta n the number of days n each month.

SELECT SELECT SELECT SELECT SELECT

EOMONTH('1/1/2011') EOMONTH('2/1/2011') EOMONTH('3/1/2011') EOMONTH('4/1/2011') EOMONTH('2/1/2012')

AS LastDayOfMonth UNION ALL -- 31 UNION ALL -- 28 UNION ALL -- 31 UNION ALL -- 30 -- 29 (leap year)

LastDayOfMonth ----------------------2011-01-31 2011-02-28 2011-03-31 2011-04-30 2012-02-29

By comb n ng EOMONTH w th DATEPART, t’s easy to get the day count for any g ven month For examp e, the fo ow ng query returns 29 for the number of days n February 2012 (a eap year) SELECT DATEPART(day, EOMONTH('2/1/2012')) AS DaysInFeb2012

New Logical Functions SQL Server 2012 a so ntroduces these two new funct ons for cond t ona operat ons ■

CHOOSE

■

IIF

These two funct ons are conven ent a ternat ves to more verbose CASE statements, and work ke the same-named V sua Bas c for App cat ons (VBA) funct ons n M crosoft Access CHOOSE accepts an nteger parameter (wh ch wou d need to be a var ab e n any usefu scenar o), fo owed by any number of parameters of any data type (wh ch can be e ther constants or var ab es) The nteger acts as an ndex (one-based) nto the st of parameters to return the tem at a spec fic pos t on, as demonstrated by the L st ng 2-38 Listing 2-38  Us ng the CHOOSE funct on to se ect from a st of tems.

DECLARE @CardTypeId int = 2 -- Master card SELECT CHOOSE(@CardTypeId, 'Amex', 'Master', 'Visa', 'Discover') AS CardType

106  Part 1  Core SQL Server Development

CardType -------Master

When dec d ng between one of two return va ues based on a Boo ean (true/fa se) cond t on, the new IIF (Immed ate IF) funct on offers a conc se way to express your og c The funct on accepts three parameters the first s the Boo ean cond t on to test, the second s the va ue to return f the cond t on s true, and the th rd s the va ue to return f the cond t on s fa se L st ng 2-39 demonstrates Listing 2-39  Us ng the IIF funct on to perform cond t ona test ng.

DECLARE @Num1 int = 45 DECLARE @Num2 int = 40 SELECT IIF(@Num1 > @Num2, 'larger', 'not larger' ) AS Result

Result ---------larger

New String Functions These two new str ng funct ons make t eas er to bu d and format str ngs n T-SQL ■

CONCAT

■

FORMAT

The CONCAT funct on concatenates str ngs just ke the concatenat on operator (+), but s to erant of NULL va ues When you concatenate us ng the + operator, the fina resu t s a ways NULL f any of the str ngs be ng concatenated are NULL Th s common annoyance s often dea t w th by wrapp ng ISNULL convers ons on a the va ues be ng concatenated CONCAT offers a c eaner approach s mp y by treat ng NULL va ues as empty str ngs Essent a y, t mp c t y converts a the spec fied va ues to str ngs before t concatenates them together Cons der th s ne of code SELECT CONCAT('Happy', ' Birthday ', 8, '/', NULL, '30') AS Greeting

Th s statement returns the str ng ‘Happy Birthday 8/30’ by str ng ng together a the va ues ­supp ed to CONCAT The funct on converts the va ues 8 and NULL to the str ngs ‘8’ and ‘’ (empty) respect ve y before concatenat on The new FORMAT funct on br ngs the fu power of NET formatt ng to T-SQL W th t, you can ­ as y express date, t me, and currency va ues n v rtua y any des red format Supp y the va ue to e format n the first parameter, spec fy the formatt ng code n the second parameter, and spec fy the cu ture n the th rd parameter (as w th PARSE and TRY PARSE, the th rd parameter s opt ona , and defau ts to the cu ture for the current y set anguage) Tab e 2-3 shows the var ous supported codes to format dates, t mes, and currenc es n any cu ture



Chapter 2  T SQL Enhancements   107

Table 2-3  Common date and t me formatt ng codes Size

Action

d

Genera date format

t

Genera t me format

D

Long date format

T

Long t me format

Ddd

Abbrev ated day of week

Dddd

Day of week

C

Currency (w th opt ona dec ma pos t ons; e.g., c2)

The code n L st ng 2-40 popu ates a tab e var ab e w th a handfu of d fferent cu tures, and then se ects from the cu tures to demonstrate var ous formatt ng capab t es of the FORMAT funct on Listing 2-40  Us ng the FORMAT funct on to perform cu ture spec fic date, t me, and currency formatt ng.

DECLARE @Cultures table(Culture varchar(10), Lang varchar(50)) INSERT INTO @Cultures VALUES ('en', 'English'), ('nl', 'Dutch'), ('ja', 'Japanese'), ('ru', 'Russian'), ('no', 'Norwegian') DECLARE @d datetime2 = DATETIME2FROMPARTS(2011, 2, 1, 16, 5, 0, 0, 7) DECLARE @m money = 199.99 SELECT Lang, Date = TimeOnly = LongDate = LongTime = Dow = Currency = FROM @Cultures

Lang --------English Dutch Japanese Russian Norwegian

FORMAT(@d, FORMAT(@d, FORMAT(@d, FORMAT(@d, FORMAT(@d, FORMAT(@m,

Date ---------2/1/2011 1-2-2011 2011/02/01 01.02.2011 01.02.2011

'd', Culture), 't', Culture), 'D', Culture), 'T', Culture), 'ddd', Culture), 'c2', Culture)

TimeOnly -------4:05 PM 16:05 16:05 16:05 16:05

LongDate -------------------------Tuesday, February 02, 2011 dinsdag 1 februari 2011 2011年2月1日 1 февраля 2011 г. 1. februar 2011

108  Part 1  Core SQL Server Development

LongTime ---------4:05:00 PM 16:05:00 16:05:00 16:05:00 16:05:00

Dow Currency --- -------Tue $199.99 di € 199,99 火 ¥199.99 BT 199,99p. ti kr 199,99

You can a so express custom date formats us ng masks (a comb nat on of formatt ng codes and punctuat on), as demonstrated by L st ng 2-41 Exerc se caut on w th th s approach, however, as you are hardcod ng formats that may not adapt proper y to d fferent cu tures Listing 2-41  Us ng the FORMAT funct on w th a mask to produce custom date and t me formatt ng.

DECLARE @d datetime2 = DATETIME2FROMPARTS(2011, 2, 1, 16, 5, 0, 0, 7) SELECT FORMAT(@d, 'ddd M/d/yyyy h:mm tt') AS DateAndTime

DateAndTime --------------------------Tue 2/1/2011 4:05 PM

In th s examp e, the M/d/yyyy n the mask pos t ons the month before the day Th s s correct n the Un ted States but reversed n European countr es S mp y us ng D (for ong date) or d (for short date) n the mask avo ds th s prob em, as SQL Server automat ca y adjusts the date d sp ay for the spec fied cu ture

Changed Mathematical Function There s on y one changed funct on n SQL Server 2012 It’s extreme y m nor, but we cover t for ­comp eteness ■

LOG

Th s funct on now a ows you to spec fy any des red base w th wh ch to compute the ogar thm Prev ous y, you cou d use LOG on y to compute the natura ogar thm, or use the LOG10 funct on to compute the ogar thm w th a base of 10 To compute the ogar thm of a number us ng any other base, t was necessary to d v de the natura ogar thm of the des red number by the natura ogar thm of the des red base For examp e, the base 2 ogar thm of 11 can be computed as fo ows SELECT LOG(11) / LOG(2) AS Base2LogOf11

In SQL Server 2012, the LOG funct on now accepts a second opt ona parameter to des gnate a spec fic base for the a gor thm Thus, the prev ous express on can now be coded as fo ows SELECT LOG(11, 2) AS Base2LogOf11

The THROW Statement Error hand ng n T-SQL was very d fficu t to mp ement proper y before SQL Server 2005 ntroduced the TRY/CATCH construct, a feature oose y based on NET’s try/catch structured except on hand ng mode The CATCH b ock g ves you a s ng e p ace to code error hand ng og c n the event that a prob em occurs anywhere ns de the TRY b ock above t Before TRY/CATCH, t was necessary to ­a ways check for error cond t ons after every operat on by test ng the bu t- n system funct on ­

Chapter 2  T SQL Enhancements   109

@@ERROR Not on y d d code become c uttered w th the many @@ERROR tests, deve opers (be ng humans) wou d too often forget to test @@ERROR n every needed p ace, caus ng many unhand ed except ons to go unnot ced In SQL Server 2005, TRY/CATCH represented a vast mprovement over constant y test ng ­ @­ERROR, but RAISERROR has (unt now) rema ned as the on y mechan sm for generat ng your @ own errors In SQL Server 2012, the new THROW statement (aga n, borrowed from throw n the NET ­mode ) s the recommended a ternat ve way to ra se except ons n your T-SQL code (a though ­R AISERROR does reta n severa capab t es that THROW acks, as we’ exp a n short y)

Re-Throwing Exceptions The new THROW statement can be used n two ways F rst, and as we just stated, t can serve as an a ternat ve to RAISERROR, a ow ng your code to generate errors when t detects an unreso vab e ­cond t on n process ng Used for th s purpose, the THROW statement accepts parameters for the ­error code, descr pt on, and state, and works much ke RAISERROR A more spec a zed use of THROW takes no parameters, and can appear on y ns de a CATCH b ock In th s scenar o, an unexpected error occurs n the TRY b ock above, tr gger ng execut on of the CATCH b ock Ins de the CATCH b ock, you can perform genera error hand ng (for examp e, ogg ng the error, or ro ng back a transact on), and then ssue a THROW statement w th no parameters Th s w re-throw the or g na error that occurred—w th ts code, message, sever ty, and state ntact—back up to the c ent, so the error can be caught and hand ed at the app cat on eve as we Th s s an easy and e egant way for you to mp ement a segmented except on hand ng strategy between the database and app cat on ayers In contrast, RAISERROR a ways ra ses a new error Thus, t can on y s mu ate re-throw ng the ­ r g na error by captur ng the ERROR MESSAGE, ERROR SEVERITY, and ERROR STATE n the CATCH o b ock and us ng the r va ues to ra se a new error Us ng THROW for th s purpose s much more s mp e and d rect, as demonstrated n L st ng 2-42 Listing 2-42 Us ng THROW n a CATCH b ock to re throw an error.

CREATE TABLE ErrLog(ErrAt datetime2, Severity varchar(max), ErrMsg varchar(max)) GO BEGIN TRY DECLARE @Number int = 5 / 0; END TRY BEGIN CATCH -- Log the error info, then re-throw it INSERT INTO ErrLog VALUES(SYSDATETIME(), ERROR_SEVERITY(), ERROR_MESSAGE()); THROW; END CATCH

110  Part 1  Core SQL Server Development

In th s code’s CATCH b ock, error nformat on s recorded to the ErrLog tab e and then the or g na error (d v de by zero) s re-thrown for the c ent to catch Msg 8134, Level 16, State 1, Line 4 Divide by zero error encountered.

To confirm that the error was ogged by the CATCH b ock as expected before be ng re-thrown, query the ErrLog tab e SELECT * FROM ErrLog GO ErrAt --------------------------2012-02-30 14:14:35.3361250

Severity -------16

ErrMsg -----------------------------------------Divide by zero error encountered.

Comparing THROW and RAISERROR The fo ow ng tab e summar zes the notab e d fferences that ex st between THROW and RAISERROR Table 2-4 Compar ng THROW and RAISERROR THROW

RA SERROR

Can on y generate user except ons (un ess re throw ng n CATCH b ock)

Can generate user (>= 50000) and system (< 50000) except ons

Supp es ad hoc text; doesn t ut ze sys.messages

Requ res user messages defined n sys.­messages (except for code 50000)

Doesn t support token subst tut ons

Supports token subst tut ons

A ways uses sever ty eve 16 (un ess ­re throw ng n a CATCH b ock)

Can set any sever ty eve

Can re throw or g na except on caught n the TRY b ock

A ways generates a new except on; the or g na ­except on s ost to the c ent

Error messages are buffered, and don t appear n rea t me

Supports WITH NOWAIT to mmed ate flush ­buffered output on error

A user except on s an error w th a code of 50000 or h gher that you define for your app cat on’s use System except ons are defined by SQL Server and have error codes ower than 50000 You can use the new THROW statement to generate and ra se user except ons, but not system except ons On y RAISERROR can be used to throw system except ons Note, however, that when THROW s used n a CATCH b ock to re-throw the except on from a TRY b ock as exp a ned n the prev ous sect on, the actua or g na except on—even f t’s a system except on—w get thrown (th s was demonstrated n the prev ous “d v de by zero” examp e) When RAISERROR s used w thout an error code, SQL Server ass gns an error code of 50000 and expects you to supp y an ad-hoc message to assoc ate w th the error The THROW statement a ways expects you to supp y an ad-hoc message for the error, as we as a user error code of 50000 or h gher Thus, the fo ow ng two statements are equ va ent THROW 50000, 'An error occurred querying the table.', 1; RAISERROR ('An error occurred querying the table.', 16, 1);



Chapter 2  T SQL Enhancements   111

Both these statements ra se an error w th code 50000, sever ty 16, state 1, and the same ­ essage text Compat b ty between the two keywords ends there, however, as vary ng usages m mpose d ­ fferent ru es (as summar zed n Tab e 2-4) For examp e, on y RAISERROR supports token ­subst tut on RAISERROR ('An error occurred querying the %s table.', 16, 1, 'Customer'); Msg 50000, Level 16, State 1, Line 22 An error occurred querying the Customer table.

THROW has no s m ar capab ty A so, wh e RAISERROR ets you spec fy any sever ty eve , THROW w a ways generate an error w th a sever ty eve of 16 Th s s s gn ficant, as eve 11 and h gher nd cates more ser ous errors than eve 10 and ower For examp e RAISERROR ('An error occurred querying the table.', 10, 1); An error occurred querying the table.

Because the sever ty s 10, th s error does not echo the error code, eve , state, and ne number, and s d sp ayed n b ack rather than the usua red that s used for sever ty eve s h gher than 10 In contrast, THROW cannot be used to s gna a non-severe error The ast mportant d fference between the two keywords s the RAISERROR assoc at on w th sys. messages In part cu ar, RAISERROR requ res that you ca sys.sp addmessage to define error ­messages assoc ated w th user error codes h gher than 50000 As exp a ned, the RAISERROR syntax n our ­ear er examp es uses an error code of 50000, and s the on y supported syntax that ets you supp y an ­ad-hoc message nstead of ut z ng sys.messages The fo ow ng code demonstrates how to define customer user error messages for ­R AISERROR F rst (and on y once), a token zed message for user error code 66666 s added to sys.­messages ­Thereafter, RAISERROR references the error by ts code, and a so supp es va ues for token ­rep acements that are app ed to the message’s text n sys.messages EXEC sys.sp_addmessage 66666, 16, 'There is already a %s named %s.'; RAISERROR(66666, 16, 1, 'cat', 'morris'); Msg 66666, Level 16, State 1, Line 34 There is already a cat named morris.

The THROW statement has no such requ rement You supp y any ad-hoc message text w th THROW You don’t need to separate y manage sys.messages, but th s a so means that THROW can’t (d rect y) everage centra y managed error messages n sys.messages ke RAISERROR does ­Fortunate y, the FORMATMESSAGE funct on prov des a workaround f you want to take advantage of the same capab ty w th THROW You just need to take care and make sure that the same error code s spec fied n the two p aces that you need to reference t (once for FORMATMESSAGE and once for THROW), as shown here

112  Part 1  Core SQL Server Development

DECLARE @Message varchar(max) = FORMATMESSAGE(66666, 'dog', 'snoopy'); THROW 66666, @Message, 1; Msg 66666, Level 16, State 1, Line 40 There is already a dog named snoopy.

Server-Side Paging Return ng paged query resu ts was d fficu t to ach eve pr or to SQL Server 2005 That re ease of SQL Server ntroduced a ser es of rank ng funct ons, nc ud ng the ROW NUMBER funct on that made t poss b e to return one page at a t me from your query Let’s first see how to use ROW NUMBER ­added n SQL Server 2005 to mp ement server-s de pag ng, and then you w earn how much eas er t s to ach eve the same goa us ng the new OFFSET/FETCH NEXT syntax ntroduced n SQL Server 2012

Using ROW_NUMBER The ROW NUMBER funct on, as ts name mp es, generates a sequent a number for each row n the resu t set returned by your query The va ue returned by the ROW NUMBER funct on can then be used n an outer query’s WHERE c ause to m t the resu t set to just the des red page L st ng 2-43 shows an examp e us ng the AdventureWorks2012 database that demonstrates th s techn que It ­returns “page 3” of the query resu ts, wh ch (assum ng a page s ze of 10) are rows 21 through 30 Listing 2-43  Return ng paged resu ts us ng a nested subquery and the ROW NUMBER funct on.

USE AdventureWorks2012 GO -- Using ROW_NUMBER introduced in SQL Server 2005 DECLARE @PageNum int = 3 DECLARE @PageSize int = 10 DECLARE @FirstRow int = ((@PageNum - 1) * @PageSize) + 1 DECLARE @LastRow int = @FirstRow + @PageSize - 1 SELECT * FROM (SELECT RowNum = ROW_NUMBER() OVER (ORDER BY LastName, FirstName), Title, FirstName, LastName FROM Person.Person ) AS a WHERE RowNum BETWEEN @firstRow AND @lastRow ORDER BY LastName, FirstName



Chapter 2  T SQL Enhancements   113

And here are the resu ts RowNum -------21 22 23 24 25 26 27 28 29 30

Title -------NULL Mr. NULL Ms. NULL NULL NULL NULL NULL NULL

FirstName ----------Bailey Ben Blake Carla Carlos Charles Chloe Connor Courtney Dalton

LastName ---------Adams Adams Adams Adams Adams Adams Adams Adams Adams Adams

(10 row(s) affected)

Now th s certa n y works, but there are two undes rab es here F rst, t requ res you to ­manufacture the row number as an add t ona co umn n your resu t set, whether or not you want or need t ­Second, the syntax s somewhat contorted; the requ red use of a nested SELECT statement and ­ u t p e ORDER BY c auses (as we as the requ red a as “AS a”) s both awkward and un ntu t ve m

Using OFFSET/FETCH NEXT Now take a ook at how the same resu t can be ach eved n SQL Server 2012 us ng the code shown n L st ng 2-44 Listing 2-44  Return ng paged resu ts us ng the OFFSET/FETCH NEXT syntax.

DECLARE @PageNum int = 3 DECLARE @PageSize int = 10 DECLARE @Offset int = (@PageNum - 1) * @PageSize SELECT Title, FirstName, LastName FROM Person.Person ORDER BY LastName, FirstName OFFSET @Offset ROWS FETCH NEXT @PageSize ROWS ONLY

The syntax cou dn’t be any s mp er—just spec fy your start ng row w th OFFSET and your page s ze w th FETCH NEXT Here are the resu ts Title -------NULL Mr. NULL Ms. NULL NULL

FirstName ----------Bailey Ben Blake Carla Carlos Charles

LastName ---------Adams Adams Adams Adams Adams Adams

114  Part 1  Core SQL Server Development

NULL NULL NULL NULL

Chloe Connor Courtney Dalton

Adams Adams Adams Adams

(10 row(s) affected)

You can see that th s query returns the same “page” as the prev ous vers on of the query that used the ROW NUMBER funct on, yet t wasn’t necessary to manufacture a row number co umn to do t, nor was t necessary to code a nested subquery Furthermore, OFFSET/FETCH NEXT performs s ght y faster than us ng ROW NUMBER, and s s gn ficant y faster than the pre–SQL Server 2005 hacks ­nvo v ng stored procedures and temp tab es If you st want row numbers returned n your resu t set, you can comb ne OFFSET/FETCH w th ROW NUMBER f des red by rep ac ng the SELECT statement n L st ng 2-44 w th SELECT RowNum = ROW_NUMBER() OVER (ORDER BY LastName, FirstName), Title, FirstName, LastName FROM Person.Person ORDER BY LastName, FirstName OFFSET @Offset ROWS FETCH NEXT @PageSize ROWS ONLY

The resu ts of th s query are dent ca to the resu ts shown for L st ng 2-43, but doesn’t requ re ­cod ng the subquery seen n that st ng A so note that OFFSET can be spec fied w thout FETCH NEXT to sk p a spec fied number of rows and return a rema n ng rows

The SEQUENCE Object H stor ca y, one notab e d fference between SQL Server and other database p atforms (such as Orac e and DB2) has been the manner n wh ch you mp ement automat ca y ass gned nteger va ues for ­pr mary keys when nsert ng new rows SQL Server prov des the IDENTITY attr bute for the int and bigint data types, whereas other p atforms requ re you to create an ndependent “sequence ­generator” object that feeds ncrement ng va ues to new rows n the tab e These are two d fferent ways to ach eve the same th ng, and SQL Server’s IDENTITY s arguab y s mp er to use, but sequence ­generators offer the r own un que advantages as we SQL Server 2012 now a so supports sequence generators as a powerfu a ternat ve to us ng the IDENTITY attr bute Sequences can be created on nteger types (both bu t- n and user-defined), and you can spec fy the m n mum, max mum, start, and ncrement (or decrement) va ues for the sequence You can a so cyc e back around to the m n mum va ue when the max mum va ue s reached (or v ce versa) Sequences are most common y used to ass gn new pr mary key va ues on INSERT operat ons as an a ternat ve to us ng IDENTITY-attr buted key co umns But because they ex st as objects ­ndependent of the data tab es that they feed new pr mary keys to, they offer more flex b ty and pose fewer ­ m tat ons For examp e, you can obta n the next va ue n the sequence before you perform the ­INSERT, and you can nsert any (un que) va ue nto the pr mary key co umn w thout requ r ng SET

Chapter 2  T SQL Enhancements   115

IDENTITY INSERT ON/OFF statements The NEXT VALUE FOR syntax can be used n any SELECT, INSERT, or UPDATE statement to request and ncrement the next va ue n the sequence Let’s see how sequences work by exam n ng the code n L st ng 2-45 Listing 2-45  Us ng a sequence to generate new pr mary keys.

CREATE TABLE Customer (Id int PRIMARY KEY, FirstName varchar(max), LastName varchar(max)) -- Create the sequence with a start, increment, and min/max settings CREATE SEQUENCE CustomerSequence AS int START WITH 1 INCREMENT BY 1 MINVALUE 0 NO MAXVALUE -- Use INSERT (NEXT (NEXT (NEXT

the sequence for new primary key values INTO Customer (Id, FirstName, LastName) VALUES VALUE FOR CustomerSequence, 'Bill', 'Malone'), VALUE FOR CustomerSequence, 'Justin', 'Thorp'), VALUE FOR CustomerSequence, 'Paul', 'Duffy')

The Customer tab e uses an nteger pr mary key, but does not spec fy the IDENTITY ­attr bute Instead, you use the new CREATE SEQUENCE statement n SQL Server 2012 to create a ­CustomerSequence object that feeds new ntegers, start ng w th one and ncrement ng by one, w th no spec fied upper m t (beyond the max mum s ze for the nteger data type; a 32-b t int data type n th s examp e) You then INSERT three new rows, each of wh ch spec fies NEXT VALUE FOR ­CustomerSequence as the new Id va ue (note the row constructor syntax, ntroduced n SQL Server 2008, that nserts the three rows w th a s ng e statement) The resu t, as you’d expect, appears ke th s SELECT * FROM Customer GO Id ----1 2 3

FirstName --------Bill Justin Paul

LastName ------------Malone Thorp Duffy

(3 row(s) affected)

Most ke y, you’ want to emu ate the exper ence of us ng the IDENTITY attr bute; that s, you may w sh to om t the pr mary key va ues from the INSERT statement and put SQL Server n charge of ­ass gn ng them to new rows Th s s eas y done by estab sh ng NEXT VALUE FOR as a DEFAULT ­constra nt on the Id co umn, as shown n L st ng 2-46

116  Part 1  Core SQL Server Development

Listing 2-46 Emu at ng IDENTITY attr buted pr mary key co umns w th sequences us ng a DEFAULT constra nt.

-- Set the default for IDENTITY-behavior ALTER TABLE Customer ADD DEFAULT NEXT VALUE FOR CustomerSequence FOR Id -- Generates customer ID 4 INSERT INTO Customer (FirstName, LastName) VALUES('Jeff', 'Smith')

W th the DEFAULT constra nt n p ace, the INSERT statement ooks and works just the same as f you were us ng an IDENTITY-attr buted pr mary key co umn Customer Jeff Sm th s ­automat ca y a ­ ss gned an Id va ue of 4 But by us ng sequences, you can enjoy a few add t ona benefits For ­examp e, you can peek at the current va ue w thout consum ng t by query ng the sys.sequences cata og v ew Among the many parameters for each sequence object n the database exposed by th s v ew, the current value co umn revea s the current y ass gned va ue SELECT current_value FROM sys.sequences WHERE name='CustomerSequence' GO current_value ------------4

You can a so use the ALTER SEQUENCE statement to change the behav or of an ex st ng sequence object, as shown n L st ng 2-47 Listing 2-47  Chang ng a sequence object us ng ALTER SEQUENCE.

ALTER SEQUENCE CustomerSequence RESTART WITH 1100 MINVALUE 1000 MAXVALUE 9999 CYCLE

Now the next va ue returned by CustomerSequence w be 1100, and t w cont nue to ­ncrement by one from there When t tops 9999, t w start aga n from 1000 and cont nue ncrement ng norma y Natura y, g ven the un queness of pr mary keys, the sequence can no onger be used to popu ate the Customer tab e 100 rows after t recyc es to 1000, because you a ready have customers w th pr mary keys start ng at 1100

Sequence Limitations Of course, sequences have some restr ct ons that need to be ca ed out Sequences cannot be used n a subquery, CTE, TOP c ause, CHECK CONSTRAINT defin t on, or as an argument to an aggregate funct on They a so can’t be used n v ews, computed co umns, and user-defined funct ons, as those object types are not a owed to cause the s de effects of sequence number generat on F na y, you

Chapter 2  T SQL Enhancements   117

cannot ssue a DROP SEQUENCE statement to de ete a sequence wh e you have tab es w th ­ex st ng DEFAULT constra nts that reference the sequence; you’ need to drop the referenc ng constra nts before you can drop the sequence

Metadata Discovery Modern deve opment too s de ver cr t ca product v ty features, such as graph ca des gners and code generators, and those capab t es re y heav y on database metadata d scovery Indeed, t has a ways been poss b e to nterrogate SQL Server for metadata nformat on You can eas y ­d scover a the ­objects n a database (tab es, v ews, stored procedures, and so on) and the r types by d rect y ­query ng system tab es (not recommended, as they can change from one vers on of SQL Server to a ­ nother) or nformat on schema v ews (wh ch are cons stent n each SQL Server vers on) It s s­gn ficant y more cha eng ng, however, to d scover the resu t set schema for T-SQL statements or stored procedures that conta n cond t ona og c Us ng SET FMTONLY ON/OFF has been the common techn que n the past for d scover ng the schema of a query’s resu t set w thout actua y execut ng the query tse f To demonstrate, execute the code shown n L st ng 2-48 Listing 2-48  Us ng the o d (now deprecated) SET FMTONLY ON/OFF techn que for d scover ng a tab e s schema.

USE AdventureWorks2012 GO SET FMTONLY ON SELECT * FROM HumanResources.Employee; SET FMTONLY OFF

Th s SELECT statement, wh ch wou d norma y return a the rows from the HumanResources.­ Employee tab e, returns no rows at a It just revea s the co umns The SET FMTONLY ON statement prevents quer es from return ng rows of data so that the r schemas can be d scovered, and th s ­behav or rema ns n effect unt SET FMTONLY OFF s encountered SQL Server 2012 ntroduces severa new system stored procedures and tab e-va ued funct ons (TVFs) that prov de s gn ficant y r cher metadata d scovery than what can be d scerned us ng the re at ve y ne egant (and now deprecated) SET FMTONLY ON/OFF approach These new procedures and funct ons are ■

sys sp descr be first resu t set

■

sys sp descr be undec ared parameters

■

sys dm exec descr be first resu t set

■

sys dm exec descr be first resu t set for object

118  Part 1  Core SQL Server Development

The sys.sp describe first result set accepts a T-SQL statement and produces a h gh y deta ed schema descr pt on of the first poss b e resu t set returned by that statement The fo ow ng code retr eves schema nformat on for the same SELECT statement you used ear er to get nformat on on a the co umns n the HumanResources.Employee tab e EXEC sp_describe_first_result_set @tsql = N'SELECT * FROM HumanResources.Employee'

F gure 2-3 shows the wea th of nformat on that SQL Server returns about each co umn n the resu t set returned by the T-SQL statement

Figure 2-3  Deta ed metadata returned by sp describe first result set.

A data management funct on named sys.dm exec describe first result set works very s m ar to sys.sp describe first result set But because t s mp emented as a TVF, t s easy to query aga nst t and m t the metadata returned For examp e, the query n L st ng 2-49 exam nes the same T-SQL statement, but returns just the name and data type of nu ab e co umns Listing 2-49  Query ng the new data management funct on to d scover nu ab e co umns n a tab e.

SELECT name, system_type_name FROM sys.dm_exec_describe_first_result_set( 'SELECT * FROM HumanResources.Employee', NULL, 1) WHERE is_nullable = 1



Chapter 2  T SQL Enhancements   119

name ----------------OrganizationNode OrganizationLevel

system_type_name ---------------hierarchyid smallint

Parameter zed quer es are a so supported, f you supp y an appropr ate parameter s gnature after the T-SQL The T-SQL n the prev ous examp e had no parameters, so t passed NULL for the “­parameters parameter ” The code n the L st ng 2-50 d scovers the schema of a parameter zed query Listing 2-50  D scover ng the schema returned by a parameter zed query.

SELECT name, system_type_name, is_hidden FROM sys.dm_exec_describe_first_result_set(' SELECT OrderDate, TotalDue FROM Sales.SalesOrderHeader WHERE SalesOrderID = @OrderID', '@OrderID int', 1)

name --------------OrderDate TotalDue SalesOrderID

system_type_name ---------------datetime money int

is_hidden --------0 0 1

You’d be qu ck to quest on why the SalesOrderID co umn s returned for a SELECT statement that returns on y OrderDate and TotalDue The answer es n the ast parameter passed to the data ­management funct on A bit va ue of 1 (for true) te s SQL Server to return the dent fy ng ­Sa­lesOrderID co umn, because t s used to “browse” the resu t set Not ce that t s marked true (1) for is hidden Th s nforms the c ent that the SalesOrderID co umn s not actua y revea ed by the query, but can be used to un que y dent fy each row n the query’s resu t set What f mu t p e resu t sets are poss b e? There’s no prob em w th th s as ong as they a have the same schema In fact, SQL Server w even try to forg ve cases where mu t p e poss b e schemas are not exact y dent ca For examp e, f the same co umn s nu ab e n one resu t set and non-nu ab e n the other, schema d scovery w succeed and nd cate the co umn as nu ab e It w even to erate cases where the same co umn has a d fferent name (but same type) between two poss b e resu t sets, and nd cate NULL for the co umn name, rather than arb trar y choos ng one of the poss b e co umn names or fa ng a together The code n L st ng 2-51 demonstrates th s w th a T-SQL statement that has two poss b e resu t sets depend ng on the va ue passed n for the @SortOrder parameter Because both resu t sets have ­compat b e schemas, the data management funct on succeeds n return ng schema nformat on

120  Part 1  Core SQL Server Development

Listing 2-51  D scover ng the schema returned from a T SQL statement w th mu t p e compat b e resu t sets.

SELECT name, system_type_name FROM sys.dm_exec_describe_first_result_set(' IF @SortOrder = 1 SELECT OrderDate, TotalDue FROM Sales.SalesOrderHeader ORDER BY SalesOrderID ASC ELSE IF @SortOrder = -1 SELECT OrderDate, TotalDue FROM Sales.SalesOrderHeader ORDER BY SalesOrderID DESC’, '@SortOrder AS tinyint', 0)

name ----------OrderDate TotalDue

system_type_name ---------------datetime money

D scovery won’t succeed f SQL Server detects ncompat b e schemas For examp e, n L st ng 2-52, the ca to the system stored procedure spec fies a T-SQL statement w th two poss b e resu t sets, but one returns three co umns but the other returns on y two co umns Listing 2-52  Attempt ng to d scover schema nformat on returned from a T SQL statement w th mu t p e ­ncompat b e resu t sets.

EXEC sys.sp_describe_first_result_set @tsql = N' IF @IncludeCurrencyRate = 1 SELECT OrderDate, TotalDue, CurrencyRateID FROM Sales.SalesOrderHeader ELSE SELECT OrderDate, TotalDue FROM Sales.SalesOrderHeader'

In th s case, the system stored procedure ra ses an error that c ear y exp a ns the prob em Msg 11509, Level 16, State 1, Procedure sp_describe_first_result_set, Line 53 The metadata could not be determined because the statement 'SELECT OrderDate, TotalDue, CurrencyRateID FROM Sales.SalesOrderHeader' is not compatible with the statement 'SELECT OrderDate, TotalDue FROM Sales.SalesOrderHeader'.

It s noteworthy to ment on that the data management funct on copes w th th s scenar o much more pass ve y G ven confl ct ng resu t set schemas, t s mp y returns NULL and does not ra se an ­error



Chapter 2  T SQL Enhancements   121

The data management funct on sys.dm exec describe first result set for object can be used to ach eve the same d scovery aga nst any object n the database It accepts just an object ID and the Boo ean “browse” flag to spec fy f h dden ID co umns shou d be returned You can use the OBJECT ID funct on to obta n the ID of the des red object The examp e n L st ng 2-53 demonstrates th s by return ng schema nformat on for the stored procedure GetOrderInfo Listing 2-53  D scover ng metadata returned from a stored procedure dent fied by ts object D.

CREATE PROCEDURE GetOrderInfo(@OrderID AS int) AS SELECT OrderDate, TotalDue FROM Sales.SalesOrderHeader WHERE SalesOrderID = @OrderID GO SELECT name, system_type_name, is_hidden FROM sys.dm_exec_describe_first_result_set_for_object(OBJECT_ID('GetOrderInfo'), 1)

name --------------OrderDate TotalDue SalesOrderID

system_type_name ----------------datetime money int

is_hidden --------0 0 1

F na y, the sys.sp describe undeclared parameters stored procedure parses a T-SQL statement to d scover type nformat on about the parameters expected by the statement, as L st ng 2-54 ­demonstrates Listing 2-54  D scover ng parameters (and the r data types) expected by a T SQL statement.

EXEC sys.sp_describe_undeclared_parameters N'IF @IsFlag = 1 SELECT 1 ELSE SELECT 0'

parameter_ordinal name suggested_system_type_id suggested_system_type_name ... ----------------- ------- ------------------------ -------------------------- ------1 @IsFlag 56 int ...

In th s examp e, SQL Server detects the @IsFlag parameter, and suggests the int data type based on the usage n the T-SQL statement t was g ven to parse

Summary We covered a ot of ground n th s chapter As M crosoft cont nues to nvest n T-SQL, deve opers are rewarded w th many powerfu enhancements to the T-SQL eng ne You’ve earned how to use tab e-va ued parameters (TVPs) to pass sets of rows across c ent, server, stored procedures, and userdefined funct ons (UDFs) The date and t me data types were a so covered, as we as the capab t es 122  Part 1  Core SQL Server Development

of MERGE, INSERT OVER DML, and GROUPING SETS You a so earned how to use the atest features n SQL Server 2012, nc ud ng a host of new T-SQL funct ons, the w ndow ng (OVER c ause) enhancements, the THROW statement, server-s de pag ng, SEQUENCE objects, and metadata d scovery mprovements T-SQL s st the ma n way you w program aga nst your data When your operat ons requ re you to nteract w th the operat ng system or use the NET Framework, you can use the CLR to wr te your quer es n V sua Bas c NET or V sua C# In the next chapter, you’ earn how



Chapter 2  T SQL Enhancements   123

C hapter 3

Exploring SQL CLR —Andrew Brust

T

he banner head ne for M crosoft SQL Server 2005 was ts ntegrat on of the M crosoft NET ­common anguage runt me (CLR) Th s arch tectura enhancement gave SQL Server the ab ty to use certa n NET c asses as bas c data types, as we as accommodate the use of NET anguages for the creat on of stored procedures, tr ggers, funct ons, and even user-defined aggregates

Th s capab ty was carr ed forward and enhanced n SQL Server 2008 and s the under y ng enab er of var ous “beyond re at ona ” SQL Server data types, nc ud ng hierarchyid (covered n Chapter 7), and geometry and geography (covered n Chapter 9) SQL CLR techno ogy rema ns mportant n SQL Server 2012 and we w cover the techno ogy, nc ud ng ts deve opment and dep oyment us ng SQL Server Data Too s (SSDT), n th s chapter

Note  Throughout th s chapter, we w refer to the CLR ntegrat on n SQL Server as SQL CLR ­features, funct ona ty, or ntegrat on, and we w refer to SQL CLR stored procedures, tr ggers, funct ons, ­aggregates, and user-defined types (UDTs) as the five bas c SQL CLR ent t es

In th s chapter, you w

earn

■

How to enab e (or d sab e) SQL CLR ntegrat on on your SQL Server

■

How SQL Server accommodates CLR code through the oad ng of NET assemb es

■

■

■

■

How to use SQL Server 2012, M crosoft V sua Stud o 2010, and SSDT together to wr te SQL CLR code and dep oy t, s mp y and qu ck y How to dep oy SQL CLR code ndependent y of V sua Stud o, us ng T-SQL statements, w th or w thout the he p of SQL Server Management Stud o (SSMS) How to create s mp e CLR stored procedures, tr ggers, funct ons, aggregates, and UDTs, use them n your databases, and ut ze them from Transact-SQL (T-SQL) How both the standard SQL Server c ent prov der and the new server-s de brary can be ­comb ned to mp ement SQL CLR funct ona ty

■

How SQL CLR secur ty works and how to configure secur ty perm ss ons for your assemb es

■

When to use SQL CLR funct ona ty, and when to opt to use T-SQL nstead

125

Getting Started: Enabling CLR Integration Before you can earn how to use SQL CLR features, you need to know how to enab e them As w th many new products n the M crosoft W ndows Server system fam y, most advanced features of SQL Server are d sab ed by defau t The reason ng beh nd th s s sound each add t ona feature that s ­enab ed prov des extra “surface area” for attacks on secur ty or ntegr ty of the product, and the added exposure s s mp y not just fied f the feature goes unused The SQL CLR features of SQL Server 2012 are soph st cated and can be very usefu , but they are a so, techn ca y, nonessent a It s poss b e to bu d h gh-performance databases and server-s de programm ng og c w thout SQL CLR ntegrat on, so t s turned off by defau t Don’t be d scouraged, though; turn ng on the feature s easy M crosoft prov des a system stored procedure for enab ng or d sab ng SQL CLR ntegrat on Connect to the server you’d ke to configure n SSDT or SSMS Then, from a query w ndow, type the fo ow ng statements, and execute the scr pt sp_configure 'clr enabled', 1  GO  RECONFIGURE  GO

That’s a there s to t! To d sab e SQL CLR ntegrat on, just use a va ue of 0, nstead of 1, as the second parameter va ue n the sp configure ca

Tip  Don’t forget that this will work from any tool that can connect to SQL Server, not just SSDT and SSMS. In fact, you could issue the previous command text from your own code using the ADO.NET SqlCommand object’s ExecuteNonQuery method as long as your code can connect to your server and your server can authenticate you as a user in the sysadmin server role. W th SQL CLR ntegrat on enab ed, you’re ready to get started wr t ng SQL CLR code Before you d ve n though, we need to d scuss V sua Stud o/SQL Server ntegrat on and when to use t

Visual Studio/SQL Server Integration SSDT and SQL Server 2012 ntegrate t ght y n a number of ways It’s mportant to rea ze, however, that the use of SSDT s comp ete y opt ona and the use of T-SQL s a suffic ent subst tute W th the re ease of SQL Server 2005, T-SQL was enhanced w th new data defin t on anguage (DDL) ­commands for ma nta n ng CLR assemb es, types, and aggregates, and ts ex st ng commands for stored ­procedures, tr ggers, and funct ons were enhanced to recogn ze code w th n dep oyed assemb es V sua Stud o can execute those commands on your beha f It can a so make wr t ng nd v dua SQL CLR c asses and funct ons eas er

126  Part I  Core SQL Server Development

U t mate y, we th nk a deve opers shou d be aware of both SSDT–ass sted and more manua c­ od ng and dep oyment methods You m ght dec de to use one method most of the t me, but n some s tuat ons you’ probab y need the other as we , so we want to prepare you As we cover each major area of SQL CLR programm ng, we w d scuss dep oyment from both po nts of v ew We’ cover some genera po nts about V sua Stud o ntegrat on now, and then we’ move on to cover SQL CLR ­deve opment

SQL Server Database Projects in Visual Studio The SSDT SQL Server Database Project type defines temp ates for the five bas c SQL CLR ent t es These temp ates nject spec fic code attr butes and funct on stubs that a ow you to create SQL CLR code eas y The attr butes are used by SSDT to dep oy your assemb y and ts stored procedures, ­tr ggers, and so on to your database Some of them are a so used by SQL Server to acknow edge and proper y use your funct ons, UDTs, and aggregates To test out the new project type and temp ates, fo ow th s procedure 1. Start V sua Stud o 2010, and then create a new project by choos ng F e New Project,

c­ ck ng New Project on the too bar, press ng Ctrl+Shift+N, or c ck ng the New Project… nk on the V sua Stud o Start Page 2. In the New Project d a og box, shown n F gure 3-1, c ck the expand g yph to the eft of the

Other Languages node n the Insta ed Temp ates tree v ew on the eft, c ck that node’s SQL Server ch d node, and then c ck SQL Server Database Project n the m dd e pane Enter your own project name f you want, and then c ck OK

Figure 3-1  The V sua Stud o 2010 New Project d a og box w th the SQL Server Database Project type

se ected.

You can eas y add preconfigured c asses for the five bas c SQL CLR ent t es to your project, but you must first dec de whether you w sh to use C# or V sua Bas c NET as the programm ng anguage for your SQL CLR Assemb y



Chapter 3  Exp or ng SQL CLR   127

3. Doub e-c ck the Properties node n the So ut on Exp orer, and then c ck the SQLCLR tab n the

resu t ng property sheet des gner Once ns de the SQLCLR tab, se ect C# from the Language combo box, as shown n F gure 3-2

Figure 3-2  The SQL CLR property sheet w th C# se ected as the programm ng anguage.

4. Now you’re ready to add a CLR ent ty to your project You can do th s from the Add New Item

d a og box, wh ch you d sp ay by se ect ng Project Add New Item from the ma n menu or by choos ng Add New Item from the project node’s shortcut menu n So ut on Exp orer If you se ect the SQL CLR C# (or SQL CLR VB) temp ate type from the “Insta ed Temp ates” st on the eft of the Add New Item d a og box, t shou d appear as shown n F gure 3-3

Figure 3-3  The V sua Stud o SQL Server Database Project Add New tem d a og box, w th SQL CLR C# temp ates d sp ayed.

128  Part I  Core SQL Server Development

After se ect ng an ent ty type, a c ass temp ate for that type w be added to your project and opened n the code ed tor w ndow Add t ona y, references to the System, System.Data, and ­System.Xml assemb es w be added to the project These references are requ red by the stubbed code that appears n the SQL CLR c ass temp ates

Automated Deployment Once opened, the use of the SQL Server Database Project temp ate adds a Pub sh opt on to the Bu d opt on of the V sua Stud o ma n menu, wh ch can be used to dep oy the assemb y and the SQL CLR ent t es w th n t SSDT can do a ot of dep oyment work for you But as you’ earn, you can perform the same tasks on your own and, n certa n c rcumstances, have more prec se contro over the dep oyment process when you do so

SQL CLR Code Attributes A number of NET code attr butes are prov ded for SQL CLR deve opers; these are conta ned n the Microsoft.SqlServer.Server namespace Many of them are nserted n your code when you use the ­var ous temp ates n the SQL Server Database Project type, as s a using statement (or an Imports statement n V sua Bas c) to a as the Microsoft.SqlServer.Server namespace tse f If you choose to ­deve op code w thout these temp ates, you must add the appropr ate attr butes, and opt ona y the using (or Imports) statement yourse f A though a these attr butes are prov ded n the same namespace, some are used exc us ve y by SSDT and others are used by both SSDT and SQL Server More SQL CLR attr butes are ava ab e for you to use than we w be ab e to cover n th s ­chapter Spec fica y, we w prov de coverage of the SqlProcedure, SqlFunction, SqlTrigger, ­SqlUser­DefinedAggregate, and SqlUserDefinedType attr butes We w not cover the SqlFacet and SqlMethod attr butes Just as certa n attr butes are not covered here, we cover on y some of the parameters accepted by the attr butes that we do cover And n some cases, we cover on y certa n va ues that can be passed to these attr butes of the many poss b e For examp e, SqlFunction accepts severa parameters, but the on y ones we w cover are Name, FillRowMethodName, and TableDefinition For SqlUserDefinedAggregate and SqlUserDefinedType, we w cover on y a s ng e va ue sett ng for the Format parameter and w not cover the severa other parameters those two attr butes accept The coverage we prov de w be more than suffic ent for you to mp ement bas c, ­ntermed ate, and certa n advanced funct ona ty w th a of the five bas c SQL CLR ent t es The attr butes and parameters that we won’t cover are usefu most y for opt m z ng your SQL CLR code, and they are we documented n SQL Server Books On ne and art c es on MSDN



Chapter 3  Exp or ng SQL CLR   129

About the Sample Code The samp e NET code for th s chapter s prov ded on the book’s compan on webs te (see the “Introduct on” for nstruct ons to down oad the samp e code) n two vers ons The pr mary mater a s supp ed as an SSDT SQL Server Database Project, access b e by open ng the so ut on fi e SQLCLRDemo.sln n the SQLCLRDemo fo der under the VS (V sua Stud o) subfo der of the samp e code fo der We a so supp y the code as a standard C ass L brary project, access b e by open ng the so ut on fi e SQLCLRDemoManual.sln n the SQLCLRDemoManua subfo der The code n each project s v rtua y dent ca , a though the C ass L brary project’s comp ed assemb y cannot be auto-dep oyed to a SQL Server database As we cover each SQL CLR feature, we’ d scuss how automated dep oyment takes p ace from the SSDT SQL Server Database Project and how command-dr ven dep oyment shou d be performed for the C ass L brary project We’ a so d scuss execut ng test scr pts As a compan on to those d scuss ons, we prov de a SQL Server Management Stud o project, access b e by open ng SQLCLRDemo.ssmssln n the SQLCLRDemo fo der under the samp e code fo der’s SSMS subfo der Th s project cons sts of a number of SQL scr pts used for test ng the samp e SQL CLR code, and a scr pt for c ean ng up everyth ng n the database created by the samp e code and tests The project a so conta ns a scr pt fi e named CreateObjects.sql, wh ch dep oys the C ass L brary assemb y and the SQL CLR ent t es w th n t

Your First SQL CLR Stored Procedure A though SQL CLR programm ng can get qu te comp ex and nvo ved, n rea ty t offers a s mp e mode that any NET deve oper can use w th h gh product v ty n re at ve y short order That’s ­because the crux of SQL CLR funct ona ty s noth ng more than the ab ty of SQL Server 2012 to oad NET ­assemb es nto your database and then a ow you to use the funct ons and types w th n the ­assemb y as you define your co umns, v ews, stored procedures, tr ggers, and funct ons To ga n a good understand ng of SQL CLR ntegrat on, you must exam ne ts features and t­ echn ques carefu y Before do ng so, however, et’s qu ck y wa k through an end-to-end scenar o for creat ng and execut ng a SQL CLR stored procedure Th s w make t eas er for you to understand the nd v dua features as we descr be them Str ct y speak ng, any NET c ass brary assemb y ( n certa n cases, us ng appropr ate NET code a ­ ttr butes n ts c asses and funct ons) can be oaded nto your database w th a s mp e T-SQL ­statement To see how eas y th s works, open a query w ndow n SSDT or SSMS us ng a ­connect on to the AdventureWorks2012 samp e database In the samp e code fo der, confirm that the fi e ­SQLCLRDemo.dll s ocated n the VS\SQLCLRDemoManua \SQLCLRDemo\b n\Debug ­subfo der If the samp e code parent fo der were C \Demos, you wou d oad the assemb y nto the AdventureWorks2012 database w th the fo ow ng T-SQL command CREATE ASSEMBLY SQLCLRDemo FROM 'C:\Demos\VS\SQLCLRDemoManual\SQLCLRDemo\bin\Debug\SQLCLRDemo.dll'

130  Part I  Core SQL Server Development

There are other syntax opt ons for the CREATE ASSEMBLY command, but for now we’ focus on the preced ng m ted usage Funct ons n an assemb y that res de w th n a c ass and perform oca computat ona tasks and certa n types of data access can be eas y exposed as SQL Server stored procedures, tr ggers, or ­funct ons As w th convent ona T-SQL stored procedures, tr ggers, and funct ons, a t takes s a s mp e T-SQL CREATE PROCEDURE, CREATE TRIGGER, or CREATE FUNCTION statement to make th s happen We’ go through each of these opt ons n th s chapter, but for now et’s create a s mp e CLR stored procedure You can v ew the source code for the SQLCLRDemo assemb y by open ng the so ut on fi e VS\SQLCLRDemoManua \SQLCLRDemoManual.sln n th s chapter’s samp e code fo der W th n the project, the fi e Sprocs.cs conta ns the fo ow ng code using System.Data.SqlClient; using Microsoft.SqlServer.Server; public partial class Sprocs {   public static void spContactsQuick()   {     SqlContext.Pipe.ExecuteAndSend(new SqlCommand("SELECT * FROM Person.Person"));   } }

The spContactsQuick method s des gned to connect to the database n wh ch ts assemb y has been oaded (AdventureWorks2012), perform a SELECT * aga nst the Person.Person tab e, and then use spec a server-s de objects to send the data back to the c ent app cat on To make th s CLR code ava ab e v a SQL Server as a stored procedure, a so ca ed spContactsQuick, you s mp y execute the fo ow ng command from an SSMS or SSDT query w ndow CREATE PROCEDURE spContactsQuick AS EXTERNAL NAME SQLCLRDemo.Sprocs.spContactsQuick

Important  Be sure to enter the Sprocs.spContactsQuick portion of the command verbatim. This phrase is case sensitive. To test the SQL CLR stored procedure, run t from an SSMS or SSDT query w ndow as you wou d any convent ona stored procedure, as shown here EXEC spContactsQuick

Or s mp y spContactsQuick

When the execut on comp etes, you shou d see the contents of the Person.Person tab e n the Resu ts tab of the query w ndow

Chapter 3  Exp or ng SQL CLR   131

As you can see from th s rather tr v a examp e, wr t ng a CLR stored procedure can be very easy and s a ot ke wr t ng c ent-s de or m dd e-t er code that performs data access us ng ADO NET The b ggest d fferences nvo ve the prov s on of a database connect on and the fact that the data must be “p ped” back to the c ent rather than mere y oaded nto a SqlDataReader and returned, man pu ated, or d sp ayed through a user nterface (UI) The presence of the ­SqlContext object a so d fferent ates SQL CLR code from convent ona ADO NET data access code We’ cover the use of the SqlContext object and ts Pipe property n the next sect on The b ts of T-SQL and C# code just shown certa n y don’t te the who e SQL CLR story The use of the ExecuteAndSend method a owed us to sk p over a number of otherw se mportant concepts There are three ways to dep oy assemb es, and you’ve seen on y a s mp fied vers on of one of those ways Secur ty cons derat ons must be taken nto account, and we haven’t even begun to ook at ­tr ggers, funct ons, aggregates, or UDTs So a though the examp e showed how easy SQL CLR ­programm ng can be, we’ now take our t me and show you the nooks and crann es

CLR Stored Procedures and Server-Side Data Access Our prev ous “qu ck and d rty” samp e ooked at CLR stored procedure deve opment, but we need to cover that top c more thorough y now We’ve a ready covered the mechan cs of wr t ng and ­dep oy ng a stored procedure, but et’s back up a b t to try and understand how CLR stored ­procedures work from a conceptua standpo nt SQL CLR stored procedure code runs n an nstance of the NET CLR that s hosted by SQL Server tse f; t s not ca ed as an externa process, as Component Object Mode (COM)–based extended stored procedures (XPs) wou d be Because SQL CLR code runs n the context of the server, t treats objects n the database as nat ve, local objects, more or ess As such, t must treat the c ent that ca s t as remote Th s contextua env ronment s, n effect, the oppos te of that under wh ch c ent and m dd e-t er ADO NET code runs There, commun cat ng w th the database requ res a remote ­connect on (even f the database s phys ca y on the same computer) and the ADO NET code runs oca y The SQL CLR reversa of th s takes a tt e gett ng used to, but once you’ve mastered th nk ng about th ngs th s way, SQL CLR code becomes easy to wr te and understand Meanwh e, as NET has no ntr ns c way of access ng oca objects on the server or transm tt ng data and messages to the c ent, you must use a spec a set of c asses to perform these tasks These c asses are conta ned n the Microsoft.SqlServer.Server namespace

Note  As an aside, it is interesting and important to note that the Microsoft.SqlServer.Server namespace is actually supplied by the System.Data.dll .NET Framework assembly. This means that you don’t need to worry about adding a reference to your project to use this namespace. The namespace’s location within System.Data.dll also further emphasizes the tight integration between .NET and SQL Server.

132  Part I  Core SQL Server Development

If you want, you can th nk of Microsoft.SqlServer.Server as a he per brary for System.Data.SqlClient It supp es the SQL CLR code attr butes we a ready ment oned, a few enumerat ons, an except on c ass, an nterface, and five other c asses SqlContext, SqlPipe, SqlTriggerContext, SqlMetaData, and SqlDataRecord We’ cover SqlMetaData and SqlDataRecord at the end of th s sect on, and we’ cover SqlTriggerContext when we d scuss CLR tr ggers ater n th s chapter We’ cover the SqlContext and SqlPipe objects r ght now At a h gh eve , the SqlContext object, wh ch s static, prov des a hand e to the server-s de context n wh ch your code runs It a so has a channe to the c ent through wh ch you can return data and text ts Pipe property, wh ch n turn prov des access to a proper y opened and n t a zed SqlPipe object A SqlPipe object can send data and messages to the ca ng c ent though severa methods Send, SendResultsStart, SendResultsRow, SendResultsEnd, and ExecuteAndSend In the preced ng code samp e, you used the SqlPipe object’s ExecuteAndSend method to mp c t y open a connect on, ca ExecuteReader on a SqlCommand object that uses that connect on, and transm t the contents of the resu t ng SqlDataReader back to the c ent A though the mp c t work done by ExecuteAndSend m ght have been conven ent for us to get started qu ck y, t’s mportant to avo d such shortcuts n deta ed d scuss ons of SQL CLR programm ng In genera , SQL CLR stored procedure code that quer es tab es n the database must open a ­connect on to that database, use the SqlCommand object’s ExecuteReader method to query the data, and then use one or a comb nat on of the Send methods to send t back The Send methods do not accept DataSet objects; they accept on y SqlDataReader objects, str ngs, and spec a SqlDataRecord objects L st ng 3-1, wh ch shows the mp ementat on of the funct on spContacts from spTest.cs n the SQLCLRDemo samp e project, s a representat ve examp e of how th s s done Listing 3-1  spContacts from spTest.cs.

[SqlProcedure] public static void spContacts() {     SqlConnection conn = new SqlConnection("context connection=true");     SqlCommand cm = new SqlCommand("SELECT * FROM Person.Person", conn);     conn.Open();     SqlDataReader dr = cm.ExecuteReader();     SqlContext.Pipe.Send("Starting data dump");     SqlContext.Pipe.Send(dr);     SqlContext.Pipe.Send("Data dump complete");     dr.Close();     conn.Close(); }



Chapter 3  Exp or ng SQL CLR   133

Note  The implementation of spContacts in the SQLCLRDemoManual project is identical to that shown in Listing 3-1, but is not decorated with the SqlProcedure code attribute. Because the Person.Person table includes xml columns, and because such columns can cause a slowdown in SQL CLR stored procedures, the query in the sample code for ­spContacts uses a column list (which excludes the xml columns) in the SELECT clause rather than the * wildcard. We retained the SELECT * syntax in the printed code for simplicity and terseness. For th s code to work, you need to use both the Microsoft.SqlServer.Server and System.Data.Sql­Client namespaces (and f you ook n the chapter’s samp e project rather than L st ng 3-1, you’ see that we have a ased both of those namespaces w th using statements) Th s s because any ­convent ona ADO NET objects you m ght use, such as SqlConnection, SqlCommand, and SqlDataReader, are supp ed from System. Data.SqlClient, just as they wou d be n a convent ona c ent app cat on or m dd e-t er assemb y And as a ready d scussed, you need the Microsoft.SqlServer.Server namespace n order to use objects such as SqlContext and SqlPipe The stored procedure temp ate n the SSDT SQL Server Database Project temp ate nc udes the using statement for Microsoft.SqlServer.Server and System.Data.SqlClient automat ca y A though server-s de code uses SqlClient objects, t does so n a spec a zed way For examp e, ­ ot ce that the context connection=true connect on str ng passed to the SqlConnection object’s n ­constructor Th s essent a y nstructs ADO NET to open a new connect on to the database n wh ch the CLR assemb y res des Not ce a so the second ca to the SqlContext.Pipe object’s Send method Here, the SqlDataReader parameter over oad of the SqlPipe object’s Send method s used to push the contents of the SqlDataReader back to the c ent You can th nk of th s method as perform ng a while (dr.Read()) oop through the SqlDataReader and return ng the va ues of each co umn for each ­terat on of the oop But nstead of hav ng to do that work yourse f, the Send method does t for you Before and after the SqlDataReader s p ped (returned to the consumer), the String ­parameter over oad of the Send method s used to send status messages to the c ent When th s stored ­procedure s run, the p ped text appears on the Resu ts tab of the query w ndow when you use the Resu ts To Text opt on n SSMS (Resu ts As Text n SSDT) and on the Messages tab when you use the Resu ts To Gr d (Resu ts As Gr d n SSDT) opt on The rest of the st ng conta ns typ ca ADO NET code, a of t us ng objects from the SqlClient ­prov der And that ustrates we the overa theme of SQL CLR programm ng do what you’d norma y do from the c ent or m dd e t er, and use a few spec a he per objects to work w th n the context of SQL Server as you do so

Piping Data with SqlDataRecord and SqlMetaData We ment oned that the SqlPipe object’s Send method can accept an object of type SqlDataRecord, and we ment oned prev ous y that Microsoft.SqlServer.Server prov des th s object as we as an object named SqlMetaData You can use these two objects together n a CLR stored procedure to return a resu t set one row at a t me, nstead of hav ng to supp y the SqlPipe object’s Send method w th a 134  Part I  Core SQL Server Development

SqlDataReader. Th s a ows (but does not requ re) you to nspect the data before send ng t back to the c ent Send ng SqlDataReader objects prevents nspect on of the data w th n the stored ­procedure because SqlDataReader objects are forward-on y resu t set structures Us ng the ExecuteAndSend method and a SqlCommand object has the same m tat on The SqlDataRecord object perm ts NET code to create an nd v dua row to be returned to the ca ng c ent Its constructor accepts an array of SqlMetaData objects, wh ch n turn descr bes the metadata for each co umn n the row L st ng 3-2, wh ch shows the mp ementat on of the spContactCount funct on from spTest.cs n the SQLCLRDemo samp e project, ustrates how to use SqlPipe.Send together w th SqlDataRecord and SqlMetaData objects to return a s ng e-co umn, s ng e-row resu t set from a stored procedure Listing 3-2  spContactCount from spTest.cs.

[SqlProcedure] public static void spContactCount() {   SqlConnection conn = new SqlConnection("context connection=true");   SqlCommand cm = new SqlCommand("SELECT COUNT(*) FROM Person.Person", conn);   SqlDataRecord drc = new SqlDataRecord(new SqlMetaData("ContactCount",  SqlDbType.Int));   conn.Open();   drc.SetInt32(0, (Int32)cm.ExecuteScalar());   SqlContext.Pipe.Send(drc);   conn.Close(); }

The code dec ares var ab e drc as a SqlDataRecord object and passes ts constructor a s ng e S­ qlMetaData object (Pass ng a s ng e object rather than an array s perm ss b e f the ­SqlDataRecord object w have on y a s ng e co umn ) The SqlMetaData object descr bes a co umn named ­ContactCount of type SqlDbType.Int

Note The SqlDbType enumeration is contained within the System.Data.SqlTypes namespace. The SQL Server Stored Procedure template inserts a using statement for this namespace. If you are creating SQL CLR code without using this template, you will need to add the using statement yourself. The rest of the code s rather stra ghtforward F rst, a context connect on and command are opened and a SELECT COUNT(*) query s performed aga nst the Person.Person tab e Because the query returns a s ng e sca ar va ue, t s run us ng the SqlCommand object’s ExecuteScalar method Next, the va ue returned by ExecuteScalar s cast nto a NET Int32 and that va ue s oaded nto ­co umn 0 (the on y returned co umn) of the SqlDataRecord object us ng ts SetInt32 method The ­SqlDataRecord s then p ped back to the c ent us ng the SqlPipe object’s Send method

Chapter 3  Exp or ng SQL CLR   135

Note  If you wanted to send back multiple SqlDataRecord objects, you would send the first object using the SqlContext object’s SendResultsStart method and then send all s­ ubsequent SqlDataRecord objects using the SendResultsRow method. You would call the SendResultsEnd method after all SqlDataRecord objects had been sent. Once the stored procedure has been dep oyed (the techn ques for wh ch we w d scuss short y), you can execute t just as you wou d any other stored procedure A though the resu t s a s ng e va ue, t s presented as a co umn and the co umn name ContactCount s shown on the Resu ts tab of the query w ndow Keep n m nd that th s COUNT(*) query resu t cou d have been returned w thout us ng the SqlMetaData and SqlDataRecord objects; the samp e s prov ded to demonstrate the use of these objects as an a ternat ve to p p ng SqlDataReader objects and text to the c ent

CLR Stored Procedure Usage Guidelines It’s mportant to understand how to perform data access and retr eva n CLR stored ­procedures As a NET deve oper, you a ready know how to perform computat ona tasks w th n your code, so our samp es ustrate server-s de data access more than anyth ng e se As proofof-concept code, these samp es are comp ete y adequate Meanwh e, you shou d avo d wr t ng CLR stored procedures that mere y perform s mp e “CRUD” (Create, Read, Update and De ete) operat ons Such tasks are better eft to ­convent ona T-SQL stored procedures, wh ch typ ca y perform these operat ons more effic ent y than ADO NET can CLR stored procedures work we when you need to perform computat ons on (or ­us ng) your data, and you requ re the express veness of a NET anguage or the r ch ­funct ona ty prov ded by the NET base c ass brar es to do so (where such express veness and base c ass brary support are m ss ng from T-SQL) For examp e, mp ement ng a “fuzzy search” us ng bus ness og c embedded n NET ­ ssemb es to determ ne wh ch data has an affin ty to other data s a good use of SQL CLR a stored procedures Regu ar-express on-based data va dat on n an update or nsert stored procedure s another good app cat on of SQL CLR ntegrat on As a genera ru e, stra ght data access shou d be eft to T-SQL “H gher-va ued” computat ons are good cand dates for SQL CLR ntegrat on We’ rev s t the SQL CLR usage quest on at var ous po nts n th s chapter

Deployment Before you can test your SQL CLR code, you must dep oy the assemb y conta n ng t and reg ster the nd v dua funct ons that you want recogn zed as stored procedures A number of dep oyment ­methods are at your d sposa ; we w pause to cover them n th s sect on, before d scuss ng test ng of your stored procedures and the other four bas c SQL CLR ent t es 136  Part I  Core SQL Server Development

Getting Ready If you’re us ng a SQL Server Database Project, you’ dep oy your assemb y us ng SSDT’s Pub sh ­funct on But before do ng so, you must be ab e to bu d the project successfu y A bu d w not on y comp e the SQL CLR assemb y tse f, but w a so check the code for var ous errors, nc ud ng ­unreso ved references to database objects Go ahead and bu d the project now by se ect ng the SQLCLRDemo project node n the So ut on Exp orer, and then se ect ng Bu d Bu d SQLCLRDemo from the ma n menu Equ va ent UI se ect ons (after se ect ng the project node) are the Bu d opt on from the project node’s shortcut menu or the Sh ft+F6 keyboard shortcut If you’re us ng the SQLCLRDemo samp e code demo project, after your attempt to bu d the p ­ roject you shou d see a message that the bu d fa ed, w th the fo ow ng error d sp ayed n the Error L st w ndow SQL71501: Trigger: [Person].[trgUpdatePerson] has an unresolved reference to object [Person]. [Person].

If, n the Error L st w ndow, you doub e-c ck the error, a code ed tor w ndow w open w th a T-SQL dep oyment scr pt generated by SSDT In t s a CREATE TRIGGER command for the trgUpdatePerson tr gger and a red squ gg y under the name of the tab e to wh ch the tr gger s be ng app ed, Person. Person The T-SQL s actua y va d, so you may wonder why SSDT m ght flag t w th an error SQL Server Database projects are meant to m rror fu y the databases to wh ch the r contents w be dep oyed A though you can create projects that conta n just a subset of a database—for examp e, the assets for a SQL CLR assemb y—SSDT w b ock you as soon as any T-SQL n the project references an object n the database that s not a so n your project In the case of your project, the dep oyment scr pt references the tab e Person.Person (because the SqlTrigger attr bute app ed to the trgUpdatePerson CLR tr gger references t) and yet there s no T-SQL scr pt n your project that defines that tab e Therefore, by SSDT’s cr ter a, the T-SQL s mproper You cou d so ve th s by add ng a scr pt for the Person.Person tab e and then add ng further scr pts, n a p ecemea fash on, for any objects upon wh ch the tab e s dependent Wh e do ng so wou d ­a ow the project to bu d, t wou d st be a work-around, and thus someth ng you ought to avo d The rea so ut on s to mport the AdventureWorks2012 database nto the project; that way your ­project w reflect a fu defin t on of the database “Import ng” the database rea y trans ates nto hav ng SSDT d scover the database’s structure and create correspond ng scr pts for each object Let’s do the mport now As a preparatory step, c ose V sua Stud o, open W ndows Exp orer and nav gate to the source code d rectory conta n ng the project fi e (not the so ut on fi e) and the ­project’s contents (e g C:\Demos\VS\SQLCLRDemo\SQLCLRDemo) Look for a fi e ca ed SQLCLRDemo.dbmdl and de ete t, then re-open the so ut on n V sua Stud o Th s preparatory step s necessary to enab e the Import Database funct ona ty that we need to use; f the step s not ­performed after a fa ed bu d, the Import Database opt on w be grayed-out Let’s now perform the mport operat on Once aga n, se ect the SQLCLRDemo project node n the So ut on Exp orer, then r ght-c ck that node and se ect Import Database from the shortcut menu (you can a so se ect Project Import Database from the ma n menu) Th s se ect on w br ng up the Import Database d a og box, wh ch s shown n F gure 3-4

Chapter 3  Exp or ng SQL CLR   137

Figure 3-4 The mport Database d a og box.

Se ect the AdventureWorks2012 database from the Source Database Connect on combo box, or c ck the New Connect on button, define a connect on to the AdventureWorks2012 database on ­localhost, and c ck OK Now c ck Start The Summary screen of the mport database d a og box w appear and the mport process w proceed When the F n sh button becomes enab ed, c ck t to ­fin sh the mport process Now bu d the project once more Th s t me the bu d shou d succeed, n wh ch case you are ready to dep oy your assemb y

Deploying Your Assembly Hav ng comp eted a successfu bu d, you’re now ready to pub sh the database, and dep oy the ­assemb y n the process Se ect Bu d Pub sh SQLCLRDemo from the ma n menu now (or r ght-c ck the SQLCLRDemo project node n the So ut on Exp orer and se ect Pub sh from the shortcut menu); th s w br ng up the Pub sh Database d a og box, shown n F gure 3-5

Figure 3-5 The Pub sh Database d a og box.

138  Part I  Core SQL Server Development

C ck the Ed t button next to the Target Database Connect on read-on y text box to br ng up the standard Connect on Propert es d a og box Configure the connect on to po nt to your target server (e g localhost), to use W ndows Authent cat on and to po nt to the AdventureWorks2012 database (make sure not to accept the defau t database name of SQLCLRDemo), then c ck OK Back n the ­Pub sh Database d a og box, c ck the Pub sh button to dep oy your CLR assemb y to the database

Note  Before clicking Publish, you can optionally click the Save Profile As button to save this publish profile. When you next publish the database, you can use the Load Profile button to retrieve the profile and avoid configuring the target database connection manually once again. For dep oy ng the C ass L brary project vers on, assum ng C \Demos s the samp e code parent d rectory, you can execute the fo ow ng T-SQL statement from w th n a query w ndow CREATE ASSEMBLY SQLCLRDemo AUTHORIZATION dbo FROM 'C:\Demos\VS\SQLCLRDemoManual\SQLCLRDemo\bin\Debug\SQLCLRDemo.dll' WITH PERMISSION_SET = SAFE GO

The AUTHORIZATION c ause a ows you to spec fy a name or ro e to wh ch ownersh p of the ­assemb y s ass gned The defau t author zat on s that of the current user, and because you are most ke y ogged n as dbo for AdventureWorks2012, n th s case the c ause s unnecessary (wh ch s why we om tted t from our prev ous examp e) The mean ng and effect of the WITH PERMISSION SET c ause s d scussed n the “Secur ty” ­sect on near the end of th s chapter For now, just note that th s c ause a ows you to spec fy the secur ty ­perm ss ons w th wh ch your assemb y runs As w th the AUTHORIZATION c ause, n th s case the WITH ­PERMISSION SET c ause s techn ca y unnecessary because SAFE s the defau t PERMISSION SET va ue used when a CREATE ASSEMBLY command s executed Regard ess, t’s a good pract ce to nc ude both c auses If your assemb y has dependenc es on other assemb es, SQL Server ooks to see whether those assemb es have a ready been oaded nto the database and, f so, confirms that the r ownersh p s the same as that of the spec fied assemb y If the dependent assemb es have not yet been oaded nto the database, SQL Server ooks for them n the same fo der as the spec fied assemb y If t finds a ­dependent assemb es n that ocat on, t oads them and ass gns them the same ownersh p as the pr mary assemb y If t does not find the dependent assemb es n that fo der or n the g oba assemb y cache (GAC), the CREATE ASSEMBLY command w fa You can supp y a str ng express on nstead of a tera n the FROM c ause, a ow ng for some ­nterest ng data-dr ven poss b t es For examp e, you cou d fetch an assemb y path reference from a tab e n your database It s a so poss b e to supp y an assemb y d rect y n ne by prov d ng a ­b nary stream n the FROM c ause nstead of a fi e spec ficat on You do th s by spec fy ng a ­varbinary tera va ue or express on (or a comma-de m ted st of varbinary va ues or express ons, when

Chapter 3  Exp or ng SQL CLR   139

­ ependent assemb es must be spec fied) that conta ns the actua b nary content of your assemb y d (or ­assemb es) Th s a ows the creat on of a database, nc ud ng any CLR assemb es t conta ns, to be comp ete y scr pted, w thout requ r ng d str but on of actua assemb y fi es The b nary stream can be embedded n the scr pt tse f or, us ng an express on, t cou d be fetched from a tab e n a database In add t on to us ng V sua Stud o dep oyment and the T-SQL CREATE ASSEMBLY statement, you can up oad the assemb y nto your database nteract ve y from SSMS S mp y r ght-c ck the ­servername/Databases/AdventureWorks2012/Programmability/Assemblies node n the Object ­Exp orer (where servername s the name of your server), and then choose New Assemb y from the shortcut menu The New Assemb y d a og box, shown n F gure 3-6, appears

Figure 3-6 The SSMS New Assemb y d a og box.

Type the assemb y path and fi e name n the Path To Assemb y text box, or use the Browse button to spec fy t nteract ve y You can spec fy AUTHORIZATION and WITH PERMISSION SET deta s n the Assemb y Owner text box (us ng the e ps s button, f necessary) and the Perm ss on Set combo box, respect ve y Regard ess of the dep oyment method you use, once your assemb y has been added to your ­ atabase, t becomes an ntegra part of that database and ts under y ng mdf fi e Th s means that f d your database s backed up and restored, or dep oyed, any assemb es w th n t move a ong w th the data tse f and need not be added manua y as a subsequent step

140  Part I  Core SQL Server Development

Deploying Your Stored Procedures In the SQL Server Database Project vers on of the samp e code, dep oyment of a the stored ­procedures s hand ed by V sua Stud o when the assemb y tse f s dep oyed Th s s due to the ­app cat on of the SqlProcedure attr bute to the methods n the StoredProcedures c ass (found n the spTest.cs fi e) The SqlProcedure attr bute accepts an opt ona Name parameter, the va ue of wh ch s the actua ca ab e stored procedure name If you do not supp y a va ue for the Name parameter, the name of the NET method s used as the stored procedure name The SqlProcedure attr bute s used on y by V sua Stud o when auto-dep oy ng SQL CLR ­assemb es Therefore, t does not appear n the source code n the C ass L brary project Dep oy ng the stored procedures from that vers on of the source code requ res ssu ng a CREATE PROCEDURE T-SQL ­command us ng the EXTERNAL NAME c ause to spec fy the assemb y, the fu y qua fied c ass name spec fier, and the method name For examp e, to oad the C ass L brary vers on of spContacts, you wou d ssue the fo ow ng command CREATE PROCEDURE spContacts AS EXTERNAL NAME SQLCLRDemo.StoredProcedures.spContacts

The preced ng command spec fies that the spContacts method, found n the c ass named StoredProcedures, n the oaded assemb y w th T-SQL name SQLCLRDemo, shou d be reg stered as a CLR stored procedure ca ab e under the name spContacts

Note  All necessary CREATE PROCEDURE commands for the Class Library project version of the sample code are contained in the CreateObjects.sql script in the SSMS project supplied with the chapter’s sample code on the book’s companion website. You will need to run that script in order to utilize the various SQL CLR entities implemented in the Class Library ­project. Before running the script, you will need to edit the CREATE ASSEMBLY command at the top so that the assembly’s file specification points to the location on your system where the assembly has been installed. By default, the script looks for the assembly in the C:\Demos\VS\SQLCLRDemoManual\SQLCLRDemo\bin\Debug folder on your PC. Note that f the CLR stored procedure had been wr tten n M crosoft V sua Bas c NET rather than C#, the c ass name spec fier wou d change to SQLCLRDemo.StoredProcedures Th s wou d necess tate a change to the dep oyment T-SQL code as fo ows CREATE PROCEDURE spContacts AS EXTERNAL NAME SQLCLRDemo.[SQLCLRDemo.StoredProcedures].spContacts

In V sua Bas c projects, the defau t namespace for a project tse f defau ts to the project name, as does the assemb y name The c ass w th n the project must be referenced us ng the defau t namespace as a prefix Because the c ass spec fier s a mu t part dot-separated name, t must be ­enc osed w th n square brackets so that SQL Server can dent fy t as a s ng e nd v s b e name Because C# projects hand e the defau t namespace sett ng a tt e d fferent y, the namespace prefix s not used n the c ass spec fier for C# assemb es

Chapter 3  Exp or ng SQL CLR   141

One ast po nt before we d scuss how to test your now-dep oyed SQL CLR stored procedures It s mportant to rea ze that the c ass spec fier and method name n the EXTERNAL NAME c ause are case sensitive, and that th s s true even for assemblies developed in Visual Basic .NET. A though th s po nt may seem perp ex ng at first, t does make sense SQL Server searches for your methods w th n your assemb es, not w th n your source code In other words, t’s ook ng w th n M crosoft Intermed ate Language (MSIL) code, not V sua Bas c NET or C# source code Because MSIL s case sens t ve ( t has to be to support case-sens t ve anguages ke C#), SQL Server must be as we as t searches w th n an assemb y for a spec fic c ass and method The fact that SQL Server s not case sens t ve by defau t (even though t once was) and that V sua Bas c NET s not a case-sens t ve anguage s of no mportance! If you attempt to reg ster a method and you rece ve an error that t cannot be found w th n the assemb y, doub e-check that the case ­usage n your command matches that of your source code

Testing Your Stored Procedures The TestStoredProcs.sql scr pt fi e n the SSMS project supp ed w th th s chapter’s samp e code w run both CLR stored procedures (spContactCount and spContacts) Open the fi e n SSMS, and then c ck the Execute button on the SQL Ed tor too bar, choose Execute from the Query menu, press F5, or press Ctrl-E (You can a so r ght-c ck anywhere ns de the query w ndow and se ect Execute from the shortcut menu ) When the scr pt runs, you shou d see the s ng e-va ued resu t of the spContactCount stored ­ rocedure appear first, as shown n F gure 3-7 Not ce that the co umn name ContactCount ­appears p on the Resu ts tab and reca that th s s a d rect resu t of your us ng the SqlMetaData object n the CLR code Be ow the spContactCount resu t, you w see the resu ts from the spContacts stored ­procedure come n Because the Person.Person tab e has a most 20,000 rows, on certa n mach nes, these resu ts m ght take some t me to retr eve

Figure 3-7  TestStoredProcs.sql scr pt code and resu ts.

142  Part I  Core SQL Server Development

Even wh e the resu ts are com ng n, the “Start ng Data Dump” status message shou d be v s b e on the Messages tab (or on the Resu ts tab f you’re us ng the Resu ts To Text opt on n SSMS) After a the rows have been fetched, you shou d see the “Data Dump Comp ete” message appear as we We have yet to cover CLR funct ons, tr ggers, aggregates, and UDTs, but you have a ready earned most of the sk s you need to deve op SQL CLR code You have earned how to create a V sua Stud o SQL Server Database Project and use SSDT’s CLR assemb y auto-dep oyment features You have a so earned how to deve op SQL CLR code n standard C ass L brary projects and how to use T-SQL ­commands to dep oy the code for you You’ve earned about the subt e d fferences between ­dep oy ng C#- and V sua Bas c NET-based SQL CLR assemb es, and we’ve covered the case-sens t ve requ rements of T-SQL–based dep oyment W th a th s under your be t, we can cover the rema n ng four bas c SQL CLR ent t es re at ve y qu ck y

CLR Functions Let’s take everyth ng we’ve d scussed about SQL CLR stored procedures and dep oyment and app y t to SQL CLR funct ons As any programmer knows, a function s a procedure that returns a va ue (or an object) Ma nstream NET funct ons typ ca y return NET types SQL CLR funct ons, on the other hand, must return a SqlType So to start w th, you need to make sure that your c asses that mp ement SQL CLR funct ons mport the System.Data.SqlTypes namespace w th a using or Imports statement The SQL Server Database Project C# temp ate for user-defined funct ons (UDFs) conta ns the ­appropr ate using statement by defau t; you w need to add the using statement manua y to standard C ass L brary code Once the namespace s mported, you can wr te the funct ons themse ves In V sua Stud o SQL Server Database Projects, they shou d be decorated w th the SqlFunction attr bute; th s attr bute accepts an ­opt ona name parameter that works dent ca y to ts SqlProcedure counterpart In our samp e code, a va ue s not supp ed for th s parameter SqlFunction s used by V sua Stud o SQL Server Database ­Projects for dep oyment of your SQL CLR funct ons For sca ar-va ued funct ons n C ass L brary projects, the SqlFunction attr bute s opt ona , so t appears n the C ass L brary samp e code on y for the tab e-va ued funct on (TVF) descr bed ater n th s sect on L st ng 3-3, wh ch shows the code for funct on fnHelloWorld from fnTest.cs n the SQLCLRDemo samp e code project, mp ements a s mp e “He o Wor d” funct on that returns a va ue of type ­SqlString Listing 3-3  fnHelloWorld from fnTest.cs.

[SqlFunction] public static SqlString fnHelloWorld() {     return new SqlString("Hello World"); }



Chapter 3  Exp or ng SQL CLR   143

Not ce that SqlType objects (such as SqlString) requ re exp c t nstant at on and constructor va ue pass ng; you cannot s mp y dec are and ass gn va ues to them The code n L st ng 3-3 nstant ates a SqlString object n ne w th n the return statement to avo d var ab e dec arat on A funct on that accepts no parameters and returns a hard-coded va ue s of tt e pract ca use ­ yp ca y, funct ons are passed va ues and perform ca cu at ons w th those va ues, and they are often T used from w th n T-SQL statements, n effect as extens ons to the funct ons bu t nto the T-SQL anguage tse f L st ng 3-4, wh ch shows the code for funct on fnToCelsius n fnTest.cs n the SQLCLRDemo samp e project, mp ements a Fahrenhe t-to-Ce s us convers on funct on Listing 3-4  fnToCelsius from fnTest.cs.

[SqlFunction] public static SqlDecimal fnToCelsius(SqlInt16 fahrenheit) {     return new SqlDecimal((((Int16)fahrenheit) - 32) / 1.8); }

The funct on accepts a Fahrenhe t temperature (as a SqlInt16), converts t to Ce s us, and returns t (as a SqlDecimal) Not ce that the code casts the nput parameter from a SqlInt16 to a NET Int16, app es a Fahrenhe t-to-Ce s us convers on formu a, and then passes the resu t to the constructor of a new SqlDecimal object Dep oyment of these funct ons s automat c n the SSDT SQL Server Database Project vers on of our samp e code For the C ass L brary vers on, use the T-SQL CREATE FUNCTION statement n a ­s m ar fash on to how the CREATE PROCEDURE statement was used n the prev ous sect on, but nc ude a data type spec ficat on for the return va ue For examp e, to dep oy the fnHelloWorld funct on, you wou d use th s statement CREATE FUNCTION fnHelloWorld() RETURNS nvarchar(4000) WITH EXECUTE AS CALLER AS EXTERNAL NAME SQLCLRDemo.UserDefinedFunctions.fnHelloWorld

Not ce the use of data type nvarchar(4000) to correspond w th the SqlString type used n the funct on’s mp ementat on The WITH EXECUTE AS CALLER c ause spec fies that the SQL CLR funct on shou d execute under the ca er’s dent ty

Tip  You can enter the CREATE FUNCTION command yourself, but note that all such ­necessary commands for the sample code SQL CLR functions are available in the CreateObjects.sql script file in the SSMS project supplied with this chapter’s sample code. You can test these funct ons n SSDT or SSMS Use the fo ow ng query to test the two funct ons (You can a so run the TestScalarFunctions.sql scr pt fi e n the SSMS samp e project ) SELECT    dbo.fnHelloWorld() AS HelloWorld,    dbo.fnToCelsius(212) AS CelsiusTemp

144  Part I  Core SQL Server Development

T-SQL funct ons can return resu t sets as we as sca ar va ues Funct ons wh ch return resu t sets are ca ed table-valued functions (TVFs) Wr t ng SQL CLR TVFs s poss b e, a though you do so ­d fferent y than you wou d SQL CLR sca ar-va ued funct ons or SQL CLR stored procedures SQL CLR TVFs must return a type that mp ements the NET nterface IEnumerable, and they must a so prov de a “fi row” method that converts an e ement of that type to a co ect on of sca ar va ues that compr se the ­correspond ng tab e row L st ng 3-5, wh ch shows the code for funct ons fnPortfolioTable and FillTickerRow n fnTest.cs n the SQLCLRDemo samp e project, mp ements a TVF named fnPortfolioTable Listing 3-5  fnPortfolioTable and FillTickerRow from fnTest.cs.

[SqlFunction(      FillRowMethodName="FillTickerRow",      TableDefinition="TickerSymbol nvarchar(5), Value decimal")] public static System.Collections.IEnumerable fnPortfolioTable(SqlString  tickersPacked) { string[] tickerSymbols; object[] rowArray = new object[2]; object[] compoundArray = new object[3]; char[] parms = new char[1]; parms[0] = ';'; tickerSymbols = tickersPacked.Value.Split(parms); rowArray[0] = tickerSymbols[0]; rowArray[1] = 1; compoundArray[0] = rowArray; rowArray = new object[2]; rowArray[0] = tickerSymbols[1]; rowArray[1] = 2; compoundArray[1] = rowArray; rowArray = new object[2]; rowArray[0] = tickerSymbols[2]; rowArray[1] = 3; compoundArray[2] = rowArray; return compoundArray; } public static void FillTickerRow(object row, ref SqlString tickerSymbol, ref  SqlDecimal value) { object[] rowArray = (object[])row; tickerSymbol = new SqlString((string)rowArray[0]); value = new SqlDecimal(decimal.Parse(rowArray[1].ToString())); }



Chapter 3  Exp or ng SQL CLR   145

Rather than mp ement ng ts own IEnumerable-compat b e type, fnPortfolioTable uses an array Th s s perfect y ega because arrays n NET mp ement IEnumerable The fnPortfolioTable ­funct on accepts a sem co on-de m ted st of stock t cker symbo s and returns a tab e w th each t cker ­symbo appear ng n a separate row as co umn TickerSymbol and a va ue for the t cker as co umn Value The structure of the returned tab e s dec ared n the TableDefinition parameter of the SqlFunction ­attr bute n SQL Server projects and n the CREATE FUNCTION T-SQL command for C ass L brary ­projects The ass gned va ues are hard-coded, and on y three rows are returned, regard ess of how many t cker symbo s are passed n As w th our other samp es, th s one s more usefu as a teach ng too than as a pract ca app cat on of TVFs Arrays are the name of the game here F rst, the Split method s used to crack the de m ted t cker st nto an array of s ng e t cker str ngs Then the TVF structures the data so that each e ement n the return va ue array (compoundArray) s tse f a two-e ement array stor ng a s ng e t cker symbo and ts va ue The funct on code needs on y to return compoundArray Next, the FillTickerRow funct on (named n the FillRowMethodName parameter of the SqlFunction attr bute) takes each ­t wo-e ement array (passed n as the first parameter) and converts ts members to nd v dua sca ars These ­sca ars are then returned v a ref parameters that start n the second parameter pos t on; the parameter names are not s gn ficant, but the r pos t ons must correspond to those of the co umns n the ­TableDefinition argument of the SqlFunction attr bute Because the FillRowMethodName parameter of the SqlFunction attr bute s requ red by SQL Server, the C ass L brary vers on of the fnPortfolioTable funct on s decorated w th that attr bute, supp y ng a va ue for that one parameter In the SQL Server Database Project vers on, a va ue s a so supp ed for the TableDefinition parameter to enab e auto-dep oyment of the TVF As w th the other funct ons, dep oyment of th s funct on s performed by SSDT n the SQL Server Database Project samp e code For the C ass L brary vers on, you can dep oy the funct on us ng the fo ow ng T-SQL command (a so conta ned n the CreateObjects.sql scr pt fi e) CREATE FUNCTION fnPortfolioTable(@TickersPacked nvarchar(4000)) RETURNS table (  TickerSymbol nvarchar(5),  Value decimal ) WITH EXECUTE AS CALLER AS EXTERNAL NAME SQLCLRDemo.UserDefinedFunctions.fnPortfolioTable

As w th fnHelloWorld, the SqlString data type s mapped to an nvarchar(4000), th s t me for one of the nput parameters Because fnPortfolioTable s a TVF, ts return type s dec ared as table, w th n ne spec ficat ons for the tab e’s defin t on Use the fo ow ng query n SSDT or SSMS to test the TVF (or run the TestTableValuedFunction.sql scr pt fi e n the SSMS samp e project) SELECT * FROM fnPortfolioTable('IBM;MSFT;ORCL') ORDER BY TickerSymbol

146  Part I  Core SQL Server Development

The fo ow ng data shou d be returned TickerSymbol Value ------------  -----IBM 1 MSFT 2 ORCL 3

Note that as of SQL Server 2008, t s poss b e to prov de a “h nt” to the database that your TVF returns data n a part cu ar order Th s can opt m ze quer es or creat on of ndexes ordered on the same express on To take advantage of th s, you must first ensure that og c n your CLR TVF code returns data n a spec fic order and, second, ensure that you spec fy that order n an ORDER c ause n the CREATE FUNCTION T-SQL command For examp e, mag ne that the SQL CLR TVF code ordered ts resu ts by the TickerSymbol co umn In that case, you cou d mod fy the T-SQL code that creates the funct on as fo ows CREATE FUNCTION fnPortfolioTable(@TickersPacked nvarchar(4000)) RETURNS table ( TickerSymbol nvarchar(5), Value decimal ) WITH EXECUTE AS CALLER ORDER (TickerSymbol) AS EXTERNAL NAME SQLCLRDemo.UserDefinedFunctions.fnPortfolioTable

A though th s code does not actually return data n TickerSymbol order, you can st use the preced ng T-SQL command for the C ass L brary vers on of the TVF If you re-executed the query n the TestTableValuedFunction.sql scr pt fi e, everyth ng wou d work correct y, because the nput data s co nc denta y supp ed n TickerSymbol order (‘IBM;MSFT;ORCL’) If, however, you mod fied the nput str ng to, for examp e, ‘IBM;ORCL;MSFT’, you wou d rece ve the fo ow ng error The order of the data in the stream does not conform to the ORDER hint specified for the CLR TVF 'fnPortfolioTable'. The order of the data must match the order specified in the ORDER hint for a CLR TVF. Update the ORDER hint to reflect the order in which the input data is ordered, or update the CLR TVF to match the order specified by the ORDER hint.

We po nt out th s error text because t s rea y the on y outward proof (other than ana ys s of ­query execut on p ans) that th s new ordered SQL CLR TVF feature s n fact supported, and we a ­ ssume that you wou d prefer not to take th s nformat on mere y on fa th P ease note that th s feature can on y be taken advantage of, pract ca y speak ng, when reg ster ng a SQL CLR TVF us ng one’s own T-SQL code That s because there s no parameter n the SqlFunction attr bute for spec fy ng an ORDER c ause, so SSDT’s auto-dep oyment cannot em t that c ause Th s means that you must use the ORDER c ause on y w th a C ass L brary project or e se om t the SqlFunction attr bute from your SQL CLR TVF code n a SQL Server Database Project and reg ster the TVF “manua y,” e ther from an ­externa scr pt or by add ng a scr pt to your SQL Server Database Project that performs the ­dep oyment



Chapter 3  Exp or ng SQL CLR   147

CLR Triggers T-SQL tr ggers are rea y just stored procedures that are ca ed by SQL Server at spec fic t mes and that can query va ues n the DELETED and INSERTED pseudo-tab es (wh ch expose “before and after” snapshots of data changed by the statement that fired the tr gger) SQL CLR tr ggers are s m ar to SQL CLR stored procedures, and they can be created for a data man pu at on anguage (DML) ­act ons that mod fy data (that s, updates, nserts, and de etes) SQL Server 2005 ntroduced the concept of data defin t on anguage (DDL) tr ggers, wh ch can ntercept and hand e act ons such as CREATE TABLE and ALTER PROCEDURE L ke DML tr ggers, DDL tr ggers can be mp emented n T-SQL or SQL CLR code We w cover both SQL CLR DML and DDL tr ggers n th s sect on SQL CLR DML tr ggers, ke the r T-SQL counterparts, have access to the DELETED and INSERTED pseudo-tab es and must be dec ared as hand ng one or more spec fic events for a spec fic tab e or, under certa n c rcumstances, a spec fic v ew A so, they can make use of the SqlTriggerContext object (through the SqlContext object’s TriggerContext property) to determ ne wh ch part cu ar event ­(update, nsert, or de ete) caused them to fire and wh ch co umns were updated Once you understand these concepts, wr t ng SQL CLR DML tr ggers s rea y qu te s mp e ­ L st ng 3-6, wh ch shows the code for the trgUpdatePerson funct on from trgTest.cs n the ­SQLCLRDemo samp e project, shows the SQL CLR code for the trgUpdatePerson DML tr gger, wh ch s des gned to funct on as a FOR UPDATE tr gger on the Person.Person tab e n the AdventureWorks2012 database Listing 3-6  trgUpdatePerson from trgTest.cs.

[SqlTrigger(Target="Person.Person", Event="for UPDATE")] public static void trgUpdatePerson() {     SqlTriggerContext context = SqlContext.TriggerContext;     string oldName = string.Empty;     string newName = string.Empty;     SqlConnection conn = new SqlConnection("context connection=true");     SqlCommand cmOld = new SqlCommand( "SELECT FirstName FROM DELETED", conn);     SqlCommand cmNew = new SqlCommand( "SELECT FirstName FROM INSERTED", conn);     conn.Open();     SqlDataReader drOld = cmOld.ExecuteReader();     if (drOld.Read())     {         oldName = (string)drOld[0];     }     drOld.Close();     SqlDataReader drNew = cmNew.ExecuteReader();     if (drNew.Read())     {

148  Part I  Core SQL Server Development

        newName = (string)drNew[0];     }     drNew.Close();     conn.Close();     SqlContext.Pipe.Send("Old Value of FirstName:" + oldName);     SqlContext.Pipe.Send("New Value of FirstName:" + newName);     for (int i = 0; i  ‘ eve -separat on str ng In th s manner, the code obta ns the parent, grandparent, great-grandparent, and so on, a the way up to the root When the oop term nates, @DisplayPath s returned to the ca er Here s the output from the fnGetFullDisplayPath funct on when run aga nst the emp oyee tab e SELECT NodeId, NodeId.ToString() AS NodeIdPath, dbo.fnGetFullDisplayPath(NodeId) AS NodeIdDisplayPath, EmployeeName FROM Employee ORDER BY NodeLevel, NodeId GO NodeId -----0x 0x58 0x68 0x78 0x5AC0 0x5B40 0x6AC0 0x7AC0 0x5AD6 0x5ADA

NodeIdPath ---------/ /1/ /2/ /3/ /1/1/ /1/2/ /2/1/ /3/1/ /1/1/1/ /1/1/2/

NodeIdDisplayPath ----------------------------Dave Dave > Amy Dave > John Dave > Jill Dave > Amy > Cheryl Dave > Amy > Wanda Dave > John > Mary Dave > Jill > Kevin Dave > Amy > Cheryl > Richard Dave > Amy > Cheryl > Jeff

EmployeeName -----------Dave Amy John Jill Cheryl Wanda Mary Kevin Richard Jeff

(10 row(s) affected)

The NodeIdDisplayPath n th s resu t set shows the emp oyee names a ong the h erarch ca paths n each row Popu at ng a h erarch ca tab e does requ re some effort up front, as you’ve seen However, once your data s n p ace, you can everage the power of the hierarchyid data type by nvok ng ts methods to eas y query and reorgan ze the tree structure We’re a most ready to d ve nto that, but first et’s take a moment to ta k about the ndex ng opt ons that you need to know about

Hierarchical Table Indexing Strategies You can create a depth-first ndex or a breadth-first ndex (or both) on your h erarch ca tab es The two types d ffer n how SQL Server phys ca y stores node references n the ndex Defin ng ­depth-first and breadth-first ndexes can have a s gn ficant mpact on performance for access ng data n ­h erarch ca tab es

Chapter 7  H erarch ca Data and the Re at ona Database   313

As the r names mp y, depth-first ndex ng stores parent and ch d node references near each other, whereas breadth-first ndex ng stores references for nodes at the same h erarch ca eve near each other Therefore, you w choose the appropr ate type of ndex based on an understand ng of how your h erarch ca data s shaped n the tab e, how t w grow, and how t w be typ ca y quer ed by ­c ent app cat ons You can create both types of ndexes as we , for effic ent access both vert ca y and hor zonta y across the tree

Note  Regardless of what indexing strategy you employ, comparison operations are a ­ lways performed in depth-first order. Thus, A < B means that A always comes before B in a ­depth-first traversal of the tree (or A is to the left of B if they are both at the same level), even if breadth-first indexing is used.

Depth-First Indexing Defin ng e ther a pr mary key ndex or a un que ndex on a hierarchyid co umn resu ts n a d ­ epth-first ndex Because the NodeId co umn (of type hierarchyid) s des gnated as the pr mary key n the ­emp oyee tab e, you have a ready created a depth-first ndex W th depth-first ndex ng (dep cted n F gure 7-3), SQL Server tr es to phys ca y store references to nodes n a subtree as near to each other as poss b e By “near each other,” we mean that SQL Server records them on d sk n the same page, f poss b e—or n the same extent, f not, and so on— n order to max m ze query performance Th s strategy y e ds h gh query performance f your h erarchy runs very many eve s deep Creat ng a depth-first ndex for such a h erarchy w resu t n very fast vert ca search ng on the tree (that s, query ng ancestors and descendants up and down a potent a y ong cha n)

Figure 7-3  Phys ca row storage when opt m zed for depth first search ng.

Breadth-First Indexing W th breadth-first ndex ng, ustrated n F gure 7-4, reference to nodes at the same eve of the h erarchy are phys ca y stored as near to each other as poss b e Th s type of ndex y e ds h gh query performance for trees that grow very broad If there are many ch dren beneath the parents n your h erarchy, you w want to create a breadth-first ndex to enab e fast hor zonta search ng across a potent a y arge number of nodes at the same eve 314  Part II  Going Beyond Relational

Figure 7-4  Phys ca row storage when opt m zed for breadth first search ng.

To define a breadth-first ndex, you create a compos te ndex on your h erarch ca tab e that nc udes two co umns the nteger co umn that ho ds the level of the node w th n the h erarchy (such as the NodeLevel co umn defined n the emp oyee tab e, based on the GetLevel method aga nst the NodeId co umn) and the hierarchyid co umn tse f (NodeId) So to create a breadth-first ndex, run the fo ow ng code CREATE UNIQUE INDEX IX_EmployeeBreadth ON Employee(NodeLevel, NodeId)

As we ment oned a ready, one tab e can have both depth-first and breadth-first ndexes by c­ reat ng one pr mary key or un que ndex on the hierarchyid co umn and another compos te ­ndex on the node- eve co umn and the hierarchyid Th s w carry s ght y more overhead for data ­man pu at on anguage (DML) act ons, because updates w need to be performed n both ndexes; however, query performance w be very fast for both hor zonta and vert ca search ng across arge h erarch es that are both very broad and very deep

Querying Hierarchical Tables W th a h erarch ca tab e popu ated and ndexed, you’re ready to start wr t ng effic ent quer es aga nst t us ng some more methods ava ab e on the hierarchyid type

The IsDescendantOf Method The IsDescendantOf method s nvoked on one hierarchyid va ue and accepts another hierarchyid va ue as ts parameter It returns a bit (Boo ean) va ue of 1 (true) f the hierarchyid that the method s nvoked on s a descendant (e ther d rect y or nd rect y) of the hierarchyid that s passed n as the parameter Thus, th s method essent a y returns a subtree whose root s the node spec fied by the parameter Because of ts vert ca traversa , t de vers very fast performance for tab es that have a depth-first ndex You can eas y query the emp oyee tab e to return a the descendants (ch d rows, grandch d rows, and so on) of a part cu ar emp oyee us ng the IsDescendantOf method For examp e, the fo ow ng query sts a of Amy’s descendants

Chapter 7  H erarch ca Data and the Re at ona Database   315

DECLARE @AmyNodeId hierarchyid SELECT @AmyNodeId = NodeId FROM Employee WHERE EmployeeId = 46 SELECT FROM WHERE ORDER

NodeId.ToString() AS NodeIdPath, * Employee NodeId.IsDescendantOf(@AmyNodeId) = 1 BY NodeLevel, NodeId

Th s query se ects a rows whose NodeId va ues are descendants of Amy (emp oyee ID 46) Here s the resu t NodeIdPath ---------/1/ /1/1/ /1/2/ /1/1/1/ /1/1/2/

NodeId -----0x58 0x5AC0 0x5B40 0x5AD6 0x5ADA

NodeLevel --------1 2 2 3 3

EmployeeId ---------46 269 389 87 90

EmployeeName -----------Amy Cheryl Wanda Richard Jeff

Title ---------------------Marketing Specialist Marketing Assistant Business Assistant Business Intern Business Intern

(5 row(s) affected)

Not ce that the resu t nc udes Amy n add t on to a of her descendants Amy s returned because she s cons dered to be her “own descendant” at the 0 eve of the subtree Thus, you have se ected a subtree that nc udes a of Amy’s descendants, no matter how many eve s deep they m ght ex st beneath her, w th Amy at the root To se ect only Amy’s mmed ate ch d rows that are just one eve beneath her, you can use the GetAncestor method (You used th s method ear er to create the uspAddEmployee stored procedure ) For examp e SELECT NodeId.ToString() AS NodeIdPath, * FROM Employee WHERE NodeId.GetAncestor(1) = (SELECT NodeId FROM Employee WHERE EmployeeId = 46) ORDER BY NodeLevel, NodeId GO NodeIdPath ---------/1/1/ /1/2/

NodeId -----0x5AC0 0x5B40

NodeLevel --------2 2

EmployeeId ---------269 389

EmployeeName -----------Cheryl Wanda

Title ---------------------Marketing Assistant Business Assistant

(2 row(s) affected)

Th s t me, the resu ts st on y Chery and Wanda, but not Amy nor any of Amy’s deeper ­ escendants That’s because you are request ng just the rows whose one- eve -up ancestor s Amy d (emp oyee ID 46)—that s, just Amy’s mmed ate ch dren

316  Part II  Going Beyond Relational

If you wanted to retr eve a the emp oyees exact y two eve s down from a part cu ar manager, you cou d pass the va ue 2 to the GetAncestor method For examp e, to se ect the emp oyees that report to the emp oyees beneath Dave (that s, to see a the emp oyees two eve s beneath h m), you cou d request rows whose grandparent s Dave (emp oyee ID 6; that s, just Dave’s grandch dren), as shown here SELECT NodeId.ToString() AS NodeIdPath, * FROM Employee WHERE NodeId.GetAncestor(2) = (SELECT NodeId FROM Employee WHERE EmployeeId = 6) ORDER BY NodeLevel, NodeId GO NodeIdPath ---------/1/1/ /1/2/ /2/1/ /3/1/

NodeId -----0x5AC0 0x5B40 0x6AC0 0x7AC0

NodeLevel --------2 2 2 2

EmployeeId ---------269 389 272 291

EmployeeName -----------Cheryl Wanda Mary Keven

Title ---------------------Marketing Assistant Business Assistant Marketing Assistant Marketing Intern

(4 row(s) affected)

Th s query returned a emp oyees that are two eve s be ow Dave The fact that some of them are cous ns and not s b ngs (that s, some of them have d fferent d rect parents) s rre evant Be ng exact y two eve s down beneath Dave qua fies them a for se ect on by th s query And because of ts hor zonta traversa , t de vers very fast performance w th a breadth-first ndex defined on the tab e To find the root node n the h erarchy, s mp y nvoke the GetRoot method on the hierarchyid data type tse f us ng the doub e-co on syntax, as shown n the fo ow ng code Th s s the same method you used to create the first emp oyee at the top of the tree (Dave, as shown n L st ng 7-2) SELECT NodeId.ToString() AS NodeIdPath, * FROM Employee WHERE NodeId = hierarchyid::GetRoot() GO NodeIdPath NodeId NodeLevel EmployeeId EmployeeName Title ---------- ------ --------- ---------- ------------ ---------------------/ 0x 0 6 Dave Marketing Manager (1 row(s) affected)

Reordering Nodes within the Hierarchy Reorgan z ng the nodes w th n the tree structure s a common ma ntenance task when work ng w th h erarch es You m ght need to a ter the tree structure by adjust ng the parent-ch d ­re at onsh ps w th n the h erarchy The hierarchyid type prov des a GetReparentedValue method that makes t easy to hand e th s k nd of ma ntenance You w start by first re ocat ng just a s ng e node w thout



Chapter 7  H erarch ca Data and the Re at ona Database   317

­ sturb ng any other nodes n the h erarchy Then you’ re ocate an ent re subtree (that s, a of a d node’s descendant nodes)

The GetReparentedValue Method You nvoke the GetReparentedValue method on the node you want to move, pass ng n two ­parameters The first spec fies the or g na parent (the source), and the second spec fies the new parent (the target) Wanda former y reported to Amy but s now report ng to J nstead (a ongs de Kev n) You therefore want to move Wanda’s current pos t on as a ch d of Amy to be a ch d of J nstead The fo ow ng code uses the GetReparentedValue method to perform that change DECLARE @EmployeeToMove hierarchyid DECLARE @OldParent hierarchyid DECLARE @NewParent hierarchyid SELECT @EmployeeToMove = NodeId FROM Employee WHERE EmployeeId = 389 -- Wanda SELECT @OldParent = NodeId FROM Employee WHERE EmployeeId = 46 -- Amy SELECT @NewParent = NodeId FROM Employee WHERE EmployeeId = 119 -- Jill -- Wanda now reports to Jill and no longer to Amy UPDATE Employee SET NodeId = @EmployeeToMove.GetReparentedValue(@OldParent, @NewParent) WHERE NodeId = @EmployeeToMove GO

The h erarchy now ooks ke F gure 7-5 Dave (6) /

Amy (46) /1/

Mary (272) /2/1/

Cheryl (269) /1/1/

Richard (87) /1/1/1/

John (271) /2/

Jeff (90) /1/1/2/

Figure 7-5  Reparent ng a s ng e node.

318  Part II  Going Beyond Relational

Jill (119) /3/

Kevin (291) /3/1/

Wanda (389) /3/2/

V ew ng the resu ts of the fo ow ng query confirms that Wanda’s NodeIdPath has changed from /1/2/ (ch d of Amy) to /3/2/ (ch d of J ) SELECT NodeId, NodeId.ToString() AS NodeIdPath, NodeLevel, dbo.fnGetFullDisplayPath(NodeId) AS NodeIdDisplayPath, EmployeeName FROM Employee ORDER BY NodeLevel, NodeId GO NodeId -----0x 0x58 0x68 0x78 0x5AC0 0x6AC0 0x7AC0 0x7B40 0x5AD6 0x5ADA

NodeIdPath ---------/ /1/ /2/ /3/ /1/1/ /2/1/ /3/1/ /3/2/ /1/1/1/ /1/1/2/

NodeLevel --------0 1 1 1 2 2 2 2 3 3

NodeIdDisplayPath ----------------------------Dave Dave > Amy Dave > John Dave > Jill Dave > Amy > Cheryl Dave > John > Mary Dave > Jill > Kevin Dave > Jill > Wanda Dave > Amy > Cheryl > Richard Dave > Amy > Cheryl > Jeff

EmployeeName -----------Dave Amy John Jill Cheryl Mary Kevin Wanda Richard Jeff

(10 row(s) affected)

Chang ng the pos t on of a s ng e node w th the GetReparentedValue method as you just d d does not affect any of that node’s ch dren (s nce on y one row was updated) Th s means that f Wanda had any ch d nodes, they wou d not move a ong w th Wanda as new descendants of J , now wou d they move up one eve n the h erarchy as d rect descendants of Amy to occupy the pos t on ­vacated by Wanda The same wou d be true f Wanda were s mp y de eted rather than moved Former ch d nodes of Wanda wou d therefore end up hav ng no parent at a , but wou d rather have on y a ­grandparent (wh ch n th s case wou d be Amy, Wanda’s former parent) The resu t s a “ho e” n your h erarchy SQL Server w not enforce ntegr ty checks on the h erarchy to catch th s cond t on, so t’s your job to hand e orphaned nodes proper y One way to ensure aga nst orphaned nodes s to ­transp ant an ent re subtree at one t me Let’s earn how to do just that

Transplanting Subtrees You can eas y move a arger number of peop e at one t me by reparent ng an ent re subtree Because you’ be mov ng a of a node’s descendants as a s ng e b ock, no orphaned nodes can resu t from th s operat on In the next update, you w move a of Amy’s subord nates to the r new manager, Kev n Th s s ach eved by reparent ng a nodes w th hierarchyid va ues that beg n w th /1/ (Amy) to /3/1/ (Kev n), except for Amy You use the IsDescendantOf method to return a of Amy’s descendants for the update Reca from our ear er d scuss on of the IsDescendantOf method that Amy s her own



Chapter 7  H erarch ca Data and the Re at ona Database   319

descendant at the 0 eve (that s, she’s the root of her own subtree) You must therefore exc ude her (emp oyee ID 46) from the rows to be updated, as shown n the fo ow ng code DECLARE @OldParent hierarchyid DECLARE @NewParent hierarchyid SELECT @OldParent = NodeId FROM Employee WHERE EmployeeId = 46 -- Amy SELECT @NewParent = NodeId FROM Employee WHERE EmployeeId = 291 -- Kevin UPDATE Employee SET NodeId = NodeId.GetReparentedValue(@OldParent, @NewParent) WHERE NodeId.IsDescendantOf(@OldParent) = 1 AND EmployeeId 46 -- This excludes Amy from the move. GO

Note  The employee ID values are hard-coded for Amy and Kevin in this code. This was done for demonstration purposes. In a real-world implementation, this logic should be in a stored procedure that accepts any two nodes as input parameters and wraps the SELECT and UPDATE statements in a transaction. Runn ng th s update moves a of Amy’s descendants to be descendants of Kev n, as shown n F­ gure 7-6 Not ce that the nodes have moved not on y across the tree, but down one eve as we (s nce Kev n s one eve deeper than Amy) Dave (6) /

Amy (46) /1/

Jill (119) /3/

John (271) /2/

Mary (272) /2/1/

Kevin (291) /3/1/

Wanda (389) /3/2/

Cheryl (269) /3/1/1/

Richard (87) /3/1/1/1/ Figure 7-6  Reparent ng an ent re subtree.

320  Part II  Going Beyond Relational

Jeff (90) /3/1/1/2/

Se ect ng a the emp oyees one more t me shows how SQL Server updated the hierarchyid va ues n NodeId for Chery , R chard, and Jeff to reflect the r new pos t ons beneath Kev n For the first t me n th s scenar o, the h erarchy now runs five eve s deep (count ng from 0 to 4), as shown here SELECT NodeId, NodeId.ToString() AS NodeIdPath, NodeLevel, dbo.fnGetFullDisplayPath(NodeId) AS NodeIdDisplayPath, EmployeeName FROM Employee GO NodeId --------0x 0x58 0x68 0x78 0x6AC0 0x7AC0 0x7B40 0x7AD6 0x7AD6B0 0x7AD6D0

NodeIdPath ---------/ /1/ /2/ /3/ /2/1/ /3/1/ /3/2/ /3/1/1/ /3/1/1/1/ /3/1/1/2/

NodeLevel --------0 1 1 1 2 2 2 3 4 4

NodeIdDisplayPath -------------------------------------Dave Dave > Amy Dave > John Dave > Jill Dave > John > Mary Dave > Jill > Kevin Dave > Jill > Wanda Dave > Jill > Kevin > Cheryl Dave > Jill > Kevin > Cheryl > Richard Dave > Jill > Kevin > Cheryl > Jeff

EmployeeName -----------Dave Amy John Jill Mary Kevin Wanda Cheryl Richard Jeff

(10 row(s) affected)

More hierarchyid Methods Three more hierarchyid methods—Parse, Read, and Write—are prov ded by the hierarchyid data type, a though they are ess often used Parse s essent a y the reverse of ToString It accepts the same s ash-de m ted str ng ­representat on that s returned by ToString and returns the equ va ent compacted varbinary hierarchyid va ue that SQL Server uses nterna y to represent the nodes Th s va ue cou d then be passed to ToString, n wh ch case you’d get the or g na s ash-de m ted str ng back It s the on y other stat c method bes des GetRoot, and so t uses the doub e-co on syntax, as fo ows SELECT hierarchyid::Parse('/2/1/1/') AS NodeId GO NodeId ---------0x6AD6 (1 row(s) affected)

Together, Parse and ToString enab e ser a zat on and deser a zat on to and from hierarchyid b nary va ues and the r str ng representat ons A though you cou d manage the str ng representat ons on

Chapter 7  H erarch ca Data and the Re at ona Database   321

your own and use Parse to convert them for storage as hierarchyid va ues, the hierarchyid va ues are best ass gned by ca ng the GetDescendant method, as we’ve demonstrated throughout our ­examp es The ast two methods, Read and Write, are the on y methods not ava ab e n T-SQL, and can be used on y n NET code aga nst the Microsoft.SqlServer.Types.SqlHierarchyId type These two methods are used to pass hierarchyid va ues nto and out of BinaryReader and BinaryWriter objects They are a so used nterna y by SQL Server as necessary, such as when read ng from or wr t ng to a hierarchyid co umn or for convers ons between varbinary and hierarchyid These methods are prov ded to enab e such nterna operat ons, as we as for ntegrat on w th your own NET code runn ng under SQL CLR They don’t otherw se serve any rea s gn ficant funct on n terms of the actua mp ementat on of a h erarch ca structure

Summary In th s chapter, we showed how to use the hierarchyid data type to cast a tree structure over data stored n ord nary database tab es You can now work w th h erarch ca structures n SQL Server, such as the new F eTab e ntroduced n SQL Server 2012 (you’ earn a about F eTab e the next ­chapter) The hierarchyid data type prov des a set of methods and ndex ng strateg es to enab e effic ent search ng and man pu at on of h erarchy nodes at the database eve Th s enab es you to app y a h erarch ca mode to trad t ona tab e data rather than resort ng to XML, n scenar os where t makes better sense to keep the data stored re at ona y than to redes gn the data for XML storage

322  Part II  Going Beyond Relational

C hapter 8

Native File Streaming —Leonard Lobel

T

he ongo ng pro ferat on of d g ta content n today’s wor d s generat ng unstructured data at a ­bew der ng rate Consequent y, modern app cat ons must cope w th a mu t tude of b nary fi es and formats You m ght have to store and process photos, v deos, documents, spreadsheets, ema ­messages, and other re ated unstructured art facts w th your database records Th s unstructured nformat on—wh ch s rea y just a b nary stream— s common y referred to as BLOB (B nary Large Object) data BLOB data needs to be assoc ated w th the structured data that ves n a re at ona database, and t must stream effic ent y nto and out of your app cat on Trad t ona y, re at ona database management systems store a the r data n tables, and that mp es structure Re at ona database systems were not ­or g na y des gned to hand e the mass ve amount of unstructured data that ex sts today To adapt, M crosoft SQL Server 2008 ntroduced FILESTREAM, an nnovat ve feature that ntegrates the re at ona database eng ne w th the NTFS fi e system In th s chapter, we’ start by exam n ng ­trad t ona BLOB so ut ons, and then exp a n how FILESTREAM prov des mportant advantages over the prev ous BLOB techn ques You w bu d some rea app cat ons and serv ces w th FILESTREAM, and then earn how the new F eTab e feature n M crosoft SQL Server 2012 de vers a og ca fi e ­system— mp emented as a FILESTREAM-enab ed tab e n your database—on top of your BLOBs

Traditional BLOB Strategies Pr or to SQL Server 2008, there were two trad t ona so ut ons for comb n ng structured tab e data w th unstructured BLOB data e ther keep BLOBs n the database (w th a your structured tab e data), or store them outs de the database ( n the fi e system) In the former case, BLOBs are stored r ght ns de the database In the atter case, the database mere y ho ds references (or, poss b y, references are der ved from other va ues n the database) that po nt to ocat ons n the fi e system where the BLOBs actua y ve Each of these strateg es has pros and cons w th respect to storage, manageab ty, performance, and programm ng comp ex ty that we’ d scuss—but ne ther of them are ntr ns ca y nat ve to the core database eng ne

323

BLOBs in the Database You can, of course, s mp y store BLOB data d rect y n the co umns of your database tab es You do th s by dec ar ng a co umn as a varbinary(max) data type, wh ch w a ow t to store a s ng e BLOB up to 2 g gabytes (GB) n s ze

Important  You should no longer use the image data type that was used to store BLOBs prior to Microsoft SQL Server 2005. The varbinary(max) data type should be used instead of image, which has been deprecated and may be removed in a future version of SQL Server. When BLOB data s stored d rect y ns de of tab es th s way, t s very t ght y ntegrated w th the database It’s easy to access BLOB data, because the BLOB s r ght there n the tab e’s co umn Because everyth ng s conta ned n a s ng e database un t, management s a so s mp fied Backup, restore, detach, copy, and attach operat ons on the database nc ude structured and BLOB data together Transact ona cons stency s another mportant benefit that you enjoy w th th s approach Because BLOB data s a phys ca part of the tab es n the database, t s e g b e to part c pate n transact ons If you beg n a transact on, update some data, and then ro back the transact on, any BLOB data that was updated s a so ro ed back Overa , the m xture of structured and BLOB data s hand ed very seam ess y w th th s mode Desp te a these advantages, however, phys ca y stor ng BLOBs n the database s pract ca on y for sma -sca e app cat ons hav ng very few/sma BLOBs Because BLOB content s stored n- ne w th structured data, you can severe y mpa r sca ab ty by b oat ng your fi egroups Query performance w degrade rap d y as a resu t, because the query processor must s ft through much arger amounts of data n your tab es that are consumed w th BLOB content The BLOBs a so don’t stream near y as ­effic ent y when backed by SQL Server varbinary(max) co umns as they wou d f they were he d externa y n the fi e system or on a ded cated BLOB store F na y, there s a so a 2 GB m t on the varbinary(max) data type

Tip  If you have modest BLOB requirements (that is, you are dealing with very few or very small BLOBs) you should store them in the database using the varbinary(max) data type ­instead of using the file system (either directly, or via FILESTREAM). Furthermore, you should consider caching small, frequently accessed BLOBs rather than repeatedly retrieving them from the database or the file system.

BLOBs in the File System To counter these concerns, you can store BLOBs outs de the database and n the fi e system nstead W th th s approach, the structured data n your re at ona tab es mere y conta ns path nformat on to the unstructured BLOB data, wh ch s he d externa y as ord nary fi es n the fi e system ­(a ternat ve y, path nformat on can be der ved from other structured data n a row) App cat ons use th s path nformat on as a reference for ocat ng and track ng the BLOB content assoc ated w th rows n the 324  Part II  Going Beyond Relational

database tab es Because they are phys ca y he d n the fi e system, a BLOB’s s ze s m ted on y by the host fi e system and ava ab e d sk space Th s approach a so de vers much better stream ng ­performance, because the fi e system s a nat ve env ronment that’s h gh y opt m zed for stream ng And because the phys ca database s much sma er w thout the BLOBs ns de t, the query processor can cont nue to de ver opt ma performance A though phys ca y separat ng structured and unstructured content th s way does address the ­ erformance concerns of BLOBs, t a so ra ses new ssues because the data s now separated not on y p phys ca y, but og ca y as we SQL Server has abso ute y no awareness of the assoc at on between data n the database and fi es stored externa y n the fi e system The r coup ng ex sts so e y at the app cat on eve Backup, restore, detach, copy, and attach operat ons on the database fi es therefore nc ude on y structured tab e data w thout any of the BLOB data that’s n the fi e system You won’t get comp ete backups, un ess you back up the fi e system as we , so now t’s another adm n strat ve burden to separate y manage the fi e system App cat on deve opment aga nst th s mode s a so more comp ex because of the extra effort r­ equ red for nk ng between the database and the fi e system It’s up to the deve oper to ­estab sh and ma nta n the references between structured data and externa BLOBs on the r own, and ­accord ng to the r own custom des gn Last, a though perhaps most s gn ficant, there s no un fied transact ona contro across both the database and the fi e system Natura y, ro ng back a database transact on won’t undo changes you’ve made n the fi e system

Introducing FILESTREAM Both of the trad t ona BLOB so ut ons present tough cha enges, so what do you do? W th FILESTREAM, SQL Server offers a way out of th s conundrum F rst, make sure you understand that FILESTREAM s techn ca y not a SQL Server data type Rather, t s mp emented as an attribute that you app y to the varbinary(max) data type—the same data type trad t ona y used to store BLOBs d rect y ns de structured tab es However, mere y app y ng th s attr bute te s SQL Server to store the BLOB n the fi e system rather than the tab e’s structured fi egroup W th the FILESTREAM attr bute app ed, you cont nue to treat the varbinary(max) co umn as though ts contents were stored n- ne w th your structured tab e data Under the covers, however, SQL Server stores and ma nta ns the data n the server’s oca NTFS fi e system, separate y from the structured content of the database that rema ns n the norma fi egroups W th FILESTREAM, structured and unstructured data are logically connected wh e physically separated The unstructured data s configured as a spec a fi egroup n the database, so t’s actua y cons dered part of the database— t s ava ab e n a og ca database operat ons, nc ud ng quer es, transact ons, and backup/restore On d sk, however, the BLOBs are stored as nd v dua phys ca fi es n the NTFS fi e system that are created and managed automat ca y beh nd the scenes SQL Server estab shes and ma nta ns the nk references between the structured fi e groups and the fi e ­system It knows about the unstructured BLOB data n the fi e system and cons ders the fi es ho d ng BLOB data to be an ntegra part of the overa database But the unstructured data does not mpede query ­performance because t s not phys ca y stored n- ne w th tab e data It’s stored n the fi e

Chapter 8  Nat ve F e Stream ng   325

s­ ystem, wh ch s a nat ve BLOB env ronment (and where t ostens b y be ongs) Log ca y, however, the ­database encompasses both the re at ona tab es and the nd v dua BLOB fi es n the fi e system You therefore cont nue to treat BLOB data as though you were stor ng t n- ne, from both a deve opment and an adm n strat ve perspect ve

Tip  Backing up the database includes all the BLOB data from the file system in the backup automatically. However, because the BLOB data is contained in its own database filegroup, you can easily exclude it from backups if desired or as needed. The end resu t s that SQL Server uses the appropr ate mechan sm for structured and unstructured data—stor ng re at ona (structured) data n tab es and BLOB (unstructured) data n ord nary fi es—so t can de ver the best poss b e performance a around Because t does th s comp ete y transparent y, you enjoy ntegrated management benefits over the database You a so enjoy s mp fied app cat on deve opment as you are no onger burdened w th the add t ona comp ex t es of manua y assoc at ng the database w th the fi e system and keep ng the two n sync Last, by everag ng the transact ona capab t es of the NTFS fi e system, BLOB updates part c pate seam ess y w th database transact ons If you’re start ng to get exc ted by a th s, that’s the dea! You’re now ready to d ve n to some rea code that puts FILESTREAM to work for you

Enabling FILESTREAM L ke many other features, FILESTREAM s d sab ed by defau t n SQL Server 2012, and you must first enab e t before the feature can be used Enab ng FILESTREAM s s ght y more nvo ved than enab ng other SQL Server features because t requ res two d st nct steps F rst, the feature needs to be enab ed for the mach ne, and then t needs to be enab ed for the server nstance These two ndependent FILESTREAM configurat on ayers are by des gn, and they draw a separat on of secur ty concerns ­between the ro e of W ndows adm n strator and database adm n strator

Enabling FILESTREAM for the Machine The first step s to enab e FILESTREAM for the mach ne by sett ng an access eve Th s step can ­actua y be performed at the t me that SQL Server s n t a y nsta ed by choos ng a FILESTREAM ­access eve dur ng setup To enab e FILESTREAM for the mach ne after SQL Server has been nsta ed, use the SQL Server Configurat on Manager to set the access eve (Th s too can be aunched from the Configurat on Too s fo der of the M crosoft SQL Server 2012 program group on the Start menu ) The SQL Server Configurat on Manager opens w th a treev ew on the eft In the treev ew, c ck SQL Server Serv ces to d sp ay the st of ava ab e serv ces n the ma n pane R ght-c ck the SQL Server nstance that you want to enab e FILESTREAM for, choose Propert es, and n the Propert es d a og box, se ect the FILESTREAM tab The three check boxes on the FILESTREAM tab a ow you to se ect the var ous eve s of FILESTREAM access F gure 8-1 shows the Propert es d a og box w th a three check boxes se ected 326  Part II  Going Beyond Relational

Figure 8-1  Enab ng F LESTREAM for the mach ne to support fi e /O stream ng access by remote c ents.

When a three check boxes are c eared, FILESTREAM s comp ete y d sab ed Se ect ng the first check box enab es FILESTREAM, but on y for Transact-SQL (T-SQL) access Th s prov des for ­utter transparency, where SQL Server w store BLOBs conta ned ns de varbinary(max) FILESTREAM co umns n the fi e system beh nd the scenes just as we’ve d scussed But t won’t a ow you to take advantage of d rect fi e stream ng between the database and your c ent app cat ons us ng more advanced FILESTREAM features, such as SqlFileStream and F eTab e that you’ be earn ng about soon The rea power of FILESTREAM comes nto p ay when you enab e d rect fi e I/O stream ng, wh ch de vers the best poss b e BLOB performance w th SQL Server You enab e d rect fi e I/O stream ng access by se ect ng the second check box Streamed fi e access a so creates a W ndows share name that s used to construct og ca Un versa Nam ng Convent on (UNC) paths to BLOB data dur ng FILESTREAM access, as you’ see further on when you use SqlFileStream and F eTab e The share name s spec fied n a text box after the second check box and s set by defau t to the same name as the server nstance (MSSQLSERVER, n F gure 8-1) In most cases, c ent app cat ons w not be runn ng on the same mach ne as SQL Server, and so you w usua y a so need to se ect the th rd check box to enab e FILESTREAM for remote c ent fi e I/O stream ng access w th SqlFileStream In add t on, the new F eTab e feature n SQL Server 2012 requ res that you check th s opt on n order to expose ts data as a fi e system through the W ndows share created by FILESTREAM (you’ exam ne F eTab e after you fin sh earn ng about FILESTREAM) So pract ca y, you must check a three checkboxes to get the most out of FILESTREAM (and to support a the exerc ses n th s chapter) An except on to th s genera pract ce m ght be when us ng M crosoft SQL Server 2012 Express ed t on as a oca data store for a c ent app cat on w th everyth ng runn ng

Chapter 8  Nat ve F e Stream ng   327

on the same mach ne In th s case, you cou d use the more secure sett ng and eave the th rd check box c eared Do ng so wou d enab e SqlFileStream and F eTab e access for the oca c ent app cat on but deny such access to remote c ents If you are us ng SQL Server Configurat on Manager to change th s sett ng after SQL Server has a ready been nsta ed, you must remember to restart the SQL Server nstance for the sett ng change to take effect

More Info  There is no T-SQL equivalent script that can set the FILESTREAM access level for the machine. However, Microsoft posts a VBScript file available over the Internet that allows you to enable FILESTREAM from the command line as an alternative to using SQL Server Configuration Manager. At press time, the download page for this script is http://www. codeplex.com/SQLSrvEngine/Wiki/View.aspx?title FileStreamEnable&referringTitle Home.

Enabling FILESTREAM for the Server Instance The second step s to enab e FILESTREAM for the server nstance The concept here s s m ar to what we just descr bed Vary ng eve s of access correspond to the checkboxes n F gure 8-1 Natura y, the access eve defined for the server nstance must be supported by the access eve defined for the mach ne Typ ca y, therefore, the mach ne and server nstance access eve s shou d be set to match one another FILESTREAM can be enab ed for the server nstance w th a s mp e ca to the sp configure system stored procedure, as fo ows EXEC sp_configure filestream_access_level, n RECONFIGURE

In the preced ng code, rep ace n w th a number from 0 to 2 to set the access eve A va ue of 0 ­ sab es the FILESTREAM feature comp ete y Sett ng the access eve to 1 enab es FILESTREAM for d T-SQL access on y, and sett ng t to 2 enab es FILESTREAM for fu access (wh ch nc udes oca or remote fi e I/O stream ng access as enab ed for the mach ne n the first step) To support the samp e SqlFileStream NET c ent app cat on that you’ be bu d ng ater n th s chapter, you must se ect eve 2 (fu access) Th s corresponds to the th rd checkbox n F gure 8-1 You can a so set the FILESTREAM access eve for the server nstance n SQL Server ­Management Stud o (SSMS) from the Advanced Server Propert es d a og box R ght-c ck any server nstance n ­Object Exp orer, choose Propert es, and then se ect the Advanced page The var ous eve s are ­ava ab e as cho ces n the F estream Access Leve drop-down st, as shown n F gure 8-2

Note  You can use either SQL Server Data Tools (SSDT) inside Visual Studio (covered in Chapter 1) or SSMS to set the access level, but only SSMS provides the ability to set it in the graphical user interface (GUI). Using SSDT, you can view the setting’s property from the SQL Server Object Explorer, but you can only change it by running the EXEC statement in a query window.

328  Part II  Going Beyond Relational

Figure 8-2  Se ect ng the F LESTREAM configurat on eve n SQL Server Management Stud o.

Creating a FILESTREAM-Enabled Database Once FILESTREAM s enab ed for both the mach ne and the server nstance, any database ­runn ng on the server nstance can support unstructured data by defin ng a fi egroup w th the spec a ­FILEGROUP…CONTAINS FILESTREAM c ause of the CREATE DATABASE statement For examp e, the statement n L st ng 8-1 creates a PhotoLibrary database that can store p ctures us ng FILESTREAM Before you can run th s code, you need to create an empty fo der named C \Demo\PhotoL brary Listing 8-1  Creat ng a F LESTREAM enab ed database w th FILEGROUP…CONTAINS FILESTREAM.

CREATE DATABASE PhotoLibrary ON PRIMARY (NAME = PhotoLibrary_data, FILENAME = 'C:\Demo\PhotoLibrary\PhotoLibrary_data.mdf'), FILEGROUP FileStreamGroup1 CONTAINS FILESTREAM (NAME = PhotoLibrary_group2, FILENAME = 'C:\Demo\PhotoLibrary\Photos') LOG ON (NAME = PhotoLibrary_log, FILENAME = 'C:\Demo\PhotoLibrary\PhotoLibrary_log.ldf')

The FILEGROUP…CONTAINS FILESTREAM c ause n th s otherw se ord nary CREATE DATABASE statement enab es the FILESTREAM feature for the PhotoLibrary database As when creat ng any database, the d rectory (or d rector es) spec fied for the pr mary and og ­ egroups must ex st at the t me the database s created In L st ng 8-1, the CREATE DATABASE fi statement w fa f the C \Demo\PhotoL brary d rectory spec fied by FILENAME n ts ON PRIMARY and LOG ON c auses does not ex st For the new FILESTREAM group spec fied by the FILEGROUP… CONTAINS FILESTREAM c ause, a FILENAME keyword s spec fied po nt ng to a d rectory (not a fi e) that must not ex st at the t me that the database s created (a though the path ead ng up to the fina d rectory must ex st), or the CREATE DATABASE statement w fa as we SQL Server takes contro of creat ng and manag ng th s d rectory—ca ed the FILESTREAM data conta ner—much as

Chapter 8  Nat ve F e Stream ng   329

t does for creat ng and manag ng the mdf and df fi es n the other fi egroups In L st ng 8-1, SQL Server ­automat ca y creates the C \Demo\PhotoL brary\Photos fo der when the CREATE DATABASE ­statement s executed, and t w use that fo der for stor ng a BLOB data (photos, n th s examp e) n the PhotoLibrary database Execute the code n L st ng 8-1 to create the database SQL Server creates the usua mdf and df fi es for you, and a so creates the Photos subd rectory for the FILESTREAM group, as shown n F gure 8-3

Figure 8-3  F LESTREAM storage n the fi e system.

Beh nd the scenes, SQL Server w store a your p ctures as fi es n the Photos subd rectory, and w transparent y assoc ate those fi es and the re at ona tab es that they og ca y be ong to n co umns defined as varbinary(max) FILESTREAM Un ess you exp c t y exc ude the FileStreamGroup1 fi egroup from a backup or restore command, a your p cture fi es n the Photos subd rectory w be nc uded w th the re at ona database n the backup or restore operat on

Note  You can create multiple FILESTREAM filegroups, with each one pointing to a ­different file system location. Doing so helps to partition BLOB data when you need to scale up, ­because you can designate specific filegroups for each varbinary(max) FILESTREAM column you define in your tables. See Microsoft Books Online for the appropriate syntax: http://msdn.microsoft.com/en-us/library/ms176061.aspx.

Creating a Table with FILESTREAM Columns You’re now ready to create a new PhotoAlbum tab e SQL Server requ res that any tab e u ­ s ng FILESTREAM storage has a uniqueidentifier co umn that s not nu ab e and spec fies the ­ROWGUIDCOL attr bute You must a so create a un que constra nt on th s co umn On y one ­ROWGUIDCOL co umn 330  Part II  Going Beyond Relational

can be defined n any g ven tab e, a though you can then dec are any number of varbinary(max) FILESTREAM co umns n the tab e that you want for stor ng BLOB data The statement n L st ng 8-2 creates the PhotoAlbum tab e w th a Photo co umn dec ared as varbinary(max) FILESTREAM Listing 8-2  Creat ng a F LESTREAM enab ed tab e.

USE PhotoLibrary GO CREATE TABLE PhotoAlbum( PhotoId int PRIMARY KEY, RowId uniqueidentifier ROWGUIDCOL NOT NULL UNIQUE DEFAULT NEWSEQUENTIALID(), Description varchar(max), Photo varbinary(max) FILESTREAM DEFAULT(0x))

W th th s statement, you sat sfy the FILESTREAM requ rement for the ROWGUIDCOL co umn, yet you won’t actua y have to do anyth ng to ma nta n that co umn By dec ar ng the RowId co umn w th ts DEFAULT va ue set to ca the NEWSEQUENTIALID funct on, you can just gnore th s co umn when nsert ng rows—s mp y not prov d ng va ues for t w cause SQL Server to automat ca y generate the next ava ab e g oba y un que dent fier (GUID) that t needs to support FILESTREAM on the tab e The co umn s set to not accept NULL va ues and s defined w th the requ red un que constra nt You have a so dec ared an nteger PhotoId co umn for the tab e’s pr mary key va ue You’ use the PhotoId co umn to dent fy nd v dua photos n the a bum, and SQL Server w use the RowId co umn to track and cross-reference photos n the fi e system w th rows n the PhotoAlbum tab e The Photo co umn ho ds the actua BLOB tse f, be ng defined as a varbinary(max) data type w th the FILESTREAM attr bute app ed Th s means that t gets treated ke a regu ar varbinary(max) co umn, but you know that ts BLOB s rea y be ng stored under the covers n the fi e system by SQL Server The Photo co umn s a so defined w th a defau t va ue of 0x, wh ch represents a zero- ength b nary stream Th s w come nto p ay ater when you start programm ng w th SqlFileStream We’re not there yet, so you can just gnore the defau t ass gnment for now

Storing and Retrieving FILESTREAM Data To start, you’ use p a n T-SQL to cast s mp e str ng data nto and out of the varbinary(max) data type of the Photo co umn Th s s certa n y contr ved, but start ng sma ke th s s the best way to earn FILESTREAM You are go ng to mon tor and observe effects on the NTFS fi e system as SQL Server ut zes t for BLOB storage Beg n w th the fo ow ng INSERT statement that adds your first row to the PhotoAlbum tab e INSERT INTO PhotoAlbum(PhotoId, Description, Photo) VALUES(1, 'Text pic', CAST('BLOB' As varbinary(max)))

Th s INSERT statement reads no d fferent y than t wou d f you were us ng a regu ar varbinary(max) co umn for the Photo co umn w thout the FILESTREAM attr bute It appears to store

Chapter 8  Nat ve F e Stream ng   331

the unstructured Photo co umn data n ne w th the rest of the re at ona co umns, and t certa n y ­appears the same way when return ng the data back w th a SELECT query, as shown here SELECT *, CAST(Photo AS varchar) AS PhotoText FROM PhotoAlbum GO PhotoId RowId Description Photo PhotoText ------- ------------------------------------ ----------- ---------- --------1 FC7D28BC-E8C6-E011-9849-080027565B78 Text pic 0x424C4F42 BLOB (1 row(s) affected)

However, f you peek ns de the FILESTREAM data conta ner, you can ver fy that SQL Server s ­actua y stor ng the Photo co umn outs de the database n the fi e system Because SQL Server ­obfuscates the fi e system that t’s manag ng for you beh nd the scenes, you can’t understand the manner n wh ch fi es and fo ders are named, organ zed, and referenced back to the re at ona ­database But just by dr ng down and prob ng the subfo ders beneath the Photos d rectory, you w d scover that there s n fact a new fi e stored n the fi e system (W ndows w prompt for perm ss on before open ng the Photos d rectory ) Th s fi e was created as a resu t of the INSERT statement V ew t by r ght-c ck ng the fi e name and choos ng Open, as shown n F gure 8-4

Figure 8-4  Exp or ng the F LESTREAM fi e system (note that the actua fo der and fi e names w

ones shown n th s figure).

not match the

If you se ect Notepad to open the fi e, you w see c ear proof that the unstructured content of the Photo co umn s stored outs de the database and n the fi e system In th s examp e, the text BLOB that was nserted nto the Photo co umn s stored n the fi e that you’ve opened n Notepad, as shown n F gure 8-5

332  Part II  Going Beyond Relational

Figure 8-5  Exam n ng unstructured F LESTREAM content n Notepad.

Th s c ear y shows how FILESTREAM data s og ca y connected to—but phys ca y separated from—the database Because the unstructured data s stored ent re y n the fi e system, you can eas y a ter ts content by d rect y updat ng the fi e n Notepad w thout nvo v ng the database To prove the po nt once aga n, et’s change the text n the fi e from BLOB to Cool, and save the changes back to the fi e system, as d sp ayed n F gure 8-6

Figure 8-6  Chang ng F LESTREAM content d rect y n the fi e system.

The a tered FILESTREAM data s reflected n the same SELECT statement you ran ear er, as shown here SELECT *, CAST(Photo AS varchar) AS PhotoText FROM PhotoAlbum GO PhotoId RowId Description Photo PhotoText ------- ------------------------------------ ----------- ---------- --------1 FC7D28BC-E8C6-E011-9849-080027565B78 Text pic 0x436F6F6C Cool (1 row(s) affected)

You are perform ng th s exerc se to ver fy that SQL Server s us ng the fi e system to store varbinary(max) FILESTREAM data However, we must stress that the purpose here s pure y ­demonstrat ve Ord nar y, you must never tamper d rect y w th fi es n the FILESTREAM data conta ner th s way FILESTREAM prov des a total abstract on over the varbinary(max) data type, and you need to cons der the phys ca fi e system as part of the database (wh ch t s, by v rtue of the FILESTREAM group assoc ated w th t)— t gets managed by SQL Server exc us ve y Later n the chapter, you’ earn how F eTab e (new n SQL Server 2012) adds yet another ayer of abstract on by expos ng a network share over FILESTREAM-based BLOBs that can be treated and accessed just ke an ord nary fi e system

Note  Normal SQL Server column-level security permissions apply to varbinary(max) FILESTREAM columns.



Chapter 8  Nat ve F e Stream ng   333

In- n ng BLOB data n T-SQL s ted ous, but t s feas b e and can be done when you are work ng w th re at ve y sma byte streams Run the fo ow ng INSERT statement to add a second row, th s t me w th a rea mage n the Photo co umn INSERT INTO PhotoAlbum(PhotoId, Description, Photo) VALUES(2, 'Document icon', 0x4749463839610C000E00B30000FFFFFFC6DEC6C0C0C0000080000000D3121200000 000000000000000000000000000000000000000000000000000000021F90401000002002C000000000C000E0000042C9 0C8398525206B202F1820C80584806D1975A29AF48530870D2CEDC2B1CBB6332EDE35D9CB27DCA554484204003B)

Now rev s t the fi e system and you’ find another fi e was just created Th s new fi e conta ns the mage represented by the byte stream n the INSERT statement you just executed R ght-c ck the fi e and choose Open, but th s t me se ect Pa nt to open the fi e w th F gure 8-7 shows the mage open and magn fied n Pa nt

Figure 8-7  Exam n ng unstructured F LESTREAM content n Pa nt.

More ke y, the BLOB source w be an externa fi e In th s case, you can use OPENROWSET w th ts BULK and SINGLE BLOB opt ons to consume the fi e (wh ch can be ocated on a remote fi e share) nto SQL Server as a varbinary(max) type For examp e, f the BLOB s access b e as a fi e named \\public\ shared\doc.ico, then the same resu t can be ach eved w th the fo ow ng INSERT statement INSERT INTO PhotoAlbum(PhotoId, Description, Photo) VALUES(2, 'Document icon', (SELECT BulkColumn FROM OPENROWSET(BULK '\\public\shared\doc.ico', SINGLE_BLOB) AS x))

Deleting FILESTREAM Data When a row s de eted from a FILESTREAM-enab ed tab e, the row s removed from the tab e and the fi es assoc ated w th the row’s varbinary(max) FILESTREAM co umns are removed from the fi e system The fi es are de eted by the FILESTREAM garbage co ector runn ng on a separate background thread, so you m ght not ce that the phys ca fi e s not removed from the FILESTREAM data conta ner ­ mmed ate y 334  Part II  Going Beyond Relational

SQL Server tr ggers the FILESTREAM garbage co ector thread when a CHECKPOINT occurs, wh ch, for databases that are not h gh y transact ona , m ght take a ong t me to occur If you want to tr gger the garbage co ector thread yourse f, you can ssue an exp c t CHECKPOINT statement to the server

Direct Streaming in .NET with SqlFileStream You just saw how to embed and extract b nary streams of data n T-SQL A s m ar T-SQL approach can be used w th ADO NET n your NET app cat ons to embed and extract byte arrays aga nst varbinary(max) FILESTREAM co umns It’s easy, and t works, but t s not the most effic ent way to transfer BLOBs nto and out of the database Beh nd the scenes, SQL Server must use ts own memory to stream BLOBs n and out, and t exposes BLOBs to c ent app cat ons as varbinary(max) data types that are not opt m zed for stream ng The proper (and fastest) way to get data nto and out of FILESTREAM co umns s to bu d a NET ­c ent app cat on (wr tten n C# or VB NET, for examp e), and use the spec a SqlFileStream c ass ( n NET 3 5 SP1 and h gher) to access the BLOBs SqlFileStream s a NET wrapper around ­OpenSqlFilestream, a funct on prov ded by the SQL Server nat ve c ent app cat on programm ng nterface (API) that g ves your app cat on safe, d rect access to varbinary(max) FILESTREAM co umn data stored n the fi e system W th th s approach, SQL Server does not use any of ts own memory for the BLOB transfer, wh ch reduces demand on server resources Instead, your app cat on, us ng ts own resources, streams d rect y aga nst the fi e system w thout the overhead of the varbinary(max) data type abstract on The resu t s ghtn ng fast BLOB performance—a though there’s a pr ce to be pa d, of course D rect stream ng w th SqlFileStream s re at ve y more comp ex than us ng T-SQL, because you need to nteract w th the FILESTREAM data store Overa , t’s not too onerous though, and as you’re about to see, t’s not d fficu t to understand and mp ement the appropr ate cod ng pattern

Understanding SqlFileStream W th SqlFileStream, NET app cat ons can use the standard FileStream c ass n the System.IO namespace to de ver h gh-performance stream ng of BLOB data aga nst the SQL Server-managed fi e system In the next sect on, you’ earn how to use SqlFileStream to store and retr eve p ctures n the Photo co umn But before we d g nto the ntr cac es of cod ng, et’s first get a h gh- eve ­understand ng of how SqlFileStream works

Note  You can also create native-code FILESTREAM applications (written in in C++, for ­example) by calling OpenSqlFilestream directly. The handle returned by this function can then use either the ReadFile and WriteFile Microsoft Win32 API functions. Our samples are .NET (managed-code) applications written in C#, but the process we describe here is ­fundamentally the same for native-code applications. You beg n by start ng a database transact on When work ng w th SqlFileStream, you a ways work w th transact ons (even for read access) There s no way to avo d them because FILESTREAM, by

Chapter 8  Nat ve F e Stream ng   335

des gn, coord nates transact ona ntegr ty across structured and unstructured data access n the database and the NTFS fi e system After start ng the transact on, you perform standard T-SQL operat ons to query or mod fy data But you don’t actua y nc ude varbinary(max) FILESTREAM co umns (BLOBs) n your T-SQL statements (ne ther for read ng nor for wr t ng) Instead, you ask SQL Server to g ve you two key p eces of nformat on that w enab e you to access the BLOBs much more effic ent y than us ng T-SQL—spec fica y, v a d rect fi e stream ng us ng SqlFileStream F rst, you need a og ca UNC path to the fi e ho d ng the BLOB on the server, wh ch you get by ca ng the PathName method on the des red varbinary(max) FILESTREAM va ue nstance Second, you need the fi e system transact on context, wh ch you get by ca ng the GET FILESTREAM TRANSACTION CONTEXT funct on (wh ch returns NULL f a transact on has not yet been estab shed) Ne ther of these tems mean anyth ng to your code; you just need to pass them on, a ong w th your des red access (read or wr te), to the SqlFileStream constructor The constructed SqlFileStream object nher ts from System.IO.Stream, so you can then read or wr te aga nst t us ng the same NET cod ng patterns you use to access ord nary streams Th s prov des you w th a stream ng “tunne ” between your app cat on and the SQL Server nterna y ­managed fi e system for each BLOB When you start the database transact on, SQL Server nterna y n t ates an NTFS fi e system ­transact on over the BLOB data and assoc ates t w th the current database transact on SQL Server then further ensures that both transact ons w subsequent y e ther comm t or ro back together On y when you comm t the database transact on does SQL Server nterna y comm t the NTFS fi e system transact on Th s permanent y saves changes to both the structured fi egroups of the database (from the T-SQL operat ons) and the fi es n the fi e system (from the stream ng I/O operat ons) at the same t me S m ar y, ro ng back the transact on undoes changes to both the database and fi e system s mu taneous y

Note The UNC reference returned by the PathName method is not a real path to the physical file system on the server. Rather, PathName returns a fabricated path used by SqlFileStream to enable direct streaming between the file system and client a ­ pplications. This UNC path always begins with the share name specified when FILESTREAM was ­enabled for the machine (Figure 8-1) and contains the GUID value in the corresponding row’s uniqueidentifier ROWGUIDCOL column. The file system is secured on the server no ­differently than the data and transaction filegroups (.mdf and .ldf files) are secured. Users should never be granted direct access to the file system on the server (although you can grant them indirect access via a FileTable, as we’ll discuss later on). Cont nu ng w th the photo brary examp e, you’ now create a W ndows Forms app cat on n C# that conta ns a the key e ements that br ng a FILESTREAM app cat on together—noth ng more, noth ng ess Your app cat on w a ow users to create new photos n the database by stream ng BLOBs nto the Photo co umn Users can a so se ect photos, wh ch w stream BLOBs back out from the Photo co umn and nto a PictureBox contro for d sp ay

336  Part II  Going Beyond Relational

Building the Windows Forms Client You’ beg n w th the user nterface (UI), wh ch s very s mp e Start M crosoft V sua Stud o 2010 and create a new C# W ndows Forms app cat on Des gn a form w th two separate group boxes one at the top of the form for nsert ng photos and another beneath t for retr ev ng photos Prov de abe s and text boxes for enter ng a photo ID, fi e name, and descr pt on n the top group box, a ong w th a nk abe to nvoke a save operat on In the bottom group box, prov de a text box and abe for ­enter ng a photo ID and a nk abe to nvoke a oad operat on Inc ude a abe to d sp ay the descr pt on returned from the database and a p cture box to d sp ay the photo BLOB returned v a FILESTREAM After perform ng some aesthet c a gnment and formatt ng, your form shou d appear someth ng ke the one shown n F gure 8-8

Figure 8-8  A s mp e F LESTREAM W ndows U form.

You’ wr te on y a very sma amount of code beh nd th s W ndows form, and mp ement the FILESTREAM og c n a separate data access c ass Add the code beh nd the c ck events for the form’s Save and Load nk buttons, as shown n L st ng 8-3 Listing 8-3  U ca s nto F LESTREAM data access c ass for sav ng and oad ng mage fi es.

private void lnkSave_LinkClicked(object sender, LinkLabelLinkClickedEventArgs e) { int photoId = int.Parse(this.txtSavePhotoId.Text); string desc = this.txtDescription.Text; string filename = this.txtFilename.Text; PhotoData.InsertPhoto(photoId, desc, filename); }



Chapter 8  Nat ve F e Stream ng   337

private void lnkLoad_LinkClicked(object sender, LinkLabelLinkClickedEventArgs e) { int photoId = int.Parse(this.txtLoadPhotoId.Text); string desc; Image photo = PhotoData.SelectPhoto(photoId, out desc); this.lblDescription.Text = desc; this.picImage.Image = photo; }

When the user supp es new photo nformat on and c cks Save, the code retr eves the photo ID, ­descr pt on, and fi e name from the three text boxes and passes them to the InsertPhoto method of the PhotoData c ass When the user spec fies a photo ID and c cks Load, the code ca s the SelectPhoto method of the PhotoData c ass to retr eve the requested mage (w th ts textua descr pt on) for d sp ay

Programming SqlFileStream Data Access A the mag c happens ns de the PhotoData c ass, wh ch s shown n L st ng 8-4

Note  As with all code in this book, the full FILESTREAM demo code in this chapter is ­available on the book’s companion website. Listing 8-4  mp ement ng a F LESTREAM data access managed c ent c ass.

using using using using using using using

System; System.Data; System.Data.SqlClient; System.Data.SqlTypes; System.Drawing; System.IO; System.Transactions;

namespace PhotoLibraryApp { public class PhotoData { private const string ConnStr = "Data Source=.;Integrated Security=True;Initial Catalog=PhotoLibrary;"; #region "Insert Photo" public static void InsertPhoto(int photoId, string desc, string filename) { const string InsertTSql = @" INSERT INTO PhotoAlbum(PhotoId, Description) VALUES(@PhotoId, @Description);

338  Part II  Going Beyond Relational

SELECT Photo.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT() FROM PhotoAlbum WHERE PhotoId = @PhotoId"; string serverPath; byte[] serverTxn; using (TransactionScope ts = new TransactionScope()) { using (SqlConnection conn = new SqlConnection(ConnStr)) { conn.Open(); using (SqlCommand cmd = new SqlCommand(InsertTSql, conn)) { cmd.Parameters.Add("@PhotoId", SqlDbType.Int).Value = photoId; cmd.Parameters.Add("@Description", SqlDbType.VarChar).Value = desc; using (SqlDataReader rdr = cmd.ExecuteReader(CommandBehavior. SingleRow)) { rdr.Read(); serverPath = rdr.GetSqlString(0).Value; serverTxn = rdr.GetSqlBinary(1).Value; rdr.Close(); } } SavePhotoFile(filename, serverPath, serverTxn); } ts.Complete(); } } private static void SavePhotoFile (string clientPath, string serverPath, byte[] serverTxn) { const int BlockSize = 1024 * 512; using (FileStream source = new FileStream(clientPath, FileMode.Open, FileAccess.Read)) { using (SqlFileStream dest = new SqlFileStream(serverPath, serverTxn, FileAccess.Write)) { byte[] buffer = new byte[BlockSize]; int bytesRead; while ((bytesRead = source.Read(buffer, 0, buffer.Length)) > 0) { dest.Write(buffer, 0, bytesRead); dest.Flush(); } dest.Close(); } source.Close();



Chapter 8  Nat ve F e Stream ng   339

} } #endregion #region "Select Photo" public static Image SelectPhoto(int photoId, out string desc) { const string SelectTSql = @" SELECT Description, Photo.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT() FROM PhotoAlbum WHERE PhotoId = @PhotoId"; Image photo; string serverPath; byte[] serverTxn; using (TransactionScope ts = new TransactionScope()) { using (SqlConnection conn = new SqlConnection(ConnStr)) { conn.Open(); using (SqlCommand cmd = new SqlCommand(SelectTSql, conn)) { cmd.Parameters.Add("@PhotoId", SqlDbType.Int).Value = photoId; using (SqlDataReader rdr = cmd.ExecuteReader(CommandBehavior. SingleRow)) { rdr.Read(); desc = rdr.GetSqlString(0).Value; serverPath = rdr.GetSqlString(1).Value; serverTxn = rdr.GetSqlBinary(2).Value; rdr.Close(); } } photo = LoadPhotoImage(serverPath, serverTxn); } ts.Complete(); } return photo; } private static Image LoadPhotoImage(string filePath, byte[] txnToken) { Image photo; using (SqlFileStream sfs = new SqlFileStream(filePath, txnToken, FileAccess.Read))

340  Part II  Going Beyond Relational

{ photo = Image.FromStream(sfs); sfs.Close(); } return photo; } #endregion } }

Let’s exp a n th s code n deta We’ start at the top w th some requ red namespace nc us ons The two using statements to take not ce of are System.Data.SqlTypes and System.Transactions The System.Data.SqlTypes namespace defines the SqlFileStream c ass that you’ be us ng to stream BLOBs No spec a assemb y reference s requ red to use th s c ass, because t s prov ded by the System.Data. dll assemb y that your project s a ready referenc ng (V sua Stud o set th s reference automat ca y when t created your project) The System.Transactions namespace defines the TransactionScope c ass that ets you code mp c t transact ons aga nst the database Th s c ass s prov ded by the System. Transactions.dll assemb y, wh ch s not referenced automat ca y You’ need to add a reference to t now, or the code w not comp e R ght-c ck the project n So ut on Exp orer and choose Add Reference In the Add Reference d a og, c ck the NET tab, and scro to find the System.Transactions component Then doub e-c ck t to add the reference

More Info  Chapter 10 provides detailed coverage of the conventional ADO.NET objects for SQL Server (SqlConnection, SqlCommand, SqlDataReader, and so on). There, we discuss explicit transactions (using the ADO.NET SqlTransaction object) versus implicit transactions (using TransactionScope). Although the former will also work with SqlFileStream, Chapter 10 explains why implicit transactions are preferred. A connect on str ng s defined at the top of the code as a hard-coded constant named ConnStr Th s s just for demonstrat on purposes; a rea -wor d app cat on wou d store the connect on str ng e sewhere (such as n a configurat on fi e, poss b y encrypted), but we’re keep ng the examp e s mp e

Important  We must reiterate that this code takes a minimalist approach for proof-ofconcept purposes only. It is not representative of best coding practices, and has no error handling. Although the using constructs in this code ensure that all objects that allocate unmanaged resources (such as database connections and file handles) are disposed of properly even if an exception occurs, your production applications must implement a more robust strategy that includes the use of try/catch/finally blocks with a mechanism for error logging and recovery.



Chapter 8  Nat ve F e Stream ng   341

The first method defined n the c ass s InsertPhoto, wh ch accepts a new photo nteger ID, str ng descr pt on, and fu path to an mage fi e to be saved to the database, as shown here public static void InsertPhoto(int photoId, string desc, string filename) { const string InsertTSql = @" INSERT INTO PhotoAlbum(PhotoId, Description) VALUES(@PhotoId, @Description); SELECT Photo.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT() FROM PhotoAlbum WHERE PhotoId = @PhotoId"; string serverPath; byte[] serverTxn; using (TransactionScope ts = new TransactionScope()) { using (SqlConnection conn = new SqlConnection(ConnStr)) { conn.Open(); using (SqlCommand cmd = new SqlCommand(InsertTSql, conn)) { cmd.Parameters.Add("@PhotoId", SqlDbType.Int).Value = photoId; cmd.Parameters.Add("@Description", SqlDbType.VarChar).Value = desc; using (SqlDataReader rdr = cmd.ExecuteReader(CommandBehavior.SingleRow)) { rdr.Read(); serverPath = rdr.GetSqlString(0).Value; serverTxn = rdr.GetSqlBinary(1).Value; rdr.Close(); } } SavePhotoFile(filename, serverPath, serverTxn); } ts.Complete(); } }

Not ce that the InsertTSql str ng constant defined at the top of the method spec fies an INSERT s­ tatement that nc udes the PhotoId and Description co umns, but not the actua Photo BLOB co umn ­Instead, the INSERT statement s fo owed mmed ate y by a SELECT statement that retr eves two p eces of nformat on you’ use to stream the BLOB nto the Photo co umn much more effic ent y than us ng ­ord nary T-SQL—name y a og ca UNC pathname to the fi e and the transact ona context token You’ reca from the d scuss on ear er that those are the two va ues needed to use SqlFileStream, and you’re about to see how exact y But a you’ve done so far s define a constant ho d ng two T-SQL statements The constant s fo owed by two var ab e dec arat ons, serverPath and serverTxn; these var ab es w rece ve the two spec a va ues when you execute those T-SQL statements ater n the code The method then creates and enters a new TransactionScope b ock Th s does not actua y beg n the ­ atabase transact on (you’ve not even connected to the database yet), but rather dec ares that a data d access w th n the b ock (and n any code ca ed from w th n the b ock) must part c pate n a ­database ­transact on Ins de the TransactionScope b ock, the code creates and opens a new S­ qlConnection ­Be ng 342  Part II  Going Beyond Relational

the first data access code ns de the TransactionScope b ock, th s a so mp c t y beg ns the database ­transact on Next, t creates a SqlCommand object assoc ated w th the open ­connect on and prepares ts command text to conta n your T-SQL statements (the INSERT fo owed by the SELECT) Invok ng the ExecuteReader method executes the T-SQL statements and returns a reader from wh ch you can retr eve the va ues returned by the SELECT statement The transact on s st pend ng at th s t me The INSERT statement does not prov de a va ue for RowId and nstead a ows SQL Server to automat ca y generate and ass gn a new uniqueidentifier ROWGUID va ue by defau t just ke before when you used T-SQL to nsert the first two rows We’ve a so po nted out that no va ue s prov ded for the Photo co umn—and th s s exact y how the defau t 0x va ue that you defined ear er for the Photo co umn comes nto p ay (we sa d we’d come back to t, and here you are) A though the row has been added by the INSERT statement, t w ro back (d sappear) f a ­ rob em occurs before the transact on s comm tted Because you d dn’t prov de a BLOB va ue for p the Photo co umn n the new row, SQL Server honors the defau t va ue 0x that you estab shed for t n the ­CREATE TABLE statement for PhotoAlbum Th s represents a zero- ength b nary stream, wh ch s c­ omp ete y d fferent than NULL Be ng a varbinary(max) co umn decorated w th the FILESTREAM attr bute, an empty fi e gets created n the fi e system that SQL Server assoc ates w th the new row At the same t me, SQL Server n t ates an NTFS fi e system transact on over th s new empty fi e and synchron zes t w th the database transact on So just ke the new row, the new fi e w d sappear f the database transact on does not comm t successfu y

Important  You cannot use SqlFileStream to populate BLOBs against NULL values in a new row’s varbinary(max) FILESTREAM column. You must always add new rows with a zero-length binary stream (0x) in the BLOB column, either by making it the default value as in this example, or by specifying it explicitly in your INSERT statement. This will result in the creation of a z­ ero-length file that can be streamed to (overwritten) by calling SqlFileStream, as described here. Immed ate y fo ow ng the INSERT statement, the SELECT statement returns Photo.PathName and GET FILESTREAM TRANSACTION CONTEXT What you’re essent a y do ng w th the WHERE c ause n th s SELECT statement s read ng back the same row you have just added (but not yet comm tted) to the PhotoAlbum tab e n order to reference the BLOB stored n the new fi e that was just created (a so not yet comm tted) n the fi e system The va ue returned by Photo.PathName s a fabr cated path to the BLOB for the se ected PhotoId The path s expressed n UNC format, and po nts to the network share name estab shed for the server nstance when you first enab ed FILESTREAM (wh ch s MSSQLSERVER n th s case, as shown n F gure 8-1) It s not a path to the fi e’s phys ca ocat on on the server, but rather conta ns nformat on SQL Server can use to der ve the fi e’s phys ca ocat on For examp e, you’ not ce that t a ways conta ns the GUID va ue n the uniqueidentifier ROWGUIDCOL co umn of the BLOB’s correspond ng row You retr eve the path va ue from the reader’s first co umn and store t n the serverPath str ng var ab e



Chapter 8  Nat ve F e Stream ng   343

Note  You’re able to read back the row you just added because your code is running ­inside the same transaction. However, with the default (and recommended) SQL Server ­transaction isolation mode set to Read Committed, no other users will be able to see this row because it has not yet been committed. That would be what is often referred to as a “dirty read,” because the data will never actually come into existence if the code fails to ­commit the pending transaction. To prevent dirty reads, SQL Server imposes a lock on the page holding the uncommitted row. It’s therefore very important for your transactions to be committed (or rolled back) as soon as possible after they are started. Any code that doesn’t absolutely need to run inside a transaction should run outside. We just exp a ned how SQL Server n t ated an NTFS fi e system transact on over the FILESTREAM data n the new row’s Photo co umn when you started the database transact on The GET FILESTREAM TRANSACTION CONTEXT funct on returns a hand e to that NTFS transact on ( f you’re not ns de a transact on, th s funct on w return NULL and your code won’t work) You obta n the transact on context, wh ch s returned by the reader’s second co umn as a SqlBinary va ue, and store t n the byte array named serverTxn Armed w th the BLOB path reference n serverPath and the transact on context n serverTxn, you have what you need to create a SqlFileStream object and perform d rect fi e access to stream the mage nto the Photo co umn You c ose the reader, term nate ts using b ock, then term nate the ­enc os ng using b ock for the SqlConnection as we Th s wou d norma y c ose the database ­connect on mp c t y, but that gets deferred n th s case because the code s st nested ns de the outer using b ock for the TransactionScope object So the connect on s st open at th s t me, and the transact on s st pend ng It s prec se y at th s po nt that you ca the SavePhotoFile method to stream the spec fied mage fi e nto the Photo co umn of the new y nserted PhotoAlbum row, overwr t ng the empty fi e just created by defau t When contro returns from SavePhotoFile, the TransactionScope object’s Complete method s nvoked and ts using b ock s term nated, s gna ng the transact on management API that everyth ng worked as expected Th s mp c t y comm ts the database transact on (wh ch n turn comm ts the NTFS fi e system transact on) and c oses the database connect on The SavePhotoFile method reads from the source fi e and wr tes to the database FILESTREAM ­storage n 512 KB chunks at a t me us ng ord nary NET stream ng techn ques, as shown here private static void SavePhotoFile (string clientPath, string serverPath, byte[] serverTxn) { const int BlockSize = 1024 * 512; using (FileStream source = new FileStream(clientPath, FileMode.Open, FileAccess.Read)) { using (SqlFileStream dest = new SqlFileStream(serverPath, serverTxn, FileAccess.Write)) { byte[] buffer = new byte[BlockSize];

344  Part II  Going Beyond Relational

int bytesRead; while ((bytesRead = source.Read(buffer, 0, buffer.Length)) > 0) { dest.Write(buffer, 0, bytesRead); dest.Flush(); } dest.Close(); } source.Close(); } }

The method beg ns by defin ng a BlockSize nteger constant that s set to a reasonab e va ue of 512 KB P cture fi es arger than th s w be streamed to the server n 512 KB b ocks at a t me The oca source mage fi e ( n clientPath) s then opened on an ord nary read-on y FileStream object Then the dest nat on fi e s opened by pass ng the two spec a va ues (serverPath and serverTxn), a ong w th a FileAccess.Write enumerat on request ng wr te access, nto the SqlFileStream constructor L ke the source FileStream object, SqlFileStream nher ts from System.IO.Stream, so t can be treated just ke any ord nary stream Thus, you atta n wr te access to the dest nat on BLOB on the ­database server’s NTFS fi e system Remember that th s output fi e s en sted n an NTFS transact on and ­noth ng you stream to t w be permanent y saved unt the database transact on s comm tted by the term nat ng TransactionScope b ock, after SavePhotoFile comp etes The rest of the SavePhotoFile method mp ements a s mp e oop that reads from the source FileStream and wr tes to the dest nat on SqlFileStream, one 512 KB b ock at a t me unt the ent re source fi e s processed, and then t c oses both streams That covers nsert ng new photos The rest of the code conta ns methods to retr eve ex st ng ­ hotos and stream the r content from the fi e system nto an Image object for d sp ay You’ find that p th s code fo ows the same pattern as the ast, on y now you’re perform ng read access The SelectPhoto method accepts a photo ID and returns the str ng descr pt on from the database n an output parameter The actua BLOB tse f s returned as the method’s return va ue n a System. Drawing.Image object You popu ate the Image object w th the BLOB by stream ng nto t from the database server’s NTFS fi e system us ng SqlFileStream, as shown here public static Image SelectPhoto(int photoId, out string desc) { const string SelectTSql = @" SELECT Description, Photo.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT() FROM PhotoAlbum WHERE PhotoId = @PhotoId"; Image photo; string serverPath; byte[] serverTxn; using (TransactionScope ts = new TransactionScope()) { using (SqlConnection conn = new SqlConnection(ConnStr)) { conn.Open();



Chapter 8  Nat ve F e Stream ng   345

using (SqlCommand cmd = new SqlCommand(SelectTSql, conn)) { cmd.Parameters.Add("@PhotoId", SqlDbType.Int).Value = photoId; using (SqlDataReader rdr = cmd.ExecuteReader(CommandBehavior.SingleRow)) { rdr.Read(); desc = rdr.GetSqlString(0).Value; serverPath = rdr.GetSqlString(1).Value; serverTxn = rdr.GetSqlBinary(2).Value; rdr.Close(); } } photo = LoadPhotoImage(serverPath, serverTxn); } ts.Complete(); } return photo; }

Once aga n, you start th ngs off by enter ng a TransactionScope b ock and open ng a c­ onnect on You then execute a s mp e SELECT statement that quer es the PhotoAlbum tab e for the record spec fied by the photo ID and returns the descr pt on and fu path to the mage BLOB, as we as the FILESTREAM transact ona context token And once aga n you use the pathname and transact ona context w th SqlFileStream to t e nto the server’s fi e system n the LoadPhotoImage method, as shown here private static Image LoadPhotoImage(string serverPath, byte[] serverTxn) { Image photo; using (SqlFileStream sfs = new SqlFileStream(serverPath, serverTxn, FileAccess.Read)) { photo = Image.FromStream(sfs); sfs.Close(); } return photo; }

Just as when you were nsert ng new photos (on y th s t me us ng FileAccess.Read nstead of ­FileAccess.ReadWrite), you create a new SqlFileStream object from the og ca pathname and ­transact on context You then pu the BLOB content d rect y from the NTFS fi e system on the server nto a new System.Drawing.Image object us ng the stat c Image.FromStream method aga nst the SqlFileStream object The popu ated mage s then passed back up to the form, where t s d sp ayed us ng the Image property of the PictureBox contro It’s t me to see a of th s n act on and g ve the app cat on a run! To nsert a new photo, spec fy a un que photo ID (you’ve a ready used 1 and 2 a ready), an mage fi e, and a descr pt on n the top group box n the PhotoForm w ndow, as shown n F gure 8-9, and then c ck Save 346  Part II  Going Beyond Relational

Figure 8-9  nsert ng a new photo nto F LESTREAM storage.

To se ect and d sp ay the photo and ts descr pt on back from the database, type ts photo ID n the bottom group box, and then c ck Load The photo s d sp ayed, as shown n F gure 8-10

Figure 8-10  Retr ev ng a photo from F LESTREAM storage.

Th s app cat on s sma , but t demonstrates everyth ng needed to everage the power of FILESTREAM n your NET c ent app cat ons The code that you need to wr te fo ows a fa r y ­stra ghtforward pattern that adapts eas y to d fferent scenar os In the next FILESTREAM app cat on, you’ stream content from a Hypertext Transfer Protoco (HTTP) serv ce and consume t n a W ndows Presentat on Foundat on (WPF) c ent us ng the very same FILESTREAM pr nc p es that you app ed n th s W ndows Forms app cat on



Chapter 8  Nat ve F e Stream ng   347

Creating a Streaming HTTP Service You’ now bu d a very s mp e serv ce to de ver photos over HTTP It w be mp emented as a norma M crosoft ASP NET Web App cat on project w th a s ng e PhotoServ ce aspx page Th s page can be ca ed by any HTTP c ent that passes n a photo ID va ue appended to the URL query str ng, and t w stream back the b nary content for the spec fied photo from the database FILESTREAM storage n SQL Server to the c ent

More Info  We use an ordinary ASP.NET page rather than a true Windows Communications Foundation (WCF) service in this example, because the ASP.NET Response object has ­built-in streaming capabilities that are relatively easy to code against. Conversely, WCF data contracts are buffered entirely in memory before sending and receiving by design, which does not work well for potentially large byte streams (such as BLOBs being served up by FILESTREAM). Two readily available techniques can be leveraged to create WCF services that stream BLOBs. First, the Message Transmission Optimization Mechanism (MTOM) protocol allows WCF data contracts to be streamed rather than buffered, using an approach very similar to the way SMTP transmits embedded email attachments. Your second option is to use WCF Data Services and implement a streaming provider, which returns BLOB data as a binary media resource separate from the normal text-based service response feed. To bu d the serv ce, fo ow these steps Start V sua Stud o and choose F e New Project Create a M crosoft V sua C# ASP NET Web App cat on project named PhotoLibraryHttpService n a so ut on named PhotoLibraryFileStreamDemo, as shown n F gure 8-11

Figure 8-11  Creat ng the stream ng HTTP serv ce project.

348  Part II  Going Beyond Relational

Add a new Web Form named PhotoService.aspx Un ke a typ ca aspx page, th s page w not return HTML content Instead, the page’s code-beh nd c ass w stream out b nary content from the database d rect y through the Response object bu t n to ASP NET Therefore, de ete a the HTML markup, eav ng on y the d rect ve at the very top that nks the aspx page w th ts ­code-beh nd c ass Now make th s the defau t startup page by r ght-c ck ng PhotoService.aspx n So ut on Exp orer and then choos ng Set As Start Page Next, sw tch to the code-beh nd c ass fi e by r ght-c ck ng aga n on the PhotoService.aspx node n So ut on Exp orer and then choos ng V ew Code Rep ace the starter code prov ded by V sua Stud o w th the code shown n L st ng 8-5 Listing 8-5  mp ement ng code for the stream ng photo serv ce.

using using using using using using using

System; System.Data; System.Data.SqlClient; System.Data.SqlTypes; System.IO; System.Transactions; System.Web.UI;

namespace PhotoLibraryHttpService { public partial class PhotoService : Page { private const string ConnStr = "Data Source=.;Integrated Security=True;Initial Catalog=PhotoLibrary;"; protected void Page_Load(object sender, EventArgs e) { int photoId = Convert.ToInt32(Request.QueryString["photoId"]); if (photoId == 0) { return; } const string SelectTSql = @" SELECT Photo.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT() FROM PhotoAlbum WHERE PhotoId = @PhotoId"; using (TransactionScope ts = new TransactionScope()) { using (SqlConnection conn = new SqlConnection(ConnStr)) { conn.Open(); string serverPath; byte[] serverTxn; using (SqlCommand cmd = new SqlCommand(SelectTSql, conn)) { cmd.Parameters.Add("@PhotoId", SqlDbType.Int).Value = photoId;



Chapter 8  Nat ve F e Stream ng   349

using (SqlDataReader rdr = cmd.ExecuteReader(CommandBehavior.SingleRow)) { rdr.Read(); serverPath = rdr.GetSqlString(0).Value; serverTxn = rdr.GetSqlBinary(1).Value; rdr.Close(); } } this.StreamPhotoImage(serverPath, serverTxn); } ts.Complete(); } } private void StreamPhotoImage(string serverPath, byte[] serverTxn) { const int BlockSize = 1024 * 512; const string JpegContentType = "image/jpeg"; using (SqlFileStream sfs = new SqlFileStream(serverPath, serverTxn, FileAccess.Read)) { byte[] buffer = new byte[BlockSize]; int bytesRead; Response.BufferOutput = false; Response.ContentType = JpegContentType; while ((bytesRead = sfs.Read(buffer, 0, buffer.Length)) > 0) { Response.OutputStream.Write(buffer, 0, bytesRead); Response.Flush(); } sfs.Close(); } } } }

Th s code bears a strong resemb ance to the photo retr eva code n the ear er W ndows Forms app cat on Th s nc udes the use of mp c t transact ons w th TransactionScope, so you’ once aga n need to add the appropr ate project reference R ght-c ck the project n So ut on Exp orer, choose Add Reference, c ck the NET tab, and then doub e-c ck the System.Transactions component The Page Load method first retr eves the photo ID passed n v a the photoId query str ng va ue If no va ue s passed, the method returns w thout stream ng anyth ng back (A ternat ve y, you cou d stream back an “ mage not found” mage ) Otherw se, as before, the photo fi e name and transact on context s obta ned after estab sh ng a connect on and transact on on the database, and nvok ng a SELECT statement ca ng the PathName method and GET FILESTREAM TRANSACTION CONTEXT funct on aga nst the photo ID spec fied n the WHERE c ause

350  Part II  Going Beyond Relational

W th these two key p eces of nformat on n hand, the StreamPhotoImage method s ca ed The method beg ns by defin ng a BlockSize nteger constant that s set to a reasonab e va ue of 512 KB (as before, p cture fi es arger than th s w be streamed to the c ent n 512 KB b ocks at a t me) And then once aga n, you use SqlFileStream to read the BLOB data from SQL Server

Note  Because this code is executing under the auspices of a web server, you may also need to grant access to the photo storage directory for the account executing the webpage. This might be ASPNET or NETWORK SERVICE if you’re using Internet Information Services (IIS) or your user account if you’re executing the page using Visual Studio’s development server. Before stream ng the b nary photo content, you need to change two propert es of the Response object In an aspx page, by defau t, the Response object’s BufferOutput property s set to true and the ­ContentType s set to text/html Here, you’ change BufferOutput to false (for true stream ng) and nform the c ent that you’re send ng a JPEG mage by chang ng the ContentType property to image/jpeg Us ng the SqlFileStream object, the code then reads from the database FILESTREAM storage n 512 KB chunks and streams to the c ent us ng the Reponse.OutputStream.Write and Response.Flush ­methods Th s s mp emented w th a s mp e oop that reads content from the SqlFileStream and sends t to the c ent v a the Response object unt the ent re fi e s processed Th s comp etes the serv ce app cat on Before mov ng on to bu d the WPF c ent, first test the serv ce Press F5 to start the app cat on When Internet Exp orer aunches PhotoService.aspx, t d sp ays an empty page because no photo ID s present n the URL’s query str ng In the Address bar, append ?photoId=3 to the URL and re oad the page The code-beh nd c ass retr eves photo ID 2 from the database and streams t back for d sp ay n the browser, as shown n F gure 8-12

Figure 8-12  Stream ng a photo over HTTP to nternet Exp orer.



Chapter 8  Nat ve F e Stream ng   351

You’ve created a funct on ng HTTP serv ce app cat on that streams p ctures from the database to any HTTP c ent It’s now ncred b y easy to bu d a sma WPF c ent app cat on that ca s the serv ce and d sp ays photos A that’s needed s the proper URL w th the des red photo ID ­spec fied n the query str ng, as you’ve just seen You’re us ng the ASP NET Deve opment Server prov ded by V ­ sua Stud o, wh ch by defau t random y ass gns a port number on localhost (port 1157 was ­ass gned th s t me, as nd cated n F gure 8-12) You’ need to estab sh a fixed port number ­nstead so that your WPF c ent can re ab y construct a URL for ca ng the serv ce Any unused port ­number w suffice, and so just p ck 22111 for th s app cat on To set the port number, r ght-c ck the ­PhotoLibraryHttpService project n So ut on Exp orer, and then choose Propert es Se ect the Web tab, se ect the Spec fic Port opt on, and then type 22111 for the port number, as shown n F gure 8-13

Figure 8-13  Sett ng a spec fic port number for the HTTP serv ce app cat on.

Building a WPF Client To bu d the WPF c ent, fo ow these steps In V sua Stud o choose F e New Project Create a new V sua C# WPF App cat on project named PhotoLibraryWpfClient Be sure to se ect Add To So ut on n the So ut on drop-down st, as shown n F gure 8-14, so that the project s added to the same ­PhotoLibraryFileStreamDemo so ut on that conta ns PhotoLibraryHttpService

352  Part II  Going Beyond Relational

Figure 8-14  Creat ng the stream ng WPF c ent app cat on.

V sua Stud o creates a new WPF app cat on w th a s ng e w ndow name Ma nW ndow xam , and then opens the w ndow des gner Rep ace the w ndow’s markup n the XAML w ndow w th the code shown n L st ng 8-6 Listing 8-6  XAML markup for WPF photo c ent.



Photo ID

Download Photo





Th s s s mp e markup that mere y defines Label, TextBox, Button, and MediaElement contro s for the w ndow, as shown n F gure 8-15



Chapter 8  Nat ve F e Stream ng   353

Figure 8-15  A s mp e F LESTREAM WPF U w ndow.

The MediaElement contro n WPF s a sca ed-down med a p ayer that s capab e of render ng a var ety of mu t med a types, nc ud ng mages and v deo, from any source A you need to do s set ts Source property to a URL that t can stream ts content from Doub e-c ck the Button contro , and then nsert the fo ow ng code n the button’s event hand er private void btnDownload_Click(object sender, RoutedEventArgs e) { string url = "http://localhost:22111/PhotoService.aspx?photoId=" + this.txtPhotoId.Text; this.mediaElement1.Source = new Uri(url); }

Th s code s mp y constructs a URL to the PhotoServ ce aspx page that s known to be runn ng on localhost port 22111, pass ng the des red photo ID n the query str ng When the MediaElement contro ’s Source property s set to that URL, the contro automat ca y ca s the serv ce and renders the photo that s streamed from the database to the serv ce and then from the serv ce to the WPF c ent over HTTP To see t work, run the app cat on, and request photo ID 2 for d sp ay, as shown n F gure 8-16 354  Part II  Going Beyond Relational

Figure 8-16  Stream ng a photo from the database over HTTP to a WPF c ent app cat on.

FILESTREAM Limitations and Considerations From a programm ng standpo nt, FILESTREAM offers a comp ete abstract on over the varbinary(max) data type Thus, app cat ons and frameworks ( nc ud ng Ent ty Framework) are unab e to d ­ st ngu sh between varbinary(max) co umns w th the FILESTREAM attr bute that are stored as fi es n the fi e system, and varbinary(max) co umns w thout the FILESTREAM attr bute that are stored ns de the ­re at ona tab es Th s prov des a h gh eve of compat b ty between FILESTREAM and ex st ng ­app cat ons that work w th varbinary(max) data However, FILESTREAM’s nterna ntegrat on w th the NTFS fi e system does create un que cha enges that affect compat b ty w th severa other SQL Server features It’s very mportant that you understand these compat b ty ssues before you start to use FILESTREAM, so that you can avo d unp easant p tfa s further down the road The essent a cons derat ons for FILESTREAM are summar zed be ow ■

M rror ng/HADR FILESTREAM does not work w th m rror ng Pr or to SQL Server 2012, th s was FILESTREAM’s most egreg ous m tat on However, FILESTREAM is compat b e w th the new H gh-Ava ab ty D saster Recovery (HADR, a so known as “a ways on”) feature n SQL Server 2012 HADR offers many mprovements over the m rror ng techno ogy t rep aces, and fortunate y, FILESTREAM compat b ty s one of them

■

Transparent Data Encrypt on (TDE) TDE (covered n Chapter 5) w the NTFS fi e system

■

not encrypt varbinary(max) FILESTREAM co umn data stored n

Rep cat on FILESTREAM can be used w th both transact ona rep cat on and merge rep cat on, w th ­severa restr ct ons and cons derat ons A subscr bers must be runn ng SQL Server 2008 or h gher For merge rep cat on, SQL Server ut zes the same uniqueidentifier ­ROWGUIDCOL



Chapter 8  Nat ve F e Stream ng   355

co umn that FILESTREAM uses (there can be on y one such co umn n a tab e, and both FILESTREAM and merge rep cat on requ re t) For th s co umn to be compat b e between both features, be sure to spec fy NEWSEQUENTIALID as ts defau t va ue n your tab e Refer to Books On ne for more nformat on about us ng rep cat on w th FILESTREAM ■

Log Sh pp ng Log sh pp ng fu y supports FILESTREAM, as ong as both the pr mary and secondary servers are runn ng SQL Server 2008 or h gher

■

Fu -Text Search (FTS) The FTS eng ne ndexes FILESTREAM co umns just as ord nary varbinary(max) co umns It s not on y 100% compat b e w th FILESTREAM, but actua y has greater s gn ficance n SQL Server 2012 w th the ntroduct on of F eTab e (see the sect on “Search ng Documents” at the end of th s chapter)

■

Database Snapshots SQL Server 2005 ntroduced database snapshots, a feature that ets you create a stat c v ew of the database that can be used for report ng or ro back purposes Database snapshots are not supported for FILESTREAM fi egroups However, you can st create a database snapshot of the standard fi egroups n the database (the FILESTREAM fi egroups w be marked as offl ne n those snapshots)

■

Snapshot Iso at on Leve SQL Server 2005 a so ntroduced snapshot so at on eve for transact ons (we cover snapshot so at on n Chapter 4) When FILESTREAM was first ntroduced w th SQL Server 2008, t was ncompat b e w th snapshot so at on As w th m rror ng, th s m tat on prevented many peop e from adopt ng FILESTREAM Fortunate y, th s m tat on was removed n SQL Server 2008 R2, and FILESTREAM can now be used w th snapshot so at on

■

Loca NTFS F e System FILESTREAM data can on y by stored on oca NTFS d sk vo umes However, remote BLOB ­storage (RBS) so ut ons ava ab e from M crosoft and th rd-party vendors w a ow you to ­configure a FILESTREAM-enab ed SQL Server database as a ded cated BLOB store for ­SharePo nt and other app cat ons

■

Secur ty To support SqlFileStream, the SQL Server nstance shou d be configured to use m xed-mode ( ntegrated) secur ty

■

SQL Server Express ed t on FILESTREAM s fu y supported by the free Express ed t on of SQL Server (though notab y, t s not supported by the LocalDB nstance of SQL Server supp ed w th SSDT) Furthermore, the

356  Part II  Going Beyond Relational

database s ze m t (4 GB n SQL Server 2008, or 10 GB n SQL Server 2008 R2 and 2012) does not nc ude varbinary(max) FILESTREAM co umns stored n the fi e system

Introducing FileTable The new F eTab e feature n SQL Server 2012 bu ds on FILESTREAM F eTab e comb nes FILESTREAM w th hierarchyid and the W ndows fi e system API to de ver exc t ng new BLOB capab t es n SQL Server As mp ed by the two words jo ned together n ts name, one F eTab e funct ons as two ­d st nct th ngs s mu taneous y 1. A F eTab e s an Ord nary Tab e 2. A F eTab e s an Ord nary F e System

F rst and foremost, a F eTab e s a regu ar SQL Server database tab e n every respect, w th one except on The schema of a F eTab e s fixed The co umns of a F eTab e and the r data types are ­pre-determ ned by SQL Server Spec fica y, every F eTab e conta ns the co umns shown n Tab e 8-1 Table 8-1  F xed F eTab e co umns Column Name

Data Type

Description

stream id

uniqueidentifier ROWGUIDCOL

Un que row dent fier

file stream

varbinary(max) FILESTREAM

BLOB content (NULL f d rectory)

name

varchar(255)

Name of fi e or d rectory

path locator

hierarchyid

Locat on of fi e or d rectory w th n the og ca fi e system

creation time

datetimeoffset(7)

Created

last write time

datetimeoffset(7)

Last mod fied

last access time

datetimeoffset(7)

Last accessed

is directory

bit

0=fi e, 1=d rectory

is is is is is is

bit

Storage attr butes

offline hidden readonly archive system temporary

Every F eTab e row represents an ord nary fi e or fo der n a fi e system whose og ca fo der s­ tructure s mp emented us ng the path locator co umn Th s s a hierarchyid va ue that denotes the ocat on of each fi e and fo der (row) w th n the og ca fi e system (tab e) The hierarchyid data type was ntroduced n SQL Server 2008 as a b nary va ue that, re at ve to other hierarchyid va ues n the same tree structure, dent fies a un que node pos t on It s mp emented as a system common ­anguage runt me (CLR) type, wh ch means that t’s a NET framework c ass, and has a set of ­methods you can use (such as GetAncestor, GetDescendant, GetReparentedValue, and IsDescendantOf ) to

Chapter 8  Nat ve F e Stream ng   357

traverse and man pu ate the h erarchy Thus, t’s perfect for cast ng the h erarch ca structure of a fi e system over a re at ona tab e, wh ch s prec se y what F eTab e does

More Info  A proper understanding of hierarchyid is required to query and manipulate a FileTable’s path locator column with T-SQL. Chapter 7 provides the necessary coverage. The path locator co umn s defined as the tab e’s pr mary key (w th a non-c ustered ndex) A separate key va ue s a so stored n the stream id co umn (w th a non-c ustered un que ndex) Th s s the uniqueidentifier ROWGUIDCOL va ue requ red by any tab e w th varbinary(max) FILESTREAM co umns, so F eTab e s no except on Un ke path locator, th s un que va ue w never change once t s ass gned to a new F eTab e row, even f the row s ater “reparented” (that s, ass gned to another ­ocat on n the h erarchy) You can use e ther path locator or stream id to un que y dent fy each row n a F eTab e But because the path locator va ue changes whenever the fo der or fi e represented by the row s “moved” n the fi e system, you shou d use the stream id va ue to store ong-term ­(permanent) references to rows n a F eTab e Every row n a F eTab e corresponds to e ther a s ng e fi e or fo der, as determ ned by the bit va ue n the is directory co umn The fi e or fo der name s stored n the name co umn as an nvarchar(255) str ng A of the other co umn names are se f-descr b ng, and are used to store typ ca fi e system nformat on such as var ous t mestamps and storage attr butes Rows represent ng d rector es have an IsDirectory co umn va ue of 1 (for true) D rectory rows have no BLOB content, and thus a ways conta n NULL n the file stream co umn Rows represent ng fi es have an IsDirectory va ue of 0 (for fa se), and the actua fi e content (the BLOB tse f) s stored n the file stream co umn Th s s a varbinary(max) data type decorated w th the FILESTREAM attr bute, wh ch means that the b nary content n the file stream co umn s stored n the NTFS fi e system that SQL Server s manag ng beh nd the scenes, rather than the structured fi egroups where a the other tab e data s stored ( n other words, standard FILESTREAM behav or, as exp a ned n the first part of th s chapter) In add t on to these 14 co umns, each F eTab e nc udes the three computed (read-on y) co umns shown n Tab e 8-2 Table 8-2  Computed F eTab e co umns Column Name

Data Type

Description

parent path locator

hierarchyid

Parent node (der ved from path locator)

file type

nvarchar(255)

F ename extens on der ved from name

cached file size

bigint

BLOB byte ength der ved from file stream

The parent path locator co umn returns the resu t of ca ng GetAncestor(1) on path locator to o ­ bta n the path locator to the parent fo der of any fi e or fo der n the tab e The file type co umn returns the extens on of the fi ename parsed from the str ng va ue n the name co umn And the

358  Part II  Going Beyond Relational

cached file size co umn returns the number of bytes stored n the file stream co umn (that s, the s ze of the BLOB stored beh nd the scenes n the SQL Server nterna y managed NTFS fi e system) W th th s fixed schema n p ace, every F eTab e has what t needs to represent a og ca fi e system Thus, SQL Server s ab e to fabr cate a W ndows fi e share over any F eTab e Th s mmed ate y exposes the F eTab e to any user or app cat on who can then v ew and update the tab e us ng standard fi e I/O semant cs (for examp e, drag-and-drop w th W ndows Exp orer, or programmat ca y w th System.IO.FileStream and the W ndows API) Thus ■

Creat ng a fi e or d rectory n the og ca fi e system adds a row to the tab e

■

Add ng a row to the tab e creates a fi e or d rectory n the og ca fi e system

F gure 8-17 shows the tota F eTab e p cture

Server Mach ne Name

Server nstance F LESTREAM Share Name

Database Name

Windows File I/O F eTab e Name

T-SQL

SqlFileStream

F eTab e Rows

varb nary(max) F LESTREAM

h erarchy d

FileTable ‘Documents’ in Database ‘mydb’ stream id

name

path locator

file stream

...

27D8D4AD-D100-39... ‘ Financials’

0xFF271A3562... NULL

...

78F603CC-0460-73... ‘ReadMe . docx’

0xFF59345688... 0x3B0E956636AE020B...

...

207D4A96-E854-01... ‘Budget . xlsx’

0xFD0011039A... 0xF359000EE30F29A2...

...

SQL Server Structured F e Groups Non BLOB data (pages, rows, co umn)

NTFS F e System Synchron zed database/NTFS transact on

BLOBs (f e contents) n varb nary(max) F LESTREAM co umns

Figure 8-17  F eTab e data can be accessed v a T SQL, SqlFileStream, or the W ndows fi e system.

Take a moment to d gest what’s happen ng here In add t on to the programmat c FILESTREAM access us ng T-SQL or SqlFileStream, SQL Server 2012 now offers a th rd nterface to FILESTREAM a

Chapter 8  Nat ve F e Stream ng   359

og ca fi e system In a sense, th s fi s the FILESTREAM gap n wh ch the fi e system tse f s comp ete y naccess b e Th s s not to say that F eTab e ets you d rect y access SQL Server’s nterna y managed NTFS fi e system; certa n y not That rema ns obfuscated and pr vate, as t cont nues to funct on n standard FILESTREAM fash on aga nst BLOB data that just happens to be n a F eTab e nstead of any other tab e What you do get s an abstract on ayer over the F eTab e that funct ons as a standard fi e system In th s og ca fi e system, everyth ng about each fi e and fo der—except the BLOB content of the fi es themse ves— s stored n the F eTab e’s structured fi egroup, whereas the BLOBs themse ves are phys ca y stored n the NTFS fi e system as ord nary varbinary(max) FILESTREAM data So you can see that there’s rea y noth ng new beneath the F eTab e ayer; as before, SQL Server synchron zes transact ona access between the row n the F eTab e and ts correspond ng BLOB content n the NTFS fi e system to ensure that ntegr ty s proper y ma nta ned between them As w th T-SQL access, th s synchron zat on occurs mp c t y when man pu at ng the F eTab e v a the exposed W ndows fi e share And, be ng an ord nary tab e n v rtua y every respect, you a so have the opt on to use SqlFileStream w th exp c t transact on synchron zat on for the fastest poss b e stream ng of BLOBs nto and out of a F eTab e A of th s s extreme y appea ng You now have tota flex b ty for BLOB storage n the database W th F eTab e, you can eas y m grate ex st ng app cat ons that work aga nst phys ca fi e systems w thout wr t ng any custom T-SQL or fancy SqlFileStream code Just use a F eTab e, et the ex st ng app cat ons cont nue work ng w thout mod ficat on, and enjoy the benefits of your fi es becom ng an ntegra part of the SQL Server database

Creating a FileTable F eTab e re es on FILESTREAM Therefore, FILESTREAM must be enab ed before you can create a ­F eTab e S m ar y, a F eTab e-enab ed database must be a FILESTREAM-enab ed database created w th a proper y defined FILEGROUP…CONTAINS FILESTREAM c ause In add t on, you must prov de a WITH FILESTREAM c ause n your CREATE DATABASE statement (or, f F eTab e-enab ng an ex st ng database, you must prov de a SET FILESTREAM c ause n your ALTER DATABASE statement) In th s c ause, use DIRECTORY NAME to spec fy the fo der name for th s database Th s fo der w appear n the root d rectory of the fi e share assoc ated w th and exposed by the SQL Server nstance Second, spec fy NON TRANSACTED ACCESS=FULL to enab e non-transacted access Th s exposes every ­F eTab e n the database as a subfo der beneath the database fo der spec fied by DIRECTORY NAME Do not et the non-transacted access sett ng confuse you Th s just a ows F eTab e data to be ­exposed v a the W ndows API, whose operat ons are non-transact ona n nature Interna y, ­transact ons are a ways used to synchron ze varbinary(max) FILESTREAM data between the database and NTFS fi e system—e ther mp c t y (whether us ng T-SQL or the W ndows API) or exp c t y ( f ­us ng SqlFileStream)—as ustrated n F gure 8-17 L st ng 8-7 demonstrates creat ng a F eTab e-enab ed database w th a mod fied vers on of the ­CREATE DATABASE statement you used n L st ng 8-1 The WITH FILESTREAM c ause n th s ­statement enab es ­F eTab e for the PhotoFileLibrary database, and exposes a PhotoFileLibrary d rectory for the database beneath the W ndows fi e share for the SQL Server nstance (MSSQLSERVER n our examp es) 360  Part II  Going Beyond Relational

Listing 8-7  Creat ng a F eTab e enab ed database.

CREATE DATABASE PhotoFileLibrary ON PRIMARY (NAME = PhotoFileLibrary_data, FILENAME = 'C:\Demo\PhotoLibrary\PhotoFileLibrary_data.mdf'), FILEGROUP FileStreamGroup CONTAINS FILESTREAM (NAME = PhotoFileLibrary_blobs, FILENAME = 'C:\Demo\PhotoLibrary\PhotoFiles') LOG ON (NAME = PhotoFileLibrary_log, FILENAME = 'C:\Demo\PhotoLibrary\PhotoFileLibrary_log.ldf') WITH FILESTREAM (DIRECTORY_NAME='PhotoFileLibrary', NON_TRANSACTED_ACCESS=FULL)

Creat ng the actua F eTab e s the eas est part Because SQL Server contro s the schema of every F eTab e, you just use a CREATE TABLE statement w th the new AS FileTable c ause and don’t nc ude any co umns, as shown n L st ng 8-8 Listing 8-8  Creat ng a F eTab e.

USE PhotoFileLibrary GO CREATE TABLE PhotoFiles AS FileTable GO

The PhotoFiles F eTab e s ready to use You w find a root PhotoFiles fo der for the F eTab e beneath the PhotoLibrary fo der created for the database n the W ndows fi e share for the server nstance, as shown n F gure 8-18

Figure 8-18  A W ndows Exp orer w ndow open to the fi e share exposed by a F eTab e.



Chapter 8  Nat ve F e Stream ng   361

Manipulating a FileTable As w th any FILESTREAM-enab ed tab e, you can nteract w th the F eTab e us ng the same T-SQL and SqlFileStream techn ques covered ear er n th s chapter In add t on, the F eTab e can be accessed by users and app cat ons v a the og ca fi e system exposed by the W ndows fi e share To demonstrate, run the code n L st ng 8-9 Th s code nserts rows nto the F eTab e to create a fo der w th two fi es ns de of t Listing 8-9  Us ng T SQL to create fo ders and fi es n a F eTab e.

-- Get root pathnames for the database and FileTable DECLARE @DbRootPath varchar(max) = FILETABLEROOTPATH() -- \\machine\instance\db DECLARE @TableRootPath varchar(max) = @DbRootPath + '\PhotoFiles' -- Create folder 'Mountains' in the FileTable's root DECLARE @TableRootNode hierarchyid = GETPATHLOCATOR(@TableRootPath) INSERT INTO PhotoFiles(name, path_locator, is_directory) VALUES ('Mountains', @TableRootNode.GetDescendant(NULL, NULL), 1) -- Get new folder's hierarchyid DECLARE @FolderPath varchar(max) = @TableRootPath + '\Mountains' DECLARE @FolderNode hierarchyid = GETPATHLOCATOR(@FolderPath) -- Add a text file to the folder DECLARE @TextFileNode hierarchyid = @FolderNode.GetDescendant(NULL, NULL) INSERT INTO PhotoFiles(file_stream, name, path_locator) VALUES( CONVERT(varbinary(max), 'This folder contains pictures of mountains.'), 'ReadMe.txt', @TextFileNode) -- Add an image file to the folder DECLARE @ImageFileNode hierarchyid = @FolderNode.GetDescendant(@TextFileNode, NULL) INSERT INTO PhotoFiles(file_stream, name, path_locator) VALUES( (SELECT BulkColumn FROM OPENROWSET(BULK 'C:\Demo\Ascent.jpg', SINGLE_BLOB) AS x), 'Ascent.jpg', @ImageFileNode)

Th s code first ca s the FILETABLEROOTPATH funct on to ass gn the root pathname for ­current ­database’s F eTab e share nto @DbRootPath Th s name s based on the mach ne name, SQL Server nstance share name, and database name You shou d a ways use FILETABLEROOTPATH to ­obta n th s nformat on, because hard-cod ng the mach ne, nstance, and database names w prevent your code from runn ng n d fferent env ronments For examp e, assum ng the PhotoFileLibrary ­database s runn ng on a mach ne named SQL2012DEV under an nstance named MSSQLSERVER, the ­FILETABLEROOTPATH funct on w return \\SQL2012DEV\MSSQLSERVER\PhotoFileLibrary The next ne of code concatenates the database root pathname w th \PhotoFiles (the name you ­ ss gned to the new F eTab e) to obta n the root pathname for the F eTab e, and ass gns t nto a @TableRootPath Th s s the parent for the new Mounta ns fo der you are about to create Node ­man pu at on n the F eTab e tree structure a ways nvo ves the hierarchyid va ue stored n the path locator co umn To get the hierarchyid va ue of any fo der or fi e n the F eTab e, you can ca the 362  Part II  Going Beyond Relational

GETPATHLOCATOR funct on and pass t the fu pathname to the des red fo der or fi e In L st ng 8-9, GETPATHLOCATOR obta ns the hierarchyid of the F eTab e’s root path and ass gns t nto @TableRootNode In the INSERT statement that fo ows, GetDescendant s nvoked on @TableRootNode to ­generate the appropr ate path locator va ue (a new hierarchyid) for the Mounta ns fo der As exp a ned n Chapter 7, the GetDescendant method s nvoked on a parent’s hierarchyid va ue, and returns a new hierarchyid va ue represent ng a ch d of that parent The method expects two other hierarchyid va ues as nput parameters, wh ch spec fies two ex st ng ch d nodes that the new hierarchyid va ue shou d be “sandw ched” n-between Because there are no ch d nodes yet, NULL s passed for both GetDescendant parameters n the INSERT statement, wh ch generates the proper hierarchyid va ue for the one ch d node n the F eTab e’s root the Mounta ns fo der The INSERT statement ass gns the GetDescendant method’s resu t to the path locator co umn of the new F eTab e row, wh ch dent fies th s row’s pos t on n the h erarchy as the first and on y ch d beneath the F eTab e root It a so ass gns ‘Mounta ns’ to the name co umn and 1 (for true) to the is directory co umn A the other co umns n the new F eTab e row are ass gned the r defau t va ues Th s means that NULL s stored n file stream, the varbinary(max) FILESTREAM co umn that stores the BLOBs for rows n the F eTab e that represent actua fi es Fo ders have no BLOBs, so F eTab e rows represent ng fo ders (that s, where IsDirectory s 1) w a ways conta n NULL n the file stream co umn Next, GETPATHLOCATOR s ca ed aga n, th s t me to obta n the hierarchyid va ue of the new y created Mounta ns fo der Th s hierarchyid va ue w now be used as the parent for two more ­GetDescendant ca s—one for each of the two ch d nodes represent ng the text fi e and mage fi e ­beneath Mounta ns For the first ch d, two NULL va ues are aga n passed to GetDescendant to ­generate a hierarchyid va ue for the text fi e, because as before, th s s the first ch d node ­beneath the parent node (wh ch s the Mounta ns fo der th s t me) The INSERT statement ass gns the ­GetDescendant resu t to the path locator co umn of the new F eTab e row, wh ch dent fies th s row’s pos t on n the h erarchy as the first and on y ch d beneath Mounta ns It a so ass gns ‘ReadMe.txt’ to the name co umn and the fi e’s BLOB content to the file stream co umn The BLOB for th s sma text fi e s easy to n- ne s mp y by convert ng an ord nary str ng to varbinary(max) A other co umns ­assume the r defau t va ues, nc ud ng is directory, wh ch defau ts to 0 (fa se) The mage fi e s created n a very s m ar manner The on y s gn ficant d fference s n the ca to GetDescendant Because a second ch d node s now be ng added to the Mounta ns fo der, the hierarchyid of the ast ex st ng ch d node (current y the text fi e node just added) s spec fied as the second GetDescendant parameter, rather than NULL It’s mportant to understand that th s techn que s requ red to ensure un queness across the ch d nodes beneath a parent; t’s not rea y “pos t on ng” the ch d nodes n any usefu way Pract ca y, you and your users w sort F eTab e data as des red (for examp e, ascend ng by name, or descend ng by creat on date) The th rd INSERT statement a ­ ss gns the GetDescendant resu t to path locator and ‘Ascent.jpg’ to the name co umn The BLOB for the mage fi e s mported nto the file stream co umn from an externa copy of Ascent.jpg us ng the ­OPENROWSET funct on w th ts BULK and SINGLE BLOB opt ons (th s mage fi e s supp ed w th the samp e code for th s chapter on the book’s compan on webs te, or you can just subst tute one of your own mages for the exerc se)



Chapter 8  Nat ve F e Stream ng   363

After runn ng the code n L st ng 8-9, use W ndows Exp orer to nav gate to the Mounta ns fo der n the fi e system exposed by the F eTab e A most ke mag c, W ndows Exp orer shows the fo der and fi es created by the three INSERT statements, as shown n F gure 8-19

Figure 8-19  W ndows Exp orer show ng fo ders and fi es nserted nto a F eTab e by T SQL.

Doub e-c ck Ascent.jpg to v ew the mage us ng W ndows Photo V ewer (the defau t ­app cat on assoc ated w th mage fi es) Beh nd the scenes, SQL Server finds the F eTab e row for the mage fi e, retr eves ts BLOB from the obfuscated fi e system where that row’s varbinary(max) FILESTREAM ­co umn s actua y stored, and streams t n to W ndows Photo V ewer (wh ch you can now c ose) Now doub e-c ck the text fi e to open t n Notepad (the defau t app cat on assoc ated w th text fi es) Instead of open ng as expected, you may be surpr sed to rece ve the error message “The request s not supported” nstead In fact, th s s not a bug, but rather a man festat on of the fact that the fi e system exposed by F eTab e does not support memory-mapped files Th s s a techn que that some app cat ons use to map areas of memory d rect y to the sectors on d sk where a oca fi e s phys ca y stored Thus, the app cat on doesn’t need to “ oad” the fi e nto memory, wh ch opt m zes ­performance Notepad uses memory mapp ng when ed t ng oca fi es, and so t can’t open a fi e d rect y from a F eTab e share on the oca database server Th s w norma y not present a prob em, because users and app cat ons access the F eTab e share remote y (not d rect y on the database server), and fi e memory mapp ng s d sab ed w th remote fi e share access To prove that there’s rea y noth ng wrong w th ReadMe.txt, make a copy of t from the F eTab e share (you can just drag and drop t to your desktop) and then doub e-c ck the copy You’ see that the copy opens n Notepad w thout a prob em You can a so map a dr ve etter to F eTab e share, and then access the fi es through the mapped dr ve etter Th s foo s W ndows nto th nk ng that the oca F eTab e share s a remote fi e share and, as a resu t, oca app cat ons w not attempt to use memory mapp ng for access ng F eTab e fi es You created a fo der and two fi es n the og ca fi e system by man pu at ng a F eTab e n T-SQL L kew se, you can have SQL Server nsert, update, and de ete rows n a F eTab e by man pu at ng fi es and d rector es n the og ca fi e system S m ar y, organ z ng fi es and fo ders us ng drag and drop 364  Part II  Going Beyond Relational

n W ndows Exp orer automat ca y adjusts the hierarchyid va ues n the path locator co umn of a ­ ffected rows n the F eTab e It’s easy to demonstrate th s b d rect ona capab ty; just use W ndows a Exp orer to drag and drop a few of your own fo ders and fi es nto the F eTab e share Then run a SELECT query aga nst the F eTab e You w see that SQL Server has nserted new rows to back the fo ders and fi es you added v a W ndows Exp orer

More Info There are several catalog views provided to help you manage FileTables in your database. There is also a special dynamic management view and related stored procedure that lets you monitor and kill open handles on FileTable resources. Consult Books Online for detailed usage and syntax of these system-defined objects that support FileTable.

Searching Documents You’ve seen how FILESTREAM and F eTab e a ow unstructured documents to be stored as first-c ass c t zens ns de the database w th unprecedented ease and performance So t’s on y natura to beg n wonder ng how to make better use of unstructured content that ves n the database, beyond mere y stream ng t to c ent app cat ons Th s s where Fu -Text Search (FTS) and the new Stat st ca Semant c Search n SQL Server 2012 come n The FTS eng ne was first ntroduced as an add-on component to SQL Server 2000, and t prov des much more powerfu w dcard search capab t es than the T-SQL LIKE operator FTS has stead y ­matured w th each product re ease, and s now a fu y ntegrated part of the re at ona database eng ne FTS can search varbinary(max) content, but pr or to FILESTREAM, t wasn’t feas b e to store many arge BLOBs n varbinary(max) co umns Now that do ng so s not on y feas b e but easy, FTS s a natura too for search ng content ns de many d fferent types of documents that s mp y get dropped nto a F eTab e FTS s ab e to parse many d fferent fi e types, nc ud ng M crosoft Word documents, M crosoft PowerPo nt decks, M crosoft Exce spreadsheets, Adobe PDF fi es, and a host of other popu ar formats It understands word-stemm ng and syntax ru es of over 50 anguages, and can a so perform prox m ty search ng w th the NEAR keyword In SQL Server 2012, FTS now a so supports property search ng and custom zab e d stance for prox m ty search ng w th NEAR SQL Server 2012 a so ntroduces Stat st ca Semant c Search, a new feature that bu ds on the FTS arch tecture, extend ng fu -text search capab t es beyond what s offered by FTS a one The ­Semant c Search eng ne d scovers key phrases n each document, and can therefore return documents deemed s m ar to one another based on common key phrases In add t on to enab ng FTS, you must separate y nsta , attach, and reg ster a spec a Semant c Language Stat st cs Database before you can start us ng Semant c Search In ts first re ease, Semant c Search supports fewer anguages than FTS does (on y about 15), and ndexes on y s ng e-word phrases However the ab ty to categor ze your documents automat ca y, and to dent fy documents that are s m ar or re ated, enab es powerfu new features for document management app cat ons



Chapter 8  Nat ve F e Stream ng   365

Summary For app cat ons that work w th BLOB data, FILESTREAM great y enhances the storage and ­performance of unstructured content n the database by everag ng the nat ve NTFS fi e system It does th s wh e ma nta n ng og ca ntegrat on between the database and fi e system that nc udes transact ona support As a resu t, you no onger need to comprom se between effic ency and ­comp ex ty as you d d n the past when faced w th the cho ce of stor ng BLOB data ns de or outs de the database You a so earned how to use SqlFileStream to de ver h gh-performance stream ng of BLOB content between the fi e system managed by SQL Server and your W ndows, web, and WPF app cat ons F na y, you earned how F eTab e n SQL Server 2012 comb nes FILESTREAM and hierachyid to furn sh a database-backed fi e system for users and app cat ons You can app y the techn ques you earned n th s chapter to a w de range of app cat ons that requ re ntegrat on between the re at ona database and a stream ng fi e system

366  Part II  Going Beyond Relational

C hapter 9

Geospatial Support —Leonard Lobel

L

ocat on-aware app cat ons are ub qu tous nowadays You find them on phones, tab ets, aptops, and desktops Enterpr se users and consumers a ke re y on Geograph c Informat on System (GIS) ­app cat ons (and GIS extens ons to trad t ona app cat ons) n the home, car, office, and everywhere n between GIS app cat ons—powered by G oba Pos t on ng Sate te (GPS) techno ogy—enhance the user exper ence w th soph st cated mapp ng nte gence In short, GIS s a about stor ng and p ­ rocess ng geospatial data (often s mp y ca ed spatial data), and th s chapter s a about the r ch ­support for spat a data n M crosoft SQL Server 2012

Note  We use the terms spatial and geospatial interchangeably throughout this chapter. In SQL Server 2012, spat a data support s a powerfu extens on to the core re at ona eng ne that enab es you to embed ocat on awareness nto the database Us ng the geometry and geography data types, you can mport, export, store, and process d fferent types of spat a data In th s chapter, we w qu ck y cover bas c spat a concepts, and then d ve nto more advanced examp es (a ong w th severa samp e app cat ons) to get you comfortab e work ng w th spat a data

SQL Server Spaces Out In SQL Server 2012, the database tse f s capab e of understand ng geospat a data (shapes—projected onto e ther a flat surface or the round earth) as a nat ve data type SQL Server can effic ent y store and process nstances of such data, so deve opers can enjoy r ch spat a funct ona ty at the database eve Th s feature (wh ch was first ntroduced n M crosoft SQL Server 2008 and s ava ab e n a ed t ons of SQL Server) offers an attract ve a ternat ve to perform ng spat a ca cu at ons at the app cat on eve , e ther through custom wr tten code (wh ch s far from tr v a ) or by us ng a th rd-party brary (wh ch s far from nexpens ve) The a gor thms used n geospat a operat ons are very comp ex (to say the east), and a c­ omprehens ve treatment of the top c s we beyond the scope of th s chapter At the same t me, abstract on of that comp ex ty s a key aspect of the spat a data types The geospat a support n SQL Server s based on the OpenGIS S mp e Features for SQL standard Th s standard enab es deve opers to qu ck y program aga nst spat a data w thout requ r ng a deep understand ng of the mathemat ca

367

formu ae beh nd spat a ca cu at ons In th s chapter, we’ start by first exp a n ng the two bas c spat a mode s Then we’ exam ne severa code demonstrat ons that everage the geometry and geography data types n a var ety of more advanced scenar os

Spatial Models Let’s beg n our d scuss on w th an exp anat on of the two spat a mode s These are known as the planar and geodetic mode s

Planar (Flat-Earth) Model The p anar mode s a flat surface where shapes are p otted us ng two-d mens ona x- and y-coord nates The coord nate system semant cs are ent re y up to you; measurements can be anyth ng from p xe s to nches, meters, and m es It makes sense to work w th the p anar mode when you’re dea ng w th re at ve y sma areas (such as a p ece of paper, bu d ng floor p an, or park ng ot), or even arger areas that are e ther conceptua y flat or st sma enough where the earth’s curvature does not skew the outcome of area and d stance ca cu at ons F atten ng the earth onto a two-d mens ona surface resu ts n a spat a d stort on that makes t ­mposs b e to ca cu ate measurements accurate y over arger spaces on the g obe’s actua shape You can see th s n F gure 9-1, wh ch shows a p anar mode map of the earth The flattened ­project on stretches the earth more and more the farther away ts parts are from the equator For examp e, not ce that Antarct ca (way down by the South Po e) appears to be s gn ficant y arger than North Amer ca Yet n fact, North Amer ca (at ~ 24 m on sq km) s s gn ficant y arger than Antarct ca (at ~ 13 m on sq km) Now draw a stra ght ne from New York to Amsterdam on th s map The ength of that ne does not rea y represent the accurate d stance between those two c t es, because the actua ne wou d be stretched and curved aga nst the shape of the g obe So c ear y, the p anar mode s nappropr ate for ca cu at ng spat a data over arge areas of the p anet

Geodetic (Ellipsoidal Sphere) Model If your database needs to perform spat a ca cu at ons that span s gn ficant areas of the earth, t needs to adopt the popu ar y he d be ef that the wor d s round It’s not perfect y round, however Due to ts rotat on, the earth s w der around the equator than t s around the po es, such that t forms an ellipsoidal sphere To accurate y p ot and compute shapes and ntersect ons on a p anetary sca e, the prec se curvature of the earth’s un que spher ca shape must be taken nto cons derat on To ach eve th s accuracy, the geodet c mode (dep cted n F gure 9-2) represents ocat ons on the p anet us ng the earth’s ong tude and at tude coord nate system Any g ven po nt on earth s ­represented as at tude and ong tude, where at tude spec fies the ang e north or south of the ­equator, and ong tude spec fies the ang e east or west of the Pr me Mer d an

368  Part II  Going Beyond Relational

North America (~ 24 m sq km) Equator

Africa (~ 30 m sq km)

Antarctica (~ 13 m sq km) Figure 9-1  P anar spat a mode (flat earth project on).

Figure 9-2  Geodet c spat a mode (round earth project on).



Chapter 9  Geospat a Support   369

Spatial Data Standards SQL Server prov des two data types—geometry and geography—des gned to work respect ve y w th the p anar and geodet c spat a mode s Shapes are projected onto spat a mode s us ng vector objects, wh ch are co ect ons of po nts, nes, po ygons (c osed shapes), and—new n SQL Server 2012—curves and arcs Both the geometry and geography data types support the three standard formats for consum ng spat a data We -Known Text (WKT), We -Known B nary (WKB), and Geography Markup Language (GML) WKT, WKB, and GML are standards governed by the Open Geospat a Consort um (OGC), an ­nternat ona body of more than 400 compan es M crosoft s an act ve and h gh y nfluent a member of the OGC, and SQL Server 2012 supports the OpenGIS S mp e Features for SQL standard—a spec ficat on pub shed by the OGC for the exchange of spat a data between d fferent database p atforms (you can earn more about the spec ficat on from http://www.opengeospatial.org/standards/sfs) The spat a data types are NET c asses that expose more than 90 ava ab e methods About t­ wo-th rds of those methods are named w th the prefix ST; for examp e, STArea, STDistance, and ­STArea The ST prefix nd cates spat a methods that mp ement the OGC spec ficat on, but ­M crosoft a so adds some of ts own extens ons to SQL Server that are not part of the OGC standard These are d st ngu shed w th method names that do not start w th ST; such as Parse, ToString, and GeomFromGml Some are stat c methods (mean ng that they are nvoked d rect y on the ­geometry or geography data type name us ng the spec a doub e-co on syntax), and some are nstance ­methods (mean ng that they are nvoked on an nstance of a geometry or geography data type us ng the ­standard “dot” notat on) A though the major ty of the spat a methods are ava ab e and ­funct on dent ca y w th both data types, a few of them are un que to on y geometry or geography Th s ­chapter exp a ns many, but certa n y not a , of the ava ab e spat a methods You shou d consu t M crosoft Books On ne for the comp ete spat a data type method reference (the geometry methods can be found at http://msdn.microsoft.com/en-us/library/bb933973.aspx and the geography methods at http://msdn.microsoft.com/en-us/library/bb933802.aspx)

Importing Well-Known Text (WKT) WKT s the eas est way to work w th spat a data n SQL Server It offers a very terse syntax for defin ng shapes us ng a handfu of keywords (such as POINT, LINESTRING, and POLYGON), and SQL Server mp ements a spat a data c ass for each of these shapes F gure 9-3 shows a the supported spat a c asses, a ong w th the r assoc ated WKT keywords and an examp e nstance of each shape Note that the CIRCULARSTRING, COMPOUNDCURVE, CURVEPOLYGON, and FULLGLOBE WKT shapes are new c asses n SQL Server 2012, and that we cover those and other SQL Server 2012 enhancements ater n the chapter

370  Part II  Going Beyond Relational

POINT

MULTIPOINT

LINESTRING

MULTILINESTRING

CIRCULARSTRING

COMPOUNDCURVE

POLYGON

MULTIPOLYGON

CURVEPOLYGON

GEOMETRYCOLLECTION

FULLGLOBE

Figure 9-3  Spat a c asses and the r assoc ated WKT keywords.

Every shape from the s mp est po nt to the most comp ex po ygons and co ect ons s expressed as a WKT str ng The WKT str ng syntax comb nes the var ous shape keywords w th numer c coord nates Tab e 9-1 shows some examp es of WKT str ngs Table 9-1  Examp es of WKT Str ngs



WKT String

Description

POINT(6 10)

A s ng e po nt at xy coord nates 6, 10

POINT( 111.06687 45.01188)

A s ng e po nt on the earth ( ong tude/ at tude coord nates)

LINESTRING(3 4,10 50,20 25)

A two part ne, drawn between three po nts spec fied as ­x y coord nates

POLYGON(( 75.17031 39.95601, 75.16786 39.95778, 75.17921 39.96874, 75.18441 39.96512, 75.17031 39.95601))

An enc osed shape on the earth drawn between the po nts spec fied as ong tude/ at tude coord nates

CIRCULARSTRING(1 5, 6 2, 7 3)

A curved ne, drawn between three po nts spec fied as ­x y coord nates

GEOMETRYCOLLECTION( POINT(6 10), CIRCULARSTRING(1 5, 6 2, 7 3), LINESTRING(3 4,10 50,20 25))

A co ect on w th three shapes; a po nt, a c rcu ar str ng, and a ne str ng

Chapter 9  Geospat a Support   371

As you can see, the same WKT syntax s used for project ng spat a ent t es onto both p anar and geodet c mode s A so not ce that geodet c coord nates are a ways expressed n WKT w th the ­ong tude va ue first, fo owed by the at tude va ue

More Info  In addition to the intersection of X (or latitude) and Y (or longitude) c­ oordinates, SQL Server supports two optional data values that you can associate with each point to represent elevation: the Z value (height) and M value (measure). Both of these values are user-defined and, if supplied, are stored and retrieved with the spatial data instance. However, elevation is not considered in any spatial calculations; it merely tags along as metadata in the coordinates. WKT s SQL Server’s defau t spat a format, so str ng tera s expressed us ng WKT syntax can be ass gned d rect y nto a geometry or geography data type S m ar y, spat a data s returned n WKT format when you nvoke the ToString method on a geometry or geography data type For examp e, the fo ow ng statement parses a WKT str ng and oads t nto the geometry data type stored n @line DECLARE @line geometry = 'LINESTRING(5 15, 22 10)'

Th s next statement shows the raw b nary data for the shape n @line, and a so uses ToString to convert t back nto WKT for d sp ay SELECT @line AS AsRaw, @line.ToString() AS AsWKT

Here s the output AsRaw AsWKT ---------------------------------------------------------------- -----------------------0x00000000011400000000000014400000000000002E40000000000000364... LINESTRING (5 15, 22 10)

The STGeomFromText and STxxxFromText Methods The geometry and geography data types a so prov de a set of stat c methods for exp c t y ­mport ng any shape—or spec fic shapes—expressed n WKT The STGeomFromText method accepts any ­WKT-expressed shape and mports t as a geometry or geography type In fact, SQL Server actua y ca ed STGeomFromText when you just supp ed the str ng tera n the prev ous ass gnment Thus, the fo ow ng ass gnment s equ va ent DECLARE @line geometry = geometry::STGeomFromText('LINESTRING(5 15, 22 10)', 0)

The STxxxFromText (for examp e, STPointFromText, STLineFromText, and STPolyFromText) methods add extra va dat on on the WKT str ng to ensure t represents the ntended shape The next ­statement uses the STLineFromText method to demonstrate yet another way of oad ng the same shape DECLARE @line geometry = geometry::STLineFromText('LINESTRING(5 15, 22 10)', 0)

372  Part II  Going Beyond Relational

Th s statement works just the same, but on y because the WKT str ng s ndeed a ne str ng For examp e, the fo ow ng statement s nva d DECLARE @line geometry = geometry::STLineFromText('POINT(10 100)', 0)

Th s statement fa s because a WKT str ng represent ng a po nt was passed to STLineFromText, wh ch s a method that can on y accept ne str ngs Msg 6522, Level 16, State 1, Line 19 A .NET Framework error occurred during execution of user-defined routine or aggregate "geometry": System.FormatException: 24142: Expected "LINESTRING" at position 1. The input has "POINT(10 1". System.FormatException: at Microsoft.SqlServer.Types.WellKnownTextReader.RecognizeToken(String token) at Microsoft.SqlServer.Types.WellKnownTextReader.ParseTaggedText(OpenGisType type) at Microsoft.SqlServer.Types.WellKnownTextReader.Read(OpenGisType type, Int32 srid) at Microsoft.SqlServer.Types.SqlGeometry.GeometryFromText(OpenGisType type, SqlChars text, Int32 srid)

Th s rather verbose error message a so revea s that the spat a data types are mp emented as common anguage runt me (CLR) types Th s means that SQL Server spat a support s conta ned ns de a NET assemb y that res des on the server, wh ch a so means that you can reference the very same ­assemb y and ut ze the same spat a types and methods n your C# and VB NET app cat ons You’ see exact y how that’s done n a samp e app cat on com ng up ater n the chapter

Importing WKB We -Known B nary (WKB) s the OGC-standard b nary equ va ent of WKT, wh ch s s ght y d fferent than the nat ve b nary format that SQL Server uses to store spat a data nterna y

The STGeomFromWKB and STxxxFromWKB Methods If your data source supp es spat a data n WKB, you can use STGeomFromWKB or one of the v­ ar ous STxxxFromWKB methods to mport t As w th WKT, there s one such method for each supported shape (for examp e, STPointFromWKB, STLineFromWKB, and STPolyFromWKB) For examp e, the fo ow ng statement mports a WKB va ue and oads t nto the geometry data type stored n @point DECLARE @point geometry = geometry::STGeomFromWKB(0x010100000000000000000059400000000000005940, 0)

Th s next statement shows the raw (SQL Server) b nary data for the shape n @point, and uses ToString to convert t back nto WKT for d sp ay SELECT @point AS AsRaw, @point.ToString() AS AsWKT

Here s the output AsRaw ---------------------------------------------0x00000000010C00000000000059400000000000005940



AsWKT --------------POINT (100 100)

Chapter 9  Geospat a Support   373

Not ce that the raw b nary va ue s s m ar to the mported WKB va ue, but the two are not actua y dent ca

Importing Geography Markup Language (GML) Geography Markup Language (GML) s another spat a data format, and was deve oped by the OGC after WKT GML s both a anguage and an open nterchange format for spat a nformat on ­throughout the Internet It uses an XML d a ect that s s gn ficant y r cher and more verbose than WKT

The GeomFromGml Method You can mport spat a data from GML nto SQL Server us ng the GeomFromGml method, as fo ows DECLARE @gml xml = '

100 100 20 180 180 180 ' DECLARE @line geometry = geometry::GeomFromGml(@gml, 0) SELECT @line AS AsRaw, @line.ToString() AS AsWKT

Here s the output AsRaw AsWKT --------------------------------------------------- -----------------------0x0000000001040300000000000000000059400000000000... LINESTRING (100 100, 20 180, 180 180)

We use WKT exc us ve y n the rest of th s chapter’s code GML s a wor d unto tse f (no pun ntended), and deta ed GML coverage s beyond the scope of th s book (the GML standard s posted on the OGC webs te at http://www.opengeospatial.org/standards/gml) The key po nt to understand s that the geometry and geography data types (and the r spat a methods) are comp ete y agnost c to the part cu ar OGC standard used for mport ng spat a data; whether t’s WKT, WKB, or GML

Spatial Data Types The geometry and geography data types are system CLR types, wh ch means they are mp emented by the NET CLR nterna y by SQL Server The geometry data type s prov ded to store and process spat a data us ng the p anar mode , whereas ts counterpart geography supports the geodet c mode One n ce th ng about the two spat a types s that most methods are supported and work the same for both types So f you’re work ng w th the geodet c mode , you’ be us ng ong tude and at tude coord nates w th the geography data type For the p anar mode , you’ be us ng x- and y-coord nates w th the geometry data type But n e ther case, you’ then use many of the same methods for dea ng w th your spat a ent t es

374  Part II  Going Beyond Relational

Working with geometry Our first examp e demonstrates the geometry data type n a very s mp e scenar o You w define and store shapes represent ng town d str cts and avenues, and then query that data to return nterest ng nformat on, such as a st of the avenues that run through each d str ct Th s tt e, one-horse town has on y three d str cts and two ma n avenues, and s sma enough to be expressed on a p anar (flat) surface us ng the geometry data type As you progress through the examp e, you’ earn how to use a number of common geospat a methods F gure 9-4 shows a map of the town, and the xy-coord nates for the po nts of each shape on the map The three d str cts are po ygons (c osed shapes), and the two avenues are ne str ngs You w store these shapes n a SQL Server database us ng the geometry data type, express ng the r coord nates n WKT syntax Then you’ query and man pu ate the d str cts and streets us ng var ous spat a methods 0, 0

Downtown

150, 0

Green Park

300, 0

50, 50

100, 100 Bea

ch S tree

t

0, 150

150, 150 Harborside

20, 180

First Avenue

300, 150

180, 180

150, 300

300, 300

Figure 9-4  Sma town map on a flat (p anar) surface.

The first th ng you need to do s create tab es to ho d the shapes that define the d str cts and streets The District tab e w store the po ygons represent ng Downtown, Green Park, and Harbors de n a geometry co umn named DistrictGeo S m ar y, you’ create a Street tab e to store the ne str ngs represent ng Beach Street and F rst Avenue n a geometry co umn named StreetGeo, as shown n ­L st ng 9-1



Chapter 9  Geospat a Support   375

Listing 9-1  Creat ng the District and Street tab es w th geometry co umns.

USE master GO IF EXISTS(SELECT name FROM sys.databases WHERE name = 'MyDB') DROP DATABASE MyDB GO CREATE DATABASE MyDB GO USE MyDB GO CREATE TABLE District (DistrictId int PRIMARY KEY, DistrictName nvarchar(20), DistrictGeo geometry) CREATE TABLE Street (StreetId int PRIMARY KEY, StreetName nvarchar(20), StreetGeo geometry)

Next, popu ate the District tab e w th the three d str cts a ong w th the r shapes, s zes, and ­coord nates as po ygons n WKT, as shown n L st ng 9-2 Listing 9-2  Us ng WKT to nsert po ygons nto geometry co umns.

INSERT INTO District VALUES (1, 'Downtown', 'POLYGON ((0 0, 150 0, 150 -150, 0 -150, 0 0))'), (2, 'Green Park', 'POLYGON ((300 0, 150 0, 150 -150, 300 -150, 300 0))'), (3, 'Harborside', 'POLYGON ((150 -150, 300 -150, 300 -300, 150 -300, 150 -150))')

Not ce how the square shapes represent ng each d str ct are conveyed n WKT format as po ygon e ements Each po nt connect ng the nes of the po ygon s expressed as an xy-coord nate ­Un ke ne str ngs (wh ch you’ use short y to define the streets), a po ygon a ways represents a c osed shape In WKT, the coord nate for the fina po nt n any po ygon must be the same coord nate used for the start ng po nt n order to c ose the shape You can see how a of the d str ct po ygons n the ­preced ng code c ose the r shapes n th s manner If you attempt to express a po ygon w thout c os ng the shape, you w rece ve a FormatException, as fo ows Msg 6522, Level 16, State 1, Line 81 A .NET Framework error occurred during execution of user-defined routine or aggregate "geometry": System.FormatException: 24119: The Polygon input is not valid because the start and end points of the exterior ring are not the same. Each ring of a polygon must have the same start and end points. System.FormatException: at Microsoft.SqlServer.Types.GeometryValidator.ValidatePolygonRing(Int32 iRing, Int32 cPoints, Double firstX, Double firstY, Double lastX, Double lastY) at Microsoft.SqlServer.Types.Validator.Execute(Transition transition) at Microsoft.SqlServer.Types.ForwardingGeoDataSink.EndFigure()

376  Part II  Going Beyond Relational

at Microsoft.SqlServer.Types.WellKnownTextReader.ParseLineStringText() at Microsoft.SqlServer.Types.WellKnownTextReader.ParsePolygonText() at Microsoft.SqlServer.Types.WellKnownTextReader.ParseTaggedText(OpenGisType type) at Microsoft.SqlServer.Types.WellKnownTextReader.Read(OpenGisType type, Int32 srid) at Microsoft.SqlServer.Types.SqlGeometry.GeometryFromText(OpenGisType type, SqlChars text, Int32 srid) at Microsoft.SqlServer.Types.SqlGeometry.Parse(SqlString s) . The statement has been terminated.

Th s rather unfr end y error message a so a udes to the fact that po ygons can have mu t p e r ngs defined ns de the exter or r ng Th s means that you can define comp ex po ygons that have one or more “ho es” ns de them us ng WKT Now popu ate the Street tab e w th ne str ngs, as shown n L st ng 9-3 Listing 9-3  nsert ng ne str ngs nto geometry co umns.

INSERT INTO Street VALUES (1, 'First Avenue', 'LINESTRING (100 -100, 20 -180, 180 -180)'), (2, 'Beach Street', 'LINESTRING (300 -300, 300 -150, 50 -50)')

The ne str ng e ements used to store the streets of the town spec fy the po nts that descr be the paths of each street on the map In th s examp e, each street has three po nts express ng the two-part ne str ngs for F rst Avenue and Beach Street Now execute the un on query n L st ng 9-4 to retr eve a the shapes from both tab es comb ned nto a s ng e resu t set Listing 9-4  Retr ev ng spat a data.

SELECT Name = StreetName, AsRaw = StreetGeo, AsWKT = StreetGeo.ToString() FROM Street UNION ALL SELECT Name = DistrictName, AsRaw = DistrictGeo, AsWKT = DistrictGeo.ToString() FROM District

Here are the resu ts of the query Name -----------First Avenue Beach Street Downtown Green Park Harborside



AsRaw ----------------0x000000000104... 0x000000000104... 0x000000000104... 0x000000000104... 0x000000000104...

AsWKT -----------------------------------------------------------LINESTRING (100 -100, 20 -180, 180 -180) LINESTRING (300 -300, 300 -150, 50 -50) POLYGON ((0 0, 150 0, 150 -150, 0 -150, 0 0)) POLYGON ((300 0, 150 0, 150 -150, 300 -150, 300 0)) POLYGON ((150 -150, 300 -150, 300 -300, 150 -300, 150 -150))

Chapter 9  Geospat a Support   377

If you execute th s query n SQL Server Management Stud o (SSMS), you can v sua ze these resu ts us ng the spat a v ewer (unfortunate y, the spat a v ewer s not ava ab e ns de of V sua Stud o ­us ng SQL Server Data Too s [SSDT], so we use SSMS n th s chapter for spat a deve opment) Because SSMS recogn zes that the resu t set conta ns spat a data, t adds a new tab named Spat a Resu ts ­n-­between the Resu ts and Messages tabs C ck the Spat a Resu ts tab and you w be presented w th a graph ca render ng of the town map, as shown n F gure 9-5

Figure 9-5  D sp ay ng the town map data n SSMS us ng the Spat a Resu ts v ewer.

For a resu t set that conta ns mu t p e spat a data co umns, you can graph any one of them us ng the first drop-down st on the r ght of the chart You can a so add egend abe s based on another co umn n the resu t set us ng the second drop-down st You can a so set the chart’s ­magn ficat on eve and togg e the d sp ay of gr d nes us ng the s der and check box contro s beneath the ­drop-down sts

The STBuffer Method You can nvoke STBuffer on any spat a nstance to pad a shape by any amount (d stance), and return t back as a new nstance The query n L st ng 9-5 demonstrates the use of STBuffer to w den the streets, wh ch converts the ne str ng nto a po ygon represent ng the padded street area

378  Part II  Going Beyond Relational

Listing 9-5 Padd ng LINESTRING shapes w th the STBuffer method.

SELECT Name = StreetName, AsRaw = StreetGeo.STBuffer(5), AsWKT = StreetGeo.STBuffer(5).ToString() FROM Street UNION ALL SELECT Name = DistrictName, AsRaw = DistrictGeo, AsWKT = DistrictGeo.ToString() FROM District

The resu ts of the query show that the ne str ng shapes for F rst Avenue and Beach Street are now be ng returned as po ygon shapes represent ng streets w th w dth Name -----------First Avenue Beach Street Downtown Green Park Harborside

AsRaw ----------------0x000000000104... 0x000000000104... 0x000000000104... 0x000000000104... 0x000000000104...

AsWKT -----------------------------------------------------------POLYGON ((20.000000000000881 -184.99999999999966, 180.00... POLYGON ((300.00000000000034 -304.9999999999992, 300.243... POLYGON ((0 0, 150 0, 150 -150, 0 -150, 0 0)) POLYGON ((300 0, 150 0, 150 -150, 300 -150, 300 0)) POLYGON ((150 -150, 300 -150, 300 -300, 150 -300, 150 -150))

In SSMS, c ck the Spat a Resu ts tab to v ew the town map w th the w dened streets, as shown n F gure 9-6

Figure 9-6  V ew ng the resu t of ca ng STBuffer n the Spat a Resu ts v ewer.



Chapter 9  Geospat a Support   379

The STCentroid and STEnvelope Methods The STCentroid method returns the po nt at the center of a g ven shape, and the STEnvelope ­method returns a rectang e that bounds a g ven shape These two methods are ava ab e on y for the ­geometry data type The query n L st ng 9-6 uses STCentroid to ocate the center po nt of each d str ct, and then uses STBuffer to pad the po nt so that t appears as a sma c rc e that s c ear y v s b e n the Spat a Resu ts v ewer (Note, however, that STBuffer does not actua y generate c rcu ar str ngs, but nstead p ots many po nts n a po ygon very c ose to one another to produce the rounded padd ng shape ) Listing 9-6  Locat ng the center of a shape w th the STCentroid method.

SELECT DistrictGeo, DistrictGeo.ToString() FROM District UNION ALL SELECT DistrictGeo.STCentroid().STBuffer(5), DistrictGeo.STCentroid().ToString() FROM District

The resu ts appear as shown n F gure 9-7

Figure 9-7 Us ng STCentroid to ocate the center of each d str ct.

380  Part II  Going Beyond Relational

The query n L st ng 9-7 uses STEnvelope to dent fy the bound ng box of each street Each street s returned (aga n, padded w th STBuffer so that streets are c ear y v s b e n the Spat a Resu ts v ewer) a ong w th a rectangu ar shape around each street generated by STEnvelope Listing 9-7  Us ng the STEnvelope method to generate a bound ng box for each street.

SELECT StreetName, StreetGeo.STBuffer(5), StreetGeo.ToString() FROM Street UNION ALL SELECT StreetName + ' Bounds', StreetGeo.STEnvelope(), StreetGeo.STEnvelope().ToString() FROM Street

The resu ts appear as shown n F gure 9-8

Figure 9-8  Resu t of ca ng STEnvelope for each street.



Chapter 9  Geospat a Support   381

The STIntersects and STIntersection Methods Now you’re go ng to wr te a query to find out wh ch streets ntersect wh ch d str cts Th s query w use a CROSS JOIN so that t returns a poss b e comb nat ons of d str cts and streets You’ qua fy the query w th a WHERE c ause to fi ter the resu ts and report on y those d str ct and street ­comb nat ons that actua y ntersect w th one another Th s s eas y ach eved w th the STIntersects method You ­ nvoke STIntersects on one spat a nstance, pass ng n another spat a nstance as ts parameter, and SQL Server returns a bit (Boo ean) va ue of 1 (true) or 0 (fa se) te ng you whether or not the two geometry shapes ntersect w th each other L st ng 9-8 demonstrates Listing 9-8  Us ng the STIntersects method to find a d str ct and street ntersect ons.

SELECT S.StreetName, D.DistrictName FROM District AS D CROSS JOIN Street AS S WHERE S.StreetGeo.STIntersects(D.DistrictGeo) = 1 ORDER BY S.StreetName

When you run th s query, the resu ts nd cate that Beach Street runs through a three d str cts, whereas F rst Avenue runs through Downtown and Harbors de, but not Green Park StreetName -------------------Beach Street Beach Street Beach Street First Avenue First Avenue

DistrictName -------------------Downtown Green Park Harborside Downtown Harborside

But you can do more than just find out whether two spat a ent t es ntersect—you can us the STIntersection method to actua y obta n a shape that represents the over app ng area of ntersect on n a new spat a ent ty L st ng 9-9 presents a mod fied vers on of the same STIntersects query that a so uses STIntersection to report wh ch pieces of each road cut through each d str ct Listing 9-9  Us ng the STIntersection method to generate road fragments for each d str ct.

SELECT S.StreetName, D.DistrictName, S.StreetGeo.STIntersection(D.DistrictGeo).STBuffer(5) AS Intersection, S.StreetGeo.STIntersection(D.DistrictGeo).ToString() AS IntersectionWKT FROM District AS D CROSS JOIN Street AS S WHERE S.StreetGeo.STIntersects(D.DistrictGeo) = 1 ORDER BY S.StreetName

The resu ts of the query show the ne str ng shapes created by STIntersection for the port on of each street that runs through each d str ct 382  Part II  Going Beyond Relational

StreetName -----------Beach Street Beach Street Beach Street First Avenue First Avenue

DistrictName -----------Downtown Green Park Harborside Downtown Harborside

Intersection -------------------0x0000000001047F0... 0x0000000001047F0... 0x000000000104870... 0x000000000104870... 0x000000000104870...

IntersectionWKT -------------------------------------------LINESTRING (50 -50, 150 -89.999999999999886) LINESTRING (150.00000000000125 -90.00000... LINESTRING (300 -150, 300 -300) LINESTRING (100 -100, 50.000000000000782... LINESTRING (180 -180, 150.00000000000071...

(5 row(s) affected)

Not ce that the port ons of F rst Avenue that do not run through any of the three d str cts are exc uded from the resu ts As F gure 9-9 shows, you can see a v sua representat on of the resu ts by c ck ng the Spat a Resu ts tab n SSMS

Figure 9-9  V ew ng the road fragments of the streets n each d str ct us ng STIntersection.

The STDimension Method The STDimension method can be nvoked on any spat a nstance, and returns a number (0, 1, or 2) that nd cates how many d mens ons are defined by the shape n the nstance S ng e po nts have 0 d mens ons; nes, c rcu ar nes, and ne str ngs have 1 d mens on; and po ygons (c osed shapes) have 2 d mens ons That means that a of the d str cts n the town are two-d mens ona objects, whereas the streets are a one-d mens ona objects The code n L st ng 9-10 d sp ays the d mens ons for a the spat a objects current y stored n the database

Chapter 9  Geospat a Support   383

Listing 9-10  Us ng the STDimension method to show the number of d mens ons for each shape.

SELECT StreetName AS Shape, StreetGeo.ToString() AS ShapeWKT, StreetGeo.STDimension() AS Dimensions FROM Street UNION ALL SELECT DistrictName AS Shape, DistrictGeo.ToString() AS ShapeWKT, DistrictGeo.STDimension() AS Dimensions FROM District

Here are the resu ts Shape ------------First Avenue Beach Street Downtown Green Park Harborside

ShapeWKT -----------------------------------------------------------LINESTRING (100 -100, 20 -180, 180 -180) LINESTRING (300 -300, 300 -150, 50 -50) POLYGON ((0 0, 150 0, 150 -150, 0 -150, 0 0)) POLYGON ((300 0, 150 0, 150 -150, 300 -150, 300 0)) POLYGON ((150 -150, 300 -150, 300 -300, 150 -300, 150 -150))

Dimensions ----------1 1 2 2 2

The STUnion, STDifference, and STSymDifference Methods Our next examp e demonstrates some more man pu at ons w th over app ng reg ons between shapes The code n L st ng 9-11 defines two over app ng po ygons, and then ca s severa methods aga nst those shapes The code st ng s fo owed by a ser es of screen snapshots show ng the Spat a Resu ts v ewer render ng of each method’s resu t Listing 9-11  Compar ng the STUnion, STIntersection, STDifference, and STSymDifference methods.

DECLARE @S1 geometry = 'POLYGON ((60 40, 410 50, 400 270, 60 370, 60 40))' DECLARE @S2 geometry = 'POLYGON ((300 100, 510 110, 510 330, 300 330, 300 100))' SELECT SELECT SELECT SELECT SELECT SELECT

@S1 UNION ALL SELECT @S2 S1_UNION_S2 = @S1.STUnion(@S2) S1_INTERSECTION_S2 = @S1.STIntersection(@S2) S1_DIFFERENCE_S2 = @S1.STDifference(@S2) S2_DIFFERENCE_S1 = @S2.STDifference(@S1) S1_SYMDIFFERENCE_S2 = @S1.STSymDifference(@S2)

The first query comb nes (un ons) the two shapes so you can see what they ook ke and how they over ap, as shown n F gure 9-10

384  Part II  Going Beyond Relational

Figure 9-10  Two over app ng shapes.

STUnion returns a new shape based on the two source shapes (d sregard ng the over app ng ­port on), as shown n F gure 9-11

Figure 9-11  Comb n ng the two over app ng shapes nto one us ng STUnion.



Chapter 9  Geospat a Support   385

Ear er, you used the STIntersection method to break down one shape (a str ng) nto mu t p e shapes (by d str ct) Once aga n, STIntersection produces a s m ar resu t by creat ng a new shape represent ng just the area of over ap (the center p ece) between the two source shapes, as shown n F gure 9-12

Figure 9-12  dent fy ng the area of over ap between the two shapes us ng STIntersection.

The STDifference method “cuts away” from one shape the over app ng area of another shape F­ gure 9-13 shows the resu t of nvok ng STDifference to cut away the part of the first shape where the second shape over aps

Figure 9-13  D fferenc ng the second shape away from the first shape us ng STDifference.

386  Part II  Going Beyond Relational

F gure 9-14 shows the reverse, where STDifference s used to cut away the part of the second shape where the first shape over aps

Figure 9-14  D fferenc ng the first shape away from the second shape us ng STDifference.

F na y, the STSymDifference method returns the “symmetr ca ” d fference between the two shapes Th s s essent a y the oppos te of the STIntersection method—a new shape s generated that ­represents everyth ng except the area of over ap between the two shapes (everyth ng but the center p ece), as shown n F gure 9-15

Figure 9-15  Obta n ng the symmetr c d fference between two shapes us ng STSymDifference.



Chapter 9  Geospat a Support   387

Working with geography Most (but not a ) methods of the geometry type are ava ab e and funct on s m ar y for the ­geography type The pr mary d fference s that you must use ong tude and at tude va ues ­Interna y, as exp a ned, SQL Server automat ca y compensates for the earth’s curvature when perform ng ­ca cu at ons aga nst our geography data In th s sect on, you’ work w th the geography data type How do you get your hands on ong tudes and at tudes? There are many ways Coord nates for arge c t es and other major ocat ons n the wor d can be eas y obta ned on the Wor d W de Web w th a qu ck search and v a read y ava ab e web serv ce APIs, nc ud ng Yahoo! and Goog e The ­o d-fash oned way st works too, so p ots can acqu re them from sect ona maps, for examp e You can a so use M crosoft Streets & Tr ps 2008, wh ch has a “ ocat on sensor” too that w te you the ong tude and at tude coord nates for any po nt or shape drawn on the map us ng the mouse Th s Streets & Tr ps feature was used to obta n the coord nates for our next examp e

On Your Mark … You’ use a rea - fe event to earn about spat a area, ength, and d stance ca cu at ons In th s ­app cat on, you are mapp ng the Pro Cyc ng Tour he d n Ph ade ph a (wh ch entered ts 27th year n 2011) Us ng ong tude and at tude coord nates, you w bu d a database that stores d fferent areas of the b ke race Then you’ wr te quer es to ca cu ate the area and ength of d fferent reg ons us ng other new spat a methods The map for th s app cat on s shown n F gure 9-16 The ent re race area s conta ned n one arge 14-s ded po ygon W th n the race area, there are two popu ar ocat ons where spectators gather and take p ctures One s the Parkway Area to the south, where the race starts and fin shes, and the other s the Wa Area to the north In th s app cat on, you’ comb ne spat a features w th FILESTREAM Photos subm tted by s­ pectators w be stored n the database as b nary arge objects (BLOBs) n the fi e system us ng the techn ques you earned ear er, dur ng our FILESTREAM coverage n Chapter 8 Refer to the sect on “Enab ng FILESTREAM for the Mach ne” n that chapter, wh ch descr bes how to enab e FILESTREAM us ng SQL Server Configurat on Manager Then execute the code n L st ng 9-12 to create the FILESTREAM-enab ed database for the app cat on Note that the path to the database, C \Demo\ EventL brary n th s examp e, must ex st before the database can be created Listing 9-12  Creat ng the EventLibrary database.

USE master GO EXEC sp_configure filestream_access_level, 2 RECONFIGURE GO CREATE DATABASE EventLibrary ON PRIMARY

388  Part II  Going Beyond Relational

(NAME = EventLibrary_data, FILENAME = 'C:\Demo\EventLibrary\EventLibrary_data.mdf'), FILEGROUP FileStreamGroup1 CONTAINS FILESTREAM (NAME = EventLibrary_group2, FILENAME = 'C:\Demo\EventLibrary\Events') LOG ON (NAME = EventLibrary_log, FILENAME = 'C:\Demo\EventLibrary\EventLibrary_log.ldf') GO USE EventLibrary GO

POLYGON(( -75.22280 40.02387, -75.21442 40.02810, -75.21746 40.03142, -75.22534 40.02586, -75.22280 40.02387))

Wall Area One of the most interesting parts of the race route. Contains the infamous 17% grade climb known as the “Manayunk Wall.”

POLYGON(( -75.17031 39.95601, -75.16786 39.95778, -75.18870 39.97789, -75.18521 39.99237, -75.18603 40.00677, -75.19922 40.01136, -75.21746 40.03142, -75.22534 40.02586, -75.21052 40.01430, -75.19192 40.00634, -75.19248 39.99570, -75.20526 39.98374, -75.19437 39.97704, -75.19087 39.96920, -75.17031 39.95601))

Race Area The entire Pro Cycling Championship race route is contained inside this irregular polygon.

POLYGON(( -75.17031 39.95601, -75.16786 39.95778, -75.17921 39.96874, -75.18441 39.96512, -75.17031 39.95601))

Parkway Area The race starts and finishes here on the Benjamin Franklin Parkway.

Figure 9-16  Pro Cyc ng Tour reg on map.

Next, create the EventRegion tab e to ho d the d fferent map reg ons and then popu ate the tab e w th the po ygons represent ng the three reg ons be ng mapped, as shown n L st ng 9-13



Chapter 9  Geospat a Support   389

Listing 9-13  Creat ng the EventRegion tab e and popu at ng t w th geograph ca data.

CREATE TABLE EventRegion (RegionId int PRIMARY KEY, RegionName nvarchar(32), MapShape geography) INSERT INTO EventRegion VALUES(1, 'Parkway Area', geography::Parse('POLYGON(( -75.17031 39.95601, -75.16786 39.95778, -75.17921 39.96874, -75.18441 39.96512, -75.17031 39.95601 ))')) INSERT INTO EventRegion VALUES(2, 'Wall Area', geography::Parse('POLYGON(( -75.22280 40.02387, -75.21442 40.02810, -75.21746 40.03142, -75.22534 40.02586, -75.22280 40.02387))')) INSERT INTO EventRegion VALUES(3, 'Race Area', geography::Parse('POLYGON(( -75.17031 39.95601, -75.16786 39.95778, -75.18870 39.97789, -75.18521 39.99237, -75.18603 40.00677, -75.19922 40.01136, -75.21746 40.03142, -75.22534 40.02586, -75.21052 40.01430, -75.19192 40.00634, -75.19248 39.99570, -75.20526 39.98374, -75.19437 39.97704, -75.19087 39.96920, -75.17031 39.95601))'))

Now execute the statement SELECT * FROM EventRegion to query for the st of reg ons you just created n the EventRegion tab e F gure 9-17 shows how the query resu ts appear n the spat a v ewer

Figure 9-17  V ew ng spat a geography data n SSMS.

390  Part II  Going Beyond Relational

Not ce the th rd drop-down st abe ed Se ect Project on Th s drop-down st was not present on the spat a v ewer n our ear er quer es because they returned geometry (p anar) data Th s query returns geography (geodet c) spat a data, so the spat a v ewer a ows you to render the resu ts us ng one of severa d fferent project ons from the add t ona drop-down st By defau t, the v ewer uses the Equ rectangu ar project on, but you can change that to observe the d fferent ways that geodet c spat a data s skewed when flatten ng the g obe us ng d fferent flat-earth project on mode s For ­examp e, change the th rd drop-down st to use the Mercator project on As shown n F gure 9-18, the same geodet c data gets skewed to a s ght y d fferent shape when flattened use the Mercator ­project on (not ce how t’s narrower than the Equ ractangu ar project on n F gure 9-17)

Figure 9-18  V ew ng the same spat a data us ng the Mercator project on.

The STArea and STLength Methods Area and ength (per meter) ca cu at ons are eas y performed us ng the STArea and STLength ­methods The query n L st ng 9-14 d sp ays the area and ength of each reg on, sorted from argest to sma est Listing 9-14  Ca cu at ng area and ength us ng the STArea and STLength methods.

SELECT RegionName, ROUND(MapShape.STArea(), 2) AS Area, ROUND(MapShape.STLength(), 2) AS Length



Chapter 9  Geospat a Support   391

FROM EventRegion ORDER BY MapShape.STArea() DESC

Here are the resu ts RegionName -------------------------------Race Area Parkway Area Wall Area

Area ---------------------6432902.35 689476.79 334024.82

Length ---------------------22165.07 4015.39 2529.11

(3 row(s) affected)

Spatial Reference IDs The first quest on you’re bound to ask when v ew ng these resu ts s, what un t of measurement does SQL Server use to express the area and ength? The answer s, t depends on the Spat a Reference ID (SRID) of the spat a data type nstance By defau t, geography nstances use an SRID va ue of 4326, wh ch s based on the metr c system Therefore, the area and ength resu ts shown n the preced ng code are g ven n square meters and meters, respect ve y You can execute the statement SELECT * FROM sys.spatial reference systems to obta n a st of a the SRIDs supported by SQL Server Some of the SRIDs support d fferent measurement systems, nc ud ng foot, C arke’s foot, Ind an foot, U S survey foot, and German ega meter Ca cu at ons ­performed between two spat a nstances requ re both nstances to use the same SRID See Books On ne for more nformat on about SRIDs

Building Out the EventLibrary Database Now t’s t me to create the EventPhoto tab e, as shown n L st ng 9-15 The RowId and Photo ­co umns, respect ve y, prov de the ROWGUIDCOL and varbinary(max) FILESTREAM co umns requ red for FILESTREAM storage and access to b nary photo fi es, as we exp ored n depth n Chapter 8 The ­Location co umn stores the po nt ( ong tude and at tude) where the photo was taken n a geography data type Th s tab e w be popu ated w th severa photos us ng the very same techn ques you used n the photo brary FILESTREAM app cat on from Chapter 8 Listing 9-15  Creat ng the EventPhoto tab e to ho d photos and the ocat ons where they were taken.

CREATE TABLE EventPhoto ( PhotoId int PRIMARY KEY, RowId uniqueidentifier ROWGUIDCOL NOT NULL UNIQUE DEFAULT NEWSEQUENTIALID(), Description varchar(max), Location geography, Photo varbinary(max) FILESTREAM DEFAULT(0x))

392  Part II  Going Beyond Relational

You’re about to create a W ndows Forms app cat on n C# that a ows the user to se ect a reg on and retr eve a the photos taken n that reg on Once aga n, us ng the same FILESTREAM techn ques you’ve a ready earned for the PhotoLibrary app cat on n Chapter 8, the user can then se ect a photo for d sp ay n a p cture box contro To support th s app cat on, you’ wr te severa stored procedures (see L st ng 9-16) GetRegions s ca ed to retr eve a st of a the reg on IDs and names, wh ch gets bound to a combo box for the user to make a reg on se ect on GetRegionPhotos accepts the ID of the reg on se ected by the user and returns a st of a the photo IDs and descr pt ons of p ctures taken n the se ected reg on, wh ch gets bound to a st box for the user to make a photo se ect on The photo st s obta ned by u ­ s ng the STIntersects method you earned about ear er n th s chapter w th the geometry type Last, ­GetPhotoForFilestream returns the PathName and GET FILESTREAM TRANSACTION CONTEXT va ues for us ng SqlFileStream to retr eve and d sp ay the photo se ected by the user Listing 9-16  Creat ng the stored procedures for the samp e event med a app cat on.

-- Present list of regions to user CREATE PROCEDURE GetRegions AS BEGIN SELECT RegionId, RegionName FROM EventRegion END GO -- Get all the photos taken in the selected region CREATE PROCEDURE GetRegionPhotos(@RegionId int) AS BEGIN DECLARE @MapShape geography -- Get the shape of the region SELECT @MapShape = MapShape FROM EventRegion WHERE RegionId = @RegionId -- Get all photos taken in the region SELECT PhotoId, Description FROM EventPhoto WHERE Location.STIntersects(@MapShape) = 1 END GO -- Get the SqlFileStream information for retrieving the selected photo CREATE PROCEDURE GetPhotoForFilestream(@PhotoId int) AS BEGIN -- Called by ADO.NET client during open transaction to use SqlFileStream SELECT Photo.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT()



Chapter 9  Geospat a Support   393

FROM WHERE

EventPhoto PhotoId = @PhotoId

END GO

Creating the Event Media Client Application To create the c ent app cat on, start M crosoft V sua Stud o 2010, and then create a new C# W ndows Forms app cat on named EventMediaSpatialApp Des gn a form named PhotoForm w th a combobox for se ect ng a reg on, a nk abe to search for photos by reg on, and a st box for d sp ay ng the resu ts A so nc ude a p cture box contro for d sp ay ng the photo se ected from the st box and a nk abe that you’ use to bu k oad four photo fi es nto the EventPhoto tab e After perform ng some aesthet c a gnment and formatt ng, your form shou d appear someth ng ke the one shown n F gure 9-19

Figure 9-19  EventMedia W ndows user nterface (U ) form.

L st ng 9-17 conta ns the comp ete code beh nd the form, wh ch ca s the stored procedures you created n L st ng 9-16 to popu ate the combobox w th reg ons, query for photos taken n any se ected reg on, and then d sp ay any photo se ected from that reg on When a photo s se ected, t streams the mage from the FILESTREAM co umn n the database to the p cture box contro ­us ng SqlFileStream There are a so methods that use SqlFileStream to oad geocoded mages nto the ­database The SqlFileStream code requ res no deta ed exp anat on here; t fo ows the prec se pattern we deta ed n Chapter 8 to save and oad the BLOB mages to and from the database Listing 9-17  EventMedia c ent app cat on code.

using using using using

System; System.Data; System.Data.SqlClient; System.Data.SqlTypes;

394  Part II  Going Beyond Relational

using using using using

System.Drawing; System.IO; System.Transactions; System.Windows.Forms;

namespace EventMediaSpatialApp { public partial class PhotoSearchForm : Form { private const string ConnStr = "Data Source=.;Integrated Security=True;Initial Catalog=EventLibrary;"; public PhotoSearchForm() { InitializeComponent(); } protected override void OnLoad(EventArgs e) { base.OnLoad(e); this.LoadRegions(); } private void lnkAddPhotos_LinkClicked (object sender, LinkLabelLinkClickedEventArgs e) { this.AddPhotos(); } private void lnkSearch_LinkClicked (object sender, LinkLabelLinkClickedEventArgs e) { this.FindRegionPhotos(); } private void lstPhotos_SelectedIndexChanged(object sender, EventArgs e) { this.DisplayPhoto(); } private void LoadRegions() { using (SqlDataAdapter adp = new SqlDataAdapter("GetRegions", ConnStr)) { adp.SelectCommand.CommandType = CommandType.StoredProcedure; DataSet ds = new DataSet(); adp.Fill(ds); this.cboRegions.DataSource = ds.Tables[0]; this.cboRegions.ValueMember = "RegionId"; this.cboRegions.DisplayMember = "RegionName"; } }



Chapter 9  Geospat a Support   395

private void FindRegionPhotos() { this.lstPhotos.SelectedIndexChanged -= new System.EventHandler(this.lstPhotos_SelectedIndexChanged); int regionId = (int)this.cboRegions.SelectedValue; using (SqlDataAdapter adp = new SqlDataAdapter("GetRegionPhotos", ConnStr)) { adp.SelectCommand.CommandType = CommandType.StoredProcedure; adp.SelectCommand.Parameters.AddWithValue("@RegionId", regionId); DataSet ds = new DataSet(); adp.Fill(ds); this.lstPhotos.DataSource = ds.Tables[0]; this.lstPhotos.ValueMember = "PhotoId"; this.lstPhotos.DisplayMember = "Description"; } this.lstPhotos.SelectedIndexChanged += new System.EventHandler(this.lstPhotos_SelectedIndexChanged); this.DisplayPhoto(); } private void DisplayPhoto() { int photoId = (int)this.lstPhotos.SelectedValue; this.picImage.Image = this.GetPhoto(photoId); } private Image GetPhoto(int photoId) { Image photo; using (TransactionScope ts = new TransactionScope()) { using (SqlConnection conn = new SqlConnection(ConnStr)) { conn.Open(); string filePath; byte[] txnToken; using (SqlCommand cmd = new SqlCommand("GetPhotoForFilestream", conn)) { cmd.CommandType = CommandType.StoredProcedure; cmd.Parameters.Add("@PhotoId", SqlDbType.Int).Value = photoId; using (SqlDataReader rdr = cmd.ExecuteReader(CommandBehavior. SingleRow)) { rdr.Read(); filePath = rdr.GetSqlString(0).Value; txnToken = rdr.GetSqlBinary(1).Value; rdr.Close(); } }

396  Part II  Going Beyond Relational

photo = this.LoadPhotoImage(filePath, txnToken); } ts.Complete(); } return photo; } private Image LoadPhotoImage(string filePath, byte[] txnToken) { Image photo; using (SqlFileStream sfs = new SqlFileStream(filePath, txnToken, FileAccess.Read)) { photo = Image.FromStream(sfs); sfs.Close(); } return photo; } private void AddPhotos() { this.InsertEventPhoto(1, "Taken from the Ben Franklin parkway near the finish line", -75.17396, 39.96045, "bike9_2.jpg"); this.InsertEventPhoto(2, "This shot was taken from the bottom of the Manayunk Wall", -75.22457, 40.02593, "wall_race_2.jpg"); this.InsertEventPhoto(3, "This shot was taken at the top of the Manayunk Wall.", -75.21986, 40.02920, "wall_race2_2.jpg"); this.InsertEventPhoto(4, "This is another shot from the Benjamin Franklin Parkway.", -75.17052, 39.95813, "parkway_area2_2.jpg"); MessageBox.Show("Added 4 photos to database"); } private void InsertEventPhoto( int photoId, string desc, double longitude, double latitude, string photoFile) { const string InsertTSql = @" INSERT INTO EventPhoto(PhotoId, Description, Location) VALUES(@PhotoId, @Description, geography::STGeomFromText(@Location, 4326)) SELECT Photo.PathName(), GET_FILESTREAM_TRANSACTION_CONTEXT() FROM EventPhoto WHERE PhotoId = @PhotoId";



Chapter 9  Geospat a Support   397

const string PointMask = "POINT ({0} {1})"; string location = string.Format(PointMask, longitude, latitude); string serverPath; byte[] serverTxn; using (TransactionScope ts = new TransactionScope()) { using (SqlConnection conn = new SqlConnection(ConnStr)) { conn.Open(); using (SqlCommand cmd = new SqlCommand(InsertTSql, conn)) { cmd.Parameters.Add("@PhotoId", SqlDbType.Int).Value = photoId; cmd.Parameters.Add("@Description", SqlDbType.NVarChar).Value = desc; cmd.Parameters.Add("@Location", SqlDbType.NVarChar).Value = location; using (SqlDataReader rdr = cmd.ExecuteReader(CommandBehavior. SingleRow)) { rdr.Read(); serverPath = rdr.GetSqlString(0).Value; serverTxn = rdr.GetSqlBinary(1).Value; rdr.Close(); } } this.SavePhotoFile(photoFile, serverPath, serverTxn); } ts.Complete(); } } private void SavePhotoFile (string photoFile, string serverPath, byte[] serverTxn) { const string LocalPath = @"..\..\Photos\"; const int BlockSize = 1024 * 512; using (FileStream source = new FileStream(LocalPath + photoFile, FileMode.Open, FileAccess.Read)) { using (SqlFileStream dest = new SqlFileStream(serverPath, serverTxn, FileAccess.Write)) { byte[] buffer = new byte[BlockSize]; int bytesRead; while ((bytesRead = source.Read(buffer, 0, buffer.Length)) > 0) { dest.Write(buffer, 0, bytesRead); dest.Flush(); } dest.Close(); } source.Close();

398  Part II  Going Beyond Relational

} } } }

The code beh nd the Add Photos nk abe c ck event nserts four photos and the r ong tude and at tude coord nates nto the database Two of the photos were taken n the Wa Area, and the other two were taken n the Parkway Area Most phone cameras and many d g ta cameras automat ca y record GPS nformat on w th each d g ta mage fi e (that s, they geocode the mage) You can use the GetPropertyItem method of the Image c ass to extract the geocoded nformat on from these d g ta mages to obta n the ocat on where they were taken Our examp e s mp y uses hard-coded ocat on va ues n the AddPhotos method

Note  As with all code in this book, the full EventMedia demo code in this chapter is ­available on the book’s companion website (see the “Introduction” for details). Included with the code are the four photo files named bike9 2.jpg, wall race 2.jpg, wall race2 2. jpg, and parkway area2 2.jpg. If you do not have access to these images, just use any .jpg ­image files you have available—doing so won’t change the point of the demo. Run the app cat on, and c ck Add Photos to popu ate the EventPhoto tab e Then se ect d fferent reg ons from the drop-down st to test out the query Based on the STIntersects method n the stored ­procedure, the app cat on sts two of the p ctures for the Wa Area, the other two p ctures for the ­Parkway Area, and a four p ctures for the Race Area (wh ch encompasses both of the other reg ons) ­Se ect ng any photo streams the mage from the database nto the p cture box, as shown n F gure 9-20

Figure 9-20  Perform ng ocat on based photo search ng and stream ng.



Chapter 9  Geospat a Support   399

The STDistance Method You use the STDistance method for d stance ca cu at ons By jo n ng the EventPhoto tab e to tse f on PhotoId, you can nvoke th s method aga nst every comb nat on of any two p ctures, as shown n ­L st ng 9-18 The va ue returned by STDistance s the exact d stance between the two p ctures expressed n meters, so d v d ng that va ue by 1000 converts the resu t to k ometers (As ment oned ear er, the defau t SRID for the geography data type uses the metr c system ) Listing 9-18 Us ng STDistance to ca cu ate the d stance between any two photos.

SELECT P1.PhotoId AS Photo1, P2.PhotoId AS Photo2, ROUND(P1.Location.STDistance(P2.location) / 1000, 2) AS Km FROM EventPhoto AS P1 JOIN EventPhoto AS P2 ON P1.PhotoId < P2.PhotoId ORDER BY P1.PhotoId

Here are the query resu ts Photo1 ----------1 1 1 2 2 3

Photo2 ----------2 3 4 3 4 4

Km ---------------------8.46 8.58 0.39 0.54 8.83 8.95

(6 row(s) affected)

Because the jo n s based on P1.PhotoId < P2.PhotoId, you don’t get resu ts for the d stance ­ etween a photo and tse f (wh ch s a ways zero), and you a so fi ter out dup cate “oppos te b ­d rect on” resu ts For examp e, the d stance from photo 1 to photo 3 s the same as the d stance from photo 3 to photo 1, so t doesn’t need to be repeated n the resu t set

Spatial Enhancements in SQL Server 2012 SQL Server 2012 adds many s gn ficant mprovements to the spat a support that was first ­ntroduced w th SQL Server 2008 In th s sect on, we’ exp ore some of these atest enhancements ­Part cu ar y notab e s support for curves (arcs), whereas SQL Server 2008 on y supported stra ght nes, or ­po ygons composed of stra ght nes As you’ see, M crosoft a so prov des methods that test for ­non-2012-compat b e (curved) shapes, and convert c rcu ar data to ne data for backward ­compat b ty w th SQL Server 2008 (as we as other mapp ng p atforms that don’t support curves)

400  Part II  Going Beyond Relational

New Spatial Data Classes The three new shapes n SQL Server 2012 are c rcu ar str ngs, compound curves, and curve po ygons A three are supported n WKT, WKB, and GML by both the geometry and geography data types, and a of the ex st ng methods work on a of the new shapes

Circular Strings A c rcu ar str ng defines a bas c curved ne, s m ar to how a ne str ng defines a stra ght ne It takes a m n mum of three coord nates to define a c rcu ar str ng; the first and th rd coord nates define the end po nts of the ne, and the second coord nate (the “anchor” po nt, wh ch es somewhere ­between the end po nts) determ nes the arc of the ne F gure 9-21 shows the shape represented by ­CIRCULARSTRING(0 1, .25 0, 0 -1)

Figure 9-21  A s mp e c rcu ar str ng.

The code n L st ng 9-19 produces four c rcu ar str ngs A of them have the same start and end po nts, but d fferent anchor po nts The nes are buffered s ght y to make them eas er to see n the spat a v ewer Listing 9-19  C rcu ar str ngs w th d fferent anchor po nts.

-- Create a "straight" circular line SELECT geometry::Parse('CIRCULARSTRING(0 8, 4 UNION ALL -- Curve it SELECT geometry::Parse('CIRCULARSTRING(0 8, 4 UNION ALL -- Curve it some more SELECT geometry::Parse('CIRCULARSTRING(0 8, 4 UNION ALL -- Curve it in the other direction SELECT geometry::Parse('CIRCULARSTRING(0 8, 4

0, 8 -8)').STBuffer(.1) 4, 8 -8)').STBuffer(.1) 6, 8 -8)').STBuffer(.1) -6, 8 -8)').STBuffer(.1)

F gure 9-22 shows the generated shapes from the prev ous code The first shape s a “stra ght c­ rcu ar” ne, because the anchor po nt s pos t oned d rect y between the start and end po nts The next two shapes use the same end po nts w th the anchor out to the r ght (4), one a b t further than



Chapter 9  Geospat a Support   401

the other (6) The ast shape a so uses the same end po nts, but spec fies an anchor po nt that curves the ne to the eft rather than the r ght (-6)

Figure 9-22  A set of c rcu ar str ngs, nc ud ng one that s perfect y stra ght.

You can extend c rcu ar str ngs w th as many curve segments as you want Do th s by defin ng another two coord nates for each add t ona segment The ast po nt of the prev ous curve serves as the first end po nt of the next curve segment, so the two add t ona coord nates respect ve y spec fy the next segment’s anchor and second end po nt Thus, va d c rcu ar str ngs w a ways have an odd number of po nts You can extend a c rcu ar str ng ndefin te y to form curves and arcs of any k nd It’s easy to form a perfect c rc e by connect ng two sem -c rc e segments For examp e, F gure 9-23 shows the c rc e produced by CIRCULARSTRING(0 4, 4 0, 8 4, 4 8, 0 4). Th s part cu ar examp e ­connects the end of the second segment to the beg nn ng of the first segment to form a c osed shape Note that th s s certa n y not requ red of c rcu ar str ngs (or ne str ngs), and that c os ng the shape by connect ng the ast segment to the first st does not resu t n a po ygon, wh ch s a ­t wo-d mens ona shape that has area Desp te be ng c osed, th s c rc e s st cons dered a one-­ d mens ona shape w th no area As you’ soon see, the curve po ygon can be used to convert c osed ne shapes nto true po ygons

Figure 9-23  Two sem c rc e segments nked together n one c rcu ar str ng.

402  Part II  Going Beyond Relational

Compound Curves A compound curve s a set of c rcu ar str ngs, or c rcu ar str ngs comb ned w th ne str ngs, that form a des red curved shape The end po nt of each e ement n the co ect on must match the start ng po nt of the fo ow ng e ement, so that compound curves are defined n a “connect-the-dots” fash on The code n ­L st ng 9-20 produces a compound curve and compares t w th the equ va ent geometry ­co ect on shape Listing 9-20  Compound curve shapes.

-- Compound curve DECLARE @CC geometry = ' COMPOUNDCURVE( (4 4, 4 8), CIRCULARSTRING(4 8, 6 10, 8 8), (8 8, 8 4), CIRCULARSTRING(8 4, 2 3, 4 4) )' -- Equivalent geometry collection DECLARE @GC geometry = ' GEOMETRYCOLLECTION( LINESTRING(4 4, 4 8), CIRCULARSTRING(4 8, 6 10, 8 8), LINESTRING(8 8, 8 4), CIRCULARSTRING(8 4, 2 3, 4 4) )' -- They both render the same shape in the spatial viewer SELECT @CC.STBuffer(.5) UNION ALL SELECT @GC.STBuffer(1.5)

Th s code creates a keyho e shape us ng a compound curve, and a so creates an dent ca shape as a geometry co ect on (though not ce that the LINESTRING keyword s not—and cannot—be spec fied when defin ng a compound curve) It then buffers both of them w th d fferent padd ng, so that the spat a v ewer c ear y shows the two dent ca shapes on top of one another, as shown n F gure 9-24

Figure 9-24  A compound curve over an dent ca y shaped geometry co ect on.



Chapter 9  Geospat a Support   403

Both the compound curve and the geometry co ect on y e d dent ca shapes In fact, the ­express on @CC.STEquals(@GC), wh ch compares the two nstances for equa ty, returns 1 (for true) The STEquals method tests for “spat a equa ty,” mean ng t returns true f two nstances produce the same shape even f they are be ng rendered us ng d fferent spat a data c asses Furthermore, reca from L st ng 9-19 and F gure 9-22 that segments of a c rcu ar str ng can be made perfect y stra ght by pos t on ng the anchor d rect y between the end po nts, mean ng that the c rcu ar str ng offers yet a th rd opt on for produc ng the very same shape So wh ch one shou d you use? Compar ng these spat a data c asses w he p you determ ne wh ch one s best to use n d fferent scenar os A geometry co ect on (wh ch was a ready supported n SQL Server 2008) s the most accommodat ng, but carr es the most storage overhead Geometry co ect ons can ho d nstances of any spat a data c ass, and the nstances don’t need to be connected to (or ntersected w th) each other n any way The ­co ect on s mp y ho ds a bunch of d fferent shapes as a set, wh ch n th s examp e just happens to be severa ne str ngs and c rcu ar str ngs connected at the r start and end po nts In contrast, the new compound curve c ass n SQL Server 2012 has the most constra nts but s the most ghtwe ght n terms of storage It can only conta n ne str ngs or c rcu ar str ngs, and each s­ egment’s start po nt must be connected to the prev ous segment’s end po nt (a though t s most certa n y not ­necessary to connect the first and ast segments to form a c osed shape as n th s e ­ xamp e) The DATALENGTH ­funct on shows the d fference n storage requ rements; DATALENGTH(@CC) returns 152 and DATALENGTH(@GC) returns 243 Th s means that the same shape requ res 38% ess storage space by us ng a compound curve nstead of a geometry co ect on A compound curve s a so more storage-effic ent than a mu t -segment c rcu ar ne str ng when stra ght nes are nvo ved Th s s because there s overhead for the mere potent a of a curve, because the anchor po nt requ res storage even when ts pos t on produces stra ght nes, whereas compound curves are opt m zed spec fica y to connect c rcu ar str ngs and (a ways stra ght) ne str ngs

Curve Polygons A curve po ygon s very s m ar to an ord nary po ygon; ke an ord nary po ygon, a curve po ygon spec fies a “r ng” that defines a c osed shape, and can a so spec fy add t ona nner r ngs to define “ho es” ns de the shape The on y fundamenta d fference between a po ygon and a curve ­po ygon s that the r ngs of a curve po ygon can nc ude c rcu ar shapes, whereas an ord nary po ygon s composed exc us ve y w th stra ght nes Spec fica y, each r ng n a curve po ygon can cons st of any comb nat on of ne str ngs, c rcu ar str ngs, and compound curves that co ect ve y define the c osed shape For examp e, the code n L st ng 9-21 produces a curve po ygon w th the same keyho e out ne that we just demonstrated for the compound curve Listing 9-21  A curve po ygon.

-- Curve polygon SELECT geometry::Parse(' CURVEPOLYGON( COMPOUNDCURVE( (4 4, 4 8),

404  Part II  Going Beyond Relational

CIRCULARSTRING(4 8, 6 10, 8 8), (8 8, 8 4), CIRCULARSTRING(8 4, 2 3, 4 4) ) )')

Th s code has s mp y spec fied the same compound curve as the c osed shape of a curve ­po ygon A though the shape s the same, the curve po ygon s a two-d mens ona object, whereas the ­compound curve vers on of the same shape s a one-d mens ona object Th s can be seen v sua y by the spat a v ewer resu ts, wh ch shades the nter or of the curve po ygon as shown n F gure 9-25

Figure 9-25  A curve po ygon.

New Spatial Methods Let’s now exp ore some of the new spat a methods n SQL Server 2012 Some of these new methods comp ement the new curved shapes, whereas others add new spat a features that work w th a shapes

The STNumCurves and STCurveN Methods These two methods can be nvoked on any geometry or geography nstance They can be used together to d scover nformat on about the curves conta ned w th n the spat a nstance The ­STNumCurves method returns the tota number of curves n the nstance You can then pass any number between 1 and what STNumCurves returns to extract each nd v dua curve, and thus terate a the curves n the nstance L st ng 9-22 demonstrates these two methods w th the same c rc e that you saw n F gure 9-23 Reca that th s s a c rcu ar str ng w th two connected segments The c rc e s defined by the two ha ves, each of wh ch s a curved ne defin ng a sem -c rc e Listing 9-22 Us ng STNumCurves and STCurveN to obta n curve nformat on from a c rcu ar str ng.

-- Create a full circle shape (two connected semi-circles) DECLARE @C geometry = 'CIRCULARSTRING(0 4, 4 0, 8 4, 4 8, 0 4)'



Chapter 9  Geospat a Support   405

-- Get the curve count (2) and the 1st curve (bottom semi-circle) SELECT CurveCount = @C.STNumCurves(), SecondCurve = @C.STCurveN(2), SecondCurveWKT = @C.STCurveN(2).ToString()

Th s query produces the fo ow ng output CurveCount SecondCurve SecondCurveWKT ---------- ----------------------------------------------- ------------------------------2 0x000000000204030000000000000000002040000000... CIRCULARSTRING (8 4, 4 8, 0 4)

You can see that STNumCurves nd cates there are two curves, and that STCurveN(2) returns the second curve If you v ew the resu ts n the spat a v ewer, you’ see just the top ha f of the c rc e Th s s the sem -c rc e defined by the second curve, wh ch s converted back to WKT as CIRCULARSTRING (8 4, 4 8, 0 4) Not ce that th s represents the second segment of the fu c rc e n L st ng 9-21

The BufferWithCurves Method You’ve a ready earned how the STBuffer method can “pad” a ne, effect ve y convert ng t nto a ­po ygon To repr se our “sma town” demo, STBuffer was used n L st ng 9-5 to w den the town streets, and the output from that code st ng c ear y showed that SQL Server generated po ygons from the ne str ngs If you ook c ose y at the resu t ng po ygon shapes n the spat a v ewer (see F gure 9-6), t appears that po nts of each ne str ng ( nc ud ng the m dpo nts) are transformed nto rounded edges n the po ygon However, the rounded edge ook s actua y produced by p ott ng many short stra ght nes that are c ustered very c ose y together, present ng the us on of a curve Th s approach s ­because curves were not prev ous y supported before SQL Server 2012 (but the STBuffer method was) C ear y, us ng nat ve curve defin t ons n a curve po ygon s more effic ent than c uster ng a ­mu t tude of stra ght nes n an ord nary po ygon For backward compat b ty, STBuffer ­cont nues to return the ( neffic ent) po ygon as before So SQL Server 2012 ntroduces a new method, ­BufferWithCurves, for th s purpose The code n L st ng 9-23 uses BufferWithCurves to pad the street nes as before, and compares the resu t w th ts stra ght- ne cous n, STBuffer Listing 9-23 Compar ng BufferWithCurves and STBuffer.

DECLARE @streets geometry = ' GEOMETRYCOLLECTION( LINESTRING (100 -100, 20 -180, 180 -180), LINESTRING (300 -300, 300 -150, 50 -50) )' SELECT @streets.BufferWithCurves(10) SELECT AsWKT = @streets.ToString(),

406  Part II  Going Beyond Relational

Bytes = DATALENGTH(@streets), Points = @streets.STNumPoints() UNION ALL SELECT @streets.STBuffer(10).ToString(), DATALENGTH(@streets.STBuffer(10)), @streets.STBuffer(10).STNumPoints() UNION ALL SELECT @streets.BufferWithCurves(10).ToString(), DATALENGTH(@streets.BufferWithCurves(10)), @streets.BufferWithCurves(10).STNumPoints()

F gure 9-26 shows the resu t ng shape (the co ect on of padded street shapes) returned by the first SELECT statement n the spat a v ewer

Figure 9-26 Us ng BufferWithCurves to pad a ne str ng w th true rounded edges.

As w th STBuffer back n F gure 9-6, the new shapes have rounded edges around the po nts of the or g na ne str ngs However, BufferWithCurves generates actua curves and, thus, produces a ­s gn ficant y sma er and s mp er po ygon The second SELECT statement n L st ng 9-23 demonstrates th s by compar ng the three shapes—the or g na ne str ng co ect on, the po ygon returned by ­STBuffer, and the curve po ygon returned by BufferWithCurves Here are the resu ts AsWKT -------------------------------------------------------------------------GEOMETRYCOLLECTION (LINESTRING (100 -100, 20 -180, 180 -180), LINESTRIN... MULTIPOLYGON (((20.000000000000796 -189.99999999999858, 179.99999999999... GEOMETRYCOLLECTION (CURVEPOLYGON (COMPOUNDCURVE ((20.000000000000796 -1...

Bytes ----151 5207 693

Points -----6 322 38

The first shape s the or g na geometry co ect on of streets ( ne str ngs) used for nput, wh ch requ res on y 151 bytes of storage, and has on y 6 po nts For the second shape, STBuffer pads the ne str ngs to produce a mu t -po ygon (a set of po ygons) that consumes 5,207 bytes and has a tota of 322 po nts—a whopp ng 3,448 percent ncrease from the or g na ne str ngs In the th rd shape, BufferWithCurves s used to produce the equ va ent padd ng us ng a co ect on of curve po ygons



Chapter 9  Geospat a Support   407

composed of compound curves, so t consumes on y 693 bytes and has on y 38 po nts—a (re at ve y) mere 458 percent ncrease from the or g na ne str ngs

The ShortestLineTo Method Th s new method exam nes any two shapes and figures out the shortest ne between them L st ng 9-24 demonstrates th s Listing 9-24  F nd ng the shortest d stance between two shapes us ng ShortestLineTo.

DECLARE @Shape1 geometry = ' POLYGON ((-20 -30, -3 -26, 14 -28, 20 -40, -20 -30))' DECLARE @Shape2 geometry = ' POLYGON ((-18 -20, 0 -10, 4 -12, 10 -20, 2 -22, -18 -20))' SELECT @Shape1 UNION ALL SELECT @Shape2 UNION ALL SELECT @Shape1.ShortestLineTo(@Shape2).STBuffer(.25)

Th s code defines two po ygons and then uses ShortestLineTo to determ ne, generate, and return the shortest stra ght ne that connects them STBuffer s a so used to pad the ne str ng so that t s more c ear y v s b e n the spat a v ewer, as shown n F gure 9-27

Figure 9-27  Us ng the ShortestLineTo method to ca cu ate the shortest stra ght ne between two shapes.

The MinDbCompatibilityLevel Method W th the added support for curves n SQL Server 2012 comes support for backward c­ ompat b ty w th prev ous vers ons of SQL Server (2008 and 2008 R2) that don’t support curves The new ­MinDbCompatibilityLevel method accepts any WKT str ng and returns the m n mum vers on of SQL Server requ red to support the shape defined by that str ng For examp e, cons der the code n L st ng 9-25

408  Part II  Going Beyond Relational

Listing 9-25  Test ng for vers on compat b ty w th the MinDbCompatibilityLevel method.

DECLARE @Shape1 geometry = 'CIRCULARSTRING(0 50, 90 50, 180 50)' DECLARE @Shape2 geometry = 'LINESTRING (0 50, 90 50, 180 50)' SELECT Shape1MinVersion = @Shape1.MinDbCompatibilityLevel(), Shape2MinVersion = @Shape2.MinDbCompatibilityLevel()

The MinDbCompatibilityLevel method returns 110 (referr ng to vers on 11 0) for the first WKT str ng and 100 (vers on 10 0) for the second one Th s s because the first WKT str ng conta ns a ­c rcu ar str ng, wh ch requ res SQL Server 2012 (vers on 11 0), whereas the ne str ng n the second WKT str ng s supported by SQL Server 2008 (vers on 10 0) and h gher

The STCurveToLine and CurveToLineWithTolerance Methods These are two methods you can use to convert curves to rough y equ va ent stra ght ne shapes Aga n, th s s to prov de compat b ty w th prev ous vers ons of SQL Server and other mapp ng ­p atforms that don’t support curves The STCurveToLine method converts a s ng e curve to a ne str ng w th a mu t tude of segments and po nts that best approx mate the or g na curve The techn que s s m ar to what we just d scussed for STBuffer, where many short stra ght nes are connected n a c uster of po nts to s mu ate a curve And, as exp a ned n that d scuss on, the resu t ng ne str ng requ res s gn ficant y more storage than the or g na curve To offer a comprom se between fide ty and storage, the CurveToLineWithTolerance method accepts “to erance” parameters to produce ne str ngs that consume ess storage space than those produced by STCurveToLine The code n L st ng 9-26 demonstrates th s by us ng both methods to convert the same c rc e from F gure 9-23 nto ne str ngs Listing 9-26  Convert ng curves to nes w th STCurveToLine and CurveToLineWithTolerance.

-- Create a full circle shape (two connected semi-circles) DECLARE @C geometry = 'CIRCULARSTRING(0 4, 4 0, 8 4, 4 8, 0 4)' -- Render as curved shape SELECT Shape = @C, ShapeWKT = @C.ToString(), ShapeLen = DATALENGTH(@C), Points = @C.STNumPoints() -- Convert to lines (much larger, many more points) SELECT Shape = @C.STCurveToLine(), ShapeWKT = @C.STCurveToLine().ToString(), ShapeLen = DATALENGTH(@C.STCurveToLine()), Points = @C.STCurveToLine().STNumPoints()



Chapter 9  Geospat a Support   409

-- Convert to lines with tolerance (not as much larger, not as many more points) SELECT Shape = @C.CurveToLineWithTolerance(0.1, 0), ShapeWKT = @C.CurveToLineWithTolerance(0.1, 0).ToString(), ShapeLen = DATALENGTH(@C.CurveToLineWithTolerance(0.1, 0)), Points = @C.CurveToLineWithTolerance(0.1, 0).STNumPoints()

The query resu ts show that the or g na c rc e consumes on y 112 bytes and has 5 po nts Invok ng STCurveToLine on the c rc e converts t nto a ne str ng that consumes 1,072 bytes and has 65 po nts That’s a b g ncrease, but the resu t ng ne str ng represents the or g na c rc e n h gh fide ty; you w not see a percept b e d fference n the two when v ew ng them us ng the spat a v ewer However, the ne str ng produced by CurveToLineWithTolerance consumes on y 304 bytes and has on y 17 po nts; a s gn ficant y sma er footpr nt, pa d for w th a not ceab e oss n fide ty As shown by the spat a v ewer resu ts n F gure 9-28, us ng CurveToLineWithTolerance produces a c rc e made up of v s b y stra ght ne segments

Figure 9-28  F de ty oss convert ng a curve to a ne us ng CurveToLineWithTolerance.

The STIsValid, IsValidDetailed, and MakeValid Methods Spat a nstance va dat on has mproved great y n SQL Server 2012 The STIsValid method ­eva uates a spat a nstance and returns a 1 (for true) or 0 (for fa se), nd cat ng f the nstance represents a va d shape (or shapes) As L st ng 9-27 demonstrates, f the nstance s nva d, the new IsValidDetailed method w return a str ng exp a n ng the reason why Listing 9-27 Us ng IsValidDetailed to determ ne why a geometry nstance s nva d.

DECLARE @line geometry = 'LINESTRING(1 1, 2 2, 3 2, 2 2)' SELECT IsValid = @line.STIsValid(), Details = @line.IsValidDetailed()

410  Part II  Going Beyond Relational

Th s ne str ng s nva d because the same po nt (2 2) s repeated, wh ch resu ts n “over app ng edges,” as revea ed by the output from IsValidDetailed IsValid ------0

Details ------------------------------------------------------------------24413: Not valid because of two overlapping edges in curve (1).

SQL Server 2012 s more to erant of nva d spat a nstances than prev ous vers ons For examp e, you can now perform metr c operat ons (such as STLength) on nva d nstances, a though you st won’t be ab e to perform other operat ons (such as STBuffer) on them The new MakeValid method can “fix” an nva d spat a nstance and make t va d Of course, the shape w sh ft s ght y, and there are no guarantees on the accuracy or prec s on of the changes made The code n L st ng 9-28 uses MakeValid to remove over app ng parts (wh ch can be caused by anoma es such as naccurate GPS traces), effect ve y convert ng the nva d ne str ng from L st ng 9-27 nto a va d spat a nstance Listing 9-28 Us ng MakeValid to convert an nva d ne str ng to a va d spat a nstance.

DECLARE @line geometry = 'LINESTRING(1 1, 2 2, 3 2, 2 2)' SELECT @line.MakeValid().ToString() AS Fixed

The WKT str ng returned by the SELECT statement shows the “fixed” ne str ng Fixed ------------------------------------------------------------------LINESTRING (3 2, 2 2, 1.0000000000000071 1.0000000000000036)

Other Enhancements The rema nder of th s sect on g ves br ef ment on to severa other noteworthy spat a enhancements added n SQL Server 2012 These nc ude better geography support, and prec s on and opt m zat on mprovements

Note  Full coverage of these enhancements (and others, such as aggregates, histograms, the nearest neighbor query plan, spatial indexes, hints, and performance improvements) is beyond the scope of this chapter. Consult SQL Server Books Online for the complete spatial reference.

Support for geography Instances Exceeding a Logical Hemisphere Prev ous vers ons of SQL Server supported geography objects as arge as (s ght y ess than) a ­og ca hem sphere (ha f the g obe) Th s m tat on has been removed n SQL Server 2012, wh ch now ­supports geography nstances of any s ze (even the ent re p anet)

Chapter 9  Geospat a Support   411

When you define a geography po ygon, the order n wh ch you spec fy the r ng’s at tude and ong tude coord nates (known as vertex order) s s gn ficant (un ke geometry, where vertex order s ns gn ficant) The coord nate po nts are a ways defined accord ng to the left-foot inside ru e; when you “wa k” the boundary of the po ygon, your eft foot s on the ns de Thus, vertex order determ nes whether you are defin ng a sma p ece of the g obe, re at ve to the arger p ece defined by the ent re g obe except for the sma p ece (that s, the rest of the g obe) Because prev ous vers ons of SQL Server were m ted to ha f the g obe, t was mposs b e to spec fy the po nts of a po ygon n the “wrong order,” s mp y because do ng so resu ted n too arge a shape (and thus, ra sed an error) That error potent a no onger ex sts n SQL Server 2012, so t’s even more cr t ca to make sure your vertex order s correct, or you’ be unw tt ng y work ng w th the exact ­“oppos te” shape If you have a geography nstance that s known to have the wrong vertex order, you can repa r t us ng the new ReorientObject method Th s method operates on y on po ygons ( t has no effect on po nts, ne str ngs, or curves), and can be used to correct the r ng or entat on (vertex order) of the po ygon, as L st ng 9-29 demonstrates Listing 9-29 Us ng ReorientObject to change the vertex order.

-- Small (less than a logical hemisphere) polygon SELECT geography::Parse('POLYGON((-10 -10, 10 -10, 10 10, -10 10, -10 -10))') -- Reorder in the opposite direction for "rest of the globe" SELECT geography::Parse('POLYGON((-10 -10, -10 10, 10 10, 10 -10, -10 -10))') -- Reorient back to the small polygon SELECT geography::Parse('POLYGON((-10 -10, -10 10, 10 10, 10 -10, -10 -10))') .ReorientObject()

Three geography po ygon nstances are defined n th s code The first geography nstance defines a very sma po ygon The second nstance uses the exact same coord nates, but because the vertex ­order s reversed, t defines an enormous po ygon whose area represents the ent re g obe except for the sma po ygon As exp a ned, such a defin t on wou d cause an error n prev ous vers ons of SQL Server, but s now accommodated w thout a prob em by SQL Server 2012 The th rd nstance ­reverses the vertex order on the same shape as the second nstance, thereby produc ng the same sma ­po ygon as the first nstance

Full Globe Support A ong w th the aforement oned support for geography nstances to exceed a s ng e og ca ­hem sphere comes a new spat a data c ass ca ed FULLGLOBE As you may have guessed, th s s a shape that represents the ent re p anet If you’ve ever wondered how many square meters there are n the ent re wor d, the query n L st ng 9-30 g ves you the answer (wh ch s 510,065,621,710,996 square meters, so you can stop wonder ng)

412  Part II  Going Beyond Relational

Listing 9-30 Us ng FULLGLOBE to ca cu ate the area of the ent re earth.

-- Construct a new FullGlobe object (a WGS84 ellipsoid) DECLARE @Earth geography = 'FULLGLOBE' -- Calculate the area of the earth SELECT PlanetArea = @Earth.STArea()

A of the common spat a methods work as expected on a fu g obe object So you cou d, for examp e, “cut away” at the g obe by nvok ng the STDifference and STSymDifference method aga nst t us ng other po ygons as cook e-cutter shapes

New “Unit Sphere” Spatial Reference ID Ear er we noted that the defau t SRID s 4326, wh ch uses the metr c system as ts un t of ­measurement Th s SRID a so represents the true e pso da sphere shape of the earth A though th s ­representat on s most accurate, t’s a so more comp ex to ca cu ate prec se e pso da mathemat cs SQL Server 2012 offers a comprom se n speed and accuracy, by add ng a new spat a reference d (SRID), 104001, wh ch uses a sphere of rad us 1 to represent a perfectly round earth You can create geography nstances w th SRID 104001 when you don’t requ re the greatest ­accuracy The STDistance, STLength, and ShortestLineTo methods are opt m zed to run faster on the un t sphere, because t takes a re at ve y s mp e formu a to compute measures aga nst a perfect y round sphere (compared to an e pso da sphere)

Better Precision Interna spat a ca cu at ons n SQL Server 2012 are now performed w th 48 b ts of prec s on, compared to 27 b ts used n SQL Server 2008 and SQL Server 2008 R2 Th s can reduce the error caused by round ng of float ng po nt coord nates for or g na vertex po nts by the nterna computat on

Integrating with Microsoft Bing Maps We’ end th s chapter w th one fina (and fun) geospat a app cat on, n wh ch you w construct a ­M crosoft B ng Maps mash-up The term “mash-up” s used to descr be a un fied presentat on ­comb n ng one set of data w th another For th s mash-up, you w create a web app cat on that ­ayers customer coord nates over B ng Maps magery to show the ocat on of customers n the ­database as pushp n cons on a map Start w th a new database Create the Customer tab e w th a geography co umn to ho d customer ocat ons and popu ate t w th some samp e data, as shown n L st ng 9-31



Chapter 9  Geospat a Support   413

Listing 9-31  Creat ng the Customer tab e and popu at ng t w th geograph ca data.

USE master GO IF EXISTS(SELECT name FROM sys.databases WHERE name = 'MyDB') DROP DATABASE MyDB GO CREATE DATABASE MyDB GO USE MyDB GO CREATE TABLE Customer (CustomerId int PRIMARY KEY, Name varchar(50), Company varchar(50), CustomerGeo geography) GO INSERT INTO Customer VALUES (1, 'Adam', 'Coho Vineyard & Winery', 'POINT(-111.06687 45.01188)'), (2, 'John', 'ACME Corp.', 'POINT(-104.06 41.01929)'), (3, 'Paul', 'Litware, Inc.', 'POINT(-111.05878 41.003)'), (4, 'Joel', 'Tailspin Toys', 'POINT(-121.05878 41.003)'), (5, 'Martin', 'ABC Travel', 'POINT(-110.05878 43.003)'), (6, 'Remon', 'Wingtip Toys', 'POINT(-113.05878 35.003)'), (7, 'Jason', 'School of Fine Art', 'POINT(-116.05878 34.003)'), (8, 'Fred', 'Fourth Coffee', 'POINT(-114.05878 43.003)')

Now create a stored procedure to retr eve a the customers and the r ocat ons, as shown n L st ng 9-32 Listing 9-32  Creat ng a stored procedure to retr eve customers and the r ocat ons.

CREATE PROCEDURE GetCustomers AS BEGIN SELECT Name, Company, CustomerGeo FROM Customer END GO

W th the database set up, you’re ready to create your B ng Maps mash-up Start V sua Stud o, and then choose F e New Project From the temp ates sted under V sua C#, Web, create an ASP NET Empty Web App cat on project named BingMapsSpatialApp, as shown n F gure 9-29

414  Part II  Going Beyond Relational

Figure 9-29  Creat ng the BingMapsSpatialApp project.

In Default.aspx, rep ace the starter markup prov ded by V sua Stud o w th the code shown n ­L st ng 9-33 Listing 9-33  Creat ng a V rtua Earth mash up w th geography data.











Th s webpage uses Asynchronous JavaScr pt and XML (AJAX) to render a map us ng M crosoft B ng Maps, upon wh ch t draws pushp n cons for each customer There are abso ute y no page postbacks to the server; a the serv ce ca s are made d rect y from the c ent browser Let’s d ssect th s page carefu y Every AJAX-enab ed webpage requ res a ScriptManager e ement, wh ch you’ve p aced at the top of the form Nested w th n the ScriptManager e ement, two serv ce references are dec ared The first ­reference po nts to CustomerQueryService.asmx, wh ch s a W ndows Commun cat ons Foundat on (WCF) Serv ce that you w create momentar y to ca the stored procedure that retr eves ­customers and the r ocat ons The second reference s to the M crosoft B ng Maps web serv ces app cat on programm ng nterface (API) at dev.virtualearth.net Th s API prov des the B ng Maps Ajax Contro that can be used to render and man pu ate a map on any webpage The map renders w th a UI that prov des zoom and pan contro s s mp y by c ck ng and dragg ng the mouse

Note  The script reference indicates the name Virtual Earth, which is the original name for Bing Maps. Currently, the Bing Maps Ajax Control API still reflects the old name by u ­ sing objects prefixed with VE. Thus, we interchange the terms “Bing Maps” and “Virtual Earth” throughout our discussion of the code. After the ScriptManager, the page dec ares a sect on that defines the rectangu ar area of the page (640 by 400 p xe s n th s examp e) n wh ch the map shou d be d sp ayed The e ement s 416  Part II  Going Beyond Relational

ass gned an ID named divBingMap that w be referenced ater by a ca nto the B ng Maps API when the map s created Beneath the map, the page d sp ays a button named btnGetCustomers that s w red to a c ent-s de event hand er named btnGetCustomers click() The rest of the page conta ns the JavaScr pt that nteracts w th the two serv ces from the user’s browser At the top of the scr pt, a page- eve var ab e named map s dec ared Th s var ab e w ho d a reference to the VEMap (V rtua Earth map) object created n the pageLoad funct on, wh ch fires on the c ent when the browser oads the page The pageLoad funct on nstant ates the map v­ ar ab e by dec ar ng t as a new VEMap object Th s resu ts n a ca to the B ng Maps web serv ce that returns a VEMap object that s stored n the map var ab e The object s bound to the tag named divBingMap (dec ared ear er n the markup) that was passed n as a parameter nto the VEMap ­constructor As a resu t, the map oads and d sp ays tse f n the divBingMap sect on of the page when the page oads Next, the button’s c ck event hand er s defined n the btnGetCustomers click() funct on The s ng e ne of code n th s funct on ca s your own CustomerQueryService.asmx web serv ce (wh ch you’re about to create), dec ared n the preced ng ScriptManager sect on Th s resu ts n an asynchronous ca to the serv ce, wh ch runs n the background to ca your stored procedure and retr eve customers and the r ocat ons from the database When the ca s made, the page spec fies the name of the ca back funct on OnDataRetrievalComplete to be nvoked when the resu ts are returned by the serv ce F na y, the OnDataRetrievalComplete funct on s defined to rece ve and process the resu ts of the c­ ustomer query, wh ch s a st of customers and the r ocat ons that s passed to the funct on as a p ­ arameter named results The st s processed as an array of objects terated w th a for oop Each e ement n the array s an object w th propert es defined for the Customer c ass (wh ch s a WCF data contract that you’re about to create w th the serv ce) For each customer, the Latitude and Longitude propert es are used to construct a VELatLong (V rtua Earth at tude/ ong tude) object stored n a var ab e named point (Note that the VELatLong constructor requ res the at tude va ue for the first ­parameter and the ong tude va ue for the second parameter, un ke WKT, wh ch uses the reverse ­order of ong tude fo owed by at tude ) A new VEShape (V rtua Earth shape) object s then created for the po nt as a VEShapeType.Pushpin shape stored n the var ab e pin The Company and Name propert es are then used to set the pushp n’s t t e and descr pt on us ng the SetTitle and ­SetDescription methods The pushp n s then added to the map by ­ nvok ng the AddShape method on the map ­var ab e, wh ch represents the map d sp ayed n the tag named divVirtualEarthMap on the page That comp etes the webpage Your next step s to create the WCF Serv ce ca ed by the page to retr eve your customers and the r ocat ons from the database In So ut on Exp orer, r ght-c ck the VirtualEarthSpatialApp project, and se ect Add New Item In the nsta ed temp ates sted under Web, WCF Serv ce, name the serv ce CustomerGeoService.svc, as shown n F gure 9-30



Chapter 9  Geospat a Support   417

Figure 9-30  Creat ng the customer query web serv ce.

To keep the examp e s mp e, you won’t separate nterface from mp ementat on Therefore, you can de ete the ICustomerGeoService.cs fi e that V sua Stud o created for you to define the nterface Then, open the serv ce’s CustomerGeoService.svc.cs code-beh nd fi e and rep ace the starter code that V sua Stud o prov ded w th the code shown n L st ng 9-34 Listing 9-34  mp ement ng a WCF Serv ce to retr eve customer spat a data.

using using using using using using using

System; System.Collections.Generic; System.Data; System.Data.SqlClient; System.Runtime.Serialization; System.ServiceModel; System.ServiceModel.Activation;

using Microsoft.SqlServer.Types; namespace BingMapsSpatialApp { [DataContract] public class Customer { [DataMember] public string Name { get; set; } [DataMember] public string Company { get; set; } [DataMember] public double Latitude { get; set; }

[DataMember]

418  Part II  Going Beyond Relational

public double Longitude { get; set; } } [ServiceContract(Namespace = "BingMapsSpatialApp")] [AspNetCompatibilityRequirements (RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)] public class CustomerGeoService { private const string ConnStr = "Data Source=.;Initial Catalog=MyDb;Integrated Security=True;"; [OperationContract] public List GetCustomers() { var customers = new List(); using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd = new SqlCommand()) { cmd.Connection = conn; cmd.CommandText = "GetCustomers"; cmd.CommandType = CommandType.StoredProcedure; using (var rdr = cmd.ExecuteReader()) { while (rdr.Read()) { var customer = new Customer(); customer.Name = rdr["Name"].ToString(); customer.Company = rdr["Company"].ToString(); // Get the CustomerGeo column bytes into a geography instance var geo = SqlGeography.Deserialize(rdr.GetSqlBytes(2)); customer.Latitude = (double)geo.Lat; customer.Longitude = (double)geo.Long; customers.Add(customer); } rdr.Close(); } } conn.Close(); return customers; } } } }



Chapter 9  Geospat a Support   419

The Customer data contract c ass s defined first, and s mp y exposes propert es to define the object that gets returned to the webpage Then the CustomerGeoService serv ce contract c ass s defined, wh ch n turn defines the GetCustomers operat on contract ca ed by the webpage The GetCustomers method extracts the at tude and ong tude va ues from the geography typed Location co umn returned by the GetCustomers stored procedure that you created n L st ng 9-32 In M crosoft NET, you can work w th the SQL Server system CLR types (such as geography) by us ng c asses defined n the Microsoft.SqlServer.Types namespace Th s namespace s dec ared by a using statement at the top of the code (h gh ghted n bo d) The SqlGeography c ass n th s namespace corresponds to the geography data type n SQL Server The Lat and Long propert es of a SqlGeography object expose the at tude and ong tude va ues conta ned n the geography nstance t encapsu ates (as double data types) Before us ng any types n th s namespace, you must first estab sh a reference to ts assemb y To do so, r ght-c ck the BingMapsSpatialApp project n So ut on Exp orer, and then se ect Add Reference On the NET tab, scro down and se ect Microsoft.SqlServer.Types, as shown n F gure 9-31, and then c ck OK

Figure 9-31  Add ng a reference to the Microsoft.SqlServer.Types assemb y.

After open ng a connect on to the database, the GetCustomers method prepares a SqlCommand object to ca the GetCustomers stored procedure n the database and then nvokes the ExecuteReader method to obta n a SqlDataReader object that returns the resu ts The method oops through a the rows n the reader returned by the stored procedure and popu ates a gener c List object w th the resu ts For each row, a Customer object s created and popu ated w th the Name and ­Company co umns, as we as ocat on nformat on extracted from the CustomerGeo co umn The ­ocat on va ue s obta ned by ca ng the stat c Deserialize method on the SqlGeography c ass, pass ng n the bytes from the CustomerGeo co umn of the reader, and rece v ng back a geography nstance The bytes are retr eved by ca ng the reader’s GetSqlBytes method, pass ng n the zero-based ndex of the des red co umn ( n th s case 2 refers to the th rd co umn, CustomerGeo) Then the customer’s Latitude and Longitude propert es (double va ues) are popu ated from the geography nstance’s Lat 420  Part II  Going Beyond Relational

and Long propert es, respect ve y The Customer object s then added to the gener c List object that s returned to the ca er after a the customers have been processed Your next and fina step s to configure the serv ce for asynchronous ca s, wh ch s requ red to support Ajax c ent app cat ons ke the one you are bu d ng Spec fica y, you need to ed t Web.config and define a custom behav or for the serv ce endpo nt to enab e web scr pt ng ca s, as shown n L st ng 9-35 Listing 9-35  Configur ng the WCF Serv ce for Ajax c ents.















After mod fy ng Web.config, you can g ve the app cat on a run To v ew the webpage, r ght-c ck Default.aspx n So ut on Exp orer, and choose Set as Start Page Then press F5 to run the a ­ pp cat on When the page oads, t runs the c ent-s de JavaScr pt you wrote n the pageLoad funct on and d sp ays the map n the browser C ck the Get Customers button to nvoke the database query and d sp ay the pushp ns correspond ng to the ocat ons of the customers returned by the query, as shown n F gure 9-32



Chapter 9  Geospat a Support   421

Figure 9-32  Runn ng the B ng Maps mash up aga nst geography data n SQL Server.

Th s page de vers fu B ng Maps capab ty Users can zoom, pan, and render aer a v ews of the map wh e the pushp ns rema n bound to your geograph c data, as shown n F gure 9-33

Figure 9-33  Zoom, pan, and aer a v ew capab t es of the B ng Maps Ajax Contro .

422  Part II  Going Beyond Relational

Summary The power of spat a process ng s now read y ava ab e to a deve opers us ng the geometry and geography data types bu t nto SQL Server 2012, enab ng the rap d deve opment of soph st cated GIS app cat ons In th s chapter, you saw how easy t s to express and project coord nates aga nst e ther the p anar (flat-earth) or geodet c (e pso da sphere) spat a mode s us ng the We -Known Text (WKT), ­We -Known B nary (WKB), and Geography Markup Language (GML) formats We wa ked you through numerous samp es and demos, and, a ong the way, exam ned the many methods ava ab e for ­man pu at ng spat a data You a so created severa geospat a app cat ons that ntegrate ocat on nte gence and mapp ng capab t es for end users A though th s chapter has on y scratched the surface of spat a data and spat a programm ng, t does prov de the foundat on you need to work w th the powerfu spat a support ava ab e n SQL Server 2012



Chapter 9  Geospat a Support   423

Par t I I I

Applied SQL chapter 10

The M crosoft Data Access Juggernaut

427

chapter 11

WCF Data Access Techno og es

509

chapter 12

Mov ng to the C oud w th SQL Azure

579

chapter 13

S QL Azure Data Sync and W ndows Phone Deve opment . . . . . . . . . . . . . . . .

619

chapter 14

Pervas ve Ins ght . . . . . . . . . . . . . . . . . . . . . . . . . . . .

675

chapter 15

xVe oc ty In-Memory Techno og es . . . . . . . . . . . .

701

425

C hapter 1 0

The Microsoft Data Access Juggernaut –Leonard Lobel

A

t some po nt, and n some way, you need to expose your SQL Server data so that c ent ­app cat ons can consume t From the perspect ve of the database, every consumer s a “c ent app cat on”—whether you are bu d ng an app cat on to commun cate d rect y w th M crosoft SQL Server (trad t ona c ent/server arch tecture), or you are bu d ng a m dd e-t er serv ce ayer that, n turn, s consumed by other types of c ent app cat ons (n-t er arch tecture) Regard ess of the nature of the consum ng app cat on, t s your respons b ty as a profess ona deve oper to arch tect a data access ayer that exchanges nformat on between SQL Server and your app cat on (or the t ers of your app cat on) n a re ab e, sca ab e, and ma nta nab e manner In the next ser es of chapters, you w earn how to do just that us ng a of the techn ques ­ava ab e w th the M crosoft NET Framework—and there are many A though the number of cho ces m ght seem daunt ng, don’t fee nt m dated We w take th ngs step by step so that you ga n a c ear ­understand ng of what a the p eces are, and how the ent re stack fits (or doesn’t fit) together

Th s chapter covers a your c ent/server opt ons, nc ud ng raw ADO NET objects, the DataSet, Language-Integrated Query (LINQ), and the ADO NET Ent ty Framework In the next chapter, you w earn how to extend your data’s reach by us ng two spec a W ndows Commun cat on ­Foundat on (WCF) techno og es WCF Data Serv ces, wh ch exposes your data us ng Open Data Protoco (OData), and WCF RIA Serv ces, a r cher framework that targets M crosoft S ver ght n part cu ar The two chapters that fo ow then carry the d scuss on up to the c oud, where you w earn how to ­create so ut ons us ng SQL Azure, SQL Azure Data Sync, W ndows Azure, and W ndows Phone 7 Th s ­comprehens ve coverage w he p you choose the r ght NET data access strategy for your SQL Server app cat ons

.NET Data Access Evolution ADO NET was re eased w th the very first vers on of the NET Framework n ear y 2002, and t ­offers you two ways to access the database programmat ca y us ng raw objects or DataSets (or ­comb nat ons of the two) You can use the raw ADO NET objects to nteract d rect y w th the database by exp c t y connect ng to t, execut ng d rect T-SQL or stored procedures, and pu ng data back

427

w th stream ng readers A ternat ve y, you can work at a h gher eve of abstract on w th D ­ ataSets, us ng DataAdapters as wrappers around the raw objects to fi and update the DataSet Because ­DataAdapters are mere y wrappers around the very same raw objects, you can use a comb nat on of raw objects and DataSets as des red In our d scuss on, we define both of these methods co ect ve y as “convent ona ADO NET,” to d st ngu sh them from data access techno og es added to ater vers ons of the NET Framework Convent ona ADO NET was the on y data access opt on for NET programmers from the t me the framework was first re eased n 2002 unt NET 3 5 was re eased n ate 2007, fo owed short y ­thereafter by NET 3 5 SP1 n 2008 S nce NET 3 5 (and espec a y SP1), th ngs have been acce erat ng qu te rap d y The advent of Language-Integrated Query (LINQ), n NET 3 5, rad ca y changed the way deve opers wr te code aga nst many d fferent types of back-end data stores For SQL Server n part cu ar, programmab ty opt ons have ncreased great y w th the re ease of LINQ to SQL n NET 3 5 and the ADO NET Ent ty Framework—first re eased as part of NET 3 5 SP1, and great y enhanced n NET 4 0 as of 2010 New app cat on programm ng nterfaces (APIs) and too s have a so emerged, such as WCF Data Serv ces and WCF RIA Serv ces (covered n Chapter 11) These too s promote rap d deve opment of n-t er database app cat ons by prov d ng even r cher abstract ons at the WCF ayer When techno ogy churns th s fast, t s on y natura to fee overwhe med Keep ng pace s certa n y cha eng ng, but th s chapter w he p you meet the cha enge w th m n ma pa n You w earn the necessary cod ng techn ques for work ng w th a of the NET data access opt ons as you study each w th pract ca , hands-on use cases Th s comparat ve approach w he p br ng the s m ar t es and ­d fferences between each opt on to ght, so that you effect ve y ga n a so d understand ng across the spectrum Anyone who has worked w th NET for any ength of t me s a ready qu te fam ar w th c­ onvent ona ADO NET, but may now quest on ts re evance n the wake of a the techno og es that have emerged s nce In fact, convent ona ADO NET—the raw data access objects n part cu ar, but the DataSet as we — s not on y st re evant, but rema ns core to the NET framework The newer APIs comp ement convent ona ADO NET—they do not rep ace t Th s s a vast qua ty-of- fe mprovement over standard M crosoft pract ce n the days of the Component Object Mode (COM) p atform that preceded the NET Framework In the days of COM, each new data access API wou d typ ca y render prev ous APIs obso ete Data access objects (DAO) prov ded an object mode around the Open Database Connect v ty (ODBC) API that was very powerfu , unt t was made obso ete by Remote Data Objects (RDO) Yet t was not ong after RDO that Act veX Data Objects (ADO) comp ete y obso esced RDO And of course, a of these COM-based APIs were rendered obso ete by the NET Framework, and the not on of managed code prov ders w th ADO NET One s mp y accepted obso escence w th the re ease of each new API, as part of the nev tab e pr ce to be pa d f one wanted to everage the atest techno og es As a resu t of th s pa nfu cyc e, product on code became st gmat zed as “ egacy code” very qu ck y n those days S nce NET, the s tuat on has mproved great y There s noth ng “c ass c” or “ egacy” (more ­po te terms for “obso ete”) about convent ona ADO NET The newer APIs ayer and extend, but most certa n y do not obso esce convent ona ADO NET And wh e they each have d fferent use cases and prov de d fferent eve s of abstract on, they a so typ ca y do not obso esce one another (though 428  Part III  Applied SQL

depend ng on one’s subject ve defin t on of the term “obso ete,” LINQ to SQL m ght be cons dered an except on to th s statement, as t has been essent a y ec psed by the atest vers on of the ADO NET Ent ty Framework) L ke DataAdapters w th DataSets, the atest APIs prov de vary ng degrees of abstract on beneath wh ch e the very same managed code prov ders that NET deve opers have been work ng w th s nce the b rth of the framework (Sq C ent, n the case of SQL Server) Th s new and mproved sty e of evo ut on serves as a testament to the stab ty and matur ty of the NET Framework, and s gn ficant y extends the she f- fe of your app cat ons A though keep ng pace st cont nues to pose a cha enge, the fact s that the newer APIs broaden rather than narrow your cho ces, and we be eve that’s a good th ng So regard ess of wh ch API you choose, the same ADO NET data prov der (Sq C ent) s used to send commands to SQL Server and stream data back to your app cat on Th s s true no matter wh ch API you are us ng DataSets are fi ed and updated us ng DataAdapters, wh ch wrap the raw connect on, command, and reader objects W th LINQ to SQL, there s a DataContext object that w a so ut ze Sq C ent to retr eve and update data n SQL Server If you are us ng ADO NET Ent ty Framework, the storage ayer s m ar y does the same WCF Data Serv ces and WCF RIA Serv ces (wh ch we cover n the next chapter) prov de abstract ons at the commun cat ons transport ayer further up n the stack, so c ent requests hand ed by those serv ce-or ented APIs w a so u t mate y bo down to raw ADO NET data access when the t me comes to h t the database Th s techno ogy stack s dep cted n F gure 10-1

WCF Data Services

WCF RIA Services

WCF

DataSet

LINQ to SQL

Entity Framework

Raw ADO.NET Objects

SQL Server

Figure 10-1  The .NET data access techno ogy stack over SQL Server.

Note  In our discussions, we will refer to the ADO.NET Entity Framework frequently as Entity Framework, or abbreviate it simply as EF. Convent ona ADO NET today st rema ns a perfect y v ab e opt on for data access n many ­scenar os It s not on y he pfu but prudent for even exper enced deve opers to refresh the r

Chapter 10  The M crosoft Data Access Juggernaut   429

­ nderstand ng of t Therefore, our coverage of convent ona ADO NET s recommended read ng even u f you’re a seasoned deve oper who may be tempted to sk p ahead to the newer techno og es further on n the chapter

Preparing the Sample Database Start by creat ng a bas c order entry database n SQL Server Then you’ bu d app cat on code over the database that quer es and updates us ng each of the ava ab e data access APIs Use e ther SQL Server Data Too s (SSDT) n V sua Stud o (covered n Chapter 1) or SQL Server Management Stud o (SSMS) to execute the scr pt shown n L st ng 10-1 and create the samp e database Listing 10-1  T SQL scr pt for creat ng the SampleDb database.

USE master GO IF EXISTS(SELECT name FROM sys.databases WHERE name = 'SampleDb') DROP DATABASE SampleDb GO CREATE DATABASE SampleDb GO USE SampleDb GO CREATE TABLE Customer( CustomerId bigint IDENTITY(1,1) NOT NULL, FirstName varchar(50) NOT NULL, LastName varchar(50) NOT NULL, Balance money NOT NULL DEFAULT 0, CreatedAt datetime2(7) NOT NULL DEFAULT SYSDATETIME(), UpdatedAt datetime2(7) NOT NULL DEFAULT SYSDATETIME(), CONSTRAINT PK_Customer PRIMARY KEY CLUSTERED (CustomerId ASC)) CREATE TABLE OrderHeader( OrderHeaderId bigint IDENTITY(1,1) NOT NULL, CustomerId bigint NOT NULL, ShipVia varchar(20) NOT NULL, OrderStatus varchar(20) NULL, Notes varchar(max) NULL, CreatedAt datetime2(7) NOT NULL DEFAULT SYSDATETIME(), UpdatedAt datetime2(7) NOT NULL DEFAULT SYSDATETIME(), CONSTRAINT PK_OrderHeader PRIMARY KEY CLUSTERED (OrderHeaderId ASC)) ALTER TABLE OrderHeader WITH CHECK ADD CONSTRAINT FK_OrderHeader_Customer FOREIGN KEY(CustomerId) REFERENCES Customer(CustomerId) CREATE TABLE Employee( EmployeeId bigint IDENTITY(1,1) NOT NULL, FirstName varchar(50) NOT NULL, LastName varchar(50) NOT NULL,

430  Part III  Applied SQL

Salary decimal(18, 9) NOT NULL, CreatedAt datetime2(7) NOT NULL, UpdatedAt datetime2(7) NOT NULL, CONSTRAINT PK_Employee PRIMARY KEY CLUSTERED (EmployeeId ASC)) CREATE TABLE CustomerEmployee( CustomerId bigint NOT NULL, EmployeeId bigint NOT NULL, CONSTRAINT PK_CustomerEmployee PRIMARY KEY CLUSTERED (CustomerId ASC, EmployeeId ASC)) ALTER TABLE CustomerEmployee WITH CHECK ADD CONSTRAINT FK_CustomerEmployee_Customer FOREIGN KEY(CustomerId) REFERENCES Customer(CustomerId) ALTER TABLE CustomerEmployee WITH CHECK ADD CONSTRAINT FK_CustomerEmployee_Employee FOREIGN KEY(EmployeeId) REFERENCES Employee(EmployeeId) GO CREATE PROCEDURE SelectCustomer(@CustomerId bigint) AS SELECT CustomerId, FirstName, LastName, Balance, CreatedAt, UpdatedAt FROM Customer WHERE CustomerId = @CustomerId GO CREATE PROCEDURE SelectCustomers AS SELECT CustomerId, FirstName, LastName, Balance, CreatedAt, UpdatedAt FROM Customer ORDER BY CustomerId GO CREATE PROCEDURE InsertCustomer( @FirstName varchar(20), @LastName varchar(20), @Balance money = 0) AS DECLARE @CreatedAt datetime2 = SYSDATETIME() INSERT INTO Customer (FirstName, LastName, Balance, CreatedAt, UpdatedAt) VALUES (@FirstName, @LastName, @Balance, @CreatedAt, @CreatedAt) SELECT CustomerId = CAST(SCOPE_IDENTITY() AS bigint), CreatedAt = @CreatedAt, UpdatedAt = @CreatedAt GO CREATE PROCEDURE UpdateCustomer( @CustomerId bigint, @FirstName varchar(20),



Chapter 10  The M crosoft Data Access Juggernaut   431

@LastName varchar(20), @Balance money, @OriginalUpdatedAt datetime2) AS DECLARE @NewUpdatedAt datetime2 = SYSDATETIME() UPDATE Customer SET FirstName = @FirstName, LastName = @LastName, Balance = @Balance, UpdatedAt = @NewUpdatedAt WHERE CustomerId = @CustomerId AND UpdatedAt = @OriginalUpdatedAt IF @@ROWCOUNT = 0 THROW 50000, 'The customer does not exist, or has been updated/deleted', 1; ELSE SELECT UpdatedAt = @NewUpdatedAt GO CREATE PROCEDURE DeleteCustomer(@CustomerId bigint) AS DELETE Customer WHERE CustomerId = @CustomerId GO CREATE PROCEDURE GetCustomerBalance( @CustomerId bigint, @Balance money OUTPUT) AS SELECT @Balance = Balance FROM Customer WHERE CustomerId = @CustomerId GO

We’ve kept the database des gn sma , wh e st demonstrat ng key concepts, such as the use of stored procedures for create, retr eve, update, and de ete (CRUD) operat ons; opt m st c concurrency checks; dent ty-based pr mary keys; and many-to-many re at onsh ps The scr pt first creates the Customer and OrderHeader tab es, a ong w th a fore gn key constra nt that estab shes a one-to-many re at onsh p between them Then t creates the Employee tab e There s a many-to-many re at onsh p between customers and emp oyees, wh ch s mp emented by the CustomerEmployee junct on tab e—the fourth and ast tab e created n the scr pt The junct on tab e conta ns just the two keys re at ng customers w th emp oyees and has fore gn key constra nts to both tab es, wh ch s how you mp ement a many-to-many re at onsh p n SQL Server The three tab es Customer, OrderHeader, and Employee each fo ow a s m ar pattern The r pr mary keys are long va ues (64-b t ntegers), and they each have an IDENTITY spec ficat on that nstructs SQL Server to automat ca y ass gn the next ava ab e nteger when nsert ng new rows Each of them

432  Part III  Applied SQL

a so has a pa r of datetime2 co umns named CreatedAt and UpdatedAt that are used for aud t ng and concurrency checks n the CRUD stored procedures, created next Your first exerc se w work d rect y aga nst tab es But n rea product on env ronments, c ent app cat ons are often den ed perm ss on to execute d rect SELECT, INSERT, UPDATE, and DELETE operat ons aga nst tab es n the database Instead, typ ca y, each tab e s “wrapped” by a set of stored procedures that expose CRUD operat ons n a contro ed manner In ater examp es, you’ work w th stored procedures nstead of d rect SQL to access the Customer tab e (For brev ty, the code on y ­creates CRUD stored procedures for the Customer tab e; you can extrapo ate from that how a s m ar set of procedures wou d be mp emented for the OrderHeader and Employee tab es ) Two stored procedures retr eve customers SelectCustomer accepts a @CustomerID ­parameter and returns a s ng e row for the spec fied customer SelectCustomers takes no ­parameters, and returns every customer n the tab e sorted by ast name, first name Both stored procedures r­ eturn every co umn from the Customer tab e Of course, your typ ca app cat on wou d have many more ­parameter zed stored procedures for retr eva (such as SelectCustomersByThis and ­SelectCustomersByThat), but these two w suffice for th s exerc se Next, the scr pt defines the InsertCustomer stored procedure that creates a new customer The stored procedure accepts one parameter for each customer co umn, except for ­CustomerId, ­CreatedAt, and UpdatedAt The c ent cannot supp y a CustomerId because SQL Server w ­automat ca y ass gn the next dent ty va ue for the pr mary key and then return t back to the c ent Va ues are a so not accepted for CreatedAt and UpdatedAt because the stored procedure w ensure that both of these va ues get set to the current t me on the database server c ock, and then return them back to the c ent a ong w th CustomerId after the INSERT InsertCustomer first dec ares @CreatedAt as a datetime2 var ab e, ass gns t the current date and t me us ng the SYSDATETIME funct on, and then executes the INSERT statement that creates the new row The INSERT statement supp es the SYSDATETIME va ue captured n @CreatedAt for both the CreatedAt and UpdatedAt co umns, and supp es the ncom ng parameter va ues for a the other ­co umns The on y co umn m ss ng from the INSERT statement s the CustomerId pr mary key (th s can’t be spec fied because you’ve put SQL Server n charge of ass gn ng the next ava ab e ­IDENTITY va ue to new pr mary keys) The stored procedure returns a the co umn va ues ass gned by the server (CustomerId, CreatedAt, and UpdatedAt) to the c ent as a s ng e-row resu t set n the SELECT ­statement mmed ate y fo ow ng the INSERT Th s SELECT statement returns the SCOPE IDENTITY funct on that g ves you the new pr mary key va ue ass gned to CustomerId, and returns the @CreatedAt va ue ass gned to both CreatedAt and UpdatedAt A c ent app cat on can use th s return nformat on to refresh ts oca v ew of the new y added customer, rather than be ng forced to retr eve the ent re customer aga n from the database The UpdateCustomer stored procedure s defined next Th s stored procedure mod fies an ex st ng customer, and enforces an opt m st c concurrency check ensur ng that users do not overwr te each other’s changes It accepts a @CustomerId parameter spec fy ng the customer row to be updated, ­fo owed by one parameter for each customer co umn to be updated, except for CreatedAt and ­UpdatedAt Th s des gn prevents the CreatedAt co umn from be ng changed once t has been ass gned by InsertCustomer, and thus prov des you w th a re ab e aud t of each customer’s or g na creat on

Chapter 10  The M crosoft Data Access Juggernaut   433

t me The ast parameter taken by UpdateCustomer s @OriginalUpdatedAt, wh ch s used for the ­mu t user concurrency check Th s parameter must be passed the or g na UpdatedAt va ue of the row to be changed, wh ch mp es that you must first retr eve a customer before you can change t UpdateCustomer first dec ares @NewUpdatedAt as a datetime2 var ab e and ass gns t the ­current date and t me on the server us ng the SYSDATETIME funct on Th s va ue overwr tes the current va ue n the UpdatedAt co umn, and then gets returned to the c ent, assum ng that the opt m st c ­concurrency check doesn’t fa The UPDATE statement executes w th a WHERE c ause that finds the des red row to be updated by @CustomerId, but a so checks that the UpdatedAt co umn st ­conta ns the va ue passed n to @OriginalUpdatedAt Because UpdatedAt gets overwr tten w th each ­mod ficat on, the WHERE c ause w no onger find the row f another user se ected and updated the same customer s nce the t me you or g na y retr eved t Th s prevents you from overwr t ng changes a ready comm tted by the other user After the UPDATE statement, @@ROWCOUNT s tested to see f the spec fied customer was actua y found and updated A va ue of 0 nd cates that e ther a mu t user confl ct occurred, or the spec fied CustomerId never ex sted n the first p ace The stored ­procedure treats e ther of those cond t ons as an error, and ssues a THROW statement that w ra se an e ­ xcept on on the NET s de Otherw se, the UPDATE has succeeded, and the stored procedure returns the new UpdatedAt va ue as a s ng e-row, s ng e-co umn resu t set w th the SELECT statement f­ o ow ng the UPDATE As w th InsertCustomer, th s return nformat on enab es c ent app cat ons to refresh the r oca v ew of updated customers DeleteCustomer s the ast—and s mp est—of the CRUD stored procedures for the Customer tab e It takes just an @CustomerId parameter and ssues a DELETE statement that de etes that customer’s row from the tab e

Note  Although this delete operation does not check for multiuser conflicts, business requirements will ultimately dictate how you implement a concurrency strategy in your application. You might need to be more stringent and put checks on deletions as well as updates, by requiring an @OriginalUpdatedAt parameter and then testing it in the WHERE clause like you did for UpdateCustomer. You may also need to devise a more sophisticated conflict resolution strategy than merely throwing an error. For example, you can provide a merge facility that lets the user select which version of each column should “win” for the update. Conversely, if the probability of a multiuser collision is very low in your particular application, you may be able to do away with concurrency checks altogether (that is, if a conflict does occur, just blindly let the “last user win,” and don’t worry about the few cases where they overwrite another user’s earlier changes). There s one more stored procedure n the database that s not used for CRUD operat ons The GetCustomerBalance stored procedure demonstrates how to return a sca ar va ue us ng an output parameter (as opposed to a resu t set w th one row and one co umn) Th s stored procedure returns the ba ance for the spec fied @CustomerId n the @Balance output parameter

434  Part III  Applied SQL

That comp etes the exp anat on of the database You are ready to access t from NET (and you w start to n a moment us ng convent ona ADO NET) But first, fire up a SQL Profi er trace

Monitoring Database Activity with SQL Server Profiler SQL Server Profi er (or SQL Profi er for short) s an nd spensab e too that ets you see the d rect ­commands as they are sent to the server by ADO NET on beha f of your runn ng app cat on You w use t throughout th s chapter to observe database act v ty beh nd the scenes, so now s a good t me to start trac ng Launch SQL Server Profi er from the Performance Too s fo der n the M crosoft SQL Server 2012 group on the Start menu Press CTRL+N to create a new trace, and c ck Connect to connect to the server SQL Profi er d sp ays the Trace Propert es d a og C ck ng Run w start the trace, but don’t c ck t just yet You’ want to ta or the trace first By defau t, SQL Profi er h des transact ona events from the trace, so you won’t get to see the t­ ransact ona statements be ng sent to SQL Server by ADO NET un ess you change the defau t ­sett ngs These nc ude the BEGIN TRANSACTION and COMMIT TRANSACTION statements ssued ­mp c t y by the TransactionScope object that you w use n an upcom ng examp e to batch updates to SQL Server To mon tor these events, c ck the Events Se ect on tab, check the Show A Events checkbox, scro the Events st down, and expand the Transact ons category Then check a the “comp ete” events beg nn ng w th TM (wh ch stands for Transact on Manager), such as TM Beg n Tran comp ete, TM Comm t Tran comp ete, and so on, as shown n F gure 10-2

Figure 10-2  Configur ng a SQL Profi er trace.



Chapter 10  The M crosoft Data Access Juggernaut   435

You a so want to m t the trace to show commands sent on y from your app cat on, because t w show commands sent to SQL Server from any c ent by defau t C ck the Co umn F ters button to d sp ay the Ed t F ter d a og Choose App cat onName from the st on the eft, expand the Like node n the treev ew on the r ght to create a new L ke fi ter, and type .Net SqlClient Data Provider Then c ck OK to c ose the d a og Now the trace w on y d sp ay commands ssued by the ADO NET ­Sq C ent prov der (wh ch come from your app cat on), and fi ter out “no se” generated by other runn ng app cat ons and serv ces that ta k to SQL Server (such as SQL Server Report ng Serv ces, for examp e) W th transact ona events enab ed and the App cat onName fi ter now set, c ck Run to start the trace

Tip  You can save these settings in a template and have them loaded by default whenever you start a new trace. Click the File menu, choose Templates, and select New Template. Give the template a name, and check the “Use as a default template” checkbox. Then set the transactional events and column filter in the Events Selection tab and save the template.

Conventional ADO.NET We’ve a ready defined convent ona ADO NET as the two ava ab e data access opt ons that were ­ntroduced w th NET 1 0; name y, the raw data access objects and the DataSet/DataAdapter ­abstract on over those objects You’ now exp ore both of these approaches, beg nn ng w th the raw objects

Using the Raw Data Access Objects The raw ADO NET data access objects are based on the fo ow ng set of nterfaces n the NET Framework, wh ch are defined n the System.Data namespace ■

IDbConnection

■

IDbCommand

■

IDbParameter

■

IDbDataReader

■

IDbTransaction

These nterfaces define the operat ons that can be conducted aga nst the database (or other ­ ack-end data source) by any concrete prov der that mp ements them For SQL Server, there s a b managed ADO NET prov der (ca ed Sq C ent) that mp ements these nterfaces n the fo ow ng set of c asses, wh ch are defined n the System.Data.SqlClient namespace ■

SqlConnection

■

SqlCommand

436  Part III  Applied SQL

■

SqlParameter

■

SqlDataReader

■

SqlTransaction

As the nterface and c ass names convey, an ADO NET prov der g ves you a connect on object so you can ta k to the database, a command object w th parameters so you can express your quer es and update data, a reader object to stream query resu ts nto your app cat on, and a transact on object to batch your updates atom ca y Arguab y, these raw objects are a you need to perform any SQL Server data access n NET Any conce vab e command can be sent to the database, any conce vab e resu t set can be retr eved back, and mu t p e updates can be transact ona zed Those rea y are a the p eces you need to bu d a data access ayer n NET And that s why they cont nue to serve as the under y ng objects used by a the var ous NET data access abstract ons ava ab e today But that s a so prec se y why you need to st cons der us ng them n s tuat ons that ca for t, as opposed to us ng someth ng “ ater and greater,” m sgu ded by the not on that you must conform to a more fash onab e way of do ng th ngs n order to ma nta n your cred b ty as a deve oper Our gu dance here s to not fa nto that trap, but rather st ck to the o d adage of us ng the r ght too for the r ght job (and ho d your head up h gh) It sn’t a ways necessary to p e ayers and ayers of abstract ons over each other; n many cases, t h nders more than he ps When your needs are s mp e and ghtwe ght, you can and shou d cont nue to use the raw objects to get the job done qu ck y and effic ent y, w thout the overhead of a DataSet, LINQ to SQL mode , or Ent ty Data Mode (EDM) added to the m x There’s abso ute y noth ng wrong w th hook ng up a s mp e connect on object to a command object configured w th parameters for ca ng a stored procedure, and stream ng back the resu ts w th a reader, when that’s more or ess a you need to do And so there rea y are no hard and fast ru es e ther for or aga nst us ng raw objects d rect y versus any of the ava ab e abstract ons offered by the more advanced a ternat ve techno og es It a comes down to your understand ng of each (wh ch you’ get from th s chapter), comb ned w th the proper judgment on your part (wh ch you can ga n on y from exper ence) You must deem when t s ­appropr ate to everage the benefit of (and create a dependency on) more robust f­ rameworks, ­commensurate w th the eve of comp ex ty that your app cat on ca s for, and the degree of ­abstract on you w sh to ayer over that comp ex ty W th th s understand ng n m nd, et’s start at the very beg nn ng connect ons and commands

Creating Connections and Commands You can’t do anyth ng w thout a connect on object That s obv ous At the same t me, you can’t do anyth ng w th just a connect on object You’re go ng to need to wrap a command object around that connect on W th those m n ma requ rements met, you have what you need to execute any d rect T-SQL or stored procedure on the server Add a few parameter objects to the command, and you can pass nput parameters to the server and even retr eve sca ar va ues back v a output ­parameters ­Invok ng the ExecuteNonQuery method on the command object causes the T-SQL or stored ­procedure to execute on the server F gure 10-3 dep cts the process

Chapter 10  The M crosoft Data Access Juggernaut   437

Command

Database

Connection Parameters

Figure 10-3  The raw ADO.NET objects used to execute a bas c command n SQL Server.

You’ wr te your first b t of NET code w th just these objects Start w th a s mp e command to add a new customer to the database Launch V sua Stud o, create a new V sua C# W ndows Forms ­app cat on, and name t DemoRawDataAccess Drag a button contro from the Too box and drop t onto the form Form1 created automat ca y by V sua Stud o In the Propert es w ndow, name the button btnDirectSql, and set the Text property to Direct SQL Then doub e-c ck the button V sua Stud o w automat ca y create a c ck event hand er for the button, and then open the code ed tor so that you can mp ement the hand er Add the code shown n L st ng 10-2 Listing 10-2  Perform ng a s mp e INSERT operat on us ng ExecuteNonQuery.

private void btnDirectSql_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string TSql = "INSERT INTO Customer (FirstName, LastName) VALUES ('Lukas', 'Keller')"; using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd = new SqlCommand()) { cmd.Connection = conn; cmd.CommandText = TSql; cmd.ExecuteNonQuery(); } conn.Close(); } }

The comp er s unaware that the SqlConnection and SqlCommand c ass names referred to n your code can be found n the System.Data.SqlClient namespace, un ess you d rect t to ook there Do so by add ng the fo ow ng using d rect ve at the very top of the source code using System.Data.SqlClient;

The code beg ns by defin ng the ConnStr constant, and po nts t to the samp e database runn ng on the oca SQL Server nstance (you must change the Data Source accord ng y f you have c­ reated 438  Part III  Applied SQL

the database on another server or nstance) Hardcod ng the connect on str ng n th s ­manner s ­unacceptab e n product on app cat ons But n order to best ock down concepts as you p ­ rogress, you’re start ng w th abso ute y m n ma code Later on, you’ store the connect on str ng n a ­configurat on fi e so that t can be a tered w thout recomp ng the app cat on Next, the TSql constant s defined Th s str ng conta ns the INSERT statement to be executed by SQL Server (note that you can a so batch mu t p e T-SQL statements to be executed n a s ng e ca , by separat ng them w th sem co ons) Hardcod ng the first and ast name va ues Lukas and Ke er s even more unrea st c than us ng a hardcoded connect on str ng, but aga n, you’ soon graduate to more pract ca cod ng techn ques n upcom ng examp es W th the connect on str ng and T-SQL statement defined, you’re ready to start work ng w th the raw ADO NET data objects for SQL Server A new SqlConnection object s then created and ass gned to the var ab e conn at the top of a using b ock The using statement ensures that the SqlConnection object d sposes ts resources when the ­object s no onger needed (see s debar) You w see the using statement app ed frequent y n our code samp es, and we encourage you to fo ow th s best pract ce whenever you are work ng w th d sposab e objects

Disposable Objects and the using Statement D sposab e objects are objects that, nterna y, a ocate unmanaged resources (mean ng ­resources not managed by the NET Framework, such as database connect ons) Because of the nondeterm n st c nature of NET garbage co ect on, there’s no way to know for certa n when these objects w actua y be “fina zed” (destroyed) Th s s true even after no more object ­references rema n Thus, there’s no nherent mechan sm n the NET Framework for ­automat ca y re eas ng unmanaged resources ( ke database connect ons) when they are no onger needed A d sposab e object therefore mp ements the IDisposable nterface, wh ch exposes a Dispose method that can be ca ed exp c t y as soon as you know that you no onger need t Ca ng Dispose nstructs the object to re ease ts unmanaged resources at that po nt n t me, regard ess of the fact that the object tse f won’t get destroyed by NET garbage ­co ect on unt some (poss b y d stant) future t me The using statement (wh ch s va d on y w th an IDisposable object) makes t very easy to work w th d sposab e objects (Th s s not to be confused w th the comp ete y unre ated ­using d rect ve that mports a namespace, ke you d d a moment ago ) The statement dec ares a var ab e ass gned to an IDisposable object that gets scoped to a code b ock fo ow ng the ­using statement W th th s pattern, you never actua y wr te the ne of code that ca s Dispose; nstead, the framework guarantees that Dispose gets ca ed on the object once t fa s out of scope (that s, at the end of the code b ock) Th s guarantee s abso ute y re ab e, even f an unhand ed except on occurs at any po nt w th n the using code b ock wh ch wou d cause the b ock to term nate premature y

After creat ng the SqlConnection object, the ConnectionString property s set to the prev ous y defined constant and the Open method s nvoked on t to estab sh the database connect on

Chapter 10  The M crosoft Data Access Juggernaut   439

A second using statement then creates a new SqlCommand object and ass gns t to the var ab e cmd It s perfect y okay to nest using b ocks ke th s; once the second one s entered, the framework guarantees that Dispose w get ca ed on both the SqlConnection and the SqlCommand objects regard ess of f, when, or where an except on occurs The SqlCommand object’s Connection property s then assoc ated w th the open connect on n conn, and ts CommandText property s set to the TSql constant (the T-SQL you want to execute) defined ear er Then the ExecuteNonQuery method s nvoked on t, wh ch passes the command a ong to SQL Server The command, n th s case, s a s ng e INSERT statement that adds a new row to the Customer tab e Because va ues for on y two co umns (FirstName and LastName) are spec fied, a other co umns n the new Customer row (bes des CustomerId) w be ass gned the defau t va ues des gnated for them when the Customer tab e was defined at the beg nn ng of the chapter The CustomerId co umn w be ass gned automat ca y to the next ava ab e dent ty va ue, because IDENTITY was spec fied on that pr mary key co umn n the Customer tab e defin t on After the command executes on the server, the nner using b ock c oses, wh ch mp c t y d sposes the SqlCommand object and dereferences the cmd var ab e F na y, you nvoke the Close method on the connect on and the outer using b ock c oses, wh ch mp c t y d sposes the SqlConnection object and dereferences the conn var ab e Run the code and c ck the button Then se ect the new row to v ew t SELECT * FROM Customer

The output confirms that the Customer tab e now has one row n t, the one you just added for Lukas Ke er CustomerId ---------1

FirstName --------Lukas

LastName -------Keller

Balance ------0.00

CreatedAt ----------------------2012-04-06 12:34:16.50

UpdatedAt ---------------------2012-04-06 12:34:16.50

As expected, the CustomerId co umn was ass gned a va ue of 1, and the Balance, CreatedAt, and UpdatedAt co umns were ass gned the defau t va ues of 0, SYSDATETIME, and SYSDATETIME, respect ve y These va ues were ass gned automat ca y based on the IDENTITY spec ficat on and defau t va ues for the Customer tab e defin t on n L st ng 10-1

Using Parameters In pract ce, you wou d never hardcode va ues for a new customer ke you just d d The va ues wou d come from e sewhere, and then get passed nto your app cat on’s data access ayer You can certa n y use str ng concatenat on to embed the nput va ues nto a dynam ca y composed T-SQL statement, but do ng so exposes your app cat on to potent a SQL nject on attacks (a secur ty r sk that we exp a n n Chapter 5) Furthermore, your code wou d need to san t ze the nput va ues (for examp e, by doub ng up tera s ng e quotat on characters n str ngs), or you further r sk runt me except ons resu t ng from str ngs that conta n T-SQL syntax errors So the proper way to hand e var ab e va ues n T-SQL s to use parameters, as demonstrated by a mod fied vers on of the prev ous code

440  Part III  Applied SQL

Drag two text box contro s from the Too box and drop them onto the form In the Propert es w ndow, name the text boxes txtFirstName and txtLastName Now drag a button contro from the Too box and drop t onto the form, name t btnParameterizedDirectSql, and set the Text property to Parameterized Direct SQL Then doub e-c ck the button and add the code n L st ng 10-3 Listing 10-3  Us ng parameters w th a command object.

private void btnParameterizedDirectSql_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string TSql = "INSERT INTO Customer (FirstName, LastName) VALUES (@FirstName, @LastName)"; using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd = new SqlCommand()) { cmd.Connection = conn; cmd.CommandText = TSql; cmd.Parameters.AddWithValue("@FirstName", this.txtFirstName.Text); cmd.Parameters.AddWithValue("@LastName", this.txtLastName.Text); cmd.ExecuteNonQuery(); } conn.Close(); } }

Th s vers on of the code s a most dent ca to the prev ous vers on, but has some mportant ­ fferences You’ not ce that the hardcoded first and ast name va ues n the INSERT statement d defined by the TSql constant have been rep aced w th parameter p aceho ders The parameter names, a ong w th the r va ues wh ch you get from the text boxes on the form, are added to the Parameters co ect on of the SqlCommand object w th the AddWithValue method

More Info  AddWithValue is a very convenient method. In one line of code, it creates a new SqlParameter object with its ParameterName and Value properties set, and adds the SqlParameter object to the Parameters collection. More elaborate ways of c­ onfiguring the Parameters collection give you fine control over the data type, length, precision, direction (input or output), and source column mapping of each parameter. You shall see an example of this shortly, when you learn how to return scalar values ­using ­output parameters. But for simple input parameters such as the ones in this example, c­ alling the AddWithValue method is sufficient.



Chapter 10  The M crosoft Data Access Juggernaut   441

By us ng parameters (rather than str ng concatenat on), your code s not vu nerab e to SQL nject on attacks, nor do you need to concern yourse f w th san t z ng the first and ast name va ues ADO NET w automat ca y parse the command text and parameter va ues to produce a T-SQL statement that s both safe and va d for execut on (for examp e, t w doub e-up tera s ng e quotat on marks entered n e ther the first or ast name text box) Now go ahead and run the app cat on Type va ues n the first and ast name text boxes, and c ck the button to create a new customer row Then, to confirm, se ect and v ew the new customer as you d d after creat ng the first customer

Calling Stored Procedures So far, you’ve used the SqlCommand object to execute a d rect T-SQL statement on the server and nsert a new customer row However, t’s a better pract ce to use stored procedures rather than d rect SQL to ­perform such updates Us ng stored procedures draws a c ear separat on between NET code and T-SQL code Pr mary benefits to th s approach nc ude ma nta nab ty, reusab ty, and secur ty D rect SQL can qu ck y ead to code dup cat on and esca ate nto a ma ntenance n ghtmare, w th NET and T-SQL og c ntertw ned and t ght y coup ed n your app cat on Good arguments can be made on both s des of the great “d rect SQL versus stored procedure debate” (read the s debar n the upcom ng ADO NET Ent ty Framework coverage), but the fact s that somet mes you just don’t have the cho ce It s very common pract ce for database adm n strators n the enterpr se to ock down tab es and deny d rect access to them by c ent app cat ons The database nstead exposes a “serv ce ayer” of stored procedures that nd rect y prov de contro ed access to the under y ng tab es Th s s unquest onab y far more secure than just ­a ow ng c ents to execute any SQL that they want d rect y aga nst your tab es Argu ng the mer ts and d sadvantages between d rect SQL and stored procedures s po nt ess n such env ronments—you’re stuck w th us ng stored procedures and that’s a there s to t Now rev se the code to ca the InsertCustomer stored procedure nstead of us ng d rect SQL Drag another button contro from the Too box and drop t onto the form, name t btnStoredProcedureInsert, and set ts Text property to Stored Procedure INSERT Then doub e-c ck the button and add the code n L st ng 10-4 Listing 10-4  Ca ng a stored procedure w th parameters.

private void btnStoredProcedureInsert_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string StoredProcName = "InsertCustomer"; using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd = new SqlCommand())

442  Part III  Applied SQL

{ cmd.Connection = conn; cmd.CommandText = StoredProcName; cmd.CommandType = CommandType.StoredProcedure; cmd.Parameters.AddWithValue("@FirstName", this.txtFirstName.Text); cmd.Parameters.AddWithValue("@LastName", this.txtLastName.Text); cmd.ExecuteNonQuery(); } conn.Close(); } }

Aga n, not ce the two d fferences from the prev ous examp e F rst, rather than cod ng the ­actua T ­ -SQL INSERT statement r ght ns de the C# method, you refer on y to the name of the stored ­procedure n the database, InsertCustomer The og c for add ng the new customer row, ­nc ud ng the actua ­INSERT ­statement, s conta ned n the stored procedure Second, you nform the ­SqlCommand object that the str ng you have set n ts CommandText property s the name of a stored p ­ rocedure and not a d rect T-SQL statement You do th s by sett ng ts CommandType property to C ­ ommandType.StoredProcedure ­(overr d ng the defau t va ue of CommandType.Text for d rect T-SQL) The rest ooks and works the same G ve the code a run to add a th rd customer, and then v ew the new customer row as before But the InsertCustomer stored procedure does more than just execute the INSERT statement to add the new customer row Referr ng back to L st ng 10-1, reca that t a so executes a SELECT ­statement r ght after the INSERT to return co umns va ues ass gned to the new row by the stored ­procedure In part cu ar, the SELECT statement returns a s ng e-row resu t set to the c ent that ­conta ns the un que pr mary key va ue ass gned to CustomerId (by v rtue of ts IDENTITY ­spec ficat on), and the current server date and t me va ues ass gned to CreatedAt and UpdatedAt (by v rtue of the r defau t va ues of SYSDATETIME, the bu t- n funct on that returns the server c ock’s current date and t me as a datetime2 data type) In th s manner, the c ent can obta n va ues ass gned to a new customer by SQL Server when contro s returned from the stored procedure that creates the new customer, rather than ssu ng another command (and ncurr ng another server round tr p) to retr eve them You can obta n a data reader to retr eve these va ues from the stored procedure by nvok ng ExecuteReader rather than ExecuteNonQuery on the SqlCommand object A though data readers are most common y used to retr eve mu t p e rows of data from SQL Server, they are a so used to retr eve a s ng e-row resu t set n cases ke th s In the next sect on, you’ exper ment w th both use cases

Iterating Data Readers If your T-SQL statement or stored procedure ca returns a resu t set (or mu t p e resu t sets), you retr eve t (or them) us ng a data reader (a so ca ed a connected data reader, and somet mes referred to as a “firehose cursor”) Data s streamed from SQL Server d rect y nto your app cat on as qu ck y as poss b e v a a data reader n a read-on y, forward-on y fash on F gure 10-4 dep cts the process



Chapter 10  The M crosoft Data Access Juggernaut   443

Command

Database

Connection

DataReader

Parameters

Figure 10-4  The raw ADO.NET objects used to execute a command n SQL Server that returns resu ts n a data

reader.

The SqlDataReader advances through the resu t set one row at a t me, g v ng your code the ­ pportun ty to extract the co umn va ues of each row What you do w th the row data s ent re y up o to you; you can nstant ate and popu ate objects, or you can process one row at a t me w thout ever cach ng the data (but do rema n cogn zant of the fact that the database connect on rema ns open wh e you terate the reader; you shou d genera y str ve to keep connect ons open for the shortest durat on of t me poss b e) The connect on, command, and parameter objects get configured just as you’ve been do ng a a ong, on y now you nvoke the ExecuteReader method rather than ExecuteNonQuery aga nst the SqlCommand object to execute the command Interna y, the ExecuteReader method nvokes the command on the server just ke ExectuteNonQuery does, but then a so creates a new ­SqlDataReader object pos t oned at the very beg nn ng of the resu t set to be streamed back, and returns the ­SqlDataReader for your code to terate one row at a t me Th s s the on y way to create a reader ­object The SqlDataReader c ass has no constructor, so you cannot nstant ate one w th the new ­keyword; you can on y obta n a new reader by ca ng ExecuteReader You w now use a data reader to retr eve the CustomerId, CreatedAt, and UpdatedAt va ues ­ ss gned to the new y nserted row (for a new customer, the same va ue s expected to returned for a CreatedAt and UpdatedAt) Drag another button contro from the Too box and drop t onto the form, name t btnStoredProcedureInsertWithReader, and set the Text property to Stored Procedure INSERT with Reader Then doub e-c ck the button and add the code n L st ng 10-5 Listing 10-5  Us ng a data reader to retr eve va ues ass gned to a new row added by the InsertCustomer stored

procedure.

private void btnStoredProcedureInsertWithReader_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string StoredProcName = "InsertCustomer"; using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd = new SqlCommand()) {

444  Part III  Applied SQL

cmd.Connection = conn; cmd.CommandText = StoredProcName; cmd.CommandType = CommandType.StoredProcedure; cmd.Parameters.AddWithValue("@FirstName", this.txtFirstName.Text); cmd.Parameters.AddWithValue("@LastName", this.txtLastName.Text); using (var rdr = cmd.ExecuteReader(CommandBehavior.CloseConnection)) { rdr.Read(); var customerId = rdr.GetInt64(0); var createdAt = rdr.GetDateTime(1); var updatedAt = rdr.GetDateTime(2); rdr.Close(); MessageBox.Show( string.Format("Customer ID {0}; Created at {1}; Updated at {2}", customerId, createdAt, updatedAt)); } } } }

Everyth ng s the same unt t comes t me to execute the command, at wh ch po nt you nvoke E­ xecuteReader to obta n a SqlDataReader object ass gned to the var ab e rdr Because SqlDataReader s a d sposab e object, you ca ExecuteReader w th a using statement b ock to ensure that the reader w a ways be d sposed proper y You shou d c ose a data reader when you are done work ng w th t, after wh ch t me you shou d a so c ose the under y ng database connect on The CommandBehavior. CloseConnection enumerat on passed to ExecuteReader offers a conven ence here, by nstruct ng ADO NET to c ose the SqlConnection automat ca y when you c ose the SqlDataReader Therefore, you’ not ce that the ne conn.Close() has been removed from th s vers on of the code, because the ­connect on w be c osed automat ca y when the reader s c osed on the ne rdr.Close() When you rece ve the reader, t s pos t oned before the first row (wh ch n th s case s the on y row) n the resu t set Invok ng the Read method advances the reader to the row so you can retr eve ts ­va ues The reader exposes a set of GetXxx methods for a ava ab e data types, each of wh ch accepts an nteger parameter that spec fies the ord na pos t on of the co umn from wh ch to ­retr eve the va ue Because the row s expected to return one ong (64-b t) nteger for the new CustomerId and two date/t me va ues for CreatedAt and UpdatedAt— n that order—you ca the GetInt64 and ­GetDateTime methods pass ng n the va ues 0, 1, and 2 respect ve y You can a so extract co umn va ues by access ng the SqlDataReader object us ng ndexed notat on In th s examp e, you cou d have obta ned the three va ues by cod ng rdr[0], rdr[1], and rdr[2], rather than us ng the GetXxx methods W th th s approach, co umn va ues are returned as the type object, so you must cast or convert to the des red type If you know the co umn names, you can use a str ng ndex rather than a numer c ndex, such as rdr[“CustomerId”], rdr[“CreatedAt”], and rdr[“UpdatedAt”] (though be aware that do ng so s s ght y s ower than us ng a numer c ndex, because ADO NET needs to find the des red co umn matched by name) You w use str ng ndexes n the next data reader examp e



Chapter 10  The M crosoft Data Access Juggernaut   445

More Info  All techniques for extracting data reader values require that you know the data type of the column you are accessing. In this case, as in most, you will have that k­ nowledge at design time. However, there are some scenarios in which you may not know what a ­column’s data type is, or indeed, what the actual columns being returned are altogether. You can call the reader’s GetDataTypeName method and pass it a column index to discover the data type of any particular column. If you need to dynamically discover all the columns in the result set being returned, you can call the GetSchemaTable method on the reader to obtain a DataTable populated with schema information. You can then glean the result set schema (before streaming in from the reader) by examining the DataTable. It will contain one DataRow per result set column, populated with descriptive information about each ­column (name, data type, scale, precision, and so on). Run the examp e now to add a fourth row to the Customer tab e The MessageBox d a og w d sp ay the CustomerId (wh ch shou d be 4, f you ran the three prev ous examp es), CreatedAt, and UpdatedAt va ues returned by the stored procedure Because the InsertCustomer stored procedure returns a resu t set that s known to a ways conta n one and on y one row, the code s based on that as be ng a re ab e assumpt on That s, the Read method s ca ed on the SqlDataReader uncond t ona y, and exact y once In many other scenar os, you w not know f you are gett ng back zero, one, or more rows n the reader, and so you can’t make any assumpt ons Your code w need to test the reader for an end-of-stream cond t on, because an except on w occur when you attempt to retr eve data f there are no rows n the resu t set, or f you have a ready advanced beyond the ast row n the resu t set The next examp e demonstrates how to mp ement th s pattern You w ca the SelectCustomers stored procedure to return the ent re set of customers, and then bu d a st of d sp ay str ngs formatted as “ ast name, first name ” Drag a st box contro from the Too box and drop t onto the form In the Propert es w ndow, name the st box lstNames Now drag a button contro from the Too box and drop t onto the form, name t btnStoredProcedureWithReader, and set the Text property to Stored Procedure with Reader Then doub e-c ck the button and add the code n L st ng 10-6 Listing 10-6  Process ng a data reader.

private void btnStoredProcedureWithReader_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string StoredProcName = "SelectCustomers"; var names = new List(); using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr;

446  Part III  Applied SQL

conn.Open(); using (var cmd = new SqlCommand()) { cmd.Connection = conn; cmd.CommandText = StoredProcName; cmd.CommandType = CommandType.StoredProcedure; using (var rdr = cmd.ExecuteReader(CommandBehavior.CloseConnection)) { while (rdr.Read()) { var name = string.Format("{0}, {1}", rdr["LastName"], rdr["FirstName"]); names.Add(name); } rdr.Close(); } } } this.lstNames.DataSource = names; }

Th s t me, the SqlCommand object’s CommandText property s set to po nt to the SelectCustomers stored procedure Then ExecuteReader s nvoked to obta n a SqlDataReader object that returns the resu ts as before Th s t me, however, there s no way to know how many rows w come back So the code mp ements a while b ock, and tests for an end-of-stream cond t on before each ca to the reader’s Read method as t oops through the resu t set If there are no customers, Read w return true the very first t me t s ca ed, and the while b ock w never execute Otherw se, the b ock w execute once for each customer row returned by the stored procedure For each row, a str ng ndex s used aga nst the SqlDataReader to extract the LastName and ­FirstName va ues so the customer name can be formatted (th s s a more conven ent, a be t s ght y s ower a ternat ve to ca ng GetString on the reader and pass ng n the ord na co umn pos t ons of the name fie ds) Each formatted name s added to the List co ect on that s n t a zed before creat ng the SqlConnection After c os ng the reader (wh ch, because CommandBehavior.CloseConnection was spec fied, a so c oses the connect on), the DataSource property of the st box b nds t to the str ng co ect on G ve the app cat on a run, and confirm that the st box d sp ays the names of a the customers you added n the ear er exerc ses

Returning Scalar Values In the first data reader examp e, you retr eved three sca ar va ues back from ca ng the ­InsertCustomer stored procedure If you on y need to return a s ng e sca ar va ue, ADO NET offers a shortcut that ­e m nates the need to work w th a data reader a together By nvok ng the ExecuteScalar method (rather than ExecuteReader) on the SqlCommand object, ADO NET s mp y returns the va ue n the first co umn of the first row of whatever resu t set gets returned by SQL Server If there are more co umns (or rows), they are gnored, and f no resu t set at a gets returned, an except on s thrown The sca ar va ue s returned as the type object, wh ch you then cast or convert as necessary to the expected data type

Chapter 10  The M crosoft Data Access Juggernaut   447

The UpdateCustomer stored procedure s a good examp e It returns the one and on y va ue ­affected by an update operat on, name y the UpdatedAt co umn va ue ass gned by the ­SYSDATETIME funct on ( n L st ng 10-1, th s s the ne of code that reads SELECT UpdatedAt = @NewUpdatedAt) A though you can use the same data reader approach you used when ca ng InsertCustomer (where you needed to retr eve three va ues), you can retr eve the new UpdatedAt va ue returned by ­UpdateCustomer w th ess code by us ng ExecuteScalar nstead of ExecuteReader, as shown n the fo ow ng code sn ppet // This code... using (var rdr = cmd.ExecuteReader()) { rdr.Read(); var updatedAt = rdr.GetDateTime(0); rdr.Close(); } // ...is equivalent to this code var updatedAt = Convert.ToDateTime(cmd.ExecuteScalar());

The ExecuteScalar method s c ear y a cod ng conven ence, but don’t th nk that t works any faster than us ng ExecuteReader A the work st goes on beh nd the scenes to bu d and return a resu t set that could potent a y conta n mu t p e co umns and rows, and then you’re say ng that you on y want the first co umn of the first row That can amount to need ess server overhead and network transport, cons der ng that you can return output parameters to send sca ar va ues back from SQL Server W th output parameters, you don’t use a s ng e-row resu t set to return sca ar va ues (a though you can effect ve y comb ne the use of output parameters w th mu t -row resu t sets to return both sca ar and tabu ar data) Instead, you add a Parameter object correspond ng to each output parameter expected by the stored procedure and set ts Direction property to ParameterDirection.Output. Then you ca ExecuteNonQuery to run the stored procedure After the stored procedure executes, you can extract the va ue(s) set by the server from the Value property of the Parameter object(s) The GetCustomerBalance stored procedure returns the ba ance of any spec fied customer us ng an output parameter Let’s see how you wr te the ADO NET code to ca GetCustomerBalance and retr eve ts return va ue Drag another text box contro from the Too box and drop t onto the form In the Propert es w ndow, name the text box txtCustomerId Now drag a button contro from the T ­ oo box and drop t onto the form, name t btnOutputParameter, and set the Text property to Output ­Parameter Then doub e-c ck the button and add the code n L st ng 10-7 Listing 10-7  Retr ev ng an output parameter va ue returned by a stored procedure.

private void btnOutputParameter_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string StoredProcName = "GetCustomerBalance"; var customerId = Convert.ToInt64(this.txtCustomerId.Text); decimal balance;

448  Part III  Applied SQL

using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd = new SqlCommand()) { cmd.Connection = conn; cmd.CommandText = StoredProcName; cmd.CommandType = CommandType.StoredProcedure; cmd.Parameters.AddWithValue("@CustomerId", customerId); var outputParam = new SqlParameter() { ParameterName = "@Balance", SqlDbType = SqlDbType.Money, Direction = ParameterDirection.Output }; cmd.Parameters.Add(outputParam); cmd.ExecuteNonQuery(); balance = (decimal)outputParam.Value; } conn.Close(); } MessageBox.Show(string.Format("Customer ID {0} balance is {1:c}", customerId, balance)); }

The stored procedure accepts two parameters The first one s the @CustomerId nput parameter (a 64-b t nteger; bigint n SQL Server, correspond ng to long n NET), and the second one s the @Balance output parameter (a money va ue n SQL Server, correspond ng to decimal n NET) The customer ID s obta ned from the va ue entered n the text box and converted nto a 64-b t nteger n customerId that w be passed for the @CustomerId nput parameter Then the balance var ab e s dec ared as a decimal that w get ass gned the va ue returned by the @Balance output parameter returned by the stored procedure, after t executes a few nes further down n the code After the connect on and command objects are configured, you add the parameters The @CustomerId nput parameter s added by ca ng AddWithValue, as you’ve been do ng thus far The @Balance output parameter, however, cannot be added n just a s ng e ne of code Instead, you need to exp c t y nstant ate the new SqlParameter nstance yourse f, set the ­appropr ate propert es, and add t to the Parameters co ect on Spec fica y, you set the ParameterName, SqlDbType, and ­Direction propert es to @Balance, SqlDbType.Money, and ParameterDirection.Output, respect ve y W th the Parameters co ect on proper y configured, ExecuteNonQuery runs the stored procedure When contro returns to the app cat on, you extract the output parameter va ue set by the stored procedure nto the balance var ab e Run the app cat on now and g ve t a try Enter any ex st ng customer ID (you shou d have four customers w th IDs 1 through 4) and c ck the button to run the stored procedure and confirm that the ba ance s be ng returned v a the output parameter A the customers entered to th s po nt have zero ba ances (because you’ve not yet exp c t y prov ded a va ue for the Balance co umn, wh ch

Chapter 10  The M crosoft Data Access Juggernaut   449

has a ­defau t va ue of 0 00) In the next sect on, you’ perform some updates that change customer ­ba ances, after wh ch you can run th s code aga n to ver fy that the stored procedure s return ng ­ba ance data as expected

Batching Updates with Transactions The ast raw ADO NET object eft to cover s a so, nterest ng y, the on y one wh ch you shou d no ­onger use, and that s SqlTransaction object ADO NET 2 0 ntroduced the TransactionScope object, wh ch s now the recommended way to manage transact ons at the app cat on eve w th ADO NET The SqlConnection object has a BeginTransaction method that, at the t me you nvoke t, ssues a BEGIN TRANSACTION command to SQL Server over the open connect on A SqlTransaction ­object s returned back for you to ho d as a hand e onto the pend ng transact on You then perform the ­necessary updates, sett ng the Transaction property of each SqlCommand object nvo ved n the ­updates to the SqlTransaction object F na y, you nvoke e ther the CommitTransaction or RollbackTransaction method on the SqlTransaction object, depend ng on the success or fa ure of a the updates These methods ssue, respect ve y, a COMMIT TRANSACTION or ROLLBACK TRANSACTION command to the server Because your NET code exp c t y ssues methods that trans ate d rect y to T-SQL statements contro ng transact ons, t s sa d to be manag ng explicit transact ons There are a number of pa n po nts w th exp c t transact ons The first d fficu ty es n the r­ equ rement that every SqlCommand object used to perform updates ns de the transact on must have ts Transaction property set to the SqlTransaction object returned by BeginTransaction Th s means that you must take care to pass a ong the SqlTransaction object to any p ace n your code that performs an update, because fa ng to ass gn t to the Transaction property of every SqlCommand object n the transact on resu ts n a runt me except on The ssue s compounded when you need to track the SqlTransaction object across mu t p e method ca s that perform updates It becomes even harder to manage th ngs when these methods need to be flex b e enough to work whether or not a ­transact on s nvo ved or requ red The prob em s worse when work ng w th any of the other techno og es we’ be cover ng ater that prov de abstract on ayers over the raw objects For examp e, because a SqlDataAdapter a ­ ctua y wraps three d st nct SqlCommand objects (for nsert, update, and de ete), you must d g beneath the data adapter and hook nto ts three under y ng command objects so that you can set the r ­Transaction propert es (We don’t genera y recommend that you m x and match d fferent data ­access APIs w th n your app cat on, but f you must transact ona ze updates across a comb nat on of ­techno og es, ­mp c t transact ons make t easy ) The TransactionScope object, ntroduced as part of a ded cated transact on management API w th NET 2 0 n 2005, ets you code transact ons mp c t y Th s s a super or approach that re eves you from a of the aforement oned burdens assoc ated w th exp c t transact ons So the gu dance here s to a ways work w th mp c t transact ons whenever poss b e You w wr te ess code that s more flex b e when you a ow the framework to hand e transact on management for you However, t s st a so mportant to understand exp c t transact ons w th the SqlTransaction object, as you m ght need to ntegrate w th and extend ex st ng code that a ready uses exp c t transact ons Therefore, we w cover both transact on management sty es to prepare you for a s tuat ons 450  Part III  Applied SQL

More Info  The transaction management API offers many more benefits besides i­mplicit transactions. For example, TransactionScope is capable of promoting a ­lightweight ­transaction (one associated with a single database) to a distributed transaction ­automatically, if and when your updates involve changes across multiple databases. Furthermore, you can enlist updates other than database changes to participate in the transaction. Chapter 4 covers these concepts in more detail. The next examp e demonstrates how to code exp c t transactons It performs a ba ance transfer between customers, a ow ng a port on of one customer’s ba ance to be subtracted, and then added to another customer’s ba ance Both update operat ons obv ous y need to succeed or fa as a who e, and so they must be transact ona zed Drag three text box contro s from the Too box and drop them onto the form In the ­Propert es w ndow, name the text boxes txtTransferFromCustomerId, txtTransferToCustomerId, and ­t xtTransferAmount Now drag a button contro from the Too box and drop t onto the form, name t btnUpdateWithExplicitTransaction, and set the Text property to Update with Explicit ­Transaction Then doub e-c ck the button and add the code n L st ng 10-8 Listing 10-8  Updat ng data us ng exp c t transact ons.

private void btnUpdateWithExplicitTransaction_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string TSql = "UPDATE Customer SET Balance += @Transfer WHERE CustomerId = @CustomerId"; var fromCustomerId = Convert.ToInt64(this.txtTransferFromCustomerId.Text); var toCustomerId = Convert.ToInt64(this.txtTransferToCustomerId.Text); var transferAmount = Convert.ToDecimal(this.txtTransferAmount.Text); using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var txn = conn.BeginTransaction()) { try { using (var cmd1 = new SqlCommand()) { cmd1.Connection = conn; cmd1.CommandText = TSql; cmd1.Transaction = txn; cmd1.Parameters.AddWithValue("@CustomerId", fromCustomerId); cmd1.Parameters.AddWithValue("@Transfer", decimal. Negate(transferAmount));



Chapter 10  The M crosoft Data Access Juggernaut   451

cmd1.ExecuteNonQuery(); } using (var cmd2 = new SqlCommand()) { cmd2.Connection = conn; cmd2.CommandText = TSql; cmd2.Transaction = txn; cmd2.Parameters.AddWithValue("@CustomerId", toCustomerId); cmd2.Parameters.AddWithValue("@Transfer", transferAmount); cmd2.ExecuteNonQuery(); } txn.Commit(); } catch { txn.Rollback(); throw; } } conn.Close(); } }

Make sure that your SQL Server Profi er trace s proper y configured and runn ng, as we exp a ned ear er (see F gure 10-2) Then set a breakpo nt at the top of the method n L st ng 10-8 and run the code Enter two ex st ng customer IDs and an amount to transfer, c ck the Update with Explicit ­Transaction button, and h t the breakpo nt Then step through the code one ne at a t me as you mon tor database act v ty n the SQL Profi er trace Two SqlCommand objects are nvo ved n the ba ance transfer, both of wh ch execute the ­UPDATE statement defined by the TSql constant The UPDATE statement accepts the two ­parameters @ CustomerId and @TransferAmount and adjusts the Balance of the spec fied customer by the ­spec fied amount The statement s executed first for the source customer, pass ng n the negated transfer amount (th s subtracts from the first customer’s ba ance), and s then executed aga n for the target customer, pass ng n the actua transfer amount (th s adds to the second customer’s ba ance) C ear y, you must use a transact on to ensure that both operat ons succeed or fa together To transact ona ze these updates exp c t y, you nvoke BeginTransaction on the SqlConnection to start the transact on (the SQL Profi er trace shows that th s method sends a BEGIN TRANSACTION statement to SQL Server) and obta n a SqlTransaction object ass gned to the var ab e txn Because SqlTransaction s a d sposab e object, you ca BeginTransaction w th a using statement b ock to ensure that the transact on gets d sposed of proper y at the end of the b ock The two updates are then ­executed us ng the standard ExecuteNonQuery pattern, w th the add t ona step of sett ng the ­Transaction property of both SqlCommand objects to the SqlTransaction After the second update, you ca txn.CommitTransaction On y then are both updates ­permanent y app ed to the database (the SQL Profi er trace shows that th s method sends a ­COMMIT ­TRANSACTION statement to SQL Server) The two updates are wrapped n a try/catch b ock so 452  Part III  Applied SQL

that you can ca txn.RollbackTransaction n the event of an error (th s wou d send a ROLLBACK ­TRANSACTION statement to SQL Server) Of course, SQL Server w never comm t the transact on f txn.CommitTransaction doesn’t execute, and w eventua y ro t back anyway, so the try/catch b ock sn’t str ct y necessary But t’s a bad dea to keep transact ons pend ng any onger than necessary; you shou d nstead hand e except ons that occur ns de transact ons so you can ro back mmed ate y In th s case, you’re not rea y hand ng the except on n the catch b ock; you just want to ensure an exp c t ro back on error So the except on s then s mp y rethrown (and now needs to be hand ed further up the stack) after ca ng txn.RollbackTransaction Now cons der a more rea st c scenar o w th mu t p e updates mp emented n separate methods It then becomes a greater burden to manage the transact on, because you need to pass the SqlTransaction object around and make sure t gets ass gned to the Transaction property of every SqlCommand object execut ng w th n the transact on Th ngs are much eas er us ng mp c t transact ons, as you’ see now

Tip  Although our coverage of implicit transactions is provided here in the section on raw data access objects, it pertains to all the layered technologies as well. You should i­mplicitly transactionalize your updates using a TransactionScope block (as demonstrated here) whether you are working with raw objects, DataAdapters with DataSets, LINQ to SQL, or Entity Framework. Drag another button contro from the Too box and drop t onto the form, name t ­ tnUpdateWithImplicitTransaction, and set the Text property to Update with Implicit b ­Transaction Then doub e-c ck the button and add the code n L st ng 10-9 Listing 10-9  Updat ng data us ng mp c t transact ons.

private void btnUpdateWithImplicitTransaction_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string TSql = "UPDATE Customer SET Balance += @Transfer WHERE CustomerId = @CustomerId"; var fromCustomerId = Convert.ToInt64(this.txtTransferFromCustomerId.Text); var toCustomerId = Convert.ToInt64(this.txtTransferToCustomerId.Text); var transferAmount = Convert.ToDecimal(this.txtTransferAmount.Text); using (var ts = new TransactionScope()) { using (var conn = new SqlConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd1 = new SqlCommand()) { cmd1.Connection = conn;



Chapter 10  The M crosoft Data Access Juggernaut   453

cmd1.CommandText = TSql; cmd1.Parameters.AddWithValue("@CustomerId", fromCustomerId); cmd1.Parameters.AddWithValue("@Transfer", decimal.Negate(transferAmount)); cmd1.ExecuteNonQuery(); } using (var cmd2 = new SqlCommand()) { cmd2.Connection = conn; cmd2.CommandText = TSql; cmd2.Parameters.AddWithValue("@CustomerId", toCustomerId); cmd2.Parameters.AddWithValue("@Transfer", transferAmount); cmd2.ExecuteNonQuery(); } conn.Close(); } ts.Complete(); } }

The key object here s TransactionScope, wh ch s not techn ca y part of ADO NET TransactionScope s prov ded by the transact on management API ntroduced w th NET 2 0, and s conta ned n the ­System. Transactions assemb y (deta ed coverage of th s API s prov ded n Chapter 4, wh ch s ded cated to ­transact ons) So before th s code w comp e, you need to reference that assemb y and mport ts namespace w th a using d rect ve R ght-c ck the project n So ut on Exp orer and choose Add Reference In the Add Reference d a og, c ck the NET tab, scro to find the System.Transactions component, and doub e-c ck t Then add the fo ow ng using d rect ve at the very top of the source code using System.Transactions;

The project shou d now bu d successfu y Once aga n, start a SQL Profi er trace and s ng estep through the code You’ now see how exp c t transact ons behave d fferent y than mp c t ­transact ons Rather than exp c t y beg nn ng, comm tt ng, and ro ng back transact ons, you nstead wrap your updates ns de of a TransactionScope b ock Then you s gna success by nvok ng the Complete method on the TransactionScope object before the end of the b ock, such that an except on occurr ng w th n the b ock means that Complete won’t get ca ed SQL Profi er revea s that the transact on does not start when you nstant ate the TransactionScope, nor does t comm t when you nvoke ts Complete method Rather, the TransactionScope b ock mere y declares that a updates ns de the b ock need to be transact ona zed Th s nc udes ca s ns de the b ock that you make to other methods wh ch may themse ves dec are add t ona TransactionScope b ocks that are nested ns de the ca er’s b ock You can nest TransactionScope b ocks as many eve s as you need to As ong as each b ock nvokes the Complete method before t ends (and no unhand ed except ons occur w th n any of the b ocks), the transact on w comm t when the outermost b ock term nates Nest ng b ocks prov des you w th great flex b ty, because the transact on s a ways created and comm tted at the outermost b ock eve Nested b ocks s mp y p ggyback off that transact on The outermost b ock creates and m ­ a nta ns 454  Part III  Applied SQL

a s ng e transact on assoc ated w th the database connect on, even f the database connect on s opened and c osed mu t p e t mes ns de of nested b ocks Th s s because ADO NET s smart enough not to actua y c ose and reopen the connect on wh e ns de of a pend ng mp c t transact on W th th s scheme, transact ons get started and comm tted automat ca y as needed at runt me So the actua database transact on beg ns mp c t y, some time after the outermost TransactionScope b ock s entered The SQL Profi er trace shows that ADO NET ssues the BEGIN TRANSACTION statement to SQL Server when the connect on s opened, severa nes nto the TransactionScope b ock S m ar y, the database transact on comm ts mp c t y, just after the outermost TransactionScope b ock s ex ted (aga n, under the cond t on that Complete has been ca ed w th n the b ock and a nested b ocks) The trace shows a COMMIT TRANSACTION statement sent to SQL Server just after the TransactionScope b ock ends, even though that occurs after you c ose the connect on (wh ch, be ng ns de a pend ng mp c t ­transact on, won’t actua y c ose the connect on) and ca the Complete method Un ke w th exp c t transact ons, there s no transact on object for you to pass around and keep track of, nor do you need to worry about s­ ett ng the Transaction property of every SqlCommand object that s part c pat ng n the update Your on y ­concern s to ca Complete before the TransactionScope b ock ends, and th s dramat ca y s mp fies the task of wr t ng transact ona code

More Info  Chapter 4 is dedicated to the subject of atomic database transactions, and includes ­detailed discussion on the various transaction isolation levels supported by SQL Server. In that chapter, we show how you can control these isolation levels from the ­application layer using ADO.NET as well. Now that you know to use the raw data access objects, you can advance to the second component of convent ona ADO NET, the DataSet

Working with DataSets Techn ca y, one can create a data access ayer ent re y on just the raw objects— ndeed, many ­successfu app cat ons have been bu t that way St , M crosoft cou d not rea st ca y have re eased the NET Framework n t a y w thout a so prov d ng the DataSet Do ng so wou d have been an unacceptab e step backward for COM-based deve opers a ready accustomed to the conven ence of the RecordSet object In c ass c ADO, the RecordSet prov des a scro ab e cursor that can random y nav gate a resu t set returned by SQL Server And so, the DataSet was ntroduced n ADO NET 1 0 as a d sconnected, mu t -tab e cache to rep ace the (usua y) connected s ng e-tab e RecordSet cursor n c ass c ADO There s by now a great dea of ex st ng DataSet-based NET code n product on that st won’t be phased out for 5 years or more (some compan es are st runn ng code today that s eas y 10 or more years o d) P us, there may st be t mes when t s appropr ate to use DataSets So deve opers w need to keep the r DataSet sk s sharp for a ong t me to come



Chapter 10  The M crosoft Data Access Juggernaut   455

To Use or Not to Use DataSets Shou d you work w th DataSets? That has been the subject of much debate s nce the dawn of NET, and s even more debatab e w th the advent of S ver ght, ADO NET Ent ty ­Framework, and the newest W nRT runt me n W ndows 8 Some deve opers d sm ss DataSets out of hand, pr mar y because—desp te the r ab ty to be strong y typed and to encapsu ate ­bus ness ­og c—they are not true bus ness objects For examp e, you need to nav gate through ­re at onsh p objects n the DataSet to connect between parent and ch d rows Th s s not ­ntu t ve to object-or ented programmers, who th nk of parent–ch d re at onsh ps n s mp er terms; each parent has as a ch d co ect on property and each ch d has a parent property Furthermore, the DataSet parad gm does not a ow for nher tance, wh ch s a so extreme y mportant to object-or ented deve opers Spec a hand ng s a so requ red for nu va ues n a DataSet Notw thstand ng these concerns, we don’t genera y advocate d sm ss ng any techno ogy out of hand Every app cat on s d fferent, and you are do ng yourse f a d sserv ce f you don’t exam ne the facets of a ava ab e cho ces on a case-by-case bas s L ke anyth ng e se, ­DataSets can be used or they can be abused, and t’s true that they do present m tat ons f you try to ut ze them as bus ness objects But f what you need s a gener c n-memory ­database mode , then that’s what a DataSet s, and that’s what a DataSet g ves you A though the ­aforement oned concerns are a va d, the fact rema ns that DataSets are very powerfu and can serve ­extreme y we as data transfer objects Furthermore, they have the un que ab ty to dynam ca y adapt the r shape accord ng to the schema of whatever data s streamed nto them, wh ch s ­someth ng that none of the newer APIs can do So to re terate, DataSets are not obso ete The DataSet s a cornerstone of the NET Framework, and t cont nues to be supported by the newest NET Framework 4 5 n the W ndows 8 W n32 ­compat b ty runt me Even when LINQ to SQL and Ent ty Framework were first re eased, M ­ crosoft was enhanc ng the DataSet For examp e, the TableAdapterManager was added n NET 3 5 to great y s mp fy h erarch ca updates, as you’ see n an examp e short y The fact s that DataSets do st have the r p ace, and you w cont nue to encounter them for a ong t me to come as you ma nta n ex st ng app cat ons Hav ng sa d that, we must a so stress that M crosoft has c ear y pos t oned ADO NET Ent ty Framework as the preferred data access techno ogy today and n the future, ec ps ng both DataSets and LINQ to SQL Furthermore, ne ther S ver ght nor the NET Framework 4 5 n the W ndows 8 nat ve W nRT runt me supports the DataSet

A DataSet conta ns one or more DataTable objects, each of wh ch n turn conta ns rows and ­co umns The DataSet has a of the bas c character st cs of a re at ona database, such as pr mary keys, defau t va ues, re at onsh ps and constra nts, but has no awareness of any part cu ar u ­ nder y ng data store In fact, t’s poss b e to use a DataSet w th mu t p e data stores, or even none at a (that s, an ent re y n-memory scenar o) It s the job of a DataAdapter to shutt e data back and forth b ­ etween 456  Part III  Applied SQL

a DataSet and a spec fic back-end data store The DataAdapter—not the DataSet—works as an ­abstract on over the raw ADO NET command objects that are used to get data nto and out of the DataSet Many ADO NET data prov ders are ava ab e today that offer a DataAdapter to br dge the D ­ ataSet w th d fferent p atforms W th the SQL Server prov der (Sq C ent), the DataSet s pa red w th the ­SqlDataAdapter F gure 10-5 dep cts these objects and shows the nteract on between the ­database, DataAdapter, and DataSet

Note  This chapter covers just the SQL Server provider, though we often refer to the SqlDataAdapter object using the more generic term DataAdapter. Similarly, we refer to data readers as SqlDataReader and DataReader interchangeably.

DataSet Table Column Column Column Column

DataAdapter Table Column Column Column

SelectCommand InsertCommand UpdateCommand DeleteCommand

Column Column

Database Figure 10-5 The DataSet and DataAdapter work together to retr eve and update data n SQL Server.

The SqlDataAdapter fac tates fi ng and updat ng (v a ts apt y named Fill and Update methods) data n the DataSet It ho ds four d st nct SqlCommand objects, exposed by the SelectCommand property (used when you ca Fill), and the InsertCommand, UpdateCommand, and DeleteCommand propert es (used when you ca Update) It s a common m sconcept on that DataSets are s ower than DataReaders, when n fact DataReaders are used nterna y when you ca Fill on a DataAdapter What t rea y comes down to s whether you are cach ng the data versus process ng t as you stream t Beh nd the scenes, the Fill method works just ke what you’d code by hand to popu ate a co ect on of objects n memory us ng the DataReader techn ques we just covered Spec fica y, ca ng Fill nvokes ExecuteReader on the SqlCommand object referenced by the SelectCommand of the DataAdapter, and thus obta ns a SqlDataReader The adapter s smart enough to open the SqlConnection (referenced by the Connection property of the SqlCommand object) first, f t sn’t a ready opened It then ca s Read to terate the resu ts one row at a t me For each row, t

Chapter 10  The M crosoft Data Access Juggernaut   457

nstant ates and popu ates a DataRow object, and adds the DataRow to a DataTable n the DataSet If mu t p e resu t sets are returned by the server, then mu t p e DataTables are automat ca y nstant ated to accommodate them After the ast row of data n the ast resu t set s processed, t c oses the reader when done F na y, f the connect on wasn’t opened before the Fill, the connect on s c osed as we At th s stage, the connect on w th the server has been severed, and the DataAdapter can be d scarded f des red Ca ng Update on the DataAdapter prov des a s m ar abstract on to push changes made n the DataSet ( nserts, updates, and de etes) back to the database Spec fica y, ADO NET exam nes the ­RowState property of each row n a DataTable to determ ne whether ExecuteNonQuery shou d be ca ed, and on wh ch SqlCommand object (InsertCommand, UpdateCommand, or D ­ eleteCommand) t shou d be ca ed Each DataRow ho ds current and or g na va ues, mak ng t easy to map both ­vers ons of any co umn to the three update commands Because Update works on y on one tab e at a t me, t can be tr cky to hand e h erarch ca updates when the database s ­enforc ng ­referent a ­ntegr ty constra nts between tab es New parent rows need to be added before the r ch d rows get added, whereas de et ons need to be coded n the reverse order Fortunate y, the ­TableAdapterManager was ntroduced n NET 3 5 to do a that work for you As you’ see n an ­upcom ng examp e, the TableAdapterManager can push a the nsert ons, mod ficat ons, and ­de et ons across an ent re ­DataSet back to SQL Server n just one ne of code The DataSet s a very powerfu data conta ner that supports both b nary and XML ser a zat on across t ers and can track ts own changes It s an eas y b ndab e, h erarch ca , d sconnected cache that features fi ter ng, sort ng, computed express ons, and even prov des transact ona capab t es (such as beg n/end ed t, and accept/reject changes) And LINQ to DataSet (wh ch we cover short y) prov des powerfu extens ons that make t even eas er to query a popu ated DataSet

Generic Versus Strongly Typed DataSets The term generic can be somewhat amb guous when used w th DataSets In one sense, and as a ready exp a ned, DataSets are a ways gener c w th respect to the r back-end data store But more ­common y, gener c DataSets refer to DataSets that have no schema defin t on, as opposed to strongly typed DataSets that do Gener c DataSets are usefu n (the very sma m nor ty of) cases when you are not aware of the schema of the data that you’re work ng w th at des gn t me In those ­s tuat ons, t’s ­actua y qu te powerfu and conven ent to have a DataSet object that starts out schema- ess (no ­defined tab es or co umns) and just dynam ca y adapts to whatever data you stream nto t (even ­Ent ty Framework can’t do that!) When you ca Fill on the SqlDataAdapter, one DataTable gets ­created for each resu t set returned by the SelectCommand and one DataColumn w th n each ­DataTable gets created for each co umn of each resu t set (the co umn data types get nferred from the resu t set) Then the rows get popu ated w th the actua data However, n the vast major ty of cases, you are aware of the schema of the data that you’re ­ ork ng w th at des gn t me, and so you want to work w th strong y typed DataSets (a so ca ed typed w DataSets) You’ start now w th examp es us ng a gener c DataSet, and then move on to create an XML Schema Defin t on (XSD) for examp es us ng a typed DataSet As you progress, you w see many benefits of work ng w th typed DataSets 458  Part III  Applied SQL

Filling and Updating a Generic DataSet Let’s get busy w th DataSets Launch V sua Stud o, create a new V sua C# W ndows Forms ­app cat on, and name t DemoDataSets Drag a button contro from the Too box and drop t onto the form Form1 created automat ca y by V sua Stud o In the Propert es w ndow, name the button btnGenericDataSet, and set the Text property to Generic DataSet Then doub e-c ck the button V sua Stud o w automat ca y create a c ck event hand er for the button, and then open the code ed tor so that you can mp ement the hand er Add the code as shown n L st ng 10-10 Listing 10-10  F

ng and updat ng w th a gener c DataSet.

private void btnGenericDataSet_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string SelectTSql = "SELECT * FROM Customer"; const string UpdateTSql = @" UPDATE Customer SET FirstName = @FirstName, LastName = @LastName, UpdatedAt = SYSDATETIME() WHERE CustomerId = @CustomerId"; var ds = new DataSet(); using (var cn = new SqlConnection()) { cn.ConnectionString = ConnStr; using (var cm = new SqlCommand()) { cm.Connection = cn; cm.CommandText = SelectTSql; using (var da = new SqlDataAdapter()) { da.SelectCommand = cm; da.Fill(ds); } } } // The self-tracking DataSet can now be serialized across the tiers var customerTable = ds.Tables[0]; if (customerTable.Rows.Count == 0) { MessageBox.Show("There are no customers"); return; } var firstCustomerRow = customerTable.Rows[0];



Chapter 10  The M crosoft Data Access Juggernaut   459

// Type casting required var customerId = (long)firstCustomerRow["CustomerId"]; var updatedAt = (DateTime)firstCustomerRow["UpdatedAt"]; MessageBox.Show(string.Format("Customer {0} was updated at {1}", customerId, updatedAt)); firstCustomerRow["FirstName"] = "Brian"; firstCustomerRow["LastName"] = "Perry"; using (var cn = new SqlConnection()) { cn.ConnectionString = ConnStr; using (var cm = new SqlCommand()) { cm.Connection = cn; cm.CommandText = UpdateTSql; cm.Parameters.Add("@FirstName", SqlDbType.VarChar, 50, "FirstName"); cm.Parameters.Add("@LastName", SqlDbType.VarChar, 50, "LastName"); cm.Parameters.Add("@CustomerId", SqlDbType.BigInt, -1, "CustomerId"); using (var da = new SqlDataAdapter()) { da.UpdateCommand = cm; da.Update(customerTable); } } } }

As w th the prev ous project, you must mport the namespace so that the comp er can recogn ze c asses n the System.Data.SqlClient namespace So a so add the fo ow ng using d rect ve at the very top of the source code using System.Data.SqlClient;

L st ng 10-10 demonstrates the fundamenta pr nc p es of data access us ng DataSets and ­ ataAdapters W th gener c DataSets, there are no XSDs, des gners, or other v sua too s nvo ved; t’s D a pure cod ng exper ence n ADO NET The code beg ns w th a hard-coded connect on str ng constant, as before Th s s the ast t me that you w fo ow th s ant -pract ce; the rema n ng examp es n the chapter store the connect on str ng n a configurat on fi e so that t can be a tered w thout recomp ng the app cat on Two str ng ­constants, SelectTSql and UpdateTSql, are then defined as d rect T-SQL statements aga nst the Customer tab e that, respect ve y, retr eve a the customers from the database and update a s ng e customer’s ­FirstName and LastName fie ds For now, you aren’t support ng de et ons and nsert ons, nor are you dea ng w th any other Customer co umns or the re ated OrderHeader tab e A new gener c DataSet s then nstant ated and ass gned to the var ab e ds The code s now go ng to fi ds w th customers, change one of the customer’s nformat on, and then save those changes back to the database Let’s start w th the fi operat on 460  Part III  Applied SQL

The SqlConnection and SqlCommand are nstant ated w th using b ocks, just as we’ve demonstrated n a the prev ous examp es Not ce, however, that the code does not actua y open the SqlConnection Th s s gna s the fi operat on to automat ca y open the connect on before t oads the resu t set and c oses t afterwards (converse y, f you d d open the connect on, the fi operat on wou d not try to open t aga n, nor wou d t c ose the connect on afterwards) Now the SqlDataAdapter object s created, and ts ­SelectCommand property s po nted to the SqlCommand object configured to SELECT the ent re customer tab e A the p eces are now n p ace to ca the Fill method When you ca the Fill method on the SqlDataAdapter, the SqlCommand object of the adapter’s S­ electCommand property s executed to fi the DataSet As exp a ned, th s happens by ca ng ­E xecuteReader and terat ng a SqlDataReader beh nd the scenes to popu ate a DataTable ns de the DataSet w th DataRows Because th s s pure y a retr eva operat on, the SqlDataAdapter doesn’t care that you haven’t set the other three SqlCommand propert es After the Fill, the DataSet (wh ch had been devo d of both schema and data) now has a s ng e popu ated DataTable n t w th a schema der ved from the resu ts Spec fica y, the Columns co ect on of the DataTable s popu ated w th ­DataColumn objects whose data types are nferred by the resu t set returned from the server At th s t me, every DataRow n the DataTable has a RowState property whose va ue s set to Unchanged The server connect on s severed at th s po nt, and the data access objects are d sposed On y the popu ated DataSet rema ns The code now makes some changes n the DataSet and updates those changes w th another DataAdapter A though everyth ng s coded n a s ng e method, th s ­demonstrates that a DataSet ex sts ent re y ndependent y of the DataAdapter, and the DataAdapter’s under y ng connect ons and commands used to shutt e data nto and out of the DataSet Between the t me that one SqlDataAdapter s used to ca Fill and the t me that another SqlDataAdapter s used to ca Update, the DataSet cou d potent a y be ser a zed across mu t p e t ers up to the ­presentat on ayer, bound to contro s n the user nterface, and then transported back down to the data access ayer to be updated

Note  We stress this point because it highlights a key difference between DataSets and the much newer and more advanced Entity Framework that is coming up later in the chapter. The independence of the DataSet as a self-tracking container makes it extremely easy to develop for n-tier scenarios. With EF, conversely, stateful entity objects rely on the ObjectContext for change tracking, posing much greater challenges in n-tier architectures. We will elaborate more on this point when discussing EF. Next the code gets a hand e onto the DataTable ho d ng the resu ts, wh ch s the first (and ­current y on y) member n the DataSet’s Tables co ect on If there are no customers t s mp y d sp ays a ­message and ex ts the method Otherw se, t gets a hand e onto the DataRow ho d ng the first customer, wh ch s the first member n the DataTable’s Rows co ect on It then uses str ng ndexes nto the DataRow to extract the CustomerId and UpdatedAt va ues of the first customer and d sp ays them n a MessageBox After the MessageBox, the customer’s name s changed to Br an Perry The code that d rect y changes the va ue of the FirstName and LastName co umns automat ca y a ters the RowState

Chapter 10  The M crosoft Data Access Juggernaut   461

­ roperty of the customer DataRow from Unchanged to Modified Th s s exact y the way the ­DataSet p wou d be affected f a user typed the changes nto a gr d on a form that was bound to the ­DataTable Furthermore, the prev ous first and ast name va ues just overwr tten are st ava ab e, so you can a ways access or g na va ues n a mod fied DataSet (across a t ers) for any purpose that your ­app cat on requ res Now the changes are ready to be saved In a rea n-t er app cat on, you can mag ne th s wou d be the po nt where the user c cks Save and the DataSet gets ser a zed back down across the t ers to the data access ayer so that t can be updated w th another SqlDataAdapter—th s one configured for ca ng Update rather than Fill Once aga n, the code nstant ates SqlConnection and SqlCommand objects And aga n, by not open ng the connect on, the adapter nterna y opens and c oses the connect on on ts own The SqlCommand w be used to update every row n the DataTable whose RowState property s set to Modified, and the Parameters co ect on of the SqlCommand object s popu ated us ng an over oad of the Add method that maps parameter names n the T-SQL statement to correspond ng co umn names n the DataTable F na y, the SqlDataAdapter object s created, ts UpdateCommand property s po nted to the SqlCommand object, and ts Update method s nvoked As exp a ned, th s causes ExecuteNonQuery to be nvoked on each DataRow n the DataTable passed to the Update method that has a RowState of Modified Start a SQL Profi er trace and g ve the code a run Watch the trace as you s ng e-step over the Fill and Update methods to see the act on as t happens beh nd the scenes (The Update method w update a rows w th a RowState of Modified, but because your code on y mod fies one customer row, you w on y see a one UPDATE n the trace ) Th s code works because t knows that the on y RowState va ue that the adapter’s Update method s go ng to encounter (other than Unchanged) w be Modified If a RowState of Added or Deleted wou d be encountered, th s code w fa , because the Update method wou d ook respect ve y to the InsertCommand and DeleteCommand propert es for the appropr ate SqlCommand to execute, and those propert es haven’t been set To support nsert ons and de et ons, you’d need to add the correct parameter zed T-SQL statements to the code, as demonsrated for updates And to support the OrderHeader tab e as we wou d requ re even more code to dea w th another tab e’s Fill and Update operat ons, as we as the extra og c to proper y hand e the order of de et ons and nsert ons across the h erarch ca compos t on of parent and ch d re at onsh ps Th s qu ck y adds up to a ot of code that you need to wr te and ma nta n for typ ca app cat ons that work w th many re ated tab es Strong y typed DataSets (a ong w th the TableAdapterManager) come to the rescue here In the next sect on you w earn how to wr te ess code and get more done us ng typed DataSets nstead of gener c DataSets

Building Strongly Typed DataSets A typed DataSet (short for strong y typed DataSet) offers numerous benefits over the gener c ­ ataSet When creat ng your app cat on, you use a graph ca des gner n V sua Stud o to v sua y D bu d the schema of your typed DataSet You ay out the DataSet by dragg ng and dropp ng tab es and re at onsh ps from the Server Exp orer, and then further custom ze ts propert es (occas ona y 462  Part III  Applied SQL

w th the ass stance of a w zard) When you perform these act ons, you are actua y ed t ng an XML Schema Defin t on (XSD) fi e n the background that descr bes a the tab es, co umns, data types, and re at onsh ps you’ve a d out for the DataSet V sua Stud o prov des a custom code-generat on too that then reads the XSD fi e and produces NET source code from t The process s tr ggered when any change s made to the XSD fi e, wh ch s any t me you make a change w th the graph ca des gner The generated code defines a c ass that nher ts from ­DataSet—so that a typed DataSet s everyth ng that a gener c DataSet s—but t a so prov des strong y typed members for a of the e ements (tab es, co umns, re at onsh ps) defined by the XSD In add t on to strong y typed DataTable schemas, NET 2 0 enhanced the comb ned v sua -des gner/ code-generat on exper ence by ntroduc ng TableAdapters These are s mp y strong y typed ­wrappers around DataAdapters that are pre-configured (as you’ see n the next examp e) w th the four ­SqlCommand objects—comp ete w th parameters, connect on str ng, and just about any other propert es you can th nk of sett ng The resu t s a drast c reduct on n the amount of code you need to wr te, because the des gner-generated code s now do ng the work to hook up a four command objects and the r ­parameters to an adapter, for each tab e Now that we’ve exp a ned some of the benefits, et’s tap nto the power of typed DataSets In S­ o ut on Exp orer, r ght-c ck the project and choose Add New Item In the Add New Item d a og, scro down to find and c ck on DataSet Name the new DataSet SampleDS.xsd and c ck Add V sua Stud o creates the fi e and opens t n the typed DataSet des gner In the center of the empty ­des gn surface, c ck the Server Exp orer nk The Server Exp orer w he p you get started qu ck y w th ­bu d ng your typed DataSet, but you’ first need to g ve t a connect on to the database In the Server Exp orer, r ght-c ck Data Connect ons and choose Add Connect on Se ect M crosoft SQL Server as the data source and c ck Cont nue In the Add Connect on d a og, type localhost for the Server name, se ect SampleDB from the database drop-down st, and c ck OK (you must choose d fferent sett ngs f you have created the database on another server or nstance) V sua Stud o ­creates the connect on beneath the Data Connections node n the Server Exp orer Expand the new connect on node n Server Exp orer, and then expand the Tables node beneath that, as shown n F gure 10-6

Figure 10-6  V ew ng data connect ons n V sua Stud o s Server Exp orer.



Chapter 10  The M crosoft Data Access Juggernaut   463

C ck the first tab e (Customer), ho d down the Sh ft key, and then c ck on the ast tab e (­ OrderHeader) to se ect a four tab es Then drag the four tab es from the Server Exp orer and drop them onto the DataSet des gn surface After rearrang ng the tab es to get a better v ew of how they re ate, your DataSet shou d ook s m ar to F gure 10-7

Figure 10-7  The typed DataSet des gn surface n V sua Stud o.

Severa very mportant th ngs have happened automat ca y at th s po nt F rst, not ce that an ­ pp cat on configurat on fi e (named app.config) has been added to the project Open ­app.config and a you’ see a e ement w th an e ement nested ns de of t The ­e ement defines a name for the connect on, and a so conta ns the comp ete connect on str ng to the database If and when you need to change the connect on str ng, you’ be ab e to ed t the ­configurat on fi e and won’t need to recomp e the app cat on as you wou d w th a hard-coded ­connect on str ng You can a so dep oy the app cat on to d fferent env ronments (deve opment, ­test ng, product on), and ed t the configurat on fi e accord ng y to adjust the connect on str ng appropr ate y for those env ronments Now c ose app.config and return to the typed DataSet des gner Next, you can see that the des gner has created tab es, co umns, and re at onsh ps n the DataSet that essent a y m m c the database structure The Server Exp orer does a very good job at d scover ng and rep cat ng schema from the database to the DataSet des gner, but t’s not perfect For examp e, t won’t p ck up on defau t va ues In the Customer tab e, c ck the Balance co umn and not ce n the Propert es w ndow that t does not have a defau t va ue, even though a defau t va ue of 0 s defined for Balance n the database You can use the Propert es w ndow to set the defau t va ue and make other tweaks as necessary after dragg ng and dropp ng tab es from Server Exp orer onto the DataSet

464  Part III  Applied SQL

F na y, the des gner d sp ays TableAdapters attached at the bottom of each tab e Be ng that there s one TableAdapter for fi ng and updat ng each DataTable, th s s a reasonab y conven ent d sp ay However, t can a so be qu te m s ead ng Remember that TableAdapters are rea y just ­DataAdapters, and that DataTables conta ned ns de of DataSets ex st ent re y ndependent y of DataAdapters So wh e you use the des gner to eas y configure each tab e together w th ts adapters, the code ­generator actua y produces d st nct DataSet and TableAdapter c asses An nstance of a typed DataSet nc udes on y typed DataTables; t does not nc ude the correspond ng TableAdapters Th s makes sense, because when the DataSet s ser a zed up to the c ent across the network, the adapter nformat on s mean ng ess TableAdapters are nstant ated ndependent y of DataSets on y on the ­app cat on server connected to the database ns de the firewa , n order to fi and update them just as you’ve been do ng a a ong By defau t, the des gner has generated a TableAdapter for each tab e, w th each one a ready configured w th the four command objects needed to fi and update the tab e It a so generated ­appropr ate d rect T-SQL statements for each command n each tab e, and configured the r parameter co ect ons Compared w th the amount of code n the prev ous gener c DataSet examp e, t s obv ous that typed TableAdapters offer a huge sav ngs n t me and effort

Mapping Stored Procedures to the Typed DataSet Now you w use the CRUD stored procedures defined for the Customer tab e nstead of the d rect SQL generated by the des gner for each of the four command objects n ts TableAdapter C ck the ­CustomerTableAdapter head ng n the des gner The Propert es w ndow shows the four command ­propert es SelectCommand, InsertCommand, UpdateCommand, and DeleteCommand Exam ne each of them and you w see engthy d rect T-SQL statements ass gned to the CommandText property and correspond ng parameters popu ated n the Parameters co ect on property You’ use the Tab eAdapter Configurat on W zard to change th s so that the CRUD stored procedures are used nstead R ght-c ck the CustomerTableAdapter head ng and choose Configure It may seem odd, but th s drops you off n the middle of the w zard, so c ck the Prev ous button to go back to the Choose a Command Type page The Use SQL Statements rad o button s se ected by defau t, wh ch means that the Customer tab e shou d use the d rect SQL statements generated automat ca y when you dragged and dropped from Server Exp orer The second rad o button w actua y create new stored ­procedures, wh ch wou d conta n the same automat ca y generated SQL statements Se ect the th rd rad o button to te the w zard that you want to use ex st ng stored procedures, as shown n F gure 10-8 C ck Next For the Se ect, Insert, Update, and De ete drop-down sts, choose SelectCustomers, InsertCustomer, UpdateCustomer, and DeleteCustomer, respect ve y When you map the Se ect, the w zard d scovers the co umns returned by the stored procedure and d sp ays them n the pane on the r ght, as shown n F gure 10-9 (the capt on “Set Se ect procedure parameters” above the co umn st s actua y ncorrect; t shou d rea y state “Se ect procedure co umns”) When you map the Insert, ­Update, and De ete, the w zard d scovers the parameters requ red by the stored procedure and ­d sp ays them n the r ght-hand pane



Chapter 10  The M crosoft Data Access Juggernaut   465

Figure 10-8  Te ng the w zard to use ex st ng stored procedures for a tab e adapter.

Figure 10-9  Mapp ng stored procedures to a tab e adapter.

Now c ck F n sh Rev s t the Propert es w ndow for the CustomerTableAdapter and you’ see someth ng very d fferent than before Rather than a engthy T-SQL statement, the four CommandText propert es mere y conta n the stored procedure name, as shown n F gure 10-10

466  Part III  Applied SQL

Figure 10-10  Tab e adapter propert es show ng mapped stored procedures.

Because stored procedures parameters are cons stent y named w th the tab e co umns, the w zard correct y mapped a the parameters to co umns It even guessed to map the parameter @OriginalUpdatedAt to the UpdatedAt co umn for the UpdateCustomer stored procedure But you st need to tweak that mapp ng In the Propert es w ndow, expand the UpdateCommand property and c ck the e pses next to Parameters to open the Parameters Co ect on Ed tor Then c ck the @OriginalUpdatedAt parameter The des gner correct y set the SourceColumn to UpdatedAt co umn even though the parameter name s prefixed w th the word Original, but the SourceVersion s st set to Current Th s w work fine as ong as the c ent never changes the UpdatedAt va ue after they retr eve t, but you can never trust what the ­c ent m ght do So nstead, you want to spec fy the or g na UpdatedAt co umn that was retr eved from the database and not worry about the c ent poss b y overwr t ng t (acc denta y or de berate y) Because each DataRow reta ns every co umn’s or g na va ue, t s easy for you to map the or g na va ue of any co umn to any stored procedure parameter that needs t (Tangent a y, t s a so very conven ent that the va dat ons ru e and ca cu at ons n your bus ness og c can access or g na va ues n a DataSet ) Change the ­SourceVersion for @OriginalUpdatedAt from Current to Original, as shown n F gure 10-11, and c ck OK Then save and bu d the project to make sure that t comp es successfu y A though the who e po nt of the des gner s to sh e d you from the deta s of the XSD and the typed DataSet c ass that gets generated from t, fam ar z ng yourse f w th both of these fi es s very en ghten ng, and w go a ong way n he p ng you understand the techno ogy So before wr t ng the code for th s fina DataSet examp e, go have a qu ck ook at the fi es beh nd the des gner



Chapter 10  The M crosoft Data Access Juggernaut   467

Figure 10-11  Mapp ng a co umn s or g na va ue to a stored procedure parameter.

R ght-c ck SampleDS.xsd n the So ut on Exp orer and choose Open W th Se ect XML (Text) Ed tor and c ck OK to v ew the XSD fi e as XML Th s is the XML that you are compos ng as you bu d your typed DataSet n the des gner w th Server Exp orer drag-and-drop, the Tab e Adapter Configurat on W zard, and the Propert es w ndow Scro through the XSD, and you w find an XML representat on of everyth ng you added to the DataSet us ng the des gner Next, expand the SampleDS.xsd fi e n the So ut on Exp orer to revea the SampleDS.Designer.cs fi e nested beneath t Doub e-c ck t to open the C# code, wh ch s generated automat ca y whenever the XSD s updated In th s code, you w see the SampleDS c ass that nher ts from DataSet, as we as tab e c asses that nher t from DataTable w th fu y descr bed co umns Scro a b t more and you w find the generated TableAdapter code, wh ch uses code s m ar to what you wrote by hand ear er to set up the four command objects and the r parameters w th an adapter for each tab e W th a of th s funct ona ty now baked nto your typed DataSet, you’re go ng get a ot more done w th a ot ess code, compared to work ng w th gener c DataSets Drag another button ­contro from the Too box and drop t onto the form In the Propert es w ndow, name the button ­btnTypedDataSet, and set the Text property to Strongly Typed DataSet Then doub e-c ck the button and add the code n L st ng 10-11 Listing 10-11  F

ng and updat ng w th a strong y typed DataSet.

private void btnTypedDataSet_Click(object sender, EventArgs e) { var ds = new SampleDS(); using (var da = new SampleDSTableAdapters.CustomerTableAdapter()) { da.Fill(ds.Customer); }

468  Part III  Applied SQL

using (var da = new SampleDSTableAdapters.OrderHeaderTableAdapter()) { da.Fill(ds.OrderHeader); } // The self-tracking DataSet can now be serialized across the tiers if (ds.Customer.Count == 0) { MessageBox.Show("There are no customers"); return; } var firstCustomerRow = ds.Customer[0]; // No type casting required var customerId = firstCustomerRow.CustomerId; var updatedAt = firstCustomerRow.UpdatedAt; var orderRows = firstCustomerRow.GetOrderHeaderRows(); MessageBox.Show( string.Format("Customer {0} was updated at {1} and has {2} order(s)", customerId, updatedAt, orderRows.Length)); // Change the customer's name firstCustomerRow.FirstName = "David"; firstCustomerRow.LastName = "Jones"; // Add a customer order ds.OrderHeader.AddOrderHeaderRow(firstCustomerRow, "Regular Mail", "Open", string.Empty, DateTime.Now, DateTime.Now); using (var tam = new SampleDSTableAdapters.TableAdapterManager()) { tam.CustomerTableAdapter = new SampleDSTableAdapters.CustomerTableAdapter(); tam.OrderHeaderTableAdapter = new SampleDSTableAdapters.OrderHeaderTableAdapter(); tam.UpdateAll(ds); } }

F rst, a new nstance of SampleDS (your typed DataSet) s created Un ke a new y ­nstant ated ­gener c DataSet, th s DataSet s fu y oaded w th schema nformat on, and has DataTable, ­DataColumn, and DataRelation objects for everyth ng defined n the XSD a ready created ns de of t the moment t s nstant ated The on y th ng m ss ng are the rows of data themse ves Next, a the customers and orders are retr eved from the database Each tab e s fi ed us ng the TableAdapter you configured n the des gner For the CustomerTableAdapter, you know that means

Chapter 10  The M crosoft Data Access Juggernaut   469

the SelectCustomers stored procedure w be ca ed For the OrderHeadersTableAdapter, the defau t des gner-generated T-SQL SELECT statement w be ca ed When you run the app cat on, s ng e step over both Fill method ca s as you exam ne the runn ng trace n SQL Profi er to ver fy that these are ndeed the statements be ng sent by ADO NET to SQL Server

Note  Realistically, your fill operations will be parameterized to filter just a subset of the data from the database. In the designer, you can create multiple data retrieval queries in addition to the one associated with the SelectCommand property. These queries can be tied to parameterized stored procedures, and the designer will generate FillByXxx methods (for example FillByNameSearch, FillByState, FillByCategory, and so on) with .NET parameters that match the T-SQL parameters of the stored procedure. This way, through its ­associated TableAdapter, each DataTable can be filled using any number of queries with varying ­parameters for controlling the subsets of data that will get retrieved from the database. Once aga n, the server connect on s now severed, the data adapters are d sposed, and on y the popu ated DataSet rema ns In an n-t er scenar o, t wou d be ser a zed across mu t p e t ers up to the presentat on ayer at th s po nt Now you start to enjoy the benefits of strong typ ng You don’t need to refer to the customer DataTable by ndex, as n ds.Tables[0] or ds.Tables[“Customer”], because the typed DataSet has a ­property ca ed Customer (that the comp er won’t et you m sspe ), and t has a Count property that te s you how many customer rows were retr eved If there are no customers, the code ­d sp ays a ­message and ex ts the method Otherw se, t gets a hand e onto the DataRow ho d ng the first ­customer, wh ch s the first member n the typed DataRow co ect on exposed by the Customer property It then extracts the CustomerId and UpdatedAt va ues of the first customer as n the ast examp e On y th s t me, t uses genu ne long and DateTime propert es exposed by the strong y typed DataRow You don’t need to cast from a generic object type or use str ng ndexes—both of wh ch are undes rab e, nconven ent, and error-prone pract ces The Inte sense feature n V sua Stud o makes t very easy to find the r ght property, and f you st manage to m sspe a property name, your project won’t bu d Because of the re at onsh p defined n the typed DataSet between Customer and OrderHeader, the typed customer DataRow s a so equ pped w th a GetOrderHeaderRows method Th s method returns an array of DataRow objects for the orders be ong ng to th s customer (wh ch s a subset of the comp ete st of orders be ong ng to a customers that you fi ed from the database) You ca ­GetOrderHeaderRows and then show a MessageBox that d sp ays the customer’s ID, updated date, and order count After the MessageBox, the customer’s name s changed to Dav d Jones Aga n, strong y typed propert es for FirstName and LastName are vast y super or to the gener c approach n the prev ous examp e, where you were constant y cast ng back and forth between object and spec fic data types You won’t be ab e to ass gn ncorrect data types or m sspe co umn names w th typed DataSets ­Mak ng these (extreme y common) m stakes w th gener c DataSets do not get caught by the ­comp er, and t w be the poor unsuspect ng user that d scovers the prob em w th an ug y error 470  Part III  Applied SQL

message d sp ayed by an unhand ed except on You shou d be gett ng the dea that our very strong ­recommendat on s to a ways use typed DataSets over gener c DataSets, except for cases where you need to d scover schema dynam ca y at runt me Gener c DataSets are the perfect too for the job n such scenar os (they’re a so great for “throw-away” apps, ke qu ck demos or proof-of-concepts) In add t on to chang ng the customer name, the code adds a new customer order Aga n, due to the re at onsh p between Customer and OrderHeader, the OrderHeader tab e property of the typed DataSet s equ pped w th an AddOrderHeaderRow method that was created automat ca y by the typed DataSet code generator Th s method accepts appropr ate y typed parameters for each co umn of a new order The first parameter dent fies the new OrderHeader’s parent Customer ­Interest ng y, th s passed as the parent Customer row, rather than the CustomerId fore gn long va ue The ­rema n ng parameters supp y va ues for the new order’s ShipVia, OrderStatus, Notes, CreatedAt, and UpdatedAt co umns The typed DataRow requ res va ues for CreatedAt and UpdatedAt, wh ch are supp ed as ­DateTime.Now (based on the app cat on’s c ock) If there were CRUD stored procedures for ­OrderHeader as there are for Customer, the nsert and update stored procedures wou d be us ng SYSDATETIME (based on SQL Server’s c ock) for these co umns, gnor ng and then rep ac ng the va ues set n the DataRow Now mag ne that the user c cks Save and the DataSet gets ser a zed back down across the t ers to the data access ayer so that ts changes can be pushed back to the database Th s t me, the ­TableAdapterManager s used, and th s hand es a the concerns of h erarch ca updates when dea ng w th compos t ona object graphs of re ated ent t es If you have referent a ntegr ty constra nts on the ­re at onsh ps n the database tab es (and we strong y recommend that you do) that aren’t enab ed for cascad ng de etes (and we strong y recommend that they aren’t), a two-phase approach needs to be taken when pers st ng nsert ons and de et ons from the DataSet back to the database In part cu ar, new parent rows need to be INSERTed before new ch d rows are, whereas removed ch d rows need to be DELETEed before removed parent rows Pr or to NET 3 5, you needed to code th s og c yourse f us ng ­nd v dua DataAdapters (or TableAdapters) But the TableAdapterManager removes a that pa n As the code demonstrates, you s mp y create an nstance of a new TableAdapterManager and prov de t w th a new TableAdapter nstance for each tab e nvo ved n the update W th a s ng e ca to the UpdateAll method, the TableAdapterManager updates both tab es ns de a transact on, automat ca y revers ng the order of nsert ons and de et ons between parent and ch d tab es as necessary Run the code and watch the SQL Profi er trace as t executes You’ve now earned the most mportant techn ques for us ng convent ona ADO NET w th SQL Server Desp te how o d convent ona ADO NET s compared w th LINQ to SQL or Ent ty Framework, these newer techno og es (wh ch are very d fferent than DataSets) are based on many of the same pr nc p es that have proven so successfu w th typed DataSets—name y the use of a des gner to author an XML document represent ng your data mode , comb ned w th a code generator to produce typed ent t es for your app cat ons to work w th programmat ca y So the firm grasp you now have on these concepts w make earn ng the newer techno og es much eas er You’ start by enter ng the wor d of LINQ



Chapter 10  The M crosoft Data Access Juggernaut   471

Language-Integrated Query (LINQ) The express veness and flex b ty of anguage- ntegrated query (LINQ), first ntroduced w th NET 3 5, brought rad ca change to the way deve opers program n C# and V sua Bas c (VB) NET That’s a bo d statement, but don’t m s nterpret t to mean that LINQ s a ways the answer There are many ­mp ementat ons of LINQ, and we w soon be exam n ng many factors that you need to cons der ­before determ n ng wh ch ones are the r ght (or wrong) cho ces for your app cat on But a LINQ mp ementat ons u t mate y rest on top of LINQ to Objects, and LINQ to Objects s always the r ght cho ce when work ng w th objects Th s means that—at the very east—you shou d be us ng LINQ ­nstead of wr t ng oops wherever poss b e to query sts, co ect ons, and arrays n your NET ­app cat on You w enjoy produc ng t ghter, more e egant code that expresses what you want to do, not how to do t

Note  LINQ is a huge topic, and this is a book on SQL Server, not LINQ. We d ­ emonstrate only a fraction of the real power of LINQ in our examples, and focus instead on how ­different LINQ implementations can be made to work with data in SQL Server. For a ­detailed and concise treatment of LINQ, we recommend the LINQ Pocket Reference (O’Reilly) by Joseph Albahari and Ben Albahari. In T-SQL, you work w th set-based operat ons It wou d be absurd to mp ement a query n T-SQL by cod ng a cursor that scans a tab e, oops through t one row at a t me, and exam nes each row for some cr ter a That’s what SELECT/WHERE s for! Yet before LINQ, that’s exact y what deve opers had been do ng w th object arrays and co ect ons How many t mes have you coded a oop that scans some enumerab e sequence of objects? You cou d be fi ter ng for a subset, ca cu at ng an aggregat on, or perhaps you just need to determ ne f a part cu ar tem s present n a co ect on LINQ was des gned to e m nate a such oop ng constructs from your code You m ght st code a foreach oop to process the resu ts of a LINQ query, but the LINQ query tse f s coded as a s ng e statement rather than a procedura oop, just ke ­set-based operat ons n the database wor d So a though you shou d a ways use LINQ to Objects to query n-memory object sequences, other LINQ mp ementat ons need to be eva uated nd v dua y When mak ng your determ nat on, the first th ng to cons der s whether you are query ng d rect y over memory-res dent data structures or externa data stores Bes des “ord nary” objects quer ed w th LINQ to Objects, memory-res dent data ­structures nc ude cached XML documents (the XDocument object, wh ch can be quer ed w th LINQ to XML) or—more re evant to th s chapter—popu ated DataSets (quer ed w th LINQ to DataSet) To work w th n-memory data structures such as these, aga n, you’ a ways want to use a spec a zed LINQ mp ementat on ( f ava ab e) over the more trad t ona , procedura approaches Because the source data res des oca y n memory, your LINQ query w be ab e to ca nto any NET method n the framework or your app cat on Then there are the other LINQ mp ementat ons des gned to query externa data stores These LINQ prov ders dynam ca y, at runt me, form an express on tree from your LINQ query to generate a query n another anguage (such as T-SQL) that w then execute aga nst an externa data store (such as SQL Server) 472  Part III  Applied SQL

Th s s the case w th both LINQ to SQL and Ent ty Framework So now you ser ous y need to quest on the w sdom of wr t ng a query n one anguage (LINQ) on y to have t trans ated nto another anguage (T-SQL) If you are a ready we versed n T-SQL, you may fee that you can ­produce better T-SQL code than the LINQ runt me Converse y, th s s a very appea ng approach when T-SQL sk s are ack ng Regard ess, t’s mportant to understand that such LINQ quer es can on y ca methods n the NET Framework that have a compat b e equ va ent n T-SQL (for examp e, the way StartsWith can be mp emented us ng LIKE), and that you obv ous y won’t be ca ng methods n your own app cat on because they don’t ex st n SQL Server We’ be exam n ng LINQ to SQL and Ent ty Framework a b t further on, but you’ start your first LINQ exerc se by extend ng the prev ous examp es to work w th LINQ to DataSet

LINQ to DataSet To re terate, LINQ to DataSet operates on a memory-res dent DataSet that s a ready popu ated Th s means that you cont nue to use convent ona ADO NET to fi the DataSet Then once you’ve fi ed the DataSet, you use LINQ to DataSet to further query over the data retr eved from the database Here s an examp e of a LINQ to DataSet query var list = (from cust in ds.Customer where !cust.LastName.StartsWith("A") select cust).ToList();

Now how do you know th s s a LINQ to DataSet query, and not LINQ to someth ng e se? We , that’s part of the dea beh nd LINQ What you’re LINQ- ng to s not supposed to be mmed ate y ­obv ous, the advantage be ng that a LINQ quer es k nd of ook the same regard ess of the ­under y ng data source So you want to zone n on what fo ows the in c ause; that’s what te s you wh ch LINQ mp ementat on s be ng used In th s examp e, the in c ause s fo owed by a DataTable n a typed DataSet, so th s s LINQ to DataSet Th s query bu ds a st of customers whose ast names don’t beg n w th the etter A, and s equ va ent to the fo ow ng code that you m ght have wr tten before LINQ to ach eve the same th ng var list = new List(); foreach (SampleDS.CustomerRow cust in ds.Currency) { if (!cust.LastName.StartsWith("A")) { list.Add(currencyCode); } }

If you’re a ready fam ar w th DataSets, you probab y know that even before LINQ, the RowFilter property of the DataView object cou d have been used to ach eve the very same th ng, ke so var dv = new DataView(ds.Customer); dv.RowFilter = "Customer NOT LIKE 'A*'";

True, the query s expressed n a s ng e statement But there are two prob ems w th th s ­approach F rst, the RowFilter property s a string property Even though t s be ng used to fi ter the strong y

Chapter 10  The M crosoft Data Access Juggernaut   473

typed Customer tab e, m sspe ng Customer n the query (or any other typo) w not get caught by the comp er Second, ask yourse f what query anguage s be ng used here, because t s most ­certa n y not T-SQL Remember that you’ve a ready h t the database to fi the Customer tab e from SQL Server Indeed, th s cou dn’t be T-SQL because the LIKE operator n T-SQL uses the percent ­symbo (%) and not the aster sk (*) symbo to denote a w dcard match So what syntax s th s? As t turns out, t’s a m n -query anguage understood on y by the RowFilter property of the DataView ­object, and a pr me examp e of why LINQ was nvented W th LINQ, you don’t need to start ­earn ng d fferent query anguages (some more obscure than others) for d fferent data sources If you use LINQ to DataSet, then you don’t need to earn th s RowFilter anguage to query DataSets; f you use LINQ to XML, then you don’t need to earn XPath to query XML documents; that’s the dea And yes, w th LINQ to SQL and Ent ty Framework, you don’t have to (necessar y) earn T-SQL to query SQL Server! You just query everyth ng w th LINQ

Querying a Generic DataSet LINQ to DataSet s easy to use, and works both w th gener c DataSets and typed DataSets Th s first ­examp e w use a gener c DataSet So drag another button contro from the Too box and drop t onto the form In the Propert es w ndow, name the button btnLinqToGenericDataSet, and set the Text property to LINQ to Generic DataSet Then doub e-c ck the button and add the code n L st ng 10-12 Listing 10-12  Query ng a gener c DataSet w th L NQ to DataSet.

private void btnLinqToGenericDataSet_Click(object sender, EventArgs e) { const string ConnStr = "Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=true;"; const string SelectTSql = "SELECT * FROM Customer"; var ds = new DataSet(); using (var cn = new SqlConnection()) { cn.ConnectionString = ConnStr; using (var cm = new SqlCommand()) { cm.Connection = cn; cm.CommandText = SelectTSql; using (var da = new SqlDataAdapter()) { da.SelectCommand = cm; da.Fill(ds); } } } var query = from cust in ds.Tables[0].AsEnumerable()

474  Part III  Applied SQL

where !cust.Field("LastName").StartsWith("A") orderby cust.Field("LastName") descending select cust; var dv = query.AsDataView(); MessageBox.Show(string.Format( "Filtered {0} of {1} customers", dv.Count, ds.Tables[0].Rows.Count)); }

None of the code n th s st ng before the LINQ query requ res any exp anat on That’s because, as we’ve sa d, you’re st h tt ng the database us ng convent ona ADO NET just ke you d d n L st ng 10-10 In th s examp e, you’ve retr eved a the customers nto a gener c DataSet, and now you’re us ng LINQ to DataSet to query the resu t set by fi ter ng out a customers whose ast name starts w th A Let’s exam ne th s LINQ to DataSet query up c ose A though the DataTable does have a Rows co ect on property that cou d theoret ca y be ­quer ed us ng LINQ to Objects, there s a spec a NET assemb y for LINQ to DataSet that supp es a few ­extens on methods des gned spec fica y for us ng LINQ aga nst cached DataSets

More Info  Visual Studio automatically set a reference to this assembly when you created the project. The assembly is named System.Data.DataSetExtensions.dll, and you will see it listed under the References node in Solution Explorer. Because the extension methods in this assembly are all defined in the System.Data namespace, you don’t need to add any special using directive to the top of the source file to make them recognizable. The key extens on method n LINQ to DataSet s AsEnumerable, wh ch you nvoke on the ­DataTable LINQ operates over any object that mp ements IEnumerable, but the DataTable object does not ­ mp ement IEnumerable (the tab e tse f s not a co ect on) W th LINQ to DataSet, you query the DataTable tse f, not ts Rows co ect on, and so you ca AsEnumerable on the DataTable to LINQ-­enab e t The rest s an ord nary LINQ query that scans the DataTable, fi ter ng and sort ng on LastName

Note  This is the first of several examples that filter customers by last names not beginning with the letter A. To get conclusive results as you walk through these examples, you should have at least one customer in the database with a last name that starts with the letter A. If you’ve not given any of the customers from earlier examples a last name that starts with A, go ahead and add another customer now with a last name beginning with A so you can test that the filtering logic works as expected. You’re a so us ng the Field extens on method to extract the LastName va ue from each ­ ataRow be ng scanned by the query Th s method makes work ng w th gener c DataSets a tt e D b t eas er by sav ng you the need to cast between object and spec fic data types when referenc ng

Chapter 10  The M crosoft Data Access Juggernaut   475

c­ o umns n a gener c DataRow Because the LastName co umn s a string type, you use Field to obta n ts va ue as a string n the where and orderby c auses w thout hav ng to d rect y cast t So now you’ve defined the LINQ query and ass gned t to the query var ab e, wh ch s dec ared ­us ng var Just ke the nput sequences they consume, LINQ quer es a ways return an IEnumerable of something; and w th LINQ to DataSet n part cu ar, that w be some spec a zed c ass that mp ements IEnumerable defined n the extens ons assemb y But you w typ ca y not need to know or care exact y wh ch c ass that s, and so var s a conven ent way of te ng the comp er to nfer the actua return type from the context of the LINQ query A you rea y care about s the fact that t returns an IEnumerable object, mean ng that the resu ts can be retr eved as a sequence of DataRows F na y, there are two more extens on methods that are un que to LINQ to DataSet, and those are AsDataView and CopyToDataTable In th s examp e, you are us ng AsDataView to execute the query and return the resu ts n an ord nary DataView (just as f you used an o d-fash oned RowFilter) The DataView conta ns the fi tered resu ts, and ho ds ve DataRow nstances t ed to the under y ng ­DataTable Changes made to the DataView are reflected n the DataSet and w get pushed back to the database when you update the DataSet w th a DataAdapter The CopyToDataTable method s ­s m ar to AsDataView, but t makes a copy of the resu ts n a new DataTable nstead of r­ eturn ng ve references n a DataView Th s s usefu when you need to c one the resu ts so that they ex st ­ndependent y of the source DataTable n the DataSet

Querying a Strongly Typed DataSet LINQ to DataSet s even eas er to use w th typed DataSets The next examp e w perform the same f­unct on as the ast one, on y th s t me us ng a typed DataSet Drag another button contro from the Too box and drop t onto the form In the Propert es w ndow, name the button btnLinqToTypedDataSet, and set the Text property to LINQ to Strongly Typed DataSet Then doub e-c ck the button and add the code n L st ng 10-13 Listing 10-13  Query ng a strong y typed DataSet w th L NQ to DataSet.

private void btnLinqToTypedDataSet_Click(object sender, EventArgs e) { var ds = new SampleDS(); using (var da = new SampleDSTableAdapters.CustomerTableAdapter()) { da.Fill(ds.Customer); } var query = from cust in ds.Customer where !cust.LastName.StartsWith("A") orderby cust.LastName descending select cust; var dv = query.AsDataView();

476  Part III  Applied SQL

MessageBox.Show(string.Format( "Filtered {0} of {1} customers", dv.Count, ds.Customer.Count)); }

Aga n, the code to fi the Customer tab e works the same as the ear er typed DataSet examp e n L st ng 10-11 And the LINQ query that fo ows a so works the same as the prev ous LINQ to gener c DataSet examp e n L st ng 10-12 But th s vers on of the LINQ query s even neater and t ghter Not ce that there are no str ng tera s at a (other than the parameter passed to StartsWith); co umn names are strong y typed and there’s no need for any cast ng to and from object, e ther exp c t y or ­us ng the Field method A so, because one of the base c asses (ca ed TypedTableBase) between the strong y typed Customer tab e and the under y ng DataTable mp ements IEnumerable, you don’t need to use the AsEnumerable method e ther Once aga n, you see that typed DataSets offer ­numerous advantages over gener c DataSets Let’s venture forward now nto the next generat on of NET data access APIs Armed w th the ­techn ques and concepts we’ve covered thus far, grasp ng them a w be eas er than you mag ne

Object Relational Modeling (ORM) Comes to .NET There s no doubt that DataSets have served programmers we over the years And f what you need s an n-memory database object mode , or a gener c data conta ner, they st rema n usefu today The DataSet a so has many features to make t “work” ke a bus ness object You’ve seen how a typed DataSet supp es propert es for the Customer tab e s m ar to how you’d code the propert es a Customer ent ty c ass yourse f And because you can embed your own bus ness og c and va dat ons d rect y nto a typed DataSet by everag ng part a c asses, DataSets can de ver many of the same encapsu at on benefits that object-or ented des gns offer when work ng w th pure bus ness ent t es But c ose s not c ose enough Though the DataSet funct ons as an exce ent data transfer object, the fact s that DataSets are not true bus ness objects The DataSet parad gm of tab es, co umns, and re at onsh ps does not a ways trans ate we to the object-or ented deas of c asses, propert es, and co ect ons A row n the typed DataSet’s Customer tab e s just that—a DataRow, w th co umns— and not rea y a convent ona bus ness ent ty object w th ts own ord nary propert es, methods, and events Because t nher ts from DataRow, t can’t a so nher t from a base c ass of your own, nor s t poss b e to have one tab e type nher t from another Furthermore, the manner n wh ch parent-ch d h erarch es of a DataSet are nav gated v a re at onsh ps and spec a zed code-generated methods s un ntu t ve to object-or ented programmers n NET who th nk of “regu ar” objects n s mp er terms A parent ent ty has a property that ho ds a co ect on of ch d ent t es, and each ch d ent ty has a property that refers back to ts parent Man pu at ng h erarch es n a DataSet s qu te d fferent, and somewhat more cumbersome Ser ous object-or ented programmers demand that the r bus ness ent ty c asses everage ­nher tance, po ymorph sm, and des gn patterns Furthermore, they ns st that the m dd e t er ­rema ns free of dependenc es on any part cu ar techno ogy The ph osophy s that data access and

Chapter 10  The M crosoft Data Access Juggernaut   477

­ resentat on ayer techno og es w come and go, but bus ness og c n the m dd e t er—wr tten p n ent ty c asses (such as the qu ntessent a public class Customer)— s bu t to ast The term “p a n o d CLR object” (POCO) s used to refer to such bus ness c asses that are des gned to be nsu ated from ­orb t ng p atform changes Investments n m dd e-t er deve opment are thus better protected, and bus ness og c code surv ves even as new techno og es rep ace o d ones n other ayers of the ­app cat on These concerns (among other factors) have dr ven the deve opment of Object Re at ona Mapp ng (ORM) techno og es, such as LINQ to SQL and Ent ty Framework, wh ch enab e you to bu d data access ayers much more a gned w th object-or ented pr nc p es than DataSets

Defining ORM Bus ness ent t es are represented as objects at the app cat on eve and as relational data n the database, and—one way or another—you need to map the two n your app cat on The key words ta c zed n that sentence define Object Re at ona Mapp ng, or ORM Take the s mp est cases where you wr te the code yourse f to shutt e va ues, one at a t me, between a ­SqlDataReader or a DataRow and the propert es of an object Such code s mapp ng re at ona data to objects, and so there’s your ORM Typ ca y though, the term ORM refers to robust frameworks des gned spec fica y to ­e m nate th s manua abor by automat ng the process n some way Less t me and effort e ­ xpended on the mundane ntr cac es of CRUD operat ons between SQL Server and your ­objects means more t me and resources ava ab e to focus on more mportant and creat ve th ngs, ke core bus ness og c, app cat on features, and user nterface des gn Both LINQ to SQL and Ent ty Framework (as we as other non-M crosoft ORM techno og es, such as the open-source NH bernate brary) comp ete y abstract the pers stence deta s between bus ness ent t es and the database away from you But ORM abstract on can be ra sed much h gher than th s One genera y assoc ates tab es, co umns, and re at onsh ps n the database w th c asses, propert es, and co ect ons n the app cat on, and th s s mp st c v ew m ght be adequate for some bas c app cat ons But n most rea -wor d scenar os, you encounter the so-ca ed ­impedance mismatch prob em, where there sn’t a ways a one-to-one correspondence b ­ etween re at ona tab es n the database and bus ness objects n the app cat on Beyond mere y ­hand ng pers stence deta s, more advanced ORMs (such as Ent ty Framework, but notab y not LINQ to SQL) prov de r ch abstract ons over the data mode tse f that can automat ca y reso ve many mpedance m smatch scenar os A common examp e s the many-to-many re at onsh p between two tab es, mp emented n the database w th a th rd junct on tab e The junct on tab e ho ds fore gn keys that re ate ent t es n the other two tab es to one another, support ng the many-to-many re at onsh p The samp e database does th s w th the CustomerEmployee junct on tab e between Customer and Employee That’s three tab es n the database phys ca y, but on y two objects conceptua y In the app cat on, you on y work w th Customer and Employee objects Each of those objects has a co ect on property that ho ds mu t p e object nstances of the other type, wh ch estab shes the many-to-many re at onsh p at the app cat on eve Ent ty Framework automat ca y

478  Part III  Applied SQL

manages the CustomerEmployee junct on tab e beh nd the scenes, so that your app cat on s concerned on y w th the two og ca ent t es Customer and Employee You’ see th s n ­act on a b t further on w th an examp e, and we w a so ta k about other mpedance m smatch ­scenar os that Ent ty Framework can hand e

Multiple ORM Offerings from Redmond Both LINQ to SQL and Ent ty Framework were des gned to g ve deve opers someth ng better than ­convent ona ADO NET, where you can bu d a data access ayer based on much more ord nary ­objects than a DataSet These ORM techno og es ra se the eve of abstract on above the raw ADO NET c asses even more than DataSets and DataAdapters As the examp es w show, you no onger nteract w th SqlConnection and SqlCommand objects at a Instead, you w use a spec a context object that acts as a contro er for query ng and updat ng data between SQL Server and objects (true ord nary objects; not “row” objects) n your app cat on Desp te these (and other) mportant d fferences w th convent ona ADO NET, the two ORM ­techno og es de ver a V sua Stud o deve oper exper ence that s str k ng y s m ar to that of typed ­DataSets LINQ to SQL and Ent ty Framework each sport a graph ca des gner that’s used as a ­front-end to author XML markup ( n d a ects other than XSD), wh ch n turn dr ves NET code ­generat on of strong y typed objects W th the know edge you a ready have of bu d ng typed ­DataSets n V sua Stud o, the LINQ to SQL and Ent ty Framework mode ng too s w seem very ­fam ar and ntu t ve That’s the good news The ava ab ty of two techno og es so over app ng n purpose s the bad news, and requ res some exp a n ng

LINQ to SQL: Then and Now LINQ to SQL was re eased n 2007 as part of NET 3 5 W th LINQ to SQL, you wr te LINQ quer es n your NET app cat on that get trans ated nto T-SQL quer es dynam ca y at runt me LINQ to SQL prov des a DataContext object that you use as the source of your LINQ quer es, prov d ng funct ona ty s m ar to the way a DataAdapter fi s a DataSet The DataContext a so tracks any changes made to a objects returned from the query resu ts You can then nstruct the DataContext to send the appropr ate nsert, update, and de ete commands back to SQL Server, much ke the way a DataAdapter updates a DataSet Here’s a s mp e examp e of a LINQ to SQL query using (var ctx = new SampleL2SDataContext()) { var q = from customer in ctx.Customers where !customer.LastName.StartsWith("A") orderby customer.LastName descending select customer; var list = q.ToList();



Chapter 10  The M crosoft Data Access Juggernaut   479

MessageBox.Show(string.Format("Non-A customer count: {0}", list.Count)); }

At runt me, LINQ to SQL w parse th s query nto an express on tree and generate a T-SQL query for SQL Server to execute The LINQ to SQL runt me mp ements the funct ona ty of the NET StartsWith method n T-SQL by append ng the percent s gn (%) w dcard symbo to a parameter zed va ue for the LIKE operator exec sp_executesql N'SELECT [t0].[CustomerId], [t0].[FirstName], [t0].[LastName], [t0]. [Balance], [t0].[CreatedAt], [t0].[UpdatedAt] FROM [dbo].[Customer] AS [t0] WHERE NOT ([t0].[LastName] LIKE @p0) ORDER BY [t0].[LastName] DESC',N'@p0 varchar(8000)',@p0='A%'

LINQ to SQL evo ved from the or g na LINQ project, and was created by the anguage ­ eve opment team at M crosoft as an abstract on ayer that prov des a s mp e, one-to-one mapp ng d between objects n your app cat on and database tab es n SQL Server The Ent ty Framework, first re eased n 2008 as part of NET 3 5 SP1, was a project of the data programmab ty team not re ated to LINQ It s substant a y more advanced and prov des many more capab t es than LINQ to SQL The EF team focused on the Ent ty Data Mode (EDM) and created Ent ty SQL as the nat ve anguage for query ng the EDM (you’ soon work hands-on w th the EDM and Ent ty SQL) W th the advent of LINQ, however, t was natura (a most compu sory) for the team to a so prov de LINQ to Ent t es as a strong y typed way of query ng the EDM, so that earn ng Ent ty Framework doesn’t (necessar y) requ re earn ng Ent ty SQL L ke typed DataSets, both LINQ to SQL and EF de ver a des gn-t me exper ence n V sua Stud o for bu d ng an object mode around the database and auto-generat ng typed c asses based on the mode In code, EF a so prov des an ObjectContext object that works s m ar y to the LINQ to SQL’s DataContext And w th LINQ to Ent t es ava ab e as an a ternat ve to Ent ty SQL, both techno og es can use LINQ to query the database mapped to the object mode It was becom ng obv ous that M crosoft had two compet ng NET data access methodo og es that targeted the same scenar os, yet they were offer ng tt e or no gu dance over wh ch one to use Deve opers became more confused, and eventua y one of these techno og es had to emerge as the v ctor Not surpr s ng y, Ent ty Framework won and LINQ to SQL ost M crosoft has fina y made ts v­ s on c ear, and (as of the second vers on of EF re eased w th NET 4 n 2010) recommends the Ent ty ­Framework over LINQ to SQL as the data access so ut on for LINQ to re at ona scenar os

More Info  A post on the ADO.NET team blog explains Microsoft’s position on the future of Entity Framework and LINQ to SQL. You can read it at http://blogs.msdn.com/b/adonet/­ archive/2008/10/29/update-on-linq-to-sql-and-linq-to-entities-roadmap.aspx. So what does th s mean for LINQ to SQL? As far s th s book s concerned, t means that our LINQ to SQL coverage s m ted to the gu dance n th s sect on You shou d not cons der LINQ to SQL for any new deve opment, and t wasn’t around 480  Part III  Applied SQL

ong enough before be ng ec psed by EF for t to have ga ned very much of a strongho d (though t has defin te y won qu te a number of fans) For a ong t me st , you w encounter most y DataSet code and re at ve y tt e LINQ to SQL n ma nta n ng ex st ng product on app cat ons Our ear er ­coverage on convent ona ADO NET prepares you for that But because EF s recommended over LINQ to SQL for new app cat ons, we have made the dec s on not to d g nto LINQ to SQL and to focus nstead on Ent ty Framework deve opment For M crosoft, gett ng the r message out has not been easy It was a tough se to recommend the first vers on of Ent ty Framework (EF1) over LINQ to SQL, because the former acked many mportant capab t es offered by the atter Th s resu ted n an oppos t on group that banded together and posted a “vote of no confidence” n the new techno ogy, d sm ss ng t as ncomp ete and c t ng a tany of comp a nts aga nst t At the top of the st was the ack of support for azy oad ng (a so known as on-demand oad ng, where ch d ent t es are oaded automat ca y as needed) and POCOs (where t sn’t necessary for ent t es to nher t from, and thus be t ed to, any base c ass) EF1 d d not support azy oad ng, and a though t d d support nher tance, ent ty c asses were st forced to der ve from EntityObject LINQ to SQL supported azy oad ng and POCO support from the beg nn ng EF1 was a so sharp y cr t c zed for do ng a poor job of dynam ca y generat ng T-SQL quer es trans ated from your LINQ quer es at runt me It was pa nfu y obv ous that Ent ty Framework was re eased before t was ready, and that t defin te y needed to catch up w th LINQ to SQL In ts n t a form, there was no way for M crosoft to even hope for w despread adopt on of Ent ty Framework Fortunate y, the second vers on of the ADO NET Ent ty Framework (wh ch was re eased w th the NET Framework 4 0 n 2010, and s actua y referred to as EF4) represents a major mprovement over the first re ease EF4 supports azy oad ng, and there are now severa ways to use POCOs so that t s no onger necessary to nher t from EntityObject Improvements were made n dynam c T-SQL code generat on as we , and EF4 w usua y generate T-SQL that’s as good as ( f not better than) what you m ght code by hand Many other cr t c sms were addressed, and new features were added as we M crosoft rea y stened th s t me, and they deserve a ot of cred t for the work they d d w th ts vast y matured second re ease, so we w over ook the fact that, offic a y, there was no EF2 or EF3 LINQ to SQL s re at ve y far more mpa red when you compare t w th EF4 LINQ to SQL can on y be used w th SQL Server, but EF4 works w th any database p atform that has a compat b e ADO NET prov der LINQ s the on y way to query w th LINQ to SQL ( m t ng ts use to C# and VB NET), whereas wh e EF offers LINQ as a strong y typed a ternat ve to the more powerfu Ent ty SQL (wh ch can be used w th any NET anguage) LINQ to SQL supports on y a one-to-one correspondence between the re at ona tab e structure and your object mode , and thus w not reso ve many of the mpedance m smatch scenar os that EF can w th ts EDM You can a so detach and attach ent t es to the ObjectContext n EF4, enab ng pract ca n-t er deve opment that was mposs b e w th EF1 (as we as LINQ to SQL) A though t’s been a rough road (and there s st more room for mprovement), the turn ng po nt has defin te y been reached where you can accept Ent ty Framework ser ous y as M crosoft’s ­ ccess strategy of cho ce A though LINQ to SQL w not evo ve, t s not deprecated e ther data a LINQ to SQL rema ns part of the NET Framework It s re at ve y stra ghtforward to m grate from LINQ to SQL to Ent ty Framework f you have good reasons to do so, but t sn’t necessary ­otherw se ­M grat on makes more sense f the app cat on s be ng extended w th funct ona ty that cou d benefit

Chapter 10  The M crosoft Data Access Juggernaut   481

from ­features ava ab e on y n EF Fortunate y, the m grat on path s fa r y stra ghtforward, as you are ­mov ng toward a w der feature set where pretty much every LINQ to SQL concept has an EF ­equ va ent

Note  In Chapter 13, you build a Windows Phone 7 application that accesses local d ­ evice storage in SQL Server Compact edition (CE) using LINQ to SQL. This represents an ­exception to the general rule of avoidance that we just advocated, and is the one scenario in which LINQ to SQL is not only appropriate but preferred. Specifically, LINQ to SQL can and should be used client-side to access against the (single-user) local SQL Server CE d ­ atabase running on a Windows Phone 7 device—which is very different than accessing a full SQL Server 2012 or SQL Azure instance running in a data center. As you will see in Chapter 13, the phone application also includes a service component that synchronizes local data on the phone with a SQL Azure database running in the cloud, and this service component uses WCF Data Services with the Entity Framework—not LINQ to SQL—to access the SQL Azure database.

Entity Framework: Now and in the Future W th a of the ear er techno og es and gu dance d scussed to th s po nt, you are now pr med and ready to d ve nto Ent ty Framework, M crosoft’s recommended data access so ut on now and for the future The very first th ng you need to earn about s the Ent ty Data Mode (EDM), wh ch es at the heart of Ent ty Framework

Building an Entity Data Model (EDM) There are three parts to the EDM F rst there s the storage schema, wh ch s a thorough d ­ escr pt on of how tab es, co umns, and re at onsh ps are phys ca y arranged n the database Then there s the ­conceptual schema, wh ch descr bes the c asses, propert es, and nav gat on paths of bus ness ­ent t es n the app cat on F na y, you have the mapping schema, wh ch defines how the storage and ­conceptua schemas re ate to one another These three p eces (co ect ve y ca ed the mode ’s ­metadata) are se f-conta ned n a s ng e .edmx fi e ns de your project You author the .edmx fi e n V sua Stud o us ng a graph ca des gner s m ar to the one used to bu d an XSD-based typed ­DataSet ear er n the chapter—w th one cr t ca d fference the EDM design surface only displays the ­conceptual schema And there n s the crux of programm ng aga nst the Ent ty Framework Your app cat on works on y w th objects defined n the conceptua schema as t appears n the des gner, and has no ­awareness of the under y ng database structure defined n the storage schema You’ soon use the Mode Browser and var ous other pane s n V sua Stud o to v ew and configure the storage schema and mapp ngs at des gn t me Then at runt me, t s the Ent ty Framework’s job to retr eve and pers st data between your app cat on and SQL Server, and to reso ve the d fferences between storage and conceptua mode s automat ca y and dynam ca y us ng the mapp ng schema Thus, a three schemas need to be present and ava ab e to the app cat on wh e t’s runn ng 482  Part III  Applied SQL

By defau t, V sua Stud o embeds the three schemas as resources w th n the app cat on to ­ uarantee that they are a ways ava ab e You can a so keep the schema defin t ons outside your g ­app cat on’s assemb y, so they can be ed ted ndependent y w thout requ r ng you to recomp e and redep oy Because the mapp ng ayer effect ve y serves as a buffer between SQL Server and your app cat on, t serves as nsu at on from database changes that are made n the future For examp e, f a co umn name s changed, you can keep the o d name n the conceptua schema by mak ng the change on y n the storage schema of the EDM, and then adjust the mapp ng schema accord ng y Your app cat on can cont nue runn ng unaffected w th the una tered conceptua schema If you have a one-to-one correspondence between every tab e n the database and every c ass n the bus ness object mode , then both the storage and conceptua schemas w match exact y, and the mapp ng schema s mp y connects them up one-to-one That s noth ng d fferent than what you get w th LINQ to SQL or typed DataSets, and the true power of the EDM sn’t rea y be ng tapped But rea -wor d app cat ons rare y have storage and conceptua schemas that match exact y There s usua y some form of impedance mismatch that needs to be reso ved (see the s debar “Defin ng ORM” ear er n the chapter), and the EDM s a powerfu too that can he p to reso ve t Many-to-many re at onsh ps are a good examp e In the samp e database, the ­CustomerEmployee tab e serves as a junct on tab e for the many-to-many re at onsh p between the Customer and ­ hys ca ­Employee tab es But th s jo n tab e s rea y on y an mp ementat on deta of the under y ng p database structure SQL Server does not nherent y support many-to-many re at onsh ps, and so you requ re CustomerEmployee to serve as an ntermed ary tab e between the Customer and Employee tab es, both of wh ch t has a one-to-many re at onsh p w th In the wor d of objects, you care on y about Customer and Employee, whose many-to-many re at onsh ps are mp emented w th ch d ­co ect ons on each s de of each ent ty nstance Through the EDM, the Ent ty Framework ­understands the phys ca tab e ayout n the storage schema and sh e ds you from t Th s makes t eas er to code your app cat on, because you stay focused on og ca ent t es and don’t get entang ed w th the phys ca database deta s As you’re about to see, EF w automat ca y jo n on the junct on tab e for quer es, and update the junct on tab e for re at onsh p changes w thout a CustomerEmployee ent ty n the conceptua schema Now that you understand the EDM, you’re ready to create one Launch V sua Stud o, create a new V sua C# W ndows Forms app cat on, and name t DemoEntityFramework In So ut on Exp orer, r ght-c ck the project and choose Add New Item In the Add New Item d a og, scro down to find and c ck on ADO.NET Entity Data Model Name the new mode SampleEF.edmx and c ck Add V sua Stud o aunches the Ent ty Data Mode W zard so that you can beg n bu d ng the mode Th s w zard s your pr mary too for creat ng or updat ng mode s from an ex st ng database, wh ch s notab y ­d fferent than the Server Exp orer drag-and-drop approach taken by the typed DataSet and LINQ to SQL des gners The database s a ready n p ace, so choose Generate From Database on the first page of the w zard and c ck Next



Chapter 10  The M crosoft Data Access Juggernaut   483

More Info  There will often be an existing database that you can base a new model on ­using the wizard, as in this scenario. But other design strategies are possible as well. EF also supports model-first development, where you start by first building the conceptual model in the Visual Studio designer. This generates Data Definition Language (DDL) statements in a T-SQL script that will create the database from the model. Code-first design is another possible strategy introduced with Entity Framework 4.2, where you start out with neither a database nor a model, and just write your entities in plain code. Both the model and ­database are then derived and generated from your code. F rst you set up the connect on If you wa ked through the ear er typed DataSet examp es, then the ­database connect on SampleDb w st be ava ab e n the Server Exp orer and the EDM W zard w se ect t by defau t Other prev ous y used connect ons w a so be ava ab e n the drop-down st for you to reca eas y, or you can c ck New Connect on to create a new one on the fly from w th n the w zard

Tip  If you didn’t follow along with the typed DataSet examples earlier, you’ll need to ­create the connection now. Click New Connection, select Microsoft SQL Server as the data source, and click Continue. In the Add Connection dialog, type localhost for the Server name, ­select SampleDB from the database drop-down list, and click OK (you must choose ­different settings if you have created the database on another server or instance). The w zard d sp ays the ent ty connect on str ng, wh ch t bases on the SampleDb connect on, as shown n F gure 10-12

Figure 10-12  Sett ng the connect on for an EDM.

484  Part III  Applied SQL

Let’s exp a n th s connect on str ng metadata=res://*/SampleEF.csdl|res://*/SampleEF.ssdl|res://*/SampleEF.msl; provider=System.Data.SqlClient; provider connection string="Data Source=localhost;Initial Catalog=SampleDb;Integrated Security=True;"

Th s m ght ook scary at first, but you can eas y dec pher what t a means from what you’ve a ready earned of the Ent ty Framework Remember that w th the EDM, you code on y aga nst the conceptua schema, wh e Ent ty Framework figures out how to do a the database work at runt me us ng the nformat on n the storage and mapp ng schemas Therefore, a three d st nct schemas must be ava ab e at runt me, and so t makes sense that a connect on str ng n EF must po nt to where they can be found The metadata keyword n the connect on str ng spec fies a p pe-de m ted st of the three schema fi es SampleEF.csdl (conceptua schema defin t on anguage), SampleEF.ssdl (storage schema defin t on anguage), and SampleEF.msl (mapp ng schema anguage) Reca from our ear er d scuss on that these three fi es are housed n the s ng e .edmx fi e ns de your project at des gn t me, and that they (by defau t) get embedded as resources n the assemb y when you bu d the ­so ut on The res://*/ des gnat on te s the EF runt me that the fi es can be ocated n the resources of the ­runn ng assemb y (or any referenced assemb es) The rest of the connect on str ng spec fies wh ch part cu ar ADO NET prov der s be ng used The provider keyword spec fies Sq C ent, wh ch nd cates SQL Server Remember that EF can be made to work w th any database p atform that has a compat b e ADO NET Ent ty Framework prov der The ­provider connection string keyword po nts to the prov der-spec fic connect on str ng, wh ch for ­Sq C ent s the fam ar SampleDb database connect on str ng Thus, an ent ty connect on str ng a ways conta ns references to the three schemas that define the mode , p us a “nested” connect on str ng n the syntax of the part cu ar back-end prov der be ng used w th the Ent ty Framework Beneath the connect on str ng, the w zard d sp ays an a ready checked checkbox te ng you that the connect on str ng w be saved n the app cat on’s configurat on fi e as SampleDbEntities Th s s just what you want, so c ck Next The w zard now scours the database to d scover a the tab es, v ews, and stored procedures that t can pu nto the mode There are four tab es and a handfu more stored procedures, and you w mport them a C ck the checkbox next to Tab es and the one next to Stored Procedures, as shown n F gure 10-13 on the next page, and then c ck F n sh The w zard bu ds the mode , and automat ca y adds the requ red reference to System.Data.Entity (the Ent ty Framework assemb y) to your project The mode then opens up n the des gner, as shown n F gure 10-14 Are you wonder ng why you see on y three ent t es when you se ected four tab es? That’s ­a ready been answered The des gner shows on y the conceptua ent t es, and the CustomerEmployee ­junct on tab e s not a part of the conceptua mode The w zard detected the ro e of th s tab e as the phys ca connect on between two og ca ent t es and exc uded t from the conceptua schema ­CustomerEmployee has a presence on y n the EDM’s mapp ng and storage schemas



Chapter 10  The M crosoft Data Access Juggernaut   485

Figure 10-13  Se ect ng database objects to be mported nto an EDM.

The Mode Browser s one of severa w ndows and panes n V sua Stud o that you use to v ew and configure the storage and mapp ng schemas To open t, r ght-c ck on an empty area of the des gn surface and choose Mode Browser In the Mode Browser, find and expand the Entity Types and Tables / Views nodes F gure 10-14 shows the Mode Browser on the r ght

Figure 10-14  The ent ty data mode des gn surface n V sua Stud o.

486  Part III  Applied SQL

The Mode Browser sts CustomerEmployee among the four tab es n the storage schema, wh ch s denoted by the suffix n SampleEFModel.Store You are go ng to become fast fr ends w th the Mode Browser as you use t to d g nto the EDM—much deeper than we w be demonstrat ng here Take the t me to browse around and get comfortab e w th a of the e ements of the mode structure

More Info  Our coverage of Entity Framework in this chapter merely scratches the surface of this powerful API. If you really want to become an EF expert, we recommend reading Julia Lerman’s Programming Entity Framework, Second Edition (O’Reilly), for an excellent and thorough treatment of the technology. Not ce how the w zard created associations between the ent t es, wh ch t based on tab e r­ e at onsh ps detected n the storage schema The many-to-many assoc at on between Customer and Employee has an aster sk (*) symbo ( nd cat ng mu t p c ty) on both ends, whereas the ­assoc at on ­between Customer and OrderHeader nd cates the one-to-many re at onsh p between those two tab es In add t on to sca ar propert es, each ent ty a so has one or more navigation properties that are based on ts assoc at on(s) w th other ent t es These nav gat on propert es surface as ch d co ect on and parent reference propert es that your app cat on can use to access re ated ent t es n code The w zard a so s ngu ar zed or p ura zed each nav gat on property appropr ate y For examp e, ook at the one-to-many re at onsh p between Customer and OrderHeader Each Customer ent ty has an OrderHeaders property (p ura ) that ho ds a co ect on of OrderHeader (s ngu ar) ent t es ­Converse y, each OrderHeader ent ty has a Customer property (s ngu ar) that po nts back to ts parent ent ty S m ar y, Customer and Employee each have p ura zed nav gat on co ect on propert es of each other (Customer has Employees, and Employee has Customers), because those ent t es are jo ned n a many-to-many re at onsh p Th s s just the way an object-or ented programmer expects the wor d to be, and s a great dea eas er to work w th n code than the DataSet when nav gat ng between re ated ent t es As w th typed DataSets, the EDM des gner acts as a front end that you use to author an XML document transparent y n the background But nstead of us ng the XSD format, th s d a ect of XML represents the metadata for the three schemas of the EDM And rather than tr gger ng the generat on of typed DataSet and TableAdapter c asses from the XML, typed ent ty and ObjectContext c asses are generated nstead Most of the t me, the des gner w sh e d you from the deta s of the under y ng XML and the code that t generates, but t defin te y pays to become fam ar w th them f you rea y want to ga n a so d understand of EF So before wr t ng the code for your first Ent ty Framework ­examp e, et’s qu ck y ook over the two fi es R ght-c ck SampleEF.edmx n the So ut on Exp orer and choose Open W th Se ect XML (Text) E ­ d tor and c ck OK to v ew the fi e as raw XML V sua Stud o w prompt you first to c ose the mode open n the des gner before open ng the raw XML (wh ch is the very XML that you are ­compos ng as you bu d your ent ty mode n the des gner) Scro through the fi e, and you w find an XML representat on of everyth ng you’ve done us ng the des gner If you co apse the nodes beneath , you w see the h gh- eve v ew of the three sect ons for the conceptua , ­storage, and mapp ng schemas As exp a ned, th s s the metadata that s requ red at runt me and gets

Chapter 10  The M crosoft Data Access Juggernaut   487

­ mbedded as a resource n the project’s assemb y when you bu d the so ut on After the metadata, e you w see the ­sect on, wh ch s used on y by V sua Stud o at des gn t me Th s s where the des gner saves ­nformat on about the ayout of shapes (s ze, pos t on, and so on) n the mode d agram Layout nformat on s gnored by the runt me, and you can gnore t now as we When you bu d your app cat on, the sect on s not nc uded w th the metadata that gets embedded as an assemb y resource Next, expand the SampleEF.edmx fi e n the So ut on Exp orer to revea the SampleEF.Designer. cs fi e nested beneath t Doub e-c ck t to open the C# code, wh ch s generated automat ca y ­whenever the .edmx fi e s updated The code s organ zed n reg ons, so start by expand ng the ­Ent t es reg on You w find three c asses for the three ent t es Customer, Employee, and OrderHeader; and you’ see that they a nher t from EntityObject Th s s based on the defau t code generat on strategy wh ch was the on y cho ce ava ab e w th EF1 W th EF4, you can use an a ternat ve code ­generator based on T4 temp ate techno ogy to produce POCO ent t es—c asses that don’t nher t from anyth ng (bes des perhaps other c asses n the mode , f your mode defines nher tance) Expand the nested reg ons further, and you’ find the defin t ons for each property w th change not ficat on og c bu t nto the property setters In add t on to the ent t es themse ves, the generated code nc udes a context c ass for ­programm ng quer es and updates aga nst the mode Expand the Contexts reg on (at the top of the source fi e, just ­beneath the namespace dec arat on) and you’ see the SampleDbEntities c ass that nher ts from ­ObjectContext and funct ons as the pr mary access po nt to your mode at runt me Th s becomes c earer when you expand the ObjectSet Propert es reg on ns de the context c ass There you w find a of the entity sets exposed by the context (one set for each ent ty), named n the p ura (Customers, Employees, and OrderHeaders) You w conduct LINQ quer es aga nst the ent ty sets exposed by the SampleDbEntities context and mater a ze the resu ts from the database nto ve object nstances The context w a so track any changes you make to the ve objects so that the changes can be saved back to the database ater That’s enough exp or ng beh nd the scenes C ose the XML document and et’s move on to the first EF examp e us ng LINQ to Ent t es

Using LINQ to Entities Ent ty SQL s the nat ve anguage n Ent ty Framework for query ng the EDM, and ater on, we w d scuss when or why you’d choose to use t But for now, start w th the eas est way to query ent t es, wh ch s to use LINQ to Ent t es Drag a button contro from the Too box and drop t onto the form Form1 created automat ca y by V sua Stud o In the Propert es w ndow, name the button btnLinqWithDirectSql, and set the Text property to LINQ to Entities Query (Direct SQL) Then doub e-c ck the button and add the code n L st ng 10-14

488  Part III  Applied SQL

Listing 10-14  Query ng an EDM w th L NQ to Ent t es us ng d rect SQL.

private void btnLinqWithDirectSql_Click(object sender, EventArgs e) { // Retrieve via direct SQL: using (var ctx = new SampleDbEntities()) { // Define, but do not execute, the query (deferred execution) // 'q' is of type System.Data.Objects.ObjectQuery var q = from customer in ctx.Customers where !customer.LastName.StartsWith("A") orderby customer.LastName descending select customer; // Implicitly execute query on the server, returning List var list = q.ToList(); MessageBox.Show(string.Format("Non-A customer count: {0}", list.Count)); } }

Before runn ng the app cat on, set a breakpo nt on the ne that dec ares the q var ab e and ­ass gns t to the LINQ query A so start a new SQL Profi er trace, f one sn’t st runn ng from the ear er examp es Now start the app cat on and c ck the button You w h t the breakpo nt r ght before the LINQ query s ass gned to q S ng e-step over the ne and sw tch mmed ate y over to the SQL Profi er trace You may be surpr sed to see that no query was sent to SQL Server Th s h gh ghts a key pr nc p e of LINQ known as deferred execution A you’ve done s define your query and ass gn t to q There s no mag ca method that you ca to actua y execute the query; execut on s mp ed, and w automat ca y k ck off anyt me you try to access a resu t from the query Th s cou d happen s mp y by try ng to foreach your way through q, by b nd ng t to a contro n the user nterface, or by ca ng a method ke ToList on t as you’re do ng here Go back to V sua Stud o, s ng e-step over the next ne, and then sw tch r ght back to the trace Sure enough, you can see the SELECT statement that was just sent to SQL Server, transformed from a LINQ to Ent t es query nto nat ve T-SQL SELECT [Extent1].[CustomerId] AS [CustomerId], [Extent1].[FirstName] AS [FirstName], [Extent1].[LastName] AS [LastName], [Extent1].[Balance] AS [Balance], [Extent1].[CreatedAt] AS [CreatedAt], [Extent1].[UpdatedAt] AS [UpdatedAt] FROM [dbo].[Customer] AS [Extent1] WHERE NOT ([Extent1].[LastName] LIKE 'A%') ORDER BY [Extent1].[LastName] DESC

It’s cr t ca to rea ze that execut ng another q.ToList, or any other attempt to “tap” at q—for ­examp e, even just gett ng a count w th q.Count—fires another query execut on n SQL Server ­Need ess round-­ tr pp ng s obv ous y someth ng to avo d, so t s mperat ve for you to rema n ­cogn zant of deferred ­execut on n your code You’ typ ca y want to query the database just once, and then ho d onto the

Chapter 10  The M crosoft Data Access Juggernaut   489

resu ts n some cached form such as an array, co ect on, or d ct onary The ­IEnumerable extens on methods ToArray, ToList, and ToDictionary are prov ded so you can do just that The generated SQL may not be pretty, but you can see that the NET StartsWith method n the LINQ query’s where c ause was correct y expressed us ng the LIKE operator n the T-SQL query’s WHERE c ause Thus, the fi ter ng and sort ng occurs n SQL Server, and on y the resu ts of nterest get returned to your app cat on EF s qu te capab e of produc ng quer es that perform at east as we as what you’d code by hand—most of the t me You’d never a as the Customer tab e as Extent1 because Extent1 s a mean ng ess word to you, but the a as n no way negat ve y mpacts the query ­performance The runt me must be flex b e enough to compose quer es far more comp ex than th s s mp e examp e, and so t w somet mes generate strange syntax that doesn’t resemb e anyth ng you’d wr te by hand, but st doesn’t necessarily represent a performance h t Of course there are except ons, so you defin te y need to keep an eye on SQL Profi er to make sure that the EF’s dynam ca y generated quer es are reasonab e Enjoy ng the conven ence offered by an abstract on ayer does not mean you can be tota y gnorant of what’s go ng on beneath the surface Th s s true even more as you ra se the eve of abstract on W th the great power that EF hands you comes the great respons b ty of understand ng how t operates and be ng aware of the ­consequences of your act ons, such as how you use deferred execut on or azy oad ng In some cases, you m ght need to refactor a LINQ query to encourage the rut me to generate a better T-SQL query In other cases, you may determ ne that t’s better to wr te the T-SQL query yourse f ns de a ­parameter zed stored procedure, and then use a s mp e LINQ query that mere y ca s the stored ­procedure That’s perfect y fine, because EF doesn’t mpose an a -or-noth ng cho ce You can a ow EF to generate d rect T-SQL from fu y expressed LINQ quer es, and a so ma nta n tota contro over T-SQL query syntax when necessary by us ng stored procedures

Note  If you have strong feelings on the subject of direct SQL and stored procedures, you are not alone. Read the sidebar “Rethinking the Great Direct SQL vs. Stored Procedure Debate” a bit further on in the chapter for a brief discussion on major factors to consider.

Mapping Stored Procedures to the EDM Even though you mported the set of CRUD stored procedures for the Customer tab e nto the mode , EF st generated a d rect T-SQL dynam ca y Import ng stored procedures mere y defines them n the storage schema, but they aren’t actua y mapped to anyth ng yet Ent ty Framework won’t know to use mported stored procedures un ess you map them to the conceptua schema us ng the des gner, wh ch you’ do next Stored procedures that retr eve data are mapped as function imports us ng the Mode Browser You w now mport a funct on to map the SelectCustomers stored procedure Doub e-c ck SampleEF.edmx n the So ut on Exp orer to open the mode n the des gner If the Mode Browser s not a ready v s b e, r ght-c ck on an empty area of the des gn surface and choose Mode Browser Beneath SampleDbModel n the Mode Browser, expand the EntityContainer: ­SampleDbEntities node R ght-c ck the Function Imports node and se ect Add Funct on Import, as shown n F gure 10-15 490  Part III  Applied SQL

Figure 10-15  mport ng a funct on nto the conceptua mode .

Name the funct on SelectCustomers, after the stored procedure you’re mapp ng the funct on to (though t certa n y doesn’t need to match) Th s s the name that you’ refer to n your next LINQ query so that the stored procedure gets ca ed nstead of d rect SQL Next, choose SelectCustomers from the drop-down st of ava ab e stored procedures n the ­storage schema F na y, because you know that th s stored procedure returns the co umns for a ­Customer ent ty, c ck the Ent t es rad o button beneath Returns A Co ect on Of, and then se ect Customer from the drop-down st Th s te s EF that t can and shou d return a co ect on of Customer ent t es when you use the SelectCustomers funct on mport n a LINQ query F gure 10-16 shows the Add Funct on Import d a og box configured to mport SelectCustomers

Figure 10-16  Mapp ng a stored procedure that returns a resu t set to a funct on that returns an ent ty co ect on.



Chapter 10  The M crosoft Data Access Juggernaut   491

Now wr te a mod fied vers on of the prev ous LINQ query that uses the SelectCustomers funct on you just mapped Drag another button contro from the Too box and drop t onto the form, name t btnLinqWithStoredProc, and set the Text property to LINQ to Entities Query (Stored Procedure) Then doub e-c ck the button and add the code n L st ng 10-15 Listing 10-15  Query ng an EDM w th L NQ to Ent t es us ng a stored procedure.

private void btnLinqWithStoredProc_Click(object sender, EventArgs e) { // Retrieve via a function import mapped to a stored procedure: using (var ctx = new SampleDbEntities()) { // Execute the stored procedure on the server immediately // 'q' is of type System.Linq.OrderedEnumerable var q = from customer in ctx.SelectCustomers() where !customer.LastName.StartsWith("A") orderby customer.LastName descending select customer; // Filter & sort stored proc results on the client, returning List var list = q.ToList(); MessageBox.Show(string.Format("Non-A customer count: {0}", list.Count)); } }

Th s LINQ query s v rtua y dent ca to the prev ous one (see L st ng 10-14), except that the S­ electCustomers funct on mport, rather than the Customers ent ty set, s spec fied as the source of the query after the in keyword Everyth ng e se s exact y the same As you’re about to see, however, th s sma change makes an enormous d fference n the way the query executes Once aga n, set a ­breakpo nt on the ne dec ar ng q and make sure you’ve got a current SQL Profi er trace runn ng Start the app cat on and c ck the button When you h t the breakpo nt, s ng e-step over the LINQ query and sw tch back to the SQL Profi er trace You m ght be surpr sed aga n to find that, th s t me, EF ca s out to SQL Server mmed ate y on the ne of code that defines the LINQ query The trace shows the stored procedure ca be ng made when you s ng e-step over that ne exec [dbo].[SelectCustomers]

But wa t The SelectCustomers stored procedure returns every customer n the database You know th s—you wrote the stored procedure! You know that t’s return ng everyth ng, so that means that—un ke the prev ous vers on—the fi ter ng and sort ng operat ons spec fied n the LINQ query are be ng executed on the client Th s s occurr ng after send ng a the Customer rows across the ­network from SQL Server to your app cat on—even rows you don’t care about, even rows that w get thrown away because they don’t pass the where cr ter a There s st deferred execut on at p ay, but t’s ­mere y the n-memory fi ter ng and sort ng of the comp ete resu t set (a ready returned by SQL Server) that gets deferred unt you ca ToList on q n the next ne of code 492  Part III  Applied SQL

So t comes down to th s If you LINQ aga nst any of the ent ty sets n the mode , then t w a ways be deferred execut on of d rect SQL aga nst the tab e(s)/v ew(s) mapped to the ent ty, and resu ts w be fi tered on the server by a dynam ca y generated WHERE c ause that’s based on the LINQ query’s where c ause Th s s a true LINQ to Ent t es query But f you LINQ aga nst a funct on mport mapped to a stored procedure nstead, then the stored procedure w execute mmed ate y, a of ts resu ts w get sent back to the c ent, where the resu ts w subsequent y get fi tered by the LINQ query’s where c ause just ke an ord nary LINQ to Objects query

More Info  This behavior stems from the fact that tables and views are composable (they can be combined with a parameterized WHERE clause in T-SQL) while stored procedures are not. Table-valued functions (TVFs) offer the best of both worlds; they provide the ­encapsulation benefits of stored procedures yet they are composable like tables and views. Unfortunately, TVFs are not currently supported in EF, though Microsoft has confirmed TVF support for the next major release of the Entity Framework. Th s h gh ghts the mportance of understand ng what goes on under the covers w th EF Once you know what’s rea y happen ng, you earn how to use (and not m suse) the techno ogy Thus, f you’re pu ng n a moderate y s zed resu t set from the database w th your stored procedure, and then extract ng a reasonab e subset of that data on the c ent n the where c ause, then that’s a perfect y acceptab e use case On the other hand, f the stored procedure returns a mass ve amount of data and the where c ause n your LINQ query fi ters the resu t set on the c ent down to just a handfu of rows, then you know you are do ng someth ng wrong The SelectCustomers stored procedure n th s examp e has no parameters, but you can certa n y map parameter zed stored procedures to funct on mports as we In that case, a WHERE c ause n the stored procedure can first fi ter the resu t set on the server (us ng parameters passed to t through the funct on mport), and then the where c ause n the LINQ query can fi ter the resu ts further on the c ent Th s s qu te the same th ng as fi ng a DataSet from the database v a a parameter zed stored procedure and then further fi ter ng on that resu t set n memory us ng LINQ to DataSet U t mate y, th s begs the quest on “where’s the where?,” and the answer s that t’s rea y up to your own judgement of how to best d str bute fi ter ng og c between c ent and server n a way that makes the most sense for your app cat on

Rethinking the Great Direct SQL vs. Stored Procedure Debate Ent ty Framework (and LINQ to SQL) has good stored procedure support, but s rea y des gned to query the database w th d rect SQL Yet many database profess ona s reject and forb d the use d rect SQL, ns st ng exc us ve y on the use of stored procedures So now s a good t me to re-v s t th s heated debate, n wh ch secur ty, performance, and ma nta nab ty are the most often c ted po nts of argument



Chapter 10  The M crosoft Data Access Juggernaut   493

Security—D rect SQL s vu nerab e to SQL nject on attacks But th s s true on y f you are bu d ng T-SQL statements dynam ca y us ng str ng concatenat on Don’t forget that even stored procedures are vu nerab e n th s respect, f they generate d rect SQL by concatenat ng str ngs and embed content of an unknown and unparsed or g n The pr mary ne of defense aga nst SQL nject on attacks t to parameter ze the query, wh ch s eas y done w th d rect SQL As ev denced by observ ng a SQL Profi er trace, EF generated a parameter zed database ­command that protects aga nst SQL nject on Performance—Th s s actua y an outdated concern, based on the not on that d rect SQL doesn’t get comp ed ke stored procedures do, and so stored procedures therefore run faster It’s true that stored procedures wou d get part a y comp ed to speed mu t p e execut ons n SQL Server vers ons 6 5 (re eased n 1996) and ear er But as of SQL Server 7 0 (re eased n 1999), that s no onger the case Instead, SQL Server 7 0 (and ater) comp es and caches the query execut on p an to speed mu t p e execut ons of the same query (where on y parameter va ues vary), and that’s true whether execut ng a stored procedure or a d rect T-SQL statement Maintainability—In product on app cat ons, t s very poor pract ce to embed T-SQL ­ rect y n NET code Do ng so t ght y coup es c ent and server og c n a way that qu ck y d snowba s nto a ma ntenance n ghtmare But th s s not the case w th LINQ to Ent t es (or w th LINQ to SQL) You are on y compos ng and ma nta n ng the LINQ query n NET code; the ­trans at on to d rect T-SQL occurs on the fly at runt me when you execute your app cat on, so th s concern s mp y doesn’t app y These cons derat ons shou d change your perspect ve somewhat If you’re fortunate enough to be n tota contro over your app cat on’s arch tecture, you can enjoy a good comprom se w th a hybr d approach You can a ow d rect SQL aga nst tab es and v ews for SELECT ­quer es on y, but cont nue us ng stored procedures for INSERT, UPDATE, and DELETE operat ons W th th s strategy, updates at the app cat on eve rest on top of a stored procedure ayer that can perform cr t ca va dat ons at the database eve You can a so cons der creat ng v ews to expose a m ted or a tered subset of tab e data, and then grant ng SELECT access on y to the v ews and not the tab es Then you can map the ent t es n your data mode to the v ews rather than tab es for query ng, wh e updates get pers sted back us ng stored procedures ­ ELETE So we recommend that you st ck to us ng stored procedures for INSERT, UPDATE, and D operat ons, wh e deny ng d rect access to the under y ng tab es (except for SELECT) Do ng so a ows you to perform add t ona va dat on that cannot be bypassed by c rcumvent ng the ­app cat on ayer and commun cat ng d rect y w th the database server At the same t me, your LINQ quer es w trans ate “natura y” to d rect SQL because SELECT perm ss ons are st granted aga nst the tab e (or v ew) be ng quer ed

If you’re st paused at the breakpo nt n the prev ous examp e, cont nue execut on now and c ose the app cat on It’s t me to start updat ng some data w th Ent ty Framework

494  Part III  Applied SQL

Saving Entity Changes When t comes to update operat ons, th ngs are much more stra ghtforward EF w e ther dynam ca y generate a d rect SQL statement (wh ch s ts defau t behav or) or ca a stored procedure ( f you’ve mapped one) for each changed row E ther way, ent t es get updated one at a t me Rev s t the mode and w re up the three stored procedures for sav ng Customer ent ty changes to the database (they were created at the beg nn ng of the chapter; refer to L st ng 10-1 f you need to rev ew them) You w nstruct EF not to generate d rect SQL statements for sav ng Customer ­ent t es n the database, but to ca these stored procedures nstead Doub e-c ck SampleEF.edmx n the ­So ut on Exp orer to open t n the des gner Then r ght-c ck anywhere on the Customer ent ty and choose Stored Procedure Mapp ng to d sp ay the Mapp ng Deta s pane (p n t down to keep t n v ew) In t a y, there are no mapp ngs defined, wh ch means that EF dynam ca y generates d rect SQL statements for INSERT, UPDATE, and DELETE You’re now go ng to change that behav or and map a three stored procedures a ong w th the r parameters and return va ues C ck and choose the InsertCustomer stored procedure from the ­ rop-down st You need to map each stored procedure parameter to the correspond ng ent ty d property, but the des gner he ps out where t can Because parameters and co umns are cons stent y named, a of the nput parameters requ red by the stored procedure were correct y mapped The on y add t ona th ng you need to do s map the three va ues returned by the stored procedure back nto the ent ty n the Resu t Co umn B nd ngs Reca (from L st ng 10-1) that the InsertCustomer stored procedure returns the CustomerId, ­CreatedAt, and UpdatedAt va ues—ass gned to the new customer row n the database—as ­sca ar v­ a ues n a ­s ng e-row resu t set W th the Resu t Co umn B nd ngs, you can shove those va ues r ght back nto the c ent nstance and refresh ts v ew of the new ent ty after the stored procedure ­executes Thus, the c ent doesn’t need to retr eve the ent re ent ty after add ng t on y to obta n these three va ues You just need to type the name of each co umn n the s ng e-row resu t set and map t to the appropr ate property C ck and type CustomerId Add the other two return va ues the same way, typ ng the r ­respect ve names CreatedAt and UpdatedAt Unfortunate y, the des gner offers no he p mapp ng the resu t b nd ngs, and a three are mapped to the CustomerId property by defau t Th s happens to be a ­correct assumpt on for the first b nd ng, but the other two b nd ngs need to be changed C ck the ­Property drop-down st for each of them and se ect C ­ reatedAt and ModifiedAt, respect ve y You’re done w th the Insert funct on and can move on to the Update funct on

Important  The ADO.NET Entity Framework 4 supports stored procedures that return scalar values either as output parameters or as a single-row result set, with one glaring ­exception. The new primary key value assigned by SQL Server can only be sent back to a newly ­created entity’s key using a single-row result set; an output parameter cannot be used. Thus, if the InsertCustomer stored procedure was written to return its three server-assigned values (one of them being the newly assigned CustomerId primary key value) using output parameters rather than a single-row result set, you could not use Entity Framework to call InsertCustomer.



Chapter 10  The M crosoft Data Access Juggernaut   495

C ck and choose the UpdateCustomer stored procedure from the ­drop-down st Once aga n, a of the same-named parameters and propert es get mapped ­automat ca y by the des gner The on y parameter that doesn’t have a match ng property s @OriginalUpdatedAt C ck the Property drop-down st and se ect UpdatedAt to set the mapp ng, but also check the Use Or g na Va ue checkbox Th s s an mportant guarantee that the UpdatedAt va ue or g na y retr eved from the database s what gets sent back to UpdateCustomer n th s parameter Because the context preserves the or g na va ues of every property, the des gner a ows you to map them to any stored procedure parameters you des re The UpdateCustomer stored procedure ra ses a concurrency error f another user changes the UpdatedAt va ue between the t me that you or g na y retr eve the customer and the t me you ­attempt to save t back Pass ng n the or g na UpdatedAt va ue n the @OriginalUpdatedAt parameter you just mapped a ows the stored procedure to perform the check If the update succeeds norma y, ­UpdateCustomer returns the new UpdatedAt va ue that got ass gned to the changed customer row n the database as a sca ar va ue n a s ng e-row resu t set As w th the Insert funct on, you want to br ng the updated sca ar va ue for UpdatedAt back nto the c ent nstance to refresh ts v ew of the mod fied ent ty after the stored procedure executes C ck and type UpdatedAt Then c ck the Property drop-down st to the r ght and se ect UpdatedAt to map the return va ue back nto the ent ty That does t for the Update funct on; the Delete funct on s next and ast (and eas est) C ck and choose the DeleteCustomer stored procedure from the ­ rop-down st The one and on y @CustomerId parameter s automat ca y mapped to the ­CustomerId d property and you’re done! F gure 10-17 shows the Mapp ng Deta s pane w th a three funct ons mapped to stored procedures

Figure 10-17  Mapp ng three stored procedures for an ent ty s nsert, update, and de ete operat ons.

496  Part III  Applied SQL

W th a the CRUD stored procedures mapped, you’re ready to wr te some more code Drag ­another button contro from the Too box and drop t onto the form, name t btnUpdatingData, and set the Text property to Updating Data Then doub e-c ck the button and add the code n L st ng 10-16 Listing 10-16  Updat ng an EDM object graph.

private void btnUpdatingData_Click(object sender, EventArgs e) { using (var ctx = new SampleDbEntities()) { var customers = (from cust in ctx.Customers where !cust.LastName.StartsWith("A") orderby cust.LastName descending select cust).ToList(); if (customers.Count == 0) { MessageBox.Show("There are no customers"); return; } var firstCustomer = customers[0]; var orderCount = firstCustomer.OrderHeaders.Count; MessageBox.Show(string.Format( "Customer {0} has {1} order(s)", firstCustomer.CustomerId, orderCount)); // Change the customer's name firstCustomer.FirstName = "Keith"; firstCustomer.LastName = "Harris"; // Add a customer order firstCustomer.OrderHeaders.Add(new OrderHeader() { ShipVia = "Regular Mail", OrderStatus = "Open", }); ctx.SaveChanges(); } }

Set a breakpo nt at the top of the method (on the using statement that nstant ates the new S­ ampleDbEntities object) Make sure you’ve got a SQL Profi er trace runn ng, and then run the ­app cat on C ck the button and you’ h t the breakpo nt Step over the ne to create the context and advance to the next ne of code Th s t me, the LINQ to Ent t es query s wrapped ns de ­parentheses and ToList s ca ed on t a n one ne of code Th s effect ve y short-c rcu ts deferred execut on, and forces the query to run mmed ate y (and on y once) The ToList method mater a zes the query resu ts nto Customer objects, and returns a popu ated List co ect on n customers When you s ng e-step over the ne, the SQL Profi er trace shows that EF generates and executes the same SELECT statement you saw used w th L st ng 10-14 Aga n, d rect SQL s be ng used because you’re

Chapter 10  The M crosoft Data Access Juggernaut   497

query ng aga nst the Customers ent ty set, and not the SelectCustomers funct on mapped to the SelectCustomers stored procedure So th s s a rea LINQ to Ent t es query that executes n SQL Server and returns on y fi tered Customer rows to your app cat on Step over the test that checks for no customers You’ve a ready added severa customers n ear er examp es, so the code shou d jump r ght to the ne that gets the first customer from the st Step over that ne, and a so over the next ne that gets the customer’s order count A though you are ­perform ng s m ar operat ons as you d d w th DataSets ear er (see L st ng 10-11), th s code s s gn ficant y more flex b e and object-or ented than that examp e Let’s exp a n further to see how The Customer object n firstCustomer has an OrderHeaders co ect on wh ch, ke any ord nary co ect on, has a Count property that te s you how many orders the customer has Th s s much more ntu t ve than need ng to ca the GetOrderHeaderRows method to get an array of ch d rows so that you can get the order count w th the array’s Length property, as you d d w th the typed DataSet

Object Context Lazy Loading and Change Tracking But wa t—your LINQ to Ent t es query retr eves on y customers from the database, but none of the r orders The ear er DataSet examp e se ected both customers and orders, so GetOrderHeaderRows s mp y returned an array of n-memory order rows a ready retr eved from the database So how s EF ab e to get the count of orders that you haven’t yet retr eved? The answer s azy oad ng, wh ch s on by defau t If you s ng e-step some more and sw tch back to SQL Profi er trace, you w see that EF ssues a second T-SQL query to retr eve the orders be ong ng to just the first customer Th s was tr ggered by your request to obta n the count through the customer’s OrderHeader co ect on’s Count property Now the context s track ng a customers retr eved by the or g na query, p us the orders for th s one customer just retr eved by azy oad ng At th s t me, a of the objects be ng tracked by the context have an EntityState property whose va ue s set to EntityState.Unchanged Compare the flex b ty afforded by EF azy oad ng w th the typed DataSet approach taken n L st ng 10-11 In that examp e, you oaded every ch d ent ty, not know ng wh ch ones w rea y be needed and wh ch won’t W th EF, t’s much eas er to str ke the opt mum ba ance of what gets oaded and when, wh ch often var es n d fferent s tuat ons You can eager- oad some re ated ent t es up front (by us ng the Include method n your n t a LINQ query to generate the appropr ate T-SQL jo n), and a ow azy oad ng to automat ca y fetch other re ated ent t es when you reference them Or you can d sab e azy oad ng by sett ng the LazyLoadingEnabled property on the context object to false and contro everyth ng yourse f Then you cou d eager- oad some re ated ent t es up front, and defer the oad ng of other re ated ent t es unt you need them (by exp c t y nvok ng the Load method on a ch d co ect on or parent reference property) Step over the next ne and d sm ss the MessageBox that d sp ays the customer ID and order count Now the customer’s name s changed by sett ng ts FirstName and LastName propert es Th s n turn automat ca y changes the EntityState of the customer to EntityState.Modified A new order s then created for the customer Aga n the code s c eaner than L st ng 10-11 Rather than be ng forced to ca a spec a method ke AddOrderHeaderRow to add a new DataRow to a DataTable, you treat your ent t es as p a n objects and co ect ons The customer object has an OrderHeaders property, wh ch s an ord nary co ect on w th an ord nary Add method The new order s created n ne d rect y ns de 498  Part III  Applied SQL

the Add method, us ng object initializers to set va ues for a ts propert es Now the context starts track ng the new order, and EF sets ts EntityState property to EntityState.Added It’s now t me to push a your data changes back to SQL Server Pers stence s s mp e; just nvoke the s ng e SaveChanges method on the context EF does a the work of ssu ng transact ona zed, proper y-ordered updates to the database for a the objects be ng tracked for changes by the context Step over the SaveChanges method ca and then sw tch over to SQL Profi er to exam ne the trace You w see that EF sent two commands to SQL Server F rst t generated and ssued a d rect INSERT statement to add the new order to the OrderHeader tab e Then, because of the stored ­procedure mapp ngs you estab shed for the Customer ent ty, EF ca ed the UpdateCustomer stored procedure to save the changes made to the customer name You can a so see that EF wrapped those two operat ons up ns de BEGIN TRANSACTION and COMMIT TRANSACTION statements ­automat ca y

Note  If you did not enable transactional events for the SQL Profiler trace as we explained earlier, you will not see the BEGIN TRANSACTION and COMMIT TRANSACTION statements in the SQL Profiler trace. Stop the trace, and start a new one with transactional events ­enabled, as shown in Figure 10-4. When you rerun the code, you will see these statements appear in the trace surrounding the insert and update operations.

The EF n-Tier Challenge We’ve cr t qued the DataSet qu te sharp y as we compared t w th EF But DataSets are st eas er than EF n n-t er scenar os Reca how the ear er DataSet examp e (back n L st ng 10-10) used two adapters (and thus, two separate connect ons)—one w th a command to do the fi and another w th three commands to do the update You d dn’t need to use two adapters; you cou d just have eas y used one adapter configured w th a four commands We d rected you to do so on y to s mu ate the d sconnected state that the DataSet exper ences n an n-t er arch tecture, n wh ch t traverses the t ers between the fi and the update Wh e d sconnected, the DataSet tracked a changes n each row’s RowState property so they cou d be saved back to the database It s ent re y se f-re ant for ­track ng ts own changes, and you aren’t requ red to ho d on to the same adapter that fi ed t n order to update t after the round-tr p The Ent ty Framework s very d fferent You must perform the fi and update us ng one and the same context, or you’ be faced w th ser ous prob ems A though each ent ty does ndeed have an ­EntityState property that funct ons ke the RowState property does w th DataSets, that property va ue s mere y a reflect on of the ent ty state stored nterna y n the context object that’s actua y ­track ng the ent ty Ent ty nstances are not se f-track ng as DataSets are; they depend on the context for change track ng Yet the context s not ser a zab e, and does not trave a ong w th the ent ty nstances t s track ng as they traverse the t ers The context therefore does not surv ve the round tr p When you retr eve objects from the database and then destroy the context, those objects are no onger ab e to track changes on the r own as you pass them through the t ers W thout the context, the EntityState property s a ways set to EntityState.Detached, and w no onger update a ­ utomat ca y

Chapter 10  The M crosoft Data Access Juggernaut   499

as you app y changes to the objects That means t becomes someone e se’s job to track the r changes wh e d sconnected from a context, f you want to be ab e to pers st those changes back to the ­database ater When the t me does come to save the changes, you w a so need to re-attach the objects to a new context w th the change nformat on, so that the context can carry out the correct commands to update the database These concerns must be addressed f you want to mp ement an n-t er arch tecture bu t on the Ent ty Framework Fortunate y, there are a number of read y ava ab e so ut ons that you can everage to a d you n the task As of EF4, you can hook up detached objects rece ved from the c ent to a new context and exp c t y set change nformat on for the update You can a so overr de V sua Stud o’s defau t code generat on strategy n the EDM des gner and produce rea se f-track ng POCO ent t es from the mode WCF Data Serv ces and WCF RIA Serv ces are two other techno og es that can he p you bu d ­n-t er app cat ons w th the Ent ty Framework Each n the r own un que way, these two APIs stream ne CRUD operat ons n d sconnected n-t er scenar os, and they both work seam ess y w th the ent ty c asses produced by the defau t EDM code generator n V sua Stud o We cover WCF Data Serv ces and WCF RIA Serv ces n the next chapter

Resolving Impedance Mismatch The mapp ng ayer n the EDM s a very powerfu aspect of Ent ty Framework W th t, you can des gn many d fferent types of abstract ons over a phys ca database structure For examp e, you can define a s ng e conceptua ent ty based on two tab es n the database that have a one-to-one re at onsh p w th each other You can a so der ve mu t p e conceptua ent t es from a s ng e tab e In the samp e ­database, there s a many-to-many re at onsh p junct on tab e that s comp ete y abstracted away from you by the mapp ng ayer These are examp es are what’s common y referred to as ­impedance mismatch—where you don’t have an exact one-to-one correspondence between tab es n the ­database and ent t es n the app cat on (see the s debar “Defin ng ORM” ear er n the chapter) Rev s t the Mapp ng Deta s pane to see how t can be used to custom ze the mode and reduce mpedance m smatch Doub e-c ck SampleEF.edmx n the So ut on Exp orer to open t n the des gner Then r ght-c ck anywhere on the Customer ent ty and choose Tab e Mapp ng to d sp ay the Mapp ng Deta s pane, as shown n F gure 10-18

Figure 10-18  Mapp ng an ent ty and ts propert es w th tab es or v ews and the r co umns.

500  Part III  Applied SQL

You used the Mapp ng Deta s pane ear er to map three update stored procedures to the ­ ustomer ent ty, but stored procedure mapp ng s on y one of two purposes that th s pane serves Its C other purpose s to et you map tab es and co umns w th ent t es and propert es You get dropped nto the appropr ate mode based on whether you choose Stored Procedure Mapp ng or Tab e ­Mapp ng from the r ght-c ck context menu n the des gner, and you can a so togg e between the two modes by c ck ng the two buttons n the eft marg n of the Mapp ng Deta s pane The mapp ng shows that the Customer ent ty s mapped to the s ng e Customer tab e, and that there s a one-to-one mapp ng between same-named propert es and co umns Th s s how the w zard set th ngs up or g na y by defau t The Mapp ng Deta s pane ets you take th ngs further than th s, by a ow ng you to map add t ona tab es or v ews from the storage schema to the og ca Customer ent ty For examp e, f you had another tab e n the database that had a one-to-one re at onsh p w th Customer, you cou d c ck the drop-down st to br ng the second tab e and ts co umns nto the ent ty mapp ng You wou d then remove the second tab e from the conceptua ­schema, and the s ng e Customer ent ty wou d have a co ect ve set of propert es whose va ues are backed by two separate tab es n the database EF wou d automat ca y jo n the two tab es together when query ng for a s ng e Customer, and a so sp t a Customer update operat on nto separate ­commands to update each of the tab es nd v dua y A so not ce the drop-down st, wh ch ets you embed a row fi ter nto the mode for the mapp ng For examp e, mag ne that nstead of hav ng two separate tab es, both Customer and Employee ent t es n the mode were based on a s ng e under y ng Person tab e n the database Th s Person tab e wou d have a PersonType co umn to d st ngu sh between customers and emp oyees The PersonType co umn (ca ed the discriminator) cou d be spec fied accord ng y n the tab e mapp ng cond t ons for the Customer and Employee ent t es As a resu t, you cou d ­cont nue work ng conceptua y w th customers and emp oyees as d st nct ent ty types (or, f you w sh to ­everage nher tance, as d st nct ent ty types that der ve from the same base Person type), and a ow EF to manage the mapp ng to the Person tab e automat ca y beh nd the scenes If you were to query for a Customer or an Employee, EF wou d test PersonType n the database query accord ng y to return rows of the correct type S m ar y, EF wou d automat ca y store the correct va ue n the PersonType co umn depend ng on whether you were sav ng a customer or an emp oyee to the database At no t me wou d your code be aware of the actua Person tab e or the PersonType co umn In fact, you cou d change the database des gn n the back-end from us ng two tab es to one (and back aga n) w thout mpact ng your app cat on’s code whatsoever

Working with Many-to-Many Relationships Let’s demonstrate th s abstract on concept w th the many-to-many re at onsh p between Customer and Employee n the samp e database Drag two st box contro s from the Too box and drop them onto the form Name one of them lstCustomers and the other lstEmployees Next, drag and drop another button contro , name t btnManyToMany, and set ts Text property to Many To Many ­Relationships Now doub e-c ck the button and add the code n L st ng 10-17



Chapter 10  The M crosoft Data Access Juggernaut   501

Listing 10-17  Work ng w th many to many re at onsh ps.

private void btnManyToMany_Click(object sender, EventArgs e) { using (var ctx = new SampleDbEntities()) { // Create 3 customers and 3 employees var custClaus = new Customer() { FirstName = "Claus", LastName = "Hansen" }; var custTerry = new Customer() { FirstName = "Terry", LastName = "Adams" }; var custDan = new Customer() { FirstName = "Dan", LastName = "Park" }; ctx.Customers.AddObject(custClaus); ctx.Customers.AddObject(custTerry); ctx.Customers.AddObject(custDan); var empDavid = new Employee() { FirstName = "David", LastName = "Alexander" }; var empAndy = new Employee() { FirstName = "Andy", LastName = "Jacobs" }; var empErik = new Employee() { FirstName = "Erik", LastName = "Ryan" }; ctx.Employees.AddObject(empDavid); ctx.Employees.AddObject(empAndy); ctx.Employees.AddObject(empErik); // Create these many-to-many relationships: // Customer : Employees | Employee : Customers // Claus : Andy, Erik | Andy : Claus, Terry // Terry : Andy, David | David : Terry, Dan // Dan : David, Erik | Erik : Claus, Dan // Can either add employees to customers... custClaus.Employees.Add(empAndy); custClaus.Employees.Add(empErik); custTerry.Employees.Add(empAndy); custTerry.Employees.Add(empDavid); // ...or add customers to employees empErik.Customers.Add(custDan); empDavid.Customers.Add(custDan); // Save new customers, employees, and their relationships to the database ctx.SaveChanges(); // Get customers belonging to employee David (Terry and Dan) this.lstCustomers.Items.Clear(); foreach (var cust in empDavid.Customers) { this.lstCustomers.Items.Add( string.Format("{0} {1}", cust.FirstName, cust.LastName)); }

502  Part III  Applied SQL

// Get employees belonging to customer Terry (Andy and David) this.lstEmployees.Items.Clear(); foreach (var emp in custTerry.Employees) { this.lstEmployees.Items.Add( string.Format("{0} {1}", emp.FirstName, emp.LastName)); } } }

Th s s rea y s mp e code A t does s create three customers and three emp oyees, and then estab shes many-to-many re at onsh ps between them by add ng nto each other’s co ect ons Not ce how t doesn’t matter wh ch d rect on you work n; you can add customers to an emp oyee’s ­Customers co ect on, or you can add emp oyees to a customer’s Employees co ect on S ng e-step through the code wh e runn ng a SQL Profi er trace to watch t work When the SaveChanges method s ca ed on the context, the trace shows a the commands sent to the database As expected, the customers are created w th nd v dua ca s to the InsertCustomer stored procedure, and the emp oyees are created w th nd v dua y generated INSERT statements nto the Employee tab e But you’ a so not ce that add t ona INSERT statements nto the CustomerEmployee junct on tab e were generated to create the many-to-many re at onsh ps n the database Th s s more EF mag c at work S m ar y, f you were to break re at onsh ps by remov ng objects from co ect ons n the app cat on, EF wou d automat ca y ssue the appropr ate DELETE statements aga nst the CustomerEmployee tab e n the database Next, the code starts a foreach oop to terate a of the customers assoc ated w th emp oyee Dav d If you s ng e-step the code’s entry nto the oop, you w see n the trace that EF automat ca y ssues a query to SQL Server to retr eve the re ated customers from the Customer tab e by jo n ng on the CustomerEmployee junct on tab e Then there s another foreach oop that s s m ar, but works n the oppos te d rect on Th s one terates a of the emp oyees assoc ated w th customer Terry, and the trace shows that EF runs a query n SQL Server to retr eve the re ated emp oyees from the Employee tab e by aga n jo n ng on CustomerEmployee

Exploring the Entity SQL Alternative You w now earn about the Ent ty SQL anguage, wh ch offers a way to query the EDM w thout us ng LINQ to Ent t es Why two ways to query the EDM? We , first understand that Ent ty SQL s actua y the nat ve anguage of EF des gned spec fica y for query ng an EDM It was created as a core part of Ent ty Framework before the LINQ revo ut on, and ex sts ndependent y of LINQ Most of the t me, you’ st use LINQ to Ent t es and enjoy the benefits of strong-typ ng and object mater a zat on that t prov des However, there may be t mes when you’ need to use Ent ty SQL to wr te a query that can’t be expressed as we (or at a ) n LINQ Ent ty SQL a so offers a way to stream query resu ts nto a data reader when you don’t need to mater a ze them nto objects, as we’ demonstrate short y



Chapter 10  The M crosoft Data Access Juggernaut   503

Ent ty SQL bears a strong resemb ance to the T-SQL anguage upon wh ch t s based The key ­ fference between them s that Ent ty SQL express ons refer to conceptua ent t es and ­propert es d n the mode , rather than phys ca tab es and co umns n the database Let’s wr te a s mp e query to ­demonstrate Drag another button contro from the Too box and drop t onto the form Name the button btnEntitySql and set ts Text property to Entity SQL Then doub e-c ck the button and add the code n L st ng 10-18 Listing 10-18  Us ng Ent ty SQL nstead of L NQ to return ent ty objects.

private void btnEntitySql_Click(object sender, EventArgs e) { const string EntitySql = @" SELECT VALUE cust FROM SampleDbEntities.Customers AS cust WHERE cust.LastName NOT LIKE 'A%' ORDER BY cust.LastName"; using (var ctx = new SampleDbEntities()) { // Define, but do not execute, the Entity SQL query (deferred execution) var q = ctx.CreateQuery(EntitySql); // Implicitly execute query on the server, returning List var list = q.ToList(); MessageBox.Show(string.Format("Non-A customer count: {0}", list.Count)); } }

Th s code s funct ona y equ va ent to what you wrote n L st ng 10-14 The on y d fference s that you’re us ng Ent ty SQL nstead of LINQ to Ent t es to query the database for the des red customers The EDM st cont nues to funct on as a mapp ng ayer between conceptua and phys ca mode s, and the context s st used to nvoke the query and track any changes made to the resu ts Observe how the code runs by s ng e-stepp ng through t w th a runn ng SQL Profi er trace as usua The CreateQuery method s nvoked on the context, rather than a LINQ to ­Ent t es query as n L st ng 10-14 Th s method accepts a str ng parameter that conta ns an Ent ty SQL ­express on expected to return a sequence of Customer ent t es As w th LINQ, th s method mere y defines the query but does not execute t Execut on s deferred unt the next ne of code that ca s ToList The ToList method ca k cks off the query n SQL Server as before You can see th s by ­watch ng the SQL Profi er trace as a step over the method ca w th the debugger Take a c oser ook at the syntax of th s Ent ty SQL query SELECT VALUE cust FROM SampleDbEntities.Customers AS cust WHERE cust.LastName NOT LIKE 'A%' ORDER BY cust.LastName

504  Part III  Applied SQL

As ment oned, Ent ty SQL quer es ook very s m ar to T-SQL quer es They can even be ­ arameter zed just ke T-SQL quer es are But remember that you’re dea ng w th the conceptua p mode here, not tab es In th s query, Customers refers to the ent ty set of Customer ent t es exposed by the Samp eDbEnt t es mode , not the Customer tab e n the database (even though they current y happen to be mapped one-to-one) S m ar y, LastName refers to a property of the Customer ent ty, not the LastName co umn of the Customer database tab e The VALUE keyword spec fies that the query returns a s ng e strong y typed object, wh ch s a Customer ent ty n th s case A so note that you must a as ent ty set names spec fied n the FROM c ause, un ke T-SQL where tab e a ases are opt ona In th s examp e, Customer ent t es n the Customers ent ty set are a ased as cust

Working with EntityClient We conc ude our Ent ty Framework coverage (and th s chapter) w th EntityClient, wh ch s yet another way to query the EDM Ent tyC ent on y understands Ent ty SQL; t doesn’t know anyth ng about LINQ Us ng Ent tyC ent, you w query the mode w th Ent ty SQL and stream the resu ts back us ng just a reader object Drag another button contro from the Too box and drop t onto the form Name the button ­btnEntityClient and set ts Text property to EntityClient Then doub e-c ck the button and add the code n L st ng 10-19 Listing 10-19  Us ng Ent ty SQL w th Ent tyC ent to return a stream ng reader.

private void btnEntityClient_Click(object sender, EventArgs e) { const string ConnStr = "name=SampleDbEntities"; const string EntitySql = @" SELECT VALUE cust FROM SampleDbEntities.Customers AS cust WHERE cust.LastName NOT LIKE @Pattern ORDER BY cust.LastName"; var names = new List(); using (var conn = new EntityConnection()) { conn.ConnectionString = ConnStr; conn.Open(); using (var cmd = new EntityCommand()) { cmd.Connection = conn; cmd.CommandText = EntitySql; cmd.Parameters.AddWithValue("Pattern", "A%"); using (var rdr = cmd.ExecuteReader(CommandBehavior.CloseConnection | CommandBehavior.SequentialAccess)) { while (rdr.Read()) { var firstName = rdr.GetString(1);



Chapter 10  The M crosoft Data Access Juggernaut   505

var lastName = rdr.GetString(2); var name = string.Format("{0}, {1}", lastName, firstName); names.Add(name); } rdr.Close(); } } this.lstCustomers.DataSource = names; } }

Look fam ar? It shou d! Compare th s code w th L st ng 10-6 way back at the beg nn ng of the chapter You’re us ng raw ADO NET objects to query the EDM w th the Ent tyC ent prov der, just ke raw objects have been used to query SQL Server d rect y w th the Sq C ent for years It’s the exact same pattern deve opers have been fo ow ng s nce NET 1 0, though there are a few subt e ­d fferences that we’ po nt out as we exp a n the code Ent tyC ent s supp ed by the System.Data.Entity assemb y that your project s a ready ­referenc ng (reca that the EDM des gner automat ca y added that reference for you when you created the mode ) Just as w th Sq C ent at the beg nn ng of the chapter, you need to nform the comp er where t can find the EntityConnection and EntityCommand c asses referred to n your code by add ng the fo ow ng using d rect ve at the top of the source fi e using System.Data.EntityClient;

The ConnStr constant defined at the top of the method s mp y po nts to the named ­connect on SampleDbEntities Th s named connect on refers to the actua ent ty connect on str ng (stored and ma nta ned n the app cat on configurat on fi e) that fu y descr bes the metadata ocat on and prov der-spec fic connect on nformat on for the mode Next, the EntitySql constant s defined, and s set to the same Ent ty SQL express on as the ast ­examp e, except that you are parameter z ng the va ue for LIKE w th @Pattern rather than hardcod ng t as ‘A%’ After n t a z ng an empty st of str ngs n names, you create a new EntityConnection n conn, set ts ConnectionString property to the ConnsStr constant you defined ear er, and nvoke the Open method on t to estab sh a connect on to the mode Next, you create an EntityCommand object n cmd and set ts Connection property to a ­ ssoc ate the command w th the open connect on The CommandText property gets set to the Ent ty SQL ­statement defined ear er n the EntitySql constant Because the Ent ty SQL has a parameter ca ed @Pattern, you supp y the parameter name and va ue to the command’s Parameters co ect on us ng the AddWithValue method just as you d d n L st ng 10-3 w th T-SQL Note, however, that you do not prefix parameter names w th the at-s gn (@) symbo n the Parameters co ect on as you do w th the Sq C ent, and so the parameter name s spec fied as Pattern rather than @Pattern Now you nvoke ExecuteReader to nvoke the query and obta n an EntityDataReader object that returns the query resu ts The EntityDataReader works just ke the SqlDataReader, w th ­severa ­add t ona constra nts F rst, not ce the CommandBehavior.SequentialAccess sett ng that you’re 506  Part III  Applied SQL

­comb n ng w th the CommandBehavior.CloseConnection sett ng that you were us ng ear er when c­ reat ng a SqlDataReader by ca ng ExecuteReader on a SqlCommand Th s add t ona sett ng s ­requ red w th Ent tyC ent, and forces str ct sequent a consumpt on of the data stream A though a data reader s an nherent y sequent a mechan sm, the SqlDataReader w st et you access co umn va ues random y w th n each row de vered sequent a y by the resu t set The same s not true w th an EntityDataReader because the data returned by an Ent ty SQL query can nc ude ­h erarch ca resu ts, n wh ch case ch d ent t es get unpacked dur ng the reader’s stream ng process Th s mposes the requ rement for you to retr eve nd v dua co umn va ues sequent a y—once you extract a co umn va ue from the current ent ty be ng returned by the reader, you w not be ab e to extract the va ue of any prev ous co umn for the same ent ty because the reader has a ready ­advanced past t Not know ng how many ent t es you’ get back, you test for an end-of-stream cond t on w th each ca to the reader’s Read method at the top of a while b ock If there are no ent t es, Read w return true the very first t me t s ca ed, and the while b ock w never execute Otherw se, the b ock w execute once for each customer ent ty returned by the query For each ent ty, the reader’s GetString method s used to extract the first and ast name ­va ues so you can format the customer name Th s po nts out another mportant d fference w th the ­SqlDataReader—co umn va ues can on y be extracted from an EntityDataReader by ord na ­pos t on us ng the GetXxx methods supp ed for each data type (such as GetString, GetDateTime, GetInt64, and so on) You cannot use ndexed notat on ke you can w th a SqlDataReader to access co umns by name (such as rdr[“FirstName”]) A so not ce that you extract the first name before the ast name, ­because they are returned by the query n that order as co umns 1 and 2 Even though you’re ­d sp ay ng ast name first, you must extract them n the order they are returned because of the ­CommandBehavior.SequentialAccess sett ng (as just exp a ned) Each formatted name s added to the List co ect on that got n t a zed at the top of the method before you created the EntityConnection After c os ng the reader (wh ch a so c oses the ­connect on because of the CommandBehavior.CloseConnection sett ng), you set the DataSource ­property on the lstCustomers st box to b nd the str ng co ect on to t Now run the app cat on and s ng e-step through t w th a SQL Profi er trace as usua Th s s a good demonstrat on of how EF can be used to execute quer es and process the r resu ts w thout nvo v ng objects or LINQ By now you’ve earned a great dea about the Ent ty Framework—more than enough to get started bu d ng rea app cat ons w th t You bu t a funct ona Ent ty Data Mode , and n the next chapter you w earn how to use WCF Data Serv ces and WCF RIA Serv ces to extend the reach of your EDMs to c ent app cat ons across the Internet



Chapter 10  The M crosoft Data Access Juggernaut   507

Summary Th s chapter has de vered a fa r y cr t ca ana ys s across the gamut of NET data access APIs and too s that M crosoft has re eased over more than the past decade, w th gu dance to he p you ­nte gent y d st ngu sh between them We wa ked through numerous examp es and demonstrated cod ng ­techn ques for work ng w th each of them, and a so prov ded mportant gu de nes to keep n m nd as you do We started by exp a n ng the cont nued re evance of convent ona ADO NET—both raw data access obects and DataSets—a ongs de newer techno og es You started w th raw connect ons, commands, readers, adapters, and DataSets to run d rect SQL statements and execute stored procedures n SQL Server You a so earned about exp c t and mp c t transact ons and SQL Server Profi er Then we ntroduced the concept of LINQ, and ts var ous mp ementat ons, and you saw how LINQ to DataSet extends convent ona ADO NET for query ng both gener c and strong y typed DataSets We then advanced the d scuss on to Object Re at ona Mapp ng, what ORM techno og es are, and the types of prob ems they are des gned to so ve We a so c ar fied M crosoft’s pos t on on the r two ORM offer ngs LINQ to SQL and ADO NET Ent ty Framework a though LINQ to SQL w cont nue ­be ng supported n future vers ons of NET, the Ent ty Framework s pos t oned as the current and future recommended data access so ut on for deve op ng app cat ons Then we dug nto the extens ve capab t es of the Ent ty Framework, start ng by exam n ng the Ent ty Data Mode and ts under y ng conceptua , storage, and mapp ng schemas From there, you bu t funct ona NET app cat ons us ng LINQ to Ent t es to query and update data n SQL Server, ­demonstrat ng powerfu key EF features such as azy oad ng, ent ty mapp ngs, change track ng, many-to-many re at onsh ps, Ent ty SQL, and Ent tyC ent But the NET data access story does not end here In the next chapter, you w earn about WCF Data Serv ces and WCF RIA Serv ces—two frameworks you can use to bu d data access serv ces and n-t er app cat ons for c ents over the Wor d W de Web

508  Part III  Applied SQL

C hapter 1 1

WCF Data Access Technologies –Leonard Lobel

W

ndows Commun cat on Foundat on (WCF) s a commun cat ons p atform that prov des a the support you need to bu d d str buted, serv ce-or ented so ut ons n the NET Framework ­M crosoft first ntroduced WCF n ate 2006 as part of the NET Framework 3 0 re ease S nce then, WCF has estab shed tse f as a cornerstone of the framework, and s regarded today as the de facto so ut on for bu d ng any k nd of serv ce n NET

Defining Services There are many types of serv ces Some are ntended for use on y w th n an organ zat on’s pr vate ntranet, where commun cat on occurs on y beh nd the corporate firewa (w th remote access enab ed v a V rtua Pr vate Network [VPN] connect ons) These serv ces can be t ght y coup ed w th NET ­c ents v a sockets and effic ent b nary remot ng protoco s Other serv ces may be exposed pub c y over the Internet Pub c serv ces must be ab e to penetrate firewa s and support any type of ­c ent, wh ch typ ca y requ res them to use eXtens b e Markup Language (XML) or the re at ve y terser ­JavaScr pt Object Notat on (JSON) message format over a standard transport (usua y HTTP) These text-based ser a zat on formats are bu k er and ess effic ent than b nary ser a zat on, but they do support a much w der range of c ents—M crosoft and non-M crosoft a ke And there are many more var ab es Some serv ces requ re authent cat on, whereas many do not—and those that do use some authent cat on methods but not others Some serv ces operate ­synchronous y, others asynchronous y Serv ces are usua y state ess, but some support c ent sess ons ­Messag ng ­protoco s vary as we ; some serv ces use S mp e Object Access Protoco (SOAP), whereas others are based on the Representat ona State Transfer (REST) protoco and return data as an Atom Pub sh ng Protoco (AtomPub) response (an XML d a ect very s m ar to the Rea y S mp y Synd cat on [RSS] feed format) Before WCF, such part cu ars were typ ca y hard-coded nto an app cat on If you bu t a pr vate serv ce us ng b nary remot ng and then dec ded to open up the serv ce to the pub c Internet, you needed to mod fy s gn ficant code n your app cat on to support that change Th s s not norma y the case w th WCF Typ ca y, you bu d your app cat on to commun cate us ng WCF proxy c asses, and configure WCF separate y to meet the requ rements of your part cu ar network ng env ronment A though WCF configurat on can be coded us ng C# or V sua Bas c (VB) NET (wh ch essent a y embeds t nto an assemb y), t s usua y (and best) managed dec arat ve y n the app cat on’s externa

509

c­ onfigurat on fi e (th s s e ther Web.config or App.config n your M crosoft V sua Stud o project) So, for nstance, to enab e pub c access to serv ces that were former y pr vate, you can s mp y open a text fi e n Notepad and add SOAP w th HTTP n add t on to the b nary ser a zat on w th s­ ockets ­configurat on that s a ready n p ace Or, you cou d just as eas y tweak the WCF configurat on to e ­ nab e asynchronous behav or for serv ce methods that former y supported on y synchronous ­behav or These are just a few examp es of the advantages offered by WCF’s configurat on-based ­approach

WCF Data Access Options You can certa n y work w th WCF d rect y to create custom serv ces and expose data from an Ent ty Data Mode (EDM) w th the ADO NET Ent ty Framework, or any other data access ayer (DAL) of your choos ng To take th s “raw” approach, you need to start w th the bas cs, or what s common y referred to as the ABC’s of WCF Addresses, B nd ngs, and Contracts You must create serv ce, operat on, and data contracts, and then configure your serv ce mode w th appropr ate endpo nt addresses and compat b e bindings to be reachab e by c ents Serv ces are usua y state ess, so you must a so hand e c ent-s de change track ng and mu t -user confl ct reso ut on ent re y on your own The earn ng curve can be qu te steep, after wh ch you w st need to expend a great dea of effort to make t work A ternat ve y, you can turn to one of the two ater techno og es that M crosoft has bu t on top of WCF, wh ch are the focus of th s chapter These are WCF Data Serv ces and WCF RIA Serv ces, and they represent two very d fferent approaches for bu d ng data-or ented serv ces Both prov de abstract ons that sh e d you from many under y ng WCF part cu ars, so you get to spend more t me focus ng on your app cat on and ess t me on p umb ng For one th ng, you don’t need to code WCF contracts or manage change track ng on the c ent; a that gets done for you W th WCF RIA Serv ces (and M crosoft S ver ght), you don’t even need to create and update serv ce references; V sua Stud o generates code automat ca y v a a spec a nk that keeps your c ent and WCF RIA Serv ces projects n sync at a t mes In th s chapter, you w earn how to use WCF Data Serv ces and WCF RIA Serv ces by expos ng the SampleDb database’s EDM that you created n the prev ous chapter as serv ces, and then bu d ng c ents over those serv ces By deta ed compar son, you w earn how to choose nte gent y between these two frameworks for any g ven scenar o

Monitoring Network Activity with Fiddler In Chapter 10, you used SQL Profi er to mon tor the database act v ty that Ent ty Framework was conduct ng beh nd the scenes Now that you are extend ng the c ent-server mode to an n-t er serv ce-or ented arch tecture, you w s m ar y want to mon tor the network ­act v ty ­further up the stack (HTTP traffic) as you bu d separate c ent and server ­components A ­number of too s are ava ab e that w et you do th s (so-ca ed “packet sn ffers”) In our ­wa kthrough, we use F dd er (wh ch s sma , free, and easy to use) to watch and see exact y how c ent requests and serv ce responses are be ng transm tted to and from WCF serv ces on the

510  Part III  Applied SQL

other end of the w re Th s s an nva uab e too for earn ng the techno ogy as you exper ment w th the examp es n th s chapter And once you start bu d ng product on systems w th the techno ogy, t w a so prove to be an nd spensab e too for rout ne debugg ng—whether you are programm ng raw WCF, WCF Data Serv ces, or WCF RIA Serv ces Go to http://www.fiddler2.com to down oad and nsta F dd er When you start the too , t d sp ays a sp t screen w th requests on top and responses on bottom, and then mmed ate y beg ns to mon tor HTTP traffic (see F gure 11-8) You w refer to F dd er output rout ne y to observe c ent/server commun cat on as you test and debug the code n th s chapter You shou d a so beg n a SQL Profi er trace so that you can mon tor the database act v ty of your WCF serv ces as we ( nstruct ons for start ng a trace can be found n Chapter 10 under the ­sect on “Mon tor ng Database Act v ty w th SQL Server Profi er“)

WCF Data Services M crosoft des gned WCF Data Serv ces as a th n ayer over Ent ty Framework that exposes data-centr c serv ces to c ent app cat ons across the Internet You can th nk of WCF Data Serv ces as un versa Web Serv ces bu t just for data The p atform s based on open ndustry standards and protoco s, wh ch means that these serv ces are consumab e by v rtua y every type of c ent n the wor d And you can qu ck y bu d a WCF Data Serv ces over an EDM w th a most no effort whatsoever Before bu d ng your very first serv ce (and a correspond ng c ent p ece), et’s first exp a n the ndustry standards that WCF Data Serv ces s based upon—wh ch are REST and the Open Data Protoco (OData) REST prov des a un form nterface for query ng and updat ng data It s based on HTTP, ­mean ng that c ent requests are ssued n the form of GET, POST, MERGE, and DELETE act ons—standard verbs understood by a HTTP c ents Any REST query can be nvoked w th an HTTP GET request by ­express ng a the e ements of the query n a proper y formed Un form Resource Ident fier (a URI, wh ch s a more genera term than Un form Resource Locator [URL]) You can even test the serv ce w th an ord nary browser by typ ng the URI d rect y nto the address bar The POST, MERGE, and ­DELETE verbs correspond respect ve y to nsert, update, and de ete operat ons supported by the serv ce Un ke GET, the pay oad (parameters, data, and other metadata) for these operat ons s passed n HTTP headers; t s not embedded n the URI So you cannot use the browser’s address bar to nsert, update, or de ete data However, t s easy to v ew HTTP headers us ng F dd er, and you w be do ng that short y to nspect the comp ete pay oad for each c ent request ssued to the serv ce Just as REST enab es un versa data access v a HTTP, OData estab shes un versa data structure v a standard ser a zat on formats (see http://www.odata.org) A c ents can hand e p a n-text formats such as JSON and XML, and so OData defines standard response formats based on both formats JSON prov des a compact structure su tab e for many bas c types of serv ces, whereas XML forms the bas s for the more verbose AtomPub feed format AtomPub s the defau t ser a zat on format n WCF Data Serv ces, because t effect ve y everages the h erarch ca nature of XML to descr be the r ch structure of data and metadata n an EDM

Chapter 11  WCF Data Access Techno og es   511

WCF Data Services without Entity Framework M crosoft des gned WCF Data Serv ces w th a data source prov der ta ored espec a y for the Ent ty Framework, mak ng t v rtua y effort ess to create serv ces over any Ent ty Data Mode Th s s rat ona , g ven that Ent ty Framework s M crosoft’s preferred NET data access so ut on Nonethe ess, there are a ternat ve data source prov ders ava ab e to make WCF Data Serv ces work w th other data access ayers as we Th s approach requ res substant a add t ona effort, deta s of wh ch are beyond the scope of th s chapter Here s a br ef ment on of the a ternat ves to gu de you n the r ght d rect on f you need to earn more M crosoft supp es two data source prov ders that you can use to expose data sources other than Ent ty Framework w th WCF Data Serv ces These are the Reflect on and Stream ng ­prov ders To use the Reflect on prov der, your data c asses mp ement IQueryable to support query ng, and ­opt ona y mp ement IUpdateable to support updates as we You can map these c asses to memory-res dent objects or any back-end data store that you want—even LINQ to SQL or typed DataSets The Stream ng prov der g ves you a spec a IDataServiceStreamProvider nterface that you can mp ement to expose arge b nary objects (BLOBs), such as FILESTREAM data n SQL Server (wh ch we cover n Chapter 8) F na y, custom serv ce prov ders et you dynam ca y define data mode s of any type Th s approach requ res the most effort to mp ement, and shou d be used on y f the other ava ab e prov ders are nadequate For more nformat on about prov ders for WCF Data Serv ces, v s t http://msdn.microsoft.com/en-us/library/dd672591.aspx

Building a WCF Data Service To bu d the serv ce, fo ow these steps Start V sua Stud o and choose F e New Project From the st of temp ate categor es on the eft, beneath V sua C#, choose Web Then se ect the ASP NET Empty Web App cat on temp ate to create a project named DemoRestService n a so ut on named DemoWcfDataServices, as shown n F gure 11-1

Figure 11-1  Creat ng an ASP.NET Web App cat on project for the WCF Data Serv ce.

512  Part III  Applied SQL

Every ASP NET app cat on s hosted on a web server, typ ca y Internet Informat on Serv ces (IIS) n product on env ronments, or the more ghtwe ght V sua Stud o Deve opment Web Server (a so known as “Cass n ”) on deve opment desktops and aptops But rather than serv ng up ord nary ­webpages, th s project w expose a spec a svc fi e that exposes an EDM us ng WCF Data Serv ces, ­us ng the same SampleDb EDM (and under y ng database) from the prev ous chapter

Creating the Entity Data Model The next step s to create the EDM (If you sk pped the prev ous chapter, you first need to create the ­database by runn ng the T-SQL scr pt n L st ng 10-1 of that chapter ) Then, you can repeat the same steps n the prev ous chapter (see the sect on “Bu d ng an Ent ty Data Mode “) to create the same EDM n th s project, or— f you have a ready bu t the EDM n the prev ous chapter— t s much eas er to s mp y copy t (and ts connect on str ng) from the prev ous chapter’s project To do so, open W ndows Exp orer to the prev ous chapter’s project, drag the SampleEF.edmx fi e, and drop t on the DemoRestService ­project n ­So ut on Exp orer Then copy the connectionStrings sect on n the App.config fi e from prev ous chapter’s project and paste t nto th s DemoRestService project’s Web.config fi e (just after the open ng ­ tag toward the top)



Note  The complete code—including the T-SQL script and a Visual Studio solution with the EDM already built—is available for download from the book’s companion website (see the “Introduction“ for details). Your EDM s now n p ace If you have been fo ow ng a ong w th the prev ous chapter, the EDM’s under y ng database now conta ns random samp e data from ear er exerc ses; f you have just created the database now, then t s empty In e ther case, refresh the database now w th new samp e data by runn ng the scr pt n L st ng 11-1 Listing 11-1  Popu at ng the database w th samp e data.

USE SampleDb GO DELETE FROM OrderHeader DELETE FROM CustomerEmployee DELETE FROM Customer



Chapter 11  WCF Data Access Techno og es   513

SET IDENTITY_INSERT Customer ON INSERT INTO Customer (CustomerId, FirstName, LastName, Balance) VALUES (1, 'Lukas', 'Keller', 35), (2, 'Andy', 'Jacobs', 10), (3, 'Mike', 'Ray', 40), (4, 'Josh', 'Barnhill', 5) SET IDENTITY_INSERT Customer OFF SET IDENTITY_INSERT OrderHeader ON INSERT INTO OrderHeader (OrderHeaderId, CustomerId, ShipVia, OrderStatus) VALUES (1, 1, 'Regular Mail', 'Shipped'), (2, 1, 'Express Mail', 'Pending'), (3, 2, 'Priority Mail', 'Shipped'), (4, 3, 'Priority Mail', 'Shipped'), (5, 3, 'Regular Mail', 'Cancelled'), (6, 3, 'Priority Mail', 'Shipped'), (7, 3, 'Express Mail', 'Pending'), (8, 4, 'Regular Mail', 'Shipped'), (9, 4, 'Express Mail', 'Shipped') SET IDENTITY_INSERT OrderHeader OFF

A that’s eft to mp ement the serv ce s to create an assoc ated svc fi e, wh ch w nstant y expose your EDM to any REST c ent R ght-c ck the DemoRestService project, choose Add New Item, and se ect Web from the st of temp ate categor es on the eft Then scro down the st of nsta ed temp ates and choose WCF Data Serv ce Name the fi e CustomerDataService.svc and then c ck Add, as shown n F gure 11-2

Figure 11-2  Creat ng an ASP.NET Web App cat on project for the WCF Data Serv ce.

Now rep ace the starter code that V sua Stud o creates by defau t w th the code shown n L st ng 11-2

514  Part III  Applied SQL

Listing 11-2  A s mp e WCF Data Serv ce.

using using using using using

System; System.Data.Services; System.Data.Services.Common; System.Linq; System.ServiceModel.Web;

namespace DemoRestService { public class CustomerDataService : DataService { public static void InitializeService(DataServiceConfiguration config) { config.SetEntitySetAccessRule("Customers", EntitySetRights.All); config.SetEntitySetAccessRule("OrderHeaders", EntitySetRights.All); config.SetEntitySetAccessRule("Employees", EntitySetRights.AllRead); config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2; } } }

Th s s a the code you need for create a fu y funct ona serv ce Any c ent can ca th s serv ce to query and update your EDM us ng REST and OData It s st WCF under the covers, but you d d not have to wr te any server-s de og c, create a s ng e WCF contract, or tweak a s ng e WCF configurat on sett ng—wh ch s qu te remarkab e As the code demonstrates, you need to do on y two s mp e th ngs F rst, nher t from DataService, where T s your Ent ty Data Mode ’s object context c ass (SampleDbEntities, n our ­current examp e) Second, mp ement the InitializeService method to contro wh ch parts of the EDM (that s, wh ch ent ty sets) are to be exposed by the serv ce, and what access eve s are perm tted (read-on y versus updateab e) In our examp e, a three ent ty sets n the EDM can be quer ed, but on y Customers and OrderHeaders are updateab e

Testing WCF Data Services with Internet Explorer Even a p a n web browser can supp y a proper y formatted URI to query th s serv ce and d sp ay the XML content of the AtomPub feed that gets returned n response Th s s a qu ck and easy way to test that your WCF Data Serv ces are work ng proper y before wr t ng even one ne of c ent code Try t out now by us ng Internet Exp orer 9 to ssue a few URI quer es to the serv ce



Chapter 11  WCF Data Access Techno og es   515

Important  These instructions work specifically with Internet Explorer; other browsers may exhibit different or undesirable behavior. If you have another browser set as your default, open up Internet Explorer explicitly and copy the URL into its address bar after your default browser opens. We also recommend 32-bit Internet Explorer over the 64-bit version, as the 64-bit Internet Explorer apparently crashes consistently in our test environment when ­displaying RSS feeds. R ght-c ck the CustomerDataService.svc fi e n So ut on Exp orer and choose V ew In Browser Th s mmed ate y starts the ASP NET Deve opment Web Server that hosts the serv ce on a random y ­ass gned port (1055 n th s examp e, but yours w be d fferent) and then aunches Internet Exp orer The browser nav gates to the svc fi e, and the serv ce responds w th the st of ava ab e ent ty sets As shown n F gure 11-3, these are the Customers, Employees, and OrderHeaders ent ty sets spec fied n the InitializeService method (L st ng 11-2)

Figure 11-3  Nav gate to the serv ce s .svc fi e to d scover the ava ab e ent ty sets.

Based on the “d scovery” of ava ab e ent ty sets, you can append any ent ty set name to the URI to request a data n that ent ty set to be returned For examp e, to v ew a the customers s mp y append Customers to the URL n the browser’s address bar The comp ete URI (aga n, w th a d fferent random y ass gned port number than 1055) s http://localhost:1055/CustomerDataService.svc/Customers

Go ahead and browse to th s URL Instead of d sp ay ng the raw AtomPub response feed, the feed ­read ng v ew (wh ch s on by defau t) n Internet Exp orer attempts unsuccessfu y to format the feed for d sp ay To v ew the AtomPub feed n ts raw XML form nstead, you need to turn off feed ­read ng v ew Open the Internet Opt ons d a og n Internet Exp orer, c ck the Content tab, and then c ck the Sett ngs button under Feeds and Web S ces to d sp ay the Feeds and Web S ce Sett ngs d a og Then uncheck Turn On Feed Read ng V ew, as shown n F gure 11-4, and c ck OK tw ce to d sm ss both d a ogs 516  Part III  Applied SQL

Figure 11-4  Turn off feed read ng v ew n nternet Exp orer to v ew the raw XML content of AtomPub responses.

Th s part cu ar change may not take effect unt Internet Exp orer s restarted So c ose the browser now and then once aga n r ght-c ck the CustomerDataService.svc fi e n So ut on Exp orer, choose V ew In Browser, and append Customers to the URL Th s t me Internet Exp orer d sp ays the ­AtomPub feed n readab e XML, and you can see that the URI query returns a customers n the ­database, as shown n F gure 11-5

Figure 11-5  Customer data returned as an AtomPub feed by WCF Data Serv ces.



Chapter 11  WCF Data Access Techno og es   517

To return a s ng e ent ty by pr mary key, append ts un que ID n parentheses after the ent ty set name For examp e, mod fy the URL as fo ows to return the s ng e customer w th a CustomerId va ue of 3 (M ke Ray) http://localhost:1055/CustomerDataService.svc/Customers(3L)

Not ce that the term nat ng L s requ red to convey the data type ( ong nteger, correspond ng to SQL Server’s 64-b t bigint type) of the pr mary key va ue The syntax can qu ck y grow contorted, as ad-hoc fi ter ng, sort ng, and a host of other d rect ves are supported as we For examp e, cons der the fo ow ng URI http://localhost:1055/CustomerDataService.svc/Customers?$filter=Balance gt 10&$expand=OrderHeade rs&$orderby=LastName

The $filter opt on n th s URI quer es for a customers w th a ba ance h gher than 10 The $expand opt on te s WCF Data Serv ces to return a the re ated OrderHeader ent t es w th each Customer ­ent ty n the response feed, and $orderby requests that the resu ts n the response shou d be sorted by LastName If you enter th s URI nto the browser’s address bar, the serv ce returns the two ­customers that meet the fi ter cr ter a Lukas Ke er and M ke Ray (hav ng ba ances of 35 and 40, respect ve y)

Note  An OData URI always resolves to a specific resource or set of resources. Taking this approach is often known as implementing resource-based services, in contrast with more traditional domain-oriented services based on operation methods. As you can see, you need to know the prec se OData convent ons to construct mean ngfu URI quer es (v s t http://www.odata.org/documentation/uri-conventions for the comp ete OData URI ­spec ficat on) Fortunate y, the WCF Data Serv ces c ent brar es for NET ( nc ud ng S ver ght and W ndows Phone 7) supp y a spec a LINQ prov der for th s purpose, common y known as LINQ to REST Th s prov der automat ca y trans ates c ent-s de LINQ quer es nto an equ va ent OData URI, mean ng that you rea y don’t need to earn the OData URI syntax f you are bu d ng M crosoft c ents over WCF Data Serv ces Th s s a huge benefit because, once aga n, you are us ng LINQ to avo d earn ng yet another query ng anguage (wh ch s a key object ve of LINQ)

Building Client Applications for WCF Data Services M crosoft prov des spec a brar es for desktop (W ndows and W ndows Presentat on Foundat on [WPF]), web (ASP NET and S ver ght), and W ndows Phone 7 c ent deve opment aga nst WCF Data Serv ces These brar es support LINQ to REST, wh ch converts your c ent-s de LINQ quer es nto the OData URI syntax expected by the serv ce They a so a prov de a statefu context object for track ng changes on the c ent and push ng updates back to the serv ce In th s sect on, you w create two W ndows c ents that consume the serv ce you bu t n the ­ rev ous sect on The first s a test c ent that demonstrates how LINQ to REST, OData, and change p track ng features are mp emented by the WCF Data Serv ces c ent brary Th s bare samp e has 518  Part III  Applied SQL

no user nterface ( n fact, we may just as we have mp emented t as a conso e app cat on), and s des gned to teach you how to code aga nst the WCF Data Serv ces c ent brar es You w then app y that know edge to bu d a more robust, nteract ve data entry c ent that supports h erarch ca CRUD (create, retr eve, update, and de ete) operat ons

Building a Test Client Let’s get started w th the test c ent R ght-c ck the DemoWcfDataServices so ut on n So ut on ­Exp orer and choose Add New Project From the st of temp ate categor es on the eft, beneath V sua C#, choose W ndows Then se ect the W ndows Forms App cat on temp ate to create a new project named DemoRestClientTest, as shown n F gure 11-6

Figure 11-6  Add ng a new W ndows Forms c ent app cat on to the WCF Data Serv ces demo so ut on.

The DemoWcfDataServices so ut on now has two projects n t an ASP NET project (the serv ce) and a W ndows Forms project (the new DemoRestClientTest project w th an empty form) Your next step s to set a reference from the c ent project to the serv ce project To do th s, r ght-c ck the ­DemoRestClientTest project n So ut on Exp orer and choose Add Serv ce Reference In the Add Serv ce Reference d a og, c ck the D scover button V sua Stud o ocates the WCF Data Serv ces fi e ­CustomerDataServices.svc n the ASP NET project Expand the treev ew on the eft to revea the ­ent t es exposed by the serv ce Then type CustomerDataService for the Namespace at the bottom of the d a og, as shown n F gure 11-7 C ck OK to estab sh the WCF Data Serv ces reference These are the same steps you wou d perform for sett ng a reference to an ord nary WCF serv ce And as w th an ord nary WCF serv ce, V sua Stud o creates a c ent-s de proxy w th strong y typed data c asses that correspond to data contracts (ent ty types) exposed by the serv ce, a ong w th the p umb ng that ser a zes and deser a zes ent t es as they are sent and rece ved across the w re But a WCF Data Serv ces c ent proxy goes even further— t a so supports LINQ to REST, OData, and change track ng, as th s c ent app cat on w demonstrate

Chapter 11  WCF Data Access Techno og es   519

Figure 11-7  Creat ng a serv ce reference from a W ndows c ent project to a WCF Data Serv ces project.

Drag a button from the Too box, drop t on to the form, and name t btnTestClient Then doub ec ck the button V sua Stud o creates a hand er for the button’s Click event and opens a w ndow to the form’s code, wh ch you shou d comp ete y rep ace w th the code shown n L st ng 11-3 (remember to change the port number n the ServiceUri constant defin t on toward the top of the code w th the port number for your env ronment) Listing 11-3  A s mp e WCF Data Serv ces c ent.

using System; using System.Linq; using System.Windows.Forms; using DemoRestClientTest.CustomerDataService; namespace DemoRestClientTest { public partial class Form1 : Form { public Form1() { InitializeComponent(); } // - The "." after "localhost" is a hack to enable Fiddler packet-sniffing // - Change the port number accordingly for your dev machine public const string ServiceUri = "http://localhost.:1055/CustomerDataService.svc"; private void btnTestClient_Click(object sender, EventArgs e) {

520  Part III  Applied SQL

// Create the client context var ctx = new SampleDbEntities(new Uri(ServiceUri)); // Construct an OData URI query var q = from cust in ctx.Customers.Expand("OrderHeaders") where !cust.FirstName.StartsWith("A") orderby cust.LastName descending select cust; // Call the service (issues an HTTP GET) var customers = q.ToList(); if (customers.Count == 0) { MessageBox.Show("There are no matching customers"); return; } var firstCustomer = customers[0]; var orderCount = firstCustomer.OrderHeaders.Count; MessageBox.Show(string.Format( "Customer {0} has {1} order(s)", firstCustomer.CustomerId, orderCount)); // Change the customer's name firstCustomer.FirstName = "Cassie"; firstCustomer.LastName = "Hicks"; // Add a customer order var newOrder = new OrderHeader() { ShipVia = "Regular Mail", OrderStatus = "Open", }; firstCustomer.OrderHeaders.Add(newOrder); ctx.UpdateObject(firstCustomer); ctx.AddRelatedObject(firstCustomer, "OrderHeaders", newOrder); // Issues an HTTP MERGE to update the customer's name, // and an HTTP POST to INSERT the customer's new order ctx.SaveChanges(); } } }

You w earn exact y how th s c ent code works by s ng e-stepp ng through t w th the debugger, as you keep an eye on background HTTP conversat on w th F dd er (and back-end database ­act v ty w th SQL Profi er) Set a breakpo nt at the very top of the button’s c ck event hand er by c ck ng n the eft marg n area on the ne private void btnTestClient Click(object sender, EventArgs e). V sua ­Stud o d sp ays a red bu et n the marg n, nd cat ng that the breakpo nt s set



Chapter 11  WCF Data Access Techno og es   521

You are now ready to run the so ut on R ght-c ck DemoRestClientTest n So ut on Exp orer, choose Set As Startup Project, and press F5 V sua Stud o bu ds the so ut on and aunches the c ent ­W ndows form C ck the button on the form and you w h t the breakpo nt you just set n the event hand er Before proceed ng from th s po nt, make sure you have both F dd er and SQL Profi er runn ng ( nstruct ons for start ng F dd er are g ven n the “Mon tor ng Network Act v ty w th F dd er“ s debar ear er n th s chapter; steps for runn ng a SQL Profi er trace are prov ded n Chapter 10) Use the debugger to s ng e-step over the first ne of code (F10 s the defau t keystroke n V sua Stud o for s ng e-stepp ng) Th s ne creates the c ent-s de context and stores t n the var ab e ctx var ctx = new SampleDbEntities(new Uri(ServiceUri));

The context c ass s named SampleDbEntities, and was generated automat ca y when you created the serv ce reference V sua Stud o generated th s c ass n a namespace der ved by append ng the name CustomerDataService that you spec fied n F gure 11-7 (when creat ng the serv ce reference) to the project name DemoRestClientTest By nc ud ng a using DemoRestClientTest.CustomerDataService statement at the very top of the st ng, the comp er s ab e to recogn ze SampleDbEntities as the ­c ent context c ass generated by the serv ce reference The name of the context c ass, SampleDbEntities, s based on the EDM exposed by the serv ce But the c ass has noth ng to do w th Ent ty Framework— t runs ent re y on the c ent Spec fica y, SampleDbEntities on the c ent s a statefu context object that der ves from DataServiceContext, and t prov des strong y typed propert es based on the SampleDbEntities EDM d scovered when the ­serv ce reference s created (or updated) Th s context s capab e of query ng ent t es from the serv ce, ­track ng c ent-s de changes, and push ng updates back to the serv ce Not ce the ServiceUri passed n to the SampleDbEntities constructor Th s s a str ng constant that po nts to the serv ce, and s defined a b t further up n the code (just before the event hand er) as ­fo ows // - Change the port number accordingly for your dev machine // - The "." after "localhost" is a hack to enable Fiddler packet-sniffing public const string ServiceUri = "http://localhost.:1055/CustomerDataService.svc";

As we’ve a ready ment oned, and as nd cated n the comments, you need to change the port number from 1055 to whatever port number s random y ass gned by the V sua Stud o Deve opment Web Server on your mach ne A ternat ve y, you cou d eave the code as- s and exp c t y set port 1055 n the Web tab of the serv ce project’s propert es page In e ther case, make sure to nc ude the “dot” after localhost or F dd er w not mon tor HTTP traffic as the code executes Now step over the next ne Th s s a strong y typed LINQ query that fi ters by first name, sorts by ast name, and returns match ng customers a ong w th the r re ated orders var q = from cust in ctx.Customers.Expand("OrderHeaders") where !cust.FirstName.StartsWith("A") orderby cust.LastName descending select cust;

522  Part III  Applied SQL

The query source ctx s a DataServiceContext (that s, a WCF Data Serv ces c ent context) It ­ rov des strong y typed propert es based on the SampleDbEntities EDM exposed by the serv ce, such p as the Customers ent ty set and the FirstName and LastName propert es of the Customer ent ty n th s examp e (unfortunate y, the Expand method uses a str ng tera rather than strong typ ng to reference the OrderHeaders nav gat on property) Th s s a LINQ to REST query, wh ch may not be mmed ate y obv ous, but s c ear y ev dent the moment you hover over the q The too t p that appears shows that the LINQ query has just been transformed dynam ca y at runt me nto the fo ow ng equ va ent OData URI query str ng http://localhost.:1055/CustomerDataService.svc/Customers()?$filter=not startswith(FirstName,'A') &$orderby=LastName desc&$expand=OrderHeaders

At th s po nt, the query has been defined, but t has not been nvoked (the HTTP GET operat on has not yet been ssued) Both F dd er and SQL Profi er show that no act v ty has yet occurred beh nd the scenes Stepp ng over the next ne of code actua y executes the query var customers = q.ToList();

In genera LINQ terms, th s ne of code s mp y means “execute the query and return a st of objects ” For the LINQ to REST prov der n th s part cu ar examp e, t means “ ssue a GET request to a REST serv ce w th an OData URI query based on the LINQ query, and deser a ze the AtomPub feed returned by the serv ce nto a st of strong y typed Customer ent t es ” That’s qu te a ot to accomp sh w th just a s ng e ne of code, yet the output from F dd er and SQL Profi er confirms that th s s exact y what occurs F rst ook at F dd er Se ect the ast HTTP request n the st on the eft (the one made to ­CustomerDataService.svc) Then c ck the Inspectors tab to v ew the request and response n a sp t pane d sp ay C ck the Raw button at the top of the request and response pane s to read them n p a n text, as shown n F gure 11-8

Figure 11-8  Us ng F dd er to v ew the WCF Data Serv ces conversat on over HTTP.



Chapter 11  WCF Data Access Techno og es   523

The request pane at the top shows the GET request ssued by the c ent, w th the OData URI query str ng generated from the LINQ query Beneath, the response pane shows the AtomPub feed resu t w th a of the ent t es returned by the query (not ce the application/atom+xml va ue returned by Content-Type n the HTTP response header) The WCF Data Serv ces c ent brary ­automat ca y ­deser a zes th s response nto a st of Customer ent t es (each of wh ch nc udes OrderHeader ­ent t es popu ated n the OrderHeaders co ect on) So the very next ne of c ent code s ab e to work ­mmed ate y w th objects returned by the serv ce Of course, the serv ce s ts n front of an EDM managed by Ent ty Framework, wh ch n turn c­ ommun cates w th SQL Server (see Chapter 10 for deta ed coverage of EF and the EDM) When the serv ce rece ves the URI query, t runs an equ va ent query aga nst the EDM, wh ch generates the ­actua T-SQL statements that u t mate y h t the database That’s a ot of ayers, so et’s spe t out c ear y once more For the request ■

The c ent defines a LINQ query

■

The WCF Data Serv ces c ent brary converts the LINQ query nto OData URI syntax

■

An HTTP GET s ssued to WCF Data Serv ces w th an OData URI query str ng

■

WCF Data Serv ces generates an equ va ent query aga nst the EDM

■

Ent ty Framework generates the requ red T-SQL statement to query the database

■

The T-SQL statement s executed n SQL Server

Now sw tch over to SQL Profi er As shown n the SQL Profi er trace, EF generates and executes a d rect T-SQL statement that sat sfies the EDM query SELECT [Project1].[C1] AS [C1], [Project1].[CustomerId] AS [CustomerId], [Project1].[FirstName] AS [FirstName], [Project1].[LastName] AS [LastName], [Project1].[Balance] AS [Balance], [Project1].[CreatedAt] AS [CreatedAt], [Project1].[UpdatedAt] AS [UpdatedAt], [Project1].[C2] AS [C2], [Project1].[C3] AS [C3], [Project1].[OrderHeaderId] AS [OrderHeaderId], [Project1].[CustomerId1] AS [CustomerId1], [Project1].[ShipVia] AS [ShipVia], [Project1].[OrderStatus] AS [OrderStatus], [Project1].[Notes] AS [Notes], [Project1].[CreatedAt1] AS [CreatedAt1], [Project1].[UpdatedAt1] AS [UpdatedAt1] FROM ( SELECT [Extent1].[CustomerId] AS [CustomerId], [Extent1].[FirstName] AS [FirstName], [Extent1].[LastName] AS [LastName], [Extent1].[Balance] AS [Balance], [Extent1].[CreatedAt] AS [CreatedAt], [Extent1].[UpdatedAt] AS [UpdatedAt],

524  Part III  Applied SQL

1 AS [C1], N'OrderHeaders' AS [C2], [Extent2].[OrderHeaderId] AS [OrderHeaderId], [Extent2].[CustomerId] AS [CustomerId1], [Extent2].[ShipVia] AS [ShipVia], [Extent2].[OrderStatus] AS [OrderStatus], [Extent2].[Notes] AS [Notes], [Extent2].[CreatedAt] AS [CreatedAt1], [Extent2].[UpdatedAt] AS [UpdatedAt1], CASE WHEN ([Extent2].[OrderHeaderId] IS NULL) THEN CAST(NULL AS int) ELSE 1 END AS [C3] FROM [dbo].[Customer] AS [Extent1] LEFT OUTER JOIN [dbo].[OrderHeader] AS [Extent2] ON [Extent1].[CustomerId] = [Extent2].[CustomerId] WHERE NOT ([Extent1].[FirstName] LIKE 'A%') ) AS [Project1] ORDER BY [Project1].[LastName] DESC, [Project1].[CustomerId] ASC, [Project1].[C3] ASC

Not ce how th s query jo ns Customer w th OrderHeader, wh ch returns re ated order data w th each se ected customer Th s s because of the c ent-s de LINQ query spec fied the Expand method When ca ng the serv ce, the LINQ to REST prov der trans ated Expand(“OrderHeaders”) nto the equ va ent OData URI syntax $expand=OrderHeaders, wh ch EF then trans ated nto the ­appropr ate T-SQL jo n After th s T-SQL query executes n SQL Server ■

The resu ts are returned to Ent ty Framework

■

EF mater a zes the resu t set nto conceptua ent t es

■

WCF Data Serv ces ser a zes the conceptua ent t es nto AtomPub format

■

The AtomPub response feed s sent back to the c ent

■

The WCF Data Serv ces c ent brary deser a zes the AtomPub response feed nto objects

Based on the samp e data that you popu ated the database w th ( n L st ng 11-1), th s query returns the three customers whose first names do not beg n w th the etter A (a ong w th the r orders) to the ­c ent At th s t me, the c ent-s de context s aware of three Customer ent t es and the r re ated ­OrderHeader ent t es

Note  WCF Data Services can be called either synchronously (blocking the client from ­executing while waiting for a response to each request) or asynchronously (­without ­blocking). Some clients, notably Silverlight and Windows Phone 7, support only ­asynchronous service calls. This Windows desktop application is working ­synchronously, which is a bit simpler to code (and delivers a less responsive user experience) than ­asynchronous communication. Later in this chapter, you will create a Silverlight client with WCF RIA Services that works asynchronously, and in Chapter 13, you will build a Windows Phone 7 OData client that makes asynchronous calls to WCF Data Services hosted in the cloud on Windows Azure.



Chapter 11  WCF Data Access Techno og es   525

Cont nue s ng e-stepp ng After first ensur ng that there s at east one customer n the st returned by the serv ce, the code extracts the first customer from the st and d sp ays the customer ID and the number of orders for that customer var firstCustomer = customers[0]; var orderCount = firstCustomer.OrderHeaders.Count; MessageBox.Show(string.Format( "Customer {0} has {1} order(s)", firstCustomer.CustomerId, orderCount));

Next, the code mod fies the customer’s name and adds another order for them S ng e-step through t now // Change the customer's name firstCustomer.FirstName = "Cassie"; firstCustomer.LastName = "Hicks"; // Add a customer order var newOrder = new OrderHeader() { ShipVia = "Regular Mail", OrderStatus = "Open", }; firstCustomer.OrderHeaders.Add(newOrder);

Not ce how th s s the same cod ng exper ence as server-s de Ent ty Framework, because there s a v rtua y dent ca data mode defined on the c ent Of course these c ent-s de ent t es may ook and fee ke EF ent t es, but they most certa n y are not The c ent has no awareness of EF’s ObjectContext (or any other data source that may s t beh nd the serv ce); t just works b ssfu y w th oca objects that are automat ca y sent back and forth between tse f and the WCF Data Serv ces The next two nes of code nform the c ent context of the changes made to the customer ctx.UpdateObject(firstCustomer); ctx.AddRelatedObject(firstCustomer, "OrderHeaders", newOrder);

The UpdateObject method marks the Customer ent ty (w th updated FirstName and LastName propert es) as “mod fied,” and the AddRelatedObject method marks the new OrderHeader ent ty added to the customer’s OrderHeaders co ect on as “new ” Th s step s essent a —you must use these context methods to exp c t y mark ent t es as changed, or the WCF Data Serv ces c ent brary w not track them as such S ng e-step through th s code, stopp ng just before the ast ne n the method that ca s SaveChanges ctx.SaveChanges();

Before execut ng the SaveChanges method, take a qu ck ook over at F dd er and SQL Profi er They show that no act v ty (from th s app cat on) has transp red s nce the query that n t a y retr eved the three customers Th s s expected—a changes made thus far have occurred ent re y on the c ent, where those changes are be ng and tracked by the WCF Data Serv ces c ent context The SaveChanges method ssues the appropr ate HTTP requests to push a the buffered up changes on the c ent back to the serv ce and

526  Part III  Applied SQL

update the database Step over the SaveChanges method ca now, and then sw tch r ght back to F dd er and Profi er to see the resu t In F dd er, you can see that two requests were sent to the serv ce The first s an HTTP MERGE that updates the customer w th the name change, and the second s an HTTP POST that adds the new order Let’s exam ne these requests (and the r responses) n deta , start ng w th the MERGE request for the update shown n L st ng 11-4 Listing 11-4  An HTTP MERGE request that updates a Customer ent ty.

MERGE http://localhost.:1055/CustomerDataService.svc/Customers(3L) HTTP/1.1 User-Agent: Microsoft ADO.NET Data Services DataServiceVersion: 1.0;NetFx MaxDataServiceVersion: 2.0;NetFx Accept: application/atom+xml,application/xml Accept-Charset: UTF-8 Content-Type: application/atom+xml If-Match: W/"datetime'2012-02-01T12%3A32%3A18.2991875'" Host: localhost.:1055 Content-Length: 991 Expect: 100-continue





2012-02-01T19:51:38.2358239Z http://localhost.:1055/CustomerDataService.svc/Customers(3L)

40.0000 2012-02-01T12:32:18.2991875 3 Cassie Hicks 2012-02-01T12:32:18.2991875



The MERGE (update) request s made w th an OData URI that spec fies the customer to be updated as Customers(3L), fo owed by severa HTTP headers and the updated data tse f Interest ng y, the User-Agent header (wh ch dent fies the HTTP c ent mak ng the request) st reads Microsoft ADO.NET Data Services (the o d name for WCF Data Serv ces) Fortunate y, the serv ce does not m nd the c ent dent fy ng tse f

Chapter 11  WCF Data Access Techno og es   527

w th the outdated brand name and accepts the request anyway The If-Match header (a few nes down) s far more noteworthy and s gn ficant Reca from Chapter 10 that the Customer ent ty n the EDM s mapped to an UpdateCustomer stored procedure, and that UpdateCustomer expects the or g na va ue of the UpdatedAt co umn to mp ement a concurrency check aga nst mu t -user confl cts The WCF Data Serv ces c ent brary detected th s or g na va ue mapp ng n the EDM when the serv ce reference was ­estab shed, so t s aware that the or g na va ue of every Customer ent ty’s UpdatedAt property needs to be preserved It s thus ab e to return that or g na va ue n the If-Match header n the MERGE request, so that the serv ce can pass t on to EF, and EF can pass t on to the UpdateCustomer stored procedure SQL Profi er shows the actua stored procedure ca executed n SQL Server exec [dbo].[UpdateCustomer] @CustomerId=3,@FirstName='Cassie',@LastName='Hicks',@ Balance=40.0000,@OriginalUpdatedAt='2012-02-01 14:51:38.6967614'

After the update, the serv ce response s returned to the c ent It conta ns severa HTTP headers, as shown n L st ng 11-5 Listing 11-5  The new UpdatedAt concurrency va ue returned to the c ent as an ETag response header.

HTTP/1.1 204 No Content Server: ASP.NET Development Server/10.0.0.0 Date: Wed, 01 Feb 2012 19:51:38 GMT X-AspNet-Version: 4.0.30319 DataServiceVersion: 1.0; Cache-Control: no-cache ETag: W/"datetime'2012-02-01T14%3A51%3A38.6967614'" Content-Length: 0 Connection: Close

Aga n, because of the or g na UpdatedAt va ue mapp ng, the serv ce retr eves the new y ass gned va ue (set by the stored procedure after a successfu update) and returns t to the c ent n the ETag header The WCF Data Serv ces c ent brary then automat ca y refreshes the c ent-s de nstance of the ent ty w th th s updated va ue, thereby synchron z ng t w th the ent ty updated n the database Now ook at the how the new order was nserted In F dd er, c ck the second request to v ew the HTTP traffic for the POST that nserted the order The request data appears n the top pane , as shown n L st ng 11-6 Listing 11-6  An HTTP POST request that nserts a new OrderHeader ent ty.

POST http://localhost.:1055/CustomerDataService.svc/Customers(3L)/OrderHeaders HTTP/1.1 User-Agent: Microsoft ADO.NET Data Services DataServiceVersion: 1.0;NetFx MaxDataServiceVersion: 2.0;NetFx Accept: application/atom+xml,application/xml Accept-Charset: UTF-8

528  Part III  Applied SQL

Content-Type: application/atom+xml Host: localhost.:1055 Content-Length: 961 Expect: 100-continue





2012-02-01T19:51:38.8188317Z

0001-01-01T00:00:00 0

0 Open Regular Mail 0001-01-01T00:00:00



As w th the MERGE request, the POST ( nsert) request s made w th an OData URI that ­spec fies the customer orders co ect on to be nserted nto as Customers(3L)/OrderHeaders, fo owed by severa HTTP headers and the new order data And as before, the request s forwarded by WCF Data Serv ces to Ent ty Framework The OrderHeader ent ty s not mapped to stored procedure mapp ngs n the EDM ke the Customers ent ty s, so th s t me EF dynam ca y generates and executes a d rect ­parameter zed INSERT statement for the new order, as shown n the SQL Profi er trace exec sp_executesql N'insert [dbo].[OrderHeader]([CustomerId], [ShipVia], [OrderStatus], [Notes], [CreatedAt], [UpdatedAt]) values (@0, @1, @2, null, @3, @4) select [OrderHeaderId] from [dbo].[OrderHeader] where @@ROWCOUNT > 0 and [OrderHeaderId] = scope_identity()',N'@0 bigint,@1 varchar(20),@2 varchar(20),@3 datetime2(7),@4 datetime2(7)',@0=3,@1='Regular Mail',@2='Open',@3='0001-01-01 00:00:00',@4='0001-01-01 00:00:00'

After the nsert, the serv ce response s returned to the c ent The response nd cates that the ­ent ty was created, and returns the ent re new ent ty to the c ent, as shown n L st ng 11-7



Chapter 11  WCF Data Access Techno og es   529

Listing 11-7  The HTTP response for a new y nserted ent ty.

HTTP/1.1 201 Created Server: ASP.NET Development Server/10.0.0.0 Date: Wed, 01 Feb 2012 19:51:39 GMT X-AspNet-Version: 4.0.30319 DataServiceVersion: 1.0; Content-Length: 1345 Location: http://localhost.:1055/CustomerDataService.svc/OrderHeaders(10L) Cache-Control: no-cache Content-Type: application/atom+xml;charset=utf-8 Connection: Close

http://localhost.:1055/CustomerDataService.svc/OrderHeaders(10L)

2012-02-01T19:51:39Z





10 3 Regular Mail Open

0001-01-01T00:00:00 0001-01-01T00:00:00



Building a Data Entry Client Now that you have a firm understand ng of the core WCF Data Serv ces concepts (REST, OData, and the c ent brar es), you are ready to bu d a more usefu app cat on Th s next WCF Data Serv ces c ent comb nes the techn ques you just earned w th a W ndows Forms user nterface and some data b nd ng to et the user ed t customers and orders

530  Part III  Applied SQL

Note  Data binding is a vast topic, and Windows Forms, ASP.NET, and WPF/Silverlight each support very different data binding architectures that are beyond the scope of this book. The minimal data binding code in our examples is provided to demonstrate client/server data access techniques with WCF Data Services (and, in the next section, WCF RIA Services). R ght-c ck the DemoWcfDataServices so ut on n So ut on Exp orer and choose Add New Project From the st of temp ate categor es on the eft, beneath V sua C#, choose W ndows Then se ect the W ndows Forms App cat on temp ate to create a new project named DemoRestClientDataEntry The DemoWcfDataServices so ut on now has three projects n t an ASP NET project (the ­serv ce) and two W ndows Forms projects (the prev ous DemoRestClientTest project and the new ­DemoRestClientDataEntry project w th an empty form) Set a reference from the ­DemoRestClientDataEntry project to the serv ce project as you d d for the test c ent R ght-c ck the DemoRestClientDataEntry project, choose Add Serv ce Reference, and c ck D scover Then type ­CustomerDataService for the Namespace and c ck OK Now drag severa contro s onto the empty form For v ew ng and ed t ng customers, create two buttons named btnNewCustomer and btnDeleteCustomer a ong w th a DataGridView named grdCustomers Create two more buttons named btnNewOrder and btnDeleteOrder a ong w th another DataGridView named grdOrders for v ew ng and ed t ng orders Then add two buttons named btnLoad and btnSave to query ng from the serv ce and send changes back to t F na y, set the AllowUserToAddRows and AllowUserToDeleteRows propert es of both DataGridView contro s Fa se (th s s because add ng and de et ng ent t es w be hand ed by code beh nd the respect ve New and De ete buttons) After perform ng some aesthet c a gnment and formatt ng, your form shou d appear someth ng ke the one shown n F gure 11-9

Figure 11-9  The WCF Data Serv ces data entry c ent user nterface.



Chapter 11  WCF Data Access Techno og es   531

Next, r ght-c ck on the form’s des gn surface and choose V ew Code to open a w ndow to the form’s code Rep ace t ent re y w th the code shown n L st ng 11-8 (once aga n, remember to change the port number n the serviceUri constant—th s t me defined n the CreateNewContext method— w th the port number for your env ronment) Listing 11-8  Code for the WCF Data Serv ces data entry c ent form.

using using using using using

System; System.Collections.Generic; System.Data.Services.Client; System.Linq; System.Windows.Forms;

using DemoRestClientDataEntry.CustomerDataService; namespace DemoRestClientDataEntry { public partial class Form1 : Form { private SampleDbEntities _context; private List _customers; private Customer _currentCustomer; private BindingSource _customersBindingSource; private BindingSource _ordersBindingSource; public Form1() { InitializeComponent(); } private void btnLoad_Click(object sender, EventArgs e) { this.CreateNewContext(); var query = from customer in this._context.Customers.Expand("OrderHeaders") select customer; this._customers = query.ToList(); this._customersBindingSource = new BindingSource(this._customers, null); this.BindGrid(this.grdCustomers, this._customersBindingSource); this.grdOrders.DataSource = null; this.ShowCustomerOrders(); } private void CreateNewContext() { // - The "." after "localhost" is a hack to enable Fiddler packet-sniffing

532  Part III  Applied SQL

// - Change the port number accordingly for your dev machine var serviceUri = new Uri("http://localhost.:1055/CustomerDataService.svc"); this._context = new SampleDbEntities(serviceUri); } private void grdCustomers_SelectionChanged(object sender, EventArgs e) { this.ShowCustomerOrders(); } private void ShowCustomerOrders() { if (this.grdCustomers.SelectedRows.Count == 1) { this._currentCustomer = (Customer)this.grdCustomers.SelectedRows[0].DataBoundItem; this._ordersBindingSource = new BindingSource(this._currentCustomer.OrderHeaders, null); this.BindGrid(this.grdOrders, this._ordersBindingSource); } } private void btnNewCustomer_Click(object sender, EventArgs e) { this._customersBindingSource.AddNew(); } private void grdCustomers_CellValueChanged (object sender, DataGridViewCellEventArgs e) { var customer = (Customer)this.grdCustomers.SelectedRows[0].DataBoundItem; var changeInfo = this._context.Entities.SingleOrDefault(ed => ed.Entity == customer); if (changeInfo == null) { this._context.AddToCustomers(customer); } else if (changeInfo.State == EntityStates.Unchanged) { this._context.UpdateObject(customer); } } private void btnDeleteCustomer_Click(object sender, EventArgs e) { var customer = (Customer)this.grdCustomers.SelectedRows[0].DataBoundItem; this._customersBindingSource.RemoveCurrent(); this.grdOrders.DataSource = null;



Chapter 11  WCF Data Access Techno og es   533

var changeInfo = this._context.Entities.SingleOrDefault(ed => ed.Entity == customer); if (changeInfo != null) { foreach (var orderHeader in customer.OrderHeaders) { this._context.DeleteObject(orderHeader); } this._context.DeleteObject(customer); } } private void btnNewOrder_Click(object sender, EventArgs e) { if (this.grdOrders.Columns.Count == 0) { this.ShowCustomerOrders(); this.grdCustomers.EndEdit(); } var orderHeader = (OrderHeader)this._ordersBindingSource.AddNew(); orderHeader.CustomerId = this._currentCustomer.CustomerId; orderHeader.Customer = this._currentCustomer; } private void grdOrders_CellValueChanged (object sender, DataGridViewCellEventArgs e) { var orderHeader = (OrderHeader)this.grdOrders.SelectedRows[0]. DataBoundItem; var changeInfo = this._context.Entities.SingleOrDefault(ci => ci.Entity == orderHeader); if (changeInfo == null) { this._context.AddToOrderHeaders(orderHeader); } else if (changeInfo.State == EntityStates.Unchanged) { this._context.UpdateObject(orderHeader); } } private void btnDeleteOrder_Click(object sender, EventArgs e) { var orderHeader = (OrderHeader)this.grdOrders.SelectedRows[0]. DataBoundItem; this._ordersBindingSource.RemoveCurrent(); this.grdOrders.DataSource = null;

534  Part III  Applied SQL

var changeInfo = this._context.Entities.SingleOrDefault(ed => ed.Entity == orderHeader); if (changeInfo != null) { this._context.DeleteObject(orderHeader); } } private void btnSave_Click(object sender, EventArgs e) { this.grdCustomers.EndEdit(); this.grdOrders.EndEdit(); try { this._context.SaveChanges(SaveChangesOptions.Batch); } catch (Exception ex) { MessageBox.Show (ex.Message, "Error", MessageBoxButtons.OK, MessageBoxIcon.Error); } this.grdCustomers.Refresh(); this.grdOrders.Refresh(); } private void BindGrid(DataGridView grd, BindingSource bs) { var readOnlyColumns = new string[] { "CustomerId", "OrderHeaderId", "CreatedAt", "UpdatedAt" }; var hiddenColumns = new string[] { "Customer" }; grd.DataSource = bs; foreach (DataGridViewColumn col in grd.Columns) { col.ReadOnly = readOnlyColumns.Contains(col.HeaderText); col.Visible = !hiddenColumns.Contains(col.HeaderText); } grd.AutoResizeColumns(); } } }

Your ast step s to w re up the event hand ers Return to the form des gner and c ck the Events button n the too bar at the top of the Propert es gr d Then, one at a t me, se ect nd v dua contro s and ass gn them to event hand ers n the code, as fo ows ■



Ass gn the btnNewCustomer button’s Click event to btnNewCustomer Click

Chapter 11  WCF Data Access Techno og es   535

■

Ass gn the btnDeleteCustomer button’s Click event to btnDeleteCustomer Click

■

Ass gn the grdCustomers gr d’s SelectionChanged event to grdCustomers SelectionChanged

■

Ass gn the grdCustomers gr d’s CellValueChanged event to grdCustomers CellValueChanged

■

Ass gn the btnNewOrder button’s Click event to btnNewOrder Click

■

Ass gn the btnDeleteOrder button’s Click event to btnDeleteOrder Click

■

Ass gn the grdOrders gr d’s CellValueChanged event to grdOrders CellValueChanged

■

Ass gn the btnLoad button’s Click event to btnLoad Click

■

Ass gn the btnSave button’s Click event to btnSave Click

Th s c ent code adds some bas c data b nd ng to the concepts you earned w th the test c ent Before you run the so ut on, et us exp a n the mportant aspects of the code n L st ng 11-8 F rst, not ce the pr vate SampleDbEntities form var ab e named context Th s means that the c ent context ma nta ns ts state dur ng the fet me of the form, track ng the user’s changes made from the t me they c ck Load to retr eve data and the t me they c ck Save to update t back C ck ng Load fires btnLoad Click wh ch nstant ates the c ent context by ca ng CreateNewContext As w th the test c ent, you must adjust the serv ce URL’s port number 1055 n the CreateNewContext method ­accord ng y, and ensure that there s a “dot” after localhost so that F dd er can mon tor HTTP requests After creat ng the context, the btnLoad Click method executes a LINQ to REST query that retr eves a customers and the r orders nto a st, stores the st n the pr vate form var ab e customers, and b nds the st to the top gr d (grdCustomers) us ng a BindingSource object As soon as the st s bound to the top gr d, the first customer row gets se ected automat ca y, so btnLoad ca s the ShowCustomerOrders method to b nd the first customer’s orders to the bottom gr d (grdOrders) When the user se ects any other customer n the top gr d, the grdCustomers Selection event hand er responds w th another ShowCustomerOrders ca to update the bottom gr d for the se ected customer The ShowCustomerOrders method uses the DataBoundItem property to obta n the object bound to the current y se ected row n the top gr d, wh ch t casts to the expected Customer ent ty type W th the se ected Customer ent ty n hand, the method b nds ts OrderHeaders co ect on to grdOrders us ng a second BindingSource object The next three methods hand e Customer nserts, updates, and de etes act ons made by the user n the top gr d In part cu ar, these methods respond to events that fire as the user ed ts data, and nform the c ent context to track changes as the user makes them When the user c cks New Customer, the btnNewCustomer method ca s AddNew on the c­ ustomer b nd ng source Th s nterna y creates a new Customer object and adds t to the st bound to ­grdCustomers, wh ch n turn appends a new row to the bottom of the gr d for the user to enter the new customer’s data The context s not yet aware of the new ent ty; t w be nformed about the new customer as soon as the user ed ts the first ce n the new gr d row In grdCustomers CellValueChanged, the code responds to ed ts made by the user on any customer row Th s method determ nes whether the user s ed t ng a new or ex st ng customer by query ng the 536  Part III  Applied SQL

c ent context’s Entities co ect on for the current customer Th s co ect on ho ds change ­nformat on for a of the ent t es be ng tracked on the c ent, and the SingleOrDefault method searches the ­co ect on for the part cu ar customer that the user s ed t ng If t s not found, that means that the context s not yet track ng the object, wh ch must mean that th s s a new customer (added when the user c cked New Customer) In th s case, AddToCustomers s ca ed on the context, nstruct ng t to track th s customer as new If the context s a ready track ng the object, and the change ­nformat on’s State property equa s EntityStates.Unchanged, t means that th s s an ex st ng customer that has not yet been mod fied In th s case, UpdateObject s ca ed on the context, nstruct ng t to track th s ­customer as mod fied It s mportant to remember that the grdCustomers CellValueChanged method fires for each ce changed n a gr d’s row, but the context needs to be nstructed to track that row’s ent ty state once and on y once That s why t’s cr t ca to check the State property For examp e, cons der the new ­customer scenar o The AddToCustomers method s ca ed the first t me a ce s ed ted n a new ­customer row When another ce s ed ted n the same row, the event hand er fires aga n, on y th s t me the new Customer ent ty s found n the Entities co ect on because of the prev ous ­AddToCustomers ca Therefore, the State property s checked to prevent the act on from be ng nterpreted ncorrect y as an update to be tracked rather than an nsert that’s a ready be ng tracked The State property for the new customer s EntityStates.Added, not EntityStates.Unchanged, wh ch prevents the event hand er from erroneous y track ng the same customer aga n as mod fied S m ar y, the first t me the user ed ts the ce or an ex st ng customer’s row, the State property s changed from EntityStates.Unchanged to EntityStates.Modified by the ca to UpdateObject, wh ch prevents ex st ng customer ent t es from be ng tracked as mod fied mu t p e t mes Once a customer s be ng tracked as e ther new or changed, t w have an entry n the context’s Entities co ect on w th a State other than EntityStates.Unchanged, n wh ch case the grdCustomers CellValueChanged event hand er does ­noth ng When the user c cks De ete Customer, the btnDeleteCustomer method first obta ns the customer be ng de eted and then ca s RemoveCurrent on the customer b nd ng source Th s nterna y de etes the Customer object from the st bound to the current gr d row, wh ch n turn removes the row from the gr d on the form It s mportant to perform these steps n th s order, because you w not be ab e to reference the de eted customer ent ty once t has been removed from the b nd ng source The code then attempts to ocate the change track ng entry for the de eted customer by ­query ng the context’s Entities co ect on as before If the user c cked De ete Customer mmed ate y after ­c ck ng New Customer, then the context has no change track ng nformat on for th s customer, and so no ­act on needs to be taken Otherw se, the method enters a oop that ca s DeleteObject on the c­ ustomer’s re ated OrderHeader ent t es, and then ca s DeleteObject for the parent Customer ent ty Th s nforms the c ent context to track the customer and ts orders as de eted The next three methods that fo ow hand e OrderHeader nserts, updates, and de etes act ons made by the user n the bottom gr d These methods mon tor for changes us ng the very same techn ques we just exp a ned, and keep the c ent context nformed of data that the user ed ts n the bottom gr d



Chapter 11  WCF Data Access Techno og es   537

F na y, the btnSave Click event hand er fires when the user c cks Save Th s method first ca s ­EndEdit on both gr ds to app y any pend ng changes (wh ch s mportant f the user c cks Save wh e n the m dd e of ed t ng a ce ; ca ng EndEdit forces the gr d’s CellValueChanged event to fire, whereas otherw se t wou d not) Then, SaveChanges s nvoked on the context to push a the ­c ent updates to the serv ce, just as we demonstrated w th the test c ent But not ce that th s code ­spec fies SaveChangesOptions.Batch, wh ch s required to support h erarch ca updates (that s, ­updates ­nvo v ng both parent and ch d ent t es) The SaveChangesOptions.Batch sett ng nstructs the WCF Data Serv ces c ent brary to batch up a the ent ty changes nto a s ng e request, rather than ­nd v dua requests as you saw w th the test c ent Th s means that on y one HTTP request w be ssued to WCF Data Serv ces no matter how many ent t es are nserted, updated, or de eted on the c ent To see everyth ng work, mon tor the HTTP traffic w th F dd er as you run the so ut on F rst, make sure that both F dd er and SQL Profi er are runn ng Then r ght-c ck DemoRestClientDataEntry n So ut on Exp orer, choose Set As Startup Project, and press F5 V sua Stud o bu ds the so ut on and aunches the form C ck the Load button Th s quer es the serv ce and then b nds the customer and order data ­returned by the serv ce to the user nterface, as shown n F gure 11-10

Figure 11-10  The c ent form popu ated w th data returned by WCF Data Serv ces.

Now ed t the data as fo ows ■

■ ■

Insert a new order for Andy Jacobs by se ect ng Andy’s row n the top gr d and c ck ng New Order Then enter any random text nto the ShipVia and OrderStatus co umns for the new order Update the ba ance for Cass e H cks by chang ng the va ue from 40 to 50 n the top gr d De ete customer Josh Barnh by se ect ng Josh’s row n the top gr d and c ck ng De ete ­Customer Reca from our exp anat on of the code that th s not on y marks Josh’s customer ent ty as de eted, but a the ch d orders as we

538  Part III  Applied SQL

You have just performed a m x of nsert, update, and de ete operat ons on the c ent Now c ck Save to push a those changes to the serv ce Sw tch over to F dd er to exam ne the HTTP conversat on that just took p ace You w see that two HTTP requests were made; the first one s the GET ssued by the LINQ to REST query when you first c cked the Load button The second one s a POST ssued by SaveChanges, and t conta ns a of the nd v dua ent ty changes batched up ns de a s ng e HTTP request, as shown n L st ng 11-9 Listing 11-9  A batch of c ent updates posted to WCF Data Serv ces as a s ng e request.

POST http://localhost.:1055/CustomerDataService.svc/$batch HTTP/1.1 User-Agent: Microsoft ADO.NET Data Services DataServiceVersion: 1.0;NetFx MaxDataServiceVersion: 2.0;NetFx Accept: application/atom+xml,application/xml Accept-Charset: UTF-8 Content-Type: multipart/mixed; boundary=batch_74726ae1-e572-4e88-a53fbeb080591cdd Host: localhost.:1055 Content-Length: 3509 Expect: 100-continue Connection: Keep-Alive --batch_74726ae1-e572-4e88-a53f-beb080591cdd Content-Type: multipart/mixed; boundary=changeset_d7d5bcea-d717-4c2e-813480e730c80738 --changeset_d7d5bcea-d717-4c2e-8134-80e730c80738 Content-Type: application/http Content-Transfer-Encoding: binary POST http://localhost.:1055/CustomerDataService.svc/OrderHeaders HTTP/1.1 Content-ID: 25 Content-Type: application/atom+xml;type=entry Content-Length: 965





2012-02-02T19:43:04.539875Z

0001-01-01T00:00:00 2



Chapter 11  WCF Data Access Techno og es   539

0 Pending Regular Mail 0001-01-01T00:00:00

--changeset_d7d5bcea-d717-4c2e-8134-80e730c80738 Content-Type: application/http Content-Transfer-Encoding: binary MERGE http://localhost.:1055/CustomerDataService.svc/Customers(3L) HTTP/1.1 Content-ID: 26 Content-Type: application/atom+xml;type=entry If-Match: W/"datetime'2012-02-02T14%3A21%3A07.89925'" Content-Length: 991





2012-02-02T19:43:04.539875Z http://localhost.:1055/CustomerDataService.svc/Customers(3L)

50.0000 2012-02-02T14:20:55.696125 3 Cassie Hicks 2012-02-02T14:21:07.89925

--changeset_d7d5bcea-d717-4c2e-8134-80e730c80738 Content-Type: application/http Content-Transfer-Encoding: binary DELETE http://localhost.:1055/CustomerDataService.svc/OrderHeaders(8L) HTTP/1.1 Content-ID: 27 --changeset_d7d5bcea-d717-4c2e-8134-80e730c80738 Content-Type: application/http Content-Transfer-Encoding: binary DELETE http://localhost.:1055/CustomerDataService.svc/OrderHeaders(9L) HTTP/1.1 Content-ID: 28

540  Part III  Applied SQL

--changeset_d7d5bcea-d717-4c2e-8134-80e730c80738 Content-Type: application/http Content-Transfer-Encoding: binary DELETE http://localhost.:1055/CustomerDataService.svc/Customers(4L) HTTP/1.1 Content-ID: 29 If-Match: W/"datetime'2012-01-01T00%3A00%3A00'" --changeset_d7d5bcea-d717-4c2e-8134-80e730c80738---batch_74726ae1-e572-4e88-a53f-beb080591cdd--

If you break down th s request, you can see that t conta ns mu t p e parts, each of wh ch ­represents a s ng e ent ty change request The serv ce URI for the ma n request s term nated w th $batch, wh ch nforms the serv ce that t s actua y rece v ng a batch of requests, a packaged up ns de the ma n HTTP request A so not ce that the Content-Type header reads multipart/mixed w th a str ng defin ng the boundar es of the batch (th s str ng appears before the first request n the batch and aga n after the ast request n the batch) Each of the requests n the batch s dent fied w th a un que Content-ID header, and they are a hand ed the same as the nd v dua requests you saw n the test c ent The first s a POST to nsert the new order for Andy Jacobs That s then fo owed by the MERGE to update the ba ance for Cass e H cks F na y, three DELETE operat on requests are ssued to de ete Josh Barnh ; the first two de ete the customer’s re ated orders, and the th rd de etes the customer tse f Sw tch over now to SQL Profi er to see the database act v ty generated by these requests As shown n F gure 11-11, EF automat ca y generated INSERT and DELETE statements to update the ­OrderHeader tab e and stored procedure ca s to UpdateCustomer and DeleteCustomer that update the Customer tab e Th s s expected, aga n, because there are stored procedure mapp ngs n the EDM for the Customers ent ty set but not OrderHeaders A so not ce that a the updates are batched ns de a transact on, wh ch ensures that they a succeed or fa together

Figure 11-11  SQL Server Profi er output show ng the T SQL statements executed by the batched WCF Data

Serv ces request.



Chapter 11  WCF Data Access Techno og es   541

Now sw tch back to F dd er and exam ne the HTTP response returned to the c ent, as shown n L st ng 11-10 Listing 11-10  A batch of WCF Data Serv ces resu ts returned as a s ng e response.

HTTP/1.1 202 Accepted Server: ASP.NET Development Server/10.0.0.0 Date: Thu, 02 Feb 2012 19:43:04 GMT X-AspNet-Version: 4.0.30319 DataServiceVersion: 1.0; Content-Length: 2898 Cache-Control: no-cache Content-Type: multipart/mixed; boundary=batchresponse_f525ef65-e8fb-4cd6-810840e1212fe166 Connection: Close --batchresponse_f525ef65-e8fb-4cd6-8108-40e1212fe166 Content-Type: multipart/mixed; boundary=changesetresponse_0eb669ef-4471-477785a6-8c0ed3993cd4 --changesetresponse_0eb669ef-4471-4777-85a6-8c0ed3993cd4 Content-Type: application/http Content-Transfer-Encoding: binary HTTP/1.1 201 Created Content-ID: 25 Cache-Control: no-cache DataServiceVersion: 1.0; Content-Type: application/atom+xml;charset=utf-8 Location: http://localhost.:1055/CustomerDataService.svc/OrderHeaders(33L)

http://localhost.:1055/CustomerDataService.svc/OrderHeaders(33L)

2012-02-02T19:43:04Z





33 2 Regular Mail Pending

542  Part III  Applied SQL

0001-01-01T00:00:00 0001-01-01T00:00:00

--changesetresponse_0eb669ef-4471-4777-85a6-8c0ed3993cd4 Content-Type: application/http Content-Transfer-Encoding: binary HTTP/1.1 204 No Content Content-ID: 26 Cache-Control: no-cache DataServiceVersion: 1.0; ETag: W/"datetime'2012-02-02T14%3A43%3A04.9305'" --changesetresponse_0eb669ef-4471-4777-85a6-8c0ed3993cd4 Content-Type: application/http Content-Transfer-Encoding: binary HTTP/1.1 204 No Content Content-ID: 27 Cache-Control: no-cache DataServiceVersion: 1.0; --changesetresponse_0eb669ef-4471-4777-85a6-8c0ed3993cd4 Content-Type: application/http Content-Transfer-Encoding: binary HTTP/1.1 204 No Content Content-ID: 28 Cache-Control: no-cache DataServiceVersion: 1.0; --changesetresponse_0eb669ef-4471-4777-85a6-8c0ed3993cd4 Content-Type: application/http Content-Transfer-Encoding: binary HTTP/1.1 204 No Content Content-ID: 29 Cache-Control: no-cache DataServiceVersion: 1.0; --changesetresponse_0eb669ef-4471-4777-85a6-8c0ed3993cd4---batchresponse_f525ef65-e8fb-4cd6-8108-40e1212fe166--

You can see that the response s a so batched Not ce that nd v dua responses ns de the batch are each tagged w th Content-ID va ues so that the WCF Data Serv ces c ent brary can corre ate each ­response that comes back w th the or g na request that went out As exp a ned for the test c ent, each nd v dua response’s content s used to refresh the c ent-s de nstances of each updated ent ty



Chapter 11  WCF Data Access Techno og es   543

Extending WCF Data Services We have conducted a fa r y deta ed study of WCF Data Serv ces St , the examp es thus far have been based on very s mp e c ent-s de quer es and stra ghtforward updates A though OData URI quer es can be qu te flex b e, they do not support the more advanced LINQ query operat ons ­Therefore, some LINQ to REST quer es s mp y cannot be converted nto an equ va ent OData URI str ng ­Furthermore, you may need to supp ement the defau t query/update funct ona ty prov ded by the serv ce w th your own custom bus ness og c that fi ters or va dates data Fortunate y, WCF Data Serv ces has severa extens b ty features ava ab e that a ow your serv ces to overcome the m tat ons of OData URI quer es and prov de much more funct ona ty than stra ghtforward CRUD operat ons We conc ude our WCF Data Serv ces coverage by d scuss ng custom serv ce operat ons, serv ce method overr des, and query nterceptors

Creating Custom Service Operations It s very easy to expose your own custom methods as add t ona serv ce operat ons S mp y define any pub c method that returns IQueryable (where T s any ent ty type n your EDM), and decorate the method w th the WebGet attr bute Then add one ne of code to the InitializeService method to expose the method as a serv ce operat on You can use th s techn que to mp ement quer es on the serv ce that are d fficu t or mposs b e to express on the c ent us ng LINQ to REST and an OData URI It s a so a great way to extend the serv ce w th custom bus ness og c To demonstrate, return to the CustomerDataService.cs fi e n the DemoRestService project (L st ng 11-2) Doub e-c ck the fi e n So ut on Exp orer and add the code shown n L st ng 11-11, just beneath the ­InitializeService method Listing 11-11  Add ng a custom operat on to WCF Data Serv ces.

[WebGet] public IQueryable GetCustomersByBalance(decimal minBalance) { return from customer in base.CurrentDataSource.Customers where customer.Balance > minBalance select customer; }

Then add th s one ne to the InitializeService method (you can add t anywhere ns de the method) config.SetServiceOperationAccessRule("GetCustomersByBalance", ServiceOperationRights.All);

Now rebu d the so ut on and WCF Data Serv ces w serv ce operat on

mmed ate y expose the method as a custom

Th s s mp e examp e may not do much, but t does ustrate a the essent a s po nts The WebGet attr bute exposes the pub c method GetCustomersByLastName as a serv ce operat on The serv ce operat on accepts a dec ma parameter, wh ch t uses n the where c ause of a LINQ query—on y 544  Part III  Applied SQL

th s s a true LINQ to Ent t es query, us ng Ent ty Framework on the server The LINQ query obta ns access to the ObjectContext v a the CurrentDataSource property defined n the serv ce’s base c ass, DataService Here s the OData URI syntax for ca ng th s serv ce operat on that returns those customers whose ba ance exceeds 15 (the M suffix n 15M represents the money data type n SQL Server, correspond ng to decimal n NET) http://localhost.:1055/CustomerDataService.svc/GetCustomersByBalance()?minBalance=15M

Any t me you mod fy your serv ce, you need to update the serv ce reference on the c ent to regenerate the proxy c asses So you wou d be correct to assume that your next step s to do just that, but unfortunate y updat ng the serv ce reference w not expose GetCustomersByBalance on the c ent s de (as t shou d) Th s s due to a m tat on n V sua Stud o’s WCF Data Serv ces c ent proxy generator too that fa s to detect custom serv ce operat ons ( t on y recogn zes ent ty sets exposed by the InitializeService method) The workaround s to manua y extend the automat ca y generated c ent-s de context c ass ­SampleDbEntities w th a pub c method that wraps the serv ce operat on Essent a y, th s means ­add ng the code that the proxy generator shou d have added for you The generated ­SampleDbEntities context c ass s marked partial, mak ng t very easy for you to fi n the m ss ng ­funct ona ty by add ng the wrapper method Try th s out w th the ear er DemoRestClientTest project by add ng the code shown n L st ng 11-12 ­ nywhere to the project You can s mp y append the code to the bottom of the form’s code you added a from L st ng 11-3 A ternat ve y, you can create a new c ass fi e for custom proxy wrapper extens ons and add the code there; t rea y doesn’t matter Just be sure you don’t add t d rect y to the generated proxy code, as that code gets overwr tten whenever you update the serv ce reference Listing 11-12  Extend ng the c ent context c ass w th a custom serv ce operat on wrapper method.

namespace DemoRestClientTest.CustomerDataService { using System.Data.Services.Client; using System.Collections.Generic; public partial class SampleDbEntities { public DataServiceQuery GetCustomersByBalance(decimal minBalance) { return base .CreateQuery("GetCustomersByBalance") .AddQueryOption("minBalance", string.Format("{0}M", minBalance)); } } }



Chapter 11  WCF Data Access Techno og es   545

Th s code extends the c ent context c ass by g v ng t a GetCustomersByBalance method The method accepts a m n mum ba ance parameter, and nterna y ca s the base c ass CreateQuery and AddQueryOption methods to produce a DataServiceQuery object Embedd ng th s code nto the context tse f makes t comp ete y reusab e, so th s p umb ng og c does not need to be dup cated anywhere e se The context c ass now makes the GetCustomersByBalance method as easy to nvoke as LINQ to REST quer es aga nst ord nary ent ty sets For examp e, th s ne of code ca s the serv ce operat on to get the customers w th a ba ance exceed ng 15 and popu ates a st w th the resu ts var list = ctx.GetCustomersByBalance(15).ToList();

You can eas y test th s out as fo ows Create another button on the form and add th s ne of code to the button’s Click event hand er Set a breakpo nt on the ne, r ght-c ck ­DemoRestClientTest n So ut on Exp orer, choose Set As Startup Project, and press F5 Then c ck the button to h t the breakpo nt, and s ng e-step over the ne of code so you can mon tor the act v ty w th F dd er and SQL Profi er to confirm the resu ts returned n list

Overriding Service Methods The serv ce’s base c ass, DataService, has two v rtua protected methods that you can overr de (ak n to, but not exact y the same as, an event) to nject custom og c that executes on each request or whenever an except on occurs Respect ve y, these are named ­OnStartProcessingRequest and HandleException, and each of them accepts an arguments parameter w th nformat on you can use n hand ng these “events ” Both of them shou d nvoke the base c ass mp ementat on before add ng extended funct ona ty L st ng 11-13 be ow demonstrates how to mp ement these method overr des Listing 11-13  Overr d ng the OnStartProcessingRequest and HandleException base c ass methods.

protected override void OnStartProcessingRequest(ProcessRequestArgs args) { base.OnStartProcessingRequest(args); // add code here... } protected override void HandleException(HandleExceptionArgs args) { base.HandleException(args); // add code here... }

The OnStartProcessingRequest overr de executes w th each ncom ng c ent request In th s method, you can exam ne the ProcessRequestArgs parameter to d scover and act upon a host of nformat on, such as the request URI, method, and headers, as we as the response headers, and an IsBatchRequest flag nd cat ng that th s s a batch (mu t -part) request

546  Part III  Applied SQL

The HandleException overr de effect ve y serves as a g oba “catch b ock” for the serv ce The HandleExceptionArgs parameter has an Exception property that hands you the unhand ed except on object The arguments parameter a so exposes a UseVerboseErrors property that contro s whether deta ed nformat on about the except on s returned to the c ent, p us severa usefu read-on y response-re ated propert es (ResponseContentType, ResponseStatusCode, and ResponseWritten)

Writing Interceptors F na y, you can wr te spec a methods to ntercept each query or change request at the ent ty set eve Th s prov des you w th a great dea of contro over serv ce behav or There are two types of nterceptors Query nterceptors a ow you to fi ter data be ng returned out of an ent ty set that s sat sfy ng any query n the serv ce Change nterceptors a ow you to exam ne each ent ty change just before t occurs, so you can take spec a act on f des red Both types of nterceptors operate at the ent ty set eve At runt me, the framework uses ­Reflect on to d scover nterceptors n the serv ce by ocat ng methods decorated w th the QueryInterceptor or ChangeInterceptor attr bute The ent ty set name s spec fied as a str ng parameter to the method ­attr bute, whereas the method name tse f s ns gn ficant Interceptor methods must be coded w th the correct s gnatures; query nterceptors are expected to return a ambda that fi ters the ent ty set, wh e change nterceptors are expected to accept two parameters conta n ng nformat on about the ent ty be ng changed, as L st ng 11-14 demonstrates Listing 11-14  mp ement ng a query nterceptor and change nterceptor for the Customers ent ty set.

[QueryInterceptor("Customers")] public Expression OnQueryCustomers() { return c => c.Balance > 15; } [ChangeInterceptor("Customers")] public void OnChangeCustomers(Customer customer, UpdateOperations op) { if (op == UpdateOperations.Delete) { throw new DataServiceException("Customers cannot be deleted"); } }

The first method dec ares a query nterceptor for the Customers ent ty As such, t returns a ambda express on that w be used to fi ter customer data returned for a quer es Th nk of t as a g oba “where c ause” for the Customers ent ty set A ambda express on s mere y a method po nter, wh ch n the case of Func refers to a method that accepts a Customer object and returns a bool resu t In th s examp e, the ambda returns true f the customer ba ance s greater than 15, effect ve y hardcod ng a fi ter that exc udes a customers w th a ba ance of 15 and ower from every



Chapter 11  WCF Data Access Techno og es   547

query that gets serv ced Note that to use the actua return data type Expression, you must add the fo ow ng using statement at the top of the code using System.Linq.Expressions;

The second method mp ements a change nterceptor, a so for the Customers ent ty Change ­nterceptors return no data, and must accept two parameters The first s an object of the ent ty set type (wh ch s Customer n th s examp e), and the second s an UpdateOperations enumerat on You can nspect the ent ty be ng updated as we as the type of update be ng attempted, and take any custom act on requ red by your serv ce Th s prov des you w th a much finer degree of contro over secur ty than the g oba read-on y versus wr tab e contro prov ded by the InitializeService method In th s examp e, the serv ce s mp y does not perm t customers to be de eted WCF Data Serv ces supports authent cat on, a though t does not mp ement authent cat on on ts own Instead, t re es on authent cat on opt ons supported by the host (typ ca y IIS), ­nc ud ng ­anonymous, bas c, d gest, W ndows, ASP NET forms, and c a ms-based authent cat on W th ­authent cat on enab ed, you can perform user-based secur ty checks n your change nterceptors, as we as user-based fi ter ng n your query nterceptors For examp e, f your WCF Data Serv ces are hosted as an ASP NET Web App cat on ( ke those n th s chapter are), you can retr eve the ­pr nc pa (secur ty nformat on) of the request by exam n ng the HttpContext.Current.User property Then you can everage that nformat on (for examp e, user name, or user ID) to mp ement user-spec fic ­nterceptors n your WCF Data Serv ces

WCF RIA Services WCF RIA Serv ces s, we , richer than WCF Data Serv ces (and a so newer) Indeed, the R n RIA means r ch, a though the fu TLA (Three-Letter Acronym) can stand for R ch Internet App cat on or R ch Interactive App cat on—depend ng on who you’re ta k ng to S nce ts ear est days, WCF RIA Serv ces was des gned to work best w th S ver ght, a though t now a so supports OData, SOAP, and JSON to reach a w der range of c ents You can bu d WCF RIA Serv ces over any data access ayer, ­nc ud ng Ent ty Framework, LINQ to SQL, or P a n O d CLR Objects (POCOs), n wh ch case you hand e the ­pers stence yourse f us ng any data access techn que you want, nc ud ng convent ona ADO NET Both WCF Data Serv ces and WCF RIA Serv ces so ve many of the same prob ems, so t s on y natura to quest on wh ch one to use The answer extends a b t beyond the standard “ t depends on your scenar o” response, s nce WCF RIA Serv ces offers a ot more than just data access funct ona ty It a so features c ent-s de se f-track ng ent t es, c ent-s de va dat on, automat c server-to-c ent code generat on, and more A fu treatment of WCF RIA Serv ces s we beyond the scope of th s sect on, where our object ve s to demonstrate how WCF RIA Serv ces fac tates data access for funct ona ty s m ar to the WCF Data Serv ces so ut on n the prev ous sect on After work ng through a WCF RIA Serv ces vers on of the same so ut on, the chapter conc udes w th a s de-by-s de compar son of the two frameworks Th s approach w g ve you a c ear understand ng of when t s better to use one over the other (or perhaps, when t m ght be best to use ne ther)

548  Part III  Applied SQL

Establishing a WCF RIA Services Link When WCF RIA Serv ces s used w th S ver ght, V sua Stud o prov des a spec a nk between your c ent and serv ce projects L ke a serv ce reference, th s nk b nds the two projects together, on y a WCF RIA Serv ces nk coup es them much more t ght y than an ord nary serv ce reference does Pub c changes on the serv ce s de are reflected automat ca y n correspond ng c asses on the c ent s de every t me you perform a bu d—you never need to worry about work ng aga nst an outdated proxy n the c ent project s mp y because you forgot to manua y update a serv ce reference You can create a V sua Stud o so ut on w th a S ver ght c ent project and a WCF RIA Serv ces project a ready nked to each other n a s ng e step You w do that r ght now to bu d the demo

Important  The complete WCF RIA Services framework is distributed as two separate ­downloads that you must install on top of Visual Studio 2010. Before you can follow along with the samp e application, download and install WCF RIA Services SP2 for Silverlight 4 and 5 from http://www.microsoft.com/download/en/details.aspx?id 28357, and then go to http://www.microsoft.com/download/en/details.aspx?id 26939 for the WCF RIA Services Toolkit. You may also need the latest Silverlight Developer Runtime, which you can download from http://go.microsoft.com/fwlink/?LinkId 146060. If you have a non-developer version of Silverlight installed, be sure to uninstall it before attempting these installations. Start V sua Stud o and choose F e New Project From the st of temp ate categor es on the eft, beneath V sua C#, choose S ver ght Then se ect the S ver ght App cat on temp ate to create a ­project named DemoSL n a so ut on named DemoWcfRiaServices, as shown n F gure 11-12

Figure 11-12  Creat ng a S ver ght App cat on project.



Chapter 11  WCF Data Access Techno og es   549

After you c ck OK, V sua Stud o d sp ays the New S ver ght App cat on d a og Check the Enab e WCF RIA Serv ces opt on at the bottom of the d a og as shown n F gure 11-13, and c ck OK

Figure 11-13  Check ng the Enab e WCF R A Serv ces opt on to nk the c ent and serv ce projects.

When you c ck OK, V sua Stud o creates a so ut on w th two projects One s the S ver ght c ent ­ roject, and s named DemoSL just as you spec fied n the New Project d a og The other s the serv ce p counterpart, wh ch s an ASP NET Web App cat on Project named DemoSL.Web, as spec fied n the New S ver ght App cat on d a og Ord nary S ver ght so ut ons a so beg n at th s start ng po nt, but because you enab ed Enab e WCF RIA Serv ces, V sua Stud o a so performs two mportant add t ona steps F rst, t adds a number of spec a references to the S ver ght c ent project In add t on to the ghtwe ght WebRequest and WebResponse c asses prov ded by System.Net n ord nary S ver ght projects, V sua Stud o sets a reference to the core WCF assemb es System.Runtime.Serialization and System.ServiceModel, as we as the WCF RIA Serv ces c ent-s de runt me assemb es System.ServiceModel.DomainServices.Client, System.ServiceModel.DomainServices.Client.Web, and System.ServiceModel.Web.Extensions As suggested by these names, WCF RIA Serv ces s a about bu d ng domain services, and these assemb es support the framework’s r ch c ent-s de exper ence by consum ng those doma n serv ces Second, t estab shes the WCF RIA Serv ces nk that b nds the two projects together Forget s­ erv ce references; there are none The WCF RIA Serv ces nk prov des far better synchron zat on than an ord nary WCF serv ce reference (or a WCF Data Serv ces reference) W th the nk estab shed, V sua Stud o cont nuous y regenerates the c ent-s de prox es to match the doma n serv ces, every t me you bu d your so ut on It auto-generates c ent-s de cop es of shared app cat on og c you define n the serv ces project, s mp y by ook ng for c asses you’ve defined n fi es named * shared cs, or * shared vb (rather than s mp y cs or vb) The nk a so enforces automat c c ent-s de va dat on and keeps va dat on ru es n sync between the doma n serv ces and the c ent at a t mes The WCF RIA Serv ces nk great y s mp fies the n-t er pattern, and makes trad t ona n-t er deve opment fee more ke the c ent/server exper ence After V sua Stud o creates the so ut on, r ght-c ck DemoSL (the c ent project) n So ut on Exp orer and choose Propert es The WCF RIA Serv ces nk s shown n the drop-down st on the bottom of the 550  Part III  Applied SQL

c ent project’s S ver ght propert es page, and the add t ona referenced assemb es are v s b e n the References node n So ut on Exp orer, as shown n F gure 11-14

Figure 11-14  A S ver ght so ut on w th a WCF R A Serv ces nk.

Creating the Entity Data Model The next step s to create the EDM, just as you d d for the WCF Data Serv ces project n the prev ous ­sect on You can e ther repeat the same steps to bu d the EDM from scratch as exp a ned n Chapter 10, or copy the SampleEF.edmx fi e (and the connectionStrings sect on n Web.config) from the WCF Data Serv ces project as fo ows Open W ndows Exp orer to the WCF Data Serv ces project, drag the SampleEF.edmx fi e, and drop t on the DemoSL.Web project n So ut on Exp orer Then copy the connectionStrings sect on n the Web.config fi e from WCF Data Serv ces project and paste t nto th s DemoSL.Web project’s Web.config fi e (just after the open ng tag toward the top)





Chapter 11  WCF Data Access Techno og es   551

Tip  This is the second time that we have instructed you to copy the EDM from one project to another. For your production applications, it is best practice to maintain a single v­ ersion of the EDM in a class library and compile it into an assembly. Then all you need to do is ­reference the assembly and add the connection string to the configuration file in all the projects that need to share the EDM. Bu d the so ut on (press Ctrl+Shift+B) before proceed ng If you don’t bu d the so ut on now, then the EDM w not be recogn zed when you attempt to bu d doma n serv ces c asses for t n the next step

Building the Domain Service and Metadata Classes R ght-c ck the DemoSL.Web project, choose Add New Item, and se ect Web from the st of temp ate categor es on the eft Then scro down the st of nsta ed temp ates and choose Doma n Serv ce C ass Name the fi e SampleRiaService.cs and then c ck Add, as shown n F gure 11-15

Figure 11-15  Creat ng a WCF R A Serv ces doma n serv ce c ass.

When you c ck Add, V sua Stud o d sp ays the Add New Doma n Serv ce C ass d a og, shown n F gure 11-16 The purpose of th s d a og s to generate temp ate code that he ps you get started wr t ng the ­ oma n serv ce c ass and assoc ated metadata c asses for ent t es n your EDM But we w take a d s ght y d fferent approach, and have you wr te the code for these c asses from scratch nstead Th s method w he p you better understand what these c asses are a about So don’t check off any ­ent t es n th s d a og, and just c ck OK

552  Part III  Applied SQL

Figure 11-16  The Add New Doma n Serv ce C ass d a og.

Because you d d not se ect any ent t es, V sua Stud o generates an empty c ass fi e So why go through th s process nstead of s mp y add ng an empty c ass fi e? The answer s that the r­ equ red WCF RIA Serv ces assemb y references were not automat ca y added to the serv ce project when the so ut on was created, as they were for the c ent project V sua Stud o (for some reason) ­defers ­automat ca y sett ng the references n the serv ce project unt the very first t me you use th s ­d a og—and t w set those references at that t me even f you don’t se ect any ent t es Now that the references are set, you can create doma n serv ce c asses and assoc ated metadata c asses ­s mp y by add ng ord nary c asses to your serv ce project (or you cou d have a so manua y added the ­references, but gett ng V sua Stud o to add them for you s a neat tt e t me-sav ng tr ck) The on y reason to use th s d a og aga n n the future s f you want to get a head start on wr t ng the CRUD code n your doma n serv ces You w now p ug n the code for the both the doma n serv ce c ass and the metadata c asses Then we w exp a n the mean ng of these c asses, and wa k through the code n deta Rep ace the ­generated empty SampleRiaService c ass w th the code shown n L st ng 11-15 Listing 11-15  The WCF R A Serv ces doma n serv ce.

using using using using



System; System.Data; System.Linq; System.ServiceModel.DomainServices.EntityFramework;

Chapter 11  WCF Data Access Techno og es   553

using System.ServiceModel.DomainServices.Hosting; using System.ServiceModel.DomainServices.Server; using System.Transactions; namespace DemoSL.Web { [EnableClientAccess()] public class SampleRiaService : LinqToEntitiesDomainService { public IQueryable GetCustomers() { var q = from cust in base.ObjectContext.Customers.Include("OrderHeaders") orderby cust.LastName, cust.FirstName select cust; return q; } public void InsertCustomer(Customer addedCustomer) { base.ObjectContext.Customers.AddObject(addedCustomer); } public void UpdateCustomer(Customer customer) { var originalCustomer = base.ChangeSet.GetOriginal(customer); if (originalCustomer == null) { base.ObjectContext.Customers.Attach(customer); } else { base.ObjectContext.Customers.AttachAsModified(customer, originalCustomer); } var orderChanges = this.ChangeSet.GetAssociatedChanges (customer, o => o.OrderHeaders); foreach (OrderHeader order in orderChanges) { var changeOperation = this.ChangeSet.GetChangeOperation(order); switch (changeOperation) { case ChangeOperation.Insert: base.ObjectContext.ObjectStateManager.ChangeObjectState (order, EntityState.Added); break; case ChangeOperation.Update: base.ObjectContext.OrderHeaders.AttachAsModified

554  Part III  Applied SQL

(order, base.ChangeSet.GetOriginal(order)); break; case ChangeOperation.Delete: base.ObjectContext.OrderHeaders.Attach(order); base.ObjectContext.DeleteObject(order); break; } } } public void DeleteCustomer(Customer removedCustomer) { base.ObjectContext.Customers.Attach(removedCustomer); base.ObjectContext.DeleteObject(removedCustomer); var orderChanges = this.ChangeSet.GetAssociatedChanges (removedCustomer, o => o.OrderHeaders); foreach (OrderHeader order in orderChanges) { base.ObjectContext.OrderHeaders.Attach(order); base.ObjectContext.DeleteObject(order); } } public override bool Submit(ChangeSet changeSet) { var result = false; using (var ts = new TransactionScope()) { var commit = true; result = base.Submit(changeSet); if (this.ChangeSet.HasError) { commit = false; } // Other out-of-band updates run in the same transaction scope if (commit) { ts.Complete(); } } return result; } } }



Chapter 11  WCF Data Access Techno og es   555

Not ce the TransactionScope object used n the Submit method Us ng TransactionScope enab es your serv ce to have other (non-doma n serv ce) updates part c pate n the doma n serv ce update The ­TransactionScope object s prov ded by the transact on management API n the System.Transactions assemb y, wh ch you need to reference n order for th s code to comp e R ght-c ck the DemoSL.Web project n So ut on Exp orer and choose Add Reference In the Add Reference d a og, c ck the NET tab, scro to find the System.Transactions component, and doub e-c ck t Next, r ght-c ck the DemoSL.Web project aga n, and choose Add C ass Name the new c ass fi e SampleRiaService.metadata.cs and c ck Add Then rep ace the empty c ass w th the code shown n L st ng 11-16 Listing 11-16  The WCF R A Serv ces metadata c asses.

using using using using

System; System.ComponentModel.DataAnnotations; System.Data.Objects.DataClasses; System.ServiceModel.DomainServices.Server;

namespace DemoSL.Web { [MetadataType(typeof(Customer.CustomerMetadata))] public partial class Customer { internal sealed class CustomerMetadata { private CustomerMetadata() { } [Composition] [Include] public EntityCollection OrderHeaders { get; set; } [RoundtripOriginal] public DateTime UpdatedAt { get; set; } } } [MetadataType(typeof(OrderHeader.OrderHeaderMetadata))] public partial class OrderHeader { internal sealed class OrderHeaderMetadata { private OrderHeaderMetadata() { } [RoundtripOriginal] public DateTime UpdatedAt { get; set; } } } }

w

Let’s exp a n what th s doma n serv ce c ass does and the purpose of these metadata c asses We start w th the metadata c asses first, and then c rc e back around to the doma n serv ce c ass

556  Part III  Applied SQL

The Metadata Classes Metadata s “data about data,” mean ng that metadata c asses conta n spec fic nstruct ons and h nts for the WCF RIA Serv ces framework to recogn ze about your data source You don’t actua y code any og c n metadata c asses Instead, the framework ooks for metadata by exam n ng your assemb y (us ng Reflect on) for types w th spec a attr butes that you are expected to supp y to better descr be your data source The framework searches first for a MetadataType attr bute on each pub c ent ty c ass exposed by the data source—wh ch s the EDM n your current DemoSL.Web project The pub c ent ty c asses are generated automat ca y by a custom too n V sua Stud o whenever the EDM changes, so add ng the MetadataType attr bute d rect y to those c asses s not a v ab e opt on (the attr bute wou d get ost the next t me the c asses are overwr tten by the EDM des gner) Instead, you create a separate code fi e and everage the part a c ass feature to add the MetadataType attr bute The auto-generated ent ty c asses are a defined w th the partial keyword by des gn, so that you can eas y and safe y extend them n th s manner L st ng 11-16 demonstrates the techn que Th s code extends the des gnergenerated Customer and OrderHeader ent ty c asses by defin ng part a c asses w th the same names and n the same namespace as those n the EDM, and decorat ng the c asses w th the MetadataType attr bute At bu d t me, the comp er merges th s code w th the auto-generated code to produce a s ng e og ca ent ty c ass The MetadataType attr bute takes a s ng e type parameter that spec fies the metadata c ass for the ent ty That s, the MetadataType attr bute tse f s mere y a po nter that te s the framework wh ch c ass conta ns the actua metadata These c asses can be defined anywhere, but to keep code neat y organ zed, they are defined as nested c asses r ght ns de of the ent ty c asses themse ves For ­examp e, the Customer c ass has a MetadataType attr bute that po nts to the metadata c ass ­ Customer.CustomerMetadata, wh ch s nested ns de the Customer c ass In L st ng 11-16, not ce how both metadata c asses Customer.CustomerMetadata and OrderHeader.OrderHeaderMetadata mp ement a pr vate constructor that does noth ng Th s suppresses the defau t pub c constructor, wh ch prevents the metadata c asses from ever be ng nstant ated Aga n, no og c goes nto these c asses, so there s never a reason to nstant ate them They just need to be defined, so that they can be reflected Once t finds the actua metadata c ass, the c ass s exam ned for attr butes that attach spec a mean ng and behav ors to nd v dua propert es of the ent ty That s, for each pub c property defined n the data source’s ent ty c ass, the framework ooks for correspond ng propert es w th the same name and type n the metadata c ass The propert es have no og c n the r get and set b ocks— nstead, they are dec ared on y so that they can be dec ared w th attr butes These attr butes are very often re ated to va dat on For examp e, there s a Required attr bute that des gnates a property as mandatory, and there s a StringLength attr bute to constra n a str ng property w th a max mum ength Va dat on ru es bubb e up to the c ent auto-mag ca y, and are enforced by databound contro s n the S ver ght app cat on w thout any cod ng effort on your part ­ efin te A though WCF RIA Serv ces supports data sources other than Ent ty Framework, there are d advantages to us ng an EDM as the data source One of them s that you don’t need to dec are ­va dat on attr butes n metadata c asses for every s ng e ent ty property—they surface ­automat ca y from the metadata n the EDM tse f Mean ng for examp e, propert es des gnated as requ red or

Chapter 11  WCF Data Access Techno og es   557

­ av ng a max mum ength n the EDM w be recogn zed as such by the S ver ght c ent automat h ca y, w thout defin ng Required and StringLength attr butes n the metadata c asses Th s avo ds dup cat ng va dat on ru es between the EDM and the metadata c asses, wh ch s why you see so few propert es n the Customer.CustomerMetadata and OrderHeader.OrderHeaderMetadata c asses The on y propert es that you need to define are those you want to decorate w th attr butes that attach spec a mean ng to them (beyond va dat on) The metadata c asses n L st ng 11-16 demonstrate the Composition, Include, and RoundtripOriginal attr butes The Composition attr bute s app ed to the OrderHeaders property n the Customer.­CustomerMetadata c ass OrderHeaders s a nav gat on property; t “nav gates” from a parent Customer object to ts ­re ated co ect on of OrderHeader objects When the WCF RIA Serv ces framework sees Composition on a ­nav gat on property, t treats the parent and ch d objects as part of the same “compos t on”—two parts of a arger ent ty to be treated as a who e Compos t ons can nc ude add t ona re ated ent t es s mp y by mark ng nav gat on propert es w th the Composition attr bute n each parent’s metadata c ass Even w th th s attr bute, however, ch d ent t es w not be retr eved automat ca y w th parent ent t es un ess you a so spec fy the Include attr bute You w therefore typ ca y a ways spec fy Include w th Composition, as shown for the OrderHeader property n L st ng 11-16 On the c ent, a compos t on s cons dered changed whenever any of ts part c pat ng ent t es are changed, and the compos t on as a who e s then transm tted back to the serv ce for updat ng In the current demo, th s means that the “customer” s cons dered changed f any order data changes— even f the Customer ent ty tse f was not mod fied—and that the ent re compos t on ( nc ud ng the unmod fied customer) s sent to the serv ce for updat ng A s ng e doma n serv ce method s then respons b e for updat ng the parent ent ty and a of ts re ated ch d ent t es There are obv ous advantages and d sadvantageous to compos t ons, so be jud c ous w th the r use There s certa n y a conven ence factor n pass ng a the ent t es one at a t me, but you a so ncur arger data transfers by pass ng nd v dua ent t es that have no changes Next, not ce the RoundtripOriginal attr bute on the UpdatedAt propert es n both the Customer.CustomerMetadata and OrderHeader.OrderHeaderMetadata c asses Th s attr bute fac tates the opt m st c concurrency checks that re y on the or g na UpdatedAt va ue to detect mu t -user confl cts Reca from Chapter 10 that th s funct ona ty s mp emented for the Customer ent ty n the EDM by the UpdateCustomer stored procedure

Note  Although we did not walk you through the same process for the OrderHeader ­entity, you would similarly implement an UpdateOrderHeader stored procedure and map it in the EDM if this were a production application. The EDM, as it is currently configured, ­generates direct DML statements to update the OrderHeader table rather than using stored ­procedures, and makes no use of the original UpdatedAt value. You can preserve and return the or g na va ue of any property s mp y by mark ng t w th ­ oundtripOriginal, wh ch s he pfu for more than just concurrency checks Th s capab ty can R ­e m nate the need for bus ness ru es to re oad data before app y ng an update Imag ne that the ­Balance property was a so marked w th RoundtripOriginal Upon update, the doma n serv ce c ass 558  Part III  Applied SQL

wou d have access to the or g na va ue so that t cou d ensure, just for examp e, that the new b ­ a ance does not exceed the prev ous ba ance by 10 percent W thout the or g na va ue round-tr pped, the ­update method n the doma n serv ce c ass wou d be forced to go back to the database for the ­or g na ba ance n order to test for the same cond t on

The Domain Service Class Now we can sh ft focus to the doma n serv ce c ass n L st ng 11-15, SampleRiaService Th s c ass has a the pub c methods for query ng and updat ng the data source, and can be extended w th any other custom operat ons you may w sh to expose to the c ent The c ass s decorated w th the ­EnableClientAccess attr bute, wh ch s what causes the c ent-s de code for the c ass to be generated automat ca y by the WCF RIA Serv ces nk each t me you bu d the so ut on The SampleRiaService c ass nher ts from LinqToEntitiesDomainService, mean ng that t uses Ent ty Framework as the data source The T spec fies the EDM’s object context c ass, SampleDbEntities A WCF RIA Serv ces doma n serv ce c ass can a so nher t from LinqToSqlDomainService to expose LINQ to SQL mode s, or nher t from DomainService to expose POCOs (ord nary bus ness objects based on any data access ayer) The first method s mp y retr eves a customers and the r orders Not ce that no attr bute s requ red on the method; s mp y defin ng pub c methods that return IQueryable exposes those methods as query operat ons that can be ca ed by the c ent The other three methods work to nsert, update, and de ete Customer ent t es rece ved back from the c ent, and because of the Composition attr bute n the Customer.CustomerMetadata c ass, they hand e ch d OrderHeader ent t es as we W thout the Composition attr bute n the metadata c ass, another three methods wou d be needed n the doma n serv ce c ass to separate y nsert, update, and de ete OrderHeader ent t es When the c ent subm ts changes back to the serv ce, a new EF object context for the EDM s nstant ated beh nd the scenes on the server Th s context s ma nta ned across ca s that the WCF RIA Serv ces runt me makes to the set of nsert, update, and de ete methods that you must supp y n the doma n serv ce c ass These methods are respons b e for attach ng ncom ng objects to the context, wh ch the base c ass exposes through ts ObjectContext property These methods a so set the proper state nformat on ( nserted, updated, or de eted) for the objects t attaches to the context, so EF b ­ ehaves as though th s context or g na y retr eved the data and has been track ng ts changes a a ong The InsertCustomer method s s mp est It has on y one ne of code n t that ca s the AddObject method on the new Customers ent ty W th compos t on, a new “customer” s stra ghtforward t’s a ways go ng to nvo ve a new parent Customer ent ty and a co ect on of new ch d OrderHeader ­ent t es So add ng the parent ent ty automat ca y adds the ch d ent t es and the object context tracks the ent re compos t on as new The UpdateCustomer method s a b t more nvo ved Th s one method must account for both ­ ustomer and OrderHeader ent t es separate y, and nstruct the object context to track the r C changes accord ng y The Customer ent ty must be attached first, but remember that t may or may not ­actua y be mod fied as part of the overa compos t on If on y order data changed, then the

Chapter 11  WCF Data Access Techno og es   559

­ ustomer ent ty needs to be attached by ca ng Attach; otherw se, t needs to be attached by ca ng C AttachAsModified The GetOriginal method s ca ed on the ChangeSet property exposed by the base c ass to make th s determ nat on Th s method returns null f there s no or g na vers on of the ent ty n the changeset, wh ch means that the Customer was not mod fied, and Attach s ca ed ­Otherw se, AttachAsModified s ca ed to nform the context that the Customer was mod fied, and what ts ­or g na va ues were The Customer object returned by GetOriginal conta ns or g na va ues on y for those propert es marked w th RoundtripOriginal n the metadata c ass (such as UpdatedAt); a other propert es of the “or g na ” customer are n t a zed to defau t va ues (nu s and zeros) Next, GetAssociatedChanges s ca ed on the ChangeSet to return the co ect on of OrderHeader changes For each e ement n the co ect on, GetChangeOperation s ca ed to obta n a Change­ Operation enumerat on va ue that te s you f the OrderHeader was nserted, updated, or de eted For an nsert, the ChangeObjectState method s nvoked on the context’s ObjectStateManager to attach the ch d ent ty as added For an update, AttachAsModified s ca ed to attach the ch d ent ty w th ts or g na va ues, just as AttachAsModified was ca ed to attach the parent Customer ent ty when t was detected as mod fied F na y, ch d de et ons work by first ca ng Attach on the OrderHeaders ent ty set, and then ca ng DeleteObject on the ObjectContext The DeleteCustomer method performs a stra ghtforward de ete of the ent re compos t on—parent ent ty, and a re ated ch d ent t es As you saw w th de eted ch d ent t es n UpdateCustomer, ent t es are de eted by first ca ng Attach and then ca ng DeleteObject After de et ng the parent Customer ent ty, GetAssociatedChanges s ca ed to return the co ect on of ch d OrderHeader ent t es There s no need to eva uate the ChangeOperation of each ent ty; t s a ways go ng to nd cate a de et on Therefore, each OrderHeader ent ty s s mp y de eted by ca ng Attach and then DeleteObject F na y, the doma n serv ce base c ass a so exposes a Submit method that you can overr de to ­ntercept that po nt n t me just before a the changes are wr tten back to the database Ins de th s method, you can add any custom process ng og c Th nk of the Submit method as the doma n ­serv ce’s “g oba update tr gger” that you can everage to nspect, adjust, or reject pend ng data changes, or take any other des red act on before comm tt ng changes to the database As shown n the Submit method overr de n L st ng 11-15, you can a so start a T ­ ransactionScope b ock and then ca other methods to perform separate, out-of-band updates W th n the ­TransactionScope b ock, ca ng the base Submit method saves the changes from the doma n serv ce update request as part of a broader database transact on that the other updates contr bute to In th s manner, a of the updates e ther succeed or fa as a who e

More Info  Chapter 10 explains how to use TransactionScope in detail, and Chapter 4 is dedicated to the topic of database transactions.

560  Part III  Applied SQL

Building the Silverlight Client W th the doma n serv ce c ass and metadata c asses n p ace n the DemoSL.Web serv ce project, you are ready to create the S ver ght user nterface n the DemoSL c ent project Th s user nterface sha support s m ar ad-hoc data entry capab t es as the W ndows Forms c ent you created w th WCF Data Serv ces n the prev ous sect on As w th the W ndows Forms c ent, th s user nterface (UI) w nc ude gr d ­contro s for customer and order data But V sua Stud o does not automat ca y br ng n a of the supported ­S ver ght contro s ( nc ud ng the DataGr d contro ) when you create your project So before you can use the D ­ ataGr d to bu d the UI, you must add a reference to System.Windows.Controls.Data R ght-c ck the ­DemoSL project n So ut on Exp orer and choose Add Reference In the Add Reference d a og, c ck the NET tab, and scro to find and se ect the System.Windows.Controls.Data component, as shown n F gure 11-17 Then c ck OK

Figure 11-17  Sett ng a reference to System.Windows.Controls.Data prov des a DataGr d contro for S ver ght.

Now open MainPage.xaml, and rep ace the markup w th the code shown n L st ng 11-17 Listing 11-17  The S ver ght c ent XAML.





Chapter 11  WCF Data Access Techno og es   561











The des gn v ew of your S ver ght user contro shou d ook s m ar to F gure 11-18

Figure 11-18  The WCF R A Serv ces data entry c ent user nterface.

Your ast step s to add the code-beh nd for the user contro R ght-c ck anywhere on the des gn ­surface and choose V ew Code Then rep ace the empty MainPage c ass w th the code shown n L st ng 11-18

562  Part III  Applied SQL

Listing 11-18  Code for the WCF R A Serv ces data entry c ent S ver ght form.

using using using using using

System; System.ServiceModel.DomainServices.Client; System.Text; System.Windows; System.Windows.Controls;

using DemoSL.Web; namespace DemoSL { public partial class MainPage : UserControl { private SampleRiaContext _ctx = new SampleRiaContext(); public MainPage() { InitializeComponent(); } private void btnLoad_Click(object sender, RoutedEventArgs e) { this.btnLoad.IsEnabled = false; var lo = this._ctx.Load( this._ctx.GetCustomersQuery(), this.LoadOperationComplete, null); this.grdCustomers.ItemsSource = lo.Entities; } private void LoadOperationComplete(LoadOperation lo) { if (lo.HasError) { MessageBox.Show(lo.Error.Message, "Load failed", MessageBoxButton.OK); lo.MarkErrorAsHandled(); } this.btnLoad.IsEnabled = true; } private void btnSave_Click(object sender, RoutedEventArgs e) { this.btnSave.IsEnabled = false; this._ctx.SubmitChanges(this.SaveOperationCompleted, null); } private void SaveOperationCompleted(SubmitOperation so) { this.btnSave.IsEnabled = true; if (!so.HasError)



Chapter 11  WCF Data Access Techno og es   563

{ return; } var sb = new StringBuilder(); sb.AppendLine(so.Error.Message); foreach (var badEntity in so.EntitiesInError) { sb.AppendFormat(string.Format("Entity {0} Error", badEntity. GetIdentity())); sb.AppendLine(); foreach (var ve in badEntity.ValidationErrors) { sb.AppendLine(" " + ve.ErrorMessage); } } MessageBox.Show(sb.ToString(), "Submit failed", MessageBoxButton.OK); so.MarkErrorAsHandled(); } private void grdCustomers_SelectionChanged (object sender, SelectionChangedEventArgs e) { var cust = this.grdCustomers.SelectedItem as Customer; if (cust != null) { this.grdOrders.ItemsSource = cust.OrderHeaders; } } private void btnNewCustomer_Click(object sender, RoutedEventArgs e) { var cust = new Customer(); this._ctx.Customers.Add(cust); this.grdCustomers.ItemsSource = this._ctx.Customers; } private void btnDeleteCustomer_Click(object sender, RoutedEventArgs e) { var cust = this.grdCustomers.SelectedItem as Customer; if (cust != null) { this._ctx.Customers.Remove(cust); this.grdCustomers.ItemsSource = this._ctx.Customers; } } private void btnNewOrder_Click(object sender, RoutedEventArgs e) { var cust = this.grdCustomers.SelectedItem as Customer;

564  Part III  Applied SQL

if (cust != null) { var order = new OrderHeader(); cust.OrderHeaders.Add(order); this.grdOrders.ItemsSource = cust.OrderHeaders; } } private void btnDeleteOrder_Click(object sender, RoutedEventArgs e) { var cust = this.grdCustomers.SelectedItem as Customer; if (cust != null) { var order = new OrderHeader(); cust.OrderHeaders.Remove(order); this.grdOrders.ItemsSource = cust.OrderHeaders; } } } }

Th s comp etes the ent re so ut on Go ahead and bu d t, but don’t run t just yet We’ first exp a n the c ent code you just added, and then you can run the app cat on to g ve t a test-dr ve The c ent S ver ght project knows a about the pub c serv ces and ent t es defined n the serv ce project W thout estab sh ng a serv ce reference, how does t obta n th s know edge? The answer s that V sua Stud o automat ca y generates c ent prox es every t me you bu d the so ut on Th s s tr ggered by the WCF RIA Serv ces nk that ex sts between the two projects The code s wr tten to a source fi e and added to the Generated Code fo der n your S ver ght project These tems are ­h dden by defau t, but you can v ew them by se ect ng the DemoSL project and then c ck ng Show A F es (the second too bar button at the top of So ut on Exp orer), as shown n F gure 11-19

Figure 11-19  The h dden generated code fi e conta n ng WCF R A Serv ces c ent prox es.



Chapter 11  WCF Data Access Techno og es   565

In add t on to the generated proxy code n DemoSL.Web.g.cs, the WCF RIA Serv ces nk ­ utomat ca y cop es any code fi e n the serv ce project w th a name end ng n shared cs (or a shared vb) nto the ­Generated Code fo der For examp e, you cou d define a serv ce project c ass n a fi e named CommonHelpers.shared.cs, and that code fi e wou d be nstant y cop ed and updated to the c ent project for use there as we Th s automated shar ng of metadata and og c between the c ent and serv ce projects s what the WCF RIA Serv ces nk s a about Exam n ng the proxy code n DemoSL.Web.g.cs s very en ghten ng It w he p you better ­understand the WCF RIA Serv ces framework and the c ent code n L st ng 11-18 Take some t me now to exp ore t, and you w find severa nterest ng c asses n there As you probab y expect, there are Customer and ­OrderHeader c asses that correspond to the same-named ent t es exposed by the serv ce’s data source These c ent-s de c asses were generated because of the MetadataType attr bute added to the ent ty ­metadata c asses n the serv ce project The most mportant generated c ass s SampleRiaContext, wh ch nher ts from DomainContext Th s s a statefu c ent context object that fac tates data access (quer es and updates) w th the serv ce, much the way the SampleDbEntities c ent context was used n the WCF Data Serv ces ­vers on of th s so ut on ear er n the chapter However, there s an mportant d fference here that makes ­c ent ­deve opment much eas er w th WCF RIA Serv ces than WCF Data Serv ces You do not need to ­manua y not fy the SampleRiaContext object of every change to every ent ty, as you are requ red to do w th the WCF Data Serv ces c ent brary These Customer and OrderHeader c asses are comp ete y se f-track ng ent t es Because these ent t es track the r own changes, your c ent code does not need to be concerned at a w th change track ng—you just query the context, and then ssue an update to t whenever you want to save changes back to the serv ce At the top of the MainPage c ass n L st ng 11-18, a new SampleRiaContext object s dec ared and nstant ated n ctx Th s creates the statefu DomainContext for the UI when the MainPage c ass s constructed, wh ch s access b e by a the code n the c ass v a the ctx var ab e If you compare L st ng 11-18 w th the equ va ent c ent code for the WCF Data Serv ces vers on of th s so ut on n L st ng 11-8, you can see there s s gn ficant y ess code and comp ex ty requ red, thanks to the se f-track ng capab t es of WCF RIA Serv ces c ent-s de ent t es At the same t me, th s S ver ght code works asynchronous y, wh ch ntroduces a new cha enge The W ndows Forms c ents you bu t for the WCF Data Serv ces ear er n the chapter made synchronous ca s, wh ch b ocks ­c ent execut on (and the UI) wh e awa t ng a response from the serv ce Th s s mp fies cod ng, but s ­express y proh b ted n S ver ght and W ndows Phone 7 app cat ons These c ent p atforms requ re the UI to rema n respons ve at a t mes, and thus, they on y perm t asynchronous (non-b ock ng) ­serv ce ca s Fortunate y, the necessary ca back mechan sm s re at ve y easy to mp ement, as we exp a n next Four methods mp ement the data access; the first two hand e oad ng and the other two hand e sav ng Every asynchronous operat on a ways nvo ves a pa r of methods; the first method nvokes the operat on and the second method (the ca back) hand es the response Code that fo ows the nvocat on n the first method cont nues to execute mmed ate y after ssu ng the serv ce ca , ­before the response has come back for t Th s keeps the UI respons ve, but a so means that you ­cannot know f or when the serv ce ca comp etes successfu y The second method gets ca ed ­automat ca y when 566  Part III  Applied SQL

the serv ce comp etes, and rece ves nformat on that you use to determ ne the success or fa ure of the operat on The btnLoad Click event hand er fires when the user c cks Load, and mmed ate y d sab es the Load button Th s prevents the user from c ck ng Load aga n, because the UI w et them do so wh e the first request s st process ng It then nvokes the Load method on the context to nvoke the query request asynchronous y var lo = this._ctx.Load( this._ctx.GetCustomersQuery(), this.LoadOperationComplete, null);

The first parameter passed to Load spec fies the method to be ca ed n the doma n serv ce c ass Every query method ava ab e n the serv ce has a correspond ng context method that dent fies t on the c ent Th s c ent-s de method s named the same as the method n the doma n serv ce c ass, fo owed by Query Thus, the first parameter to Load ca s the context’s GetCustomersQuery method, wh ch references the GetCustomers method n the doma n serv ce The next Load parameter spec fies the ca back; that s, the method to be ca ed when the serv ce method comp etes Th s method s expected to match a part cu ar s gnature; spec fica y, t can be any method that returns void and accepts a LoadOperation object, where T s the ent ty type returned by the serv ce (Customer, n th s case) The LoadOperationComplete method uses th s exact s gnature, wh ch a ows t to be used as the ca back to rece ve contro when the serv ce operat on comp etes The ne of code fo ow ng the Load method ca

s most nterest ng

this.grdCustomers.ItemsSource = lo.Entities;

Th s code b nds the ItemsSource property of the customer’s gr d to the Entities property of the LoadOperation object returned by the Load method In trad t ona asynchronous ­programm ng, data b nd ng does not occur unt the ca back method rece ves the serv ce response But th s ne of code executes r ght after nvok ng the asynchronous ca , wh ch s before the resu ts have been returned to the c ent At the t me the ItemsSource property s set on the customer’s gr d, the c ent s aware of the schema of the data that t expects to rece ve, but t hasn’t actua y rece ved the data tse f yet Instead, the gr d w auto-mag ca y d sp ay the data as soon as t arr ves Th s ­behav or s ustrated when you run the app cat on; you w not ce that the co umn headers of the gr d appear the nstant Load s c cked, whereas the rows are popu ated w th data a moment ater The LoadOperationComplete method has the requ red s gnature expected for the ca back; t r­ eturns void and accepts a LoadOperation. B nd ng s a ready taken care of by the ­framework as just exp a ned, so the pr mary purpose of the ca back s to re-enab e the Load button and check f an error occurred dur ng the oad operat on The HasError property returns true f there was an error, and the Error property exposes the Exception object that was thrown on the serv ce Because the UI hand es the error, t a so ca s MarkErrorAsHandled on the LoadOperation object If th s method s not ca ed, the framework re-throws the except on at runt me to make sure the prob em does not s p by unnot ced



Chapter 11  WCF Data Access Techno og es   567

The asynchronous pattern of the two save methods works exact y the same The btnSave Click event hand er fires when the user c cks Save It d sab es the Save button (prevent ng the user from c ck ng Save aga n wh e a prev ous request s process ng), and then nvokes the SubmitChanges method on the context to k ck off the asynchronous save request this._ctx.SubmitChanges(this.SaveOperationCompleted, null);

The c ent does not ca the nd v dua pub c methods for nsert, update, and de ete defined n doma n serv ce d rect y As exp a ned when d scuss ng the doma n serv ce c ass mp ementat on, the WCF RIA Serv ces framework manages a server-s de EF object context and orchestrates the ca s to the doma n serv ce methods nterna y as appropr ate A you need to do s ca SubmitChanges The SubmitChanges method takes a parameter that spec fies the ca back, wh ch s expected to be any method that returns void and accepts a SubmitOperation object The SaveOperationComplete method uses th s exact s gnature, and s spec fied as the ca back to rece ve contro when the serv ce operat on comp etes The SaveOperationComplete ca back method re-enab es the Save button and checks for ­errors that m ght have occurred dur ng the save operat on Just ke the LoadOperation object, there s a HasError property to te you f there was an error, and an Error property ho d ng the Exception thrown on the serv ce The SubmitOperation object’s EntitiesInError property exposes a co ect on that reports the spec fic deta s of each ent ty that has va dat on prob ems If an error s detected, these deta s are comb ned us ng a StringBuilder and d sp ayed to the user F na y, MarkErrorAsHandled s gna s that the error was hand ed, wh ch prevents an except on from be ng thrown

More Info  We presented each operation as two distinct methods to help you grasp the idea of asynchronous programming. In practice, you will find it common instead to e ­ mbed the callback as an anonymous method inside of the calling method using a Lambda. You will learn the Lambda technique in Chapter 13, when you build a Windows Phone 7 ­application that calls WCF Data Services hosted in Windows Azure. The same a ­ synchronous programming concepts apply whether you are “inlining” the callback method using a Lambda or coding a separate callback method, as shown in Listing 11-18. The rest of the c ent code s re at ve