Learning and Operating Presto: Fast, Reliable SQL for Data Analytics and Lakehouses 9781098141851

Table of contents :
Preface
Why We Wrote This Book
Who This Book Is For
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments
Angelica Lo Duca
Tim Meehan
Vivek Bharathan
Ying Su
1. Introduction to Presto
Data Warehouses and Data Lakes
The Role of Presto in a Data Lake
Presto Origins and Design Considerations
High Performance
High Scalability
Compliance with the ANSI SQL Standard
Federation of Data Sources
Running in the Cloud
Presto Architecture and Core Components
Alternatives to Presto
Apache Impala
Apache Hive
Spark SQL
Trino
Presto Use Cases
Reporting and Dashboarding
Ad Hoc Querying
ETL Using SQL
Data Lakehouse
Real-Time Analytics with Real-Time Databases
Introducing Our Case Study
Conclusion
2. Getting Started with Presto
Presto Manual Installation
Running Presto on Docker
Installing Docker
Presto Docker Image
Dockerfile
The etc/ directory
node.properties
jvm.config
config.properties
log.properties
catalog/.properties
Building and Running Presto on Docker
The Presto Sandbox
Deploying Presto on Kubernetes
Introducing Kubernetes
Configuring Presto on Kubernetes
presto-coordinator.yaml
presto-workers.yaml
presto-config-map.yaml
presto-secrets.yaml
Adding a New Catalog
Running the Deployment on Kubernetes
Querying Your Presto Instance
Listing Catalogs
Listing Schemas
Listing Tables
Querying a Table
Conclusion
3. Connectors
Service Provider Interface
Connector Architecture
Popular Connectors
Thrift
Writing a Custom Connector
Prerequisites
Plugin and Module
ExamplePlugin
ExampleConnectorFactory
ExampleModule
ExampleConnector
ExampleHandleResolver
Configuration
ExampleConfig
SessionProperties
TableProperties
Metadata
Data model
Handles
ExampleMetadata
ExampleClient
Input/Output
ExampleSplitManager
ExampleSplit
ExampleRecordSetProvider and ExampleRecordSet
ExampleRecordCursor
Deploying Your Connector
Apache Pinot
Setting Up and Configuring Presto
Setting up Pinot
Configuring Pinot
Configuring Presto with Pinot
Presto-Pinot Querying in Action
Conclusion
4. Client Connectivity
Setting Up the Environment
Presto Client
Docker Image
Kubernetes Node
Connectivity to Presto
REST API
Python
R
JDBC
Node.js
ODBC
Other Presto Client Libraries
Building a Client Dashboard in Python
Setting Up the Client
Building the Dashboard
Connecting to and querying Presto
Preparing the results of the query
Building the first graph
Building the second graph
Conclusion
5. Open Data Lakehouse Analytics
The Emergence of the Lakehouse
Data Lakehouse Architecture
Data Lake
File Store
File Format
Table Format
Query Engine
Metadata Management
Data Governance
Data Access Control
Building a Data Lakehouse
Configuring MinIO
Populating MinIO
Configuring HMS
Configuring Spark
Registering Hudi Tables with HMS
Connecting and Querying Presto
Conclusion
6. Presto Administration
Introducing Presto Administration
Configuration
Properties
How to configure a cluster
Sessions
Using sessions
JVM
Memory
Out-of-memory errors
Garbage collection
Monitoring
Console
Using the console for monitoring
Using the console for debugging
Using the console for going over the interactive plan
REST API
Metrics
JMX connector
REST API
JMX exporters
Management
Resource Groups
Configuring resource groups
Resource groups properties
Example
Verifiers
Setting up the system
Configuring the MySQL database
Configuring the Presto verifier
Running a test
Session Properties Managers
Configuring a session property manager
Namespace Functions
Setting up the system
Configuring a function
Running a test
Conclusion
7. Understanding Security in Presto
Introducing Presto Security
Building Secure Communication in Presto
Encryption
Keystore Management
Configuring HTTPS/TLS
Running a Presto client
Running the Presto console
Authentication
File-Based Authentication
Running a Presto client
Running the Presto console
LDAP
Kerberos
Prerequisites
Configuring the Presto coordinator and workers
Configuring the Presto client
Creating a Custom Authenticator
Authorization
Authorizing Access to the Presto REST API
Configuring System Access Control
Authorization Through Apache Ranger
Building a custom audit function
Conclusion
8. Performance Tuning
Introducing Performance Tuning
Reasons for Performance Tuning
The Performance Tuning Life Cycle
Query Execution Model
Approaches for Performance Tuning in Presto
Resource Allocation
Storage
Query Optimization
Aria Scan
Table Scanning
Repartitioning
Implementing Performance Tuning
Building and Importing the Sample CSV Table in MinIO
Converting the CSV Table in ORC
Defining the Tuning Parameters
Running Tests
Default parameters
Reducing CPU usage
Query optimization
Aria scan
Conclusion
9. Operating Presto at Scale
Introducing Scalability
Reasons to Scale Presto
Common Issues
Design Considerations
Availability
Manageability
Performance
Protection
Configuration
How to Scale Presto
Multiple Coordinators
Presto on Spark
Spilling
Using a Cloud Service
Conclusion
Index

Polecaj historie

Learning and Operating Presto: Fast, Reliable SQL for Data Analytics and Lakehouses 9781098141851

The Presto community has mushroomed since its origins at Facebook in 2012. But ramping up this open source distributed S

187 66 5MB Read more

Learning and Operating Presto: Fast, Reliable SQL for Data Analytics and Lakehouses 9781098141851, 9781098141790, 1098141857

The Presto community has mushroomed since its origins at Facebook in 2012. But ramping up this distributed SQL query eng

234 24 4MB Read more

SQL for Data Analytics: Perform fast and efficient data analysis with the power of SQL 1789807352, 9781789807356

Take your first steps to become a fully qualified data analyst by learning how to explore large relational datasets. Key

2,037 378 22MB Read more

SQL for Data Analytics: Perform Fast and Efficient Data Analysis with the Power of SQL 1789807352, 9781789807356

Take your first steps to become a fully qualified data analyst by learning how to explore large relational datasets. Key

4,522 1,107 22MB Read more

SQL for Data Analytics: Perform fast and efficient data analysis with the power of SQL [1 ed.] 1789807352, 978-1789807356

Take your first steps to become a fully qualified data analyst by learning how to explore large relational datasets. Ke

4,527 1,281 12MB Read more

Learning Analytics Cookbook: How to Support Learning Processes Through Data Analytics and Visualization 3030433765, 9783030433765

https://www.springer.com/gp/book/9783030433765 This book offers an introduction and hands-on examples that demonstrate

1,174 198 4MB Read more

Data Analytics and KPIs for Project Success

Determining exactly how to measure if project, team, and leadership performance has been successful is key to developing

1,580 211 269KB Read more

Machine Learning Approach for Cloud Data Analytics in IoT 9781119785804

1,145 176 16MB Read more

Data Analytics and Adaptive Learning: Research Perspectives 1032154705, 9781032154701

Data Analytics and Adaptive Learning offers new insights into the use of emerging data analysis and adaptive techniques

341 137 36MB Read more

SQL for Data Science: Data Cleaning, Wrangling and Analytics with Relational Databases [1 ed.] 9783030575915, 9783030575922, 3030575918

This textbook explains SQL within the context of data science and introduces the different parts of SQL as they are need

6,898 798 3MB Read more