Efficient Go: Data-Driven Performance Optimization 9781098105716

Table of contents :
Preface
Why I Wrote This Book
How I Gathered This Knowledge
Who This Book Is For
How This Book Is Organized
Conventions Used in This Book
Using Code Examples
Acknowledgments
Feedback Is Welcome!
O’Reilly Online Learning
How to Contact Us
1. Software Efficiency Matters
Behind Performance
Common Efficiency Misconceptions
Optimized Code Is Not Readable
Code after optimization can be more readable
Readability now versus past
You Aren’t Going to Need It
Hardware Is Getting Faster and Cheaper
Software expands to fill the available memory
Software gets slower more rapidly than hardware becomes faster
Technological limits
Faster execution is more energy efficient
We Can Scale Horizontally Instead
Time to Market Is More Important
The Key to Pragmatic Code Performance
Summary
2. Efficient Introduction to Go
Basics You Should Know About Go
Imperative, Compiled, and Statically Typed Language
Designed to Improve Serious Codebases
Governed by Google, Yet Open Source
Simplicity, Safety, and Readability Are Paramount
Packaging and Modules
Dependencies Transparency by Default
Consistent Tooling
Single Way of Handling Errors
Strong Ecosystem
Unused Import or Variable Causes Build Error
Unit Testing and Table Tests
Advanced Language Elements
Code Documentation as a First Citizen
Backward Compatibility and Portability
Go Runtime
Object-Oriented Programming
Generics
Is Go “Fast”?
Summary
3. Conquering Efficiency
Beyond Waste, Optimization Is a Zero-Sum Game
Reasonable Optimizations
Deliberate Optimizations
Optimization Challenges
Understand Your Goals
Efficiency Requirements Should Be Formalized
Resource-Aware Efficiency Requirements
Acquiring and Assessing Efficiency Goals
Example of Defining RAER
Got an Efficiency Problem? Keep Calm!
Optimization Design Levels
Efficiency-Aware Development Flow
Functionality Phase
1. Test functionality first
2. Do we pass the functional tests?
3. If the tests fail, we have to fix, implement, or design the missing parts
Efficiency Phase
4. Efficiency assessment
5. Are we within RAERs?
6. Find the main bottleneck
7. Choice of level
8. Optimize!
9. Release and enjoy!
Summary
4. How Go Uses the CPU Resource (or Two)
CPU in a Modern Computer Architecture
Assembly
Understanding Go Compiler
CPU and Memory Wall Problem
Hierachical Cache System
Pipelining and Out-of-Order Execution
Hyper-Threading
Schedulers
Operating System Scheduler
Go Runtime Scheduler
When to Use Concurrency
Summary
5. How Go Uses Memory Resource
Memory Relevance
Do We Have a Memory Problem?
Physical Memory
OS Memory Management
Virtual Memory
mmap Syscall
OS Memory Mapping
Go Memory Management
Values, Pointers, and Memory Blocks
Go Allocator
Garbage Collection
Summary
6. Efficiency Observability
Observability
Example: Instrumenting for Latency
Logging
Tracing
Metrics
Efficiency Metrics Semantics
Latency
CPU Usage
Memory Usage
runtime heap statistics
OS memory pages statistics
Summary
7. Data-Driven Efficiency Assessment
Complexity Analysis
“Estimated” Efficiency Complexity
Asymptotic Complexity with Big O Notation
Practical Applications
If you know precise complexity, you don’t need to measure to know expected resource requirements
It tells us if there is any easy optimization to our code
It helps us assess ideas for a better algorithm as an optimization
It tells us where the bottleneck is and what part of the algorithm is critical
The Art of Benchmarking
Comparison to Functional Testing
Benchmarks Lie
Reliability of Experiments
Human Errors
Reproducing Production
Performance Nondeterminism
Benchmarking Levels
Benchmarking in Production
Macrobenchmarks
Microbenchmarks
What Level Should You Use?
Summary
8. Benchmarking
Microbenchmarks
Go Benchmarks
Understanding the Results
Tips and Tricks for Microbenchmarking
Too-High Variance
Find Your Workflow
Test Your Benchmark for Correctness!
Sharing Benchmarks with the Team (and Your Future Self)
Running Benchmarks for Different Inputs
Microbenchmarks Versus Memory Management
Compiler Optimizations Versus Benchmark
Macrobenchmarks
Basics
Go e2e Framework
Understanding Results and Observations
Server-side latency
CPU time
Memory
Common Macrobenchmarking Workflows
Summary
9. Data-Driven Bottleneck Analysis
Root Cause Analysis, but for Efficiency
Profiling in Go
pprof Format
go tool pprof Reports
Top
Graph
Flame Graph
Peek, Source, and Disassemble
Capturing the Profiling Signal
Common Profile Instrumentation
Heap
Goroutine
CPU
Off-CPU Time
Tips and Tricks
Sharing Profiles
Continuous Profiling
Comparing and Aggregating Profiles
Summary
10. Optimization Examples
Sum Examples
Optimizing Latency
Optimizing bytes.Split
Optimizing runtime.slicebytetostring
Optimizing strconv.Parse
Optimizing Memory Usage
Moving to Streaming Algorithm
Optimizing bufio.Scanner
Optimizing Latency Using Concurrency
A Naive Concurrency
A Worker Approach with Distribution
A Worker Approach Without Coordination (Sharding)
A Streamed, Sharded Worker Approach
Bonus: Thinking Out of the Box
Summary
11. Optimization Patterns
Common Patterns
Do Less Work
Trading Functionality for Efficiency
Trading Space for Time
Trading Time for Space
The Three Rs Optimization Method
Reduce Allocations
Reuse Memory
Recycle
Don’t Leak Resources
Control the Lifecycle of Your Goroutines
Reliably Close Things
Exhaust Things
Pre-Allocate If You Can
Overusing Memory with Arrays
Memory Reuse and Pooling
Summary
Next Steps
A. Latencies for Napkin Math Calculations
Index

Polecaj historie

Efficient Go: Data-Driven Performance Optimization [1 ed.] 1098105710, 9781098105716

With technological advancements, fast markets, and higher complexity of systems, software engineers tend to skip the unc

494 118 9MB Read more

PostgreSQL 10 High Performance: Expert techniques for query optimization, high availability, and efficient database maintenance 1788474481, 9781788474481

Leverage the power of PostgreSQL 10 to design, administer and maintain a high-performance database solution Key Features

970 134 4MB Read more

PostgreSQL 10 high performance expert techniques for query optimization, high availability, and efficient database maintenance 9781788474481, 6481208432, 9781788472296, 9781788392013, 1788474481

PostgreSQL is increasingly utilized in all kind of applications, starting from desktop to web and mobile applications. I

795 188 3MB Read more

Hands-On High Performance with Go 9781789801774

2,885 503 11MB Read more

The Art of Performance Engineering: Learn Performance Optimization techniques to ensure high performance of your application

918 174 3MB Read more

Design and Performance Optimization of Renewable Energy Systems 0128216026, 9780128216026

Design and Performance Optimization of Renewable Energy Systems provides an integrated discussion of issues relating to

1,371 337 29MB Read more

PostgreSQL Query Optimization: The Ultimate Guide to Building Efficient Queries 9781484268858

719 74 10MB Read more

PostgreSQL Query Optimization: The Ultimate Guide to Building Efficient Queries 9781484268841, 9781484268858

4,643 799 10MB Read more

Mastering Entity Framework Core 2.0: Dive into entities, relationships, querying, performance optimization, and more, to learn efficient data-driven development 9781788296212, 1788296214

Learn how to leverage the features of the new Entity Framework Core APIs and use them to build pure .NET Core applicatio

649 79 23MB Read more

Hyperparameter Optimization in Machine Learning: Make Your Machine Learning and Deep Learning Models More Efficient 9781484265796

2,431 508 3MB Read more