Develop your own Bluetooth Low Energy Applications: for Raspberry Pi, ESP32 and nRF52 with Python, Arduino and Zephyr 3895765007, 9783895765001

Table of contents :
Search…
Develop your own Bluetooth Low Energy Applications
All rights reserved.
Content
Preface
1 • Introduction
1.1 What is Bluetooth Low Energy?
1.2 Layered architecture
1.3 How to communicate with BLE devices
1.3.1 Without a connection
1.3.2 With a connection
1.4 Advantages of BLE
1.4.1 Low power consumption
1.4.2 Ubiquitous
1.4.3 Low cost
1.5 Disadvantages of BLE
1.5.1 Short range
1.5.2 Limited speed
1.5.3 You need a gateway
1.6 Platforms used in this book
1.6.1 Python/Bleak (Raspberry Pi, PC)
1.6.2 C++/NimBLE-Arduino (ESP32)
1.6.3 C/Zephyr (nRF52)
1.7 How to use this book
1.8 Summary and further exploration
2 • Preparing your development environment
2.1 Python and Bleak on your PC or Raspberry Pi
2.2 The Arduino platform with NimBLE-Arduino for the ESP32
2.2.1 Install Arduino CLI
2.2.2 Install the ESP32 Arduino core
2.2.3 Detect your ESP32 board
2.2.4 Install the NimBLE-Arduino library
2.3 The Zephyr development environment for nRF5 devices
2.3.1 Build a Zephyr application
2.3.2 Flash a Zephyr application
2.4 The nRF Connect for Desktop application
2.5 The nRF Connect mobile app
2.6 The Bluetooth Low Energy app in nRF Connect for Desktop
2.7 Wireshark and a BLE sniff er dongle
2.7.1 Downloading Wireshark and the nRF Sniffer for Bluetooth LE
2.7.2 Installing the nRF Sniffer for Bluetooth LE firmware
2.7.3 Installing the nRF Sniff er capture tool
2.7.4 Installing the BLE profi le
2.7.5 Testing a BLE packet capture
2.8 Summary and further exploration
3 • Broadcasting data with advertisements
3.1 Device roles
3.2 Advertising packets
3.2.1 Advertising channels
3.2.2 Advertising packet structure
3.3 Discovering advertisements with Bleak
3.3.1 Scanning for devices
3.3.2 Detection callbacks
3.3.3 Active and passive scanning
3.4 Public and random Bluetooth addresses
3.5 The iBeacon specification
3.6 Decoding iBeacon advertisements using Bleak
3.7 Discovering advertisements with NimBLE-Arduino
3.8 Decoding manufacturer-specifi c data using NimBLE-Arduino
3.8.1 Decoding iBeacon advertisements
3.8.2 Decoding Microsoft advertising beacons
3.9 Broadcasting iBeacon advertisements with Zephyr
3.9.1 Advertising data structures in Zephyr
3.9.2 Enabling Bluetooth
3.9.3 Advertising
3.9.4 Building and fl ashing the code
3.9.5 Investigating the advertised packets
3.10 Broadcasting sensor data as manufacturer-specific data with Zephyr
3.10.1 Hardware
3.10.2 Project structure
3.10.3 Source code
3.10.4 Decoding the BME280 sensor data
3.11 Advertise scan response data with Zephyr
3.12 Summary and further exploration
4 • Connections and services
4.1 Device roles
4.2 Attributes
4.3 Services, characteristics, and descriptors
4.3.1 Services
4.3.2 Characteristics
4.3.3 Descriptors
4.4 Discovering services and characteristics with nRF Connect
4.5 A minimal GATT server
4.6 Discovering services and characteristics with Bleak
4.7 Reading and writing characteristics using Bleak
4.7.1 Reading characteristics
4.7.2 Reading characteristics by their handle
4.7.3 Writing characteristics
4.8 Notifi cations and indications
4.8.1 Read heart rate notifi cations
4.8.2 Read notifi cations from multiple devices
4.9 Creating a heart rate monitor with NimBLE-Arduino
4.10 Creating a GATT server with Zephyr
4.10.1 Exposing the Device Information service
4.10.2 Creating a BLE sensor with Zephyr
4.10.3 Reading the sensor characteristic
4.10.4 Sniffi ng packets in an unencrypted BLE connection
4.11 Receiving service data without a connection
4.11.1 Scanning for service data
4.11.2 Receiving Exposure Notifi cation advertisements
4.12 Summary and further exploration
5 • Securing BLE connections
5.1 BLE security architecture
5.2 Pairing and bonding
5.2.1 Phase 1: Exchange of pairing information
5.2.2 Phase 2: Pairing
5.2.2.1 LE Legacy Connection pairing
5.2.2.2 LE Secure Connection pairing
5.2.3 Phase 3: Bonding
5.3 Security modes and levels
5.4 Encrypting the BLE connection to a Zephyr sensor
5.4.1 Implementing Security Mode 1 Level 2
5.4.2 Securely connecting to your sensor board
5.4.3 Sniffi ng the pairing procedure with Wireshark
5.5 Authenticating a BLE connection
5.5.1 Implementing Secure Connections Only Mode
5.5.2 Securely connecting with the board
5.5.3 Sniffi ng the pairing procedure with Wireshark
5.6 Privacy
5.7 Summary and further exploration
6 • Profiles and roles
6.1 Common BLE profiles
6.1.1 Generic profiles
6.1.2 GATT profiles
6.2 Understanding a profile specification
6.2.1 Introduction
6.2.2 Configuration
6.2.3 Proximity Reporter Requirements
6.2.4 Proximity Monitor Requirements
6.2.5 Connection Establishment
6.2.6 Security Considerations
6.2.7 GATT Interoperability Requirements
6.2.8 Acronyms and Abbreviations
6.2.9 References
6.3 Understanding a service specification
6.3.1 Introduction
6.3.2 Service Declaration
6.3.3 Service Characteristics
6.3.4 Service Behaviors
6.3.5 Acronyms and Abbreviations
6.3.6 References
6.4 Understanding the definition of a characteristic
6.4.1 Description
6.4.2 Defi nition
6.5 Implementing a Proximity Reporter in Zephyr
6.6 Implementing a Proximity Monitor in NimBLE-Arduino
6.7 Summary and further exploration
7 • Reverse engineering BLE devices
7.1 Investigating the LED badge
7.2 Decompiling the mobile app
7.3 Sniffing BLE traffic between the LED badge and the mobile app
7.4 Writing arbitrary images to the LED badge using Bleak
7.4.1 Finding LED badges
7.4.2 Writing images to the LED badge
7.5 Summary and further exploration
8 • Lowering power consumption
8.1 Measuring power consumption with the Nordic Semiconductor Power Profiler Kit II
8.1.1 Ampere Meter mode
8.1.2 Source Meter mode
8.2 Measuring an iBeacon’s power consumption
8.3 Lowering power consumption by disabling hardware
8.4 Lowering the power consumption by using a larger advertising interval
8.5 Estimating battery life
8.6 Summary and further exploration
9 • Conclusion
9.1 Other BLE development platforms
9.2 More about Bluetooth Low Energy
9.3 Some ideas for further exploration
10 • Appendix
10.1 Where to find BLE specifications
10.2 16-bit UUID ranges
10.3 Verifying a product’s Bluetooth qualifications
10.4 Establishing a serial connection to a device over USB
10.4.1 Check the port
10.4.2 Install the USB-to-serial driver
10.4.3 Give the user access
10.4.4 Start the serial connection
10.5 Sniffing BLE traffic on your Android device using the Bluetooth HCI snoop log
10.5.1 Investigating the Bluetooth HCI snoop log file with Wireshark
10.5.2 Sniffi ng live BLE traffi c in Wireshark with the Android Debug Bridge
10.6 Tips for specifi c hardware
10.6.1 Programming boards that have the Adafruit nRF52 bootloader
10.6.2 Programming boards with Arduino BOSSA
Index

Citation preview

books books

books

for Raspberry Pi, ESP32 and nRF52 with Python, Arduino and Zephyr Bluetooth Low Energy (BLE) radio chips are ubiquitous from Raspberry Pi to lightbulbs. BLE is an elaborate technology with a comprehensive specification, but the basics are quite accessible. A progressive and systematic approach will lead you far in mastering this wireless communication technique, which is essential for working in low power scenarios. In this book, you’ll learn how to: > Discover BLE devices in the neighborhood by listening to their advertisements. > Create your own BLE devices advertising data. > Connect to BLE devices such as heart rate monitors and proximity reporters. > Create secure connections to BLE devices with encryption and authentication. > Understand BLE service and profile specifications and implement them. > Reverse engineer a BLE device with a proprietary implementation and control it with your own software. > Make your BLE devices use as little power as possible.

Koen Vervloesem has been writing for over 20 years on Linux, open-source software, security, home automation, AI, programming, and the Internet of Things. He holds a Master’s degree in Computer Science Engineering, a Master’s degree in Philosophy, and an LPIC-3 303 Security certificate. He is a board member of the Belgian privacy activist organisation the Ministry of Privacy.

This book shows you the ropes of BLE programming with Python and the Bleak library on a Raspberry Pi or PC, with C++ and NimBLE-Arduino on Espressif’s ESP32 development boards, and with C on one of the development boards supported by the Zephyr real-time operating system, such as Nordic Semiconductor's nRF52 boards. Starting with a very little amount of theory, you’ll develop code right from the beginning. After you’ve completed this book, you’ll know enough to create your own BLE applications. Elektor International Media BV www.elektor.com

Develop your own Bluetooth Low Energy Applications • Koen Vervloesem

Develop your own Bluetooth Low Energy Applications

BLE

Develop your own Bluetooth Low Energy Applications for Raspberry Pi, ESP32 and nRF52 with Python, Arduino and Zephyr

Koen Vervloesem

Develop your own

Bluetooth Low Energy Applications for Raspberry Pi, ESP32 and nRF52 with Python, Arduino and Zephyr

● Koen Vervloesem

Boek BLE 220329 UK.indd 3

06/05/2022 15:54

Bluetooth Low Energy Applications

● This is an Elektor Publication. Elektor is the media brand of Elektor International Media B.V.

PO Box 11, NL-6114-ZG Susteren, The Netherlands Phone: +31 46 4389444

● All rights reserved. No part of this book may be reproduced in any material form, including photocopying, or

storing in any medium by electronic means and whether or not transiently or incidentally to some other use of this publication, without the written permission of the copyright holder except in accordance with the provisions of the Copyright Designs and Patents Act 1988 or under the terms of a licence issued by the Copyright Licencing Agency Ltd., 90 Tottenham Court Road, London, England W1P 9HE. Applications for the copyright holder's permission to reproduce any part of the publication should be addressed to the publishers.

● Declaration

The Author and Publisher have used their best efforts in ensuring the correctness of the information contained in this book. They do not assume, and hereby disclaim, any liability to any party for any loss or damage caused by errors or omissions in this book, whether such errors or omissions result from negligence, accident, or any other cause.

● British Library Cataloguing in Publication Data

A catalogue record for this book is available from the British Library

● ISBN 978-3-89576-500-1 ISBN 978-3-89576-501-8

Print

eBook

● © Copyright 2022: Elektor International Media B.V. (2022-05 / 1st) Prepress Production: D-Vision, Julian van den Berg

Printed in the Netherlands by Ipskamp Printing, Enschede

Elektor is part of EIM, the world's leading source of essential technical information and electronics products for pro engineers, electronics designers, and the companies seeking to engage them. Each day, our international team develops and delivers high-quality content - via a variety of media channels (including magazines, video, digital media, and social media) in several languages - relating to electronics design and DIY electronics. www.elektormagazine.com

●4

Boek BLE 220329 UK.indd 4

06/05/2022 15:54

Content Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Chapter 1 • Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.1 What is Bluetooth Low Energy? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 1.2 Layered architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.3 How to communicate with BLE devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.3.1 Without a connection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.3.2 With a connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.4 Advantages of BLE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.4.1 Low power consumption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.4.2 Ubiquitous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.4.3 Low cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.5 Disadvantages of BLE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.5.1 Short range . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.5.2 Limited speed. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.5.3 You need a gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.6 Platforms used in this book. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.6.1 Python/Bleak (Raspberry Pi, PC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 1.6.2 C++/NimBLE-Arduino (ESP32) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 1.6.3 C/Zephyr (nRF52) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 1.7 How to use this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 1.8 Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Chapter 2 • Preparing your development environment . . . . . . . . . . . . . . . . . . . . . . 24 2.1 Python and Bleak on your PC or Raspberry Pi . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.2 The Arduino platform with NimBLE-Arduino for the ESP32 . . . . . . . . . . . . . . . . . . 25 2.2.1 Install Arduino CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.2.2 Install the ESP32 Arduino core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.2.3 Detect your ESP32 board . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.2.4 Install the NimBLE-Arduino library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.3 The Zephyr development environment for nRF5 devices . . . . . . . . . . . . . . . . . . . . 30 2.3.1 Build a Zephyr application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.3.2 Flash a Zephyr application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

●5

Boek BLE 220329 UK.indd 5

06/05/2022 15:54

Bluetooth Low Energy Applications 2.4 The nRF Connect for Desktop application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.5 The nRF Connect mobile app. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 2.6 The Bluetooth Low Energy app in nRF Connect for Desktop . . . . . . . . . . . . . . . . . 34 2.7 Wireshark and a BLE sniffer dongle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.7.1 Downloading Wireshark and the nRF Sniffer for Bluetooth LE . . . . . . . . . . . . 36 2.7.2 Installing the nRF Sniffer for Bluetooth LE firmware . . . . . . . . . . . . . . . . . . 36 2.7.3 Installing the nRF Sniffer capture tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 2.7.4 Installing the BLE profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 2.7.5 Testing a BLE packet capture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 2.8 Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Chapter 3 • Broadcasting data with advertisements . . . . . . . . . . . . . . . . . . . . . . . . 43 3.1 Device roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.2 Advertising packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.2.1 Advertising channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.2.2 Advertising packet structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 3.3 Discovering advertisements with Bleak. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.3.1 Scanning for devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.3.2 Detection callbacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.3.3 Active and passive scanning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 3.4 Public and random Bluetooth addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.5 The iBeacon specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.6 Decoding iBeacon advertisements using Bleak . . . . . . . . . . . . . . . . . . . . . . . . . . 58 3.7 Discovering advertisements with NimBLE-Arduino . . . . . . . . . . . . . . . . . . . . . . . . 61 3.8 Decoding manufacturer-specific data using NimBLE-Arduino . . . . . . . . . . . . . . . . . 66 3.8.1 Decoding iBeacon advertisements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 3.8.2 Decoding Microsoft advertising beacons. . . . . . . . . . . . . . . . . . . . . . . . . . . 69 3.9 Broadcasting iBeacon advertisements with Zephyr. . . . . . . . . . . . . . . . . . . . . . . . 73 3.9.1 Advertising data structures in Zephyr . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 3.9.2 Enabling Bluetooth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 3.9.3 Advertising. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 3.9.4 Building and flashing the code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

●6

Boek BLE 220329 UK.indd 6

06/05/2022 15:54

3.9.5 Investigating the advertised packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 3.10 Broadcasting sensor data as manufacturer-specific data with Zephyr . . . . . . . . . . 82 3.10.1 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 3.10.2 Project structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 3.10.3 Source code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 3.10.4 Decoding the BME280 sensor data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 3.11 Advertise scan response data with Zephyr . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 3.12 Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Chapter 4 • Connections and services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 4.1 Device roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 4.2 Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 4.3 Services, characteristics, and descriptors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 4.3.1 Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 4.3.2 Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 4.3.3 Descriptors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 4.4 Discovering services and characteristics with nRF Connect . . . . . . . . . . . . . . . . . 101 4.5 A minimal GATT server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 4.6 Discovering services and characteristics with Bleak . . . . . . . . . . . . . . . . . . . . . . 105 4.7 Reading and writing characteristics using Bleak. . . . . . . . . . . . . . . . . . . . . . . . . 108 4.7.1 Reading characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 4.7.2 Reading characteristics by their handle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 4.7.3 Writing characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 4.8 Notifications and indications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 4.8.1 Read heart rate notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 4.8.2 Read notifications from multiple devices . . . . . . . . . . . . . . . . . . . . . . . . . 119 4.9 Creating a heart rate monitor with NimBLE-Arduino. . . . . . . . . . . . . . . . . . . . . . 124 4.10 Creating a GATT server with Zephyr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 4.10.1 Exposing the Device Information service . . . . . . . . . . . . . . . . . . . . . . . . 131 4.10.2 Creating a BLE sensor with Zephyr . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 4.10.3 Reading the sensor characteristic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 4.10.4 Sniffing packets in an unencrypted BLE connection . . . . . . . . . . . . . . . . . 148

●7

Boek BLE 220329 UK.indd 7

06/05/2022 15:54

Bluetooth Low Energy Applications 4.11 Receiving service data without a connection . . . . . . . . . . . . . . . . . . . . . . . . . . 149 4.11.1 Scanning for service data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 4.11.2 Receiving Exposure Notification advertisements . . . . . . . . . . . . . . . . . . . 151 4.12 Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 Chapter 5 • Securing BLE connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 5.1 BLE security architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 5.2 Pairing and bonding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 5.2.1 Phase 1: Exchange of pairing information . . . . . . . . . . . . . . . . . . . . . . . . 158 5.2.2 Phase 2: Pairing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 5.2.2.1 LE Legacy Connection pairing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 5.2.2.2 LE Secure Connection pairing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 5.2.3 Phase 3: Bonding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 5.3 Security modes and levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 5.4 Encrypting the BLE connection to a Zephyr sensor. . . . . . . . . . . . . . . . . . . . . . . 165 5.4.1 Implementing Security Mode 1 Level 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 166 5.4.2 Securely connecting to your sensor board . . . . . . . . . . . . . . . . . . . . . . . . 173 5.4.3 Sniffing the pairing procedure with Wireshark . . . . . . . . . . . . . . . . . . . . . 175 5.5 Authenticating a BLE connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 5.5.1 Implementing Secure Connections Only Mode . . . . . . . . . . . . . . . . . . . . . 177 5.5.2 Securely connecting with the board. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 5.5.3 Sniffing the pairing procedure with Wireshark . . . . . . . . . . . . . . . . . . . . . 188 5.6 Privacy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 5.7 Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 Chapter 6 • Profiles and roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 6.1 Common BLE profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 6.1.1 Generic profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 6.1.2 GATT profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192 6.2 Understanding a profile specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 6.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 6.2.2 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 6.2.3 Proximity Reporter Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

●8

Boek BLE 220329 UK.indd 8

06/05/2022 15:54

6.2.4 Proximity Monitor Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 6.2.5 Connection Establishment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 6.2.6 Security Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 6.2.7 GATT Interoperability Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 6.2.8 Acronyms and Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 6.2.9 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 6.3 Understanding a service specification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 6.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 6.3.2 Service Declaration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 6.3.3 Service Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 6.3.4 Service Behaviors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 6.3.5 Acronyms and Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 6.3.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 6.4 Understanding the definition of a characteristic . . . . . . . . . . . . . . . . . . . . . . . . . 203 6.4.1 Description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 6.4.2 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 6.5 Implementing a Proximity Reporter in Zephyr . . . . . . . . . . . . . . . . . . . . . . . . . . 204 6.6 Implementing a Proximity Monitor in NimBLE-Arduino . . . . . . . . . . . . . . . . . . . . 210 6.7 Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216 Chapter 7 • Reverse engineering BLE devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 7.1 Investigating the LED badge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 7.2 Decompiling the mobile app . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 7.3 Sniffing BLE traffic between the LED badge and the mobile app . . . . . . . . . . . . . 222 7.4 Writing arbitrary images to the LED badge using Bleak . . . . . . . . . . . . . . . . . . . 225 7.4.1 Finding LED badges. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 7.4.2 Writing images to the LED badge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 7.5 Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 Chapter 8 • Lowering power consumption. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 8.1 Measuring power consumption with the Nordic Semiconductor Power Profiler Kit II 232 8.1.1 Ampere Meter mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 8.1.2 Source Meter mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

●9

Boek BLE 220329 UK.indd 9

06/05/2022 15:54

Bluetooth Low Energy Applications 8.2 Measuring an iBeacon’s power consumption . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 8.3 Lowering power consumption by disabling hardware . . . . . . . . . . . . . . . . . . . . . 237 8.4 Lowering the power consumption by using a larger advertising interval . . . . . . . . 238 8.5 Estimating battery life . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 8.6 Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240 Chapter 9 • Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 9.1 Other BLE development platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 9.2 More about Bluetooth Low Energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 9.3 Some ideas for further exploration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244 Chapter 10 • Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 10.1 Where to find BLE specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 10.2 16-bit UUID ranges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 10.3 Verifying a product’s Bluetooth qualifications . . . . . . . . . . . . . . . . . . . . . . . . . 247 10.4 Establishing a serial connection to a device over USB . . . . . . . . . . . . . . . . . . . . 248 10.4.1 Check the port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248 10.4.2 Install the USB-to-serial driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248 10.4.3 Give the user access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 10.4.4 Start the serial connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 10.5 Sniffing BLE traffic on your Android device using the Bluetooth HCI snoop log. . . 250 10.5.1 Investigating the Bluetooth HCI snoop log file with Wireshark . . . . . . . . . 250 10.5.2 Sniffing live BLE traffic in Wireshark with the Android Debug Bridge . . . . . 251 10.6 Tips for specific hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 10.6.1 Programming boards that have the Adafruit nRF52 bootloader . . . . . . . . . 251 10.6.2 Programming boards with Arduino BOSSA . . . . . . . . . . . . . . . . . . . . . . . 253 Index

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

● 10

Boek BLE 220329 UK.indd 10

06/05/2022 15:54

Preface

Preface Bluetooth Low Energy (BLE) is one of the most accessible wireless communication standards. You don’t need any expensive equipment to develop BLE devices such as wireless sensor boards, proximity beacons, or heart rate monitors. All you need is a computer or a Raspberry Pi, an ESP32 microcontroller board, or a development board with a Nordic Semiconductor nRF5 (or an equivalent BLE SoC from another manufacturer). On the software side, BLE is similarly accessible. Many development platforms, most of them open source, offer an API (application programming interface) to assist you in developing your own BLE applications. This book shows you the ropes of BLE programming with Python and the Bleak library on a Raspberry Pi or PC, with C++ and NimBLE-Arduino on Espressif’s ESP32 development boards, and with C and the Zephyr real-time operating system on Nordic Semiconductor’s nRF52 boards. While Bluetooth Low Energy is a complex technology with a comprehensive specification, getting started with the basics is relatively easy. This book takes a practical approach to BLE programming to make the technology even more approachable. With a minimal amount of theory, you’ll develop code right from the start. After you’ve completed this book, you’ll know enough to create your own BLE applications. Koen Vervloesem, February 2022

● 11

Boek BLE 220329 UK.indd 11

06/05/2022 15:54

Bluetooth Low Energy Applications

Chapter 1 • Introduction Bluetooth Low Energy (BLE) is a complex technology, but the basics are quite accessible. In this book, you’ll learn how to: • • • • • •

discover BLE devices in the neighborhood by listening to their advertisements create your own BLE devices advertising data connect to BLE devices such as heart rate monitors and proximity reporters create secure connections to BLE devices with encryption and authentication understand BLE service and profile specifications and implement them reverse engineer a BLE device with a proprietary implementation and control it with your own software • reverse engineer a BLE device with a proprietary implementation and control it with your own software • make your BLE devices use as little power as possible This chapter introduces Bluetooth Low Energy and its advantages. It also lists the software and hardware platforms used in this book, as well as the reasons to choose them.

1.1 What is Bluetooth Low Energy? Bluetooth is a wireless communication standard in the 2.4 GHz Industrial, Scientific, and Medical (ISM) frequency band. These days, if you hear about Bluetooth support in a product, it is almost always Bluetooth Low Energy (BLE). It’s a radical departure from the original Bluetooth standard, which is now called Classic Bluetooth. Bluetooth Low Energy and Classic Bluetooth are actually different protocols. Classic Bluetooth is essentially a wireless version of the traditional serial connection. If you want to print a document, transfer a file or stream audio, you want this to happen as fast as possible. Therefore, the focus of development in Classic Bluetooth was on attaining faster and faster speeds with every new version. However, Classic Bluetooth wasn’t a good fit for devices with low power consumption, for instance those powered by batteries. That’s why Nokia adapted the Bluetooth standard to enable it to work in low-power scenarios. In 2006, they released their resulting technology onto the market, dubbed Wibree. The Bluetooth Special Interest Group (SIG), the organization that maintains the Bluetooth specifications, showed interest in this new development. After consulting with Nokia, they decided to adopt Wibree as part of Bluetooth 4.0, with the new name, Bluetooth Low Energy. Classic Bluetooth remained available for high-throughput applications. Note: In practice, many chipsets support both Classic Bluetooth and Bluetooth Low Energy, especially in laptops and smartphones.

● 12

Boek BLE 220329 UK.indd 12

06/05/2022 15:54

Chapter 1 • Introduction

1.2 Layered architecture The Bluetooth Core Specification (https://www.bluetooth.com/specifications/specs/ core-specification/) is more than 3200 pages long. And this is only the core specification; there are many supplemental documents for BLE. However, BLE has a layered architecture. Many end-user applications only use the upper layers, so you don’t need to know the details of the architecture’s lower layer. This book doesn’t refer to the Bluetooth Core Specification, but mainly to the supplemental specifications, which concern the upper layers.

Figure 1.1 The layered architecture of Bluetooth Low Energy The BLE architecture consists of three main blocks:tController, Host, andnApplication. Controller This has the lower-level layers: the Physical Layer (PHY), Link Layer (LL) and Direct Test Mode (DTM). These are the layers where the Bluetooth radio does its work. The controller communicates with the outside world using the antenna, in a frequency band around 2.4 GHz. It communicates with the host using a standardized interface between the two blocks – the Host Controller Interface (HCI). 1 Host This is the block with which the end user or application developer comes in contact. The Logical Link Control and Adaptation Protocol (L2CAP) defines channels and signaling commands. On top of it, the Security Manager Protocol (SMP) handles secure connections (with authentication and encryption), and the Attribute Protocol (ATT) defines how to expose and access data as attributes. The Generic Attribute Profile (GATT) 2 builds on the Attribute Protocol to define how to discover services and their characteristics and how to read and write their values. The upper layer of the Host block is the Generic Access Profile (GAP), which defines how devices can discover other devices and connect, pair, and bond to them. The host communicates with the controller using its part of the host controller interface, and applications communicate with the host depending on the APIs exposed by the operating system.

1 2

As the HCI is an interface between Host and Controller, it’s shown in both layers Yes, GATT doesn’t look like an acronym for Generic Attribute Profile, but the Bluetooth specification uses this because GAP is already taken by Generic Access Profile. You’ll get used to it.

● 13

Boek BLE 220329 UK.indd 13

06/05/2022 15:54

Bluetooth Low Energy Applications

Application This layer builds on top of the Generic Attribute Profile to implement application-specific characteristics, services, and profiles. A characteristic defines a specific type of data, such as an Alert Level. A service defines a set of characteristics and their behaviors, such as the Link Loss Service. A profile is a specification that describes how two or more devices with one or more services communicate with each other. An example is the Proximity profile, which has two roles: Proximity Monitor and Proximity Reporter. The application layer is where the example programs in this book reside, and they make use of the layers in the Host block. The three blocks don’t have to run on the same processor. In fact, there are three common configurations – one single-chip and two dual-chip: Single-chip (SoC) Controller, host, and application code run on the same chip. The host and controller communicate through function calls and queues in the chip’s RAM. Most simple devices such as BLE sensors use this configuration; it keeps the cost down. Some smartphones also use this configuration if they have a SoC with Bluetooth built in. Dual-chip over HCI A dual-chip solution with application and host on one chip, and the controller on another chip, communicates over HCI. Because HCI is a standardized interface, it lets you combine different platforms. For instance, on a Raspberry Pi, the Wi-Fi and BLE chip implements a BLE controller. If you connect a BLE dongle to an older Raspberry Pi, this dongle also implements a BLE controller. 3 BlueZ, the Raspberry Pi Linux kernel’s Bluetooth stack, implements a BLE host. So, BlueZ communicates with the BLE controller in the built-in BLE chip or on the BLE dongle (see Figure 1.2). In the former case, the HCI uses SDIO, and in the latter, UART over USB. 4 Many smartphones and tablets also use the dual-chip over HCI configuration, with a powerful processor running the host and a Bluetooth chip running the controller. Dual-chip with connectivity device Another dual-chip solution is one with the application running on one chip and the host and controller on another chip. The latter is then called the connectivity device because it adds BLE connectivity to the other device. This approach is useful if you have an existing hardware device that you want to extend with BLE connectivity. Because there’s no standardized interface in this case, the communication between the application processor and the connectivity device needs to make use of a proprietary protocol implemented by the connectivity device. A three-chip solution with controller, host, and application each running on its own chip is also possible. However, because of the associated cost, this is typically only done for development systems. 3 4

In fact, you can easily create your own BLE dongle with an nRF52840 Dongle and the controller part of Zephyr’s BLE stack. Just build the sample samples/bluetooth/hci_usb from the Zephyr samples and flash it to your dongle. The next chapter explains how to do this BlueZ even supports multiple controllers simultaneously.

● 14

Boek BLE 220329 UK.indd 14

06/05/2022 15:54

Chapter 1 • Introduction

Figure 1.2 A dual-chip implementation of the BLE architecture

1.3 How to communicate with BLE devices Bluetooth Low Energy has two ways to communicate between devices: with and without a connection.

1.3.1 Without a connection Without a connection means that the device just broadcasts information in an advertisement. Every BLE device in the neighborhood is able to receive this information. Some examples of BLE devices broadcasting data are: Proximity beacons These devices, often following Apple’s iBeacon standard, broadcast their ID. Receivers calculate their approximate distance to the beacons based on the advertisement’s Received Signal Strength Indicator (RSSI). Sensors Many temperature and humidity sensors broadcast their sensor values. Most devices do this in an unencrypted fashion, but some of them encrypt the data to prevent it being read by every device in the neighborhood. Mobile phones After the COVID-19 pandemic started in 2020, Google and Apple collaborated on the Exposure Notifications standard for contact tracing. As part of this technology, Android phones and iPhones broadcast unique (but anonymous) numbers. Other phones can pick up these numbers and use them later to warn users that they have been in contact with someone who is known to have had COVID-19.

1.3.2 With a connection The other way to communicate between BLE devices is with a connection. One device (the client) scans for BLE advertisements to find the device it wants to connect to. Then, optionally, it may do an active scan to ask the device (the server) which services are offered. After the client connects to the server, the client can use the server’s services. Each BLE service is a container of specific data from the server. You can read this data, or (with some services) write a value to the server.

● 15

Boek BLE 220329 UK.indd 15

06/05/2022 15:54

Bluetooth Low Energy Applications

Some examples of BLE devices using a connection are: Fitness trackers Your smartphone can connect to a fitness tracker and read your heart rate, the tracker’s battery level, and other measurements. Sensors Some environmental sensors let you read their sensor values over a BLE connection. Proximity reporters These devices sound an alert when their connection to another device is lost.

1.4 Advantages of BLE 1.4.1 Low power consumption As its name implies, Bluetooth Low Energy is optimized for low-power applications. Its whole architecture is designed to reduce power consumption. For instance, setting up a connection, reading or writing data, and disconnecting happens in a couple of milliseconds. The radio is often the most energy-consuming part of a device. Therefore, the idea is to turn on the Bluetooth radio, create a connection, read or write data, disconnect, and turn off the radio again until the next time the device has to communicate. This way, a well-designed BLE temperature sensor is able to work on a coin cell for ten years or more. You can use the same approach with other wireless technologies, such as Wi-Fi, but they require more power and take more time to set up a connection.

1.4.2 Ubiquitous BLE radio chips are ubiquitous. You can find them in smartphones, tablets, and laptops. This means that all those devices can talk to your BLE sensors or lightbulbs. Most manufacturers create mobile apps to control their BLE devices. You can also find BLE radios in many single-board computers, such as the Raspberry Pi, and in popular microcontroller platforms such as the ESP32. 5 This makes it quite easy for you to create your own gateways for BLE devices. And, platforms such as the Nordic Semiconductor nRF5 series of microcontrollers with BLE radio even make it possible to create your own battery-powered BLE devices.

1.4.3 Low cost There’s no cost to access the official BLE specifications. Moreover, BLE chips are cheap, and the available development boards (based on an nRF5 or an ESP32) and Raspberry Pis are quite affordable. This means you can just start with BLE programming at minimal cost.

5

The first Raspberry Pi with built-in BLE was the Raspberry Pi Model 3B (released in 2016).

● 16

Boek BLE 220329 UK.indd 16

06/05/2022 15:54

Chapter 1 • Introduction

1.5 Disadvantages of BLE 1.5.1 Short range BLE has a short range (for most devices, less than 10 meters) compared to other wireless networks, such as Zigbee, Z-Wave, and Thread. It’s not a coincidence that these competitors all have a mesh architecture, in which devices can forward their neighbors’ messages in order to improve range. Low-power wide area networks (LPWANs), such as LoRaWAN, Sigfox, and NB-IoT, have even longer ranges. In 2017, the Bluetooth SIG added Bluetooth Mesh, a mesh protocol. This builds upon BLE’s physical and link layers with a whole new stack above them. However, Bluetooth Mesh isn’t as well-established as the core BLE protocol, at least not for home use.

1.5.2 Limited speed The BLE radio has a limited transmission speed. For Bluetooth 4.2 and earlier, this is 1 Mbps, while for Bluetooth 5 and later, this can be up to 2 Mbps. This makes BLE unsuitable for high-bandwidth applications.

1.5.3 You need a gateway Wi-Fi devices have their own IP addresses, so you can communicate with them directly from other IP-based devices, and they are integrated into your LAN (local area network). Bluetooth doesn’t have this: to integrate your BLE devices with other network devices, you need a gateway. This device has to translate Bluetooth packets to IP-based protocols such as MQTT (Message Queuing Telemetry Transport). That’s why many BLE device manufacturers have smartphone apps that function as device gateways. 6

1.6 Platforms used in this book This book focuses on Bluetooth Low Energy programming on three platforms: Programming

Library

language Python

Bleak

Software

Hardware

platform

platform

Windows, Linux,

Raspberry Pi or PC

macOS C++ C

NimBLE-Arduino

Arduino framework

ESP32

Zephyr7

nRF52

Table 1.1 BLE platforms used in this book These choices were made in order to demonstrate a wide range of applications compatible with many software and hardware platforms.

6 7

You can run IPv6 over BLE with 6LoWPAN, but this isn’t yet well-established. A more developed solution in the same 2.4 GHz frequency band is Thread, which uses 6LoWPAN over the IEEE 802.15.4 wireless protocol. There’s no library for Zephyr in this table because the Zephyr operating system includes all functionality we need for BLE development.

● 17

Boek BLE 220329 UK.indd 17

06/05/2022 15:54

Bluetooth Low Energy Applications

1.6.1 Python/Bleak (Raspberry Pi, PC) Python is an easy-to-use programming language that works on all major operating systems. There are a lot of Python Bluetooth Low Energy libraries, but many of them support only a single operating system. Bleak (https://bleak.readthedocs.io), which stands for Bluetooth Low Energy platform Agnostic Klient, is a welcome exception. It supports: • Windows 10, version 16299 (Fall Creators Update) or higher • Linux distributions with BlueZ 5.43 or higher (also on a Raspberry Pi) • OS X 10.11 (El Capitan) or macOS 10.12+

Figure 1.3 With Python and Bleak, a Raspberry Pi is the perfect small computer to communicate with your BLE devices. Bleak is a GATT client: it’s able to connect to BLE devices that act as GATT servers. It supports reading, writing, and getting notifications from GATT servers, and it’s also able to discover BLE devices and read advertising data broadcast by them. While Bleak doesn’t implement a GATT server, in practice this isn’t a big limitation. GATT servers are typically implemented on constrained devices, so, for this purpose, the ESP32 and nRF52 hardware platforms are a better match. 8

1.6.2 C++/NimBLE-Arduino (ESP32) If you’re looking at microcontrollers, the Arduino framework has become quite popular, not only on the original Arduino boards, which don’t have BLE functionality, but also on ESP32 development boards, which do.

8

If you really want to implement a GATT server on a Raspberry Pi or your PC, have a look at Bless (https://github.com/kevincar/bless). This is an acronym for Bluetooth Low Energy Server Supplement, and the library follows a similar API style to Bleak.

● 18

Boek BLE 220329 UK.indd 18

06/05/2022 15:54

Chapter 1 • Introduction

Figure 1.4 Espressif’s ESP32-PICO-KIT v4 is one of the many ESP32 development boards that you can use with NimBLE-Arduino for BLE development. Programming for the Arduino framework is done in a variant of C++, but the framework and many Arduino libraries hide much of C++’s complexity. Even if you only know some C (which is much less complex than C++), you’ll be able to use the Arduino framework. One of the more popular BLE libraries for Arduino on the ESP32 is NimBLE-Arduino (https:// github.com/h2zero/NimBLE-Arduino). It’s a fork of NimBLE (https://mynewt.apache.org/ latest/network/), which is part of the Apache Mynewt real-time operating system. With NimBLE-Arduino, you can easily create your own GATT server or client. Note: If you prefer to use Espressif’s ESP-IDF instead of the Arduino framework, you can use esp-nimble-cpp (https://github.com/h2zero/esp-nimble-cpp). In fact, NimBLE-Arduino and esp-nimble-cpp use the same API. This means that many NimBLE-Arduino examples in this book can be reused with ESP-IDF with only a few minor changes.

1.6.3 C/Zephyr (nRF52) For even more constrained devices, typically battery-powered, you need a specialized realtime operating system (RTOS). This book uses the Zephyr Project (https://zephyrproject. org) on nRF52840-based devices from Nordic Semiconductor. Zephyr has a completely open-source Bluetooth Low Energy stack.

Figure 1.5 Nordic Semiconductor’s nRF52840 Dongle is an affordable development platform for BLE applications with Zephyr. Zephyr’s BLE stack is highly configurable. You can build Zephyr firmware for three configuration types:

● 19

Boek BLE 220329 UK.indd 19

06/05/2022 15:54

Bluetooth Low Energy Applications

Combined build Builds the BLE controller, BLE host, and your application for a one-chip configuration. Host build Builds the BLE host and your application, along with an HCI driver to let your device communicate with an external BLE controller on another chip. 9 Controller build Builds the BLE controller with an HCI driver to let your device communicate with an external BLE host on another chip. With some basic knowledge of C, you can create your own BLE devices with Zephyr, such as BLE beacons, sensor boards, and proximity reporters. Zephyr has extensive documentation of its Bluetooth API, as well as a lot of ready-to-use examples that you can build upon. For the examples in this book, you’ll use the combined build type.

1.7 How to use this book This is a book about programming. I expect you to have some programming experience. Knowledge of C, C++, and Python would be perfect. However, if you only know one or two of these programming languages, that won’t be a problem. The necessary C++ knowledge is minimal, so a basic knowledge of Python and C will suffice. You need some (affordable) hardware for the examples in this book: • The Python examples are hardware-agnostic due to the use of Bleak. So, whether you want to use a Raspberry Pi or another Linux computer, a Windows computer, or a Mac, it won’t make much difference. • For the NimBLE-Arduino examples, any ESP32 development board will suffice.

10

• For the Zephyr examples, I recommend a Nordic Semiconductor nRF52840 Dongle, but you can use any BLE microcontroller supported by Zephyr. Some other manufacturers have variants of the dongle, such as the April USB Dongle 52840 (with an external antenna) or the nRF52840 MDK USB Dongle from makerdiary (with a case). Consult the appendix at the end of this book if you want to use one of these, because they come with a different bootloader.

9 10

For instance, on Nordic Semiconductor devices, the BLE controller can be a SoftDevice controller. Note that the ESP8266, the ESP32’s predecessor, isn’t suitable for this book; it doesn’t have BLE.

● 20

Boek BLE 220329 UK.indd 20

06/05/2022 15:54

Chapter 1 • Introduction

Figure 1.6 From left to right: April Brother's April USB Dongle 52840, Nordic Semiconductor's nRF52840 Dongle, and makerdiary's nRF52840 MDK USB Dongle.. If you’re really serious about BLE development with Zephyr, I also recommend Nordic Semiconductor’s nRF52840 Development Kit. This is the bigger (and slightly less affordable) brother of the nRF52840 Dongle, offering more capabilities to debug your code easily. If you have one of these, you can do your initial development on it, and move your code to the nRF52840 Dongle when you’ve solved the most complex issues.

Figure 1.7 Nordic Semiconductor’s nRF52840 Development Kit has some powerful debugging capabilities. So, at the minimum, you need your computer, one ESP32 development board and one BLE microcontroller supported by Zephyr. Note: I recommend buying a couple of Nordic Semiconductor nRF52840 Dongles. As you’ll see in the next chapter, with the right firmware, you can turn these into adapters to explore other BLE devices and connect to them. You can also turn them into BLE sniffers that pick up BLE packets in the air and sniff traffic between other devices. They’re quite affordable too, with a price of around €10.

● 21

Boek BLE 220329 UK.indd 21

06/05/2022 15:54

Bluetooth Low Energy Applications

At the end of this book, you should be able to create your own BLE software based on the examples here. Here’s a short overview of what I’ll cover in this book: Chapter 1: Introduction An introduction to Bluetooth Low Energy, its layered architecture, and its advantages and disadvantages Chapter 2: Preparing your development environment A preparation of your development environment for Python/Bleak, Arduino/NimBLE-Arduino, and Zephyr, as well as some useful tools for exploring BLE devices, and a BLE packet sniffer in Wireshark Chapter 3: Broadcasting data with advertisements A deep dive into advertising packets, which are fundamental for device discovery, but also a powerful way to broadcast data without the need to connect to a BLE device Chapter 4: Connections and services The use of BLE connections to read and write characteristics and receive notifications and indications from peripherals Chapter 5: Securing BLE connections Security measures to encrypt and authenticate BLE connections and use resolvable private addresses to thwart user tracking Chapter 6: Profiles and roles A step-by-step guide to implementing a BLE profile, as well as how to understand profile and service specifications Chapter 7: Reverse engineering BLE devices Reverse engineering a BLE LED badge by investigating its services and characteristics, decompiling the accompanying mobile app, and sniffing BLE traffic between the device and its mobile app Chapter 8: Lowering power consumption An investigation into and optimization of a BLE implementation’s power consumption Chapter 9: Conclusion A wrap-up of this book, with some resources for learning more about Bluetooth Low Energy and some ideas for further exploration Appendix Some specialized tips that could come in handy in various situations

● 22

Boek BLE 220329 UK.indd 22

06/05/2022 15:54

Chapter 1 • Introduction

Note: All of the code examples from this book are published at https://github.com/ koenvervloesem/bluetooth-low-energy-applications. Read the instructions in this GitHub repository for more information about how to download them. The repository also lists errors that may have been found in this book since its publication, as well as information about changes that impact the examples herein.

1.8 Summary and further exploration In this introductory chapter, I explained the basics of Bluetooth Low Energy and its architecture. I also explained the two different ways to communicate with BLE devices: with and without a connection. After this, you learned about some of the advantages and disadvantages of BLE. At the end of this chapter, I introduced the platforms used in this book, both hardware and software. Of course, these platforms aren’t the only ones you can use for BLE programming. For instance, I don’t cover BLE programming on mobile devices (Android or iOS). However, with the BLE knowledge learned from this book, you’ll be able to understand the BLE APIs offered by other platforms. In the next chapter, you’ll learn how to set these platforms up, so that your development environment is ready for the rest of the book.

● 23

Boek BLE 220329 UK.indd 23

06/05/2022 15:54

Bluetooth Low Energy Applications

Chapter 2 • Preparing your development environment To start developing BLE applications, you need a development environment. In this chapter, you’ll learn how to set up the tools needed for the rest of this book. These are: • • • •

Python and the Bleak library on your PC or Raspberry Pi the Arduino platform with NimBLE-Arduino for the ESP32; the Zephyr development environment for nRF5 devices the nRF Connect mobile app to explore BLE devices from your Android or iOS smartphone • nRF Connect for Desktop’s Bluetooth Low Energy app to explore BLE devices on your computer • Wireshark and a BLE sniffer dongle to sniff BLE traffic All of these tools are cross-platform: they work on Windows, Linux, and macOS (and the nRF Connect mobile app on Android and iOS). Note: I’m not advocating the use of any specific editor or integrated development environment (IDE) in this book. Consult your favorite editor/IDE’s documentation If you wish to use the examples from the book with it. 2.1 Python and Bleak on your PC or Raspberry Pi Many examples in this book are written in the Python programming language. If you’re running Linux, you probably already have Python installed by default. Just make sure that it’s Python 3 and not the deprecated Python 2. If you’re using Windows or macOS, download the latest Python version from the Python website (https://www.python.org). At the time of writing, this is Python 3.10.2. For Windows, choose the Windows installer, not the Windows embeddable package. Be sure to choose the correct version for your architecture (32-bit or 64-bit; probably the latter). After starting the installer, enable Add Python 3.x to PATH. This way, you’ll be able to run Python just by entering python from the command line. After finishing the installation, open a command line (in Windows: click Start, type cmd into the Search or Run line, and press Enter). Then, query Python’s version to check whether it runs: $ python3 --version Python 3.10.2

Warning: I’m a Linux user, and all of the examples in this book use Linux. So, the command prompt is $ and not C:\WINDOWS\system32, and paths are delineated by / and not by \. If necessary, make sure you change these examples for your own operating system.

● 24

Boek BLE 220329 UK.indd 24

06/05/2022 15:54

Chapter 2 • Preparing your development environment

The Windows installer also installs Python’s package manager, pip. This allows you to install extra Python packages, such as Bleak. If you’re using Linux, you’ll probably have to install pip yourself. On Ubuntu, it’s done like this: $ sudo apt install python3-pip

Some Linux distributions allow you to have both Python 2 and Python 3 installed simultaneously, as well as pip and Python packages for Python 2 and Python 3. The commands for Python and pip are then generally called python and pip for Python 2, and python3 and pip3 for Python 3. In this book, I’ll use python3 and pip3 to make sure that you’re using the correct version, should you happen to have both versions installed. Check whether you have pip installed and whether it’s the version for Python 3:

11

$ pip3 --version pip 20.0.2 from /usr/lib/python3/dist-packages/pip (python 3.8)

Finally, install Bleak as a Python package: $ pip3 install bleak Collecting bleak ...

At the end, you should see a line that starts with Successfully installed. You’re now ready to use the Bleak library in your Python BLE projects. Note: Make sure you regularly upgrade Bleak to the latest version, using pip3 install -U bleak. 2.2 The Arduino platform with NimBLE-Arduino for the ESP32 The Arduino platform (https://www.arduino.cc) is generally used with the Arduino IDE (https://www.arduino.cc/en/software). There are ways of using other IDEs, such as Visual Studio Code (https://code.visualstudio.com) with PlatformIO (https://platformio.org). Because I don’t want to require you to use a specific IDE if you have another favorite IDE, all examples in this book use a command-line environment. For Arduino, this means Arduino CLI (https://arduino.github.io/arduino-cli/). Note: If you want to use the Arduino examples with another IDE, consult the IDE’s documentation on how to install Arduino support for the ESP32 and install the NimBLE-Arduino library. 11

As you see, my Linux system doesn’t have the newest Python version, but this isn’t a hindrance. The Python code in this book doesn’t use any special features from newer versions, and has been tested on Python versions 3.8, 3.9, and 3.10.

● 25

Boek BLE 220329 UK.indd 25

06/05/2022 15:54

Bluetooth Low Energy Applications

2.2.1 Install Arduino CLI There are several ways of installing Arduino CLI. You can find them all on https://arduino. github.io/arduino-cli/latest/installation/. The easiest way is to simply download the latest version from https://github.com/arduino/arduino-cli/releases and extract the arduino-cli executable to a directory that’s in your PATH environment variable. Note: If you install Arduino CLI this way, make sure you regularly update Arduino CLI to the latest version. On macOS and Linux, you can install Arduino CLI with the Homebrew package manager (https://brew.sh). Just run brew install arduino-cli and update all packages using brew upgrade. After installation, you should be able to run arduino-cli without any arguments and get the following output with all available subcommands: $ arduino-cli Arduino Command Line Interface (arduino-cli). Usage: arduino-cli [command] Examples: arduino-cli [flags...] Available Commands: board

Arduino board commands.

burn-bootloader Upload the bootloader. cache

Arduino cache commands.

compile

Compiles Arduino sketches.

completion

Generates completion scripts

config

Arduino configuration commands.

core

Arduino core operations.

daemon

Run as a daemon on port 50051

debug

Debug Arduino sketches.

help

Help about any command

lib

Arduino commands about libraries.

outdated

Lists cores and libraries that can be upgraded

sketch

Arduino CLI sketch commands.

update

Updates the index of cores and libraries

upgrade

Upgrades installed cores and libraries.

upload

Upload Arduino sketches.

version

Shows version number of Arduino CLI.

Flags: --additional-urls strings

Comma-separated list of additional URLs for the Boards Manager.

--config-file string

The custom config file (if not

● 26

Boek BLE 220329 UK.indd 26

06/05/2022 15:54

Chapter 2 • Preparing your development environment

specified the default will be used). --format string

The output format, can be {text|json}. (default "text")

-h, --help

help for arduino-cli

--log-file string

Path to the file where logs will be written.

--log-format string

The output format for the logs, can be {text|json}.

--log-level string

Messages with this level and above will be logged. Valid levels are: trace, debug, info, warn, error, fatal, panic

-v, --verbose

Print the logs on the standard output.

Use "arduino-cli [command] --help" for more information about a command.

Next, initialize the Arduino CLI configuration file: arduino-cli config init

You’re now ready to use Arduino CLI.

2.2.2 Install the ESP32 Arduino core By default, the Arduino platform supports only Arduino-compatible hardware. However, you can add support for additional boards by installing so-called ‘Arduino cores’ (https://docs. arduino.cc/learn/starting-guide/cores). Espressif maintains an Arduino core for its ESP32 (https://github.com/espressif/arduinoesp32). You can install it by adding the core’s package index to Arduino’s board manager: arduino-cli config add board_manager.additional_urls https://raw. githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_index.json

Then, update the index of cores, and search for a core with esp32 in its name. After a few messages about downloading and installing missing tools, you should see the esp32 core listed: $ arduino-cli core update-index Updating index: package_index.json downloaded Updating index: package_index.json.sig downloaded Updating index: package_esp32_index.json downloaded $ arduino-cli core search esp32 ID

Version Name

esp32:esp32

2.0.2

esp32

● 27

Boek BLE 220329 UK.indd 27

06/05/2022 15:54

Bluetooth Low Energy Applications

Then install the latest version of this core: $ arduino-cli core install esp32:esp32 Downloading packages... esp32:riscv32-esp-elf-gcc@gcc8_4_0-esp-2021r2 downloaded esp32:xtensa-esp32-elf-gcc@gcc8_4_0-esp-2021r2 downloaded esp32:xtensa-esp32s2-elf-gcc@gcc8_4_0-esp-2021r2 downloaded esp32:[email protected] downloaded esp32:[email protected] downloaded esp32:[email protected] downloaded esp32:[email protected] downloaded Installing esp32:riscv32-esp-elf-gcc@gcc8_4_0-esp-2021r2... esp32:riscv32-esp-elf-gcc@gcc8_4_0-esp-2021r2 installed Installing esp32:xtensa-esp32-elf-gcc@gcc8_4_0-esp-2021r2... esp32:xtensa-esp32-elf-gcc@gcc8_4_0-esp-2021r2 installed Installing esp32:xtensa-esp32s2-elf-gcc@gcc8_4_0-esp-2021r2... esp32:xtensa-esp32s2-elf-gcc@gcc8_4_0-esp-2021r2 installed Installing esp32:[email protected]... esp32:[email protected] installed Installing esp32:[email protected]... esp32:[email protected] installed Installing esp32:[email protected]... esp32:[email protected] installed Installing platform esp32:[email protected]... Configuring platform.... Platform esp32:[email protected] installed

2.2.3 Detect your ESP32 board You can now check which ESP32 boards are available in the esp32:esp32 core you installed (I have truncated the output for brevity): $ arduino-cli board listall esp32 Board Name

FQBN

AI Thinker ESP32-CAM

esp32:esp32:esp32cam

ALKS ESP32

esp32:esp32:alksesp32

Adafruit ESP32 Feather

esp32:esp32:featheresp32

... Widora AIR

esp32:esp32:widora-air

XinaBox CW02

esp32:esp32:cw02

u-blox NINA-W10 series (ESP32)

esp32:esp32:nina_w10

Find your ESP32 development board in the output, and take note of its fully qualified board name (FQBN). You’ll need this when uploading a program (Arduino calls this a ‘sketch’) to your board. You can also connect your board via USB and ask Arduino CLI to detect it:

● 28

Boek BLE 220329 UK.indd 28

06/05/2022 15:54

Chapter 2 • Preparing your development environment

$ arduino-cli board list Port

Type

Board Name FQBN Core

/dev/ttyUSB0 Serial Port (USB) Unknown

In this example, you see that the port is detected, but the board name isn’t. Note: If Arduino CLI doesn’t detect a port, have a look at the appendix at the end of this book. You’ll probably have to install a driver first or give the currently logged-in user the appropriate permissions for the serial port.

2.2.4 Install the NimBLE-Arduino library Arduino CLI also has an easy way to find and install libraries. First, search for the correct library: $ arduino-cli lib search nimble Updating index: library_index.json downloaded Name: "NimBLE-Arduino" Author: h2zero Maintainer: h2zero Sentence: Bluetooth low energy (BLE) library for arduino-esp32 based on NimBLE. Paragraph: This is a more updated and lower resource alternative to the original bluedroid BLE library for esp32. Uses 50% less flash space and approximately 100KB less ram with the same functionality. Nearly 100% compatible with existing application code, migration guide included. Website: https://github.com/h2zero/NimBLE-Arduino Category: Communication Architecture: esp32 Types: Contributed Versions: [1.0.0, 1.0.1, 1.0.2, 1.1.0, 1.2.0, 1.3.0, 1.3.1 1.3.3, 1.3.4, 1.3.5, 1.3.6, 1.3.7] Provides includes: NimBLEDevice.h

This gives you a lot of information about the library: the exact name you need to install it, the author, a description, its official website, the architectures it supports, the available versions, and a lot more. Now, install it: $ arduino-cli lib install NimBLE-Arduino Downloading [email protected]... [email protected] downloaded Installing [email protected]... Installed [email protected]

You’re now ready to use the NimBLE-Arduino library in your Arduino BLE sketches.

● 29

Boek BLE 220329 UK.indd 29

06/05/2022 15:54

Bluetooth Low Energy Applications

2.3 The Zephyr development environment for nRF5 devices The installation instructions for the command-line Zephyr development environment depend on the operating system you’re using. The Getting Started Guide (https://docs.zephyrproject.org/latest/getting_started/index.html) in Zephyr’s documentation lists the full instructions for Windows, macOS, and Ubuntu. After you’ve followed these instructions, you’ll have a Zephyr development installed in ~/ zephyrproject/zephyr (on Linux or macOS) or %HOMEPATH%\zephyrproject\zephyr (on Windows). The project contains a lot of sample projects in the samples subdirectory. Also, make sure to install Nordic Semiconductor’s nrfutil Python package: pip3 install nrfutil

You’ll need it to flash your Zephyr applications to the nRF52840 Dongle.

2.3.1 Build a Zephyr application To test whether you’ve set up your development environment correctly, build the basic sample, Blinky, which blinks the LED on the board: $ cd ~/zephyrproject/zephyr $ west build -p auto -b nrf52840dongle_nrf52840 samples/basic/blinky ... -- The C compiler identification is GNU 10.2.0 -- The CXX compiler identification is GNU 10.2.0 -- The ASM compiler identification is GNU -- Found assembler: /home/koan/zephyr-sdk-0.12.3/arm-zephyr-eabi/bin/arm-zephyreabi-gcc -- Configuring done -- Generating done -- Build files have been written to: /home/koan/zephyrproject/zephyr/build -- west build: building application [1/139] Preparing syscall dependency handling [132/139] Linking C executable zephyr/zephyr_prebuilt.elf [139/139] Linking C executable zephyr/zephyr.elf Memory region

Used Size

Region Size

%age Used

FLASH:

12920 B

1020 KB

1.24%

SRAM:

4192 B

256 KB

1.60%

IDT_LIST:

0 GB

2 KB

0.00%

Zephyr uses the west command to build applications and manage repositories. To build an application, you use west build. In this case, you’ll often use the following options: -p auto

● 30

Boek BLE 220329 UK.indd 30

06/05/2022 15:54

Chapter 2 • Preparing your development environment

automatically cleans byproducts from an earlier build if necessary. You should always use this option when switching applications. -b nrf52840dongle_nrf52840 specifies the board for which you want to build the application. You can get the full list of board names with west boards. Consult https://docs.zephyrproject.org/latest/boards/index.html for specific information about each supported board. samples/basic/blinky specifies the application you want to build. This path is either relative to the current directory (as in this case) or an absolute path. In this case, you have built the basic Blinky application for the nRF52840 Dongle. Note: Make sure to run the west command from your Zephyr directory. If you’re building a Zephyr application outside the samples directory, refer to its full path on the command line.

2.3.2 Flash a Zephyr application Now it’s time to flash your Zephyr application to your board. The exact way to do this depends on the board. With boards such as the nRF52840 Development Kit, you can just run west flash. This flashes the firmware using the board’s built-in Segger J-Link debug interface. With the nRF52840 Dongle, the procedure is a bit more elaborate. The board is factory-programmed with Nordic Semiconductor’s bootloader. You need to use the nrfutil command to create firmware packages supported by this bootloader and flash them to the board. Package the application as follows: nrfutil pkg generate --hw-version 52 --sd-req=0x00 --application build/zephyr/ zephyr.hex --application-version 1 blinky.zip

The --hw-version 52 option specifies that you’re creating a firmware package for the nRF52 family of devices, and --sd-req=0x00 specifies that you don’t need the SoftDevice BLE stack. 12 Then you specify the hex file of the application you want to package (having built this with west build), and an application version (just use 1). The command ends with the filename of the zip file it should generate. Then, reset the board into its bootloader. Do this by pressing the RESET button. This is the small button on the far side of the board from the USB connector. Note that the button’s actuator doesn’t face upwards: you’ll have to push it from the side, toward the USB connector. After this, the red LED starts pulsating. This is the sign that the bootloader is running. 12

Zephyr has its own full BLE stack.

● 31

Boek BLE 220329 UK.indd 31

06/05/2022 15:54

Bluetooth Low Energy Applications

Figure 2.1 Press the RESET button from the side to place the nRF52840 Dongle in DFU (Device Firmware Upgrade) mode. Now that the bootloader is accepting device firmware updates, you can flash the zip file to your board: nrfutil dfu usb-serial -pkg blinky.zip -p /dev/ttyACM0

Note: Have a look at the appendix to know how to find the correct device name (/dev/ ttyACM0 in this case). After the command exits, the red LED goes off and the green LED starts blinking. The Blinky application is running on your board. Note: If you have an nRF52840 dongle from another manufacturer with the Adafruit nRF52 bootloader (https://github.com/adafruit/Adafruit_nRF52_Bootloader), consult the appendix at the end of this book for the procedure to flash your Zephyr application. Examples of such devices are April Brother’s April USB Dongle 52840 and makerdiary’s nRF52840 MDK USB Dongle.

2.4 The nRF Connect for Desktop application Because this book is using some software from Nordic Semiconductor, you should install nRF Connect for Desktop (https://www.nordicsemi.com/Products/Development-tools/ nrf-connect-for-desktop), Nordic Semiconductor’s cross-platform development software. Download the latest package for your platform (Windows/Linux/macOS) and run it.

● 32

Boek BLE 220329 UK.indd 32

06/05/2022 15:54

Chapter 2 • Preparing your development environment

Note: On Linux, you first need to make the AppImage file that you downloaded executable, for instance with chmod +x nrfconnect-3.9.1-x86_64.AppImage. To be able to install firmware to the nRF52840 Dongle from nRF Connect for Desktop, you need Nordic Semiconductor’s Programmer application. You can install this from nRF Connect for Desktop by clicking on Install next to Programmer. On Linux and macOS, you also need to install the SEGGER J-Link software (https://www. segger.com/downloads/jlink/), which is used by the Programmer application to flash the firmware to your device. The Windows version of nRF Connect for Desktop already includes this software. Note: If you have an nRF52840 dongle from another manufacturer that uses the Adafruit nRF52 bootloader, you can’t use the Programmer application. With these devices, you can just drag and drop the firmware from your operating system’s file explorer. Consult the appendix at the end of this book for the procedure.

2.5 The nRF Connect mobile app If you have an Android or iOS smartphone, the nRF Connect mobile app (https://www. nordicsemi.com/Products/Development-tools/nrf-connect-for-mobile) is an indispensable tool. It allows you to scan, explore, and communicate with BLE devices. You can download the app for free from the Google Play Store or Apple’s App Store. You’ll learn how to use nRF Connect later in this book, but, for now, you can tap Scan at the top-right. This shows you all of the BLE devices that your phone discovers, and each device’s name, Bluetooth address and RSSI (Received Signal Strength Indicator). If you tap on the name of a device, it shows you more information, such as the device type, advertising type, services, and manufacturer-specific data. You’ll learn about the meaning of all this information in the rest of this book.

● 33

Boek BLE 220329 UK.indd 33

06/05/2022 15:54

Bluetooth Low Energy Applications

Figure 2.2 Exploring BLE devices from your phone with the nRF Connect mobile app You can also connect to a device. Just press Connect and the app connects to the device. It shows you all available services, which you can tap on to see all of their characteristics, and even read and write their values. Warning: Don’t forget to tap Disconnect at the top-right to close the connection to the device if you don’t need it anymore. Most BLE devices only allow a connection to one device, and you don’t want to block other connections. You’ll use nRF Connect in the next chapters to explore BLE devices. I’ll explain in detail how to work with it in the relevant sections.

2.6 The Bluetooth Low Energy app in nRF Connect for Desktop Nordic Semiconductor has an interesting desktop app for discovering devices and exploring their services and characteristics. In nRF Connect for Desktop, click on Install next to the Bluetooth Low Energy app, and then on Open to open the application. The application requires a connection to a development kit or dongle with a BLE chip, so plug an nRF52840 Dongle into your computer’s USB port. Then click on the arrow next to Select device at the top left of the Bluetooth Low Energy application and select the available device. Confirm using Yes that you want to program the device, and again that you want to update the bootloader.

● 34

Boek BLE 220329 UK.indd 34

06/05/2022 15:54

Chapter 2 • Preparing your development environment

Note: If you get an error about permissions for USB devices in Linux, visit https:// github.com/NordicSemiconductor/nrf-udev. For Debian-based systems, just follow the instructions to install the correct udev rules to solve this. On other Linux distributions, just copy the files from nrf-udev_1.0.1-all/lib/udev/rules.d/ in the repository to /etc/ udev/rules.d/. After this, the Bluetooth Low Energy application recognizes your nRF52840 Dongle as a BLE adapter, which you can use to scan for devices. Click on Start scan at the top right to find BLE devices. If you want to know more about a device, click on Detail. Information about the address type, advertising type, services, and manufacturer-specific data unfolds. Then, you can click on Connect, where you can read and write characteristics. Warning: Don’t forget to click on the gear icon next to the device name and then Disconnect to close the connection to the device if you don’t need it anymore. Most BLE devices only allow a connection to one device, and you don’t want to block other connections.

Figure 2.3 Exploring BLE devices with the Bluetooth Low Energy application in nRF Connect for Desktop You’ll use the Bluetooth Low Energy application in nRF Connect for Desktop in the next chapters to explore BLE devices. In the relevant chapters, I’ll explain how to work with it in detail.

2.7 Wireshark and a BLE sniffer dongle The nRF Connect mobile app and the Bluetooth Low Energy application in nRF Connect for Desktop are adequate for a first exploration of the capabilities of a BLE device, but, if you really want to investigate the exact sequence of BLE packets sent over the air, you’ll need a BLE sniffer.

● 35

Boek BLE 220329 UK.indd 35

06/05/2022 15:54

Bluetooth Low Energy Applications

There are various solutions available for this purpose, some of these very powerful. In this book, I’m using Wireshark with the nRF Sniffer for Bluetooth LE firmware on a Nordic Semiconductor nRF52840 Dongle. This is a very affordable solution, as you only need a dongle costing €10 and some free software.

2.7.1 Downloading Wireshark and the nRF Sniffer for Bluetooth LE Wireshark (https://www.wireshark.org) is a powerful cross-platform open-source network protocol analyzer. Most people use it to analyze network problems on their Wi-Fi or Ethernet network. With an external capture plug-in, you can also use it to analyze Bluetooth Low Energy traffic with the Nordic Semiconductor nRF52840 Dongle. On Windows or macOS, download the latest version of the installer from Wireshark’s home page. At the time of writing, this is Wireshark 3.6.1. On Linux, you can install Wireshark from the relevant package manager. Then, download the nRF Sniffer for Bluetooth LE firmware (https://www.nordicsemi.com/ Products/Development-tools/nrf-sniffer-for-bluetooth-le/). At the time of writing, this is version 4.1.0. Unpack the zip file to a new directory.

2.7.2 Installing the nRF Sniffer for Bluetooth LE firmware In the nRF Sniffer for Bluetooth LE’s hex directory, you’ll find some firmware files with a hex extension. The one for the nRF52840 Dongle is called sniffer_nrf52840dongle_ nrf52840_4.1.0.hex. Launch the Programmer application from nRF Connect for Desktop, insert the nRF52840 Dongle into a USB port on your computer and press the RESET button at its edge. The status light will now start pulsating red. This means that the dongle is now in DFU (Device Firmware Upgrade) mode, ready to receive new firmware. Click Select device in the Programmer application and then select the device. If you’ve previously flashed firmware to the device, the application shows the firmware layout, with the bootloader, SoftDevice BLE stack, and application. At the top-right, click on Add HEX file, then Browse… and select the correct firmware file, sniffer_nrf52840dongle_nrf52840_4.1.0.hex. The file’s memory layout is also shown. Then, click Write to flash the firmware to the dongle.

● 36

Boek BLE 220329 UK.indd 36

06/05/2022 15:54

Chapter 2 • Preparing your development environment

Figure 2.4 Flashing the sniffer firmware with the Programmer application in nRF Connect for Desktop Note: Don’t panic if you get an error message saying that the device was not found. Have a closer look at the log messages at the bottom: as long as you see All dfu images have been written to the target device, all is well. The Programmer application just complains because, after flashing the firmware, the device resets and is no longer in DFU mode. If you have April Brother’s April USB Dongle 52840, I highly recommend it as your BLE sniffer dongle. With its external antenna, it has a great range. However, it comes with an alternative bootloader that isn’t supported by the nRF Connect for Desktop’s Programmer application. Consult the appendix for instructions on how to flash your hex file to it.

Figure 2.5 With its external antenna, April Brother’s April USB Dongle 52840 makes a great BLE sniffer.

● 37

Boek BLE 220329 UK.indd 37

06/05/2022 15:54

Bluetooth Low Energy Applications

2.7.3 Installing the nRF Sniffer capture tool Now that your nRF52840 Dongle has the BLE sniffer firmware, it’s time to make Wireshark understand how to communicate with the dongle to sniff BLE traffic. For this purpose, nRF Sniffer for Bluetooth LE firmware’s zip file has a Wireshark capture plugin in the extcap directory. Open a command-line window in the extcap directory and then install the required software using the following command: pip3 install -r requirements.txt

Then, you need to copy the sniffer plugin to Wireshark’s directory for external capture plugins. First, start Wireshark, then open the Help / About Wireshark menu and open the Folders tab. Double-click on Personal Extcap path and confirm that you want to create this directory. Then, copy the complete content of the BLE sniffer’s extcap directory (including the SnifferAPI subdirectory) to that directory. On my Linux system, this is ~/.config/wireshark/extcap.

Figure 2.6 Check in which directory you need to install the external extcap plugin for the BLE sniffer. Open a command-line window in your personal extcap directory and verify whether the plugin finds the extcap interface. On Windows, this goes like this: nrf_sniffer_ble.bat --extcap-interfaces

● 38

Boek BLE 220329 UK.indd 38

06/05/2022 15:54

Chapter 2 • Preparing your development environment

On Linux and macOS you use: ./nrf_sniffer_ble.sh --extcap-interfaces

This should output something like this: koan@tux:~/.config/wireshark/extcap$ ./nrf_sniffer_ble.sh --extcap-interfaces extcap {version=4.1.0}{display=nRF Sniffer for Bluetooth LE}{help=https://www. nordicsemi.com/Software-and-Tools/Development-Tools/nRF-Sniffer-for-Bluetooth-LE} interface {value=/dev/ttyACM0-None}{display=nRF Sniffer for Bluetooth LE} control {number=0}{type=selector}{display=Device}{tooltip=Device list} control {number=1}{type=selector}{display=Key}{tooltip=} control {number=2}{type=string}{display=Value}{tooltip=6 digit passkey or 16 or 32 bytes encryption key in hexadecimal starting with ‘0x’, big endian format. If the entered key is shorter than 16 or 32 bytes, it will be zero-padded in front’}{validation=\b^(([0-9]{6})|(0x[0-9a-fA-F]{1,64})|([0-9A-Fa-f]{2}[:-]){5} ([0-9A-Fa-f]{2}) (public|random))$\b} control {number=3}{type=string}{display=Adv Hop}{default=37,38,39} {tooltip=Advertising channel hop sequence. Change the order in which the sniffer switches advertising channels. Valid channels are 37, 38 and 39 separated by comma.}{validation=^\s*((37|38|39)\s*,\s*){0,2}(37|38|39){1}\s*$}{required=true} control {number=7}{type=button}{display=Clear}{tooltop=Clear or remove device from Device list} control {number=4}{type=button}{role=help}{display=Help}{tooltip=Access user guide (launches browser)} control {number=5}{type=button}{role=restore}{display=Defaults}{tooltip=Resets the user interface and clears the log file} control {number=6}{type=button}{role=logger}{display=Log}{tooltip=Log per interface} value {control=0}{value= }{display=All advertising devices}{default=true} value {control=0}{value=[00,00,00,00,00,00,0]}{display=Follow IRK} value {control=1}{value=0}{display=Legacy Passkey}{default=true} value {control=1}{value=1}{display=Legacy OOB data} value {control=1}{value=2}{display=Legacy LTK} value {control=1}{value=3}{display=SC LTK} value {control=1}{value=4}{display=SC Private Key} value {control=1}{value=5}{display=IRK} value {control=1}{value=6}{display=Add LE address} value {control=1}{value=7}{display=Follow LE address}

Note the line that starts with interface. If you don’t see this line in the output, the script hasn’t discovered your dongle. Then, press F5 in Wireshark to refresh the interfaces, open the View / Interface Toolbars menu and enable the nRF Sniffer for Bluetooth LE Item.

● 39

Boek BLE 220329 UK.indd 39

06/05/2022 15:54

Bluetooth Low Energy Applications

2.7.4 Installing the BLE profile Wireshark should now be able to sniff BLE traffic, but, before testing this, you should also install a BLE profile so that Wireshark shows more information in its capture output. To do so, open the Help / About Wireshark menu again, then the Folders tab, and double-click Personal configuration. Copy the nRF sniffer’s complete Profile_nRF_Sniffer_Bluetooth_LE directory (not the contents of the directory, but the directory itself) to the profiles directory in Wireshark’s personal configuration directory. Then open the Edit / Configuration Profiles… menu in Wireshark, and select Profile_nRF_Sniffer_Bluetooth_LE and click OK.

2.7.5 Testing a BLE packet capture Wireshark is now ready to sniff BLE packets using your nRF52840 Dongle. Plug the dongle into a USB port, press F5 in Wireshark to refresh the interfaces and double-click on nRF Sniffer for Bluetooth LE. If you have some BLE devices around, you should see a continuous stream of advertisements. In the rest of this book, you’ll constantly use Wireshark to investigate these packets.

Figure 2.7 Wireshark capturing BLE packets with the nRF Sniffer for Bluetooth LE firmware on the nRF52840 Dongle If you don’t see any packets, or if you get an error message, make sure you’ve copied the extcap and profile directories to the right place. To give you an idea, on my Linux system, all files for Wireshark are in ~/.config/wireshark and the content looks like this (I’ve only shown the relevant files): ├── extcap │   ├── nrf_sniffer_ble.bat │   ├── nrf_sniffer_ble.py

● 40

Boek BLE 220329 UK.indd 40

06/05/2022 15:54

Chapter 2 • Preparing your development environment

│   ├── nrf_sniffer_ble.sh │   ├── requirements.txt │   └── SnifferAPI │  

├── CaptureFiles.py

│  

├── Devices.py

│  

├── example_linux.py

│  

├── example.py

│  

├── Exceptions.py

│  

├── Filelock.py

│  

├── __init__.py

│  

├── Logger.py

│  

├── myVersion.py

│  

├── Notifications.py

│  

├── Packet.py

│  

├── Pcap.py

│  

├── SnifferCollector.py

│  

├── Sniffer.py

│  

├── Types.py

│  

├── UART.py

│  

└── version.py

└── profiles   └── Profile_nRF_Sniffer_Bluetooth_LE  

├── preferences

 

└── recent

Also have a look at the Nordic Semiconductor documentation’s Troubleshooting page (https://infocenter.nordicsemi.com/topic/ug_sniffer_ble/UG/sniffer_ble/troubleshooting. html) for the nRF Sniffer for Bluetooth LE.

2.8 Summary and further exploration In this chapter, you installed the three major development environments used in this book: Python with the Bleak library, the Arduino framework with the NimBLE-Arduino library, and the Zephyr real-time operating system. With these environments installed, you’re ready to run the example programs in this book. I’ve set up a command-line environment in this chapter, but, if you prefer a graphical interface to develop your applications, you may certainly use one. An interesting choice is Visual Studio Code (https://code.visualstudio.com) with PlatformIO (https://platformio. org). The latter allows developers to use the same project structure and the same graphical interface for various development platforms, including the ones used in this book. Consult PlatformIO’s documentation on how to use Python, the Arduino framework, and Zephyr this way. 13

13

For Zephyr, you have two choices in PlatformIO: the one developed by PlatformIO (https://docs. platformio.org/en/latest/frameworks/zephyr.html) and Nordic Semiconductor’s nRF Connect for VS Code (https://www.nordicsemi.com/Products/Development-tools/nRF-Connect-for-VS-Code).

● 41

Boek BLE 220329 UK.indd 41

06/05/2022 15:54

Bluetooth Low Energy Applications

You also installed some tools by Nordic Semiconductor to explore BLE devices: the nRF Connect mobile app for your Android or iOS smartphone and nRF Connect for Desktop’s Bluetooth Low Energy app on your computer. They’re useful tools to discover BLE devices around you, see what services they offer, and interact with them. You’ll also use them to debug the BLE devices you build in this book. To be able to investigate the exact sequence of BLE packets sent over the air, you installed Wireshark with the nRF Sniffer for Bluetooth LE firmware on an nRF52840 Dongle. You’ll use this BLE sniffer a lot in the next chapters, and it will give you real hands-on experience with BLE.

● 42

Boek BLE 220329 UK.indd 42

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

Chapter 3 • Broadcasting data with advertisements In the introduction, you learned that there are two ways to communicate with BLE devices: with a connection and without. In this chapter, you’ll learn how to communicate without a connection. This is called advertising. Most types of advertising packets are broadcast, meaning that any BLE device in the neighborhood is able to receive these packets. That’s why advertising packets are fundamental for device discovery. But, apart from discovering devices, advertising packets have another goal: they’re an easy way to broadcast data to multiple receivers. A broadcasting device doesn’t know whether its packets are received by any other device or whether the latter device is even listening. So, broadcasting data with BLE advertisements is an unreliable operation, but, for many tasks, it’s perfectly adequate. For instance, if a BLE environmental sensor broadcasts its temperature every second, it doesn’t matter if some of its packets are lost. In this chapter you’ll learn about: • • • • • • • • • • •

the different device roles relevant to advertising the structure and types of advertising packets the difference between active and passive scanning the various types of BLE addresses how to discover advertisements with Python and Bleak how to decode iBeacon advertisements or other manufacturer-specific data using Python and Bleak how to discover advertisements using NimBLE-Arduino how to decode iBeacon advertisements, Microsoft advertising beacons, or other manufacturer-specific data using NimBLE-Arduino how to broadcast iBeacon advertisements using Zephyr how to broadcast sensor data as manufacturer-specific data using Zephyr how to advertise scan response data using Zephyr

In the above list, you’ve probably noticed that I’m using only Python and NimBLE-Arduino for scanning, and only Zephyr for advertising. This is the most common use case. For advertising, you’ll generally want to use a low-power platform, such as a microcontroller with Zephyr. In contrast, for scanning, you’ll generally use a more powerful platform, such as a computer or ESP32 board.

3.1 Device roles Before delving into the details of advertising, it’s important to know the different roles that BLE devices can have for advertisements, because this defines their behavior. The Generic Access Profile (GAP) defines four roles:

● 43

Boek BLE 220329 UK.indd 43

06/05/2022 15:54

Bluetooth Low Energy Applications

Broadcaster Sends non-connectable advertising packets, broadcasting them to any device in the neighborhood. Other devices can’t connect to a broadcaster. Observer Scans for non-connectable advertising packets sent by broadcasters Peripheral Sends connectable advertising packets, so other devices know they’re able to connect. Once connected, the peripheral device is also known as a slave in the link layer. Central Initiates a connection to a peripheral. Once connected, the central device is also known as a master in the link layer. Because this chapter only discusses communication without connections, you’ll learn how to create a broadcaster and an observer here. The next chapter talks about connections, and there you’ll learn how to create a peripheral and a central. Note: A BLE device can have multiple GAP roles simultaneously. The broadcaster and observer roles interact like this in a one-to-many unidirectional manner:

Figure 3.1 A broadcaster sends data to all observers in the neighborhood.

3.2 Advertising packets Before explaining the structure of advertising packets, it’s important to talk about the channels they’re transmitted on.

3.2.1 Advertising channels Bluetooth Low Energy uses 40 2 MHz channels in the 2.4 GHz ISM band. Three of these are advertising channels: channels 37 (2402 MHz), 38 (2426 MHz), and 39 (2480 MHz). The other 37 (channels 0 to 36) are data channels, used for connections. Note that the advertising channels are spread around the full BLE spectrum: one in the beginning, one at the end, and one between channels 10 and 11. This is to avoid interference from other devices operating in the same spectrum. 14

14 The BLE advertising channels are placed this way so that they fall outside Wi-Fi channels 1, 6, and 11.

● 44

Boek BLE 220329 UK.indd 44

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

BLE also uses frequency-hopping to reduce overall signal interference. For instance, when an advertising BLE device transmits advertising packets, it does this on the three advertising channels, one after the other: channel 37, channel 38, channel 39, channel 37, channel 38, and so on. You can verify this with Wireshark. Start sniffing BLE traffic as explained in the previous chapter. Then, in the toolbar for BLE, select one device. After you have received a few advertising packets from the device, click on a packet and open the Nordic BLE Sniffer section in the packet details pane. There you see, for instance, Channel: 37. If you navigate to the next packet with the arrow down key, you see Channel: 38, for the next one Channel: 39, and then Channel: 37 again, in a continuous cycle. Also, look at the Time column: the packets are sent on the three channels immediately (within one or a few milliseconds) after each other.

Figure 3.2 Wireshark showing the advertising channel for each BLE advertising packet Note: I have removed some columns in Wireshark’s Packet List pane that are only used for data packets (in connections). Right-click on the column header and disable the columns you don’t want to see. For this chapter, I suggest disabling the SN, NESN, More data, and Event counter columns.

● 45

Boek BLE 220329 UK.indd 45

06/05/2022 15:54

Bluetooth Low Energy Applications

3.2.2 Advertising packet structure If you click on the packet details pane’s Bluetooth Low Energy Link Layer section, you’ll see the whole link layer packet separated into its parts. The most important part is the Packet Header, and, more specifically, the PDU Type in it. 15 The PDU type specifies the advertising packet’s type. In this book, you’ll see the following seven advertising packet types: 16 ADV_IND Connectable undirected advertising indication ADV_DIRECT_IND Connectable directed advertising indication ADV_NONCONN_IND Non-connectable undirected advertising indication ADV_SCAN_IND Scannable undirected advertising indication SCAN_REQ Active scanning request SCAN_RSP Active scanning response CONNECT_REQ Connection request You can see the packet type in the Packet List pane’s Info column in Wireshark. Each of these packet types has a different payload format for its data. Another important part of the header is the Length field. This is a 6-bit value specifying the length of the data in the packet, with possible values from 6 to 37. Why at least 6? Because each advertising packet includes the address of the advertising device, and this is a 6-byte Bluetooth address. So, apart from the address, every advertising packet can hold up to 31 bytes of data.

15 16

PDU stands for Protocol Data Unit. There are more PDU types, but not all devices support them. For instance, Bluetooth 5 introduced extended advertisements, but only devices that support this feature can discover them.

● 46

Boek BLE 220329 UK.indd 46

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

Figure 3.3 Wireshark showing the packet header of an advertising packet Note: The BLE specification stipulates that bytes are transmitted with the least significant bit first, and most multi-byte fields with the least significant byte first (which is called little-endian). This is important to remember when you’re processing advertising packets. You can already see this in this screenshot in the panel with the raw bytes at the bottom: the bytes 20 c7 1e 4b 75 d8 code the BLE address d8:75:4b:1e:c7:20. After the header, with the device address as the last part, the data is sent. You’ll see how to process and transmit this data in the code examples in the rest of this chapter. The data of an advertising packet is just a sequence of one or more advertising data structures. Each of these starts with a length byte, which specifies the number of bytes following in this data structure. You can also see this in Wireshark:

● 47

Boek BLE 220329 UK.indd 47

06/05/2022 15:54

Bluetooth Low Energy Applications

Figure 3.4 Wireshark showing the different types of advertising data For instance, the Device Name advertising structure has Length 15 (coded as hexadecimal value 0x0f), type Device Name (value 0x09), and then 14 bytes. These represent the ASCII values of the string Gigaset keeper. Note: The Length byte is 15 here because it counts the bytes of both the Type (one byte) and the Device Name (14 bytes).

3.3 Discovering advertisements with Bleak After this bit of advertising channel and packet structure theory, it’s time to put it into practice. Coding a BLE central always starts with the question: how do I find BLE devices in the neighborhood? Because every advertising packet includes the device’s Bluetooth address, this question comes down to: how do I discover advertising packets? With Bleak, it’s easy. By default, it even installs the command-line program, bleak-lescan, a simple BLE scanner, which scans for BLE advertising packets for five seconds using the default BLE adapter (have a look at bleak-lescan --help if you want to change these parameters). It then shows the Bluetooth addresses and names of the devices it has discovered: $ bleak-lescan 7D:26:FB:36:BB:E8: 7D-26-FB-36-BB-E8 54:64:0D:D5:77:AD: 54-64-0D-D5-77-AD 03:53:E4:16:21:AC: 03-53-E4-16-21-AC DC:A6:32:59:6F:41: DC-A6-32-59-6F-41 75:C9:99:26:5D:F4: 75-C9-99-26-5D-F4

● 48

Boek BLE 220329 UK.indd 48

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

45:77:8E:66:4D:5F: 45-77-8E-66-4D-5F E4:00:95:18:57:1A: nut 7C:2F:80:E3:A1:A7: Gigaset keeper C8:03:24:74:7E:0E: C8-03-24-74-7E-0E C4:7C:8D:6A:69:DC: Flower care C4:7C:8D:67:65:AD: Flower care E7:2E:00:B1:38:96: LYWSD02

If a device doesn’t advertise its name, Bleak uses the Bluetooth address as the name, with the colon (:) replaced by a dash (-).

3.3.1 Scanning for devices You can also scan for devices using the following code: """Scan for BLE devices with Bleak.""" import asyncio from bleak import BleakScanner

async def main(): """Scan for BLE devices.""" devices = await BleakScanner.discover() for device in devices: print(device)

asyncio.run(main())

Run this program: python3 scanner.py

This produces exactly the same output that bleak-lescan did before: after five seconds of scanning, you get a list of discovered devices. Note: Because Bleak uses asynchronous I/O, you have to import Python’s asyncio library. Every time you execute an asynchronous routine (such as the BleakScanner. discover() method) with the await keyword, this routine is able to pause while waiting on its result. It lets other routines run in the meantime.

3.3.2 Detection callbacks Bleak also offers the ability to notify you immediately upon discovered each device, rather than showing only the devices discovered within the first five seconds of scanning. This is done with a detection callback, and the code (modified from Bleak’s detection callback example) looks like this:

● 49

Boek BLE 220329 UK.indd 49

06/05/2022 15:54

Bluetooth Low Energy Applications

"""Scan for BLE devices with a detection callback. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio from bleak import BleakScanner from bleak.backends.device import BLEDevice from bleak.backends.scanner import AdvertisementData

def device_found( device: BLEDevice, advertisement_data: AdvertisementData ): """Show advertisement data on detection of a BLE device.""" print( device.address, "| RSSI:", device.rssi, "|", advertisement_data, )

async def main(): """Register detection callback and scan for devices.""" scanner = BleakScanner() scanner.register_detection_callback(device_found) await scanner.start() await asyncio.sleep(5.0) await scanner.stop()

asyncio.run(main())

So, you create a BleakScanner object, register a detection callback, start the scanner, wait for five seconds, then stop the scanner. Every time the BleakScanner object receives an advertisement, it calls the device_found() function with the device and advertisement data as its arguments. In this callback function, you print the device’s address and RSSI value, as well as the advertisement data. Run this program. Its output will look like this:

● 50

Boek BLE 220329 UK.indd 50

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

$ python3 scanner_detection_callback.py 42:42:F0:E7:80:B3 | RSSI: -92 | AdvertisementData(service_data={‘0000fd6f-00001000-8000-00805f9b34fb’: b’\xcf\x99(\x8a\x18sAP\xdb5Y_\x8b\xa2Q\xc1`\x1d*F’}, service_uuids=[‘0000fd6f-0000-1000-8000-00805f9b34fb’]) DC:A6:32:59:6F:41 | RSSI: -96 | AdvertisementData(manufacturer_data={76: b’\x02\ x15\xd13\x8a\xce\x00-D\xaf\x88\xd1\xe5|\x12HIf\x00\x01\x9b\xe0\xc5’}) 48:62:22:C4:CE:C5 | RSSI: -79 | AdvertisementData(service_data={‘0000fe50-00001000-8000-00805f9b34fb’: b’\xa2\x84’}, service_uuids=[‘0000fe50-0000-1000-800000805f9b34fb’]) E7:2E:00:B1:38:96 | RSSI: -81 | AdvertisementData(local_name=’LYWSD02’, service_ data={‘0000fe95-0000-1000-8000-00805f9b34fb’: b’p [\x04\x06\x968\xb1\x00.\xe7\t\ x06\x10\x02\xda\x02’}, service_uuids=[‘0000181a-0000-1000-8000-00805f9b34fb’, ‘0000fef5-0000-1000-8000-00805f9b34fb’]) 7C:2F:80:E3:A1:A7 | RSSI: -79 | AdvertisementData(local_name=’Gigaset keeper’, manufacturer_data={384: b’\x02\x15\x01\x00\x80\xe3\xa1\ xa7\xc5’}, service_uuids=[‘00001800-0000-1000-8000-00805f9b34fb’, ‘00001801-0000-1000-8000-00805f9b34fb’, ‘00001802-0000-1000-8000-00805f9b34fb’, ‘00001803-0000-1000-8000-00805f9b34fb’, ‘00001804-0000-1000-8000-00805f9b34fb’, ‘0000180a-0000-1000-8000-00805f9b34fb’, ‘0000180f-0000-1000-8000-00805f9b34fb’, ‘0000fef5-0000-1000-8000-00805f9b34fb’, ‘6d696368-616c-206f-6c65-737a637a796b’])

You will immediately see various BLE devices being discovered. After five seconds of messages scrolling by, the program exits. The AdvertisementData objects in this output already give you a sneak peek of the data that you’ll process in the rest of this chapter. You’ll encounter the following attributes for this type of object, each of them equivalent to an advertising data type: 17 local_name A UTF-8 string, possibly truncated, with the device name manufacturer_data Manufacturer-specific data, encoded as a dictionary with the 16-bit company identifier as a key and its data as the value service_data Service data, encoded as a dictionary with the UUID as the key and its data as the value service_uuids A list of service UUIDs, one for each service the device offers You’ll learn how to use each of these advertising data types later in this book.

17 18

18

You can find the full list of BLE data types on page 10 of the Supplement to the Bluetooth Core Specification, version 10. Service data and service UUIDs are explained in the next chapter.

● 51

Boek BLE 220329 UK.indd 51

06/05/2022 15:54

Bluetooth Low Energy Applications

3.3.3 Active and passive scanning Look at the last found device in the previous output of the scanner_detection_callback. py script. You see a device with local name "Gigaset keeper" that has manufacturer-specific data and service UUIDs. Try to find a similar device among your own BLE devices, preferably with the same advertising data types. Now run the scanner script again while you’re sniffing BLE traffic with Wireshark, with the same device selected in the BLE toolbar. First, you see the device sending an ADV_IND advertisement every second, containing advertising data of the Flags, 16-bit Service Class UUIDs, and Device Name types. Bleak doesn’t expose the flags, but the other ones are the AdvertisementData class’s service_uuids and local_name attributes. Then you see a SCAN_REQ advertisement sent by your computer, with the Gigaset keeper device’s target address. This happens because Bleak does an active scan by default: for every device it finds, it requests extra information, with a SCAN_REQ packet directed at that device. The addressed device responds with a SCAN_RSP advertisement, which is also called scan response data. In this case, this device responds with manufacturer-specific data. Some devices respond with their device name. What data is returned for a SCAN_RSP packet depends on the type of device. As long as Bleak is actively scanning, you’ll see the SCAN_REQ and SCAN_RSP packets. After completion of the scan, the device returns to only sending an ADV_IND advertisement every second. This sequence looks like this:

Figure 3.5 With an active scan, the scanner listens to the advertising packets and sends a scan request to each device to get more information from its scan response.

● 52

Boek BLE 220329 UK.indd 52

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

On Windows, you can request a passive scan from Bleak: your computer then just listens to the advertisements without sending SCAN_REQ packets to the devices to request more information. You can test this by running the same scanner_detection_callback.py script, but with the scanner object instantiated as follows: scanner = BleakScanner(scanning_mode="passive")

If you look at the BLE packets in Wireshark, you’ll see that there are no SCAN_REQ and SCAN_RSP packets this time: 19

Figure 3.6 With a passive scan, the scanner just listens to the advertising packets.

3.4 Public and random Bluetooth addresses Before delving more into how to process broadcast data, let’s first talk about how you identify BLE devices. Every BLE device has a 48-bit Bluetooth address, just as every Wi-Fi or Ethernet device has a 48-bit MAC address. However, there are various types of Bluetooth addresses. The main types are: Public address A fixed and globally unique address that is required to be registered with the IEEE (Institute of Electrical and Electronics Engineers). Just like a MAC address, it’s a 48bit extended unique identifier (EUI48), with a 24bit company ID (an OUI, or organizationally unique identifier) assigned to the device manufacturer and 24 bits that the manufacturer assigns to each device. Random address An address that’s programmed into the device or generated by the application, but which is not guaranteed to be unique. It does not require registration with the IEEE. In Wireshark’s Packet Details pane, you can find the type of address (public or random) in the Bluetooth Low Energy Link Layer’s Packet Header part. One of the flags there says 19

On Linux, Bleak currently supports active scanning only. Newer BlueZ versions have an advertisement monitor API, which should handle passive scanning on Linux. Follow https://github.com/hbldh/bleak/ issues/606 for its support status in Bleak.

● 53

Boek BLE 220329 UK.indd 53

06/05/2022 15:54

Bluetooth Low Energy Applications

Tx Address: Public or Tx Address: Random. The device’s address type is also shown in the menu that appears when you click on All advertising devices next to Device in the nRF Sniffer toolbar. Wireshark also replaces the OUI of a public address in the packet list with the first eight letters of the company name, which is convenient – it makes it easier to find a specific device type, such as a Raspberry Pi, in the list of Bluetooth packets.

Figure 3.7 Wireshark showing the address type and conveniently replacing the public address’s OUI with the first letters of the company name Note: With Bleak, you can find the type of address in a device’s details attribute. However, the specifics differ between Windows, Linux, and macOS. See https://bleak.readthedocs. io/en/latest/api.html#bleak.backends.device.BLEDevice for more information. A random address can be one of two types: Static address A fixed address, randomly chosen by the manufacturer or developer. The two most significant bits are always 1. This type of address can be fixed for the lifetime of the device or set when the device boots, but it can never be changed while the device is running.

● 54

Boek BLE 220329 UK.indd 54

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

Private address An address that changes periodically, for instance, every fifteen minutes. This is meant to protect the privacy of the user walking around with this device. Every modern phone operating system uses private addresses to prevent others from tracking you by using your phone’s Bluetooth address. 20 There are two types of private address: Non-resolvable address An address that can’t be used by anyone to track the device. Its two most significant bits are always 0, and the remaining 46 bits are random. Of the 46 random bits, at least one should be 0, and at least one should be 1. Resolvable address An address that can’t be used by anyone to track the device, except by one or more trusted devices. For these devices, the identity of the device can be "resolved" thanks to a shared key. 21 A resolvable address has 01 as its two most significant bits. Summarizing this in an illustration, the various BLE address types are:

Figure 3.8 BLE address types

3.5 The iBeacon specification One popular type of manufacturer-specific data is iBeacon. As an example of how to interpret this manufacturer-specific data, I’ll show you how iBeacon advertisements work. The advantage is that iBeacon-enabled hardware (commonly called "Bluetooth beacons") is very common. And, even if you don’t have a Bluetooth beacon, you can simulate it easily with an app on your mobile phone. You can also do this with the Zephyr-based iBeacon firmware you’ll build at the end of this chapter for the nRF52840 Dongle. The iBeacon specification, published by Apple, is officially called Proximity Beacon. The idea is to have Bluetooth beacons advertise their presence in order to calculate their approximate distance. You can find the specification on https://developer.apple.com/ibeacon/. 20 21

See section 5.6 for more information about private addresses. This is the Identity Resolving Key (IRK), explained in section 5.2.3.

● 55

Boek BLE 220329 UK.indd 55

06/05/2022 15:54

Bluetooth Low Energy Applications

On page 6 of the Proximity Beacon Specification document, Apple writes: Proximity beacons must use a non-connectable undirected Advertising PDU, ADV_ NONCONN_IND, and implement an Apple-specific advertising payload. Proximity beacons must broadcast the entire 30-byte advertising packet in all Advertising frequencies using a fixed 100 ms advertising interval. Some devices don’t follow this specification to the letter. For instance, they use an ADV_ IND PDU type (because they allow connections) or a different advertising interval. However, the manufacturer-specific advertisement type’s data always has the same format because iBeacon-compatible software would otherwise not be able to read this data. The specification lists the format of the iBeacon advertising packet. This always consists of two advertising data structures: flags (of length 2) and manufacturer-specific data (of length 26). That’s why an iBeacon advertising packet is always 30 bytes long (1 + 2 + 1 + 26). Here’s the structure of the complete packet:

Figure 3.9 Structure of an iBeacon’s manufacturer-specific data The first data structure has type 0x01, which signifies that it’s a flags data structure. The value 0x06 sets the LE General Discoverable Mode and BR/EDR Not Supported bits to 1. The second data structure has type 0xff, which signifies that it’s manufacturer-specific data. The first two bytes of this manufacturer-specific data always represent the company ID. You can find all registered company identifiers (also called Company Identifier Code, or CIC) on https://www.bluetooth.com/specifications/assigned-numbers/company-identifiers/. Normally, the company ID is the device manufacturer’s ID. However, Apple allows other manufacturers to use its company ID for iBeacon devices if they agree to the license. Note: The company ID is a field of two bytes sent as a little-endian value. If you look at an iBeacon packet capture in Wireshark, the bytes sent are 4c 00. However, Apple’s real company ID is 00 4c, or 76 in decimal. With manufacturer-specific data, the format of all other bytes after the company ID is up to the manufacturer to specify. Many manufacturers don’t document this, but Apple does specify the structure of iBeacon advertising data.

● 56

Boek BLE 220329 UK.indd 56

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

In the document "Getting Started with iBeacon" (https://developer.apple.com/ibeacon/ Getting-Started-with-iBeacon.pdf), Apple explains the meaning of the iBeacon packet’s fields. The UUID, major, and minor are all identifying values for the iBeacon, but you can use them in a hierarchical nature. The Proximity UUID is a Universally Unique Identifier (UUID), a 16-byte (128-bit) value, generally written with hyphens between the different components, as in fda50693-a4e24fb1-afcf-c6eb07647825. It’s recommended that you generate a random UUID for use on all of your beacons. This way, you can distinguish your beacons from those outside of your control. Note: On Linux and macOS systems, you can easily generate a random UUID with the uuidgen command. You use the major and minor values to further subdivide your beacons into different groups. For instance, you could give all beacons on the same floor or in the same room the same major value. And, if there are multiple beacons on the same floor or in the same room, you give them a different minor value to distinguish them from each other. This way, the combination of UUID, major and minor value uniquely identifies each beacon. The last field of the iBeacon advertising data is Measured Power, which is used to calibrate your beacon. Apple describes the procedure in its "Getting Started with iBeacon" document. You have to install the beacon and then sample the signal strength at a one-meter distance for a minimum of 10 seconds. Average these values and write the average measured power to the device. The exact way depends on the manufacturer, but often it’s done by connecting to the device over BLE and then writing the value to a specific characteristic. After this, the iBeacon packets advertise this measured power. You can use the measured RSSI together with this measured power at one meter in the received packet to compute an approximate distance to the beacon. 22 So, have a look at an iBeacon’s manufacturer-specific data in Wireshark. Wireshark already decodes the length, type and company ID, and the rest of the bytes are shown in the Data part of the Manufacturer Specific advertising data structure. If you decode this data following the structure in Figure 3.9, you’ll get something like this:

22 The accuracy of BLE RSSI measurements depends largely on the physical materials surrounding the beacons, such as walls, human bodies and so on. So, while the RSSI correlates highly with distance, physical objects or human bodies between the sender and receiver can make the sender appear more far away.

● 57

Boek BLE 220329 UK.indd 57

06/05/2022 15:54

Bluetooth Low Energy Applications

Figure 3.10 Example of an iBeacon’s manufacturer-specific data Note: Measured power is a signed integer. This means that the value 0xd8 in this example (which is decimal value 216), isn’t 216 dBm, but rather 216 256 = 40 dBm.

3.6 Decoding iBeacon advertisements using Bleak Now that you know how to identify iBeacon advertisements with Wireshark and decode the format manually, let’s see how to do this in Python with Bleak. The basic structure of an iBeacon scanner is the scanner with detection callback from section 3.3.2. However, instead of just showing all detected devices’ advertisement data, you first have to filter for iBeacon advertisements. Knowing the structure of an iBeacon advertising packet, this is easy: • Check whether there’s manufacturer-specific data for company ID 0x004c • Check whether the data starts with 0x0215 As you’ve seen previously, Bleak already does some preprocessing: the manufacturer-specific data is turned into a dictionary with the company ID as the key and the rest of the data as the value. So, the first thing you’d do is determine whether there’s the 0x004c key in the manufacturer-specific data. If not, ignore the data. If there is, check whether the first two bytes are 0x0215. If not, ignore the data. If they are, parse the rest of the data and show the UUID, major, minor, and measured power. Summarizing this in a flowchart, the actions of the callback function should look like this:

Figure 3.11 The structure of an iBeacon scanner program

● 58

Boek BLE 220329 UK.indd 58

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

To parse the different fields in the iBeacon data packet, I’ll use Construct (https://construct. readthedocs.io), a Python library that helps with parsing and serialization of binary data. Install it first: pip3 install construct

Then, the Python code to decode an iBeacon packet looks like this: """Scan for iBeacons. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio from uuid import UUID from construct import Array, Byte, Const, Int8sl, Int16ub, Struct from construct.core import ConstError from bleak import BleakScanner from bleak.backends.device import BLEDevice from bleak.backends.scanner import AdvertisementData ibeacon_format = Struct( "type_length" / Const(b"\x02\x15"), "uuid" / Array(16, Byte), "major" / Int16ub, "minor" / Int16ub, "power" / Int8sl, )

def device_found( device: BLEDevice, advertisement_data: AdvertisementData ): """Decode iBeacon.""" try: apple_data = advertisement_data.manufacturer_data[0x004C] ibeacon = ibeacon_format.parse(apple_data) uuid = UUID(bytes=bytes(ibeacon.uuid)) print(f"UUID

: {uuid}")

print(f"Major

: {ibeacon.major}")

print(f"Minor

: {ibeacon.minor}")

print(f"TX power : {ibeacon.power} dBm") print(f"RSSI

: {device.rssi} dBm")

● 59

Boek BLE 220329 UK.indd 59

06/05/2022 15:54

Bluetooth Low Energy Applications

print(47 * "-") except KeyError: # Apple company ID (0x004c) not found pass except ConstError: # No iBeacon (type 0x02 and length 0x15) pass

async def main(): """Scan for devices.""" scanner = BleakScanner() scanner.register_detection_callback(device_found) while True: await scanner.start() await asyncio.sleep(1.0) await scanner.stop()

asyncio.run(main())

First, it defines a Struct object from the Construct library, and calls it ibeacon_format. A Struct is a collection of ordered (and usually named) fields. 23 Each field in itself is an instance of a Construct class. This is how you define bytes’ data type in an iBeacon data structure. In this case, the fields are: • Const(b"\x02\x15"): a two-byte value. This is a constant because this value is always fixed for an iBeacon data structure. • Array(16, Byte): an array of 16 bytes that define the UUID. • Int16ub for both the major and minor numbers, which are both unsigned big-endian 16-bit integers. • Int8sl for the measured power, which is a signed 8-bit integer. Now, when the device_found() function receives manufacturer-specific data from Apple, it can easily parse it. It just calls the parse() function on the ibeacon_format object, with the bytes of the manufacturer-specific data as its argument. The result is an object of the class construct.lib.containers.Container, with the fields that are defined in the ibeacon_ format struct. That’s why you can just refer to the fields such as ibeacon.major, ibeacon. minor, and ibeacon.power.

23

A Struct object in Construct is comparable to a struct in the C programming language.

● 60

Boek BLE 220329 UK.indd 60

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

However, ibeacon.uuid returns a construct.lib.containers.ListContainer object, printed as a list of separate numbers. To format it like a UUID, first convert it to bytes and then create a UUID object from these bytes. Note: In contrast to the flowchart in Figure 3.11, this Python program doesn’t explicitly check for the company ID and the first two bytes in the manufacturer-specific data. The code just assumes it’s received iBeacon data, and catches exceptions if this assumption is false: the KeyError exception is triggered if there’s no 0x004c key in the manufacturer_data dictionary, and ConstError if the first two bytes of the data don’t equal b"\x02\ x15". This common Python coding style is called EAFP (easier to ask for forgiveness than permission), and, in many cases, it makes the code easier to follow. The other style, that of testing for all conditions before, is called LBYL (look before you leap). If you run this program, it will scan continuously for iBeacons and show their information: $ python3 ibeacon_scanner.py UUID

: fda50693-a4e2-4fb1-afcf-c6eb07647825

Major

: 1

Minor

: 2

TX power : -40 dBm RSSI

: -80 dBm

----------------------------------------------UUID

: d1338ace-002d-44af-88d1-e57c12484966

Major

: 1

Minor

: 39904

TX power : -59 dBm RSSI

: -98 dBm

-----------------------------------------------

It will keep scanning indefinitely. Just press Ctrl+C to stop the program.

3.7 Discovering advertisements with NimBLE-Arduino Let’s move on to running some code on an ESP32 board. The NimBLE-Arduino project has a couple of examples. One of these is a heavily-commented Arduino sketch to scan for and report on all Bluetooth advertisements on the serial monitor: NimBLE_Scan_Continuous. ino. This code is perfect for introducing NimBLE-Arduino: /** Example of continuous scanning for BLE advertisements. * This example will scan forever while consuming as few resources as * possible and report all advertisments on the serial monitor. * * Created: on January 31 2021 *

Author: H2zero

* */

● 61

Boek BLE 220329 UK.indd 61

06/05/2022 15:54

Bluetooth Low Energy Applications

#include "NimBLEDevice.h" NimBLEScan *pBLEScan; class MyAdvertisedDeviceCallbacks : public NimBLEAdvertisedDeviceCallbacks { void onResult(NimBLEAdvertisedDevice *advertisedDevice) { Serial.printf("Advertised Device: %s \n", advertisedDevice->toString().c_str()); } }; void setup() { Serial.begin(115200); Serial.println("Scanning..."); /** *Optional* Sets the filtering mode used by the scanner in the * BLE controller. * *

Can be one of:

*

CONFIG_BTDM_SCAN_DUPL_TYPE_DEVICE (0) (default)

*

Filter by device address only, advertisements from the same

*

address will be reported only once.

* *

CONFIG_BTDM_SCAN_DUPL_TYPE_DATA (1)

*

Filter by data only, advertisements with the same data will only

*

be reported once, even from different addresses.

* *

CONFIG_BTDM_SCAN_DUPL_TYPE_DATA_DEVICE (2)

*

Filter by address and data, advertisements from the same address

*

will be reported only once, except if the data in the

*

advertisement has changed, then it will be reported again.

* *

Can only be used BEFORE calling NimBLEDevice::init.

*/ NimBLEDevice::setScanFilterMode(CONFIG_BTDM_SCAN_DUPL_TYPE_DEVICE); /** *Optional* Sets the scan filter cache size in the BLE * controller. When the number of duplicate advertisements seen by * the controller reaches this value it will clear the cache and * start reporting previously seen devices. The larger this number, * the longer time between repeated device reports. Range 10 - 1000. * (default 20) * *

Can only be used BEFORE calling NimBLEDevice::init.

*/

● 62

Boek BLE 220329 UK.indd 62

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

NimBLEDevice::setScanDuplicateCacheSize(200); NimBLEDevice::init(""); // Create new scan pBLEScan = NimBLEDevice::getScan(); // Set the callback for when devices are discovered, no duplicates. pBLEScan->setAdvertisedDeviceCallbacks( new MyAdvertisedDeviceCallbacks(), false); // Set active scanning, this will get more data from the advertiser. pBLEScan->setActiveScan(true); // How often the scan occurs / switches channels; in milliseconds, pBLEScan->setInterval(97); // How long to scan during the interval; in milliseconds. pBLEScan->setWindow(37); // Do not store the scan results, use callback only. pBLEScan->setMaxResults(0); } void loop() { // If an error occurs that stops the scan, it will be restarted // here. if (pBLEScan->isScanning() == false) { // Start scan with: duration = 0 seconds(forever), no scan end // callback, not a continuation of a previous scan. pBLEScan->start(0, nullptr, false); } delay(2000); }

The program consists of three major parts: a class with a callback function, the setup() function, and the loop() function. The latter two are part of every Arduino sketch. The setup() function has all initialization instructions, which run once when the device boots, and, after that, the loop() function is repeated continuously. At the top of your application code, you always need to add #include "NimBLEDevice.h". This provides access to all classes from NimBLE-Arduino. Then, you define a pointer to a NimBLEScan object called pBLEScan, because you need to be able to refer to that object both in setup() and loop(). Now, let’s have a look at the setup() function. After initializing the serial output, the code calls two of the NimBLEDevice class’s static member functions. By default, NimBLE-Arduino reports advertisements from the same Bluetooth address during the same scan only once. If this is fine for your application, you can remove this call to NimBLEDevice::setScanFilterMode(). In the next call, to NimBLEDevice::setScanDuplicateCacheSize(), you

● 63

Boek BLE 220329 UK.indd 63

06/05/2022 15:54

Bluetooth Low Energy Applications .

set the cache size of this scan filter. When the number of received duplicate advertisements reaches this value, the BLE controller clears the cache. Both calls are optional, but if you use them, you must call them before calling NimBLEDevice::init(). The latter initializes the NimBLE-Arduino library and prepares the Bluetooth stack for your application. In this case, an empty string is passed as an argument to the function. This means that this code won’t advertise a name. If you wanted your application to advertise a name, you would pass a string with the device’s name to the function. With pBLEScan = NimBLEDevice::getScan() you get a pointer to a scan object. This will be used for everything that has to do with scanning. In the next few lines, you use this object to set some properties. The most important one is pBLEScan->setAdvertisedDeviceCallbacks(new MyAdvertisedDeviceCallbacks(), false). This creates a new MyAdvertisedDeviceCallbacks object and uses this for callbacks when your program receives new advertisement data. Because the second argument is false, the callback function won’t be called on duplicate advertisements. The other function calls in the setup() function change some parameters for the scans. You first set it to an active scan (so the program sends a SCAN_REQ advertisement to request SCAN_RSP responses), then you set the scan interval (how often the scan occurs), the scan window (how long to scan during the interval), and the number of scan results to store. This example doesn’t store any scan results because the callback function processes them immediately. Note: As you can see from this example code, NimBLE-Arduino gives you a lot of flexibility in how you scan for BLE advertisements, in stark contrast to Bleak on a computer or Raspberry Pi. With all of this set up, the loop() function can be quite short. All it does is check whether the scan object is still scanning. If not, it restarts the scan. Then it adds a delay of two seconds. The loop() function executes repeatedly. The heart of this program is the MyAdvertisedDeviceCallbacks class – a subclass of NimBLEAdvertisedDeviceCallbacks. This is a class with one member function: onResult(). This function is called whenever a new scan result is detected, with a pointer to the device that was found. In this case, the callback function prints only a string representation of the device to the serial console. In the next section, you’ll see what other information you can get out of the NimBLEAdvertisedDevice object. Now, build this code for your ESP32 board by running the following from the NimBLE_Scan_ Continuous directory:

● 64

Boek BLE 220329 UK.indd 64

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

$ arduino-cli compile -b esp32:esp32:pico32 -e NimBLE_Scan_Continuous Sketch uses 544266 bytes (41%) of program storage space. Maximum is 1310720 bytes. Global variables use 26992 bytes (8%) of dynamic memory, leaving 300688 bytes for local variables. Maximum is 327680 bytes.

Make sure to change the argument after -b to your board’s fully-qualified board name. 24 With -e, the built firmware is copied to your Arduino sketch’s directory. The final argument is the path to your that directory. Note: The Arduino platform requires that you put your sketch’s .ino file in a directory with the same base name as it (without the .ino). So, in this case, you have a NimBLE_ Scan_Continuous directory containing the NimBLE_Scan_Continuous.ino file. Now connect the ESP32 board to your computer with a USB cable and upload the code to the it: $ arduino-cli upload -p /dev/ttyUSB0 -b esp32:esp32:pico32 -i NimBLE_Scan_ Continuous/build/esp32.esp32.pico32/NimBLE_Scan_Continuous.ino.bin esptool.py v3.0-dev Serial port /dev/ttyUSB0 Connecting.... Chip is ESP32-PICO-D4 (revision 1) Features: WiFi, BT, Dual Core, Embedded Flash, Coding Scheme None Crystal is 40MHz MAC: d8:a0:1d:40:54:34 Uploading stub... Running stub... Stub running... Changing baud rate to 921600 Changed. Configuring flash size... Auto-detected Flash size: 4MB Compressed 8192 bytes to 47... Wrote 8192 bytes (47 compressed) at 0x0000e000 in 0.0 seconds (effective 34205.8 kbit/s)... Hash of data verified. Compressed 17120 bytes to 11164... Wrote 17120 bytes (11164 compressed) at 0x00001000 in 0.2 seconds (effective 814.1 kbit/s)... Hash of data verified. Compressed 544384 bytes to 295901... Wrote 544384 bytes (295901 compressed) at 0x00010000 in 5.6 seconds (effective 24 The esp32:esp32:pico32 board name is for the ESP32-Pico-Kit, one of Espressif’s official development kits.

● 65

Boek BLE 220329 UK.indd 65

06/05/2022 15:54

Bluetooth Low Energy Applications

782.9 kbit/s)... Hash of data verified. Compressed 3072 bytes to 128... Wrote 3072 bytes (128 compressed) at 0x00008000 in 0.0 seconds (effective 7644.6 kbit/s)... Hash of data verified. Leaving... Hard resetting via RTS pin...

Note: Be sure to specify the correct port number, board name, and the full path to the .bin file when running the compiled sketch. If you now establish a serial connection to the board, you should see various BLE devices being detected. 25 For example: $ screen /dev/ttyUSB0 115200 Advertised Device: Name: , Address: 5b:24:a3:42:34:33, manufacturer data: 4c0010050318eadc89, txPower: 12 Advertised Device: Name: LYWSD02, Address: e7:2e:00:b1:38:96, serviceUUID: 0x181a Service Data: UUID: 0xfe95, Data: p [

8

Advertised Device: Name: Gigaset keeper, Address: 7c:2f:80:e3:a1:a7, manufacturer data: 80010215010080e3a1a7c5, serviceUUID: 0x1803 Advertised Device: Name: F15, Address: eb:76:55:b9:56:18, appearance: 193, manufacturer data: eff0eb7655b95618, serviceUUID: 0x180d Service Data: UUID: 0x180f, Data: Advertised Device: Name: Flower care, Address: c4:7c:8d:67:65:ad, serviceUUID: 0xfe95 Service Data: UUID: 0xfe95, Data: 1 Advertised Device: Name: , Address: d7:c1:a2:d6:43:61, manufacturer data: 990403c80703ce5b00affea203bf0b1d Advertised Device: Name: Ruuvi 7E0E, Address: c8:03:24:74:7e:0e, manufacturer data: 99040510a548fcce8d001cffdc0408b756288a2ec80324747e0e, serviceUUID: 6e400001-b5a3-f393-e0a9-e50e24dcca9e Advertised Device: Name: , Address: d7:c1:a2:d6:43:61, manufacturer data: 990403c80703ce5b00b0fea303c10b11

In the next section, you’ll see how to process all of this data.

3.8 Decoding manufacturer-specific data using NimBLE-Arduino While the basic Arduino sketch in the previous section just showed a string representation of each advertised device it detected, you can do a lot more in the callback function.

25

Consult the appendix at the end of this book for how you do this on your operating system.

● 66

Boek BLE 220329 UK.indd 66

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

In this section, you’ll learn how to decode manufacturer-specific data. I’ll show this for two types of data structures: iBeacons and Microsoft advertising beacons.

3.8.1 Decoding iBeacon advertisements NimBLE-Arduino already has built-in support for decoding iBeacons; this functionality is available in the NimBLEBeacon class. Here’s an example of how you decode iBeacon advertisements with NimBLE-Arduino: /** Scan for iBeacon advertisements. * * Copyright (c) 2021 Koen Vervloesem * * SPDX-License-Identifier: MIT * * Based on h2zero’s example BLE_Beacon_Scanner.ino. * */ #include "NimBLEBeacon.h" #include "NimBLEDevice.h" #define ENDIAN_CHANGE_U16(x) ((((x)&0xff00) >> 8) + (((x)&0xff) haveManufacturerData() == true) { std::string strManufacturerData = advertisedDevice->getManufacturerData(); if (strManufacturerData.length() == 25 && strManufacturerData[0] == 0x4c && strManufacturerData[1] == 0x00 && strManufacturerData[2] == 0x02 && strManufacturerData[3] == 0x15) { NimBLEBeacon iBeacon = NimBLEBeacon(); iBeacon.setData(strManufacturerData); Serial.printf("UUID

: %s\n",

iBeacon.getProximityUUID().toString().c_str()); Serial.printf("Major

: %d\n",

ENDIAN_CHANGE_U16(iBeacon.getMajor())); Serial.printf("Minor

: %d\n",

ENDIAN_CHANGE_U16(iBeacon.getMinor())); Serial.printf("TX power : %d dBm\n",

● 67

Boek BLE 220329 UK.indd 67

06/05/2022 15:54

Bluetooth Low Energy Applications

iBeacon.getSignalPower()); Serial.printf("RSSI

: %d dBm\n",

advertisedDevice->getRSSI()); Serial.println( "-----------------------------------------------"); } } } }; void setup() { Serial.begin(115200); Serial.println("Scanning for iBeacons..."); NimBLEDevice::init(""); pBLEScan = NimBLEDevice::getScan(); pBLEScan->setAdvertisedDeviceCallbacks( new MyAdvertisedDeviceCallbacks(), true); pBLEScan->setInterval(97); pBLEScan->setWindow(37); pBLEScan->setMaxResults(0); } void loop() { if (pBLEScan->isScanning() == false) { pBLEScan->start(0); } delay(2000); }

The setup() function looks roughly the same as in the previous section. However, there are some notable differences. This example doesn’t set a scan filter mode or duplicate cache size, and the callback should also be called with duplicates. This is because an iBeacon advertisement is, by definition, a duplicate: all of these advertisements from the same device are identical. You also don’t set active scanning, because you don’t need this for iBeacons: all the data you’re interested in can be read in a passive scan. The loop() function does the same as in the previous example. The only difference is that it uses a shorter version of the start() member function. This doesn’t use a scan end callback and uses the default value false for the continuation of a previous scan. Then, NimBLEAdvertisedDeviceCallbacks class’s onResult() function is where decoding the iBeacon happens. You first check whether the advertised device has manufacturer-specific data. If it has, this data is copied to a string. Then, you check the string’s length (25), whether the first two bytes are 0x4c and 0x00 (for Apple’s company ID) and that the next

● 68

Boek BLE 220329 UK.indd 68

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

two bytes are 0x02 and 0x15 (for the type and length of Apple data). Revisit section 3.5 for the details of the iBeacon format. If all of those conditions are true, you can simply create a NimBLEBeacon object, set its data to the received manufacturer-specific data and then read the UUID, major, minor and TX power. The bytes in the major and minor values should be swapped, and that’s what the ENDIAN_CHANGE_U16 macro is for. Note: You need to include the NimBLEBeacon.h header file to use the NimBLEBeacon class. The code then prints the iBeacon’s properties to the serial connection, including the device’s RSSI value, just as you did in the example using Bleak. If you build and then upload this code to your ESP32, you’ll see the iBeacon advertisements flowing by on the serial console: $ screen /dev/ttyUSB0 115200 UUID

: d1338ace-002d-44af-88d1-e57c12484966

Major

: 1

Minor

: 21337

TX power : -59 dBm RSSI

: -88 dBm

----------------------------------------------UUID

: d1338ace-002d-44af-88d1-e57c12484966

Major

: 1

Minor

: 21337

TX power : -59 dBm RSSI

: -84 dBm

-----------------------------------------------

3.8.2 Decoding Microsoft advertising beacons Because NimBLE-Arduino already has built-in support for iBeacon decoding, the previous section was easy. Let’s see how you would write your own decoder for other manufacturer-specific data. An interesting format is the Microsoft Advertising Beacon (https://docs. microsoft.com/en-us/openspecs/windows_protocols/ms-cdp/77b446d0-8cea-4821-ad21fabdf4d9a569). Many Windows devices send this type of advertisement. The manufacturer-specific data structure is specified as follows:

Figure 3.12 Structure of a Microsoft advertising beacon’s manufacturer-specific data

● 69

Boek BLE 220329 UK.indd 69

06/05/2022 15:54

Bluetooth Low Energy Applications

Microsoft’s company ID for manufacturer-specific data is 0x0006, which is sent as 0x0600 in little-endian mode. The Scenario type is always 0x01 for a Microsoft advertising beacon. The two highest bits of the Version and device type are always set to 0. The lower six bits code the device type according to the following table: Value

Meaning

1

Xbox One

6

Apple iPhone

7

Apple iPad

8

Android device

9

Windows 10 Desktop

11

Windows 10 Phone

12

Linus device

13

Windows IoT

14

Surface Hub

26

Table 3.1 Device type in Microsoft advertising beacon The three highest bits of Version and flags are set to 001, and the lower five bits to 0. Salt is a four-byte random number, and the Device hash is an SHA256 hash of the salt and the device ‘thumbprint,’ truncated to 16 bytes. An example of how you decode these Microsoft advertising beacons using NimBLE-Arduino looks like this: /** Scan for Microsoft advertising beacons. * * Copyright (c) 2021 Koen Vervloesem * * SPDX-License-Identifier: MIT * */ #include "NimBLEDevice.h" NimBLEScan *pBLEScan; // See // https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-cdp/77b446d08cea-4821-ad21-fabdf4d9a569 const char *deviceTypeMicrosoft[] = {"", "Xbox One", "", 26

Yes, Microsoft’s documentation page calls it a "Linus device" instead of a "Linux device".

● 70

Boek BLE 220329 UK.indd 70

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

"", "", "", "Apple iPhone", "Apple iPad", "Android device", "Windows 10 Desktop", "", "Windows 10 Phone", "Linus device", "Windows IoT", "Surface Hub"}; class MyAdvertisedDeviceCallbacks : public NimBLEAdvertisedDeviceCallbacks { void onResult(NimBLEAdvertisedDevice *advertisedDevice) { if (advertisedDevice->haveManufacturerData() == true) { std::string strManufacturerData = advertisedDevice->getManufacturerData(); uint8_t deviceType; uint8_t salt[4]; uint8_t deviceHash[16]; if (strManufacturerData[0] == 0x06 && strManufacturerData[1] == 0x00 && strManufacturerData[2] == 0x01) { deviceType = strManufacturerData[3] & 0b00111111; memcpy(&salt, &strManufacturerData[6], 4); memcpy(&deviceHash, &strManufacturerData[10], 16); Serial.printf("Device type: %s (%d)\n", deviceTypeMicrosoft[deviceType], deviceType); Serial.printf("RSSI

: %d dBm\n",

advertisedDevice->getRSSI()); Serial.printf("Salt

: ");

for (int i = 0; i < 4; i++) { Serial.printf("%02x", salt[i]); } Serial.println(); Serial.printf("Device hash: "); for (int i = 0; i < 16; i++) { Serial.printf("%02x", deviceHash[i]); } Serial.println();

● 71

Boek BLE 220329 UK.indd 71

06/05/2022 15:54

Bluetooth Low Energy Applications

Serial.println( "---------------------------------------------"); } } } }; void setup() { Serial.begin(115200); Serial.println("Scanning for Microsoft advertising beacons..."); NimBLEDevice::init(""); pBLEScan = NimBLEDevice::getScan(); pBLEScan->setAdvertisedDeviceCallbacks( new MyAdvertisedDeviceCallbacks(), true); pBLEScan->setInterval(97); pBLEScan->setWindow(37); pBLEScan->setMaxResults(0); } void loop() { if (pBLEScan->isScanning() == false) { pBLEScan->start(0); } delay(2000); }

As you see, much of the code resembles that of the iBeacon scanner from subsection 3.8.1. The only substantial difference is in the onResult() member function of the MyAdvertisedDeviceCallbacks class, which checks for manufacturer-specific data with the correct Company Identifier Code and extracts some information from it. Then, it shows this information in the serial console. Build this code and upload it to your ESP32, and connect to the serial console. If you have a Windows computer in the neighborhood, you should see some messages like these: $ screen /dev/ttyUSB0 115200 Device type: Windows 10 Desktop (9) RSSI

: -56 dBm

Salt

: c0b81cdf

Device hash: 9b0189566300efacc336eca5aa88f59f ---------------------------------------------

● 72

Boek BLE 220329 UK.indd 72

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

3.9 Broadcasting iBeacon advertisements with Zephyr The Zephyr project has a lot of sample applications with source code and instructions, and one of these is an iBeacon broadcaster (https://docs.zephyrproject.org/latest/samples/ bluetooth/ibeacon/README.html). As this is the first Zephyr example in this book, I’ll just use this existing sample with limited functionality and explain how it works. You’ll find that this deceptively simple application hides a lot of details that are important to know before starting with the more complex Zephyr code in the rest of this book. Here’s the sample code to broadcast an iBeacon advertisement: /* * Copyright (c) 2018 Henrik Brix Andersen * * SPDX-License-Identifier: Apache-2.0 */ #include #include #include #include #include #include #ifndef IBEACON_RSSI #define IBEACON_RSSI 0xc8 #endif /* * Set iBeacon demo advertisement data. These values are for * demonstration only and must be changed for production environments! * * UUID:

18ee1516-016b-4bec-ad96-bcb96d166e97

* Major: 0 * Minor: 0 * RSSI:

-56 dBm

*/ static const struct bt_data ad[] = { BT_DATA_BYTES(BT_DATA_FLAGS, BT_LE_AD_NO_BREDR), BT_DATA_BYTES(BT_DATA_MANUFACTURER_DATA, 0x4c, 0x00, /* Apple */ 0x02, 0x15,

/* iBeacon */

0x18, 0xee, 0x15, 0x16, /* UUID[15..12] */ 0x01, 0x6b,

/* UUID[11..10] */

0x4b, 0xec,

/* UUID[9..8] */

0xad, 0x96,

/* UUID[7..6] */

0xbc, 0xb9, 0x6d, 0x16, 0x6e, 0x97, /* UUID[5..0] */

● 73

Boek BLE 220329 UK.indd 73

06/05/2022 15:54

Bluetooth Low Energy Applications

0x00, 0x00,

/* Major */

0x00, 0x00,

/* Minor */

IBEACON_RSSI) /* Calibrated RSSI @ 1m */ }; static void bt_ready(int err) { if (err) { printk("Bluetooth init failed (err %d)\n", err); return; } printk("Bluetooth initialized\n"); /* Start advertising */ err = bt_le_adv_start(BT_LE_ADV_NCONN, ad, ARRAY_SIZE(ad), NULL, 0); if (err) { printk("Advertising failed to start (err %d)\n", err); return; } printk("iBeacon started\n"); } void main(void) { int err; printk("Starting iBeacon Demo\n"); /* Initialize the Bluetooth Subsystem */ err = bt_enable(bt_ready); if (err) { printk("Bluetooth init failed (err %d)\n", err); } }

First, you include a couple of header files. For this and all other Zephyr examples in this book, the Bluetooth headers are important to include. Then, define the IBEACON_RSSI constant, which sets the measured power in the iBeacon advertisement. The value 0xc8 is equal to decimal value 200. As this field in the iBeacon’s specification’s advertising data structure is encoded as a signed integer, this is interpreted as 200 256 = 56 dBm.

3.9.1 Advertising data structures in Zephyr Then, you see the definition of an array of struct bt_data elements. The bt_data struct describes an advertising data structure with a type, length, and a pointer to the data. Its definition (in bluetooth/bluetooth.h) is:

● 74

Boek BLE 220329 UK.indd 74

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

/** Description of different data types that can be encoded into * advertising data. Used to form arrays that are passed to the * bt_le_adv_start() function. */ struct bt_data { u8_t type; u8_t data_len; const u8_t *data; };

In the iBeacon program, the array consists of two elements. This shouldn’t be a surprise, as you saw in section 3.5 that an iBeacon advertising packet consists of two advertising structures: flags and manufacturer-specific data. In principle, you could fill the array with bt_data structs you create yourself, but Zephyr defines some helper macros. Their definition (again, from bluetooth/bluetooth.h) looks like this: /** @brief Helper to declare elements of bt_data arrays * *

This macro is mainly for creating an array of struct bt_data

*

elements which is then passed to bt_le_adv_start().

* *

@param _type Type of advertising data field

*

@param _data Pointer to the data field payload

*

@param _data_len Number of bytes behind the _data pointer

*/ #define BT_DATA(_type, _data, _data_len) \ { \ .type = (_type), \ .data_len = (_data_len), \ .data = (const u8_t *)(_data), \ } /** @brief Helper to declare elements of bt_data arrays * *

This macro is mainly for creating an array of struct bt_data

*

elements which is then passed to bt_le_adv_start().

* *

@param _type Type of advertising data field

*

@param _bytes Variable number of single-byte parameters

*/ #define BT_DATA_BYTES(_type, _bytes...) \ BT_DATA(_type, ((u8_t []) { _bytes }), \ sizeof((u8_t []) { _bytes }))

● 75

Boek BLE 220329 UK.indd 75

06/05/2022 15:54

Bluetooth Low Energy Applications

So, with the BT_DATA macro, you construct a bt_data struct with the type, data, and data length that you specify (using a pointer to the data). BT_DATA_BYTES is a macro that uses the BT_DATA macro and automatically fills it with the right length for data with a known size. If you return to the iBeacon program, you see that it fills the array with two elements using the BT_DATA_BYTES macro. The first element is of type BT_DATA_FLAGS and with data BT_LE_AD_NO_BREDR. You can find the definition of these constants in bluetooth/gap.h. 27 The second element is of type BT_DATA_MANUFACTURER_DATA. The data used to construct this element follows the format specified in Figure 3.9: Apple’s two-byte company ID, the type (0x02), and length (0x15) for the iBeacon data type, and then the UUID, major, minor, and measured power. 28 All in all, the result of all these macros is an array of advertising data structures that look like this, schematically:

Figure 3.13 An iBeacon packet’s advertisement data in Zephyr

3.9.2 Enabling Bluetooth Until now, the code has been all about setting up the right data structures to advertise. Let’s have a look at the main() function. This actually does just one thing: calling the bt_ enable() function. You have to call this function before any other calls that use the board’s Bluetooth hardware; it initializes the Bluetooth subsystem and returns 0 on success and a negative error code otherwise. 27 28

For non-connectable advertising packets, the flags data structure is optional. The major and minor fields are generally set to zero when they aren’t used.

● 76

Boek BLE 220329 UK.indd 76

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

The argument to the bt_enable() function is a callback function that’s called when initializing Bluetooth is completed. It’s in this callback function, bt_ready(), that the core of the program resides.

3.9.3 Advertising The callback function is called with one argument – an error code that is 0 on success, or an error code consisting of a negative value otherwise. So, in bt_ready(), you first check whether Bluetooth initialization failed, and, if not, you start advertising. The definition of bt_le_adv_start(), the function used to start advertising, is: /** @brief Start advertising * *

Set advertisement data, scan response data, advertisement parameters

*

and start advertising.

* *

@param param Advertising parameters.

*

@param ad Data to be used in advertisement packets.

*

@param ad_len Number of elements in ad

*

@param sd Data to be used in scan response packets.

*

@param sd_len Number of elements in sd

* *

@return Zero on success or (negative) error code otherwise.

*/ int bt_le_adv_start(const struct bt_le_adv_param *param, const struct bt_data *ad, size_t ad_len, const struct bt_data *sd, size_t sd_len);

So, to call this function, you need to supply advertising parameters, the advertising data structure’s array and its length, as well as the array of data structures for the scan response packet and its length. The iBeacon example program doesn’t use any scan response data, so the last two arguments are NULL and 0. The array of advertising data structures is ad, which you constructed in the beginning of the code. With ARRAY_SIZE, a macro defined in sys/util.h, you compute the number of elements in the array. When using a C compiler, this macro is defined as: #define ARRAY_SIZE(array) (sizeof(array) / sizeof((array)[0]))

The first argument to the bt_le_adv_start() function specifies the advertising parameters. In this example, this argument is BT_LE_ADV_NCONN. if you look up its definition in bluetooth/bluetooth.h, you’ll see that this defines non-connectable advertising with a private address:

● 77

Boek BLE 220329 UK.indd 77

06/05/2022 15:54

Bluetooth Low Energy Applications

/** Non-connectable advertising with private address */ #define BT_LE_ADV_NCONN BT_LE_ADV_PARAM(0, BT_GAP_ADV_FAST_INT_MIN_2, \ BT_GAP_ADV_FAST_INT_MAX_2, NULL) The BT_LE_ADV_PARAM macro is another helper macro: /** * @brief Helper to declare advertising parameters inline * * @param _options

Advertising Options

* @param _int_min

Minimum advertising interval

* @param _int_max

Maximum advertising interval

* @param _peer

Peer address, set to NULL for undirected advertising or

*

address of peer for directed advertising.

*/ #define BT_LE_ADV_PARAM(_options, _int_min, _int_max, _peer) \ ((struct bt_le_adv_param[]) { \ BT_LE_ADV_PARAM_INIT(_options, _int_min, _int_max, _peer) \ })

This teaches you that BT_GAP_ADV_FAST_INT_MIN_2 and BT_GAP_ADV_FAST_INT_ MAX_2 are the minimum and maximum advertising intervals. You can find their definitions in bluetooth/gap.h: #define BT_GAP_ADV_FAST_INT_MIN_2

0x00a0

/* 100 ms

*/

#define BT_GAP_ADV_FAST_INT_MAX_2

0x00f0

/* 150 ms

*/

So, your iBeacon will use an advertising interval of between 100 ms and 150 ms. BT_LE_ADV_PARAM uses another macro, BT_LE_ADV_PARAM_INIT, and its definition is: /** * @brief Initialize advertising parameters * * @param _options

Advertising Options

* @param _int_min

Minimum advertising interval

* @param _int_max

Maximum advertising interval

* @param _peer

Peer address, set to NULL for undirected advertising or

*

address of peer for directed advertising.

*/ #define BT_LE_ADV_PARAM_INIT(_options, _int_min, _int_max, _peer) \ { \ .id = BT_ID_DEFAULT, \ .sid = 0, \ .secondary_max_skip = 0, \ .options = (_options), \ .interval_min = (_int_min), \

● 78

Boek BLE 220329 UK.indd 78

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

.interval_max = (_int_max), \ .peer = (_peer), \ }

All in all, this means that the code starts non-connectable advertising with its default Bluetooth ID, a private address, no special advertising options, and with an advertising interval of between 100 ms and 150 ms. Note: Don’t feel discouraged if all these macros seem daunting at first. After you’ve seen a couple of Zephyr programs, you start to see how they all fit together. The beauty of Zephyr is that its source code is completely open. If you don’t understand what a specific macro or function is doing, just look up its declaration in the header file or even in the corresponding C file where it’s implemented.

3.9.4 Building and flashing the code Now that you understand what the code does, it’s almost time to build it. First, you need to know something about Zephyr’s build system, which is based on CMake (https://www. cmake.org). Building a Zephyr application builds both the application and Zephyr itself, and compiles them into a single binary that you flash to the board. In its simplest form, a Zephyr application consists of the following files: ├── CMakeLists.txt ├── prj.conf └── src └── main.c

The main.c file is the application’s source code, which you already saw in the previous subsections. For more complex applications, you can have more C files in this src directory, or even files written in assembly language. The CMakeLists.txt file tells Zephyr’s build system where to find all application files. For this project, it looks like this: # SPDX-License-Identifier: Apache-2.0 cmake_minimum_required(VERSION 3.13.1) find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE}) project(ibeacon) target_sources(app PRIVATE src/main.c) if(IBEACON_RSSI) zephyr_compile_definitions(IBEACON_RSSI=${IBEACON_RSSI}) endif()

● 79

Boek BLE 220329 UK.indd 79

06/05/2022 15:54

Bluetooth Low Energy Applications

This specifies that this project uses the Zephyr package defined in the ZEPHYR_BASE environment variable and that its sources are in the src/main.c file. It also specifies that you can define the IBEACON_RSSI variable to set the variable with the same name in the C file. The prj.conf file provides a Kconfig configuration file, which specifies application-specific values for one or more kernel configuration options. For this project, it looks like this: CONFIG_BT=y CONFIG_BT_DEBUG_LOG=y

The first option enables Bluetooth and the second one enables the debug log for Bluetooth. Now, if you want to build this project for the nRF52840 Dongle, go into your Zephyr installation’s zephyr directory and then run: west build -b nrf52840dongle_nrf52840 samples/bluetooth/ibeacon

If all goes well, it ends without any error messages. If you want to set the IBEACON_RSSI variable from the command line, you can do this like this: west build -b nrf52840dongle_nrf52840 samples/bluetooth/ibeacon -- -DIBEACON_RSSI=0xd8

Then, flash the code to the board. How to do this depends on the board and the bootloader you’re using. Have a look at subsection 2.3.2 in the previous chapter for the details.

3.9.5 Investigating the advertised packets Now that your board is running the iBeacon firmware, you can discover it using the iBeacon scanner from section 3.6 (with Python) or section 3.8 (with the ESP32). You should see the UUID 18ee1516-016b-4bec-ad96-bcb96d166e97, major 0, minor 0, and a TX power of 56 dBm (if you didn’t change the IBEACON_RSSI value) or 40 dBm (if you changed the IBEACON_RSSI value to 0xd8). Now, start a Bluetooth Low Energy capture with Wireshark. Normally, you don’t know the Bluetooth address of an iBeacon, and this isn’t relevant anyway: all relevant information is in the manufacturer-specific data. Luckily, Wireshark has a powerful display filter language. If you add a display filter to the text field above the nRF sniffer toolbar, you can limit the shown Bluetooth packets to the ones you’re interested in. To limit the packets shown to those with Apple’s manufacturer-specific data, add the following display filter: btcommon.eir_ad.entry.company_id == 0x004c

● 80

Boek BLE 220329 UK.indd 80

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

Note: Wireshark automatically does the conversion from big endian (0x004c) to little endian (0x4c00) for the company ID. You can even limit it more, to only show iBeacon data: (btcommon.eir_ad.entry.company_id == 0x004c) && (btcommon.eir_ad.entry.data[:2] == 02:15)

This limits the packets shown to ones with manufacturer-specific data of company ID 0x004c and with the first two bytes equal to 0x0215. But, how do you create these display filters? To filter on company ID, just click on the Company ID field in the packet details pane of an iBeacon packet, right-click and then choose Apply as Filter and then Selected. This adds a display filter for all packets with the selected value for the company ID. The extra filter for the first two bytes is a bit more work to figure out. Just select the full Data field of in the packet details pane of an iBeacon packet, right-click, then choose Apply as Filter and then …and Selected. This adds the filter as an extra requirement to the already active filter. But now you’re filtering on all packets with exactly the same data as this packet: btcommon.eir_ad.entry.data == 02:15:18:ee:15:16:01:6b:4b:ec:ad:96:bc:b9:6d:16:6e:97:00:00:00:00:d8

If you just want to filter on the first two bytes, adding [:2] to the data field does the trick. These first two bytes are then compared to 02:15. Now, with the packets limited to iBeacon packets, click on a packet and have a closer look at the Packet Details pane. The PDU Type in the packet header is ADV_NCONN_IND and the Tx Address is Random, which is equivalent to the BT_LE_ADV_NCONN advertising parameter set in Zephyr. If you click on the Advertising Data, you see two advertising data structures, Flags and Manufacturer Specific. The Flags data structure has the BR/EDR Not Supported set to 1, which was done in the Zephyr code with BT_LE_AD_NO_BREDR. And the Manufacturer Specific data structure has the data that you’ve seen described a couple of times now in the previous sections. Note: The Length field of an advertising data structure is equal to the length of the data including the Type field: 2 for the flags and 26 for the manufacturer-specific data. Compare this with the bt_data struct in Zephyr: the data_len field there is the length of the data field only: 1 for the flags and 25 for the manufacturer-specific data.

● 81

Boek BLE 220329 UK.indd 81

06/05/2022 15:54

Bluetooth Low Energy Applications

3.10 Broadcasting sensor data as manufacturer-specific data with Zephyr You can use manufacturer-specific data to broadcast any data, for instance from a connected sensor. However, the first two bytes of the manufacturer-specific data must be a valid company ID, which is assigned to Bluetooth SIG members. You’re not allowed to just broadcast any company ID; you have to use the company ID that is assigned to you, or for which you have received a license from the owner to use. So, if you want to release a BLE product that advertises manufacturer-specific data, you should apply for a Bluetooth SIG membership, which gives you your own unique company ID. However, until you have a company ID assigned, you can use the special value 0xffff, but only for internal and interoperability tests. The example in this section uses 0xffff. Warning: You can’t use the company ID 0xffff in end products you’re shipping to customers. Let’s create a Zephyr app that reads temperature, pressure, and humidity from a connected Bosch BME280 sensor, and advertises the sensor values in the manufacturer-specific data.

3.10.1 Hardware You can do the hardware part in two ways. For testing purposes, you can solder headers to Nordic Semiconductor’s nRF52840 Dongle and then connect a BME280 sensor I²C breakout board as follows: BME280

nRF52840 Dongle

SDA

0.31

SCL

0.29

GND

GND

VCC

VDD

Table 3.2 Connecting the BME280 to the nRF52840 Dongle. This looks like this:

Figure 3.14 Connecting the BME280 sensor to the nRF52840 Dongle using I²C

● 82

Boek BLE 220329 UK.indd 82

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

Note: The nRF52840 Development Kit is already populated with female headers. You can use the same pins on that board. For a more reliable method, choose (or design) a board with built-in BME280 sensor. The RuuviTag environmental sensor (https://ruuvi.com/ruuvitag/) is an example that works with the code in this section: 29

Figure 3.15 The RuuviTag is an interesting environmental sensor supported by Zephyr. Note: To be able to flash your Zephyr application to the RuuviTag (board name ruuvi_ ruuvitag), the RuuviTag Development Kit (https://ruuvi.com/products/ruuvitagdevelopment-kit/) is recommended. This is essentially a Nordic Semiconductor nRF52DK evaluation kit with a custom development shield for mounting the RuuviTag. An alternative way to flash the RuuviTag is with the SWDIO and SWDCLK pins located on the back of the sensor board connected to a hardware debugger with SWD support, such as the Black Magic Probe (https://1bitsquared.com/products/black-magic-probe).

3.10.2 Project structure The project’s directory structure looks like this: ├── CMakeLists.txt ├── nrf52840dongle_nrf52840.overlay ├── nrf52840dk_nrf52840.overlay ├── prj.conf └── src ├── bme280.c ├── bme280.h └── main.c

The CMakeLists.txt file specifies that the project’s sources are in two files: src/bme280.c and src/main.c:

29

There are various hardware variants of the RuuviTag. In recent versions, the Sensirion SHTC3 sensor (which is also supported by Zephyr) has replaced the Bosch BME280 with an alternative. See https://f. ruuvi.com/t/ruuvitag-hw-versions/5150/3 for more information.

● 83

Boek BLE 220329 UK.indd 83

06/05/2022 15:54

Bluetooth Low Energy Applications

# SPDX-License-Identifier: MIT cmake_minimum_required(VERSION 3.13.1) find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE}) project(advertise_bme280) target_sources(app PRIVATE src/bme280.c src/main.c) Then there’s the nrf52840dongle_nrf52840.overlay file: /* * Copyright (c) 2021 Koen Vervloesem * * SPDX-License-Identifier: MIT */ /* * Configuration of a BME280 device on an I2C bus. * * Device address 0x76 is assumed. Your device may have a different * address; check your device documentation if unsure. */ &i2c0 { status = "okay"; sda-pin = ; scl-pin = ; bme280@76 { compatible = "bosch,bme280"; reg = ; label = "BME280_I2C"; }; };

A devicetree is a hierarchical data structure that describes hardware. Zephyr uses a devicetree to describe the hardware available on its supported boards. If you look up the RuuviTag’s devicetree, (in zephyr/boards/arm/ruuvi_ruuvitag/ruuvi_ruuvitag.dts), you’ll see that it defines a BME280 sensor on the SPI bus. Obviously, the nRF52840 Dongle’s devicetree (in zephyr/boards/arm/nrf52840dongle_nrf52840/nrf52840dongle_nrf52840. dts) lacks the BME280 sensor definition, because one isn’t built in. However, you can add a definition with a devicetree overlay. This overlay defines a BME280 sensor on the I²C bus with SDA pin 31 and SCL pin 29, with address 0x76. So, with this overlay, you can use the BME280 sensor connected to pins 0.31 (SDA) and 0.29 (SCL). Note: The Adafruit BME280 breakout boards have a default I²C address of 0x77, while many other boards use I²C address 0x76. Change the address in the devicetree overlay according to your breakout board.

● 84

Boek BLE 220329 UK.indd 84

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

Because the devicetree overlay’s filename is the board name, nrf52840dongle_nrf52840, with the .overlay extension, Zephyr’s build system automatically detects if you’re building the project for the nRF52840 Dongle. If you’re building the project for another board, it will ignore this overlay. I’ve also added an nrf52840dk_nrf52840.overlay devicetree overlay with the same content if you want to run this example on an nRF52840 Development Kit. The prj.conf file enables Bluetooth, I²C, the sensor subsystem, and the BME280 component: # Enable Bluetooth CONFIG_BT=y # Enable BME280 sensor CONFIG_I2C=y CONFIG_SENSOR=y CONFIG_BME280=y

3.10.3 Source code The bme280.c file has functions to: • • • • •

get a devicetree node describing a Bosch BME280 sensor fetch a sample of the sensor read the temperature read the pressure read the humidity

/* * Read BME280 sensor data. * * Copyright (c) 2021 Koen Vervloesem * * SPDX-License-Identifier: MIT */ #include #include #include #include #include /* * Get a device structure from a devicetree node with compatible * "bosch,bme280". (If there are multiple, just pick one.) */ const struct device *bme280_get_device(void) {

● 85

Boek BLE 220329 UK.indd 85

06/05/2022 15:54

Bluetooth Low Energy Applications

const struct device *dev = DEVICE_DT_GET_ANY(bosch_bme280); if (dev == NULL) { /* No such node, or the node does not have status "okay". */ printk("\nError: no device found.\n"); return NULL; } if (!device_is_ready(dev)) { printk("\nError: Device \"%s\" is not ready; " "check the driver initialization logs for errors.\n", dev->name); return NULL; } printk("Found device \"%s\", getting sensor data\n", dev->name); return dev; } void bme280_fetch_sample(const struct device *dev) { sensor_sample_fetch(dev); } int16_t bme280_get_temperature(const struct device *dev) { struct sensor_value temperature; sensor_channel_get(dev, SENSOR_CHAN_AMBIENT_TEMP, &temperature); return (int16_t)(temperature.val1 * 100 + temperature.val2 / 10000); } uint16_t bme280_get_pressure(const struct device *dev) { struct sensor_value pressure; uint32_t p; // Pressure without offset sensor_channel_get(dev, SENSOR_CHAN_PRESS, &pressure); p = (uint32_t)(pressure.val1 * 1000 + pressure.val2 / 10000); return (uint16_t)(p - 50000); } uint16_t bme280_get_humidity(const struct device *dev) { struct sensor_value humidity; sensor_channel_get(dev, SENSOR_CHAN_HUMIDITY, &humidity); return (uint16_t)(humidity.val1 * 100 + humidity.val2 / 10000); }

● 86

Boek BLE 220329 UK.indd 86

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

An in-depth explanation of these functions is out this book’s scope. For this program, it suffices to know that sensor devices in Zephyr return their results as a sensor_value struct with two int32_t members: val1 is the integer part of the value and val2 is the fractional part of the value, in one-millionth parts. The calculations in the functions to read the temperature, pressure, and humidity are just meant to be able to store the values as 16-bit integers. You can read a more in-depth explanation in Zephyr’s documentation about the sensor subsystem (https://docs.zephyrproject.org/latest/reference/peripherals/sensor.html). Note: Most pressure values will be around 100,000 Pa (1000 hPa). The function to read the pressure value subtracts 50,000 Pa so that the code can store this value in a 16-bit unsigned integer (the range is from 0 to 65,535). This is a trick I learned from the advertising data format of the original RuuviTag firmware (https://docs.ruuvi.com/ communication/bluetooth-advertisements/data-format-5-rawv2). The corresponding header file with the function declarations looks like this: /* * Read BME280 sensor data. * * Copyright (c) 2021 Koen Vervloesem * * SPDX-License-Identifier: MIT */ #ifndef BME280_H_ #define BME280_H_ const struct device *bme280_get_device(void); void bme280_fetch_sample(const struct device *dev); int16_t bme280_get_temperature(const struct device *dev); uint16_t bme280_get_pressure(const struct device *dev); uint16_t bme280_get_humidity(const struct device *dev); #endif /* BME280_H_ */

Then, the main C file looks like this: /* * Advertise BME280 sensor data in BLE manufacturer-specific data. * * Copyright (c) 2021 Koen Vervloesem * * SPDX-License-Identifier: MIT */

● 87

Boek BLE 220329 UK.indd 87

06/05/2022 15:54

Bluetooth Low Energy Applications

#include #include #include #include #include #include #include #include "bme280.h" #define ADV_PARAM

\

BT_LE_ADV_PARAM(0, BT_GAP_ADV_SLOW_INT_MIN,

\

BT_GAP_ADV_SLOW_INT_MAX, NULL) static struct bt_data ad[] = { BT_DATA_BYTES(BT_DATA_FLAGS, BT_LE_AD_NO_BREDR), BT_DATA_BYTES( BT_DATA_MANUFACTURER_DATA, 0xff, 0xff, /* Test company ID */ 0x00, 0x00, /* Temperature, int16, little-endian */ 0x00, 0x00, /* Pressure - 50000, uint16, little-endian */ 0x00, 0x00) /* Humidity, uint16, little-endian */ }; void update_ad_bme280(const struct device *dev) { int16_t temperature; uint16_t pressure, humidity; bme280_fetch_sample(dev); temperature = bme280_get_temperature(dev); memcpy(&(ad[1].data[2]), &temperature, 2); pressure = bme280_get_pressure(dev); memcpy(&(ad[1].data[4]), &pressure, 2); humidity = bme280_get_humidity(dev); memcpy(&(ad[1].data[6]), &humidity, 2); } void main(void) { int err; printk("Starting firmware...\n"); // Initialize BME280 const struct device *bme280 = bme280_get_device();

● 88

Boek BLE 220329 UK.indd 88

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

if (bme280 == NULL) { return; } // Initialize the Bluetooth subsystem err = bt_enable(NULL); if (err) { printk("Bluetooth init failed (err %d)\n", err); return; } printk("Bluetooth initialized\n"); // Start advertising sensor values update_ad_bme280(bme280); err = bt_le_adv_start(ADV_PARAM, ad, ARRAY_SIZE(ad), NULL, 0); if (err) { printk("Advertising failed to start (err %d)\n", err); return; } while (1) { k_sleep(K_MSEC(980)); // Update advertised sensor values update_ad_bme280(bme280); err = bt_le_adv_update_data(ad, ARRAY_SIZE(ad), NULL, 0); if (err) { printk("Advertising update failed (err %d)\n", err); return; } } }

This is actually not much more complex than the iBeacon example. First, be sure to include the BME280 functions’ header file. After this, the code consists of three parts: define the advertising parameters and advertising data, define a function to update these advertising data with sensor values, and then the main() function. In the iBeacon example, you just used BT_LE_ADV_NCONN for the advertising parameters. As you saw there, this is a macro that comes down to non-connectable advertising with its default Bluetooth ID, a private address, no special advertising options, and an advertising interval of between 100 ms and 150 ms.

● 89

Boek BLE 220329 UK.indd 89

06/05/2022 15:54

Bluetooth Low Energy Applications

In this case, you don’t want to advertise every 100 ms. You’re probably fine with a new sensor value every second. So, how do you change this? Return to section 3.9.3 for the BT_LE_ADV_NCONN macro. You can use its expansion, but with other values for the minimum and maximum advertising interval: #define ADV_PARAM BT_LE_ADV_PARAM(0, BT_GAP_ADV_SLOW_INT_MIN,BT_GAP_ADV_SLOW_INT_ MAX, NULL)

You can find the definition of the interval values in bluetooth/gap.h: #define BT_GAP_ADV_SLOW_INT_MIN

0x0640

/* 1 s

*/

#define BT_GAP_ADV_SLOW_INT_MAX

0x0780

/* 1.2 s

*/

As in the iBeacon example, the advertising data in the bt_data struct consists of two advertising data structures: flags and manufacturer-specific data. For the manufacturer-specific data, this code uses test company ID 0xffff, and the next six bytes contain the sensor’s measured temperature, pressure, and humidity. These are initially just filled with 0s, because the program will update them later. Updating the manufacturer-specific data with new sensor values is the task of the update_ ad_bme280() function. Its single argument is a pointer to a device struct. With bme280_ fetch_sample(dev) (one of the functions defined in bme280.c) you fetch a sample from the sensor and store it in a BME280 driver internal buffer. Then, you read the sensor’s temperature, pressure, and humidity (again with the functions you defined in bme280.c). You copy each of these results to the appropriate position in the advertising data. For instance, have a look at the temperature, which is copied like this: memcpy(&(ad[1].data[2]), &temperature, 2);

Now look at the advertisement data you have defined after the temperature is copied:

Figure 3.16 The advertisement data with the BME280 sensor data in Zephyr With ad[1] you select the advertising data structure with the manufacturer-specific data. The data field then points to the bytes of the manufacturer-specific data. There, the third and fourth bytes correspond to the temperature. So, &(ad[1].data[2]) is the address of

● 90

Boek BLE 220329 UK.indd 90

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

this third byte in the manufacturer-specific data, and the memcpy() function copies the two temperature bytes to this location. The main() function first initializes the BME280 sensor and then initializes the Bluetooth subsystem. If both initializations succeed, it calls update_ad_bme280(bme280) to update the manufacturer-specific data with new sensor values. Then, you start advertising with the bt_le_adv_start() function. Note that for the advertising parameters you use the ADV_PARAM macro you defined before. After this first advertising event, you don’t have to stop advertising, update the advertising data, and the start advertising again; Zephyr has a function specifically to update advertising data when you’re already advertising: bt_le_adv_update_data(). It has the same arguments as the bt_le_adv_start() function, but without the advertising parameters (which stay the same). So, the rest of the program is just an infinite loop where you update the advertising data structure with the sensor values and then propagate this new data to the advertising that is in progress. Now, build and flash the firmware. Then, confirm with the nRF Connect mobile app or with Wireshark that your sensor board is indeed advertising every second.

Figure 3.17 You can see your sensor board’s manufacturer-specific data in nRF Connect. Note that the data is sent little-endian: 0x0e0a is actually 0x0a0e or 2574 (25.74 °C). Note: A helpful display filter in Wireshark for displaying only your own devices (which use the test company ID in manufacturer-specific data) is btcommon.eir_ad.entry. company_id == 0xffff.

● 91

Boek BLE 220329 UK.indd 91

06/05/2022 15:54

Bluetooth Low Energy Applications

3.10.4 Decoding the BME280 sensor data Now that your sensor board is advertising the BME280 sensor’s temperature, pressure, and humidity, it’s time to decode its manufacturer-specific data. With Bleak and Construct, it’s quite easy to decode this data: """Read BME280 sensor values from BLE advertisement data. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio from construct import Int16sl, Int16ul, Struct from construct.core import StreamError from bleak import BleakScanner from bleak.backends.device import BLEDevice from bleak.backends.scanner import AdvertisementData bme280_format = Struct( "temperature" / Int16sl, "pressure" / Int16ul, "humidity" / Int16ul, )

def device_found( device: BLEDevice, advertisement_data: AdvertisementData ): """Decode BME280 sensor values from advertisement data.""" try: data = advertisement_data.manufacturer_data[0xFFFF] sensor_data = bme280_format.parse(data) print(f"Device

: {device.name}")

print(f"Temperature: {sensor_data.temperature / 100} °C") print(f"Humidity

: {sensor_data.humidity / 100} %")

print( f"Pressure

: {(sensor_data.pressure + 50000) / 100} hPa"

) print(24 * "-") except KeyError: # Test company ID (0xffff) not found pass except StreamError:

● 92

Boek BLE 220329 UK.indd 92

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

# Wrong format pass

async def main(): """Register detection callback and scan for devices.""" scanner = BleakScanner() scanner.register_detection_callback(device_found) while True: await scanner.start() await asyncio.sleep(1.0) await scanner.stop()

asyncio.run(main())

In the beginning, you define a struct with the three fields in your manufacturer-specific data: a signed 16-bit integer and two unsigned 16-bit integers, all of them little-endian. Then, in the callback function that’s called on any found device (device_found()), you look for the manufacturer-specific data for company ID 0xffff and parse it with the Construct struct. If this works, the code shows the temperature, humidity, and pressure. The temperature and humidity should only be divided by 100 to get the real value. For the pressure, you first need to add the offset of 50,000 Pa that the bme280_get_pressure() function in bme280.c previously subtracted in order to fit the value into a 16-bit unsigned integer. Then, you need only some error handling. If the manufacturer-specific data is of another company ID, just ignore it. And, if parsing the data with Construct results in a StreamError, also ignore it. If you run this Python script while your board is advertising its sensor values, the result looks like this: $ python3 bme280_scanner.py Temperature : 25.57 °C Humidity

: 53.08 %

Pressure

: 1020.0 hPa

-----------------------Temperature : 25.57 °C Humidity

: 53.09 %

Pressure

: 1020.0 hPa

------------------------

This will keep scanning indefinitely. Just press Ctrl+C to stop the program.

● 93

Boek BLE 220329 UK.indd 93

06/05/2022 15:54

Bluetooth Low Energy Applications

3.11 Advertise scan response data with Zephyr There’s still one thing you could add to your BME280 advertising firmware: scan response data. What’s the point of scan response data? As you remember from section 3.3.3, with an active scan, the scanned device replies with a SCAN_RSP packet. Because this needs an explicit SCAN_REQ packet from the scanner and a SCAN_RSP from the broadcaster, as an addition to the already-advertised packets, this is more expensive in energy terms. So, scan response data is interesting for mostly static data, such as a device name. It wouldn’t be efficient to advertise the device name in every packet with manufacturer-specific data. But, you can advertise it as scan response data every time the device receives a scan request. The flow of messages then looks like this:

Figure 3.18 The sensor board responds to an active scan with its device name. Note: The regular advertisements are of the type ADV_SCAN_IND instead of ADV_ NONCONN_IND. This is undirected advertising which can’t be connected to, but which can respond to a scan request. The necessary changes to your main.c are minimal. First add a bt_data array with the scan response data: static const struct bt_data sd[] = { BT_DATA(BT_DATA_NAME_COMPLETE, CONFIG_BT_DEVICE_NAME, (sizeof(CONFIG_BT_ DEVICE_NAME) - 1)) };

BT_DATA_NAME_COMPLETE is a macro constant for the type of advertising data structure for a device name (you can find its value in bluetooth/gap.h). Then, the only thing you have to change is add the sd array and its size to the calls to bt_le_adv_start() and bt_le_adv_update_data(). For instance: // Start advertising sensor values update_ad_bme280(bme280); err = bt_le_adv_start(ADV_PARAM, ad, ARRAY_SIZE(ad), sd, ARRAY_SIZE(sd));

● 94

Boek BLE 220329 UK.indd 94

06/05/2022 15:54

Chapter 3 • Broadcasting data with advertisements

And: update_ad_bme280(bme280); err = bt_le_adv_update_data(ad, ARRAY_SIZE(ad), sd, ARRAY_SIZE(sd));

Finally, add the variable CONFIG_BT_DEVICE_NAME to your prj.conf, with the device name as its value: CONFIG_BT_DEVICE_NAME="BME280"

If you build your code with these changes and flash the firmware, have a look in Wireshark at how your device responds to an active scan. You can now also add the device name in your Python script to decode the packets. Just add this line in the callback function: print(f"Device

: {device.name}")

Bleak assigns the device name from the scan response data to the BLEDevice object’s name attribute.

3.12 Summary and further exploration In this chapter, I introduced advertising as the most basic, but already powerful, way to communicate with BLE devices. You learned about the different device roles relevant to advertising, about the structure and types of advertising packets, about the difference between active and passive scanning and about the various types of BLE addresses. After this theoretical foundation, the bulk of this chapter was filled with code to broadcast and receive advertising data. You learned how to discover and decode advertising data with Python and Bleak on your PC, and with NimBLE-Arduino on an ESP32 board. You also learned how to broadcast data with Zephyr on an nRF52840 board. You’ve learned a lot about iBeacon advertisements as an example of manufacturer-specific data. But one of the goals of iBeacons is that you can determine your approximate distance to them. How to calculate your distance based on the beacon’s RSSI and measured power at one meter is a fascinating topic on its own. You could do some experiments to come up with your own formula or read up on the many resources on this topic. For an interesting view on Bluetooth distance estimation, read David G. Young’s blog post, "How Far Can You Go?" (http://www.davidgyoungtech.com/2020/05/15/how-far-can-you-go). David G. Young is the main developer of the Android Beacon Library (https://altbeacon. github.io/android-beacon-library/), a project of AltBeacon (https://altbeacon.org), an alternative proximity beacon specification. If you’re interested in an alternative to iBeacon, you could try to implement AltBeacon in your own applications. Another alternative specification is Eddystone (https://github.com/google/eddystone). Google has discontinued Ed-

● 95

Boek BLE 220329 UK.indd 95

06/05/2022 15:54

Bluetooth Low Energy Applications

dystone support in its products, but the specification is open and you can still implement it in your own devices without having to pay for a license. And, if you’re searching for an easy way to scan for various types of BLE beacons, have a look at the beacontools Python project (https://github.com/citruz/beacontools). Also interesting to know is that iBeacon is just one of many types of manufacturer-specific data defined by Apple. It’s all part of the Apple Continuity Protocol. This has been reverse engineered by researchers and their findings are published at https://github.com/furiousMAC/continuity. An iBeacon advertising packet always has 0x0215 after Apple’s company ID and then 21 bytes (0x15 in hexadecimal). However, other Apple device advertising packets can have multiple structures in the manufacturer-specific data, each with its own data type and length. 30 If you want to decode the BLE advertisements of various types of devices, have a look at the Theengs Decoder project (https://theengs.github.io/decoder/). It’s an efficient, portable, and lightweight C library that’s able to decode the BLE advertisements of more than 30 devices. The project has an example Arduino sketch based on NimBLE-Arduino and an example Python script based on Bleak. If you’ve read this chapter, you should be able to integrate Theengs Decoder into your own Arduino or Python projects. It’s also quite easy to add support for other devices: you need just add a decoder specification as a JSON string in a C header file. In this way, I’ve added support for the RAWv1 and RAWv2 data formats of the RuuviTag to Theengs Decoder. As a final note, I want to stress that the examples in this book all use so-called legacy advertisements, which are supported by all Bluetooth Low Energy versions. BLE 5.0 has extended advertisements added. This allows you to broadcast more than 31 bytes of data. After a whole chapter about advertisements, it’s time to look at the other way of communicating with BLE devices: connections. That’s is the topic of the next chapter.

30 «This type of format is called TLV (type-length-value), in contrast to LTV (length-type-value), which is used to put multiple advertising data structures in an advertising packet.

● 96

Boek BLE 220329 UK.indd 96

06/05/2022 15:54

Chapter 4 • Connections and services

Chapter 4 • Connections and services While the previous chapter focused on BLE broadcasters and observers, this chapter talks about peripherals and centrals and how they communicate using connections. In this chapter you’ll learn about: • the GAP and GATT roles relevant for BLE connections • the three types of attributes according to the GATT specification: services, characteristics, and descriptors • how to discover services and characteristics with Python and Bleak • how to read and write characteristics with Python and Bleak • how to receive notifications and indications with Python and Bleak, as well as with NimBLE-Arduino • how to create a GATT server with Zephyr and send notifications or indications • how to sniff packets from an unencrypted BLE connection • how to read service data without a connection

4.1 Device roles Let’s reiterate the two roles that are important for BLE connections, as defined by the Generic Access Profile (GAP): Peripheral Sends connectable advertising packets, so other devices know they’re able to connect. Once connected, the peripheral device is also known as a slave in the link layer. Central Initiates a connection to a peripheral. Once connected, the central device is also known as a master in the link layer.

Figure 4.1 One central (your phone or computer) can connect to multiple peripherals. Broadcasting is a one-to-many method of communication that works in one direction: from the broadcaster to the observers. In contrast, connections are a one-to-one method of communication that works in both directions. The central always initiates a connection to a peripheral. However, after the connection, both devices can send data to each other on their own initiative. Most peripherals can only have a connection with one central at a time. That’s because they stop advertising when they’re in a connection, so other centrals can’t find the peripheral. However, since Bluetooth 4.1, connections with multiple centrals are possible. This is im-

● 97

Boek BLE 220329 UK.indd 97

06/05/2022 15:54

Bluetooth Low Energy Applications

plemented by being both in an advertising state and on a connection. 31 For simplicity, the example programs for peripherals in this book are limited to a connection with one central. As soon as you’re dealing with connections, you not only need GAP roles, but also GATT (Generic Attribute Profile) roles. GATT defines these two roles: Client Sends requests to a server and receives its responses Server Receives requests from a client and returns responses GAP and GATT roles are completely independent from each other. For instance, if you connect your phone to your smartwatch, the smartwatch is the peripheral and the phone is the central. Both GAP roles stay the same during the connection. However, the GATT roles will change depending on which device is requesting data, and this can change during the connection: • When your phone requests your heart rate from your smartwatch, the phone is the GATT client and the smartwatch is the GATT server. • When your smartwatch requests the current time from your phone, the smartwatch is the GATT client and the phone the GATT server.

4.2 Attributes If you look at it conceptually, you can consider a GATT server as a collection of data. In BLE, this data is structured following the Attribute Protocol (ATT). Each piece of data is described by an attribute, which has: A handle A 16bit address that’s unique for this attribute. A handle doesn’t change during a connection. If you’re bonded to the server (see chapter 5), the handle doesn’t even change across connections. A type A 16bit, 32bit or 128bit UUID. Normally, a UUID is 128 bits long, but the Bluetooth specification has defined short forms to preserve space in BLE advertisements. 32bit UUID XXXXXXXX is the shorter version of 128bit UUID XXXXXXXX-0000-1000-8000-00805f9b34fb. 16bit UUID XXXX is a shorter version of 0000XXXX-0000-1000-8000-00805f9b34fb. For example, 16bit UUID 0x2a06 is the short form of 00002a06-0000-1000-800000805f9b34fb. 32

31 32

See https://www.bluetooth.com/blog/how-one-wearable-can-connect-with-multiple-smartphones-or-tablets-simultaneously/ for a thorough explanation of this scenario. You can find the list of 16bit UUIDs in the list of Assigned Numbers on the Bluetooth SIG web site: https://www.bluetooth.com/specifications/assigned-numbers/

● 98

Boek BLE 220329 UK.indd 98

06/05/2022 15:54

Chapter 4 • Connections and services

Permissions Flags that describe whether you can read and/or write the attribute, which level of security is required, and whether you need authorization to access the attribute. These security aspects will be explained in Chapter 5. A value The actual value of the attribute, with a 512-byte maximum length. The raw bytes of the attribute’s value should be interpreted depending on the attribute’s type. To be more precise, you can look at a GATT server as a database or a table of attributes, with handles in increasing order. Note: BLE data is always part of a GATT server. A GATT client doesn’t hold any data.

4.3 Services, characteristics, and descriptors According to the GATT specification, all attributes come in one of three types: services, characteristics, and descriptors. These have the following hierarchy:

Figure 4.2 A GATT server consists of services, which consist of characteristics, which can have descriptors. So, each GATT server has one or more services, each service groups one or more related characteristics, and each characteristic can have zero or more descriptors. I’ll explain each of these concepts now.

● 99

Boek BLE 220329 UK.indd 99

06/05/2022 15:54

Bluetooth Low Energy Applications

4.3.1 Services A service groups one or more related characteristics. The attribute that declares the service, called the Service Declaration attribute, has a UUID as its value. This is the (short) UUID that’s assigned by the Bluetooth SIG to that specific service in the Assigned Numbers document, or a vendor-specific UUID. Note: A service declaration has no security measures because GATT clients need to be able to read it to discover the GATT server’s services. It’s a read-only attribute. After a Service Declaration attribute in the GATT server’s attributes database come the attributes describing the characteristics of this service. The full list of attributes that make up the service (including the service’s characteristics’ attributes and their descriptors) is called the service definition. Note: With another type of attribute, an Include Declaration, you can include other services inside a service definition to avoid duplicating data. This included service is called a secondary service, while the standard type of service described above is known as a primary service. In this book, I’m using only primary services. For some services, the GATT server can have multiple instances of the same service. Have a look at the service’s specification to determine whether it allows this or if it only allows one instance.

4.3.2 Characteristics While services are meant just to group data, the real data your GATT server exposes is in its characteristics. A characteristic always has at least two attributes: Characteristic Declaration Has bit fields with properties describing the operations that the GATT client can perform on the characteristic, the handle of the attribute that contains the characteristic value, and the characteristic UUID. This is the (short) UUID in the Assigned Numbers document that’s assigned to that specific characteristic by the Bluetooth SIG, or a vendor-specific UUID. Characteristic Value Contains the actual value of the data as its value. Its type is the characteristic UUID. How you interpret the value depends on the characteristic. For characteristics specified by the Bluetooth SIG, this is defined in the GATT Specification Supplement document. Note: A characteristic declaration has no security measures, because GATT clients need to be able to read it to discover characteristics of the GATT server. It’s a read-only attribute. The Characteristic Definition consists of the attributes that make up this characteristic: the characteristic declaration, the characteristic value, and the characteristic’s descriptors. Note: You can’t have ‘free’ characteristics: they’re always part of a service.

● 100

Boek BLE 220329 UK.indd 100

06/05/2022 15:54

Chapter 4 • Connections and services

4.3.3 Descriptors A descriptor (the full name is actually "characteristic descriptor") offers more metadata about a characteristic and its value. It’s optional, but if it exists, it does this in just one attribute, a characteristic descriptor declaration. The GATT specification defines some standard types for descriptors: Client Characteristic Configuration Descriptor (CCCD) Allows you to enable and disable notifications and indications, two server-initiated forms of communication. You’ll learn more about this later in this chapter. Characteristic Presentation Format Descriptor Characteristic value’s format, such as a string, an integer or floating-point number, or a Boolean value Characteristic User Description Descriptor A human-readable name for the characteristic Extended Properties Descriptor Extra bit fields with properties that don’t fit in the characteristic declaration You can also have vendor-defined descriptors. Note: A characteristic descriptor declaration has no security because GATT clients need to be able to read it to discover the characteristic’s metadata. It’s a read-only attribute.

4.4 Discovering services and characteristics with nRF Connect If you want to discover a BLE device’s services and characteristics, the nRF Connect mobile app is a useful tool to start with. Just tap on Scan, then one of the devices found, and look at the list of service UUIDs that the device advertises:

● 101

Boek BLE 220329 UK.indd 101

06/05/2022 15:54

Bluetooth Low Energy Applications

Figure 4.3 You can find a peripheral’s advertised service UUIDs with the nRF Connect mobile app. If you tap on the Connect button next to the device, the app initiates a connection to the GATT server and shows the various services it offers. If you tap on a service, a list of characteristics unfolds. For each characteristic, the app shows the name, UUID, properties, and descriptors.

Figure 4.4 Once connected to a peripheral, you can search through all the services and characteristics in nRF Connect and read their data.

● 102

Boek BLE 220329 UK.indd 102

06/05/2022 15:54

Chapter 4 • Connections and services

If you tap on the downward-facing arrow next to a characteristic, the app reads its value and shows it after the characteristic’s properties. You can do the same with the Bluetooth Low Energy app in the desktop version of nRF Connect. When you click on Start scan, click on Details below one of the devices, and look at the list of services the device advertises:

Figure 4.5 With nRF Connect’s Bluetooth Low Energy app, you can discover devices and their services from your computer. If you click on the Connect button next to the device, the app initiates a connection to the GATT server and shows you the various services it offers. Click on a service to get a list of its characteristics and their values. To write a value to a characteristic that has write in its properties, change the characteristic’s value and click on the check mark button next to it.

● 103

Boek BLE 220329 UK.indd 103

06/05/2022 15:54

Bluetooth Low Energy Applications

Figure 4.6 Once connected to a peripheral in nRF Connect’s Bluetooth Low Energy app, you can read and write its characteristics. Warning: Don’t forget to disconnect from the device after you’re done. Most BLE peripherals can’t connect to multiple centrals at the same time. If you want to see more details about a BLE connection, use Wireshark. If you’re doing a packet capture in Wireshark at the moment you’re initiating a connection, you can see which BLE packets are used under the hood, beginning with a CONNECT_REQ packet. Note: In this chapter, I’m not going to show detailed packet captures with Wireshark. I’m also not going to explain the specific sequence of packets sent in a connection, as I did in the previous chapter for advertisements. A connection’s packet sequence is much more complex than the basic advertisements from the previous chapter. You’ll be using the abstractions offered by the Bleak, NimBLE-Arduino, and Zephyr APIs, which hide all this complexity.

● 104

Boek BLE 220329 UK.indd 104

06/05/2022 15:54

Chapter 4 • Connections and services

4.5 A minimal GATT server At its minimum, a GATT server has two services: a Generic Access Profile service and a Generic Attribute Profile service. For the former, the Device Name and Appearance characteristics are mandatory, while for the latter its only characteristic, Service Changed, is optional. This means that a GATT server’s smallest possible attribute database looks like this (with the device name and appearance values in this example taken from an existing device): Handle

Type

Permissions

Value

0x0001

0x2800 (Primary Service)

read

0x1800 (Generic Access)

0x0002

0x2803 (Characteristic)

read

0x2a00 (Device Name)

0x0003

0x2a00 (Device Name)

read (option-

"Gigaset keeper"

ally write) 0x0004

0x2803 (Characteristic)

read

0x2a01 (Appearance)

0x0005

0x2a01 (Appearance)

read (option-

0

ally write) 0x0006

0x2800 (Primary Service)

read

0x1801 (Generic Attribute)

Table 4.1 A minimal GATT server In practice, you’ll see a lot more attributes in BLE devices, but these are the ones you can be sure about. Note: The exact UUIDs for the types and values can be found in the 16bit UUIDs document on the Bluetooth SIG’s Assigned Numbers page.

4.6 Discovering services and characteristics with Bleak The Bleak library allows you to connect to a BLE peripheral in your own Python programs. Instead of creating a BleakScanner object as we did in the previous chapter, this time, you create a BleakClient object and connect to it. A program to connect to a device and show all its services, characteristics and descriptors, and their values looks like this: """Service explorer for BLE devices. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio import sys

● 105

Boek BLE 220329 UK.indd 105

06/05/2022 15:54

Bluetooth Low Energy Applications

from bleak import BleakClient

async def main(address): """Connect to a BLE device and show its services.""" async with BleakClient(address) as client: for service in client.services: print(f"- Description: {service.description}") print(f"

UUID: {service.uuid}")

print(f"

Handle: {service.handle}")

for char in service.characteristics: value = None if "read" in char.properties: try: value = bytes( await client.read_gatt_char(char) ) except Exception as error: value = error - Description: {char.description}")

print(f" print(f"

UUID: {char.uuid}")

print(f"

Handle: {char.handle}")

print(f"

Properties: {‘, ‘.join(char.properties)}")

print(f"

Value: {value}")

for descriptor in char.descriptors: desc_value = None try: desc_value = bytes( await client.read_gatt_descriptor( descriptor ) ) except Exception as error: desc_value = error print( f"

- Description: {descriptor.description}"

) print(f"

UUID: {descriptor.uuid}")

print(f"

Handle: {descriptor.handle}")

print(f"

Value: {desc_value}")

if __name__ == "__main__":

● 106

Boek BLE 220329 UK.indd 106

06/05/2022 15:54

Chapter 4 • Connections and services

if len(sys.argv) == 2: address = sys.argv[1] asyncio.run(main(address)) else: print("Please specify the Bluetooth address.")

You run it like this with the Bluetooth address of the device you want to explore as an argument: $ python3 service_explorer.py 7C:2F:80:E3:A1:A7 - Description: Device Information UUID: 0000180a-0000-1000-8000-00805f9b34fb Handle: 16 - Description: PnP ID UUID: 00002a50-0000-1000-8000-00805f9b34fb Handle: 27 Properties: read Value: b’\x01\x80\x01\x02\x00\x00\x01’ - Description: System ID UUID: 00002a23-0000-1000-8000-00805f9b34fb Handle: 25 Properties: read Value: b’\x124V\xff\xfe\x9a\xbc\xde’ - Description: Software Revision String UUID: 00002a28-0000-1000-8000-00805f9b34fb Handle: 23 Properties: read Value: b’v_3.20.8.35’ - Description: Firmware Revision String UUID: 00002a26-0000-1000-8000-00805f9b34fb Handle: 21 Properties: read Value: b’v_3.0.11.549’ - Description: Model Number String UUID: 00002a24-0000-1000-8000-00805f9b34fb Handle: 19 Properties: read Value: b’DA1458xB101-6

‘

- Description: Manufacturer Name String UUID: 00002a29-0000-1000-8000-00805f9b34fb Handle: 17 Properties: read Value: b’Gigaset 12122017’ - Description: Generic Attribute Profile UUID: 00001801-0000-1000-8000-00805f9b34fb

● 107

Boek BLE 220329 UK.indd 107

06/05/2022 15:54

Bluetooth Low Energy Applications

Handle: 12 - Description: Service Changed UUID: 00002a05-0000-1000-8000-00805f9b34fb Handle: 13 Properties: read, indicate Value: b’\x01\x00\xff\xff’ - Description: Client Characteristic Configuration UUID: 00002902-0000-1000-8000-00805f9b34fb Handle: 15 Value: b’\x02\x00’

I’ve trimmed the output because it had a lot more than just those two services. But you already see the structure with the services, characteristics, and descriptors. The Device Information service has a lot of read-only characteristics, including the manufacturer name and software revision. The Generic Attribute Profile service has one characteristic, Service Changed, which has one descriptor, Client Characteristic Configuration. Note: On Linux, the Generic Access Profile service and its characteristics aren’t shown because BlueZ hides those. See https://github.com/hbldh/bleak/issues/553 if you want to read the Device Name characteristic directly by its UUID. So, how does the code work? Instead of explicitly connecting and disconnecting the BleakClient object, the main() function starts a context manager with the line async with BleakClient(address) as client:. Inside this block, you’re sure the client object is connected to the peripheral, and, after the block is finished, it automatically disconnects from the peripheral. The outer for loop iterates over all discovered services and shows their information. For each service, a second for loop iterates over all characteristics, tries to read their value (with client.read_gatt_char(char)) and then shows their information. And, ultimately, the most inner loop does the same for all descriptors in this characteristic. Note: Read Bleak’s API documentation to see what else you can do with the BleakGATTService, BleakGATTCharacteristic and BleakGATTDescriptor classes used in this program. The program starts by checking the number of command-line arguments. If it’s two (the command name and one argument), it starts the event loop and runs the main() function with the device address as its argument.

4.7 Reading and writing characteristics using Bleak For most applications you’re not interested in a peripheral’s full list of attributes as shown in the previous section. You’re specifically targeting a small number of characteristics to read or write, depending on your application’s purpose.

● 108

Boek BLE 220329 UK.indd 108

06/05/2022 15:54

Chapter 4 • Connections and services

4.7.1 Reading characteristics The Python program in the previous section already showed you how to read all characteristics, but now I’ll show you how to read a specific characteristic directly by its UUID. For instance, here’s how you write a Python program with Bleak to show some information read from the Device Information service’s characteristics, implemented by a lot of BLE peripherals: """Show device information for a BLE device. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio import sys import bleak DEVICE_NAME_UUID = "00002a00-0000-1000-8000-00805f9b34fb" MODEL_NUMER_STRING_UUID = "00002a24-0000-1000-8000-00805f9b34fb" MANUFACTURER_NAME_STRING_UUID = "00002a29-0000-1000-8000-00805f9b34fb"

async def main(address): """Connect to device and show some information.""" try: async with bleak.BleakClient(address) as client: try: device_name = await client.read_gatt_char( DEVICE_NAME_UUID ) print(f"Device name : {device_name.decode()}") except bleak.exc.BleakError: pass try: model_number = await client.read_gatt_char( MODEL_NUMER_STRING_UUID ) print(f"Model number: {model_number.decode()}") except bleak.exc.BleakError: pass try:

● 109

Boek BLE 220329 UK.indd 109

06/05/2022 15:54

Bluetooth Low Energy Applications

manufacturer_name = await client.read_gatt_char( MANUFACTURER_NAME_STRING_UUID ) print(f"Manufacturer: {manufacturer_name.decode()}") except bleak.exc.BleakError: pass except asyncio.exceptions.TimeoutError: print(f"Can’t connect to device {address}.")

if __name__ == "__main__": if len(sys.argv) == 2: address = sys.argv[1] asyncio.run(main(address)) else: print("Please specify the Bluetooth address.")

A good, structured approach for BLE programming always starts with defining constants for your characteristics’ UUIDs. In this example, I’m using the Generic Access service’s device name characteristic, and the Device Information service’s model number string and manufacturer name string characteristics. Then the main() function starts a connection. I wrapped it in a try / except block to show a human-readable warning if the connection doesn’t succeed. In that case, there’s probably no GATT server running. If the connection succeeds, you can read the three aforementioned characteristics. Note: In the service explorer example from the previous section, the code read the characteristic using the BleakGATTCharacteristic object received from the service’s list of characteristics. In this new example, the characteristics are directly read by their 128bit UUID. The read_gatt_char() method also accepts an integer handle or a uuid. UUID object as an argument. It’s important to know whether the characteristics you’re reading are mandatory or optional, so that your program can handle all scenarios. Your first thought is probably to check a characteristics’ existence before reading it. But a more ‘Pythonic’ way is to just read it and make sure to have an exception handler catch the error if reading doesn’t succeed. To be sure, I added an exception handler for all reads, even for the mandatory device name characteristic. In this case I just ignore it when a characteristic doesn’t exist (pass does nothing), but you can handle the situation differently, for instance by showing an error message.

● 110

Boek BLE 220329 UK.indd 110

06/05/2022 15:54

Chapter 4 • Connections and services

Take into account that Bleak always returns the characteristic value as a bytearray object. To show it as a string, you need to call the decode() method on this object. Note: If the characteristic value has a different type than a string, you need to decode the byte array differently. You’ll see some examples in the rest of the book. For complex, structured types you can use Construct, as you did in the previous chapter to decode advertising data structures. If you run this example on one of your BLE peripherals, it should show something like this: $ python3 device_information.py E7:2E:00:B1:38:96 Device name : LYWSD02 Model number: LYWSD02 Manufacturer: miaomiaoce.com

4.7.2 Reading characteristics by their handle In the previous example, you read some characteristics by their UUIDs. But, what if a BLE device has more than one characteristic with the same UUID? This isn’t so far-fetched. For instance, a proximity reporter (see Chapter 6) can have an Alert Level characteristic (UUID 0x2a06) in both the Link Loss service and the Immediate Alert service. What if you tried to read the Alert Level characteristic with the following code? """Read the Alert Level characteristic of a BLE device by its UUID. Note: This doesn’t work if the device has multiple Alert Level characteristics. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio import sys import bleak ALERT_LEVEL_UUID = "00002a06-0000-1000-8000-00805f9b34fb"

async def main(address): """Connect to device and read its Alert Level characteristic.""" try: async with bleak.BleakClient(address) as client: alert_level = await client.read_gatt_char( ALERT_LEVEL_UUID

● 111

Boek BLE 220329 UK.indd 111

06/05/2022 15:54

Bluetooth Low Energy Applications

) print(f"Alert level: {alert_level.decode()}") except asyncio.exceptions.TimeoutError: print(f"Can’t connect to device {address}.")

if __name__ == "__main__": if len(sys.argv) == 2: address = sys.argv[1] asyncio.run(main(address)) else: print("Please specify the Bluetooth address.")

If you run this Python script with the Bluetooth address of a proximity reporter that implements both the Link Loss service and the Immediate Alert service, you’ll get the following exception: $ python3 alert_level_by_uuid.py 7C:2F:80:E3:A1:A7 Traceback (most recent call last): File "alert_level_by_uuid.py", line 26, in asyncio.run(main(address)) File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_ complete return future.result() File "alert_level_by_uuid.py", line 13, in main alert_level = await client.read_gatt_char(ALERT_LEVEL_UUID) File "/home/koan/.local/lib/python3.8/site-packages/bleak/backends/bluezdbus/ client.py", line 677, in read_gatt_char characteristic = self.services.get_characteristic(char_specifier) File "/home/koan/.local/lib/python3.8/site-packages/bleak/backends/service.py", line 179, in get_characteristic raise BleakError( bleak.exc.BleakError: Multiple Characteristics with this UUID, refer to your desired characteristic by the `handle` attribute instead.

As the error message clearly says, Bleak discovers multiple characteristics with the UUID you’re trying to read. It also suggests that you refer to the characteristic by its handle. So, you first have to decide which of the Alert Level characteristics you want to read: the Link Loss service or the Immediate Alert service. Then, you get that characteristic from that service and, finally, read the characteristic. If you’re interested in the Link Loss service, the code looks like this:

● 112

Boek BLE 220329 UK.indd 112

06/05/2022 15:54

Chapter 4 • Connections and services

"""Read the Alert Level characteristic of a BLE device by its characteristic object. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio import sys import bleak LINK_LOSS_UUID = "00001803-0000-1000-8000-00805f9b34fb" ALERT_LEVEL_UUID = "00002a06-0000-1000-8000-00805f9b34fb"

async def main(address): """Connect to device and read its Alert Level characteristic.""" try: async with bleak.BleakClient(address) as client: link_loss_service = client.services.get_service( LINK_LOSS_UUID ) alert_level_characteristic = ( link_loss_service.get_characteristic(ALERT_LEVEL_UUID) ) alert_level = await client.read_gatt_char( alert_level_characteristic ) print(f"Alert level: {int(alert_level[0])}") except asyncio.exceptions.TimeoutError: print(f"Can’t connect to device {address}.")

if __name__ == "__main__": if len(sys.argv) == 2: address = sys.argv[1] asyncio.run(main(address)) else: print("Please specify the Bluetooth address.")

If you run this code with the same proximity reporter’s Bluetooth address, you’ll get the correct characteristic value:

● 113

Boek BLE 220329 UK.indd 113

06/05/2022 15:54

Bluetooth Low Energy Applications

$ python3 alert_level_by_char.py 7C:2F:80:E3:A1:A7 Alert level: 1

Note: In this code, you don’t refer to the characteristic by its handle, as suggested in the exception, but by the characteristic found in the associated service. Most developers will find this to be a more natural approach.

4.7.3 Writing characteristics With Bleak, writing characteristics is just as easy as reading them. You’ll see more complex examples later in this book, but let’s just use a simple example here to show you the ropes. This example uses a popular plant sensor made by Xiaomi, the Mi Flora. It measures soil moisture and conductivity, as well as ambient light and temperature. It uses vendor-specific GATT services, which have already been reverse engineered. A lot of libraries in multiple programming languages exist to read the Xiaomi Mi Flora’s sensor measurements. For this subsection, I’ll show you how to write a command to the device’s "device mode change" characteristic to blink the LED on the device’s top. 33 The Python program to do this looks like this: """Blink the LED of the Xiaomi Mi Flora plant sensor. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio import sys import bleak DEVICE_MODE_CHANGE_UUID = "00001a00-0000-1000-8000-00805f9b34fb" BLINK_COMMAND = bytes([0xFD, 0xFF])

async def main(address): """Connect to Mi Flora and blink the LED.""" try: async with bleak.BleakClient(address) as client:

33

The meaning of all the Xiaomi Mi Flora characteristics is explained in https://github.com/vrachieru/ xiaomi-flower-care-api. In chapter 7, you’ll learn how to reverse engineer BLE devices such as this one.

● 114

Boek BLE 220329 UK.indd 114

06/05/2022 15:54

Chapter 4 • Connections and services

try: await client.write_gatt_char( DEVICE_MODE_CHANGE_UUID, BLINK_COMMAND ) except bleak.exc.BleakError: print(f"Can’t blink device {address}") except asyncio.exceptions.TimeoutError: print(f"Can’t connect to device {address}.")

if __name__ == "__main__": if len(sys.argv) == 2: address = sys.argv[1] asyncio.run(main(address)) else: print("Please specify the Bluetooth address.")

The blink command is just the two bytes 0xfdff, so the program defines a constant with those bytes, BLINK_COMMAND. Then, as soon as you have a connection, you write to the client with the write_gatt_char() method. The first argument is the characteristic (BleakGATTCharacteristic object, integer handle, string of the UUID, or uuid.UUID object) and the second is the data to write, as a bytes or bytearray object. If you run this Python script with the Bluetooth address of a Xiaomi Mi Flora device, you’ll see the LED at the top light up for a second before it turns off again.

4.8 Notifications and indications You now know how to read characteristics from a BLE device with Bleak. But, if you want to read a changing value, such as a heart rate or a battery level, continuously, reading the value repeatedly in a loop isn’t really energy-efficient. The value might not change between two read requests. Schematically, this sequence looks like this:

● 115

Boek BLE 220329 UK.indd 115

06/05/2022 15:54

Bluetooth Low Energy Applications

Figure 4.7 The central can read the peripheral’s heart rate continuously, but this isn’t energy-efficient. BLE has better methods for this purpose: notifications and indications. After the client has ‘subscribed’ to a characteristic’s notifications, the server sends a characteristic value to the client each time it changes. This sequence looks like this:

Figure 4.8 The central subscribes to peripheral’s heart rate notifications. Indications work similarly, but they require a confirmation: after having received an indication, the client sends a confirmation to the server that it has received the value. The server does not send another indication before having received this confirmation:

● 116

Boek BLE 220329 UK.indd 116

06/05/2022 15:54

Chapter 4 • Connections and services

Figure 4.9 The central subscribes to the peripheral’s heart rate indications and sends a confirmation for each received value.

4.8.1 Read heart rate notifications As an example, let’s see how you can continuously read a BLE fitness tracker’s heart rate with notifications: """Subscribe to heart rate notifications of a fitness tracker. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio import sys import bleak HEART_RATE_MEASUREMENT_UUID = "00002a37-0000-1000-8000-00805f9b34fb"

def heart_rate_changed(handle: int, data: bytearray): """Show heart rate.""" print(f"Heart rate: {data[1]}")

async def main(address): """Subscribe to heart rate notifications.""" try: async with bleak.BleakClient(address) as client: print(f"Connected to {address}") await client.start_notify(

● 117

Boek BLE 220329 UK.indd 117

06/05/2022 15:54

Bluetooth Low Energy Applications

HEART_RATE_MEASUREMENT_UUID, heart_rate_changed ) print("Notifications started...") while True: await asyncio.sleep(1) except asyncio.exceptions.TimeoutError: print(f"Can’t connect to device {address}.")

if __name__ == "__main__": if len(sys.argv) == 2: address = sys.argv[1] asyncio.run(main(address)) else: print("Please specify the Bluetooth address.")

The main() function connects with the device and then calls start_notify() on the client. The first argument is the Heart Rate Measurement characteristic’s UUID, and the second is a callback function to be called on each notification message. Then, the function sleeps continuously. The callback function, heart_rate_changed(), has two arguments: the handle of the characteristic and its value as a bytearray object. The Heart Rate Measurement characteristic has a special structure, which you can find on page 139 of GATT Specification Supplement 5. The heart rate measurement value can be a 16bit or an 8bit integer, depending on a flag in the first byte of the byte array. In this case, I just assume it’s an 8bit integer (as it is in most cases), so the code just shows the integer value of the second byte. If you run this Python script and supply the Bluetooth address of a fitness tracker while it’s measuring your heart rate, you should see something like this: $ python3 heart_rate.py F3:BE:3E:97:17:A4 Connected to F3:BE:3E:97:17:A4 Start notifications... Heart rate: 58 Heart rate: 62 Heart rate: 61

Exit the program by pressing Ctrl+C.

● 118

Boek BLE 220329 UK.indd 118

06/05/2022 15:54

Chapter 4 • Connections and services

You can write similar programs for any characteristic that has the notify or indicate property. 34 Just check first with nRF Connect (see section 4.4) or the service explorer Python script (see section 4.6) whether the characteristic supports notifications or indications. If "notify" is a mandatory property of the characteristic, as is the case for the Heart Rate Measurement characteristic, it’s fine to just call start_notify() for the characteristic. But, if the property is optional, you should check first whether "notify" or "indicate" are in the properties of a characteristic before you call start_notify()for this characteristic. 35 Note: If you try to adapt this example for notifications on the Battery Level characteristic (0x2a19) in Linux, you won’t succeed. This is impossible with BlueZ 5.48 or higher. You need to use a regular read request instead in order to read the battery level.

4.8.2 Read notifications from multiple devices It’s also possible to connect to multiple devices in the same Python program and get notifications from all of them. A simple implementation to get heart rate measurements from multiple fitness trackers would look like this: """Subscribe to heart rate notifications of multiple fitness trackers. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio import sys from bleak import BleakClient, BleakScanner DEVICE_NAME_UUID = "00002a00-0000-1000-8000-00805f9b34fb" HEART_RATE_MEASUREMENT_UUID = "00002a37-0000-1000-8000-00805f9b34fb" addresses = [] heart_rate_sensors = []

def device_found(device, _): """Add found device to list of devices.""" if device.address in addresses: heart_rate_sensors.append(device) print(f"Found device {device.name}")

34 35

If a characteristic has both properties, Bleak automatically chooses one of the two. You would do this with something like if "notify" in char.properties or "indicate" in char.properties.

● 119

Boek BLE 220329 UK.indd 119

06/05/2022 15:54

Bluetooth Low Energy Applications

def heart_rate_changed(handle: int, data: bytearray): """Show heart rate.""" print(f"Heart rate: {data[1]}")

async def connect(device): """Connect to fitness tracker and subscribe to heart rate notifications.""" try: async with BleakClient(device) as client: device_name = ( await client.read_gatt_char(DEVICE_NAME_UUID) ).decode() print(f"Connected to {device_name}") await client.start_notify( HEART_RATE_MEASUREMENT_UUID, heart_rate_changed ) print(f"Start notifications for {device_name}...") while True: await asyncio.sleep(1) except asyncio.exceptions.TimeoutError: print(f"Can’t connect to device {device.address}.")

async def main(): """Register detection callback, scan for devices and connect to them.""" scanner = BleakScanner() scanner.register_detection_callback(device_found) await scanner.start() await asyncio.sleep(5.0) await scanner.stop() await asyncio.gather( *(connect(device) for device in heart_rate_sensors) )

if __name__ == "__main__": if len(sys.argv) >= 2: addresses = sys.argv[1:] asyncio.run(main()) else: print("Please specify at least one Bluetooth address.")

● 120

Boek BLE 220329 UK.indd 120

06/05/2022 15:54

Chapter 4 • Connections and services

You supply multiple Bluetooth addresses on the command line, and the program puts these in the addresses list. Then it runs the main() function. This first creates a BleakScanner object, registers a detection callback that’s called when a device is found, and starts scanning for five seconds. The detection callback, device_found(), adds the found device when its address is in the list of supplied addresses. So, after the scan is done, heart_rate_sensors contains the devices. Then the main() function starts a task for each heart rate sensor found and calls its connect() function. This starts notifications with the heart_rate_changed() callback function, as you saw in the previous subsection. However, if you run this program on two or more fitness trackers, you’ll see a problem: you can’t differentiate the notifications from different devices. The output looks like this: $ python3 heart_rate_multiple.py F3:BE:3E:97:17:A4 EB:76:55:B9:56:18 Found device F15 Found device InfiniTime Connected to InfiniTime Start notifications for InfiniTime... Heart rate: 58 Connected to F15 Start notifications for F15... Heart rate: 75 Heart rate: 75 Heart rate: 74 Heart rate: 74 Heart rate: 73 Heart rate: 73 Heart rate: 59 Heart rate: 73

The notification callback function is called with only the characteristic handle and data as its arguments. Luckily there’s a trick, explained by Bleak creator Henrik Blidh in a GitHub discussion of the project. 36 The code, which is just a slight change from the previous code, looks like this: """Subscribe to heart rate notifications of multiple fitness trackers and show the origin of the notifications. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ 36

https://github.com/hbldh/bleak/discussions/467

● 121

Boek BLE 220329 UK.indd 121

06/05/2022 15:54

Bluetooth Low Energy Applications

import asyncio import functools import sys from bleak import BleakClient, BleakScanner DEVICE_NAME_UUID = "00002a00-0000-1000-8000-00805f9b34fb" HEART_RATE_MEASUREMENT_UUID = "00002a37-0000-1000-8000-00805f9b34fb" addresses = [] heart_rate_sensors = []

def device_found(device, _): """Add found device to list of devices.""" if device.address in addresses: heart_rate_sensors.append(device) print(f"Found device {device.name}")

def heart_rate_changed( device_name: str, handle: int, data: bytearray ): """Show device name and heart rate.""" print(f"{device_name}: {data[1]} bpm")

async def connect(device): """Connect to fitness tracker and subscribe to heart rate notifications.""" try: async with BleakClient(device) as client: device_name = ( await client.read_gatt_char(DEVICE_NAME_UUID) ).decode() print(f"Connected to {device_name}") await client.start_notify( HEART_RATE_MEASUREMENT_UUID, functools.partial(heart_rate_changed, device_name), ) print(f"Start notifications for {device_name}...") while True: await asyncio.sleep(1) except asyncio.exceptions.TimeoutError: print(f"Can’t connect to device {device.address}.")

● 122

Boek BLE 220329 UK.indd 122

06/05/2022 15:54

Chapter 4 • Connections and services

async def main(): """Register detection callback, scan for devices and connect to them.""" scanner = BleakScanner() scanner.register_detection_callback(device_found) await scanner.start() await asyncio.sleep(5.0) await scanner.stop() await asyncio.gather( *(connect(device) for device in heart_rate_sensors) )

if __name__ == "__main__": if len(sys.argv) >= 2: addresses = sys.argv[1:] asyncio.run(main()) else: print("Please specify at least one Bluetooth address.")

The difference with the previous code is that it uses the functools.partial() function to inject the device name (which is just read from the Device Name characteristic) into the callback function. The callback function, heart_rate_changed(), is then called with the device name as the first argument and then the other two arguments from the original callback function. This allows you to print the device name next to the heart rate measurement. 37 If you run this Python script, you’ll see something like this: $ python3 heart_rate_multiple_origin.py F3:BE:3E:97:17:A4 EB:76:55:B9:56:18 Found device F15 Found device InfiniTime Connected to F15 Start notifications for F15... Connected to InfiniTime Start notifications for InfiniTime... F15: 71 bpm F15: 71 bpm InfiniTime: 46 bpm F15: 71 bpm F15: 70 bpm F15: 70 bpm 37

The original example from Henrik Blidh injects the client object into the callback function.

● 123

Boek BLE 220329 UK.indd 123

06/05/2022 15:54

Bluetooth Low Energy Applications

F15: 70 bpm F15: 70 bpm F15: 69 bpm F15: 69 bpm F15: 68 bpm InfiniTime: 49 bpm F15: 68 bpm F15: 67 bpm

As you see, the program connects to both devices and then shows the heart rate and the corresponding device name for each notification. 38

4.9 Creating a heart rate monitor with NimBLE-Arduino As an example of BLE notifications in NimBLE-Arduino, let’s create a heart rate monitor. This Arduino sketch for an ESP32 board connects to a heart rate sensor and prints its notifications in the serial output. The code looks like this: /* Monitor readings from a BLE heart rate sensor. * * Copyright (C) 2021 Koen Vervloesem ([email protected]) * * SPDX-License-Identifier: MIT * * Based on the NimBLE_Client example from H2zero. */ #include #define UUID_SERVICE "180d" #define UUID_CHARACTERISTIC "2a37" static NimBLEAdvertisedDevice *advDevice; static bool doConnect = false; static uint32_t scanTime = 0; // 0 = scan forever class ClientCallbacks : public NimBLEClientCallbacks { void onConnect(NimBLEClient *pClient) { Serial.println("Connected"); pClient->updateConnParams(120, 120, 0, 60); } 38

You can also see that the PineTime’s heart rate measurements (with the InfiniTime firmware) are unrealistically low for me, as I’m not an elite athlete with resting heart rate under 50 bpm. Also, the PineTime only sends a notification when the measured heart rate changes, while the F15 sends it every second, regardless of whether it’s changed.

● 124

Boek BLE 220329 UK.indd 124

06/05/2022 15:54

Chapter 4 • Connections and services

void onDisconnect(NimBLEClient *pClient) { Serial.print(pClient->getPeerAddress().toString().c_str()); Serial.println(" Disconnected - Starting scan"); NimBLEDevice::getScan()->start(scanTime, nullptr); } /* Called when the peripheral requests a change to the connection * parameters. * Return true to accept and apply them or false to reject and keep * the currently used parameters. Default will return true. */ bool onConnParamsUpdateRequest(NimBLEClient *pClient, const ble_gap_upd_params *params) { if (params->itvl_min < 24) { // 1.25ms units return false; } else if (params->itvl_max > 40) { // 1.25ms units return false; } else if (params->latency > 2) { // Intervals allowed to skip return false; } else if (params->supervision_timeout > 100) { // 10ms units return false; } return true; } }; /* Define a class to handle the callbacks when advertisements are * received. */ class AdvertisedDeviceCallbacks : public NimBLEAdvertisedDeviceCallbacks { void onResult(NimBLEAdvertisedDevice *advertisedDevice) { Serial.print("Advertised Device found: "); Serial.println(advertisedDevice->toString().c_str()); if (advertisedDevice->isAdvertisingService( NimBLEUUID(UUID_SERVICE))) { Serial.println("Found Our Service"); // Stop scan before connecting. NimBLEDevice::getScan()->stop(); // Save the device reference in a global for the client to use. advDevice = advertisedDevice; // Ready to connect now. doConnect = true; } }

● 125

Boek BLE 220329 UK.indd 125

06/05/2022 15:54

Bluetooth Low Energy Applications

}; /* Notification / Indication receiving handler callback */ void notifyCB(NimBLERemoteCharacteristic *pRemoteCharacteristic, uint8_t *pData, size_t length, bool isNotify) { if (length) { uint16_t heart_rate_measurement = pData[1]; if (pData[0] & 1) { heart_rate_measurement += (pData[2] getAddress()); if (pClient) { if (!pClient->connect(advDevice, false)) { Serial.println("Reconnect failed"); return false; } Serial.println("Reconnected client"); } else { /* We don’t already have a client that knows this device, * we will check for a client that is disconnected that we can * use. */ pClient = NimBLEDevice::getDisconnectedClient(); }

● 126

Boek BLE 220329 UK.indd 126

06/05/2022 15:54

Chapter 4 • Connections and services

} // No client to reuse? Create a new one. if (!pClient) { if (NimBLEDevice::getClientListSize() >= NIMBLE_MAX_CONNECTIONS) { Serial.println( "Max clients reached. No more connections possible"); return false; } pClient = NimBLEDevice::createClient(); Serial.println("New client created"); pClient->setClientCallbacks(&clientCB, false); pClient->setConnectionParams(40, 56, 0, 51); pClient->setConnectTimeout(5); if (!pClient->connect(advDevice)) { /* Created a client but failed to connect, don’t need to keep it * as it has no data. */ NimBLEDevice::deleteClient(pClient); Serial.println("Failed to connect, deleted client"); return false; } } if (!pClient->isConnected()) { if (!pClient->connect(advDevice)) { Serial.println("Failed to connect"); return false; } } Serial.print("Connected to: "); Serial.println(pClient->getPeerAddress().toString().c_str()); Serial.print("RSSI: "); Serial.println(pClient->getRssi()); /* Now we can read/write/subscribe the characteristics of the * services we are interested in. */ NimBLERemoteService *pSvc = nullptr; NimBLERemoteCharacteristic *pChr = nullptr;

● 127

Boek BLE 220329 UK.indd 127

06/05/2022 15:54

Bluetooth Low Energy Applications

pSvc = pClient->getService(UUID_SERVICE); if (pSvc) { // Make sure it’s not null. pChr = pSvc->getCharacteristic(UUID_CHARACTERISTIC); } if (pChr) { // Make sure it’s not null. if (pChr->canNotify()) { if (!pChr->subscribe(true, notifyCB)) { // Disconnect if subscribe failed. pClient->disconnect(); return false; } } else if (pChr->canIndicate()) { if (!pChr->subscribe(false, notifyCB)) { // Disconnect if subscribe failed. pClient->disconnect(); return false; } } } else { Serial.println( "Heart Rate Measurement characteristic not found."); } Serial.println("Done with this device!"); return true; } void setup() { Serial.begin(115200); Serial.println("Starting NimBLE Client"); NimBLEDevice::init(""); NimBLEScan *pScan = NimBLEDevice::getScan(); pScan->setAdvertisedDeviceCallbacks( new AdvertisedDeviceCallbacks()); pScan->setInterval(60); pScan->setWindow(30); pScan->setActiveScan(true); pScan->start(scanTime, nullptr); } void loop() { // Loop here until we find a device we want to connect to.

● 128

Boek BLE 220329 UK.indd 128

06/05/2022 15:54

Chapter 4 • Connections and services

while (!doConnect) { delay(1); } doConnect = false; // Found a device we want to connect to, do it now. if (connectToServer()) { Serial.println("Success, scanning for more..."); } else { Serial.println("Failed to connect, starting scan..."); } NimBLEDevice::getScan()->start(scanTime, nullptr); }

This is a lot of code, but it handles a lot of tasks concerned with a BLE connection, and it shows that NimBLE-Arduino offers you a lot of flexibility. In the beginning, two important definitions are UUID_SERVICE and UUID_CHARACTERISTIC, string representations of the short UUIDs for the Heart Rate service and the Heart Rate Measurement characteristic. For the main part, you should look at the end of the code, in the setup() and loop() functions. In setup(), you begin a serial connection, initialize NimBLEDevice, and get a pointer to a scan object. The latter is then used to set callbacks (when your program receives new advertisement data), set the scan interval and scan window, and use an active scan. 39 After all this initialization, the scan is started for scanTime. The code has initialized this variable to 0, which means that it scans indefinitely. After setup(), the loop() function is called, which continuously checks for the value of doConnect. As long as it’s false (the value it’s initialized to in the beginning of the code), the loop keeps running. Now, because the scan has started, the member function onResult() of the AdvertisedDeviceCallbacks class is called for every new scan result. This function checks whether the detected device advertises the service UUID for the Heart Rate service. If it does, it stops the scan, saves the pointer to the device in global variable advDevice and sets doConnect to true.

39

Both the scan interval and scan window are the recommended values listed on page 16 of the Heart Rate Profile specification. You can find this document on https://www.bluetooth.com/specifications/ specs/heart-rate-profile-1-0/.

● 129

Boek BLE 220329 UK.indd 129

06/05/2022 15:54

Bluetooth Low Energy Applications

So now, the while loop in the loop() function stops, doConnect is set to false again for later connections, and the code connects to the found device with connectToServer(). Whether this succeeds or not, a new scan starts after this. The function connectToServer() is this Arduino sketch’s largest function, primarily because it handles a lot of situations. First, it checks whether there’s already a NimBLEClient object for the device you’re going to connect to: this is returned by NimBLEDevice::getClientByPeerAddress(advDevice->getAddress()). If there is one, it connects to it with pClient->connect(advDevice, false). The advDevice variable is the pointer to the heart rate sensor found by the scan. The second parameter, false, to the connect() function means that attribute objects this client may already have created aren’t deleted. Now, if there’s no NimBLEClient object already for the device, the code tries to reuse one by calling NimBLEDevice::getDisconnectedClient(). If there’s no client to reuse, the code creates a new one with NimBLEDevice::createClient(). Then, it sets a client connection callback, as well as these connection parameters: • the minimum and maximum connection interval, in 1.25 ms units

40

• the number of packets allowed to skip • the timeout time before disconnecting, in 10 ms units After this, the code sets the number of seconds to wait for the connection attempt to complete. If the connection succeeds, you get the Heart Rate service and then its Heart Rate Measurement characteristic. You check whether the characteristic supports notifications and then subscribe to these notifications with notifyCB() as the callback function to be called for each notification. The first parameter to the subscribe() function is true for notifications and false for indications. If notifications aren’t supported, the code checks whether indications are, and subscribes to them if they are. After all this, your ESP32 device is connected to your heart rate sensor and receives notifications. For each notification, notifyCB() is called. Its arguments are pointers to the characteristic and its data, as well as the number of bytes in the data and a boolean value that’s true for notifications and false for indications. The code in this callback decodes the data from the Heart Rate Measurement characteristic. But, instead of the simplified decoding from subsection 4.8.1, where only the second byte of the data is used, this code also checks the Heart Rate Value Format bit of the first byte. On page 10 of the Heart Rate Service specification (https://www.bluetooth.com/ specifications/specs/heart-rate-service-1-0/), you see that the first byte has some flags, where bit 0 is Heart Rate Value Format bit: 40

The numbers 40 and 56 are equivalent to 50 and 70 ms, respectively - the recommended values for the minimum and maximum connection interval listed on page 18 of the Heart Rate Profile specification.

● 130

Boek BLE 220329 UK.indd 130

06/05/2022 15:54

Chapter 4 • Connections and services

The Heart Rate Value Format bit (bit 0 of the Flags field) indicates if the data format of the Heart Rate Measurement Value field is in a format of UINT8 or UINT16. When the Heart Rate Value format is sent in a UINT8 format, the Heart Rate Value Format bit shall be set to 0. When the Heart Rate Value format is sent in a UINT16 format, the Heart Rate Value Format bit shall be set to 1. So, in the callback function, pData[0] & 1 (because 2 to the power of 0 is 1) tests whether the rightmost bit (bit 0) of this byte is set to 1. The second byte read from the characteristic, pData[1], is a byte with the heart rate, and if the Heart Rate Value Format bit is 1, the third byte, pData[2], is the heart rate’s next byte. If the heart rate is represented by two bytes, pData[2] has to be shifted 8 bits to the left. This is what pData[2] user_data; return bt_gatt_attr_read(conn, attr, buf, len, offset, value, sizeof(char_value)); } static uint8_t indicate; static uint8_t indicating; static struct bt_gatt_indicate_params ind_params; static void ccc_cfg_changed(const struct bt_gatt_attr *attr, uint16_t value) { indicate = (value == BT_GATT_CCC_INDICATE) ? 1 : 0; } // Primary Service Declaration BT_GATT_SERVICE_DEFINE( service, BT_GATT_PRIMARY_SERVICE(&service_uuid), BT_GATT_CHARACTERISTIC(&char_uuid.uuid, BT_GATT_CHRC_READ | BT_GATT_CHRC_INDICATE, BT_GATT_PERM_READ, read_characteristic,

● 138

Boek BLE 220329 UK.indd 138

06/05/2022 15:54

Chapter 4 • Connections and services

NULL, char_value), BT_GATT_CCC(ccc_cfg_changed, BT_GATT_PERM_READ | BT_GATT_PERM_WRITE), ); // Advertising data static const struct bt_data ad[] = { BT_DATA_BYTES(BT_DATA_FLAGS, (BT_LE_AD_GENERAL | BT_LE_AD_NO_BREDR)), BT_DATA_BYTES(BT_DATA_UUID128_ALL, BT_UUID_CUSTOM_SERVICE_VAL), }; // Indicate callbacks static void indicate_cb(struct bt_conn *conn, struct bt_gatt_indicate_params *params, uint8_t err) { printk("Indication %s\n", err != 0U ? "fail" : "success"); } static void indicate_destroy(struct bt_gatt_indicate_params *params) { printk("Indication complete\n"); indicating = 0U; } // GATT callbacks void mtu_updated(struct bt_conn *conn, uint16_t tx, uint16_t rx) { printk("Updated MTU: TX: %d RX: %d bytes\n", tx, rx); } static struct bt_gatt_cb gatt_callbacks = {.att_mtu_updated = mtu_updated}; // Connection callbacks static void connected(struct bt_conn *conn, uint8_t err) { if (err) { printk("Connection failed (err 0x%02x)\n", err); } else { printk("Connected\n"); } } static void disconnected(struct bt_conn *conn, uint8_t reason) { printk("Disconnected (reason 0x%02x)\n", reason); } static struct bt_conn_cb conn_callbacks = { .connected = connected,

● 139

Boek BLE 220329 UK.indd 139

06/05/2022 15:54

Bluetooth Low Energy Applications

.disconnected = disconnected, }; static void bt_ready(void) { int err; printk("Bluetooth initialized\n"); err = bt_le_adv_start(BT_LE_ADV_CONN_NAME, ad, ARRAY_SIZE(ad), NULL, 0); if (err) { printk("Advertising failed to start (err %d)\n", err); return; } printk("Advertising successfully started\n"); } void main(void) { int err; printk("Starting firmware...\n"); // Initialize BME280 bme280 = bme280_get_device(); if (bme280 == NULL) { return; } // Initialize the Bluetooth subsystem err = bt_enable(NULL); if (err) { printk("Bluetooth init failed (err %d)\n", err); return; } bt_ready(); // Register GATT and connection callbacks bt_gatt_cb_register(&gatt_callbacks); bt_conn_cb_register(&conn_callbacks); // Implement indications every second while (1) { k_sleep(K_SECONDS(1));

● 140

Boek BLE 220329 UK.indd 140

06/05/2022 15:54

Chapter 4 • Connections and services

if (indicate) { if (indicating) { continue; } if (update_data_bme280(bme280)) { ind_params.attr = &service.attrs[2]; ind_params.func = indicate_cb; ind_params.destroy = indicate_destroy; ind_params.data = char_value; ind_params.len = sizeof(char_value); if (bt_gatt_indicate(NULL, &ind_params) == 0) { indicating = 1U; } } } } }

First you define a 128bit UUID for a BLE service and its characteristic. You can generate a random UUID with the uuidgen command on Linux or macOS. You just have to make sure that you split the UUID into its component parts before using the BT_UUID_128_ENCODE macro. Note: You generally use a random UUID ending with 0 for a custom service, and then use the same UUID, but ending with 1, for its first characteristic. Increment the UUID for additional characteristics. The characteristic value is defined as an array of six bytes (char_value). This contains the 16bit values for the temperature, pressure, and humidity as explained in the BME280 example in the previous chapter. update_data_bme280() gets the sensor measurements from the BME280 and updates the corresponding bytes in char_value. Note: You only want to indicate or notify data when the data has changed since the previous indication or notification. That’s why the update_data_bme280() function compares the new values to the previous ones. If at least one sensor value changes, the function returns 1. The read_characteristic() function is a callback function: it’s called upon reading a characteristic value. This updates the BME280 data and reads the BLE attribute’s data and returns it. Likewise, ccc_cfg_changed() is a callback function that’s called when the Client Characteristic Configuration has changed.

● 141

Boek BLE 220329 UK.indd 141

06/05/2022 15:54

Bluetooth Low Energy Applications

Now, with the appropriate UUIDs and callback functions defined, it’s time for the primary service declaration. This is done with the BT_GATT_SERVICE_DEFINE macro, which has a name and then the attributes in the declaration. This service contains three macros: BT_GATT_PRIMARY_SERVICE Declares a primary service attribute with a specified UUID. BT_GATT_CHARACTERISTIC Declares a characteristic attribute with its value. BT_GATT_CCC Declares a Client Characteristic Configuration attribute. The characteristic declaration with BT_GATT_CHARACTERISTIC has a lot of arguments. First comes the characteristic’s UUID, then its properties (you want it to support reads and indications), then its access permissions (just read), and then the callback functions for read (read_characteristic()) and write (none) operations. The last argument refers to the value of the characteristic, which is the char_value array you defined earlier. With the BT_GATT_CCC macro, you define a CCC attribute with read and write permissions, which calls the ccc_cfg_changed() function when its value changes. Next comes the advertising data. You must make sure that you advertise the 128bit UUID for the service you defined. The next few functions are all callback functions for indications, GATT, and the connection. In this code, they just print some debug information. The bt_ready() function starts advertising with the device name defined in prj.conf, and the advertising data defined earlier in the code. With all this code set up, the main() function initializes the BME280 sensor and Bluetooth subsystem, starts advertising the service UUID, registers the GATT and connection callbacks, then starts indicating, if indications are enabled. Every second, it checks whether the update_data_bme280() function returns a non-zero value. If it does, the indicate parameters are set up (including the current sensor values in char_value) and a call to bt_gatt_indicate() indicates an attribute value change. After you’ve flashed this firmware to a RuuviTag or an nRF52840 Dongle with the BME280 sensor, connect to the device using the nRF Connect app. Then you can read the characteristic with the sensor values or enable indications to receive new sensor values when there’s an update. This is how your device appears in nRF Connect when you’re scanning for BLE devices:

● 142

Boek BLE 220329 UK.indd 142

06/05/2022 15:54

Chapter 4 • Connections and services

Figure 4.11 Your device appears in nRF Connect with your custom UUID and the device name you configured. When you press Connect, you see the Generic Attribute and Generic Access service and an Unknown Service with the UUID you configured in your code:

Figure 4.12 Your device has three services. Zephyr automatically creates the first two, and the third one is your custom service. The Generic Access service has Device Name and Appearance characteristics with the values you supplied in prj.conf:

● 143

Boek BLE 220329 UK.indd 143

06/05/2022 15:54

Bluetooth Low Energy Applications

Figure 4.13 Zephyr has created the Generic Access service with the characteristics you defined in the project configuration file. The unknown service has an Unknown Characteristic with the UUID you configured in your code. This has INDICATE and READ as its properties, and a Client Characteristic Configuration descriptor that shows you whether indications are enabled. Press the arrow next to the characteristic to read the current value of your sensor. If you press the two arrows at the right, you enable indications and receive a new value when there’s an update:

● 144

Boek BLE 220329 UK.indd 144

06/05/2022 15:54

Chapter 4 • Connections and services

Figure 4.14 You can subscribe to the indications to receive updated sensor values. If you have opened a serial connection to your device (which is possible for the nRF52840 Development Kit), you’ll see two messages for every indication: Updated MTU: TX: 23 RX: 23 bytes Connected Indication success Indication complete Indication success Indication complete ...

Disconnect from the device in the nRF Connect app when you’re done.

4.10.3 Reading the sensor characteristic Now, you can also create a Python script that subscribes to your device’s indications and decodes the sensor values. This looks like this: """Subscribe to indications of a BME280 sensor. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio import sys

● 145

Boek BLE 220329 UK.indd 145

06/05/2022 15:54

Bluetooth Low Energy Applications

from construct import Int16sl, Int16ul, Struct from construct.core import StreamError import bleak bme280_format = Struct( "temperature" / Int16sl, "pressure" / Int16ul, "humidity" / Int16ul, ) BME280_SENSOR_UUID = "63bf0b19-2b9c-473c-9e0a-2cfcaf03a771"

def sensor_value_changed(handle: int, data: bytearray): """Show sensor values for an indication.""" try: sensor_data = bme280_format.parse(data) print(f"Temperature: {sensor_data.temperature / 100} °C") print(f"Humidity

: {sensor_data.humidity / 100} %")

print( f"Pressure

: {(sensor_data.pressure + 50000) / 100} hPa"

) print(24 * "-") except StreamError: # Wrong format pass

async def main(address): """Connect to BME280 sensor and subscribe to indications.""" try: async with bleak.BleakClient(address) as client: print(f"Connected to {address}") await client.start_notify( BME280_SENSOR_UUID, sensor_value_changed ) print("Indications started...") while True: await asyncio.sleep(1) except asyncio.exceptions.TimeoutError: print(f"Can’t connect to device {address}.")

● 146

Boek BLE 220329 UK.indd 146

06/05/2022 15:54

Chapter 4 • Connections and services

if __name__ == "__main__": if len(sys.argv) == 2: address = sys.argv[1] asyncio.run(main(address)) else: print("Please specify the Bluetooth address.")

The structure of this code is a combination of the decoding with Construct from subsection 3.10.4, and the heart rate measurement notifications from 4.8.1. You subscribe to your sensor’s custom characteristic’s indications, decode the characteristic’s values in the sensor_value_changed() callback function, then print the values. Note: Bleak’s start_notify() method automatically enables notifications or indications, whichever of the two is available. You don’t have to worry about the difference. If you run this Python script with your device’s Bluetooth address as the argument, it shows this output: $ python3 bme280_indications.py C5:98:17:63:C3:E3 Connected to C5:98:17:63:C3:E3 Temperature: 26.04 °C Humidity

: 50.1 %

Pressure

: 1010.56 hPa

-----------------------Indications started... Temperature: 26.02 °C Humidity

: 50.13 %

Pressure

: 1010.56 hPa

-----------------------Temperature: 26.01 °C Humidity

: 50.17 %

Pressure

: 1010.56 hPa

-----------------------...

Just press Ctrl+C to disconnect from the device and stop the program. Note: The message Indications started… appears later than the first message that contained an indication with sensor values. This is just a consequence of the asynchronous nature of the calls.

● 147

Boek BLE 220329 UK.indd 147

06/05/2022 15:54

Bluetooth Low Energy Applications

4.10.4 Sniffing packets in an unencrypted BLE connection The BLE sensor you created with Zephyr isn’t using any security: there’s no authentication, and the BLE packets aren’t encrypted. This means that anyone can connect to the sensor and read its sensor measurements. Moreover, anyone can sniff the BLE packets transmitted between the sensor and its client in a connection. Let’s see how you can sniff traffic between a BLE client and the sensor. Start a Wireshark packet capture on your computer with nRF Sniffer for Bluetooth LE and select BME280 sensor as the device. Then, on another device, connect to the sensor, for instance with nRF Connect, and read the custom characteristic with the sensor values. Meanwhile, Wireshark will show you all packets exchanged between the client and the server, including the Read Request and Read Response. You can just view the value, which isn’t encrypted:

Figure 4.15 If a BLE connection isn’t encrypted, anyone can sniff its packets.

● 148

Boek BLE 220329 UK.indd 148

06/05/2022 15:54

Chapter 4 • Connections and services

This is clearly not a good way to exchange sensitive data. In the next chapter, you’ll see how to secure BLE traffic.

4.11 Receiving service data without a connection In the previous chapter, you encountered a couple of advertising data types. You learned how to read data from BLE devices without connecting to them, by receiving manufacturer-specific data advertisements. Another advertising data type is service data. These advertising structures are broadcast just like manufacturer-specific data, but the difference is that the data uses the same format as defined by the service with the same UUID. That’s why I decided to postpone talking about service data in this chapter, because, in the previous chapter, you didn’t yet know what services and characteristics were.

4.11.1 Scanning for service data If you run the advertising data scanner from subsection 3.3.2, you’ll see that some of the detected AdvertisementData objects have a service_data attribute. This is a dictionary, with service UUIDs as keys and the corresponding data as their values. You can easily filter for this type of advertisement with the following Python script: """Scan for BLE service data. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio from bleak import BleakScanner from bleak.backends.device import BLEDevice from bleak.backends.scanner import AdvertisementData

def device_found( device: BLEDevice, advertisement_data: AdvertisementData ): """Show service data on detection of a BLE device.""" if advertisement_data.service_data: print(device.address) for service, data in advertisement_data.service_data.items(): print(f"- {service}: 0x{data.hex()}")

async def main(): """Register detection callback and scan for devices.""" scanner = BleakScanner()

● 149

Boek BLE 220329 UK.indd 149

06/05/2022 15:54

Bluetooth Low Energy Applications

scanner.register_detection_callback(device_found) await scanner.start() await asyncio.sleep(5.0) await scanner.stop()

asyncio.run(main())

If you run this program, it will scan for five seconds and show the service data it finds: $ python3 scan_service_data.py 50:1B:A1:C4:7B:9B - 0000fd6f-0000-1000-8000-00805f9b34fb: 0xba567af839557bd57bb25e67edb3eb9f180bf8e8 55:F8:9A:54:4C:F6 - 0000fe50-0000-1000-8000-00805f9b34fb: 0xa91d C4:7C:8D:67:65:AD - 0000fe95-0000-1000-8000-00805f9b34fb: 0x310298000aad65678d7cc40d B5:D1:EB:C8:93:FB - 00005242-0000-1000-8000-00805f9b34fb: 0x410cb5d1ebc893fb0000 74:19:AF:B3:DE:F9 - 0000fe9f-0000-1000-8000-00805f9b34fb: 0x000000000000000000000000000000000000000 0 EB:76:55:B9:56:18 - 0000180f-0000-1000-8000-00805f9b34fb: 0x08

If you look at the list of assigned numbers, you’ll see that the last line is the Battery Service (0x180f). The value in this service data advertisement is the value of the Battery Level characteristic. This is a one-byte value representing the current battery level as a percentage from 0% to 100%. In this example output, this device’s battery level 0x08, or 8%. If you’re searching for service data advertisements in Wireshark, add the display filter btcommon.eir_ad.entry.type == 0x16. You’ll see something like this then:

● 150

Boek BLE 220329 UK.indd 150

06/05/2022 15:54

Chapter 4 • Connections and services

Figure 4.16 With the appropriate display filter, you can scan for service data in Wireshark. As you remember from subsection 3.2.2, each advertising data structure contains a length byte, its type, and then the data. In this case, you see that the raw data for the service data shows length 0x04, Service Data type 0x16, the 16-bit UUID 0x180f for the Battery Service (in little-endian order 0x0f18), and then the value of the service data - the byte, 0x08.

4.11.2 Receiving Exposure Notification advertisements Service data aren’t frequently used for the Bluetooth SIG assigned services. You’ll encounter them more often for vendor-specific services, and many times the specification is not readily available, so you have to reverse engineer it. 42 However, one type of vendor-specific service data with a publicly available specification is the joint Google/Apple Exposure Notification specification. 43 This is used by contact tracing applications on Android and iOS smartphones to combat the spread of the SARSCoV2 virus that causes COVID19. 42 43

See chapter 7 for an approach for reverse engineering BLE devices. You can find the specification at https://covid19.apple.com/contacttracing.

● 151

Boek BLE 220329 UK.indd 151

06/05/2022 15:54

Bluetooth Low Energy Applications

According to page 5 of the Exposure Notification specification, its advertisements must use the ADV_NONCONN_IND type. The advertiser address should be private non-resolvable.44 The advertising payload should look like this:

Figure 4.17 Structure of the Exposure Notification packet sent by contact tracing apps. As you see, the advertisement consists of three advertising data structures: Flags, Complete 16bit Service UUID and Service Data - 16bit UUID. You can also verify this in Wireshark. A display filter to find all Exposure Notification packets in the neighborhood is btcommon.eir_ad.entry.uuid_16 == 0xfd6f. 45 The Rolling Proximity Identifier (RPI) in the service data is an identifier derived from an encryption key. This identifier changes about every 15 minutes, together with the private address of the device. This is to prevent others from tracking the device. The Associated Encrypted Metadata (AEM) contains some metadata about the device and app, such as a version and a transmit power level. However, this metadata is encrypted. All in all, you can’t get any specific information from these Exposure Notification advertisements without having the correct encryption keys. However, you can at least detect them easily and show the Rolling Proximity Identifier and Associated Encrypted Metadata: """Scan for Exposure Notifications. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio from construct import Array, Byte, Struct from bleak import BleakScanner 44 45

See section 3.4 for the various types of BLE addresses. In my experiments scanning my own Android phone with the Belgian Coronalert app, I found that its advertisements didn’t have the Flags data structure.

● 152

Boek BLE 220329 UK.indd 152

06/05/2022 15:54

Chapter 4 • Connections and services

from bleak.backends.device import BLEDevice from bleak.backends.scanner import AdvertisementData EXPOSURE_NOTIFICATION_UUID = "0000fd6f-0000-1000-8000-00805f9b34fb" exposure_notification_format = Struct( "rpi" / Array(16, Byte), "aem" / Array(4, Byte) )

def device_found( device: BLEDevice, advertisement_data: AdvertisementData ): """Show exposure notification data.""" try: exposure_notification_data = advertisement_data.service_data[ EXPOSURE_NOTIFICATION_UUID ] en = exposure_notification_format.parse( exposure_notification_data ) print(f"Address

: {device.address}")

print( f"Rolling Proximity Identifier

: 0x{bytes(en.rpi).hex()}"

) print( f"Associated Encryption Metadata: 0x{bytes(en.aem).hex()}" ) print(66 * "-") except KeyError: # No Exposure Notification service data pass

async def main(): """Register detection callback and scan for devices.""" scanner = BleakScanner() scanner.register_detection_callback(device_found) while True: await scanner.start() await asyncio.sleep(1.0) await scanner.stop()

asyncio.run(main())

● 153

Boek BLE 220329 UK.indd 153

06/05/2022 15:54

Bluetooth Low Energy Applications

This code again uses Construct to parse the data packet in its two parts, the 16byte Rolling Proximity Identifier and the 4byte Associated Encrypted Metadata. Have a look at section 3.6, where the same approach was used to decode iBeacon advertisements. The only substantial difference is that the Exposure Notification scanner uses the service_data attribute instead of the manufacturer_data attribute from the AdvertisementData object in the callback function. If you run this code, it will show you exposure notifications from surrounding devices: $ python3 scan_exposure_notification.py Address

: 53:6B:69:79:4C:CB

Rolling Proximity Identifier

: 0x1d3a473586880f7e542f5b2f57c7b3de

Associated Encryption Metadata: 0xc985e6d7 -----------------------------------------------------------------Address

: 12:F8:0F:D9:12:B7

Rolling Proximity Identifier

: 0x701eec075a28f8623d5ae118c317d6df

Associated Encryption Metadata: 0xbbdc3deb -----------------------------------------------------------------...

4.12 Summary and further exploration In this chapter, you learned about connecting to BLE devices. You learned about the GAP and GATT roles relevant to BLE connections, and about the three types of attributes according to the GATT specification: services, characteristics, and descriptors. After this short theoretical foundation, the bulk of this chapter showed you code for working with services and characteristics in many ways: discovering, reading and writing, and receiving notifications and indications. You also learned how to receive service data without a connection. You can expand on many of the code examples in this chapter. I have deliberately simplified them to focus on the core ideas of this chapter. For instance, you could adapt the heart rate monitor with NimBLE-Arduino so that it allows you to choose which heart rate sensor to connect to. And, if you connect a display, you can add a user interface. Have a look at the project on https://github.com/koenvervloesem/M5Core2-Heart-Rate-Display for some inspiration. If you want to try out more sensor applications, have a look at https://github.com/ theBASTI0N/ruuvitag_fw_zephyr. This is feature-complete Zephyr firmware for the RuuviTag. If you prefer other sensor boards, you could try the Nordic Thingy:52 (https://www. nordicsemi.com/Products/Development-hardware/Nordic-Thingy-52) or the ST SensorTile. box (https://www.st.com/en/evaluation-tools/steval-mksbox1v1.html). Both devices are packed with sensors and supported by Zephyr. In the next chapter, you’ll learn how to add security to your BLE connections.

● 154

Boek BLE 220329 UK.indd 154

06/05/2022 15:54

Chapter 5 • Securing BLE connections

Chapter 5 • Securing BLE connections In the previous chapter, you learned how to set up services on a BLE device and then connect to the device to read and write these services’ characteristics. However, these examples never had any security measures to prevent unauthorized access. You’ve probably already heard a couple of news stories about Bluetooth’s insecurity. However, it’s almost never the Bluetooth protocol itself that’s insecure, but the implementation. The Bluetooth specification actually has a lot of security features, especially in newer versions. Security isn’t a binary term: security or no security. It’s a continuum ranging from no security to very high security with various intermediate security levels in between. None of Bluetooth Low Energy’s security features are mandatory. It’s up to you to choose the right security level depending on the product you’re developing and its intended use. Note: Some Bluetooth profile specifications (see Chapter 6) do state specific security requirements. You have to follow these if you’re developing an application that implements one of the roles in the profile. However, with stronger security comes less usability. The most secure connection methods for BLE require that each device has a display and that the user manually verifies that the numeric codes shown on the displays match. Because this is detrimental to the user experience, most devices don’t bother with these secure methods. As a result, they’re vulnerable to man-in-the-middle attacks, where someone can eavesdrop on the connection or even alter the messages. This chapter is a bit more theoretical than previous ones, because you first have to grasp a lot of concepts before you can start using secure connections in BLE. But, after we lay the theoretical groundwork, you’ll soon apply this knowledge to code you can run on your own devices. In this chapter, you’ll learn about: • • • • • • •

Bluetooth Low Energy’s security architecture pairing and bonding LE Legacy Connection pairing and LE Secure Connection pairing various pairing methods security modes and security levels how to encrypt and authenticate BLE connections in Zephyr how to use a resolvable private address in Zephyr

5.1 BLE security architecture Look back at Bluetooth Low Energy’s architecture diagram in the introductory chapter. You’ll see that the Host block has a Security Manager Protocol component in the same layer as the Attribute Protocol. The Security Manager Protocol’s main responsibility is to

● 155

Boek BLE 220329 UK.indd 155

06/05/2022 15:54

Bluetooth Low Energy Applications

generate and exchange the correct encryption keys between BLE devices so that they’re able to communicate securely. However, security isn’t implemented only in the Security Manager Protocol component, but also in other layers of the Bluetooth protocol stack. For instance, a GAP peripheral defines attribute permissions in its attribute table. Those permissions specify what security features you need in order to read or write those attributes. For instance, the permissions could allow for reading an attribute over an encrypted link, but only allow writing to the attribute from an authenticated device. So, if an unpaired device connects and tries to read the attribute, the read fails and pairing is triggered. After paring, the read succeeds. However, the write only succeeds when pairing results in an authenticated link.

5.2 Pairing and bonding Two concepts are essential to understand security in BLE connections: pairing and bonding. Pairing If two devices have not yet communicated securely but want to do so, they must pair. This means that they authenticate with each other by verifying that they share the same secret (such as a PIN code), they encrypt the data that they exchange, and that they exchange secret keys. Bonding If two devices have paired and both store the exchanged secret keys in their own local security database, they’re bonded. This makes starting a later secure connection faster. If just one device stores the secret keys, reconnection will fail and both devices will have to pair again. A sequence diagram of pairing and bonding looks like this:

● 156

Boek BLE 220329 UK.indd 156

06/05/2022 15:54

Chapter 5 • Securing BLE connections

Figure 5.1 Pairing and bonding between two BLE devices happens in three phases. The procedure begins with the establishment of a link layer connection between both devices. It happens in three phases, and pairing consists of the first two phases. The master (central) is the initiator of the security procedure, and the slave (peripheral) is the responder. The latter can also request the start of a pairing procedure by sending a Security Request message to the master. However, it’s always the master that starts the pairing process by send the Pairing Request message. In this first phase, both devices exchange their security requirements and the features they support by using the Pairing Request and Pairing Response messages. The exchanged information determines which pairing method (also called association model) is used in the second phase. In the second phase, both devices use the Security Manager Protocol (SMP) to exchange the keys that will be used to establish an encrypted connection. So, after Phase 2, the connection between devices is encrypted. Another purpose of the second phase is (optional) authentication, to protect against manin-the-middle (MITM) attacks. In a man-in-the-middle attack, an attacker puts themself in the middle –between two parties who think they’re communicating directly with each other. He intercepts messages from both parties and relays them to each other while secretly reading them. So, they think they’re communicating directly with each other, but, in reality, the attacker is in control of the whole connection. They can even drop, modify, or add messages, without either party knowing that this is the attacker’s work. By using a pairing method that offers authentication in the second phase, you prevent man-in-themiddle attacks on a BLE connection.

● 157

Boek BLE 220329 UK.indd 157

06/05/2022 15:54

Bluetooth Low Energy Applications

Note: This second phase (and only this phase) differs between LE Legacy Connections (the original BLE connections introduced in Bluetooth 4.0) and LE Secure Connections (introduced in Bluetooth 4.2). Bonding happens in the third phase. This is optional, but it lets devices skip the first two phases when they want to reconnect securely later. This is possible because they’ve both stored the required keys to reestablish the encrypted connection.

5.2.1 Phase 1: Exchange of pairing information The Pairing Request and Pairing Response messages exchanged in the first phase between the initiator and responder are almost identical. They contain information about the devices’ capabilities and requirements. After this exchange, both devices are able to determine which input and output capabilities the other has (it’s useful to know if a PIN code may be requested), and which pairing type (legacy or secure) and pairing method each supports. If both devices request LE Secure Connection pairing, they will use this. 46 If only one device requests it, they will use LE Legacy Connection pairing. Information about input and output capabilities is essential to decide which pairing method can be used. BLE knows of three types of input capabilities: Capability

Description

No input

The user has no ability to indicate yes or no.

Yes/No

The user can indicate yes or no.

Keyboard

The device has a numeric keyboard. This means that the user is able to input the numbers 0 through 9 and a confirmation, and also has a mechanism to indicate yes or no.

Table 5.1 Input capabilities And, there are two types of output capabilities: Capability

Description

No output

The device doesn’t have the ability to communicate a 6-digit decimal number.

Numeric

The device does have the ability to

output

communicate a 6-digit decimal number.

Table 5.2 Output capabilities

46

The BLE specification mandates this.

● 158

Boek BLE 220329 UK.indd 158

06/05/2022 15:54

Chapter 5 • Securing BLE connections

Now, if you combine those input and output capabilities, every device has one of these six I/O combinations: No output

Numeric output

No input

NoInputNoOutput

DisplayOnly

Yes/No

NoInputNoOutput

DisplayYesNo

Keyboard

KeyboardOnly

KeyboardDisplay

Table 5.3 Input and output capabilities Note: The BLE specification doesn’t define a pairing method that uses Yes/No for input and doesn’t have any output. For this combination, the input isn’t used, and the I/O capability is represented as NoInputNoOutput.

5.2.2 Phase 2: Pairing In the second phase, both devices choose an appropriate pairing method based on their I/O capabilities. This phase depends on the pairing type (legacy or secure).

5.2.2.1 LE Legacy Connection pairing With LE Legacy Connection pairing (possible in all Bluetooth versions), the devices use two keys: Temporary Key (TK) Generated as a temporary key to start the pairing process. Short Term Key (STK) Generated based on the TK and two randomly generated values exchanged between the devices. Devices have three methods of generating a TK in LE Legacy Connection pairing mode:

● 159

Boek BLE 220329 UK.indd 159

06/05/2022 15:54

Bluetooth Low Energy Applications

Pairing

Just Works

Passkey Entry

Out of Band (OOB)

0

Randomly generated six-digit

Randomly generated 128bit value.

method TK

number, padded with leading zeroes to form a 128bit value. Entropy

0 bits

20 bits

128 bits

MITM

No

Yes

Yes

The key is

One device shows the number

The devices exchange the TK

completely

on its display and the other

over a medium other than BLE.

predicta-

asks the user to enter it. This

An example is NFC (Near-field

ble, so the

is only possible if the devices

communication), a contactless

connection is

have some input and output

technology that works over a

unauthenti-

capabilities.

distance of a couple of centime-

protection Comment

cated.

ters. If one of the devices has a camera, another useful medium is a QR code.

Table 5.4 LE Legacy Connection pairing methods The Entropy row shows the strength of the temporary key for each pairing method. The larger the entropy, the more difficult it is to attack the pairing process. As is clear from the entropy values, Just Works doesn’t have any security, and Out of Band is the most secure of the legacy pairing methods. 47 Warning: As the TK is used to derive the STK, a weak TK also results in a weak STK, and hence a weak pairing process. If someone is able to observe the exchange of packets during a weak pairing process using a BLE packet sniffer, they can brute-force the TK and then obtain all of the exchanged keys. From these pairing methods, Just Works doesn’t offer any authentication: anyone can connect using Just Works. Only with Passkey Entry or Out of Band, the TK is exchanged between both devices visually (with Passkey Entry) or via another medium (OOB). As a result, knowledge of the TK can be taken as confirmation that the authenticating device is the authorized one. These are the combinations of I/O capabilities and the resulting pairing method for LE Legacy Connection pairing:

47

At least if the out-of-band mechanism is secure and the generated value for TK is fully random.

● 160

Boek BLE 220329 UK.indd 160

06/05/2022 15:54

Chapter 5 • Securing BLE connections

Initiator Display Only

Display YesNo

Responder

Keyboard

NoInput

Keyboard

Only

NoOutput

Display

Display Only

Just Works

Just Works

Passkey Entry: responder displays, initiator inputs

Just Works

Passkey Entry: responder displays, initiator inputs

Display

Just Works

Just Works

Passkey Entry: responder displays, initiator inputs

Just Works

Passkey Entry: responder displays, initiator inputs

Passkey Entry: initiator displays, responder inputs

Passkey Entry: initiator displays, responder inputs

Passkey Entry: initiator and responder inputs

Just Works

Passkey Entry: initiator displays, responder inputs

Just Works

Just Works

Just Works

Just Works

Just Works

Passkey Entry: initiator displays, responder inputs

Passkey Entry: initiator displays, responder inputs

Passkey Entry: responder displays, initiator inputs

Just Works

Passkey Entry: initiator displays, responder inputs

YesNo

Keyboard Only

NoInput NoOutput KeyboardDisplay

Table 5.5 LE Legacy Connection pairing methods based on I/O capabilities As you can see, if either the initiator or the responder don’t have any input or output, the only possible pairing method is Just Works, which is unauthenticated. So, authenticated communication between BLE devices is only possible when using some kind of input or output. At the end of Phase 2, the STK is used to calculate a session key, which is used to encrypt the link from this point on.

5.2.2.2 LE Secure Connection pairing With LE Secure Connection pairing (possible since Bluetooth 4.2), both devices use the ECDH (Elliptic-Curve Diffie–Hellman) key exchange protocol. Each generates a key pair with a public and a private key, after which they exchange each other’s public keys, and they use this to generate a shared secret key, the long-term key (LTK). This makes LE Secure Connection pairing significantly more secure than LE Legacy Connection pairing. Devices have four ways to generate the LTK in LE Secure Connection pairing: 48

48

Note that Just Works, Passkey Entry, and Out of Band don’t work the same as the identically-named pairing methods in LE Legacy Connection pairing.

● 161

Boek BLE 220329 UK.indd 161

06/05/2022 15:54

Bluetooth Low Energy Applications

Just Works The devices exchange their public keys together with other generated values. Because of the ECDH key exchange, this offers substantially stronger protection against passive eavesdropping than the identically-named method in LE Legacy Connection pairing. However, there’s still no protection against man-in-the-middle attacks. Passkey Entry An identical six-digit number is exchanged between two devices. The user enters the same number into each device, or one device shows a randomly generated number on its display and the other asks the user to enter it. This method is much more secure against manin-the-middle attacks than the same method in LE Legacy Connection pairing. This is only possible if the devices have some input and output capabilities. Out of Band (OOB) The devices exchange their public keys and other values over a medium other than BLE. An example is NFC (Near-field communication), a contactless technology that works over a distance of a couple of centimeters. If one of the devices has a camera, another useful medium is a QR code. Numeric Comparison This starts the same as Just Works but adds an extra step that protects against man-inthe-middle attacks. In this extra step, both devices generate a six-digit confirmation value independently and both display their values to the user. The user checks that the values match and confirms this, so a man-in-the-middle attack can be ruled out. This is the most secure pairing method of all, and it just needs two buttons to confirm yes or no. So, if both devices support Bluetooth 4.2, this is the preferred method. Note: In contrast to LE Legacy Connection pairing, the data input by the user during the pairing method is only used for authentication purposes in LE Secure Connection Pairing. It isn’t used to derive cryptographic keys, so a low level of entropy doesn’t result in weak keys. This makes LE Secure Connection pairing resistant to a passive attack where the pairing process is observed and then a brute-force attack on the TK is made. These are the combinations of I/O capabilities and the resulting pairing methods for LE Secure Connection pairing:

● 162

Boek BLE 220329 UK.indd 162

06/05/2022 15:54

Chapter 5 • Securing BLE connections

Initiator

Display

Display YesNo

Only

Keyboard

NoInput

Keyboard

Only

NoOutput

Display

Responder Display Only

Just Works

Just Works

Passkey Entry: responder displays, initiator inputs

Just Works

Passkey Entry: responder displays, initiator inputs

Display

Just Works

Numeric Comparison

Passkey Entry: responder displays, initiator inputs

Just Works

Numeric Comparison

Passkey Entry: initiator displays, responder inputs

Passkey Entry: initiator displays, responder inputs

Passkey Entry: initiator and responder inputs

Just Works

Passkey Entry: initiator displays, responder inputs

Just Works

Just Works

Just Works

Just Works

Just Works

Passkey Entry: initiator displays, responder inputs

Numeric Comparison

Passkey Entry: responder displays, initiator inputs

Just Works

Numeric Comparison

YesNo

Keyboard Only

NoInput NoOutput KeyboardDisplay

Table 5.6 LE Secure Connection pairing methods based on I/O capabilities The cells with Numeric Comparison shown in bold are the only differences with the LE Legacy Connection pairing table.

5.2.3 Phase 3: Bonding After Phase 2, the connection is encrypted with the key generated in that phase. For LE Legacy Connection pairing, this is the session key based on the STK, while for LE Secure Connection pairing, this is the LTK. With LE Legacy Connection pairing, the LTK is generated in Phase 3. Warning: The primary purpose of Phase 2 is to encrypt the link to securely distribute keys. However, with LE Legacy Connection pairing, the link will continue to be encrypted using the STK-based session key, which may have a low entropy or even zero entropy. Only after a reconnection, the stronger LTK is used for link encryption. In Phase 3, the devices also generate: Connection Signature Resolving Key (CSRK) Used to sign data and verify signatures over an unencrypted link

● 163

Boek BLE 220329 UK.indd 163

06/05/2022 15:54

Bluetooth Low Energy Applications

Identity Resolving Key (IRK) Used to resolve private addresses (see section 5.6) For bonding to occur, both devices have to store these keys so that they can securely reconnect later without having to re-pair. The same CSRK is stored on both devices. The IRK is unique for each device, so the master stores the slave’s IRK and the slave stores the master’s IRK. This way, they’re able to resolve each other’s private addresses.

5.3 Security modes and levels Two other important concepts in BLE security, defined by the Generic Access Protocol (GAP), are security modes and security levels. A BLE connection can be in one of the following security modes, each of them having several security levels: Security Mode 1 is concerned with encryption. It has four possible security levels: Level 1 No security (no authentication and no encryption) Level 2 Unauthenticated pairing with encryption Level 3 Authenticated pairing with encryption Level 4 Authenticated LE Secure Connection pairing with encryption Security Mode 1 Level 1 is the security level you used in the previous chapter. Security Mode 2 is concerned with data signing. It has two possible security levels: Level 1 Unauthenticated pairing with data signing Level 2 Authenticated pairing with data signing Data signing is only possible over an unencrypted link, and it’s only used for a Signed Write command. It’s rarely used, because there are better ways to handle data signing. You should use Security Mode 1 Level 2, 3, or 4 (all encrypted). BLE link encryption adds a Message Integrity Check (MIC) field to all packets, which has the same purpose as data signing.

● 164

Boek BLE 220329 UK.indd 164

06/05/2022 15:54

Chapter 5 • Securing BLE connections

Security Mode 3 is concerned with authenticated broadcasting. security levels:

49

It has three possible

Level 1 No security (no authentication and no encryption) Level 2 Unauthenticated broadcast code Level 3 Authenticated broadcast code There’s also a special security mode, Secure Connections Only Mode. This is a mode in which the device supports Security Mode 1 Level 4 only. Each BLE connection starts in Security Mode 1 Level 1, which means no authentication and no encryption. After this, the connection can be upgraded to any security level using the chosen pairing method. A connection is considered either authenticated or unauthenticated based on the pairing method. Just Works is unauthenticated (both for LE Legacy Connection pairing and LE Secure Connection pairing), while the other methods are authenticated. A connection between two devices can be in one Security Mode, but work at different Security Levels. For instance, different characteristics may require different Security Levels: one characteristic can be read in Security Level 1, while the same or another characteristic requires Security Level 3 or 4 for write access. 50

5.4 Encrypting the BLE connection to a Zephyr sensor Let’s revisit the BLE sensor from subsection 4.10.2. You created this Zephyr application that notified you about temperature, humidity, and pressure measurements with the BME280 sensor. You were able to connect to this device without any authentication, and it didn’t encrypt the connection. In this section, you’re going to add security to this application. Note: For this section, and especially for the next one, I recommend using the nRF52840 Development Kit, which allows you to read debugging information on the serial console. If you want to do this with an nRF52840 Dongle, have a look at how this is done in the following code: https://github.com/koenvervloesem/openhaystack-zephyr.

49

50

This is the only security mode that’s defined for connectionless communication in BLE. More specifically, it’s the security mode for broadcast isochronous channels, introduced in Bluetooth 5.2. This type of broadcast is used for time-synchronized communication, such as Bluetooth LE Audio. An encryption key is generated from a broadcast code and then used to encrypt all data broadcast to devices in a group. There’s also a mixed security mode for devices that need both unsigned and signed data.

● 165

Boek BLE 220329 UK.indd 165

06/05/2022 15:54

Bluetooth Low Energy Applications

5.4.1 Implementing Security Mode 1 Level 2 In this section, I build on the example from the previous chapter to add security. The first addition you need in prj.conf is to enable the Security Manager Protocol, as well as the enabling of Zephyr’s Settings API to store bonding information. You also need to increase the stack size for the Settings API: # Enable Bluetooth peripheral CONFIG_BT=y CONFIG_BT_PERIPHERAL=y # Enable Bluetooth Security Manager Protocol CONFIG_BT_SMP=y # GAP attributes CONFIG_BT_DEVICE_NAME="BME280 sensor" CONFIG_BT_DEVICE_APPEARANCE=1344 # Use Settings API for BLE bonding CONFIG_BT_SETTINGS=y CONFIG_SETTINGS=y CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048 CONFIG_FLASH=y CONFIG_FLASH_PAGE_LAYOUT=y CONFIG_FLASH_MAP=y CONFIG_FCB=y # Enable BME280 sensor CONFIG_I2C=y CONFIG_SENSOR=y CONFIG_BME280=y

For the source code, you can just reuse the bme280.c and bme280.h files from the previous chapter. For the most part, main.c stays the same. You only need a couple of additions to set up a secure connection: /* * Indications for BME280 sensor data in a custom BLE service, * using Security Mode 1 Level 2. * * Copyright (c) 2021 Koen Vervloesem * * SPDX-License-Identifier: MIT */ #include #include

● 166

Boek BLE 220329 UK.indd 166

06/05/2022 15:55

Chapter 5 • Securing BLE connections

#include #include #include #include #include #include #include #include #include #include #include #include "bme280.h" // Define BLE service 63bf0b19-2b9c-473c-9e0a-2cfcaf03a770 #define BT_UUID_CUSTOM_SERVICE_VAL

\

BT_UUID_128_ENCODE(0x63bf0b19, 0x2b9c, 0x473c, 0x9e0a,

\

0x2cfcaf03a770) static struct bt_uuid_128 service_uuid = BT_UUID_INIT_128(BT_UUID_CUSTOM_SERVICE_VAL); // Define BLE characteristic 63bf0b19-2b9c-473c-9e0a-2cfcaf03a771 static struct bt_uuid_128 char_uuid = BT_UUID_INIT_128(BT_UUID_128_ENCODE(0x63bf0b19, 0x2b9c, 0x473c, 0x9e0a, 0x2cfcaf03a771)); // Initialize characteristic value (temperature, pressure, humidity) static uint8_t char_value[6] = {0, 0, 0, 0, 0, 0}; const struct device *bme280 = NULL; // Update data with BME280 sensor measurement // Returns 1 if at least one sensor measurement changed compared to // the previously stored value. uint8_t update_data_bme280(const struct device *dev) { int16_t temperature; uint16_t pressure, humidity; uint8_t changed = 0; bme280_fetch_sample(dev); temperature = bme280_get_temperature(dev); if (memcmp(&(char_value[0]), &temperature, 2)) {

● 167

Boek BLE 220329 UK.indd 167

06/05/2022 15:55

Bluetooth Low Energy Applications

memcpy(&(char_value[0]), &temperature, 2); changed = 1; } pressure = bme280_get_pressure(dev); if (memcmp(&(char_value[2]), &pressure, 2)) { memcpy(&(char_value[2]), &pressure, 2); changed = 1; } humidity = bme280_get_humidity(dev); if (memcmp(&(char_value[4]), &humidity, 2)) { memcpy(&(char_value[4]), &humidity, 2); changed = 1; } return changed; } // Callback function for reading characteristic static ssize_t read_characteristic(struct bt_conn *conn, const struct bt_gatt_attr *attr, void *buf, uint16_t len, uint16_t offset) { update_data_bme280(bme280); const char *value = attr->user_data; return bt_gatt_attr_read(conn, attr, buf, len, offset, value, sizeof(char_value)); } static uint8_t indicate; static uint8_t indicating; static struct bt_gatt_indicate_params ind_params; static void ccc_cfg_changed(const struct bt_gatt_attr *attr, uint16_t value) { indicate = (value == BT_GATT_CCC_INDICATE) ? 1 : 0; } // Primary Service Declaration BT_GATT_SERVICE_DEFINE( service, BT_GATT_PRIMARY_SERVICE(&service_uuid), BT_GATT_CHARACTERISTIC(&char_uuid.uuid, BT_GATT_CHRC_READ | BT_GATT_CHRC_INDICATE, BT_GATT_PERM_READ_ENCRYPT,

● 168

Boek BLE 220329 UK.indd 168

06/05/2022 15:55

Chapter 5 • Securing BLE connections

read_characteristic, NULL, char_value), BT_GATT_CCC(ccc_cfg_changed, BT_GATT_PERM_READ_ENCRYPT | BT_GATT_PERM_WRITE_ENCRYPT), ); // Advertising data static const struct bt_data ad[] = { BT_DATA_BYTES(BT_DATA_FLAGS, (BT_LE_AD_GENERAL | BT_LE_AD_NO_BREDR)), BT_DATA_BYTES(BT_DATA_UUID128_ALL, BT_UUID_CUSTOM_SERVICE_VAL), }; // Indicate callbacks static void indicate_cb(struct bt_conn *conn, struct bt_gatt_indicate_params *params, uint8_t err) { printk("Indication %s\n", err != 0U ? "fail" : "success"); } static void indicate_destroy(struct bt_gatt_indicate_params *params) { printk("Indication complete\n"); indicating = 0U; } // GATT callbacks void mtu_updated(struct bt_conn *conn, uint16_t tx, uint16_t rx) { printk("Updated MTU: TX: %d RX: %d bytes\n", tx, rx); } static struct bt_gatt_cb gatt_callbacks = {.att_mtu_updated = mtu_updated}; // Connection callbacks static void connected(struct bt_conn *conn, uint8_t err) { char addr[BT_ADDR_LE_STR_LEN]; bt_addr_le_to_str(bt_conn_get_dst(conn), addr, sizeof(addr)); if (err) { printk("Failed to connect to %s (%u)\n", addr, err); return; } printk("Connected %s\n", addr); if (bt_conn_set_security(conn, BT_SECURITY_L2)) { printk("Failed to set security\n");

● 169

Boek BLE 220329 UK.indd 169

06/05/2022 15:55

Bluetooth Low Energy Applications

} } static void disconnected(struct bt_conn *conn, uint8_t reason) { char addr[BT_ADDR_LE_STR_LEN]; bt_addr_le_to_str(bt_conn_get_dst(conn), addr, sizeof(addr)); printk("Disconnected from %s (reason 0x%02x)\n", addr, reason); } static void security_changed(struct bt_conn *conn, bt_security_t level, enum bt_security_err err) { char addr[BT_ADDR_LE_STR_LEN]; bt_addr_le_to_str(bt_conn_get_dst(conn), addr, sizeof(addr)); if (!err) { printk("Security changed: %s level %u\n", addr, level); } else { printk("Security failed: %s level %u err %d\n", addr, level, err); } } static struct bt_conn_cb conn_callbacks = { .connected = connected, .disconnected = disconnected, .security_changed = security_changed, }; // Bluetooth initialization static void bt_ready(void) { int err; printk("Bluetooth initialized\n"); if (IS_ENABLED(CONFIG_SETTINGS)) { settings_load(); } err = bt_le_adv_start(BT_LE_ADV_CONN_NAME, ad, ARRAY_SIZE(ad), NULL, 0); if (err) { printk("Advertising failed to start (err %d)\n", err); return;

● 170

Boek BLE 220329 UK.indd 170

06/05/2022 15:55

Chapter 5 • Securing BLE connections

} printk("Advertising successfully started\n"); } void main(void) { int err; printk("Starting firmware...\n"); // Initialize BME280 bme280 = bme280_get_device(); if (bme280 == NULL) { return; } // Initialize the Bluetooth subsystem err = bt_enable(NULL); if (err) { printk("Bluetooth init failed (err %d)\n", err); return; } bt_ready(); // Clear all bonds for debugging purposes // bt_unpair(BT_ID_DEFAULT, BT_ADDR_LE_ANY); // Register GATT and connection callbacks bt_gatt_cb_register(&gatt_callbacks); bt_conn_cb_register(&conn_callbacks); // Implement indications every second while (1) { k_sleep(K_SECONDS(1)); if (indicate) { if (indicating) { continue; } if (update_data_bme280(bme280)) { ind_params.attr = &service.attrs[2]; ind_params.func = indicate_cb; ind_params.destroy = indicate_destroy;

● 171

Boek BLE 220329 UK.indd 171

06/05/2022 15:55

Bluetooth Low Energy Applications

ind_params.data = char_value; ind_params.len = sizeof(char_value); if (bt_gatt_indicate(NULL, &ind_params) == 0) { indicating = 1U; } } } } }

If you’re using this on a device without keyboard and display (I/O capability NoInputNoOutput), such as the nRF52840 Dongle or the RuuviTag, the maximum security level you can reach is Security Mode 1 Level 2 (unauthenticated pairing with encryption). This encrypts the BLE traffic in the connection so no one else can sniff it while the connection is active, but you’re never sure what device is connected. In the primary service declaration, change the permissions to BT_GATT_PERM_READ_ ENCRYPT for your custom characteristic and BT_GATT_PERM_READ_ENCRYPT | BT_ GATT_PERM_WRITE_ENCRYPT for the client characteristic configuration descriptor. This means that clients can only read the characteristic and enable indications if they’re on an encrypted connection. Connection callback connected() does more now: if the connection succeeds, the connection’s security level is set to Level 2 with the bt_conn_set_security(conn, BT_SECURITY_L2) call. If the device has no bonding information for the peer and isn’t already paired, then this function call initiates the pairing procedure. A new connection callback is security_changed(): this just prints the changed security level or an error message to the debug output. Warning: Don’t forget to add the security_changed() callback to the conn_callbacks structure. Then, in the bt_ready() function, you also load the settings. This allows the Bluetooth subsystem to load bonding information from storage, so bonding survives power loss. 51

51

Don’t forget to add #include in order to use the settings API.

● 172

Boek BLE 220329 UK.indd 172

06/05/2022 15:55

Chapter 5 • Securing BLE connections

Note: If you ever want to clear bonding information or other persistent data from your board, for the Nordic Semiconductor nRF52840 Development Kit you can run the command nr"prog -e. This program is part of the nRF Command Line Tools. You can also uncomment the line bt_unpair(BT_ID_DEFAULT, BT_ADDR_LE_ANY); after calling the bt_ready() function. This clears all bonding information after every reset for debugging purposes. Note that you also have to clear bonding information for the other paired device, such as your phone. In nRF Connect for Mobile, tap on the three dots at the right of the Connect button and choose Delete bond information.

5.4.2 Securely connecting to your sensor board If you build this firmware and flash it to your device, you can connect to it, for instance with nRF Connect for Mobile. This time the app will show a pairing request:

Figure 5.2 Android asks you if you want to pair your phone with the device. If you tap Pair, the device pairs and bonds with your mobile phone. On your serial connection, you should see something like this: Updated MTU: TX: 23 RX: 23 bytes Connected 6A:AC:09:D2:83:91 (random) Security changed: 6A:AC:09:D2:83:91 (random) level 2

Now, you can subscribe to indications in the nRF Connect app. It also shows Bonded with your device, even after you disconnect. The next connection should be faster because both devices are already bonded. You can also use this Python script with Bleak to connect to your sensor board and subscribe to its indications: """Subscribe to indications of a BME280 sensor in Security Mode 1 Level 2. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio import sys

● 173

Boek BLE 220329 UK.indd 173

06/05/2022 15:55

Bluetooth Low Energy Applications

from construct import Int16sl, Int16ul, Struct from construct.core import StreamError import bleak bme280_format = Struct( "temperature" / Int16sl, "pressure" / Int16ul, "humidity" / Int16ul, ) BME280_SENSOR_UUID = "63bf0b19-2b9c-473c-9e0a-2cfcaf03a771"

def sensor_value_changed(handle: int, data: bytearray): """Show sensor values for an indication.""" try: sensor_data = bme280_format.parse(data) print(f"Temperature: {sensor_data.temperature / 100} °C") : {sensor_data.humidity / 100} %")

print(f"Humidity print( f"Pressure

: {(sensor_data.pressure + 50000) / 100} hPa"

) print(24 * "-") except StreamError: # Wrong format pass

async def connect(address): """Connect and pair to BME280 sensor and subscribe to indications.""" try: async with bleak.BleakClient(address) as client: print(f"Connected to {address}") paired = await client.pair(protection_level=2) print(f"Paired: {paired}") await client.start_notify( BME280_SENSOR_UUID, sensor_value_changed ) print("Indications started...") while True: await asyncio.sleep(1)

● 174

Boek BLE 220329 UK.indd 174

06/05/2022 15:55

Chapter 5 • Securing BLE connections

except asyncio.exceptions.TimeoutError: print(f"Can’t connect to device {address}.")

if __name__ == "__main__": if len(sys.argv) == 2: address = sys.argv[1] asyncio.run(connect(address)) else: print("Please specify the BLE MAC address.")

This is actually the same script as in section 4.10.3, with one addition: paired = await client.pair(protection_level=2)

After the connection succeeds, this line of code tries to pair with the device before subscribing to its indications. How you confirm the pairing request depends on your operating system.

5.4.3 Sniffing the pairing procedure with Wireshark Nordic Semiconductor’s Infocenter shows you how to sniff a connection’s pairing procedure in various security configurations: https://infocenter.nordicsemi.com/index. jsp?topic=%2Fug_sniffer_ble%2FUG%2Fsniffer_ble%2Fintro.html. In this section, I’ll show you one configuration: LE Legacy Connections pairing using the Just Works pairing method. First, force your Zephyr code’s connection to LE Legacy Connections pairing, so you can see how insecure this is with your own eyes. Add CONFIG_BT_TINYCRYPT_ECC=n to prj. conf to disable LE Secure Connections. The code is already using the Just Works pairing method, so, after this change, nRF Sniffer for Bluetooth is able to sniff the pairing procedure without any further action. Just pair both devices and look at the decrypted messages in Wireshark.

● 175

Boek BLE 220329 UK.indd 175

06/05/2022 15:55

Bluetooth Low Energy Applications

Figure 5.3 If Wireshark is able to observe the pairing procedure of a device using LE Legacy Connections pairing with the Just Works pairing method, it can decrypt the whole encrypted connection. If the nRF Sniffer has successfully sniffed the pairing procedure of two BLE devices previously, it remembers the Long Term Key (LTK) needed to decrypt the connection. So, if you later start a new sniffing session in Wireshark, you can still sniff this connection. If you haven’t sniffed the pairing procedure but you know the Long Term Key (for instance by sniffing with Wireshark on another computer or by reading it from your device’s security database), click next to Key in the nRF Sniffer for Bluetooth LE toolbar and choose Legacy LTK or SC LTK. Then, type the key (in big-endian hexadecimal format) with a leading 0x into the Value field.

5.5 Authenticating a BLE connection The previous section used the Just Works pairing method, which doesn’t authenticate both devices. If you want to have an authenticated BLE connection, you need more input/output capabilities. For this section, I’m going to use the Nordic Semiconductor nRF52840 Development Kit. This development board has four buttons, but it doesn’t have a display. However, you can read output in your favorite terminal program over the serial connection. So, you can use the buttons for Yes/No input and the serial connection to show a 6-digit decimal number. If you look at Table 5.6, you see that the combination of a DisplayYesNo responder and a DisplayYesNo or KeyboardDisplay initiator makes the Numeric Comparison LE Secure Connection pairing method, which is the most secure pairing method of all.

● 176

Boek BLE 220329 UK.indd 176

06/05/2022 15:55

Chapter 5 • Securing BLE connections

5.5.1 Implementing Secure Connections Only Mode I’m going to configure the Zephyr firmware for the nRF52840 DK to use Secure Connections Only Mode: any device that wants to use the BLE services has to pair in Security Mode 1 Level 4. First, your CMakeLists.txt looks like this: # SPDX-License-Identifier: MIT cmake_minimum_required(VERSION 3.13.1) find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE}) project(peripheral_secure_only) target_sources(app PRIVATE src/service.c src/main.c) This adds both C files as target sources. And your prj.conf looks like this: # Enable Bluetooth peripheral CONFIG_BT=y CONFIG_BT_PERIPHERAL=y CONFIG_BT_DEBUG_LOG=y # Enable Secure Connections Only mode CONFIG_BT_SMP=y CONFIG_BT_TINYCRYPT_ECC=y CONFIG_BT_SMP_SC_ONLY=y # Use settings for BLE bonding CONFIG_BT_SETTINGS=y CONFIG_SETTINGS=y CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048 CONFIG_FLASH=y CONFIG_FLASH_PAGE_LAYOUT=y CONFIG_FLASH_MAP=y CONFIG_FCB=y # GATT services CONFIG_BT_GATT_DYNAMIC_DB=y CONFIG_BT_DIS=y CONFIG_BT_DIS_PNP=n CONFIG_BT_DEVICE_NAME="Secure Only" # Input Output capabilities CONFIG_CONSOLE=y CONFIG_GPIO=y

● 177

Boek BLE 220329 UK.indd 177

06/05/2022 15:55

Bluetooth Low Energy Applications

To enable Secure Connections, you need to enable CONFIG_BT_TINYCRYPT_ECC, and to force Secure Connections Only Mode you set CONFIG_BT_SMP_SC_ONLY to y. For the GATT services, CONFIG_BT_GATT_DYNAMIC_DB enables support for building a dynamic GATT database, which is a different way than was used in previous examples. And at the end of the configuration file, enable the console for output and GPIO for input (using the buttons). The custom service and two characteristics are defined in service.c: #include #include #include #include #include #include #include #include #include #include #include #include "service.h" // Custom service #define BT_UUID_SERVICE

\

BT_UUID_DECLARE_128(0x50, 0x22, 0xe5, 0xd4, 0x9c, 0xc1, 0x4e,

\

0x9e, 0xbe, 0x6e, 0xda, 0x13, 0x87, 0x87,

\

0x07, 0x77) // Characteristic to write to the peripheral device #define BT_UUID_INPUT

\

BT_UUID_DECLARE_128(0x51, 0x22, 0xe5, 0xd4, 0x9c, 0xc1, 0x4e,

\

0x9e, 0xbe, 0x6e, 0xda, 0x13, 0x87, 0x87,

\

0x07, 0x77) // Characteristic to read from the peripheral device #define BT_UUID_OUTPUT

\

BT_UUID_DECLARE_128(0x52, 0x22, 0xe5, 0xd4, 0x9c, 0xc1, 0x4e,

\

0x9e, 0xbe, 0x6e, 0xda, 0x13, 0x87, 0x87,

\

0x07, 0x77) static struct bt_gatt_attr attrs[] = { BT_GATT_PRIMARY_SERVICE(BT_UUID_SERVICE), BT_GATT_CHARACTERISTIC(BT_UUID_INPUT, BT_GATT_CHRC_WRITE, BT_GATT_PERM_WRITE_AUTHEN, NULL,

● 178

Boek BLE 220329 UK.indd 178

06/05/2022 15:55

Chapter 5 • Securing BLE connections

write_characteristic, NULL), BT_GATT_CHARACTERISTIC(BT_UUID_OUTPUT, BT_GATT_CHRC_READ, BT_GATT_PERM_READ_AUTHEN, read_characteristic, NULL, NULL), }; static struct bt_gatt_service service = BT_GATT_SERVICE(attrs); void service_init() { bt_gatt_service_register(&service); } uint8_t saved_number = 0; ssize_t write_characteristic(struct bt_conn *conn, const struct bt_gatt_attr *attr, const void *buf, uint16_t len, uint16_t offset, uint8_t flags) { const uint8_t *new_number = buf; if (!len) { return BT_GATT_ERR(BT_ATT_ERR_INVALID_ATTRIBUTE_LEN); } saved_number = *new_number; printk("Write characteristic: %d\n", saved_number); return len; } ssize_t read_characteristic(struct bt_conn *conn, const struct bt_gatt_attr *attr, void *buf, uint16_t len, uint16_t offset) { printk("Read characteristic\n"); return bt_gatt_attr_read(conn, attr, buf, len, offset, &saved_number, sizeof(saved_number)); }

In the attributes database, the first characteristic is defined with write permissions for authenticated pairing with encryption (BT_GATT_PERM_WRITE_AUTHEN). The second characteristic has read permissions for authenticated pairing with encryption (BT_GATT_ PERM_READ_AUTHEN). The former has a callback function, write_characteristic(), that’s called when a GATT client writes to the characteristic. The latter has a callback function, read_characteristic(), that’s called when a GATT client reads from the characteristic. Then, the service_init() function registers the service you defined previously. This is using the dynamic GATT database support that you enabled in prj.conf.

● 179

Boek BLE 220329 UK.indd 179

06/05/2022 15:55

Bluetooth Low Energy Applications

The write_characteristic() callback function, which is called when a GATT client writes to the first characteristic, stores the first byte of the written value to the saved_number variable. As an error-handling step, first ensure that the length of the written data isn’t zero. If it’s zero, return the BT_GATT_ERR macro with the ATT error code BT_ATT_ERR_INVALID_ATTRIBUTE_LEN. 52 The read_characteristic() function just reads the value in saved_number and returns it to the GATT client, which reads the corresponding characteristic. So, the function of both characteristics is that the client can write an 8bit number to one characteristic and read the same number later from the second characteristic. The corresponding header file with the function declarations looks like this: void service_init(); ssize_t write_characteristic(struct bt_conn *conn, const struct bt_gatt_attr *attr, const void *buf, uint16_t len, uint16_t offset, uint8_t flags); ssize_t read_characteristic(struct bt_conn *conn, const struct bt_gatt_attr *attr, void *buf, uint16_t len, uint16_t offset);

Then, in main.c, the core logic of the application is implemented: /* * Example of a custom BLE service using Secure Connections Only Mode. * * Copyright (c) 2021 Koen Vervloesem * * SPDX-License-Identifier: MIT * * Based on an example of the Bluetooth LE Security Guide. */ #include #include #include #include #include #include #include

52

You can find the full list of ATT error codes at https://docs.zephyrproject.org/apidoc/latest/att_8h.html or in zephyr/include/bluetooth/att.h in your local Zephyr directory.

● 180

Boek BLE 220329 UK.indd 180

06/05/2022 15:55

Chapter 5 • Securing BLE connections

#include #include #include #include #include #include #include "service.h" // Configure GPIO for buttons // Button Yes #define SW0_NODE DT_ALIAS(sw0) #if !DT_NODE_HAS_STATUS(SW0_NODE, okay) #error "Unsupported board: sw0 devicetree alias is not defined" #endif static const struct gpio_dt_spec button_yes = GPIO_DT_SPEC_GET_OR(SW0_NODE, gpios, {0}); // Button No #define SW1_NODE DT_ALIAS(sw1) #if !DT_NODE_HAS_STATUS(SW1_NODE, okay) #error "Unsupported board: sw1 devicetree alias is not defined" #endif static const struct gpio_dt_spec button_no = GPIO_DT_SPEC_GET_OR(SW1_NODE, gpios, {0}); static struct gpio_callback gpio_btn_yes_cb; static struct gpio_callback gpio_btn_no_cb; static struct k_work button_yes_work; static struct k_work button_no_work; #define DEVICE_NAME CONFIG_BT_DEVICE_NAME #define DEVICE_NAME_LEN (sizeof(DEVICE_NAME) - 1) bool authenticating = false; struct bt_conn *default_conn; static const struct bt_data ad[] = { BT_DATA_BYTES(BT_DATA_FLAGS, (BT_LE_AD_GENERAL | BT_LE_AD_NO_BREDR)), BT_DATA(BT_DATA_NAME_COMPLETE, DEVICE_NAME, DEVICE_NAME_LEN), }; static void pairing_passkey_display(struct bt_conn *conn,

● 181

Boek BLE 220329 UK.indd 181

06/05/2022 15:55

Bluetooth Low Energy Applications

unsigned int passkey) { char addr[BT_ADDR_LE_STR_LEN]; bt_addr_le_to_str(bt_conn_get_dst(conn), addr, sizeof(addr)); printk("Passkey for %s: %06u\n", addr, passkey); } static void auth_confirm(struct bt_conn *conn, unsigned int passkey) { default_conn = conn; char addr[BT_ADDR_LE_STR_LEN]; char passkey_str[7]; bt_addr_le_to_str(bt_conn_get_dst(conn), addr, sizeof(addr)); snprintk(passkey_str, 7, "%06u", passkey); printk("\nConfirm passkey for %s: %s\n\n", addr, passkey_str); printk("Press button 1 for YES or button 2 for NO\n"); authenticating = true; } static void auth_cancel(struct bt_conn *conn) { printk("Pairing cancelled\n"); authenticating = false; } static struct bt_conn_auth_cb pairing_cb_display = { .passkey_display = pairing_passkey_display, .passkey_confirm = auth_confirm, .cancel = auth_cancel, }; static void connected(struct bt_conn *conn, uint8_t err) { if (!err) { printk("Connected\n"); default_conn = bt_conn_ref(conn); if (bt_conn_set_security(default_conn, BT_SECURITY_L4)) { printk("Failed to set security\n"); } } } static void disconnected(struct bt_conn *conn, uint8_t reason) { if (default_conn) { printk("Disconnected\n"); bt_conn_unref(default_conn); default_conn = NULL; }

● 182

Boek BLE 220329 UK.indd 182

06/05/2022 15:55

Chapter 5 • Securing BLE connections

} static void security_level_changed(struct bt_conn *conn, bt_security_t level, enum bt_security_err err) { printk("Security level changed to %d\n", level); } static struct bt_conn_cb conn_callbacks = { .connected = connected, .disconnected = disconnected, .security_changed = security_level_changed, }; static void bt_ready(int err) { if (err) { return; } service_init(); if (IS_ENABLED(CONFIG_SETTINGS)) { settings_load(); } err = bt_le_adv_start(BT_LE_ADV_CONN, ad, ARRAY_SIZE(ad), NULL, 0); if (err) { return; } } void button_yes_work_handler(struct k_work *work) { if (!authenticating) { return; } printk("User indicated YES\n"); bt_conn_auth_passkey_confirm(default_conn); authenticating = false; } void button_no_work_handler(struct k_work *work) { if (!authenticating) { return; } printk("User indicated NO\n"); bt_conn_auth_cancel(default_conn);

● 183

Boek BLE 220329 UK.indd 183

06/05/2022 15:55

Bluetooth Low Energy Applications

authenticating = false; } void button_yes_pressed(const struct device *gpiob, struct gpio_callback *cb, uint32_t pins) { printk("Button Yes pressed\n"); if (!authenticating) { return; } k_work_submit(&button_yes_work); } void button_no_pressed(const struct device *gpiob, struct gpio_callback *cb, uint32_t pins) { printk("Button No pressed\n"); k_work_submit(&button_no_work); } void configure_buttons(void) { int ret; // Button Yes k_work_init(&button_yes_work, button_yes_work_handler); ret = gpio_pin_configure_dt(&button_yes, GPIO_INPUT); if (ret != 0) { printk("Error %d: failed to configure %s pin %d\n", ret, button_yes.port->name, button_yes.pin); return; } ret = gpio_pin_interrupt_configure_dt(&button_yes, GPIO_INT_EDGE_TO_ACTIVE); if (ret != 0) { printk("Error %d: failed to configure interrupt on %s pin %d\n", ret, button_yes.port->name, button_yes.pin); return; } gpio_init_callback(&gpio_btn_yes_cb, button_yes_pressed, BIT(button_yes.pin)); gpio_add_callback(button_yes.port, &gpio_btn_yes_cb); printk("Set up button at %s pin %d\n", button_yes.port->name, button_yes.pin); // Button No k_work_init(&button_no_work, button_no_work_handler);

● 184

Boek BLE 220329 UK.indd 184

06/05/2022 15:55

Chapter 5 • Securing BLE connections

ret = gpio_pin_configure_dt(&button_no, GPIO_INPUT); if (ret != 0) { printk("Error %d: failed to configure %s pin %d\n", ret, button_no.port->name, button_no.pin); return; } ret = gpio_pin_interrupt_configure_dt(&button_no, GPIO_INT_EDGE_TO_ACTIVE); if (ret != 0) { printk("Error %d: failed to configure interrupt on %s pin %d\n", ret, button_no.port->name, button_no.pin); return; } gpio_init_callback(&gpio_btn_no_cb, button_no_pressed, BIT(button_no.pin)); gpio_add_callback(button_no.port, &gpio_btn_no_cb); printk("Set up button at %s pin %d\n", button_no.port->name, button_no.pin); } void main(void) { printk("Starting application...\n"); int err; configure_buttons(); err = bt_enable(bt_ready); if (err) { return; } bt_conn_cb_register(&conn_callbacks); bt_conn_auth_cb_register(&pairing_cb_display); }

First, the GPIOs for the two buttons are defined. This only works if your board has two buttons with sw0 and sw1 aliases in their devicetree. For the nRF52840 DK, those buttons are Button 1 and Button 2, respectively. After this, the code declares a callback and work item for each button. Next come some callback functions for authentication. pairing_passkey_display() is called by the Bluetooth stack with a randomly generated 6-digit passkey. This is the point at which you display this number to the user. In this case, the number is shown to the console with printk(), but on a board with a display you should show the number on the display.

● 185

Boek BLE 220329 UK.indd 185

06/05/2022 15:55

Bluetooth Low Energy Applications

The auth_confirm() function is called by the Bluetooth stack when the user must confirm whether the passkeys displayed on both devices match. In this case, the address of the central is shown together with the generated passkey, and the user is asked to confirm or cancel, using Button 1 or 2. Then, auth_cancel() is called if the user indicates that the passkeys don’t match or if pairing is cancelled. Those three callbacks are specified in the pairing_cb_display struct. Next come three other callback functions, related to the connection. The connected() function is called when a central is connected. It sets the security level to 4 (with the BT_SECURITY_L4 constant), which triggers the pairing process. When the central disconnects, disconnected() is called, and, when the security level of the connection changes, security_level_changed() is called. Those three callbacks are specified in the conn_callbacks structure. Then, in bt_ready() the service is initialized and the settings with the security keys are loaded. The code then starts advertising. The advertising data has been defined in the ad structure previously, with the default flags and the device name. Then come a couple of functions related to the buttons. The button_yes_work_handler() function does what’s needed when the user has pressed the Yes button: it confirms the passkey with bt_conn_auth_passkey_confirm(default_conn). The button_no_work_ handler() function does what’s needed when the user has pressed the No button: it cancels the connection with bt_conn_auth_cancel(default_conn). button_yes_pressed() is called when the user presses the first button. It submits the correct work item to the system work queue. The same holds for button_no_pressed(), which concerns the second button. 53 Then, in configure_buttons() this is all linked together. The button_yes_work work structure is initialized, with button_yes_work_handler() as the work handler that’s called when the corresponding work item is processed. Then, the GPIO pin is configured as input, as well as its pin interrupts. The callback is added that’s called when the user presses a button. All of this is done first for the Yes button and then for the No button. With all of this set up, the main() function only has to configure the buttons, enable Bluetooth, register the connection callbacks and register the authentication callbacks.

5.5.2 Securely connecting with the board Build this firmware and flash it to your device (board name nrf52840dk_nrf52840 if you’re using the nRF52840 DK). Open a serial connection on your computer, for instance using screen: 54 screen /dev/ttyACM0 115200 53 54

Consult Zephyr’s documentation about workqueue threads if you want to know more about this feature: https://docs.zephyrproject.org/latest/reference/kernel/threads/workqueue.html See the appendix for more information about using the serial connection.

● 186

Boek BLE 220329 UK.indd 186

06/05/2022 15:55

Chapter 5 • Securing BLE connections

This will show the following messages from your device: *** Booting Zephyr OS build zephyr-v2.6.0-420-ge62c3c533a37

***

Starting application... Set up button at GPIO_0 pin 11 Set up button at GPIO_0 pin 12 [00:00:00.377,227] bt_hci_core: HW Platform: Nordic Semiconductor (0x0002) [00:00:00.377,227] bt_hci_core: HW Variant: nRF52x (0x0002) [00:00:00.377,258] bt_hci_core: Firmware: Standard Bluetooth controller (0x00) Version 2.6 Build 99 [00:00:00.377,563] bt_hci_core: No ID address. App must call settings_ load() [00:00:00.386,688] bt_hci_core: Identity: EE:12:12:76:12:46 (random) [00:00:00.386,688] bt_hci_core: HCI: version 5.2 (0x0b) revision 0x0000, manufacturer 0x05f1 [00:00:00.386,718] bt_hci_core: LMP: version 5.2 (0x0b) subver 0xffff

You see the messages you print in your application with printk, such as the setup of the buttons, as well as the Bluetooth stack’s debug messages. Next, open the nRF Connect app on your mobile phone and scan for devices. Click on Connect next to the device with the name Secure Only. This triggers a call of the connected() function, and it prints Connected to the console. Then the security level is set to level 4, which triggers a message on your phone that asks you if you want to pair with the device:

Figure 5.4 Android asks you if you want to pair your phone with the device. If you tap Pair, there’s a second message with the passkey:

Figure 5.5 Verify whether this passkey matches the one shown on the other device.

● 187

Boek BLE 220329 UK.indd 187

06/05/2022 15:55

Bluetooth Low Energy Applications

Now, look at the console of your nRF52840 DK. The auth_confirm() function is called, and it asks you to confirm the passkey: Confirm passkey for 71:51:B8:E0:8D:E2 (random): 414296 Press button 1 for YES or button 2 for NO

The passkeys in nRF Connect and on the console match, so you may safely press the Yes button on your nRF52840 DK and tap Pair on your phone. After this, in security_level_changed(), the security level is changed to level 4, and you can securely read and write the device’s characteristics: Button Yes pressed User indicated YES Security level changed to 4 Read characteristic Write characteristic: 1 Read characteristic Disconnected

On the last line of this console output, you see that I’ve disconnected from the device in nRF Connect. Now, when you reconnect to the device, you don’t need to confirm a passkey anymore. Thanks to the stored bonding information, reconnection happens immediately, even after a power loss on your device: Connected Security level changed to 4

Note: Try and see what happens when you press Button 2 (NO) on the nRF52840 DK after you’re asked to confirm the passkey.

5.5.3 Sniffing the pairing procedure with Wireshark If you sniff the pairing procedure for this device in Wireshark, you’ll see that the initiator (your phone) sends a Pairing Request, and the responder (your nRF52840 device) sends a Pairing Response. In these packets, both devices state their I/O capabilities: Keyboard, Display for the initiator and Display Yes/No for the responder.

● 188

Boek BLE 220329 UK.indd 188

06/05/2022 15:55

Chapter 5 • Securing BLE connections

Figure 5.6 Sniffing the Pairing Request and Pairing Response packets during the pairing procedure is a great way to debug pairing issues. Then you see both devices exchanging their ECDH public key, which they use to generate the long-term key (LTK). After this, the connection is encrypted with the LTK, but, because Wireshark doesn’t know this key, it can’t decrypt the packets. Note: You can add the line CONFIG_BT_DEBUG_SMP=y to your prj.conf to enable debug support for the Bluetooth Security Manager Protocol (SMP). This option prints out private security keys such as the Long Term Key, which you can use in Wireshark to decrypt the connection’s packets.

5.6 Privacy While most of this chapter was about encryption and authentication, privacy is another important concept in security. As you’ve seen in Chapter 3, every BLE device advertises its existence with advertisement packets containing its address. If this is the device’s public address, anyone in the neighborhood is able to track your location, just by scanning for this address. To thwart user tracking, the BLE specification offers private addresses (see section 3.4). These are addresses that change periodically, for instance once every fifteen minutes. Every modern phone operating system uses private addresses to prevent others from tracking you by using your phone’s Bluetooth address.

● 189

Boek BLE 220329 UK.indd 189

06/05/2022 15:55

Bluetooth Low Energy Applications

A special type of private address is the resolvable private address (RPA). This is an address that can’t be used by anyone to track the device, except by one or more trusted devices. Only devices that are paired with the device have the required IRK (Identity Resolving Key) to "resolve" the identity of the device – its public address. Other devices only see an address that changes every fifteen minutes. 55 But, without the IRK, you can’t be sure that the device on time T is the same as the device on time T + 15 minutes, because the addresses seem random. Zephyr makes it quite simple to use a resolvable private address in your peripheral. Just add the following build configuration properties in your project’s prj.conf: CONFIG_BT_PRIVACY=y CONFIG_BT_RPA_TIMEOUT=60

The default value of CONFIG_BT_RPA_TIMEOUT is 900 (measured in seconds), or 15 minutes. However, to test this feature, set it to 60 seconds. Now, build and flash your code, and scan for your device in nRF Connect. You’ll see a resolvable private address next to your device. If you rescan a minute later, you’ll see that the device has another address. Now, connect your phone to the device and pair. Now, if you wait a minute, the address shown next to the bonded device doesn’t change. If you disconnect from the peripheral and do a scan on another phone or computer, you’ll see another random address next to the device. This still changes every minute. Meanwhile, your bonded phone keeps showing the same address. This shows that, thanks to the bonding, your phone can resolve the peripheral’s address and identify it, while other centrals aren’t able to.

5.7 Summary and further exploration In this chapter I explained BLE’s security architecture and the procedures for pairing and bonding. You learned about LE Legacy Connection pairing and LE Secure Connection pairing, as well as their respective pairing methods. You also learned about the different security modes and security levels that BLE offers. After this theoretical foundation, you learned how to encrypt and authenticate BLE connections in Zephyr, as well as how to use a resolvable private address in Zephyr to prevent your device from being tracked. You also learned how to sniff two BLE devices’ pairing procedure to decrypt the entire encrypted connection for debugging purposes. Note: The code examples in this chapter focused on Zephyr. In section 6.6 in the next chapter, you’ll also see an example of an Arduino-NimBLE program that uses BLE security.

55

This is just an example; the device can choose the interval, but 15 minutes is a common choice.

● 190

Boek BLE 220329 UK.indd 190

06/05/2022 15:55

Chapter 5 • Securing BLE connections

If you want to read more details about the BLE pairing and bonding procedures, have a look at the five-part blog series that the Bluetooth SIG published about this topic: https:// www.bluetooth.com/blog/bluetooth-pairing-part-1-pairing-feature-exchange/. And, if you want a deep-dive into BLE security, download the Bluetooth SIG’s Bluetooth LE Security Guide (https://www.bluetooth.com/bluetooth-resources/le-security-study-guide/), which includes Zephyr code for some example applications. Of course, you should always consult the Bluetooth Core Specification for precise information about Bluetooth security features.

● 191

Boek BLE 220329 UK.indd 191

06/05/2022 15:55

Bluetooth Low Energy Applications

Chapter 6 • Profiles and roles In the previous chapters, you used various services and characteristics in the code examples. However, these don’t stand on their own. BLE defines another concept: profiles. A profile is a specification that describes how two or more devices with one or more services communicate with each other. Each of these devices implements a specific role. For instance, the Proximity profile enables two devices to monitor their proximity. This is done by detecting a dropped connection or a path loss above a specified level. The Proximity profile defines two roles: the Proximity Reporter that has a Link Loss service, and the Proximity Monitor that connects to the Proximity Reporter and sets the alert level by writing its Link Loss service’s Alert Level characteristic. In this chapter, you’ll learn how to understand a BLE service specification and a BLE profile specification, with the Proximity profile as an example. This will teach you how to go from reading these specifications to implementing them in your own devices. You’ll also implement a Proximity Reporter in Zephyr and a Proximity Monitor in NimBLE-Arduino.

6.1 Common BLE profiles Profiles are fundamental to Bluetooth Low Energy. There are two types: generic profiles and GATT profiles.

6.1.1 Generic profiles The BLE specification defines two generic profiles, and you’ve already used them a lot in this book: Generic Access Profile (GAP) This profile defines how BLE devices can discover other devices, connect to them and bond to them. It’s a mandatory profile for all BLE devices. Generic Attribute Profile (GATT) This profile defines how to discover services and their characteristics and how to read and write their values.

6.1.2 GATT profiles While the generic profiles are defined in the BLE specification, there are other profiles that build upon the GATT profile. Each of these profiles is specific to a use case, which is defined in its own specification document. You can find these documents at https://www.bluetooth. com/specifications/specs/. Some interesting GATT profiles are: Blood Pressure Profile Collects data from a blood pressure sensor

● 192

Boek BLE 220329 UK.indd 192

06/05/2022 15:55

Chapter 6 • Profiles and roles

Environmental Sensing Profile Collects data, such as temperature and humidity, from an environmental sensor Heart Rate Profile Collects data from a heart rate sensor (e.g. a fitness tracker) HID over GATT Profile Connects a keyboard, mouse or remote control using HID (Human Interface Device) services Mesh Profile Implements a mesh network for Bluetooth Low Energy, Bluetooth Mesh Phone Alert Status Profile Alerts the user of a companion device, such as a smartwatch, about the alert status of a connected phone Proximity Profile Monitors the proximity of another device and sounds an alert when it’s too far away Time Profile Gets and sets a device’s date and time Weight Scale Profile Collects weight data from a scale

6.2 Understanding a profile specification If you want to implement the Proximity profile, search for the term ‘proximity’ from https:// www.bluetooth.com/specifications/specs/. You’ll find the Proximity Profile 1.0.1 specification, adopted on July 14, 2015. Click on the link to download the specification as a PDF.

● 193

Boek BLE 220329 UK.indd 193

06/05/2022 15:55

Bluetooth Low Energy Applications

Figure 6.1 The first step in implementing a BLE profile is searching its specification. Every profile specification has roughly the same structure. As an example, the Proximity Profile specification has the following sections: • • • • • • • • •

Introduction Configuration Proximity Reporter Requirements Proximity Monitor Requirements Connection Establishment Security Considerations GATT Interoperability Requirements Acronyms and Abbreviations References

● 194

Boek BLE 220329 UK.indd 194

06/05/2022 15:55

Chapter 6 • Profiles and roles

Let’s have a look at each of these sections for the Proximity Profile specification. I recommend reading the specification section by section while reading each section’s explanation on the following few pages.

6.2.1 Introduction The introduction explains what behavior the profile defines. For the Proximity Profile specification: • The Proximity profile defines the behavior when a device moves away from a peer device so that the connection is dropped or the path loss increases above a preset level, causing an immediate alert. This alert can be used to notify the user that the devices have become separated. As a consequence of this alert, a device may take further action, for example to lock one of the devices so that it’s no longer usable. • The Proximity profile can also be used to define the behavior when the two devices come closer together such that a connection is made, or the path loss decreases below a preset level. After this, the introduction explains that the profile requires the Generic Attribute Profile (GATT), so you know that it’s a GATT profile. There are no further requirements for the Bluetooth version: the specification is compatible with any Bluetooth version that includes the Generic Attribute Profile. The introduction also has a subsection about conformance: what you need to make sure of if you want to verify your device as part of the Bluetooth qualification program. Note: Even if you don’t have any plans for Bluetooth qualification because you’re only developing devices or applications for your own personal hobby use, you should consider all of the requirements listed in the specification. This improves compatibility with other devices.

6.2.2 Configuration This section lists the roles defined by the profile, as well as each role’s services. The Proximity profile defines two roles: Proximity Reporter A GATT server and GAP peripheral Proximity Monitor A GATT client and GAP central The Proximity Reporter has three services: • Link Loss Service (mandatory) • Immediate Alert Service (optional) • Tx Power Service (optional)

● 195

Boek BLE 220329 UK.indd 195

06/05/2022 15:55

Bluetooth Low Energy Applications

The specification also allows you to implement the Proximity Monitor role or the Proximity Reporter role, together with other profiles. Your device can even implement the Proximity Monitor and Proximity Reporter roles at the same time. The section also lists requirements for the connectable mode, discoverability, and advertising.

6.2.3 Proximity Reporter Requirements This section lists the requirements for the device implementing the Proximity Reporter role. In this case, the requirements are simple: • The Proximity Reporter shall have only one instance of the Link Loss service and may have one instance of both the Immediate Alert service and the Tx Power service. This is also represented in a table: Service

Proximity Reporter

Link Loss service

M

Immediate Alert service

C1

Tx Power service

C1

Table 6.1 Service requirements for the Proximity Reporter role The meanings of the codes in the second column are: M Mandatory C1 If a device exposes only one of the Immediate Alert or Tx Power services, then neither the Immediate Alert service nor the Tx Power service shall be used in this profile. C1 is a subtle requirement: what this means is that a device isn’t allowed to implement only the Immediate Alert service or only the Tx Power service beyond the mandatory Link Loss service. There are only two cases allowed: you implement just the Link Loss service, or you implement all three services. For each of these three services, the profile doesn’t impose any requirements beyond those defined in the service specification. Note: This section of the profile specification refers to the service specification documents, which you’ll have to read later to implement these services. You’ll see how to understand a service specification in section 6.3.

● 196

Boek BLE 220329 UK.indd 196

06/05/2022 15:55

Chapter 6 • Profiles and roles

6.2.4 Proximity Monitor Requirements The next section lists the requirements for the device implementing the Proximity Monitor role. It starts with the same table as for the Proximity Reporter role: Service

Proximity Reporter

Link Loss service

M

Immediate Alert service

C1

Tx Power service

C1

Table 6.2 Service requirements for the Proximity Monitor role The meaning of the codes in the second column is: M Mandatory C1 If a device supports only either the Immediate Alert or Tx Power services, then neither the Immediate Alert service nor the Tx Power service shall be used in this profile. The meaning of C1 is the same as for the Proximity Reporter. There are only two cases allowed for the Proximity Monitor: you support just the Link Loss service or you support all three services. After this, the listed requirements are a bit more complex than for the Proximity Reporter role. The section explains the requirements for the following procedures: Procedure

Support

Service discovery

M

Characteristic discovery

M

Configuration of alert on link

M

loss Alert on link loss

M

Reading Tx power

O

Alert on path loss

O

Table 6.3 Procedure requirements for the Proximity Monitor role The meanings of the codes in the second column are: M Mandatory O Optional

● 197

Boek BLE 220329 UK.indd 197

06/05/2022 15:55

Bluetooth Low Energy Applications

For each of these procedures, the requirements are listed in detail. For instance, for service discovery, this section explains that there are two ways to do this: • using the GATT Discover All Primary Services sub-procedure • using the GATT Discover Primary Services by Service UUID sub-procedure These procedures are defined in the Bluetooth specification, but, in practice, you’ll use your preferred BLE library’s API. 56 If you do it the second way, you’ll need to specify the service UUID. This is where working with a BLE profile specification becomes a bit cumbersome. The UUIDs’ numeric values are never shown in a profile specification. They’re always listed symbolically, for instance as "Link Loss," "Immediate Alert," and "Tx Power." To find these numeric values, consult the 16bit UUID Numbers Document (see the appendix at the end of this book for the URL). Search for one of the service names in the document, and you’ll find the following table fragment: Allocation type

Allocated UUID

Allocated for

GATT Service

0x1802

Immediate Alert

GATT Service

0x1803

Link Loss

GATT Service

0x1804

Tx Power

Table 6.4 Service UUIDs used by the Proximity Monitor for service discovery In the same vein, the requirements for the other procedures are explained, such as how to discover characteristics, when to write the Alert Level characteristic in the Link Loss service, and when to alert.

6.2.5 Connection Establishment This section of the profile specification describes how the connection between Proximity Reporter and Proximity Monitor is established. These are detailed low-level requirements and recommendations, such as the GAP mode, advertising interval, connection interval/connection latency, and scan interval/scan window. It’s important to follow these recommendations for maximum compatibility with other devices implementing this profile. How you do this depends on the BLE framework you’re using: • With Bleak, you don’t have control over these low-level connection parameters

56

For instance, the GATT Discover All Primary Services sub-procedure is used by calling the get_services() method on a BleakClient object with Bleak, or by calling a NimBLEClient object’s getServices() member function using NimBLE-Arduino.

● 198

Boek BLE 220329 UK.indd 198

06/05/2022 15:55

Chapter 6 • Profiles and roles

• With NimBLE-Arduino, the connection parameters are set using the NimBLEClient object’s ConnectionParams() member function, and the scan interval and scan window with its setInterval() and setWindow() member functions • For Zephyr, connection management is extensively documented at https://docs. zephyrproject.org/latest/reference/bluetooth/connection_mgmt.html

6.2.6 Security Considerations This section lists the profile’s security requirements. The most important one is that both devices support Security Mode 1 Level 2 (unauthenticated pairing with encryption) or Security Mode 1 Level 3 (authenticated pairing with encryption). See section 5.3 for the details about these security levels. There’s also a note about possible attacks. The section advises against using the Proximity profile as the only protection of valuable assets.

6.2.7 GATT Interoperability Requirements This section lists the required GATT sub-procedures for both the Proximity Monitor and Proximity Reporter roles: GATT sub-procedure

Proximity Monitor

Proximity Reporter

Discover All Primary

C.1

M

C.1

M

M

M

N/A

M

O

M

Read Characteristic Value

M

M

Write Characteristic

M

M

Services Discover Primary Services by Service UUID Discover All Characteristics of a Service Discover Characteristic by UUID Discover All Characteristic Descriptors

Value

Table 6.5 Required GATT sub-procedures for the Proximity Monitor and Proximity Reporter roles The meanings of the codes are: M Mandatory O Optional

● 199

Boek BLE 220329 UK.indd 199

06/05/2022 15:55

Bluetooth Low Energy Applications

C.1 Either Discover All Primary Services or Discover Primary Services by Service UUID shall be supported by the Proximity Monitor N/A Not applicable I already explained the two service discovery methods in subsection 6.2.4.

6.2.8 Acronyms and Abbreviations This section lists a table of acronyms and abbreviations used in the profile specification.

6.2.9 References This section lists some references to other specification documents, such as the Bluetooth Core Specification, the service specifications, and the Bluetooth SIG Assigned Numbers document.

6.3 Understanding a service specification Now that you understand the Proximity profile, it’s time to investigate its services. Each service is explained in its own service specification document, which you can also find at https://www.bluetooth.com/specifications/specs/. As an example, let’s investigate Link Loss Service 1.0.1, adopted on July 14, 2015. Every service specification has roughly the same structure. The Link Loss Service specification has the following sections: • • • • • •

Introduction Service Declaration Service Characteristics Service Behaviors Acronyms and Abbreviations References

Let’s have a look at all of these sections for the Link Loss Service specification. I recommend reading the specification section by section while reading the sections’ explanations on the next few pages.

6.3.1 Introduction The introduction explains what this service does. For the Link Loss Service specification, this is: • The Link Loss Service uses the Alert Level characteristic (as defined in [2]) to cause an alert in the device when the link is lost. The introduction has a subsection about conformance: what you need to make sure of if you want to verify your device as part of the Bluetooth qualification program.

● 200

Boek BLE 220329 UK.indd 200

06/05/2022 15:55

Chapter 6 • Profiles and roles

Note: Even if you don’t have any plans for Bluetooth qualification because you’re only developing devices or applications for your own personal use, you should consider all of the requirements listed in the specification. This improves compatibility with other devices. After this, the introduction lists: • upon which other GATT services this service depends (none in this case) • which Bluetooth version the service requires (it’s compatible with any Bluetooth version that includes the Generic Attribute Profile) • over which transport it operates (only Bluetooth Low Energy, no Classic Bluetooth) • which application error codes the service defines (none in this case) • which GATT sub-procedures it requires, beyond those required by GATT (in this case Write Characteristic Value)

6.3.2 Service Declaration According to this section, the Link Loss Service shall be instantiated as a Primary Service. 57 It also mandates that there’s only one instance of the service on a device. The service’s UUID is defined as "Link Loss." The UUIDs’ numeric values are never listed in BLE service specification documents. So, you’ll have to search the UUID for the Link Loss service in the 16bit UUID Numbers Document. Searching for the service’s name results in the following table fragment: Allocation type

Allocated UUID

Allocated for

GATT Service

0x1803

Link Loss

Table 6.6 Link Loss Service’s UUID

6.3.3 Service Characteristics This section defines the service’s characteristics. In this case, there is just one, Alert Level, which is indicated as mandatory. Moreover, the Link Loss Service shall have only one instance of the Alert Level characteristic. The UUIDs’ numeric values are never listed in BLE service specification documents. So, you’ll have to search the UUID for the Alert Level characteristic in the 16bit UUID Numbers Document. Searching for the characteristic’s name results in the following table fragment: Allocation type

Allocated UUID

Allocated for

GATT Characteristic and Object Type

0x2a06

Alert Level

Table 6.7 Alert Level characteristic’s UUID

57

See subsection 4.3.1 for the difference between a primary and a secondary service.

● 201

Boek BLE 220329 UK.indd 201

06/05/2022 15:55

Bluetooth Low Energy Applications

The section also lists a table containing the properties with which the characteristic has to comply. In this case, being able to read and write the characteristic is mandatory, while other properties, such as broadcasting and notifications, aren’t permitted. The service also doesn’t impose any security requirements for this characteristic. After these technical requirements, the section explains what the characteristic does: • The Alert Level characteristic is used to expose the current link loss alert level, which is used to determine how the device alerts when the link is lost. Finally, the section explains the behavior of the characteristic when used with the following GATT sub-procedures: GATT Characteristic Read Value Returns the current link loss alert level GATT Write Characteristic Value Sets the current link loss alert level to either No Alert, Mild Alert, or High Alert This is another case where a BLE service specification document can be confusing: what are these alert levels’ numeric values? That’s where you need to consult yet another specification document, the GATT Specification Supplement, which you can find at https://www.bluetooth.com/specifications/specs/ gatt-specification-supplement-5/. I’ll defer this to section 6.4.

6.3.4 Service Behaviors This section defines the service’s behavior. In this case, the Link Loss Service is all about what to do when the connection between devices is lost, so there’s just one subsection, Disconnection Behavior. The behavior is defined as: • When this service is instantiated in a device and the connection is lost without any prior warning, the device shall start alerting to the current link loss alert level. However, if the connection is terminated using a link layer procedure, the device shall not alert and shall ignore the current link loss alert level. This is made more concrete for the three possible alert levels: No Alert The device doesn’t alert Mild Alert The device alerts

● 202

Boek BLE 220329 UK.indd 202

06/05/2022 15:55

Chapter 6 • Profiles and roles

High Alert The device alerts in the strongest possible way The specific action for the mild and high alerts is left to the implementation. This could be by making noises, flashing lights, or any other method that alerts the user. This section also defines the conditions met to stop alerts: • after an implementation-specific timeout • after user interaction on the device • when the physical link is reconnected

6.3.5 Acronyms and Abbreviations This section lists a table of acronyms and abbreviations used in the service specification.

6.3.6 References This section lists some references to other specification documents, such as the Bluetooth Core Specification and the Bluetooth SIG Assigned Numbers document.

6.4 Understanding the definition of a characteristic The only thing you’ve not yet seen is the full definition of the Alert Level characteristic. All GATT characteristics’ normative definitions are listed in the GATT Specification Supplement (https://www.bluetooth.com/specifications/specs/gatt-specification-supplement-5/), except for those defined in the Bluetooth Core Specification or in Bluetooth Service specifications. Most of these definitions are quite short because the behavior of the characteristic in a service is already explained in the service specification. The definition in the GATT Specification Supplement just lists the concrete structure of the data, as well as the permitted numeric values. Every characteristic definition has the same structure: • Description • Definition Let’s have a look at these sections for the Alert Level characteristic:

6.4.1 Description This section has a brief explanation of what the characteristic does: • The Alert Level characteristic is used to specify the degree of alerting for a device.

● 203

Boek BLE 220329 UK.indd 203

06/05/2022 15:55

Bluetooth Low Energy Applications

6.4.2 Definition This section defines the structure of the data read from or written to the characteristic. This is shown as a table of ‘fields.’ Every field has a specific data type, such as an 8bit or a 16bit unsigned integer, a bit field, or other special kind of structure. In this case, the Alert Level characteristic has just one field, the Alert Level. Its data type is listed as uint8 (an 8bit unsigned integer), with size 1 (in bytes). After this, the permitted values are listed for each field. In this case, there’s just the Alert Level field, and the table lists the following values: Description

Value

No Alert

0x00

Mild Alert

0x01

High Alert

0x02

Reserved for Future Use

0x03-0xff

Table 6.8 Permitted values for the Alert Level field So, there you have the different alert levels’ numeric values, which you’ll have to use in your application.

6.5 Implementing a Proximity Reporter in Zephyr You’ve read the specification documents for the Proximity Profile and the Link Loss Service. You’ve supplemented these with the numeric values from the 16bit UUID Numbers Document and the GATT Specification Supplement. So, you’ve gained all of the knowledge you need to implement both the Proximity Reporter and Proximity Monitor roles in your own devices. In this section, I’ll implement a Proximity Reporter in Zephyr. Note: This chapter shows minimal implementations of the Proximity Reporter and Proximity Monitor roles. They implement only the mandatory Link Loss service and not the optional Immediate Alert and Tx Power services. Those services can be left as an exercise to the reader. The prj.conf file looks as follows: # Enable Bluetooth peripheral CONFIG_BT=y CONFIG_BT_PERIPHERAL=y # Enable Bluetooth Security Manager Protocol CONFIG_BT_SMP=y

● 204

Boek BLE 220329 UK.indd 204

06/05/2022 15:55

Chapter 6 • Profiles and roles

# GAP attributes CONFIG_BT_DEVICE_NAME="Proximity Reporter" # Appearance: Generic Keyring CONFIG_BT_DEVICE_APPEARANCE=576 # Use settings for BLE bonding CONFIG_BT_SETTINGS=y CONFIG_SETTINGS=y CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=2048

And the C source code: /* * Minimal Proximity Reporter implementing the Link Loss service. * * Copyright (c) 2021 Koen Vervloesem * * SPDX-License-Identifier: MIT */ #include #include #include #include #include #include #include #include #include #include #include #include #include uint8_t alert_level = 0; void alert_on_link_loss() { switch (alert_level) { case 1: printk("Mild Alert\n"); break; case 2: printk("High Alert\n"); break;

● 205

Boek BLE 220329 UK.indd 205

06/05/2022 15:55

Bluetooth Low Energy Applications

} } // Callback function for reading characteristic static ssize_t read_alert_level(struct bt_conn *conn, const struct bt_gatt_attr *attr, void *buf, uint16_t len, uint16_t offset) { uint8_t level = alert_level; return bt_gatt_attr_read(conn, attr, buf, len, offset, &level, sizeof(level)); } // Callback function for writing characteristic static ssize_t write_alert_level(struct bt_conn *conn, const struct bt_gatt_attr *attr, const void *buf, uint16_t len, uint16_t offset, uint8_t flags) { uint8_t *value = attr->user_data; alert_level = *value; memcpy(value, buf, len); return len; } // Primary Service Declaration BT_GATT_SERVICE_DEFINE( service_lls, BT_GATT_PRIMARY_SERVICE(BT_UUID_LLS), BT_GATT_CHARACTERISTIC( BT_UUID_ALERT_LEVEL, BT_GATT_CHRC_READ | BT_GATT_CHRC_WRITE, BT_GATT_PERM_READ_ENCRYPT | BT_GATT_PERM_WRITE_ENCRYPT, read_alert_level, write_alert_level, &alert_level), ); // Advertising data static const struct bt_data ad[] = { BT_DATA_BYTES(BT_DATA_FLAGS, (BT_LE_AD_GENERAL | BT_LE_AD_NO_BREDR)), BT_DATA_BYTES(BT_DATA_UUID16_ALL, BT_UUID_16_ENCODE(BT_UUID_LLS_VAL)), }; // GATT callbacks void mtu_updated(struct bt_conn *conn, uint16_t tx, uint16_t rx) { printk("Updated MTU: TX: %d RX: %d bytes\n", tx, rx); }

● 206

Boek BLE 220329 UK.indd 206

06/05/2022 15:55

Chapter 6 • Profiles and roles

static struct bt_gatt_cb gatt_callbacks = {.att_mtu_updated = mtu_updated}; // Connection callbacks static void connected(struct bt_conn *conn, uint8_t err) { char addr[BT_ADDR_LE_STR_LEN]; bt_addr_le_to_str(bt_conn_get_dst(conn), addr, sizeof(addr)); if (err) { printk("Failed to connect to %s (%u)\n", addr, err); return; } printk("Connected %s\n", addr); // Use unauthenticated pairing with encryption if (bt_conn_set_security(conn, BT_SECURITY_L2)) { printk("Failed to set security\n"); } } static void disconnected(struct bt_conn *conn, uint8_t reason) { char addr[BT_ADDR_LE_STR_LEN]; bt_addr_le_to_str(bt_conn_get_dst(conn), addr, sizeof(addr)); printk("Disconnected from %s (reason 0x%02x)\n", addr, reason); alert_on_link_loss(); } static void security_changed(struct bt_conn *conn, bt_security_t level, enum bt_security_err err) { char addr[BT_ADDR_LE_STR_LEN]; bt_addr_le_to_str(bt_conn_get_dst(conn), addr, sizeof(addr)); if (!err) { printk("Security changed: %s level %u\n", addr, level); } else { printk("Security failed: %s level %u err %d\n", addr, level, err); } } static struct bt_conn_cb conn_callbacks = {

● 207

Boek BLE 220329 UK.indd 207

06/05/2022 15:55

Bluetooth Low Energy Applications

.connected = connected, .disconnected = disconnected, .security_changed = security_changed, }; static void bt_ready(void) { int err; printk("Bluetooth initialized\n"); if (IS_ENABLED(CONFIG_SETTINGS)) { settings_load(); } err = bt_le_adv_start(BT_LE_ADV_CONN_NAME, ad, ARRAY_SIZE(ad), NULL, 0); if (err) { printk("Advertising failed to start (err %d)\n", err); return; } printk("Advertising successfully started\n"); } void main(void) { int err; printk("Starting firmware...\n"); // Initialize the Bluetooth subsystem err = bt_enable(NULL); if (err) { printk("Bluetooth init failed (err %d)\n", err); return; } bt_ready(); // Clear all bonds for debugging purposes // bt_unpair(BT_ID_DEFAULT, BT_ADDR_LE_ANY); // Register GATT and connection callbacks bt_gatt_cb_register(&gatt_callbacks); bt_conn_cb_register(&conn_callbacks); }

● 208

Boek BLE 220329 UK.indd 208

06/05/2022 15:55

Chapter 6 • Profiles and roles

In the beginning, the alert level is defined as an 8bit unsigned integer. In practice, you should implement the different alert levels using a piezoelectric buzzer or an LED. In this case, I simply implemented it by calling the printk() function, which shows the alert on the console. After this, there are two callback functions: one called when the central reads the alert level characteristic, and one called when the central writes the alert level characteristic. The former reads the alert_level variable and returns its value, while the latter writes the value that the central writes to the attribute into the alert_level variable. Note: For simplicity’s sake, there’s no error handling here to handle the writing of invalid values. Then comes the primary service declaration for the Link Loss Service. Its UUID is defined as BT_UUID_LLS in bluetooth/uuid.h. The Alert Level characteristic is defined with the UUID BT_UUID_ALERT_LEVEL, defined in the same header file. The declaration lists the characteristic as readable and writable, with encrypt permissions; the characteristic is only accessible in an encrypted connection. The declaration also refers to the read_alert_level() and write_alert_level() callback functions and to the alert_level variable. Then ,the advertising data structure is defined. The Link Loss service’s 16bit UUID is defined as BT_UUID_16_ENCODE(BT_UUID_LLS_VAL). In the connection() callback, the security mode is set to Level 2. In the disconnection() callback, the device alerts. The rest of the code is the standard code to initialize Bluetooth, register the callbacks, and start advertising. After flashing this to an nRF52840 DK, open a serial connection, for instance using screen:58 screen /dev/ttyACM0 115200

Then connect to the Proximity Reporter, for instance with the nRF Connect mobile app. Write an alert level 1 or 2 to the Link Loss service’s Alert Level characteristic, and move away from the Proximity Reporter with your phone. When the device disconnects, your Proximity Reporter shows an alert on the serial console. Note: There are a lot of changes needed to make this a conforming Proximity Reporter. For instance, in the disconnected() callback function, you should check the reason for the disconnection and only alert when the connection is lost without any warning. You can find the HCI error codes in the zephyr/include/bluetooth/hci_err.h header file. Another improvement is to add some error handling for the writing of the characteristic value.

58

See the appendix for more information about using the serial connection.

● 209

Boek BLE 220329 UK.indd 209

06/05/2022 15:55

Bluetooth Low Energy Applications

6.6 Implementing a Proximity Monitor in NimBLE-Arduino This section implements a minimal Proximity Monitor role with NimBLE-Arduino. The code goes as follows: /* Minimal implementation of the Proximity Monitor role of the * Proximity Profile. * * This implementation supports the Link Loss service of a Proximity * Reporter. * * Copyright (C) 2021 Koen Vervloesem ([email protected]) * * SPDX-License-Identifier: MIT * * Based on the NimBLE_Client example from H2zero */ #include #define UUID_LINK_LOSS_SERVICE "1803" #define UUID_ALERT_LEVEL_CHARACTERISTIC "2a06" static NimBLEAdvertisedDevice *advDevice; NimBLERemoteService *pSvc = nullptr; NimBLERemoteCharacteristic *pChr = nullptr; static bool doConnect = false; static uint32_t scanTime = 0; // 0 = scan forever static uint8_t alert_level = 1; // Default value is Mild Alert void alert_on_link_loss() { switch (alert_level) { case 1: Serial.println("Mild Alert"); break; case 2: Serial.println("High Alert"); break; } } class ClientCallbacks : public NimBLEClientCallbacks { void onConnect(NimBLEClient *pClient) { Serial.println("Connected"); pClient->updateConnParams(120, 120, 0, 60); }

● 210

Boek BLE 220329 UK.indd 210

06/05/2022 15:55

Chapter 6 • Profiles and roles

void onDisconnect(NimBLEClient *pClient) { Serial.print(pClient->getPeerAddress().toString().c_str()); Serial.println( "Disconnected - Alerting on link loss and starting scan"); alert_on_link_loss(); NimBLEDevice::getScan()->start(scanTime, nullptr); } /* Called when the peripheral requests a change to the connection * parameters. * Return true to accept and apply them or false to reject and keep * the currently used parameters. Default will return true. */ bool onConnParamsUpdateRequest(NimBLEClient *pClient, const ble_gap_upd_params *params) { if (params->itvl_min < 24) { // 1.25ms units return false; } else if (params->itvl_max > 40) { // 1.25ms units return false; } else if (params->latency > 2) { // Intervals allowed to skip return false; } else if (params->supervision_timeout > 100) { // 10ms units return false; } return true; } }; /* Define a class to handle the callbacks when advertisements are * received. */ class AdvertisedDeviceCallbacks : public NimBLEAdvertisedDeviceCallbacks { void onResult(NimBLEAdvertisedDevice *advertisedDevice) { if (advertisedDevice->isAdvertisingService( NimBLEUUID(UUID_LINK_LOSS_SERVICE))) { Serial.print("Advertised Device found: "); Serial.println(advertisedDevice->toString().c_str()); Serial.println("Found Link Loss Service"); // Stop scan before connecting NimBLEDevice::getScan()->stop(); // Save the device reference in a global for the client to use

● 211

Boek BLE 220329 UK.indd 211

06/05/2022 15:55

Bluetooth Low Energy Applications

advDevice = advertisedDevice; // Ready to connect now doConnect = true; } } }; /* Create a single global instance of the callback class to be used by * all clients. */ static ClientCallbacks clientCB; /* Handles the provisioning of clients and connects / interfaces with * the server. */ bool connectToServer() { NimBLEClient *pClient = nullptr; // Check if we have a client we should reuse first if (NimBLEDevice::getClientListSize()) { /* Special case when we already know this device, we send false as * the second argument in connect() to prevent refreshing the * service database. This saves considerable time and power. */ pClient = NimBLEDevice::getClientByPeerAddress(advDevice->getAddress()); if (pClient) { if (!pClient->connect(advDevice, false)) { Serial.println("Reconnect failed"); return false; } Serial.println("Reconnected client"); } else { /* We don’t already have a client that knows this device, * we will check for a client that is disconnected that we can * use. */ pClient = NimBLEDevice::getDisconnectedClient(); } } // No client to reuse? Create a new one. if (!pClient) { if (NimBLEDevice::getClientListSize() >= NIMBLE_MAX_CONNECTIONS) { Serial.println( "Max clients reached. No more connections possible");

● 212

Boek BLE 220329 UK.indd 212

06/05/2022 15:55

Chapter 6 • Profiles and roles

return false; } pClient = NimBLEDevice::createClient(); Serial.println("New client created"); pClient->setClientCallbacks(&clientCB, false); pClient->setConnectionParams(50, 70, 0, 420); pClient->setConnectTimeout(5); if (!pClient->connect(advDevice)) { /* Created a client but failed to connect, don’t need to keep it * as it has no data */ NimBLEDevice::deleteClient(pClient); Serial.println("Failed to connect, deleted client"); return false; } } if (!pClient->isConnected()) { if (!pClient->connect(advDevice)) { Serial.println("Failed to connect"); return false; } } Serial.print("Connected to: "); Serial.println(pClient->getPeerAddress().toString().c_str()); Serial.print("RSSI: "); Serial.println(pClient->getRssi()); /* Now we can read/write the charateristics of the services we * are interested in */ pSvc = pClient->getService(UUID_LINK_LOSS_SERVICE); if (pSvc) { // Make sure it’s not null pChr = pSvc->getCharacteristic(UUID_ALERT_LEVEL_CHARACTERISTIC); } if (pChr) { // Make sure it’s not null // Write new alert level alert_level = 1; // Mild Alert pChr->writeValue(&alert_level, 1, true);

● 213

Boek BLE 220329 UK.indd 213

06/05/2022 15:55

Bluetooth Low Energy Applications

// Read alert level alert_level = pChr->readValue(); Serial.print("Alert level: "); Serial.println(alert_level); } else { Serial.println("Alert Level characteristic not found."); } Serial.println("Done with this device!"); return true; } void setup() { Serial.begin(115200); Serial.println("Starting NimBLE Client"); NimBLEDevice::init(""); /* Set security properties: * No bonding, no MITM protection, secure pairing */ NimBLEDevice::setSecurityAuth(false, false, true); NimBLEDevice::setSecurityIOCap(BLE_HS_IO_NO_INPUT_OUTPUT); NimBLEScan *pScan = NimBLEDevice::getScan(); pScan->setAdvertisedDeviceCallbacks( new AdvertisedDeviceCallbacks()); pScan->setInterval(60); pScan->setWindow(30); pScan->setActiveScan(true); pScan->start(scanTime, nullptr); } void loop() { // Loop here until we find a device we want to connect to while (!doConnect) { delay(1); } doConnect = false; // Found a device we want to connect to, do it now if (connectToServer()) {

● 214

Boek BLE 220329 UK.indd 214

06/05/2022 15:55

Chapter 6 • Profiles and roles

Serial.println("Success, scanning for more..."); } else { Serial.println("Failed to connect, starting scan..."); } NimBLEDevice::getScan()->start(scanTime, nullptr); }

In the beginning, the code defines the UUIDs for the Link Loss service and the Alert Level characteristic. The alert_level variable holds the alert level to be written to the proximity reporter, and its initialized to 1 (Mild Alert). The alert_on_link_loss() function is used to show an alert on the proximity monitor’s serial console. The AdvertisedDeviceCallbacks class has a member function, onResult(), which checks for the existence of the Link Loss service on an advertised device, and then connects to the device. The connectToServer() function connects to the proximity reporter and writes the alert level to the Alert Level characteristic. The setup() function sets some security properties: no bonding, no man-in-the-middle protection, and secure pairing. With setSecurityIOCap(BLE_HS_IO_NO_INPUT_OUTPUT) the NoInputNoOutput capability is used. As a result, the proximity monitor will connect with Security Mode 1 Level 2. We’ll use a scan interval of 60 ms and a scan window of 30 ms. Both values are the ones listed in section 5.2.2 of the Proximity Profile specification. Now, build this code for your ESP32 board: $ arduino-cli compile -b esp32:esp32:pico32 -e Proximity_Monitor

Then, connect the ESP32 board to your computer with a USB cable and upload the code to the ESP32: $ arduino-cli upload -p /dev/ttyUSB0 -b esp32:esp32:pico32 -i Proximity_Monitor/ build/esp32.esp32.pico32/Proximity_Monitor.ino.bin

Establish a serial connection to the board: $ screen /dev/ttyUSB0 115200

Now, when your ESP32 finds your Proximity Reporter, it will connect to it. When the device disconnects, the proximity reporter alerts: Disconnected from D8:A0:1D:40:54:36 (public) (reason 0x08) Mild Alert

● 215

Boek BLE 220329 UK.indd 215

06/05/2022 15:55

Bluetooth Low Energy Applications

The proximity monitor also shows an alert on the serial console: Disconnected - Alerting on link loss and starting scan Mild Alert

Note: This is just a minimal implementation of a proximity monitor. For instance, it just connects to the first proximity reporter it finds. You could also add buttons to your device so can choose the alert level using a button. You should also implement a reliable way to reconnect to the proximity reporter.

6.7 Summary and further exploration In this chapter, I went through the complete specification of a BLE profile and one of its services with the service’s characteristics. I also implemented both of this profile’s roles – one in Zephyr and one in NimBLE-Arduino. If you’re interested in the Proximity profile, there are still a lot of things you can do to create a complete and fully conforming implementation of the Proximity Reporter and Proximity Monitor roles explained in this chapter. But, more than just an explanation of the Proximity profile, this chapter is meant to be used as an example step-by-step guide for implementing any BLE profile. You learned how to understand a profile specification and a service specification, you learned where to find the definition of a characteristic, and where to find the UUID values for services and characteristics. But, most of all, you learned how to connect all the dots of these various BLE specifications. With this chapter as a guideline, you’re able to tap into the power of GATT profiles. Not all devices implement a well-known GATT profile for which you can read the specification. In the next chapter, you’ll learn how to reverse engineer devices when you don’t have a specification.

● 216

Boek BLE 220329 UK.indd 216

06/05/2022 15:55

Chapter 7 • Reverse engineering BLE devices

Chapter 7 • Reverse engineering BLE devices For many BLE devices, you don’t have the luxury of them having implemented a GATT profile specified by the Bluetooth SIG. That means that you can’t just read their specifications, as you did in the previous chapter. You’ll have to reverse engineer the device to understand how you can use it. In this chapter, I’m going to reverse engineer a BLE LED badge as an example of this approach. I’m going to find out how it works by: • investigating its services and characteristics • decompiling the accompanying mobile app • sniffing BLE traffic between the mobile app and the device The goal is to create a Python script to control the LED badge, so that I don’t need the official mobile app anymore.

7.1 Investigating the LED badge A while ago I bought this LED badge from AliExpress:

Figure 7.1 This Bluetooth LED badge seems like an interesting device to reverse engineer. The display has an 11×55 LED-pixel display, and it’s available in various colors. It supports Bluetooth, but the version isn’t specified. 59 59

There are various models of the LED badge. The NB1155 used in this chapter has 11×55 LED pixels, while the NB1144 has 11×44.

● 217

Boek BLE 220329 UK.indd 217

06/05/2022 15:55

Bluetooth Low Energy Applications

There’s a QR code on the back of the LED badge and on the instruction leaflet that is delivered with it. Scanning it results in a URL that returns an HTTP 404 error. However, the product page on AliExpress says to search for an app called "Bluetooth LED Name Badge" on Google Play. I found the app, made by Shenzhen Lesun Electronics Co., Ltd:

Figure 7.2 The Bluetooth LED Name Badge app on the Google Play Store can send commands to the LED badge. This shows that the app can send text and icons to the LED badge, and that it has some settings, such as scrolling speed. First, let’s turn the LED badge on and use nRF Connect for Mobile to see what it’s advertising. At the left side, you see a Micro-USB port and two small buttons. If you press twice on the lower button, the display shows a BLE icon. In nRF Connect, you can see that it’s now advertising:

● 218

Boek BLE 220329 UK.indd 218

06/05/2022 15:55

Chapter 7 • Reverse engineering BLE devices

Figure 7.3 The Bluetooth LED badge as seen in nRF Connect for Mobile You’re seeing manufacturer-specific data and two custom BLE services: 0xfee7 and 0xfee0. The name of the device is LSLED, followed by some unprintable characters (probably a bug in the firmware). If you connect to it, you see its services, along with the following characteristics: Service 0xfee7

0xfee0

Characteristic

Properties

0xfec7

WRITE

0xfec8

INDICATE

0xfec9

READ

0xfee1

NOTIFY, READ, WRITE

Table 7.1 BLE badge characteristics If you read characteristic 0xfec9, it returns the same value as the manufacturer-specific data. Reading 0xfee1 doesn’t return any data, and subscribing to its notifications doesn’t return anything, either. The Characteristic User Description of 0xfee1 is "Data." So, it seems we can’t find anything useful out about this device using this method.

7.2 Decompiling the mobile app The Bluetooth LED Name Badge app for Android is obviously able to communicate with the LED badge, so, another approach to reverse engineering is to try and find out what the app does. One way to do this is to decompile the app and try to understand its source code. Download the APK file for the Android app. Some companies put the APK files on their own websites. This wasn’t the case here, but you can still download the APK using a third-party APK downloader site, such as https://apkpure.com. Just paste the app’s Google Play Store URL (https://play.google.com/store/apps/details?id=com.yannis.ledcard, in this case) into the Apkpure’s search field. After this, you’re able to download the file. 60

60

Another way to get the APK file is to install the app on your phone and then extract the file from its file system.

● 219

Boek BLE 220329 UK.indd 219

06/05/2022 15:55

Bluetooth Low Energy Applications

This APK file contains the Dalvik byte code that Android executes. To understand what the app does, we need its source code. While the developer has compiled its source code to Dalvik, we can reverse this using a decompiler. A powerful decompiler that turns Dalvik byte code into Java source code is jadx (https:// github.com/skylot/jadx). It’s a Java program that comes in a command-line version as well as in a graphical interface. You need the 64bit version of Java 8 or later to be able to run it. Install the most recent release from https://github.com/skylot/jadx/releases. It works on Windows, Linux, and macOS. In Windows, you can start the graphical interface by double-clicking the jadx-gui file in the bin directory. On other operating systems, you can start the graphical interface by running bin/jadx-gui from the command line. Jadx asks you to open a file. Select the APK file you downloaded and click on Open file. The program now decompiles the app and shows a tree structure of its packages and Java files at the left. Now the search begins. You already have a couple of leads: the UUIDs of the services and characteristics you found in nRF Connect for Mobile. In the Navigation / Text search menu, you can enter some search terms. As it turns out, this is easier said than done. The code doesn’t mention the 0xfee7 service or its characteristics. It does mention the other service and its characteristic, in the com. yannis.ledcard.ble.BleDevice class: public static final String UUID_CHARACTERISTICS_WRITE = "fee1"; public static final String UUID_SERVICE = "fee0";

You can now search for these constants, UUID_CHARACTERISTICS_WRITE and UUID_ SERVICE, in the code. The easiest way to do this is to right-click on the name of the constant and then choose Find usage. Then you can click on one of the found usages to open the corresponding file in that location. However, doing this didn’t make me any wiser. So, I tried to search for some other clues in the app’s code. After a while, I found the com.yannis.ledcard.util.LedDataUtil class, which shows some promising code that deals with pictures and display modes. Specifically, I found the following Java method that seems to create some kind of header for data: public static byte[] get64(List list, int i) { Iterator it = list.iterator(); while (it.hasNext()) { Log.d("abcdef", "get64------------SendContent:" + it.next().toString()); } byte[] bArr = new byte[64]; bArr[0] = 119; bArr[1] = 97;

● 220

Boek BLE 220329 UK.indd 220

06/05/2022 15:55

Chapter 7 • Reverse engineering BLE devices

bArr[2] = 110; bArr[3] = 103; bArr[4] = 0; bArr[5] = 0; bArr[6] = getFlash(list); bArr[7] = getMarquee(list); byte[] modeAndSpeed = getModeAndSpeed(list); for (int i2 = 0; i2 < 8; i2++) { bArr[i2 + 8] = modeAndSpeed[i2]; } byte[] msgLength = getMsgLength(list, i); for (int i3 = 0; i3 < 16; i3++) { bArr[i3 + 16] = msgLength[i3]; } bArr[32] = 0; bArr[33] = 0; bArr[34] = 0; bArr[35] = 0; bArr[36] = 0; bArr[37] = 0; byte[] date = getDate(); for (int i4 = 0; i4 < 6; i4++) { bArr[i4 + 38] = date[i4]; } for (int i5 = 0; i5 < 19; i5++) { bArr[i5 + 44] = 0; } bArr[63] = 0; return bArr; }

This clearly shows some fixed header (the first 6 bytes), a mode and speed, a message length, and a date. When I searched for this method’s usage in the rest of the app’s code, it confirmed my suspicion: it was called by a getSendHeader() in the com.yannis.ledcard. mode.MainMode class. Overall, I think I would be able to reverse engineer what the app is doing just by delving further into this code. However, it wasn’t as clear as I thought it would be, so I changed strategies. I thought it would be better to start using the app with the LED badge and sniff the traffic between both devices. Then I could combine the knowledge gained from the app’s source code with live traffic exchanged between the app and the LED badge for a better understanding.

● 221

Boek BLE 220329 UK.indd 221

06/05/2022 15:55

Bluetooth Low Energy Applications

7.3 Sniffing BLE traffic between the LED badge and the mobile app Open Wireshark on your computer and start packet capture with nRF Sniffer for Bluetooth LE. 61 On the BLE badge, press twice on the lower button until it shows the Bluetooth icon. Then, select the device in Wireshark to show only packets from and to that specific device. Install Bluetooth LED Name Badge on your Android phone and open the app. In Wireshark, you see that your phone does a scan request, and the BLE badge answers with its device name in the scan response. Select the type of device in the app. For the 11×55 LED badge, the type is 11. Tap yes. The app shows a list with messages to send. The default configuration is to send one message: Welcome. Tap on Send. The app then connects to the LED badge and shows the Welcome message on its display: 62

Figure 7.4 The Bluetooth LED badge shows a welcome message. In Wireshark, you’ll see a CONNECT_REQ packet, followed by a request for attributes, then a couple of write requests and responses. If you only want to see the write requests for a better overview, right-click on the Write Request opcode in the Bluetooth Attribute Protocol part of the packet details and choose Apply as Filter / Selected. Or, enter btatt. opcode == 0x12 as the display filter. In this case, the app sent 9 Write Request packets with each 16 bytes of data: 77 61 6e 67 00 00 00 00 30 30 30 30 30 30 30 30 00 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 e5 0a 18 0c 37 25 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 c6 c6 c6 c6 d6 fe ee c6 82 00 00 00 00 00 7c c6 fe c0 c6 7c 00 00 38 18 18 18 18 18 18 18 3c 00 00 00 00 00 7c c6 c0 c0 c6 7c 00 00 00 00 00 7c c6 c6 c6 c6 7c 00 00 00 00 00 ec fe d6 d6 d6 c6 00 00 00 00 00 7c c6 fe c0 c6 7c 00 00 00 00 61 62

There’s another way to sniff the BLE traffic directly from your Android device with the Bluetooth HCI snoop log. Consult the appendix for this method. Note that there’s no authentication. Everyone in your LED badge’s neighborhood can show messages to it, at least when it’s in listening mode (showing the Bluetooth icon).

● 222

Boek BLE 220329 UK.indd 222

06/05/2022 15:55

Chapter 7 • Reverse engineering BLE devices

If you look back at the get64() Java method from the previous section, you’ll recognize the parts of the header: 77 61 6e 67 is the hexadecimal equivalent of the decimal values 119 97 110 103. Then we get 00 twice, and then again 00 twice, which are the values for getFlash() and getMarquee(), respectively. The next eight bytes are all 0x30, which represent the mode and speed. The next 16 bytes represent a message length. Because the app shows a list of eight messages, this could be the list of the lengths of these messages. The first two bytes here are 00 07, which indeed is the number of characters in the text, Welcome. The hypothesis is that the next 14 bytes represent the lengths of the next seven messages (none, in this case). Then, there are six 00s, and the next six bytes represent the date, which is e5 0a 18 0c 37 25, in this case. If you convert this to decimal, it’s 229 10 24 12 55 37. I ran this app on October 24, 2021, at 12:55:37, so the month, day and time were correct, but, the 229 seemed weird for the year. But if you know that it’s one byte, you know it can’t represent the full year. 2021 in hexadecimal is 0x07e5, and e5 (decimal 229) is indeed the first value of the date in the header. Finally, the header concludes with twenty 00s. After this, five packets of 16 bytes somehow represent the seven characters of the Welcome text. Let’s find out how. But, let’s first look at the output of a simpler message. Open the app again and click on the first message. Change the text to W and enable Flash. Return to the main screen, click on the second message, and add the text, e. For this second message, set the speed to 8, the mode to right, and enable Marquee. Return to the main screen. Then enable the slider next to the second message and click on Send while looking at the Bluetooth Attribute Protocol packets in Wireshark. The LED badge now shows the letter W moving to the left and flashing on and off. After this, it shows the letter e moving to the right at double speed and a frame of moving pixels around it. In this case, the app sent six Write Request packets, each with 16 bytes of data: 77 61 6e 67 00 00 01 02 30 71 30 30 30 30 30 30 00 01 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 e5 0a 18 10 06 13 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 c6 c6 c6 c6 d6 fe ee c6 82 00 00 00 00 00 7c c6 fe c0 c6 7c 00 00 00 00 00 00 00 00 00 00 00

The first packet starts with the same 6 bytes, but then the value of getFlash() is 01 and that of getMarquee() is 02. This is because you enabled Flash for the first message and Marquee for the second message.

● 223

Boek BLE 220329 UK.indd 223

06/05/2022 15:55

Bluetooth Low Energy Applications

The next byte is still 0x30, because you didn’t change anything to the mode and speed of the first message. But the byte after this is now 0x71. You changed the speed to 8 and the mode to right. So, the left nibble of that byte apparently encodes the speed (speed 4 is encoded as 3 and speed 8 as 7) and the right nibble the mode. 63 The second packet shows 00 01 as the length of the first message and the same for the length of the second message – this confirms our hypothesis. Then the date is encoded. There are two packets left now. If you convert the hexadecimal values to binary and show them byte by byte under each other, split into eleven rows (because the display is 11 pixels tall) they look like this: 00000000 11000110 11000110 11000110 11000110 11010110 11111110 11101110 11000110 10000010 00000000 00000000 00000000 00000000 00000000 01111100 11000110 11111110 11000000 11000110 01111100 00000000

At the end, the data is padded with 0s to fill 16 bytes (not shown here). You can clearly see the letters W and e. If you apply the same decoding to the first message with Welcome and put the bitmaps of the letters next to each other, you see the following result:

63

A nibble is half a byte (4 bits). It’s represented by one hexadecimal character. It’s called a nibble (a small bite) because "byte" is a homophone of the English word, "bite."

● 224

Boek BLE 220329 UK.indd 224

06/05/2022 15:55

Chapter 7 • Reverse engineering BLE devices

00000000000000000000000000000000000000000000000000000000 11000110000000000011100000000000000000000000000000000000 11000110000000000001100000000000000000000000000000000000 11000110000000000001100000000000000000000000000000000000 11000110011111000001100001111100011111001110110001111100 11010110110001100001100011000110110001101111111011000110 11111110111111100001100011000000110001101101011011111110 11101110110000000001100011000000110001101101011011000000 11000110110001100001100011000110110001101101011011000110 10000010011111000011110001111100011111001100011001111100 00000000000000000000000000000000000000000000000000000000

This clearly shows a 1 for each enabled pixel and a 0 for each disabled one. You now have a pretty good idea of the format of the data to send to the LED badge in order to show some text on the display. Let’s put this all together and create a Python script to write your own images to the LED badge.

7.4 Writing arbitrary images to the LED badge using Bleak The Bluetooth LED Name Badge also allows you to send some pre-defined little 11×11 bitmaps. But now that you know the data format to send, you can also send an arbitrary bitmap, for instance filling the entire 11×55 display. Let’s do this with Bleak.

7.4.1 Finding LED badges Let’s first create a script that scans for BLE badges, based on their names, for five seconds, and shows their Bluetooth addresses and RSSIs: """Find BLE LED badges. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio from bleak import BleakScanner num_devices = 0

def device_found(device, advertisement_data): """Show device details if it’s a BLE LED badge.""" global num_devices if device.name.startswith("LSLED"): num_devices += 1 print(

● 225

Boek BLE 220329 UK.indd 225

06/05/2022 15:55

Bluetooth Low Energy Applications

f"{device.address} ({device.name}) - RSSI: {device.rssi}" )

async def main(): """Scan for BLE devices.""" print("Searching for LED badges...") scanner = BleakScanner() scanner.register_detection_callback(device_found) await scanner.start() await asyncio.sleep(5.0) await scanner.stop() if not num_devices: print("No devices found")

if __name__ == "__main__": asyncio.run(main())

If you run this with two LED badges that are both in listening mode, the script shows: $ python3 find_led_badge.py Searching for LED badges... ED:67:38:80:0D:E2 (LSLED) - RSSI: -74 ED:67:38:7F:99:B5 (LSLED) - RSSI: -83

7.4.2 Writing images to the LED badge Let’s put all the knowledge gained in the previous section together. I’ll create a script that writes the appropriate commands to the LED badge to show an image: """Display an image on a BLE LED badge. Copyright (c) 2022 Koen Vervloesem SPDX-License-Identifier: MIT """ import asyncio import sys from PIL import Image import bleak

● 226

Boek BLE 220329 UK.indd 226

06/05/2022 15:55

Chapter 7 • Reverse engineering BLE devices

WRITE_CHAR_UUID = "0000fee1-0000-1000-8000-00805f9b34fb" COMMAND1 = bytes( [ 0x77, 0x61, 0x6E, 0x67, 0x00, 0x00, 0x00, 0x00, 0x34, 0x30, 0x30, 0x30, 0x30, 0x30, 0x30, 0x30, ] ) COMMAND2 = bytes([0x00, 0x07, *(14 * [0x00])]) COMMAND3 = bytes(16 * [0x00]) COMMAND4 = bytes(16 * [0x00])

def chunks(lst, n): """Yield successive n-sized chunks from lst.""" for i in range(0, len(lst), n): yield lst[i : i + n]

def commands_for_image(image): """Return commands to show an image on the BLE LED badge.""" image_bytes = bytearray() with Image.open(image) as im: px = im.load() for i in range(7):

# 7x8 = 56 -> 7 bytes next to each other

for row in range(11): row_byte = 0 for column in range(8): try: pixel = int( px[(i * 8) + column, row][3] / 255

● 227

Boek BLE 220329 UK.indd 227

06/05/2022 15:55

Bluetooth Low Energy Applications

) except IndexError: pass

# Ignore the 56th pixel in a full row

row_byte = row_byte | (pixel Discover BLE devices in the neighborhood by listening to their advertisements. > Create your own BLE devices advertising data. > Connect to BLE devices such as heart rate monitors and proximity reporters. > Create secure connections to BLE devices with encryption and authentication. > Understand BLE service and profile specifications and implement them. > Reverse engineer a BLE device with a proprietary implementation and control it with your own software. > Make your BLE devices use as little power as possible.

Koen Vervloesem has been writing for over 20 years on Linux, open-source software, security, home automation, AI, programming, and the Internet of Things. He holds a Master’s degree in Computer Science Engineering, a Master’s degree in Philosophy, and an LPIC-3 303 Security certificate. He is a board member of the Belgian privacy activist organisation the Ministry of Privacy.

This book shows you the ropes of BLE programming with Python and the Bleak library on a Raspberry Pi or PC, with C++ and NimBLE-Arduino on Espressif’s ESP32 development boards, and with C on one of the development boards supported by the Zephyr real-time operating system, such as Nordic Semiconductor's nRF52 boards. Starting with a very little amount of theory, you’ll develop code right from the beginning. After you’ve completed this book, you’ll know enough to create your own BLE applications. Elektor International Media BV www.elektor.com

Develop your own Bluetooth Low Energy Applications • Koen Vervloesem

Develop your own Bluetooth Low Energy Applications

BLE

Develop your own Bluetooth Low Energy Applications for Raspberry Pi, ESP32 and nRF52 with Python, Arduino and Zephyr

Koen Vervloesem