Leveraging Machine Learning to Enhance Acoustic Eavesdropping Attacks (Part 1 of 4)

This multi-part series explores how machine learning can enhance eavesdropping on cellular audio using gyroscopes and accelerometers — inertial sensors commonly built into mobile devices to measure motion through Micro-Electro-Mechanical Systems (MEMS) technology. The research was conducted over the summer by one of our interns, Alec K., and a newly hired full-time engineer, August H.

Introduction

Acoustic eavesdropping attacks are a potentially devastating threat to the confidentiality of user information, especially if these attacks are implemented on smartphones, which are now ubiquitous. However, conventional microphone-based attacks are limited on smartphone devices by the fact that the user must consent to the collection of microphone information by applications. Recently, researchers on eavesdropping have taken to performing side-channel attacks that leverage information leaks from a piece of hardware to reconstruct some kind of secret (i.e. the audio we want to listen in on).

Unlike the microphone, which requires explicit user permission to access, sensors like the gyroscope and accelerometer do not require explicit user consent for an application to access their readings on Android. These devices are sensitive to the vibrations caused by sound, and since some Android devices allow sampling these sensors at frequencies up to 500 Hz, it is possible to reconstruct sound using these devices.

The researchers of the Gyrophone project exploited this to extract small amounts of speech data from the gyroscope using signal processing and machine learning techniques. AccEar extended this work to the accelerometer, and was able to restore about 85% of speech using a more complicated machine learning technique called a Generative Adversarial Network (GAN).

In this blog series, we will explore why side-channel attacks like Gyrophone and AccEar were so successful, reproduce their results, and attempt to improve upon them. To that end, we acquired a Google Pixel 7a for testing to confirm that the Gyrophone and AccEar exploits were still feasible. Then, we recorded test data on the gyroscope and accelerometer and preprocessed the data. Finally, we performed extensive AI research and modeling to restore the original speech from the sensor readings.

The next part of this blog series will cover the preliminary testing we performed to confirm the feasibility of exploiting this vulnerability.