Can you measure brain activity with a smartphone? Maybe!
Hey! So I haven’t posted on this blog for a while (grad school more or less took over my life), but I’d thought I’d write a little about some of the experiments I’ve been doing looking at whether smartphones can measure brain activity based on blood flow and optical plethysmography. Optical plethysmography is a term for measuring blood volume in the body by looking at light transmitted through tissue: it relies on the fact that blood is pretty good at absorbing light so when blood vessels are “fuller” more light is absorbed. Plethysmography has actually found a lot of application in smartphones already, primarily to measure heart rate--a number of apps use the phone’s camera and flashlight LED to provide pulse measurements based on oscillations in light absorption. This method is potentially useful for looking at brain function because when areas of the brain are activated the local blood vessels dilate (through a process called neurovascular coupling) to deliver more oxygen to the area. Dilation increases the amount of blood present in the tissue, which is a signal that could theoretically be detected by photoplethysmography. In fact, researchers have reported using photoplethysmography to measure blood volume in the brain however the experiment involved artificially manipulating blood pressure rather than measuring the natural hemodynamic response during a cognitive task. To test whether phone-based photoplethysmography could be used to detect changes in blood flow due to brain activation, I wrote an Android app for my Galaxy S7 which determines the average brightness of red light in the camera’s field of view. When pressed against the skin with the camera LED on, this yields a fairly uniform image.
An example of what an image from this setup looks like.
The app averages red values over a period of 250 milliseconds (so the final measurement rate is four measurements per second) and transmits them to a server. This arrangement makes it easy to stream data from the phone: any app that knows the phone’s “stream id” can connect to the server and retrieve realtime readings. With the phone and streaming system set up, I wanted to know if the data being streamed could actually be used to measure cognitive brain functions, As a first test, I decided to use an oddball protocol: a simple cognitive task where a subject hears two beeps, one common and one rare, and hearing the rare beep requires the subject to make a response. The oddball protocol is a good test of a brain imaging method because the rare stimulus reliably evokes a larger response over much of the brain (particularly the frontal cortex) than the common stimulus. To determine whether the rare beeps produced a larger response than the common beeps, I used an event related design (a common method in EEG and fMRI studies) to look at changes in brain activity immediately after the beep occurred. I placed the phone with its LED and camera approximately over the Fp1 point.
Using EEGLAB to compute event-related changes, I found that the rare beeps produced a larger brain response than the common beeps!
Right click and select "view" to get a higher resolution image Response to the “typical” sound (top) and the “oddball” sound (bottom) in terms of the change in intensity of red light detected by the camera’s sensor. The bottom part of each plot shows the average response in milliseconds relative to stimulus onset, the top part shows the response in each trial. All scales are the same between plots. Note that the oddball stimulus evokes a stronger and more consistent response around 2000 milliseconds.
Average responses superimposed. The green trace is the oddball response, blue is the typical stimulus response. Polarity here is inverted relative to the previous two plots. Note that the absolute magnitude of this change is very small; a camera sensor measures redness value on a scale from 1-256, while the change in value is only 0.8 points. This is to be expected; most of the light emitted from the LED is absorbed or reflected before it reaches the brain and passes back to the sensor. Still, I was kind of suspicious that other factors (like movement of the sensor on the head) might have contributed to this signal. As a control, I did another experiment which presented both tones with the same chance (each presented 50% of the time) and where I responded to every tone.. This provided a way to isolate the signal due to the oddball effect from other signals like the brain’s recognition that “something happened” or artifacts due to movement. In these data, the spike that occurred 2 seconds following the oddball stimulus was not present, although a decrease in the red value occurred at longer latencies. This suggests that the early positive spike may represent a true oddball response while the later decreases represent a more general effect (i.e. of planning or executing movement).
Response to all tones in the control experiment, with the same scale and units as the plots above.
Another thing I was curious about was how well this method could measure brain function over longer periods of time. To test this, I positioned the camera approximately over the left dorsolateral prefrontal cortex (an area involved in memory) while I performed working memory tasks. By varying the difficulty of the working memory task (switching between a dual 1-back and dual-3-back task), I was able to vary the working memory demand while keeping other factors (i.e. movement) fairly constant. Theoretically, this should result in increased engagement for the higher working memory load, and therefore a stronger signal compared to a no-task baseline. However, a lot of factors can interfere when recording over a period of several minutes—things like a decrease of signal quality or change in heart rate potentially create “artifactual” differences between conditions. To reduce the impact of these variables, I exploited the fact that photoplesymogram signals contain a “high frequency” oscillating component (due to the pulse) as well as (hopefully) a low frequency component corresponding to the brain cerebrovascular response. In theory, differences between conditions due to changes in signal quality or other non-brain factors (like movement and muscle tension) should affect both the high and low frequency signal. By using a technique called multiple regression, we can determine how much the overall photoplethysmography signal differs between conditions. Because we assume that the high frequency signal is related mostly to processes we are not interested in, we can also ask “what amount of difference between conditions is NOT attributable to difference in the high frequency signal?”. The differences that can’t be attributed to this signal are the ones we are interested in because they more likely represent true brain activity. I also applied the same technique to eliminate differences between conditions that were due to motion or posture (by streaming accelerometer readings from the phone and including them as covariates in the same way as the high frequency signal power was included.)
Across 6 trials (3 in each condition) the difference between the levels of memory load was statistically significant (p=0.049), and highly significant when looking specifically at a signal smoothed with a moving average (p < 0.001). There was a consistent trend toward decreased red light detection (indicating higher absorption and greater blood volume) in the high-load condition compared to the low-load condition.
Representative example of low pass filtered red value over time in dual-1-back and dual-3-back conditions. The data is low pass filtered because a large amount of the signal consists of high frequency oscillations due to heartbeat.
Discussion
These results suggest to me that there may be something going on here. Is it conclusive? No. There are still potential sources of error to be ruled out, and I’ll be working on some experiments over the next few months to do so. However I do think it is interesting enough to be worth discussing.
There is also a lot of room to improve this technology in the areas of data acquisition, processing, and statistical analysis. To facilitate this, I’ve released several components as public-domain software, including:
*Physphone, an Android app for acquiring and transmitting data
*Biostream, the software that facilitates streaming data
*Python data acquisition and statistics utilities that connect to Biostream To use the app, you can download it from here. Data is streamed to the server, and can be retrieved using the stream ID generated by the app. To retrieve data, simply send a request to http://biostream-1024.appspot.com/get?stream=<stream id>. You can also view the source code for all these components on GitHub.
Finally, if anyone is interested in looking at the data I’ve collected, please let me know on here!