Personal Experiment #2: Comparing precision and accuracy between multiple accelerometers – Protocol
My second personal experiment aims assess the (relative) utility of digital health activity trackers. This experiment isn’t an N-of-1 trial like our first installment. This is more of a classic measurement error / calibration experiment. So without further adieu, the design…
Aim: This experiment aims to assess the precision and accuracy of three commercially available activity trackers at measuring steps taken by a user of the device.
Experiment: I will wear three activity tracker devices, the Fitbit One, the Fitbug Orb, and the Jawbone UP, as directed by each manufacturer. For fourteen consecutive days I will wear the three devices, take 500 steps, and manually count each step using a commercially available step counter.
Prior to beginning the 500 steps, I will record the number of steps registered on each device for the day for a device’s baseline reading. After taking my 500th step, I will record the number of steps registered on each device. I will subtract the baseline step count from the final step count to arrive at the number of steps measured by each device during my 500 steps.
I will perform two trial runs of 100 steps each before the first experiment to familiarize myself with the process of uploading data from each device.
I will record measurements from the three devices being tested in the following order: 1) Jawbone UP, 2) Fitbit One, 3) Fitbug Orb
Primary outcome: Accuracy of each device compared to the “gold standard” measurement of self-recorded steps via step counter.
Secondary outcomes: Precision of each device over the course of the 14 trials. Relative accuracy of each device compared to the other two devices.
Analysis plan: [I’m going to write an updated post with the analysis plan before looking at the data. I figure I’ll make myself do it if I just get started with the experiment.]
Here is a link to the google spreadsheet where I’ll be collecting the data.
Potential sources of error:
- Additional steps counted by Fitbit, Fitbug, and/or UP either 1) before experimenter begins counting steps or 2) after 500 steps have been recorded but before readings are taken from a given device.
- Miscounting of steps by experimenter when operating the manual step counter – this could include over- or under-estimating total number of steps taken while measurements are being recorded.
- Error introduced by changing the terrain over which the 500 steps are taken
- Laptop with internet access
- iPhone to collect data from Fitbug device