How Do I Upload My Exported Fitbit Data

How to download all of your raw fitbit data

There are many just-out-of-curiosity reasons someone would want to mine all of their ain raw fitbit data. In this commodity I address how one would actually go about retrieving raw fitbit data, bold that they had the interest and know-how to do so.

2019 update

FitBit now offers downloading all of your data in one-fell dive! Become to your fitbit account settings from a web browser to queue a large cipher file. You volition get an email that states:

Activity required: Confirm your Fitbit information export asking

Follow the instructions from at that place, and you lot are washed! The original weblog mail below still holds programmatic value, and FitBit limits how often you can re-download the all-of-your-data zip file, and then there are still reasons to use the Awarding Programming Interface approach I draw hither. Thank you to Medium user homunculus for commenting virtually this choice back in August. The feature has been around since at least April 2019.

Provisos

The phrase "raw data" in this context does not refer to the accelerometer signals similar the velocity of your hand as you jog — those signals undergo instantaneous on-chip processing to place each atomic footstep or stair climb. Bluetooth bandwidth and bombardment life considerations constrain your fitbit device to telemeter mere aggregated information to your telephone, tablet, or computer.

Raw data in this context refers to the data in its most granular form available, so-called data exhaust that a fitbit generates and uploads to a continuously updated database somewhere in the cloud. Presently, the almost granular class of data available is the minute-by-minute time serial of your steps, heart rate, and sleep.

FitBit (the company, singled-out from fitbit, the tracker/app) country that "your information vest to you lot": anyone can access his or her own data at any time. While true, retrieving your information in practice requires overcoming some significant technical barriers. Over the concluding two years, I take adult and applied a Python framework to query and compile my fitbit data into easy-to-utilise formats for conducting interactive information science and personal physiology. Here's what I constitute out.

Programmatically accessing fitbit data

See the top of the blog post for an update from this year — y'all can now download all of your fitbit data in one roughshod dive!

And then far as I know, there is no way to download all of your fitbit data e'er in i fell dive. — This blog post, 2018

FitBit offers Awarding Programming Interfaces (APIs) on their programmer site, with a Web portal containing moderately avant-garde instructions on how to get data. After you take logged in to your fitbit account, yous can read the documentation to meet how the arrangement works.
You will accept to write queries with RESTful HTTP requests. It takes a little getting used to if you lot have not done this before, merely an example may exist illustrative. If you want to get all of your "intraday middle charge per unit data" from September 21, 2018, y'all would write this long HTTP query string:

          https://api.fitbit.com/ane/user/-/activities/heart/engagement/2018-09-21/1d/1min.json        

…but you wouldn't be washed yet. You demand to include an instruction for what yous desire to do with that data — get, alter, or delete information technology — and y'all need to testify y'all are authorized to do so. One of the nigh popular Python packages of all time, requests, allows you lot to programmatically execute these commands if you know Python. Every mod computing linguistic communication has an equivalent to requests, but I will focus on Python. In Python yous would exercise:

          import requests
response = requests.get(query_str, headers=secret_header)        

where query_str is the query cord above and secret_header contains information about your authorization.

Authorisation

Only you can see your "intraday" data- the name FitBit adopts for minute-by-infinitesimal fourth dimension series of your heart rate and steps. FitBit uses standard cryptographically secure authentication procedures to verify your identity, and they brand information technology specially difficult to retrieve intraday data: you need to annals a free personal app to get API client credentials. This 2016 blog mail describes how to become your secret_token for the header. Once yous have your token, you can write:

          secret_header = {'Authorization': 'Bearer {}'.format(secret_token)}        

Beware that the secret token should not be shared publicly — afterward all, it's secret.

Saving and inspecting data

The response object you get back contains the data in json format, which has multiple tiers of raw information and metadata. You can save the unabridged information locally for backup purposes (recommended):

          import json
with open(path, 'w') every bit f:
            json.dump(response.json(), f)        

…where path is a descriptive filename you can make up such as HR_20180921_1d_1min.json. When you're ready to analyze your data, you can open up the awkward json format into a more user-friendly pandas DataFrame:

          import pandas equally pd
with open(path, 'r') every bit f:
            json_data = json.load(f)
df = pd.DataFrame(json_data['activities-center-intraday']['dataset'])        

At present you tin can start inspecting, merging, and transforming information to brand some awesome plots. In my feel, I have had to practice some standard cleaning operations, like setting the alphabetize to be a engagement and a fourth dimension, and renaming "value" to something more informative like "heart_rate" or simply "HR". The topic of data cleaning, merging, and analysis is the topic for a whole other mail service.

My intraday heart rate information for January 1, 2017, from midnight to midnight.

Scaling upwards and scaling out

At this point, the adjacent stride is to get ALL THE Information. You would probably want to write a for-loop over all bachelor dates, and all available data types. The main problem y'all volition come across is that FitBit just allows y'all to asking 150 parcels of data per hour. This seemingly perverse rate limit may feel frustrating, simply is a common practise in public APIs — we volition but accept to piece of work around the throttling. Insert the cute flim-flam below into your for-loop to show a live progress-bar while the figurer waits for the adjacent hour to strike:

          from tqdm import tqdm
import time          if response.status_code == 429:
            wait = int(response.headers['Fitbit-Rate-Limit-Reset'])+thirty            
            for seconds in tqdm(range(wait)):
            time.sleep(i)        

If yous accept a heart-rate enabled fitbit (like a Accuse 2 or Alta HR), then you lot will have access to at least three intraday data streams: 1) steps, two) heart rate, and 3) sleep. If yous have, say, 2 years worth of these data, you will end up with thousands of files:
ii years * 365 days/year * three streams/day * i file/stream = 2190 files

…assuming y'all save each day and each stream's unabridged json information to its own file. You can only download 150 files at a time, so it will take y'all about 15 hours to get all 2 years of these information. The FitBit documentation does provide queries for more days at a time, so you could experiment with those alternatives if you are — rightfully — impatient to wait 15+ hours.

Ancillary data

You lot may also want other types of data, such as fitness activity logs, their optional GPS-enabled TCX mapping files, weight logs (assuming you have a fitbit Aria, or you manually log your weight), and more. I will not go over how to download these files, but the methods are similar enough and well-documented on the FitBit Web API.

Recap and outlook

How realistic is it that the average fitbit owner would be able to do all of these steps? Not very likely. All the same, the boilerplate fitbit user likely gains most of the bachelor insights from the very good fitbit App and web dashboards. The raw fitbit intraday data are fascinating, simply gleaning insights from them requires some reasonably avant-garde data science skills, or at least a lot of dedication.

hottingerhishmithad.blogspot.com

Source: https://towardsdatascience.com/how-to-download-all-of-your-raw-fitbit-data-d5bcf139d7ed

Share this post