This past weekend I participated in the NASA SpaceApps challenge with two classmates, Andres and Brian. We chose the Earth Day Challenge as part of the Data Visualization category. I'm so late writing this because right after I slept for 15 hours, woke up, and studied for two tests the next day on Tuesday. After my tests, I studied for another exam I had today.
Today I am relaxing. I'll do some programming problems on Hackerrank or Project Euler and work on the PortalFace Project for my web engineering class, but I just wanted to share my experience at the competition before it is no longer fresh in my mind (arguably already too late).
I have never competed in any sort of coding competition in my life, I never thought that I was good enough to really contribute to a team let alone go it alone but Andres convinced me I should work with him and two friends at NASA's challenge (one of them was a designer who did not show up, I ended up having to do his work as well).
NASA had five categories (Citizen Science, Creative, Software, Hardware, Data Visualization), and we met before the competition to look over the ones we thought would be the coolest to work on. We decided on a remote LEGO robot control system, renewable energy potential mapping, making a near earth object database, and our earth day challenge.
The challenge was to show how the earth has changed over time with things like air quality and climate change. Our original idea was to have a globe you could spin that would have a slider and drop-down to select the data you want. When we examined it realistically we decided to start with just the US land area and then if we had time move on from there. This turned out to be a really good decision later on.
The first SIX hours were spent looking just for data. Government and really any form of public database is extremely hard to navigate, and even when you find the data you have to get access to it. I ended up finding the primary data used in our project (NOAA temperature, rain, and snow), but they took forever to get it to us (we had to submit a request and did not get access until 4 hours before time was up). Even when we got this data we had to write scripts to format the data for our site.
The site would be built entirely on an R framework called Shiny. We knew we would be using R for the data visualization itself but when Brian (he worked a lot in R on his co-op at Google so he was our expert) mentioned there was a web app framework we could use we were ecstatic, because this would set up our work structure as Andres and I mining and formatting data to throw over to Brian to format while he set up the framework. After we got data we would then move to building the rest of the site (it had bootstrap built in!) while helping Brian the whole time (he hardly needed it so this was good).
We ended up staying up the entire 26 hours, and learned many things. Most importantly we learned there was a reason why no one else competed in our category locally (manipulating data is hard). We naturally took first place in our category, with a partially finished app that accomplished a lot but had a ways to go. We managed to get temperature, rainfall, and snowfall by county working from 1990 to present but fell short of getting air quality (I got the data but we did not have time to include it). We were also supposed to display data from 1970-present but when we tried to run the 4,082,000 lines of data through R with a quad-core, 16gb RAM, and 40GB swap, we had to kill the program or crash the computer (which we did not want since it was running our server and we were working on it remotely).
The app looks decent but is not complete, especially since we did not find an online hosting solution. You can check out the project page at climatetimemachine.com which will also have a link to the github repo for our code.
I suprised myself at what I could contribute to a team in such a fast-paced environment, and I will now start to actively seek out more experiences like this in the future.