Never used the command line & where is the light?
In our April session we had a great talk about light pollution and a hands on tutorial about how to use GDAL/OGR via the command line. We started with our guest Helga Kuechly (@helsche), a Berlin & Potsdam based geo-ecologist. She explained to us, “where the light is“. Helga showed what nighttime satellite imagery can be used for and gave an introduction to her research about light pollution.
We uploaded here slides to our new 😃 slideshare account: https://www.slideshare.net/MaptimeBER/where-is-the-light-helga-kuechly-maptimeberlin
Several interesting nighttime image data sources and visualizations were mentioned in her talk.
Helga for example demonstrated a comparison between lamps in Berlin. Even nowadays, decades after the fall of the wall, clearly the former east-west division of Berlin can be seen from sky.
Very impressive is NASA´s black marble visualization. Black marble images can be found at: http://www.space.com/19049-black-marble-earth-night-photos.html
For anyone who planes to work with nighttime data, Helga recommend some very interesting URLs:
- Berlin Mosaic 2010: http://dx.doi.org/10.1594/PANGAEA.785492
- DMSP & VIIRS data: https://ngdc.noaa.gov/eog/download.html
- Nightpod ISS data: https://eol.jsc.nasa.gov/
-Loss of the night app: http://www.verlustdernacht.de/Loss_of_the_Night_App_engl.html
It should not be unmentioned, that she and her joint researchers did some interesting publications in the field of light pollution:
Light pollution literature database: http://darkshy.org/resources/research/alan-database/
1. Kuechly, H. U., Kyba, C. C. M., Ruhtz, T., Lindemann, C., Wolter, C., Fischer, J., & Hölker, F. (2012). Aerial survey and spatial analysis of sources of light pollution in Berlin, Germany. Remote Sensing of Environment, 126, 39–50.
2. Kyba, C. C. M., Garz, S., Kuechly, H., de Miguel, A. S., Zamorano, J., Fischer, J., & Hölker, F. (2015). High-resolution imagery of earth at night: New sources, opportunities and challenges. Remote Sensing, 7(1), 1–23.
Helga works for the LUP GmbH (http://www.lup-umwelt.de/en/forschung-entwicklung/) located in Potsdam and as a freelancer, e.g. for NGOs like WWF, in the field of nature conservation.
Next to Helga Harald Schernthanner (@hatschito) and Nico Bellack (@bellackn) did a hands on tutorial about the command line in general and its great use, when combined with GDAL/OGR, the geospatial data abstraction library, which is probably the most well known program library when it comes to geospatial applications. The word cloud below shows where GDAL/OGR is inside.
The final outcome of their tutorial was a heat map, showing all ice cream shops in Berlin, completely assembled just in the command line. Here you can find the slides of their talk: https://www.slideshare.net/MaptimeBER/gdal-ogr-intro2542017
They started with the very powerful wget command. A command to retrieve content from web servers. Wget was used to filter and download OpenStreetMap data. The following request retrieves all ice cream shops in Berlin via wget and the Overpass API: https://overpass-api.de. By the way, another great tool to filter the OpenStreetMap is Overpass Turbo: http://overpass-turbo.eu/
Next you can see the command to download Berlin's ice cream shops:
wget -O ice_cream.osm https://overpass-api.de/api/xapi?
node[amenity=ice_cream][bbox=12.93,52.45,13.71,52.65]
Here we explain point by point, how the command is constructed. In the slides you can find further commands, explained point by point.
1. Wget = command to retrieve files via HTTP / HTTPS / FTP
2. -O = write output to document
3. URL = Url to the Overpass API
4. [amenity = ice_cream] = OpenStreetMap map features:
https://wiki.openstreetmap.org/wiki/Map_Features
5. [bbox=12.93,52.45,13.71,52.65] = Bounding box for the request in Lat / Long lower-left(SW) to upper right (NE) coordinate
Based on this first example, they turned to GDAL/OGR and at the end our we had our first shell script. Here you can see and copy our final small script for your own purpose. Feel free to copy & paste 😃.
######################################################
####################################################### #Shebang#!/bin/bash
#wget download data via the Overpass APIwget -O ice_cream.osm https://overpass-api.de/api/xapi?node[amenity=ice_cream][bbox=12.93,52.45,13.71,52.65]
#Convert .osm to shape fileogr2ogr -f "ESRI Shapefile" ice ice_cream.osm -explodecollections
#Reproject to UTM 32ogr2ogr -t_srs EPSG:25833 icecream_shops_UTM33.shp ice/points.shp -overwrite
#Grid the datagdal_grid -l icecream_shops_UTM33 -a count:radius1=1000:radius2=1000:angle=0:min_points=0:nodata=0 -ot Float32 icecream_shops_UTM33.shp ice_cream_shops_4000_4000_mean.tif
To initiate the script, a so called Shebang has to be written: #!/bin/bash
To execute the script, store it as .sh file and run it in the terminal via the command bash + scriptname.sh Finally, we get an almost automatically derived ice cream heat map of Berlin. Just some styling had to be done in QGIS.
Our Maptime ice-cream shop heat map
To round up our session, Nico briefly showed a web-based GDAL application prototype, he built during a study project at the University of Potsdam, automatically interpolating air pollution data of Berlin‘s BLUME sensor network. Below you can see a flowchart of the work-flow of his prototype:
The data from the Blume network can be downloaded here: http://www.stadtentwicklung.berlin.de/umwelt/luftqualitaet/luftdaten/index.shtml
Harald works in the field of GIS and remote sensing at University of Potsdam and co-organizes MaptimeBER. Nico is a M.Sc. Student in the field of geoinformatics at University of Potsdam.