Discover Top Posts Tagged with #topojson

Signori si vota

Dopo un’interminabile campagna elettorale è arrivato il momento di votare per il referendum costituzionale. E quindi di gestire la presentazione dei dati del voto in real time sulle pagine del sito del Sole 24 Ore: uno speciale con tutti i comuni di Italia, navigabile tramite mappe geografiche che si basano sulla libreria D3.js (e più specificamente su dati in formato topojson). Anche la testatina presente nella home page del sito dipende dal meccanismo di elaborazione dei dati, provenienti direttamente dal Ministero dell’Interno.

L’emozione di vivere l’evento dall’interno di un grande giornale è difficile da raccontare. Il susseguirsi delle notizie, gli exit poll, mantenere sotto controllo le diverse pagine (anche sul mobile site) che riportano affluenze e voti... e alle 6 di mattina, chiusura degli scrutini. Con l'adrenalina che ti tiene sveglio. Grazie a tutti i collaboratori per la professionalità dimostrata.

Nella pagina di accesso dello Speciale Referendum Costituzionale la mappa italiana divisa per regioni con i risultati del voto.

#referendumcostituzionale #sole24ore #dservicemedia #topojson #d3

Как сделать свою карту на D3js

Сегодня я расскажу вам, как с помощью JavaScript и d3 нарисовать карту, подобную моей.

#d3js #topojson #карты #javascript

An easy, lightweight path-animation technique inspired by the NYT

#d3 #topoJSON #maps #mapping #cartography #ddj #djl

DataMaps needed a grunt task to convert GeoJSON data into TopoJSON, so I whipped up this Grunt plugin to handle the task.

Source code available here.

#topojson #datamaps #d3 #geojson #maps

Making Geospatial Sausage

I question that people often ask about topojson is, what's the algorithm for creation the topology. Hell it's a question I asked. Here as best as I can manage, is how the sausage is made for topojson, warning, lots of JavaScript to follow.

First a refresher on GeoJSON, the following is from the spec:

https://gist.github.com/calvinmetcalf/6373239

Before we get into topojson, a disclaimer, at most one of your reading this is Mike Bostock, the rest of you, DON'T CODE IN THIS STYLE it is tricky to following, easy to make mistakes in and it wouldn't be close to passing the lint step on any of my projects. That being said I can't really fault Mike he ships some incredible stuff very frequently, certainly more then me and as far as I'm concerned when you have the results he does you can code in whatever style you want, but until then use jshint.

Anywho TopoJSON uses this function 'each' a lot:

https://gist.github.com/calvinmetcalf/6373252

which references this type function which I am linking because it's long. These two objects are in many ways the bread and butter of TopoJSON, types takes an objects containing functions for what to do to the input geojson, so this bounds function

https://gist.github.com/calvinmetcalf/6373287

specifies what to do for each point so it can compute the bounding box..

One of the things that TopoJSON does is quantizes the input, this more or less like rasterizing the input coordinates. settings the quantization factor to 1e4 is another way of setting it to 10,000, which says set up my coordinates to be a box 10,000x10,000. Then remember that bounding box we just computed? We can now set it up so that whatever our minimum x and y are equal to 0, and the maximum ones to 10,000 and scale everything in between accordingly,

https://gist.github.com/calvinmetcalf/6373341

which means that you can quantize a value by subtracting the minimum, and then dividing by the scale value, this is then rounded to the nearest integer. This is an important part of the algorithm for finding arcs that match because it means that the arcs are simplified in a way that they will converge on the same values. Especially with this next function which looks for lines that go through the same point multiple times and removes the dups

https://gist.github.com/calvinmetcalf/6373367

Do you see how it works? Of course not, I haven't explained the 'coincidences' function there. Its some sort of homebrew hashtable which I think allows easy access to sets based on their members, it's also used to find arcs by the lines that make up them, and lines by the points that make up them. I haven't fully wrapped my brain around this one yet.

As an aside, the features get looped through 4 times plus more times before the function is called, i.e. if you want to simplify it. The minimum number of loops is 2 if you are able to supply a bounding box yourself or 3 if you can't. Before the next steps can happen the coincidences have to be dealt with in the above function. That function relies on quantization which needs a bounding box. So in theory if you had a bounding box you could do one loop that simplified, quantilizied and found coincidences, and then loop again with the bellow stuff.

We then have another big each call, which does a lot of stuff including transforming properties, the relevant part that changes the geometries is here. Which calls two functions lineClosed and lineOpen which are just shortcuts to line.

If were were on cribs, this would be the bedroom.

https://gist.github.com/calvinmetcalf/6373449

This is the difference between open and closed lines, in geojson a polygon is closed and thus repeates the first and last point. Next it tries to find matches in the line segments and if it can't it rotates any loops so you start from the bottom, it helps to remember that 'a' is the in progress arc, 'n' is the length of the arc, but 'p' and 't' are either the arc, or all arcs depending on where you are in the function,

https://gist.github.com/calvinmetcalf/6373459

Then we dedup points and finish up here

https://gist.github.com/calvinmetcalf/6373541

i'm going to be honest the previous section of code only makes sense to me around 11am when I am so caffeinated I have a speech impediment. This section checks whether a point should be added to a current arc or be the start of a new arc, this makes somewhat more sense in the python port

The actual arc creation is relatively straightforward

https://gist.github.com/calvinmetcalf/6373554

The bulk of it happends in the matchForward and matchBackwards functions which return true if the arc is a dupe and take care of putting it into the object. This is by no means the only way to do this portion, the python port uses a hash table here (not the one used for point and arc concurrences, that one has is a very different type of hash function which will collide when used on arcs for this)

The last steps of assembling the Topojson and delta encoding the arcs happens here

https://gist.github.com/calvinmetcalf/6377118

Note the map function where the actual delta encoding occurs.

So to sum up, first I really wouldn't consider that there is an algorithm for deduping the arcs because; A: It's a heuristic and I'm that kind of asshole and B: It isn't one discrete thing, it's several steps each one relying on the one before but also modular and replaceable. To summarize those steps are:

Quantize the points, I hesitate to actually include this as it is optional for deduping the arcs, but it does help by increasing the changes of a match being found by removing the cartographers worst enemy: points closed very close together but not exactly in the same spot.

Find the concurrent lines and make sure we break up our arcs at the same point, no matter how good our arc matching algorythm is it'll fail to match the arc a->b->c to the arc z->a->b->c->d. For this a homebrew hashtable is used.

Find the arcs that are matching.

From the man himself:

linearize (extract lines & rings)

cut (at junctions)

dedup (canonicalize duplicate arcs)

I'd like to thank Mike Bostock for both writing this awesome library and making it available to the world as free software.

I hope you, the reader, are marginally less confused then before and seriously kids, don't use one letter variable names, always use curly brackets even when they are optional, and don't assign variables inside comparisons.

Checkout my python port and if you like using TopoJSON help us write the spec.

#gis #javascript #topojson #geojson

On Incremental Topojson

One of the main reasons I ported topojson to python was so that I could add topojson as a format to esri2open.

But after I ported it I found myself with a problem, esri2open works on each feature one at a time writing directly to disc. The reason is simple, to avoid caring about the size of the data set, when you write directly to disc like this if doesn't matter how many features there are, you could write out a terabyte sized geojson file.

The original node.js topojson parser doesn't work like this at all, it loops through the input data 4 or more times depending on the options. From the looks of it it seems you need to loop through at least 3 times, though if you know the bounding box in advance you can skip one loop.

So if I have to loop through it multiple times I need to at least limit the amount of data I have in memory at once, which brings me to one of the nice things about python, a grotesquely large standard library which includes like 7 database formats. This means I can use the database to store the features I'm not using and use python generators when I'm done to prevent to much getting into memory while writing it to disc at the end.

You can grab it on the incremental branch of topojson.py or from this branch of esri2open.

Currently not working well on windows. Currently not working well on windows when tested exclusively on gigantic datasets.

Going forward the incremental version will probably live separately from the main one as it can only handle a subset of the topojson spec (coincidentally the subset that can be exported from desktop GIS software)

#gis #python #topojson #geospatial