I’ve been meaning to spend some time looking at the various mapping APIs and find out exactly what’s involved in the basic dots-on-maps implementations we see around the web on an increasingly frequent basis. This is a brief account of how I got on messing with Google Maps, a dataset of fuel stations, and some microformats. (Short attention span? See the result.)
At the London JavaScript Evening last month, we saw a talk from Tom Carden about Mapstraction, a mapping abstraction layer that will talk to the mapping APIs from Google, Yahoo!, Microsoft and eventually OpenStreetMap. That sounded pretty useful, as it would enable an implementer to switch providers if necessary without rewriting code. Conceivably, you could even switch providers on-the-fly if your primary provider was running slow or timing out. Neat. Anyway, I didn’t use that. I first wanted to get down and dirty with a map to see what was really involved. I chose Google Maps to work with, as it’s by far the most commonly used. I also had some rough idea of what was capable from a user perspective, so it was a comfortable choice.
Coincidentally, around the same time my friend Chris had bought himself a nice old Land Rover that had been converted to run on LPG gas fuel. Refuelling stations for LPG (or Autogas) aren’t all that common in the UK, at least compared to the options for regular petrol and diesel – and the official consumer site for LPG promotion – BoostLPG – only offers a list of refuelling stations as a Word or PDF document. Chris and I both thought that was a bit rubbish, as not only is it hard to use and to spot geographically close stations, it’s also not possible to load that data into an electronic device in a useful way. We figured it’d be fun to take the data and represent it in a way that could be mapped.
After some hard work extracting the data from the Word document (top tip: Word has a Convert Table to Text option which will give you a delimited representation of the data) and imported into a MySQL database, the next step was to get the address information into a form that could be plotted on a map. Google Maps accepts map points as latitude and longitude co-ordinates, whereas we had some mangled street names and postal codes. The process of converting one to the other is called geocoding and is very easy for US addresses. Yahoo! has a Geocoding API you can hook up, as do others. In the UK it’s less straightforward, as the source data isn’t so freely available, despite the government organisation that owns it being funded by tax payers. (Yes, the British public has paid for this data to be created, and then is being asked to buy it back to use it. It’s a scandal.) Fortunately, I eventually found a free UK geocoder that did the job nicely.
Once you have your points as latitude and longitude co-ordinates, the mapping part is really trivial. You have to sign up for an API key (which is tied to the URL you’ll be developing from, but you can grab as many keys for as many URLs as you need), and Google give you a sample HTML document and a comprehensive API reference to work with. If you’re comfortable with DOM Scripting, it really is child’s play to get the map centred where you need it and plot some points.
A problem I quickly ran into is that there are around 1200 LPG fuel stations in England, Scotland and Wales, and plotting 1200 points on a map isn’t exactly a quick process. In fact, it tipped that very crucial point where Firefox displays a slow script warning dialogue, giving the user the option to terminate. I’ve heard bad things said about that dialogue box, but as far as I’m concerned it’s a very good and important feature – it guards against idiots like me taking down users’ browsers by doing something stupid like trying to plot 1200 points on a map. I quickly concluded that I’d need to break my maps down by county, so that I was only dealing with a subset of stations at a time.
So I had my proof of concept – I could very happily plot a few dozen items on a map. However, to turn this into production code rather than just a toy, I needed to work out how I was going to write a script to accept a dynamic set of data. I chose to write the items onto the page marked up with the hCard microformat (which contains fields for the geo information needed for the map) and then use JavaScript to iterate through the hCards on the page, extract the data and plot the items on the map.
The nice thing about this approach is, taken to the next level, I could have written by script to parse literally any hCard in the page, making my map code pretty portable. In this instance I loosely tethered it to my specific implementation for brevity, but I needn’t have done. Playing by a set of known rules for data structure makes the code a heck of a lot more portable and easier for others to understand.
Anyway, I ended up putting this online as a tiny site called Get LPG. By passing any detail page through Brian Suda’s X2V (I’m using the Technorati hosted version), users can download the list of stations into the address books on their phone or PDA for reference on the move. I also dabbled with producing GPX files (containing GPS waypoints). It’d be great to see a geo2gpx script for parsing the geo microformat into GPX files – perhaps I should have a stab at that at some point.
After I’d done all this, I remembered Jeremy’s Adactio Austin which does pretty much the same thing. Annoying that, as I could have pinched some ideas from Jeremy and saved some time! Ah well.
Comments
Nice work Drew and a pretty handy site you’ve created there.
“the British public has paid for this data to be created, and then is being asked to buy it back to use it. It’s a scandal.”
The Guardian agrees with you. http://www.freeourdata.org.uk/
I had a similar problem with a large number of points. Way down my to do list is to solve the general problem, by having some sort of load on demand for the markers. My short-term solution was to split the UK into a three by three grid of squares. Most one square was sea, so it wasn’t a great solution. Unfortunately the data I plotted was personal, so I can’t reveal it online.
One interesting challenge I had was centering the map for each county. In the end I took a not-very-scientific approach of using a mean average of the points I was plotting. It’s not perfect, but it does tend to give you a map with lots of points in view, and with this dataset, points tend to cluster themselves around major towns, which is a good visual reference point for users, so it works.
One of the issues I can see with the way you’ve broken it down is if like me you live on the boundary of one of the areas. It means you can’t see the “cluster” around you. Would a better solution not be to ask for a post code and plot a 50 mile radius around it? Or is that not possible with the data at your disposal?
Yes, that would be a nice feature, and probably possible with the data I have. But who has the time? This was just a chance to try out some mapping – my goals were primarily selfish.
Hehe! Know the feeling well… When I get an LPG car I’ll come after you for the source code. ;)
One problem with the vCard download is that although the vCard contains multiple contacts, Outlook will only extract one to add to its address book, making the vCard download pretty pointless. Has anyone ever found a solution to this? Do other address book apps have this problem? For this kind of thing to be truly useful, we need to be able to import vCards containing multiple contacts (is Microsoft listening?). Come to think of it, Thunderbird needs to be able to import vCards too.
Hey Drew – nice work. Since when has Taunton been in Bath, though? :D
One other thing – have you told BoostLPG about the site to get a bit of linklove?