Gav's Blog

My friend Ajay has prompted me to create this post, and I’ll try to add to it over time as more cameras with inbuilt GPS. Why is inbuilt GPS important? Well it takes all the hassle out of geotagging photos. As you may have read in some of my previous posts, geotagged images are really useful for Emergency Management.

Whilst there are plenty of solutions available, I’m not going to provide the post-processing options here. I only want to record those that embed the co-ordinates at the time of taking the photo. No products that require post-processing are included.

Digital SLRs inbuilt GPS

• None yet, but maybe this year, there are rumours the Canon 60D may have inbuilt GPS.

Digital SLRs with Accessory

• Nikon D5000, D90, D300, D700, D3,D3X – requires  Nikon GP-1 accessory or a di-GPS
• Canon EOS 40D, EOS 50D, EOS 5D Mark II, EOS 1D Mark III, EOS 1Ds Mark III – requires Canon Wireless File Transmitter accessory (model is dependent on camera body). More info on Canon GPS, and manual page. It additionally requires a third party GPS as well that can provide position using the NMEA protocol.

Point and Shoots

• Nikon Coolpix P6000
• Ricoh 500SE
• Leica V-Lux 20 – as well as having inbuilt GPS, it also appears to have an inbuilt database of 500k points-of-interest that can be used to show nearby sightseeing opportunities (added: 2010-04-22)
• Sony DSC-HX5V Digital Camera

GPS Receivers with Camera

• Garmin Oregon 550, 550T

Mobile Phones with GPS

• Apple iPhone 3G, 3G S
• Nokia – a number of models that I’ll list in due course

Survey Quality solutions

• ikeGPS

Frankly, Nikon appear to have to produce a far smaller and lighter GPS solution for their cameras. Canon requires not only a bulky grip, but still requires a GPS to be added as well. The Nikon GP-1 or di-GPS look to be far more appropriate for field work for emergency management. Additionally, the Nikon solution can be used in conjunction with a battery grip, which allows additional batteries into camera (two, instead of the usual one). The Canon grip increases the bulk of the camera, but the camera itself cannot use a battery grip to extend the battery life in the field. Of course additional batteries can still be carried and swapped – although to replace the camera battery, the WFT must be removed. Finally, the WFT3/4 also require their own battery to operate.

As a Canon user, I am most disappointed with their solution to geotagging-at-shutter-click, and the Nikon approach appears far superier as it adds very little bulk to the camera, and doesn’t get in the way of using a battery grip to double the life of the camera without changing batteries.

This is a brief product announcement I provided to The Box, it was published on Tuesday the 2nd of June, 2009. It is archived here for my records.

Garmin announced their latest high-end hand-held outdoors GPS unit recently, and the headline feature is the inclusion of a 3.2MP camera with built in geotagging. This means that any photos taken with the unit will be instantly plot-able on maps, and will be a convenient tool for people enjoying the great outdoors and travelling. Other nice upgrades to the flagship Oregon line include support for a 3d electronic compass for accurate bearings when standing still; increased storage for waypoints, tracks, routes; capacity for a massive 5000 paperless geocaches; and fast USB 2.0 transfer when connecting to a computer at last.

Garmin Oregon 550 Product Page

This is a copy of an article I had published in The Box on Tuesday, the 26th of May 2009. This is a copy of my originally submitted text, an archive for my my records. It is a topic that has seen a bit of interest. I’ve also been interviewed on National Radio’s Panel on this topic. I will probably be writing a more detailed article about the problem in due course.

A recent report from the US Government Accountability Office has identified possible trouble ahead for the Global Positioning System (GPS). Due to governance failures of those responsible for the GPS, there is the risk that satellites may not be able to be replaced faster than the rate at which they fail over the next 10 years. Whilst replacement satellites have been ordered and developed, some technical and project management issues have delayed the launch schedule, with the next launch planned for November 2009. This doesn’t mean that the GPS will just stop working. There are currently 31 active satellites in the GPS constellation – only 24 are required for the agreed level of service. There are 13 satellites that are more than 12 years old, and are increasingly likely to fail. This happens as the solar panels age, and they produce less electricity to power the satellite. There are options for extending satellite life by turning off less critical secondary payloads that draw less power. A few satellites can fail without having a significant impact on end users. If the number of active satellites drops to 24 or below, GPS receivers will probably be less accurate as fewer satellites would be visible at any given time. Given the GPS is a strategic military asset for the US, it is highly unlikely it will be allowed to fail completely. This may drive innovation in GPS receivers to support multiple satellite navigation systems to reduce reliance on a single system.

What seems like a lifetime ago, I wrote my initial thoughts on the Garmin Colorado after having owned it for a few weeks. Now I’d like to return and offer more definitive thoughts on it after nearly a year of fairly heavy use. In this time it has been with me on somewhere around 1000 cache hunts, and a few overseas trips – mostly to Australia, but also Singapore and Sri Lanka. Naturally, it has seen a fair bit of domestic usage in little ole New Zealand as well.

I’m not going to spend any time going over what the Garmin Colorado is – there are plenty of articles on the net. I want to focus on the big pro’s and con’s associated with the Colorado. (Many of these comments may or may not also apply to the Oregon, I haven’t had enough hands on time with one to see what improvements they made from the Colorado.)

Made of Win
Win 1 – Stunning Maps. The screen and map display on the Colorado is absolutely gorgeous and blows away all of the previous generations of Garmin mapping handheld GPS units. The clarity of of the mapping display and the ability to clearly represent more layers at the same time is just great. I really struggle going back to older mapping units like the 60 series with far lower resolution and clarity in the maps.

Win 2 – USB Storage Device. Whilst it doesn’t sound like much, the change in metaphor for sending/receiving information from the GPS is a huge plus. You can accomplish most things purely by mounting the GPS as a USB device, and copying/deleting files from the GPS. No longer is it necessary to have Garmin GPS drivers to be able to communicate with the GPS. This is incredibly handy for travelling. If you prepare a whole lot of geocaching GPX files before you go, you can just store them on the device, and as you move around delete the active ones you are finished with, and store the others under a filename that the GPS won’t recognise e.g. use .bak or similar. Then, just rename the file from .gpx.bak to .gpx and next time you boot up, the GPS will load the new geocaches. This works the same for maps when travelling from country to country. If you have enough storage space to store multiple countries .img files from OpenStreetMap in the main memory, then as you move from one country to another, you can just rename/delete the old file, and rename the next country you’re heading to as the current gmapsupp.img – it works very well. Compare this with the alternative of either creating a massive gmapsupp file with all the countries you are travelling to, or having to take MapSource/RoadTrip with you and reloading maps as you go.

Win 3 – Paperless Caching (PC). The basic paperless caching functionality works pretty well. Two thousand caches is a far more reasonable limit than 500/1000 waypoints. The cache specific icons are great and make it so easy to see the types of caches at a glance on the map. The field notes work really well and speed up the logging of caches no end. It is very useful having the cache descriptions and logs, and we used this feature well on the road when figuring which caches to do next on a road trip.

Win 4 – Roller. What can I say, it is absolutely brilliant for zooming in and out on the maps.

Win 5 – Mount. The new mount connecter is very solid and works well on bikes and car dash mounts.

With that said, there are some areas where the Colorado let’s you down fairly significantly.

Epic Fail
Fail 1 – Archived Tracklogs. The 60 series had a near perfect means of archiving tracklogs. You could set it up to record to the SD card, and the unit would create a file for each day. Never, ever, did the GPS unit end up writing over already recorded tracklog data. Sadly, the same cannot be said for the Colorado. The archive track capability of the Colorado will only store 20 GPX files of approximately 1MB/5k trackpoints each before it will start overwriting the archived tracklogs. As a keen OpenStreetMap track logger, and also someone that wants to keep tracklogs from travel so that one day I can geotag some of my photos, I was livid the first time that I have found the Colorado was overwriting archived tracklogs. I really liked the 60 series handling where you could leave it going, and just trust that you come back and once every couple of months copy of all the archived tracks. Not so with the Colorado and it is one of my biggest frustrations. This means that archiving tracklogs becomes a hassle and a chore. This is such a big dealbreaker to me I am now considering what my next GPS is going to be. I want to be able to travel for weeks, have everything archived AND HAVE ABSOLUTELY NOTHING DELETED WITHOUT ASKING ME FIRST. Why can’t the system save tracklogs to the external SD card just like the 60/76 series do?

Fail 2 – Paperless Caching. Whilst PC has been extremely successful for traditional caches, there are many, many failures and oversights. Consider that earthcaches don’t have their own icon, and for some reason Garmin has seen fit to default them back to Traditional caches, when at a minimum they should be Virtuals. How hard is it really for Garmin to update the software to display them with their correct icons, or at the very least as a Virtual? Why, are all the child waypoints associated with a cache loaded as actual waypoints, rather than being linked to and accessible as part of the cache? All they do is take up the waypoint allocation, and they aren’t easily accessible from a cache description. Why does the list of nearest geocaches not show the cache type icon – instead you have to click through each cache to see what type it is.

Fail 3 – Lack of user-centric design. I’m not sure how Garmin tested the Colorado, but there are some silly design decisions that make no sense at all. When changing the location of a waypoint (e.g. editing the co-ordinates to enter a waypoint for a multicache) you can only see 10 of the 20 characters at any time. This makes it very frustrating when copying co-ordinates from a cache. The only way you can see them all at once is if you save the waypoint, then find it, and view the co-ordinates on the screen before selecting ‘Go’. This is an impractical solution. This is perhaps my most frustrating experiences, but there are others, and these point to a poorly designed interface that doesn’t appear to have any real thought around the development. How hard would it have been on the lat/long entry screen to have both the latitude and longitude visible at once? There is certainly enough screen real estate to support it.

So, what does this mean?
To me, I think there is something inherently flawed in Garmin’s design process. A number of these usability errors SHOULD have been picked during testing before release. One wonders if Garmin skimped on the end-user testing and rushed the Colorado to market. The Colorado has the feel of Windows Vista about it – some really nice new features, but some absolutely fundamental design errors that are real deal-breakers.

Don’t get me wrong, I’m not about to sell the Colorado and move back to a 60 or 76 series – I just couldn’t handle looking at the maps on one of them again, or losing the paperless caching. But, more than ever, my recent experience with losing archived tracklogs on the Colorado has got me thinking about the best approach to paperless caching and decent tracklogging – what is the next step.

Perhaps it is a fundamental mistake by me, thinking that I can have a single GPS that does everything I need it to. Even now, I still need my iPhone with caching details in addition to the Colorado. Perhaps I need to use a simple tracklogging GPS unit such as the Wintec G Rays 2 for logging tracks and not rely on the Colorado for this – it after all will automatically delete the archived data (which I don’t think is mentioned in the manual, just on the unofficial Colorado wiki).

The iPhone may end up being my saviour for paperless caching. Currently, we are tied to Garmin’s model and method of paperless caching on their units – you either use their approach, or you use another piece of hardware.

I’m starting to think that perhaps I can go the other way. Perhaps I should focus on looking for paperless caching capabilities in the iPhone. After all, all we need is something like GSAK/GPXSonar for iPhone, with an inbuilt map viewer that understands Garmin img map files, and that solution would likely blow away anything that Garmin could come up with (based on my 11 months with the Colorado). Sure, you could still use a basic eTrex high-sensitivity unit with caches loaded as POI’s, and take all the intelligent processing to a device capable of acting in an intelligent manner.

Garmin has had at least 12 months to improve the capabilities of the Colorado via software updates. All these issues, and hundreds more have been identified in the Garmin Coloroado wiki. Yet few have actually been resolved.

The core problem here comes down to Garmin being a hardware vendor, and they are still locked into the thinking that once the hardware is sold, only critical bugs will be fixed, and everything else will be left to the next generation of units – so you have to purchase new hardware to get a real upgrade.

This is in almost complete contrast to the Apple iPhone. With the iPhone, the software sees regular updates, and there have been some significant updates to old hardware. For example, the 1st generation iPhone and iPod touch were able to be upgraded to the v2 OS for either free or minimal fee, but the user received significant new capabilities. Come this June or July, even the original users of these devices will be able to upgrade to v3 of the OS and reap nearly all the benefits (yes, 1st gen. iPhones don’t have A-GPS, so the users will miss out on that – one of the reasons I didn’t get an iPhone until it had GPS).

On top if this, the iPhone allows developers to create competing applications. This means if you are so inclined, you can write a caching application if you choose. Whilst only geocaching.com will be able to have the tight integration with their website, I still believe there are possibilities for developers to deliver some stunning paperless geocaching applications that will blow away any capability that a hardware vendor such as Garmin appears to be capable of developing.

The software approach means that applications are sometimes updated every couple of months for existing users, and this is an area that Garmin has perhaps had their biggest fail. Garmin has failed to improve the utility in their existing units by fixing minor issues that the community have identified, and they are not releasing software updates as frequently as they should be. Any Garmin release generally addresses only a limited number of issues, especially when compared to the large number of outstanding issues identified by the user community.

This has perhaps been my big lesson with the Garmin Colorado. Garmin don’t appear capable of maintaining and improving unit capabilities once the product has been sold – certainly not when compared to iPhone updates. If Garmin keeps up like this, they are going to be made increasingly irrelevant by programmable devices like the Apple iPhone that will deliver extremely powerful paperless caching applications in spades.

The corollary, of course, is that if Garmin want to continue to deliver more complex and software-driven units to the market, they need to be far more responsive to the issues raised by the community, and provide more significant software updates.

The summary of course is that I’ll continue to use the Colorado – it is after all a sunk cost for me, and it does do some basic things pretty well. But am I seriously going to be keeping an eye out for alternative solutions that are more responsive to user feedback. I’m hoping that by sharing my experience with the Colorado, it will get you thinking about your choices more too.

I’ve been mostly quiet on the geospatial front recently as I’ve been busy with work, but there are a couple of things worth commenting on.

1. Environment Canterbury GIS Beta blog. Not the first to the council GIS blogging game, but certainly early, is my local regional council in Canterbury. They’ve started a blog to engage with the community, and built their first mashup. This is a great step, and I’m excited to see more of the public GIS practitioners provide a means of direction communication and engagement with the communities they represent. Very exciting.
2. A reminder that whilst we’ve still got a long way to go, things could actually be worse.

Getting back to mashups, I believe that they are a nice simple and cost effective means of communicating simple geospatial needs to users – ECan’s swimming water quality mashup is an example of that – a simple means of seeing visually the quality of swimming places around Canterbury.

However, the real benefits are going to come when that information is freed from the encumberance of being tied to an Internet connection. Lots of recreational activities that we undertake are not always near Internet connectivity, and mobile data access is still prohibitively expensive to be checking these sort of data sources over summer when away from your computer and Internet connection.

As a very keen recreational user of handheld GPS for geocaching and other activities, I am ever hopeful that councils such as Environment Canterbury will start to consider making some of their underlyign data available for download so that it can be bundled into offline maps that can be used in car, auto and mobile phone, GPS devices.

This will complete the full loop – councils create our communities (geospatially at the very minimum), they record geospatial data about our communities, and with mashups – we are finally starting to get some of this geospatial data back.

I look forward to the day when councils are extremely open with the geospatial data and make it available so that we can put in our choice of portable mapping device

Back in July, I posted about dc.gov releasing some data. I was a bit slow replying to a comment made by Nat Torkington then, and felt that a reply actually required a new post to elaborate further on why I’m so supportive of governments – be they local or national – releasing data that has been paid for by the rate/tax-payer. Nat said:

“Isn’t it the case that the USA doesn’t have an authoritative roading database, either? That’s why Navteq, TeleAtlas, and Google have to drive the roads.”

Whilst the US doesn’t have an authoritative roading database either, the release of the TIGER line shapefiles has spurred the development of free and open maps – e.g. the inclusion of Tiger data in OpenStreetMap, and the production of free and open maps for GPS units. This mirrors what has occurred in New Zealand with the likes of the NZ Open GPS Maps project, utilising the free information made available from Land Information NZ.

However, this leaves us with two broad types of maps both with their problems – commercial datasets with restrictive usage conditions and free datasets maintained by volunteers that may not be sustainable in the long term. In New Zealand, the commercial dataset providers are primarily Terralink, Critchlow’s and Eagle Technology, with some more affordable sets made available by Kim Ollivier. The free maps are primarily catered for by the New Zealand OpenStreetMap project and the NZ Open GPS Maps project.

My problem is that there is a lot of inefficiency in the current way that mapping data is managed in New Zealand (and this probably applies internationally). Why do we have four+ commercial sources for roading data and two volunteer driven projects all duplicating each other, as well as Government agencies that have legislative responsibilities for roading infrastructure?

Well, it is because LINZ is not currently funded to provide a centralised repository for all this information – they are too busy focusing on the cadestral database where they make their money. Instead we are producing inefficient silos of information, that are all subtly different. I have been prodding at a few people to try and get the NZ OpenStreetMap and Open GPS Maps projects to try and consolidate the underlying database to OSM, and I believe that this will occur over the long term, but there are a number of issues to work through before this will happen.

As Nat indicated in the original post – in the US Navteg, TeleAtlas and Google drive the roads there, and we’ve got at least Terralink, Google and probably others driving the roads here. In addition we have active volunteers also driving roads and correcting errors in OpenStreetMap and the Open GPS Maps project – I personally provide GPS tracklogs to OSM, and have also placed the 2007/8 High Speed Data Survey in there. The interesting part is that all of the errors are being corrected from the original LINZ roading dataset. So, because the New Zealand Government has not funded LINZ to maintain the roading dataset, make it widely available under permissive licensing terms, and allow feedback and corrections to be suggested for review and possible inclusion, we now have a massively inefficient approach to mapping roads in New Zealand.

All of these projects have sprung up because LINZ is not funded to provide the correct road dataset in the first place.

We can’t support that in a small country in New Zealand where only corporates, local authorities, and central government agencies can afford the commercial roading datasets due to expense. I know at least one of the commercial datasets costs over $100,000 to license. What this means is that small-and-medium sized businesses are being left out in the cold from using geospatial information to improve the way they do business as it is too expensive, and rate/tax-payers do not have affordable access to the information for tourism, recreational and safety purposes.

As the Immediate Past President of the NZ Recreational GPS Society, I’ve seen people balking in our forums at having to pay extra for decent road or topographical maps. Some of these are expensive because the GPS map vendor has needed to license the underlying data from a commercial provider. In addition to the cost, vendors also have to implement measures to stop the reverse-engineering and redistribution of this licensed data. However, like most forms of Digital Rights Management (some may say Restrictions), the technical mechanisms cause their own problems. I’ve just been helping with one person that has been suffering through Garmin’s Map Unlock process that is poorly communicated to customers, and provides nothing but roadblocks in an effort to set up the maps on the user’s computer and GPS. And even when he hopefully does have the maps unlocked, he will only be able to install them on one GPS!

Perhaps as a comparison, I am not able to download and install a copy of the Yellow Pages on my iPhone so that I can use it in a disconnected manner, but I can download the free and open Zenbu iPhone application that bundles all the data – so if for whatever reason I am out of mobile coverage, I can still use this data as it is stored locally on the device. I don’t believe that commercial directory services would be very comfortable about releasing their datasets to be installed on mobile devices, as they would risk the loss of their database in which the perceived value of their business resides. So having data released under permissive liceneses is also essential for new applications such as storing massive geospatial resources in our pockets.

That said, I’m not really in favour any more of the Government attempting to build a single massive dataset any more, as I think Government has proven that it cannot build these IT things effectively because there is too much management by committee, and the commercial vendors that provide the infrastructure are just looking for a jackpot if they win the tender (e.g. tender prices of $9-48 million for the failed National Address Register (NAR) project). I don’t see the need for the Government to build what is effectively their own OpenStreetMap infrastructure when we can just use something like OSM. Honestly, NZ Govt should just approach OpenStreetMap and look at an arrangement where Government can publish geospatial datasets into OSM with the ability to set some layers (such as say electoral and property boundaries which shouldn’t be editable) as read only, and the rest as editable – e.g. roads and walking tracks that can be maintained by everyone. If the publisher of a layer doesn’t want the original layer edited, then in some circumstances editable child layers should be allowed – e.g. so I can add a new walking track to a layer that hasn’t yet been updated to reflect it, and the owner of the original dataset can then look at whether they want to accept the change back into their layer.

Commercial geospatial datasets put nothing but roadblocks in the way for new and creative uses of geospatial data. I have no problem with commercial datasets providing value-add to the data, but the fundamental data such as roads and the like should be made as open and accessible as possible to encourage adoption and standardisation upon that dataset – this will also consolidate feedback and error correction. If I find an error now, I can’t report it to LINZ – they won’t listen. What benefit do I have in reporting a roading error to a commercial provider? Indeed the only benefit I get is if I report the error to a free and open project.

Adoption and standardisation of fundamental datasets are important to ensure consistency between map sets. Right now on my GPS I have two maps sets that both provide roads and you don’t have to look far to find discrepancies between the two datasets – but guess what, they are both derived from the LINZ road centrelines.

If left to commercial providers, geospatial data will be left as an expensive tool that only large organisations can afford.

The sooner governments in general recognise this, start funding the publishing and maintenance of fundamental datasets, the sooner we will see a real renaissance in how spatial information is used by the average organisation and individual. That is why I am so supportive of dc.gov releasing all their data.

I originally wrote this article for The Box, the Tuesday Technology section of The Press in Christchurch, New Zealand – it appeared on the 23rd September 2008. It also appeared on Stuff.co.nz.

Have you ever wanted to quickly find all the photos taken at your family bach? Chances are that unless you’ve been meticulous in filing your photos or tagging them with keywords, this could take quite a bit of time. Wouldn’t it be easier if you could click on your bach on a map, and bring up all of your photos within 1km, or display all of your holiday photos on a map? This is the promise of geotagging.

Simply put, geotagging records the latitude and longitude of the camera at the time the photo was taken and stores it in the image file.

Geotagging is not a new technology. It has been possible to geotag images for several years now, but previously only enthusiasts or professionals geotagged their photos, as it added extra steps to processing photos. It also required a GPS receiver that could record tracks – a breadcrumb trail of where the GPS had been. By matching the time in the GPS track log with the time that photos were taken, it is possible to reasonably estimate where the photo was taken. This took extra time and effort, and except for a dedicated few it was not worth the effort. The GPS receivers also added extra weight and bulk to carry around.

Recently GPS functionality has greatly shrunk in size and power demands, making it more friendly for the photographer by enabling the technology to be directly embedded in cameras and mobile phones.  Already a number of camera phones support geotagging photos – including much of the Nokia N series, and the recently released Apple iPhone 3G. Nikon has embedded a GPS receiver in their new Coolpix P6000 compact, and provide an optional GP-1 GPS attachment for recent Nikon digital SLR cameras. As more devices support geotagging, especially more affordable cameras and mobile phones, the possibilities (and the risks) are going to grow exponentially.

Combining GPS receivers with other devices makes the whole geotagging process transparent and automatic, and requires no effort from the user. This is going to rapidly open up opportunities for all sorts of geotagged data. But is it just technology for technologies sake? Not really, there are existing applications for geotagging, and even more to be come.

Travel photography just begs for geotagging. It is a great means of recording where holiday snapshots were taken, as you can easily show people where you took the photos. Not only that, but as people upload geotagged photos, they become a great travel planning tool as you can see photos that other have taken in a location that you are travelling to, and find sights nearby that you otherwise may have missed.

Real estate also stands to gain from geotagged photos – by being able to quickly load photos of properties for sale into an online searchable map, it will be easier to browse location and appearance at the same time. Councils and infrastructure companies have been using geotagged images for a number of years now to assist with managing assets – imagine being able to take a photo of a pothole and send the image to the council without having to try and explain where it is. Geotagging even has applications after a disaster – teams performing reconnaissance of an affected area can take geotagged photos whilst they are there, and when they return to an operations centre, the images and their exact location can be loaded into a mapping system to help authorities gain a better understanding of the extent of damage.

Location-based technology does come with inherent risks – mostly privacy related. Although many people are comfortable posting photos online, they may not be comfortable allowing people to determine the location where the photo was actually taken. This may be particularly relevant in the case of photos taken at home. No doubt we will see tools evolve to help people manage the privacy associated with geotagged photos, but in the meantime it is worth thinking about the content of a photo before uploading geotagged photos online.

The benefits of geotagging for the most outweigh the risks, and will likely lead to novel applications. The Apple iPhone 3G already has interesting applications taking advantage of geotagged images. Exposure provides mobile to Flickr – a photo sharing website. The ‘Near Me’ function will get your current GPS co-ordinates from the iPhone, and use that to display geotagged photos from Flickr that were taken near your current location. You can then view a photo, plot its location on a map, and if you desire, Google Maps will give you directions on how to get there.

Geotagged images don’t have to be shared online to reap the rewards – it may be that the biggest benefit is just providing another means of managing the vastly expanding data in your own photo library!

Always being one keen to live on the bleeding edge, I’ve been using the beta software for my Garmin Colorado. It has been great for driving around and recording tracklogs for the OpenStreetMap project.

One of the downside of using the beta software though, means that you can be exposed to bugs. I discovered that recently when attempting to open some of my recent GPX tracklogs and the software just refused to open them. After a bit of hunting, I found a relatively easy means of detecting errors and fixing them.

The tool to use is an XML validator called Xerces produced by the Apache Foundation. On a Mac, I download the appropriate binary package, and I copied the binary files in xerces/bin to /usr/local/bin, and the libraries from xerces/lib to /usr/local/lib. You can then run the program SAXCount that counts the number of elements in an XML file – the side benefit that we’re after is that it is good at detecting and reporting errors that many GPX applications are not capable of.

After working through a few minor problems on the NZ GPS forums, I had Xerces up and validating GPX – including with Garmin’s extensions. Note that if you get an error about trying to connect to Garmin’s server to download the schema, e.g. an error like…

Fatal Error at file , line 0, char 0
  Message: An exception
occurred! Type:NetAccessorException, Message:Could not open file:

http://www.garmin.com/xmlschemas/GpxExtensions/v3/GpxExtensionsv3.xsd

I believe this is a combination of Garmin redirecting the original link to a new location, and SAXCount not handling the redirect very well. If you strike this problem, this post in the forums has the fix. I’d basically recommend keeping a version of the fixed Garmin header ready to cut and paste into each GPX so that SAXCount can actually download each xsd. I’ve been using this one…

<?xml version="1.0" encoding="UTF-8" standalone="no"?><gpx
xmlns="http://www.topografix.com/GPX/1/1"
xmlns:gpxx="http://www.garmin.com/xmlschemas/GpxExtensions/v3"
xmlns:gpxtpx="http://www.garmin.com/xmlschemas/TrackPointExtension/v1"
creator="Colorado 300" version="1.1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.topografix.com/GPX/1/1

http://www.topografix.com/GPX/1/1/gpx.xsd

http://www.garmin.com/xmlschemas/GpxExtensions/v3

http://www8.garmin.com/xmlschemas/GpxExtensions/v3/GpxExtensionsv3.xsd

http://www.garmin.com/xmlschemas/TrackPointExtension/v1

http://www8.garmin.com/xmlschemas/TrackPointExtensionv1.xsd">

From there it is a simple step to validate.

SAXCount -v=always -n -s -f test.gpx

All going well you’ll get something back similar to.

test.gpx: 24 ms (7478 elems, 2498 attrs, 0 spaces, 38613 chars)

This means that everything checked out ok. Otherwise, it will let you know the lines that have errors, making it quick and easy to open in a text editor to edit or delete the corrupt elements. One trick I’ve noted is that by default, a Colorado GPX has no line breaks in it. A trick here is to search for /trkpt><trkpt and replace it with /trkpt>\r<trkpt – this will insert a linebreak, ensuring that each trkpt element starts on a new line and SAXCount can refer to it by line number for easier identification.

I’ve been meaning to blog about this sooner, but have been pretty busy with work. A chance email on a NZ GIS list that I belong to two weeks ago, inspired me to go out on a limb and see if I could get some Government data. I saw a post from someone within the Transit (soon to be merged into the New Zealand Transport Agency) refering to working with 2.2 million trackpoints from a roading survey. I started a private email discussion, and after a couple discussions, I soon had 2.2 million trackpoints from the 2008 High Speed Data Collection survey of New Zealand State Highway network.

My intention of obtaining this data was to be able to convert it to GPX files and upload it as a raw data survey layer to OpenStreetMap (OSM) so that it could be used as the basis for mapping New Zealand’s State Highway network in OSM.

I had some help from John McCombs from Integrated Mapping in Christchurch who very kindly reprojected all the points to WGS84. I then spent 4 evenings last week converting to GPX and uploading the files to OSM.

Was this data essential to mapping the highways in OSM? No. But it was a great experiment to see if a New Zealand Government Agency was willing to release data under acceptable terms and conditions – this dataset is licensed under the Creative Commons v3 Attritbution ShareAlike license, and effectively turn the raw data over for public consumption. Naturally, this doesn’t contain all of the detailed geometry that is collected during the survey, so not all of the data was made available, but we got the most important – latitude and longitude, and a lot of them!

For more information, see the following links.

• Transit High Speed Data Collection on gis.org.nz wiki
• Download the GPX on OpenStreetMap

One of the key points I was trying to make, was indicating that citizens are actually interested in accessing government data such as this, and that agencies should take a more proactive approach to releasing data for the world. After all, data is global these days – put it on the Internet and anyway can access it.

I find it hard to believe that the Institution of Professional Engineers in New Zealand is actually silly enough to promote the use of GPS technology to track vehicles and use that to institute user-pays billing on our roads. I have no problem with finding ways to deal with traffic congestion, and getting more people out of cars and into other forms of transport where possible and appropriate. But my problem is this.

The institution has released a report saying change is needed to the system where motorists pay for road use with a flat-rate excise tax. The report advocates the use of GPS systems to gather information on vehicle movements and charge accordingly. (from above linkie)

The privacy issues with GPS are a mile wide. To be able to implement congestion charging, which is often based on managing peak traffic at certain times, it is necessary to know exactly where a car is, and at what time – the basic information stored in a GPS tracklog. To be able to bill the driver, would this then mean that a GPS unit in a car is going to have to collect and send this information to some billing server that is able to process it, and analyse location and time, from which it will then produce the bill. Of course, this data would then likely have to be kept in case of any billing disputes – which means what is potentially very private and sensitive data is not going to disappear any time soon.

Of course, it may spur a whole lot of interest in DYI GPS jammers. In the meantime, you may want to check out a previous article I wrote on protecting your privacy with GPS tracklogs. GPS units produce private data, and the proposal from IPENZ is nothing more than a lame technical solution that shows a complete disregard for privacy.

Note – this has been written based only on news reports and not reading the IPENZ report in depth. Either it hasn’t been posted on their website yet, or it has only been made available to IPENZ members. I have emailed their media contact, but haven’t heard back yet.