Gav's Blog

Read books, repeat quotations, draw conclusions on the wall

Archive for the ‘standards’ tag

Detecting errors in GPX by validation

without comments

Always being one keen to live on the bleeding edge, I’ve been using the beta software for my Garmin Colorado. It has been great for driving around and recording tracklogs for the OpenStreetMap project.

One of the downside of using the beta software though, means that you can be exposed to bugs. I discovered that recently when attempting to open some of my recent GPX tracklogs and the software just refused to open them. After a bit of hunting, I found a relatively easy means of detecting errors and fixing them.

The tool to use is an XML validator called Xerces produced by the Apache Foundation. On a Mac, I download the appropriate binary package, and I copied the binary files in xerces/bin to /usr/local/bin, and the libraries from xerces/lib to /usr/local/lib. You can then run the program SAXCount that counts the number of elements in an XML file – the side benefit that we’re after is that it is good at detecting and reporting errors that many GPX applications are not capable of.

After working through a few minor problems on the NZ GPS forums, I had Xerces up and validating GPX – including with Garmin’s extensions. Note that if you get an error about trying to connect to Garmin’s server to download the schema, e.g. an error like…

Fatal Error at file , line 0, char 0
  Message: An exception
occurred! Type:NetAccessorException, Message:Could not open file:

http://www.garmin.com/xmlschemas/GpxExtensions/v3/GpxExtensionsv3.xsd

I believe this is a combination of Garmin redirecting the original link to a new location, and SAXCount not handling the redirect very well. If you strike this problem, this post in the forums has the fix. I’d basically recommend keeping a version of the fixed Garmin header ready to cut and paste into each GPX so that SAXCount can actually download each xsd. I’ve been using this one…

<?xml version="1.0" encoding="UTF-8" standalone="no"?><gpx
xmlns="http://www.topografix.com/GPX/1/1"
xmlns:gpxx="http://www.garmin.com/xmlschemas/GpxExtensions/v3"
xmlns:gpxtpx="http://www.garmin.com/xmlschemas/TrackPointExtension/v1"
creator="Colorado 300" version="1.1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.topografix.com/GPX/1/1

http://www.topografix.com/GPX/1/1/gpx.xsd

http://www.garmin.com/xmlschemas/GpxExtensions/v3

http://www8.garmin.com/xmlschemas/GpxExtensions/v3/GpxExtensionsv3.xsd

http://www.garmin.com/xmlschemas/TrackPointExtension/v1

http://www8.garmin.com/xmlschemas/TrackPointExtensionv1.xsd">

From there it is a simple step to validate.

SAXCount -v=always -n -s -f test.gpx

All going well you’ll get something back similar to.

test.gpx: 24 ms (7478 elems, 2498 attrs, 0 spaces, 38613 chars)

This means that everything checked out ok. Otherwise, it will let you know the lines that have errors, making it quick and easy to open in a text editor to edit or delete the corrupt elements. One trick I’ve noted is that by default, a Colorado GPX has no line breaks in it. A trick here is to search for /trkpt><trkpt and replace it with /trkpt>\r<trkpt – this will insert a linebreak, ensuring that each trkpt element starts on a new line and SAXCount can refer to it by line number for easier identification.

Written by Gavin Treadgold

August 9th, 2008 at 11:44 pm

Posted in GPS, Geospatial

Tagged with , ,

Feeding the eGovernment beast :)

without comments

Late last week I stumbled across the Research e-Labs blog, that comprises part of the larger eGovernment efforts. They are currently consulting on the new eGovernment feed standard. They are proposing a move to Atom 1.0 as it appears to provide better mechanisms for semantically marking up the information contained in the feed. This is an exciting proposition, especially if they expand the defined tags to support reports and datasources – this is on top of the already defined news, jobs, consultation and events. They have a good presentation available too.

Written by Gavin Treadgold

July 27th, 2008 at 10:04 pm

Great post on freeing Govt data

without comments

Having just been through the process of obtaining and publishing some Government data, I was interested to see Dennis McDonald covering an Australian blog post on eGovernment on ‘Making data freely available‘.

Craig makes some very good points in his conclusions, and I’ve got one that I would like to add based on recent experience with the State Highway trackpoint data I sourced in NZ and uploaded to OpenStreetMap.

In the followup to uploading the trackpoints to OSM, a group of NZ OSM mappers have been discussing how to classify roads in New Zealand – basically how the road hierarchy is classified. Turns out that this is a hard problem and there is not a single standardised approach taken to classifying roads in New Zealand. After asking for guidance, I became aware of a number of issues and capture these on the gis.org.nz wiki.

The simple fact was that there was not a nationally consistent approach to classifying the road hierarchy in New Zealand. I expect it is similar in Australia, and possibly worse with the additional layer of State government.

This highlights an additional key role that I think Government has to play in national data collection.

Government should set the standards for data collection to ensure that datasets are nationally consistent to enable simple aggregation of disparate datasets.

It doesn’t have to actually perform the aggregation, although that would be nice. It just has to ensure that standards are used to enabled aggregation. Using the road hierarchy classification example from above, this means that a road of a certain class in one part of the country, means exactly the same in another. This would probably occur by an engagement-based approach that determines a controlled vocabulary to define types of roads.

Without this responsibility, citizens are doomed to a million-and-one datasets that cannot be easily aggregated to produce coherent and consistent national datasets.

Written by Gavin Treadgold

July 1st, 2008 at 11:06 pm

Posted in Geospatial

Tagged with , ,

Comments on Identity Verification Service

without comments

I just heard today that the Department of Internal Affairs is consulting on an opt-in single sign-on identity verification service (IVS) that may be used by government agencies to identify us online when interacting with said agencies.

I have included my submission below for reference.

We would like to know whether you are likely to use the Internet to verify your identity with a government agency.

Yes – but it must work on any operating system and web browser. I use a variety of operating systems and web browsers including:

  • Operating Systems – Apple OS X, Fedora Core Linux, and Microsoft Windows
  • Browsers – Firefox and Safari

I will not be able to use the service if it is tied to Microsoft Internet Explorer/Windows platform. I expect that all the good work that the State Services Commission has been doing on standards and interoperability will be applied to IVS as well.

We would like to hear from you regarding the type of services you might want to access that require you to verify your identity.

  • Inland Revenue for management of personal/business taxes, KiwiSaver?
  • Government Electronic Tender Service (GETS)
  • NZ Qualifications Authority for NZQA Learner’s Record
  • Local Government

We would like to know what you think of being able to verify your identity with businesses and other organisations.

I would support the service being made available to local government.

I am initially dubious about IVS being made available to businesses until such time as more details are made available. I trust the Government to run their IT systems to a higher level of security than most businesses. I am also concerned that if the IVS was made available to non-governmental users, that uptake may well make the IVS to be more than an opt-in service – businesses may use incentives that Government cannot to strongly promote registration and use of the service.

I would however support a limited number of business sectors to utilise the IVS – in particular those that provide online financial services such as banks, fund managers and sharebrokers. It is preferable to have them using a national framework rather than having a token for each organisation AND government on my keyring. Note that this would present some risks – in particular the risk of a distributed-denial-of-service (DDOS) attack against the IVS infrastructure. If the IVS does grow to become widely used, and includes the financial sector, then a DDOS against poorly planned IVS infrastructure may have significant negative consequences – even if just in perception of the service. Naturally, as IVS grows in usage, it would have the potential to become national critical infrastructure and would need to be managed as such.
Read the rest of this entry »

Written by Gavin Treadgold

December 4th, 2007 at 1:33 pm

Standards for printing photos

without comments

I can sympathise with Jeff’s frustration at not being able to get frames for paper sizes from the ISO standard but I think he has it wrong when he suggests that Epson needs to step up and produce US paper sizes. There is a followup, from Scott Kelby’s blog that explains Epson’s position further.

With most of the world actively using ISO paper sizes, it would be far easier for framer producers to recognise that they should start producing frames that support internationally recognised standards, rather than trying to maintain antiquated North American paper sizes. This is just inefficient for international producers, and I’m sure they would love the US and Canada to adopt international standard paper sizes, so that they don’t have to keep producing paper for the North American market. If North Americans want to keep using different paper from the rest of the world, then surely their internal market would pick up on the opportunity?

Read the rest of this entry »

Written by Gavin Treadgold

October 20th, 2007 at 3:58 pm

Posted in Photography

Tagged with ,

Sahana: An Open Source Disaster Management System

without comments

I wrote this article for the July 2006 International Association of Emergency Managers Bulletin.

In February 2004, I wrote an article for the IAEM Bulletin outlining some of the benefits that open source software had the potential to provide for emergency managers. At that time, little open source software existed for emergency management, and I had just come out of a simple attempt in 2003 to create a Web-based disaster management system. That effort failed, for while there was a well-recognized need for open source disaster management software, there were no real drivers to encourage development of a solution.

2004 Tsunami Spurs Development of Sahana

The driver came with the tsunami that struck Sri Lanka on Dec. 26, 2004, which prompted the development of a free and open source solution called Sahana. Within a couple of days, the need for a system to manage vast quantities of information became obvious, along with the need to attempt to coordinate 1,300 NGOs responding to hundreds of thousands of displaced people. In the following days and weeks, a Web-based system for managing disaster information was built on-the-fly based on the most pressing needs. Accordingly, the following were the first modules developed:

  • People Registry – track and match victims of a disaster.
  • Organization Registry – register, connect and track NGOs involved in response.
  • Camp Management System – register and track camps.
  • Request/Assistance Management System – record, track and match requests and offers of assistance.

Sahana development was initially led by the Lanka Software Foundation and supported by volunteers from the Sri Lankan IT industry. As the immediate need for Sahana subsided in the months following the tsunami, more international contributors became involved in the project, myself included. These ranged from programmers wanting to help out, to those who wanted to offer assistance drawing upon their disaster experiences, including emergency managers.The positive feedback to Sahana prompted further development to add more response and recovery capabilities applicable to any disaster management situation.Longer-term, the goal is to use Sahana as a means of encouraging comprehensive emergency management in communities by supporting preparation and mitigation. This will start by providing tools to incorporate plans and reference material, such as communication directories in advance and other techniques to encourage greater interagency co-ordination before an event.

Capabilites

Sahana has been designed to operate in a diverse range of environments due to the nature of disasters. It can run on Web servers and laptops, and has even been installed on a PDA. Over time, it will support both standalone and networked modes of operation and allow communication between multiple Sahana servers, including synchronization of data.There are a number of future modules planned or under development:

  • Disaster Impact Assessment.
  • Alerting.
  • Inventory/Supply Chain/Logistics.
  • Volunteer Coordination.
  • Intelligence.
  • Response/Rescue Team Management.

In addition, there are a number of key technologies identified for inclusion:

  • Mapping/GIS, and GPS integration – it can already use Google Maps.
  • Biometrics.
  • Provision of information via open standards:
    • Common Alerting Protocol (OASIS/CAP).
    • Emergency Data Exchange Protocol (OASIS/EDXL).
    • Various OpenGIS Protocols (OpenGIS Consortium).
  • Support of existing paperbased forms.
  • PDA forms for remote fieldwork.

Deployment

Sahana has seen official deploy ments in multiple events, including the Sri Lankan response to the tsunami in 2004, the 2005 earth quake in Pakistan and the 2006 mudslide in the Philippines. It has also recently seen unofficial deployment in support of the Yogjakartra earthquake and in preparation for an eruption of Mt. Merapi. Sri Lanka’s largest NGO is also deploying Sahana within their disaster unit.

Recent Events

In mid-May 2006, a workshop was held in New York that brought together key members of the Sahana development community and IBM. The meeting served two purposes:

  1. To discuss IBM support of the project, and
  2. To consider further development of modules for Sahana that could be used during response to a pandemic.

A pandemic presents an interesting opportunity for the deployment of Web-based disaster management systems, as most infrastructure should be operating normally (relative to a hurricane or earthquake).The Sahana project is interested in contributions, be they time or financial. Time contributions can be made in providing design advice based upon disaster experience, writing the code, testing Sahana or helping to write the documentation. Financial contributions will be used to target module development, such as sponsoring development of a specific module or supporting the core development team that works full time. An international community maintains Sahana, and all contributions are provided back to that community at no cost – a share-and-share-alike ethos to ensure that everyone benefits. Sahana is free to use and has no licensing costs associated with it.

Links

Written by Gavin Treadgold

July 1st, 2006 at 12:00 pm