Gav's Blog

Over the years I’ve had a few ideas how we could improve the gathering of intelligence for an emergency, and how this could be linked into both situation reporting and as an action tracking tool for operations and other general management tasks.

Most of this thought has been directed at how to implement it in a Sahana product, but it could conceivably be applied to any product for emergency management, or even broader business/organisation management. Aspects of this solution already exist across both Ushahidi and Sahana – but neither provide the comprehensive solution yet.

This need was also something that I saw during my involvement in Building Safety Evaluation during the Canterbury Earthquake in 2010. It is also potentially a far more robust means of managing the collection of intelligence associated with an emergency.

I’ll talk about this using a modular approach – making the assumption that different groups of users will have different types of access, to ensure the protection of submitted information (see here for a reason why).

Capturing the Information
The first module would be the Intelligence Gathering module. It is where information from other sources is collected from other sources.

In the CrisisCommons context, this might allow an anonymous volunteer to submit cut-and-paste text from a news article on a website or from a situation report. For the purpose of this concept, I’m going to ignore the copyright issue – but do want to flag that this may be an issue with the collection of information from the media where a lot of information is copyright.

For Building Safety Evaluation, this may be the unstructured reports that we received that ‘the wall on this building looks like it is about to fall on a neighbouring building’.

So this module basically allows for the entry and recording of what is mostly unstructured information – it may be from a website, a phone call, SMS, even a scribbled piece of paper that someone passes you in the EOC.

Such a system could be easily configured to allow members of the public, or crowdsourcing volunteers to enter such information without having to register or have an account – thereby keeping the barrier to collecting raw information low.

Adding Structure and Metadata
Having this information in digital form is just the start however, the next step is to get a team of trusted individuals to then review the submitted information and critique it for quality, actionability, and credibility. At they same time they would ideally try and add other metadata to the record.

Does it currently contain a freeform address? Then the reviewer would then associate an address with the record, and this would properly structure the address information. If a geocoder is available then latitude and longitude records would also be entered.

We had some issues in Christchurch whereby some addresses that were reported are not official addresses recognised by the council property system – this usually happens with ‘vanity’ address. Our Kestrel office in Kestrel has a common/vanity address of 35 Riccarton Road, but for council and utility purposes, our building is actually 39 Riccarton Road. I spent a bit of time in September, and again following the Boxing Day aftershock checking some of the incoming addresses that were provided, and then record a Council GIS identifier once we had correctly identified the address. Again, this would be another means of tagging the raw data with something that adds valuable metadata to incoming information.

Does it have a phone number? Associate a structured phone number.

What does the record refer to? Add tags from a controlled taxonomy so that the record can be filtered – e.g. if the record refers to building damage, it may be tagged with ‘building evaluation’. If it is a report of a missing person, it should be tagged with ‘missing person’.

This is perhaps the most time-consuming part, but it is also most required, as it opens up a lot more potential for actually managing and sharing the information.

Wrapping it all in Management Tools and Reporting
Now we can finally get down to the crux of what we’re trying to achieve – take raw unstructured information and provide it in a form that information systems can understand it, present it, and search it in far more valuable ways.

If we assume that all incoming information about building safety is tagged with ‘building evaluation’ then we can provide a web page that allows someone in the building safety evaluation team to review all the incoming reports that are relevant to them.

At this point we go the final step, as we start allowing people in these focused teams to start associating actions and history to the original record. You may have a small team with Build Evaluation reviewing incoming records for ‘building evaluation’ when they see it, because the address (and potentially lat/long and council identifiers), it should be trivial to see if other records have been entered that refer to the same building, or nearby buildings. Without adding this metadata previously, it would be a lot harder to automated some of this information management.

We can then link multiple records that refer to the same building – such as different reports over time, or a neighbouring property that may refer to the building.

The best part though, is when we start adding actions – for example, if an Urban Search and Rescue Team is tasked to a building, then that action (Sending a team to perform an intial rapid building assessment) can be associated to that building and the team, and of course the original record that reported it. This means that if someone enquires if anything is being done, we have the history of who was tasked where and when.

When the team returns, we can mark the action as completed – we have a record that it has been completed. Not only that, but any quick comments from the team could be added as a new record associated with that building. Likewise, any digital photos, or even scanned copies of the rapid building damage assessment forms could be attached.

Scanned forms are of course interesting, as you could scan them initially and add them to the system as images, but also flag those to be reviewed to create metadata so that the form data is now accessible via metadata – such as the building status as determined by the assessment Safe/Green, Restricted/Yellow or No Access/Red. Whilst Optical Character Recognition (OCR) could speed this process, after seeing the handwriting of engineers, I’d suggest that human review and triage of key information on the forms would get more usable information into the system sooner. And yes – the idea is of course a tablet application that digitises the information in the field and uses an Emergency Data eXchange Language (EDXL) extension to submit the information via EDXL-DE back to a server.

Of course, with all this structured metadata now wrapped around the original unstructured reports – this opens up so much potential for reporting and where appropriate sharing this information using standards such as the EDXL for achieving true information interoperability.

This is something that both Ushahidi and Sahana have been working since the response to the Haiti earthquake when we were trying to provide management tools in Sahana Eden to wrap around the crowdsourced information that was being collected by Ushahidi.

Recently discussion on the CrisisCommons email list raised an issue about security pertaining to crowdsourced data – and the ease with which the information can be deleted by an anonymous malicious individual when using tools such as etherpad or Google Docs with open editing rights.

In this case an anonymous user was deleting data as quickly as it was entered in a shared public document. What is a more concerning risk is perhaps the subtle editing of crowdsourced information, where the edits are not obvious enough to be detected – such as the subtle and malicious modification of facts and figures.

For tech volunteers, there is a careful balance to be struck between protecting information (in this particular case its availability and integrity) and not creating significant barriers to entry.

The first obvious solution is that access on the document be restricted to authorised users. This means that only those individuals that are trusted can be expected to contribute to the collection and management of unstructured crowdsourced information.

This is less than ideal as it means that new users that volunteer immediately following an emergency haven’t developed a trust relationship with, for example, the CrisisCommons community, and are unable to immediately contribute.

I believe that with the simple use of a two-tier approach, one can easily protect the quality of the final document(s), whilst still making it easy for new volunteers to contribute.

You effectively create two types of document:

1. Public and open documents – which are open to all to edit, and are effectively a rough scratchpad for collecting unstructured information.
2. Trusted documents – which are open for only a limited pool of trusted users to edit, but draw from the content provided in the public and open documents.

The trusted editors effectively become the curators of the information, and once content has been copied and edited from the open documents, malicious anonymous users won’t be able to waste other volunteers time through deletion or editing.

There are other process benefits to this approach. For example, you may create a public document particular topics of the emergency – such as infrastructure, health/medical and background information (e.g. weather forecasts, population demographics etc) and these multiple individual documents may map to a single section within the trusted document to produce an edited and trusted version of crowdsourced information.

Still, from an operational perspective, this is a far from ideal approach, and there are certainly more robust approaches available to turn this into a process that can be used for intelligence gathering and situation reporting.

Three months ago I blogged about the Conficker worm and its relevance for emergency managers. Since then, I’ve rumours that a number of health agencies were still having problems with their email systems. The reason I raise this again, is that now, with a large national response to a potential pandemic taking place, one hopes that Conficker has been well and truly removed from all Health systems (both Ministry and DHB).

If Conficker is still impacting on health agency IT systems during this period of increased activity, then honestly, heads need to roll at MOH.

I’ve only recently started following the NZ Health WebEOC blog, but it is exciting to see this sort of information sharing taking place. Congratulations to Charles and the team for the work involved. I found in their feed today an article about the Ministry of Health suffering from the recent Conficker worm outbreak over the past few days. There is more info here from Computerworld.

First, what is Conficker? From Wikipedia.

Conficker disables a number of system services such as Windows Automatic Update, Windows Security Center, Windows Defender and Windows Error Reporting. It then connects to a server, where it receives further orders to propagate, gather personal information, and downloads and installs additional malware onto the victim’s computer. The worm also attaches itself to certain critical Windows processes such as svchost.exe, explorer.exe and services.exe.

What is interesting is that the security hole that Conficker utilises to gain control of the Windows operating systems was plugged in a security patch released on 23 October 2008. That means in theory that all those systems that have been compromised in the past week were systems that had not had the patch applied that was released in late October. The security patch to protect against Conficker-like attacks for Windows 2000, Windows XP and Windows Server 2003 was marked as critical and should have been installed in a timely manner.

What are some lessons from an emergency management and business continuity perspective?

1. If you’re running Microsoft Operating Systems – you must keep them patched, and do it in a timely manner. Windows represents the largest near-homogenous family of operating systems in the world. This makes them the primary target for the developers of botnets and malicious software. Whilst I recognise that it takes time to deploy patches in a large organisation such as the Ministry of Health – an organisation will always be at risk if it doesn’t install security updates in a timely manner. All Microsoft ‘Critical’ patches should be patched within weeks of release.

2. Where possible, organisations should attempt to diversify the installed base of operating systems in an organisation. If you solely run Microsoft operating systems then a worm has the potential to take down an entire organisation. If you run a heterogeneous  computing environment that has a variety of operating systems (e.g. Windows, Unix and OS X), then any outbreak of malicious software will only directly impact some of the systems. In our small business I support all three of these platforms. We have Windows and OS X clients, and servers on Linux, OS X Server, OS X Desktop, and this is one of the main reasons I refused to deploy solely Windows software for client and server when setting up our business. Reliance on a homogeneous computing environment decreases overall IT resiliency.

3. Emergency Management Information Systems (EMIS) should ideally be able to be segregated from the production systems. Malicious software doesn’t have to infect a system to have an impact on it. Even if the malicious software just consumes 100% of the network bandwidth, that will be enough to create a continuity issue by denying access to critical systems – such as servers. Therefore, EMIS should really be configured on a separate network so that even if the internal network bandwidth has been fully consumed, and access to the Internet severely restricted to limit the spread, critical systems can still be provided to the wider world. Network segmentation can be used to limit the impact upon critical systems. Direct access to the emergency network segment could be provided from network jacks in the EOC. Once again, these should be on an entirely independent network segement  to ensure that emergency operations can continue during an outbreak of malicious software on the main LAN.

Finally, emergency managers should also make themselves aware of the Centre for Critical Infrastructure Protection (CCIP), and consider signing up for vulernability alert emails. These are sent out for critical advisories associated with information security risks, and can be good prompts for getting in touch with IT, and making sure that your systems are patched and up-to-date.

Update 2009-01-27: I see that the Manager of the CCIP went public yesterday saying the CCIP advised MOH of the security patch in October. The real question is whether the Ministry has custom applications installed on all its systems (e.g. including clients), or if they are just talking about server applications. If most of the desktops are only running Office and a groupware application such as Outlook or Notes, then they should have been able to be relatively easily patched before December. It is well recognised that patching servers running legacy applications takes longer to test for complications before deploying patches.

I had downloaded Fedora Core 8 to install on one of our work servers, and I noted that only sha1sum’s are provided to verify the downloaded iso now. In the past I had used md5sums and had installed md5sum on my Mac to achieve this. Anyway, it turns out there is a simple solution to verify the file anyway. 

openssl dgst -sha1 Fedora-8-i386-DVD.iso

I just heard today that the Department of Internal Affairs is consulting on an opt-in single sign-on identity verification service (IVS) that may be used by government agencies to identify us online when interacting with said agencies.

I have included my submission below for reference.

We would like to know whether you are likely to use the Internet to verify your identity with a government agency.

Yes – but it must work on any operating system and web browser. I use a variety of operating systems and web browsers including:

• Operating Systems – Apple OS X, Fedora Core Linux, and Microsoft Windows
• Browsers – Firefox and Safari

I will not be able to use the service if it is tied to Microsoft Internet Explorer/Windows platform. I expect that all the good work that the State Services Commission has been doing on standards and interoperability will be applied to IVS as well.

We would like to hear from you regarding the type of services you might want to access that require you to verify your identity.

• Inland Revenue for management of personal/business taxes, KiwiSaver?
• Government Electronic Tender Service (GETS)
• NZ Qualifications Authority for NZQA Learner’s Record
• Local Government

We would like to know what you think of being able to verify your identity with businesses and other organisations.

I would support the service being made available to local government.

I am initially dubious about IVS being made available to businesses until such time as more details are made available. I trust the Government to run their IT systems to a higher level of security than most businesses. I am also concerned that if the IVS was made available to non-governmental users, that uptake may well make the IVS to be more than an opt-in service – businesses may use incentives that Government cannot to strongly promote registration and use of the service.

I would however support a limited number of business sectors to utilise the IVS – in particular those that provide online financial services such as banks, fund managers and sharebrokers. It is preferable to have them using a national framework rather than having a token for each organisation AND government on my keyring. Note that this would present some risks – in particular the risk of a distributed-denial-of-service (DDOS) attack against the IVS infrastructure. If the IVS does grow to become widely used, and includes the financial sector, then a DDOS against poorly planned IVS infrastructure may have significant negative consequences – even if just in perception of the service. Naturally, as IVS grows in usage, it would have the potential to become national critical infrastructure and would need to be managed as such.
Read the rest of this entry »