Gav's Blog

This is a followup email that went out today to prompt new leadership of the Sahana PHP project.

In my long email nearly a week ago, I suggested that we dissolve the existing PMC. I would now like to start community discussion about forming a new PMC to provide the leadership that the PHP project requires, and has not received for a long time.

A brief reminder why I think the PMC should be dissolved
No, actually, it is the leadership of the PHP project’s fault, and as one of the members of the PMC, I have to shoulder some of that blame like many other people here. Fran et al raised many issues in 2008 about not only the core framework of Sahana PHP, but also infrastructure. He and others tried to work within the PHP project, BUT NOTHING EVER HAPPENED. He tried to work within the rules but the PHP PMC failed him completely. In the end, Fran et al did exactly the right thing to do with open source software and fork/recode. The PHP PMC never ever made any decisions to modernise or improve the core framework. We never responded to Fran’s issues, and now, nearly a whole year later, we still have not made much progress on project infrastructure. It is entirely the PHP PMC’s fault for showing a lack of leadership, and not getting things done. I am honestly at the point where I believe the current PHP PMC is dysfunctional, and I would like to see a brand new leadership team form by those that want to take the reigns of Sahana (PHP). The only thing that Fran et al did wrong was to use the Sahana brand without permission.
Likewise, there have been many stunning contributions to Sahana (PHP) both in bug fixing and adding new features. These developers have also been caught up in the complete lack of leadership provided by the Sahana (PHP) PMC.

Tearing the old PMC down
If I had been off the mark in my original comments, I would have expected a lot of replies in the negative. I received none – neither publicly or privately. So, I can only assume that you agree with what I said, or you didn’t read my email

I would like to put forward the following proposals to the community – this would effectively close down the existing PMC.
1. That the existing pmc@ email alias be closed down.
2. That we capture for historical and recognition purposes all the members of the PMC and record this on sahanafoundation.org

Do we need a new PMC? Can’t we just do it on maindev?
I would like to think that we can do everything in the open this time, e.g. the new PMC shouldn’t have a separate and private email list. Most of the reasons that the PMC was private originally are now being managed by the Board anyway. We need people to stand up and put their names behind the project, set direction, and lead. Being a part of the new PMC is about taking responsibility and publicly stating that you’re prepared to step up and lead the PHP project, and putting your name behind it.

Standing up a new PMC
I would also like discussion about how we could create a new PMC structure for the PHP project. I believe the membership of the PMC should follow a very different model to the old PMC. The old PMC was based around committership – which this is appropriate in a developer-led project (e.g. Apache HTTPD) with traditional CVS. I don’t believe it is an appropriate model where we are producing a domain-specific application. At the same time, as we move to a distributed CVS, the concept of commitership becomes far less important, and it all becomes about responsibility for merges back into the main repository.

I see there are two key areas that a new PMC needs:
1. It needs domain leadership – I would like to see people from the likes of CUNY and NIH be a part of leading the project from a domain perspective. They have some of our best end-user interaction, and having them on the PMC will be a key means of getting end-user feedback incorporated at a strategic level in the project.
2. It needs technical leadership – to act quickly and responsively to ideas and requests from the community around the likes of PHP framework choice, infrastructure, merges, and release management process (recognising that releases also need to be linked to the domain and end-user needs).

I think there are two simple membership rules (in concept) required for the new PMC.
1. Anyone that is prepared to take a leadership role in the project can join.
2. Anyone that becomes inactive or fails to ‘get things done’ is removed from the PMC (e.g. lack of participation in voting)

The PMC has to be about ‘getting things done’ and should no longer be a part of a coder>committer>PMC pathway. As this is not a developer-led project, but rather a domain-led project, the traditional developer pathway makes no sense.

First Actions for the new PMC
There are some urgent actions required by members of a new PHP PMC.
* Developing project and mentor capacity for GSOC 2010
* Consider tagging 0.6.2 as a dead branch (in terms of focusing volunteers at least)
* Identify the trunk for future development focus (I believe we should adopt the RELIEF branch)
* Future direction
* Oversee the PHP framework discussion
* Can PHP be differentiated from Python
* Is it an effective use of volunteer resource to move Sahana PHP to a modern PHP framework, or is it more effective to adopt Sahana Python (given they are already a solid 15 months ahead)

Implications for GSOC
If the Sahana Software Foundation is announced as a GSOC mentoring organisation later this week, and we don’t have an active leadership team for the PHP project, then it will be a lot harder to justify GSOC slots going to the PHP project. Remember that this year each project is going to be responsible for managing project selection, mentors etc. As a GSOC Admin for the Sahana Software Foundation – I (and David) need to see strong leadership and support available within the project to be able to make slots available to the project.

Want PHP to succeed? Then it’s time to step up?
Who is prepared to take responsibility and lead the PHP project forward? There are many names that come to mind given recent discussions – Kethees, Chad, Greg, Glenn, Chamindra. Please consider forming a new and active PMC this week as there is a lot that needs to happen. It is critically important that a leadership team is formed quickly and able to start coordinating some of the actions outlined above.

The future of the PHP project is in your hands – if you want it to succeed, now is the time to step up and play your part.

Cheers Gavin

This is an email I mailed to the Sahana developer community and the Sahana Software Foundation Board on the 10th of March following recent discussions about branding and the perceived competition between Sahana (PHP) and Sahana (Python).

Sigh. So much for getting on top of my inbox.

/me takes all official role hats off, stores them in the soapbox and steps up

I’m intending to lay down a challenge to many preconceived ideas and current thinking. I believe little will be achieved unless we can collectively – as a community – come to some agreements, and get more things done. So I’m going to be brutally honest – it is time we discussed some of the failings, accept them, and look at where we should go from here.

What is Sahana?
Sahana is not software. Sahana is an ideal to use free and open source software to improve information management before, during and after emergencies, disasters, and providing humanitarian aid. The ideal of Sahana will live long beyond any software project we currently have. The ideal itself was around before[1][2] Sahana and will likely outlive anything we do. I am still committed to this even if all our software projects and the foundation were to turn to dust.

Sahana (PHP) was the original Sahana.
No it wasn’t. The original Sahana was Mambo. And PHP. And Perl. And Java. And probably some other languages that I can’t remember. Sahana (PHP) was not the first, it was phase 2 and that was rewritten from the ground up. Perhaps one of our biggest mistakes in hindsight was not coming up with a clearer name then other than Sahana phase 1 and phase 2.

Our existing contributions will be lost.
No they won’t. This is open source, and once a contribution has been made the contribution is there until someone updates or modifies your original contribution. The code is always accessible from repositories. This isn’t a closed-source proprietary project. The only contributions that will be lost will be future contributions of those that opt out of the community. As long as you hold to the ideal of free and open source software for emergencies, disasters and humanitarian aid – then you’re in the right place.

It’s Sahana (Pythons)’s fault. Or the Board’s.
No, actually, it is the leadership of the PHP project’s fault, and as one of the members of the PMC, I have to shoulder some of that blame like many other people here. Fran et al raised many issues in 2008 about not only the core framework of Sahana PHP, but also infrastructure. He and others tried to work within the PHP project, BUT NOTHING EVER HAPPENED. He tried to work within the rules but the PHP PMC failed him completely. In the end, Fran et al did exactly the right thing to do with open source software and fork/recode. The PHP PMC never ever made any decisions to modernise or improve the core framework. We never responded to Fran’s issues, and now, nearly a whole year later, we still have not made much progress on project infrastructure. It is entirely the PHP PMC’s fault for showing a lack of leadership, and not getting things done. I am honestly at the point where I believe the current PHP PMC is dysfunctional, and I would like to see a brand new leadership team form by those that want to take the reigns of Sahana (PHP). The only thing that Fran et al did wrong was to use the Sahana brand without permission.

Personally, I have become increasingly unwilling to contribute my time to the PHP project because it takes a long time to get things done. If they are done at all. Fran et al have demonstrated that they are prepared to do the hard yards, and focus on development. I applaud their attitude, and when it comes to where I’m willing to put my volunteer time, it is increasingly coming down on the Sahana (Python) side. Honestly, I should have just shut up and learnt to code (again) many years ago.

Likewise, there have been many stunning contributions to Sahana (PHP) both in bug fixing and adding new features. These developers have also been caught up in the complete lack of leadership provided by the Sahana (PHP) PMC.

Why hasn’t Sahana (PHP) adopted modern frameworks and technologies?
Don’t blame this on the PMC or Board. I don’t recall seeing formal proposals come to either for approval. Again, like many other things, plenty of discussion, but no-one ever appeared to pick it up and run with it. There was talk of an experimental Zend branch, but I haven’t heard further discussion on this, or a call for other PHP developers to join in and support the experiment. Again – I assume that the talk has gone nowhere.

On Branding.
Quite simply, both projects (PHP and Python), have fallen into the same trap of branding based on a developer worldview, rather than thinking about how we communicate in 5 words or less what our products do to end users, and how we differentiate them. We need a complete and utter rethink of branding, and I am increasingly favouring the Apache approach whereby Sahana becomes an umbrella brand and as Sanjiva highlighted, the various projects become Sahana {project}.

Disaster Management System will probably need to go too. Increasingly, the international approach is towards Comprehensive Emergency Management[3]. We have to fit in with our end users, and for the increasing majority of them, this is Emergency Management. The concept of Humanitarian Aid is only used in countries without well developed EM systems, and many development projects are working towards helping countries develop more robust EM arrangements e.g. WorldBank and/or Asian Development Bank projects are building local emergency management capacity and capability. So from a domain perspective, Emergency Management is probably the key phrase to focus on for branding purposes for potential end users.

Should the Sahana name be earned?
In due course, yes it should. We are not at the point now where it is workable, but I would like to believe that in a few years, we could define minimum standards for interoperability, and any software application (as opposed to library, or other project) would have to meet some minimum standards of interoperability to be entitled to be called Sahana {project}. Sahana must become a quality standard, and any application that wears its badge should meet some minimum requirements. As well as standards, there are other aspects that should be considered including testing/quality, and the capacity to support remote deployments for extended periods.

Where to from here for Sahana (PHP)?
Personally, I think Sahana (PHP) has a lot of work to do to recapture momentum and accept that big change is needed. Some next steps would probably be:
* accept that 0.6.2 is a dead branch and archive it
* accept that Camp Roberts RELIEF 10-1 is the active branch and treat this as trunk
* dissolve the existing PMC
* let a new leadership team form from whoever is interested in leading the project forward – this should not be developer-only, but should include developers, users and anyone that has a stake in Sahana (PHP) and is prepared to lead it onwards and upwards.
* immediately start work on a new framework (someone suggested moving to Zend a while back – don’t forget this is open source software and you can create an unofficial branch, and just do it).
* or, and this is potentially a very difficult decision, consider whether there is too much work to update Sahana (PHP) and consider moving to support Sahana (Python). If it doesn’t take much work to modernise Sahana (PHP), then great, but if it involves a massive amount of work, then it is better for us all that we build upon the work that has already been done by Sahana (Python).

Shouldn’t businesses be driving Sahana?
There is a mixture of approaches to development of open source software, including anarchy, business-supported and non-profit. Which is the right one? Depends on the solution(s) being developed. I think that long-term, we’ll need both non-profit and business. There are situations when a non-profit e.g. Foundation is a better vehicle for development and attracting funding for developers. At other times businesses are much better for managing contractual development, deployment and support. At this point, I think the Foundation is filling much of the vacuum, but I hope that this year, we will see much more active leadership and support from both Respere and AidIQ. I actively welcome these two companies to step up and push the development of their respective projects. In all honesty, they have better interaction with end users than many of us, and are best placed to close the feedback loop to improving our software applications.

Is it time for a tough decision?
I believe it is.

It is time for me to fully lay my cards on the table. I’ve been involved with Sahana phase 2 since mid-2005. Like many others here, I have invested not only hundreds of volunteer hours in Sahana (probably over a thousand now, but I gave up counting a long time ago) but also a few thousand dollars in airfares, and have probably lost a great many dollars in terms of opportunity cost. I’m as much the volunteer contributor as anyone here, and I haven’t even managed to contribute something tangible such as code

That said, I care little for the past, because we can’t change it. I care everything about the future and where we should head to from here – the future is everything, it is the only thing we can change, and we must focus on that.

I think we would be doing a significant disservice to our potential end users by continuing the confusion of having two software application products that we have not managed to clearly differentiate.

Following the initial Sri Lankan deployment in early 2005, the decision was made that what was effectively Sahana phase 1, was not a good foundation to build upon, and that a ground up rewrite was needed. Sahana phase 2 was born. But we are now continually running up against leadership and fundamental framework issues in phase 2. I believe we are now at the point where it is becoming fairly clear that Sahana phase 2 is not providing a platform that is going to take us well into the decade we have just started. It has given us 5 great years, but I don’t believe it can take us another five. We definitely do not have a leadership team within the current PMC to accomplish that.

I would like to suggest, again personally and with no role hats on, that we accept that it is perhaps best that we wrap up new development on Sahana phase 2. This of course does not mean it is the end or that contributions are wasted. I believe we should encourage any end user interested in Sahana phase 2 to approach Respere and obtain development and support from them. No code will ever be wasted, and it would quite likely have better project leadership than we are currently seeing. The only code that would be wasted would be if we were to now attempt to migrate Sahana (PHP) to a modern framework when the majority of that work has already been completed with Sahana (Python).

I, personally, believe it is time for the volunteer development community to move on. I believe that it is now more appropriate than ever to encourage volunteer developers to focus on Sahana (Python) and to actively promote this as Sahana phase 3 – which fits with the ongoing approach since 2005. The Haiti deployment in early 2010 of Sahana (Python) clearly indicates that it is a modern and workable system. It has a team with strong leadership and the ability to make decisions and move the project forward. It has responded to many of the issues raised against Sahana (PHP). With the surge of development that is occurring since the Haiti earthquake, it now has a solid 18 months of development behind it, a significant real-world deployment, and has been seen by a lot of potential end users. It is probably easier now to port modules from PHP to Python than it will be to implement a new framework in PHP.

We need a modern and attractive focal point for volunteer developers to focus on. I don’t believe that we can provide that using Sahana (PHP) but we can with Sahana (Python).

Please consider what I have presented carefully. If we continue to divide the volunteer development community I think we will all fail – most of us know this, but few have been prepared to air it until recently. Better that we now make a clear decision, within the public community, to end serious volunteer development on Sahana (PHP) and try to rally as many developers as possible around Python as the phase 3 successor to phase 2.

A number of developers have indicated that they don’t mind PHP or Python, but you don’t like having to choose or having resources divided. I have been mulling this for a long time, and more recently discussing it with individuals, but I think the choice has at last become fairly obvious.

The time has come for Sahana phase 2 to move to maintenance-only (and of course paid support and further development is always available from Respere), and for the volunteer community, as a whole, to accept Sahana (Python) as the main development version for volunteer developers. Naturally, if volunteers want to continue support and development of PHP there is nothing stopping them, it is after all open source software.

Sahana (PHP) has done a fantastic job over the past five years, but I think it is time we seriously consider handing the baton over to Sahana (Python) to take us the next five. This is essential not because of the technology, but for community stability and providing a united front to the world at large.

Respectfully yours,

Gavin

[1] http://sourceforge.net/projects/osveoc/ (registered 2003-08-08)
[2] http://www.rediguana.co.nz/gav/2004/02/01/emergency-management-and-open-source-software/
[3] http://www.iaem.com/publications/documents/PrinciplesofEmergencyManagement.pdf

This is the presentation I gave this evening to the Canterbury Branch of the New Zealand Computer Society.

This is the original text I submitted to The Box feature on Disaster Tech on Tuesday the 2nd of June, 2009. It is archived here for my records. It also includes some additional content that didn’t make it to the print edition.

On December 26, 2004, the Boxing Day tsunami killed over 35 thousand people and displaced over half a million people in Sri Lanka alone. A massive humanitarian crisis played out in numerous other countries also affected by the magnitude 9+ Great Sumatra-Andaman earthquake and resulting tsunami. Within days it became apparent that an information system was needed to manage the massive amounts of information being generated about who was doing what, and where – at one point there were approximately 1,100 registered NGO’s operating in Sri Lanka.

It was decided by a group of Sri Lankan IT professionals that a system needed to be built to better manage the information as they couldn’t find any existing free solutions that could be quickly deployed. Free, was critical, as they couldn’t afford any commercial solutions.

Sahana was implemented within a week by around four hundred IT volunteers, and it was named after the Sinhalese word for relief. Initially it provided tools for tracking missing persons, organisations involved in response, locations and details of camps set up in response to the tsunami, and a means of accepting requests for resources such as food, water and medicine.

Following the tsunami, the Swedish International Development Agency provided funding to take the lessons learnt from writing and deploying software during a disaster, and to rebuild Sahana from the ground up, and release it as free and open source software to the world. After all, Sri Lanka had needed an open and available system to manage disaster information, surely other countries should benefit from their experience?

Since 2005, Sahana has been officially deployed to earthquakes in Pakistan, Indonesia, China and Peru; a mudslide in the Philippines; and has been deployed in New York City as a preparedness measure to help manage storm evacuations.

Being free and open source software has been critical to Sahana’s success. The more accessible a system is, the more likely it is to be adopted, used and improved. Even in developed countries, many disaster agencies are poorly funded and often cannot justify significant expenditure on systems – commercial systems are too expensive. With pressure being applied to many public budgets, the significance of this is even greater now. Perhaps the greatest benefit of applying open source approaches is that it encourages a collaborative and communal approach to improving the system. As more countries with experience in disaster management contribute to its development, this will also act as a form of expertise transfer to countries that may not have as much experience with disasters.

Following Hurricane Katrina, there were nearly 50 websites created to track missing and displaced persons – all using different systems, all collecting duplicate information, and few of them sharing. Many of the potential benefits of the technology were lost due to a lack of co-ordination and massive replication of data. Access to tools such as Sahana will be more efficient as they can be deployed faster than solutions developed after an event occurs.

Normally, management involves a ‘leisurely’ process to collect as much information as possible, to then decide what actions should be taken. This is completely the opposite immediately following a disaster whereby decisions have to be made, sometimes with little or no information and no time to gather it.

A key benefit that IT can provide is in linking silos of information held by different organisations – everyone has a better shared picture of what has happened, what is occurring now, and what is planned.

Software, however, is just one aspect. There is a need for open data (such as maps and statistics) and standards to ensure that the multitude of systems can connect to each other and share information.

The most important aspect is having the relationships between organisations set up in advance of a disaster. This results in organisations having the confidence to connect their systems and share information. Without shared information the rest of the system will lose many potential benefits that IT can bring to disaster management.

Often, little or no information is available to support decision-making – emergency managers are forced to make complex decisions without having the luxury of all the required information.

A disaster can produce a massive number of tasks requiring hundreds of organisations and thousands of people to co-ordinate activity – meaning that there will always be some prioritisation needed. What should be done first? What can wait until later? How should an impacted community prioritise response and recovery with limited resources?

The benefits are not just limited to agencies and NGO’s. The next evolutionary step will be to adopt an approach called ‘crowd sourcing’ whereby members of the community are provided with tools to interact with each other and emergency managers.

This may be achieved with applications that run on mobile phones linking people and even submitting information from the field directly to Sahana servers. Imagine the situation where a passerby can take a georeferenced photo of some disaster damage, and if communications networks are working, send that directly to the system emergency managers are using to manage the event. There are a numberof efforts underway looking at how social networks and websites such as Facebook and Twitter can be utilised during a disaster.

Disaster IT is really a force multiplier. It won’t usually save lives, but it will allow a better shared understanding of the problems, and will lead to more effective and co-ordinated response. It allows those responding to an event, whether an organisation or individual, to quickly access information and better inform decision-making. This can lead to less suffering and a quicker recovery for affected communities.

Design for Disaster
Computer systems can often be fragile by their design – they are especially reliant upon power and communications. If any of these are lost during a disaster, the value of a system can quickly be lost if it has not been designed to operate in adverse environments. Here are some design decisions that are very important for disaster applications:

• Low bandwidth – we’ve all become accustomed to sucking bandwidth through massive broadband pipes, but during a disaster network connectivity for emergency managers may be limited to dialup speeds over satellite or digital radio connections. Disaster software needs to be designed for very efficient transfer of information, and should never assume vast quantities of bandwidth are available. At at extreme, some information may even be transferred by SMS or USB memory stick.
• Intermittent connectivity – during a disaster communications will likely fail multiple times before they are finally restored. This means that most ’software as a service’ or web applications on the Internet will be of little use to emergency managers. Disaster software needs to be stored and run locally, and be able to work without a connection to the Internet.
• Synchronisation – one of the best techniques for designing around low bandwidth and intermittent connectivity, is to design a system to be able to synchronise information between two systems when communications are available. When communications later fail, both systems will have a copy of the same data, and can access it locally until communications are restored.
• Low power – power can, and will fail during a disaster, so disaster software needs to be designed to run on low power devices. Laptops and notebooks are good targets as they are self-contained, have built-in batteries, and can be charged from solar cells or generators. Large, power hungry servers can be difficult to move and support in a disaster environment.

How I became involved

One might ask how a Kiwi became involved in Sahana. Ever since training as a Civil Defence volunteer in the late 90’s, I had an interest in how information technology could be used to improve disaster management. The tsunami in 2004 acted as the catalyst for Sri Lankan computer programmers to produce Sahana. I have been volunteering with the project since 2005. In September 2005, he helped facilitate a workshop in Colombo that formed the basis for the current version of Sahana. In March this year he attended a Sahana conference and Board meeting in Sri Lanka. At the Board meeting the existing ‘owner’ of Sahana – the Lanka Software Foundation – agreed to hand the project over to the open source community. Gavin is a member of the transition Board that is in the process of forming an international non-profit foundation that can accept financial donations, and act as the ‘custodian’ of Sahana.

How you can help

There are numerous ways Sahana is looking for help. Once registered, we will be able to accept financial donations that will be used to fund development. In the meantime, we are looking for open source programmers with web development skills (including mapping). If you’re not a programmer, we are always looking for translators that can convert the english text and documentation into many different languages. Perhaps most importantly, we are looking for experienced emergency managers to help provide design advice to the Sahana community and guide the developers.

The issue of community-produced maps has reared its head on the IAEM email list today – closely linked to my post back on the 26th. The following issue was raised, and I wanted to share my reply to this.

Lack of citation was my major concern with the other available maps that have been in wide circulation. The second concern with the other maps is that they showed push-pins when they did not have or could not cite the data to support specific points.

My reply follows:

I think you’ll find that most of those maps do actually have references, in the case of the Google Maps mash-ups, they are contained in the hundreds of comments accessible from the same page as the maps. In fact, it is generally from the posting of these references in the comments, that the Google Maps get updated. What they have failed to to is to make it easier to reference the citations, by not including the reference in the popup bubble above the marker. But if you read through all the comments, you’ll likely find most of the citations there.

Another big failure is to create a timeline/history so that one can see the growth/change in numbers over time for each marker. Most of the maps are purely a snapshot of the here-and-now, and give no context via history.

The real point that emergency managers should take away from this is the following.

Agencies that ‘own’ the source information (e.g. CDC, WHO, and health agencies in every other country in this case), really should be publishing authoritative georeferenced data at the source. If agencies did this, then there would be no need for these ‘amateur’ cartographic efforts to hack together information from news, rumours and other sources. It would sure save a lot of time and effort in people trying to recreate information that already exists and either hasn’t been released, or has not been converted to a georeferenced format.

Likewise, it isn’t really the role of companies to provide this information. Once again, they are just filling a gap that we, as emergency managers, have failed to meet.

The mashup culture is a direct result of a failure by emergency managers to make information available in a form that end users clearly want it (as evidenced by the time and effort they will put into recreating the data in the form that they want to use it).

Perhaps we really should start thinking seriously about how we can produce authoritative information in formats that our communities want.

If you have a look at the example map I created in under an hour on the 26th, you’ll note that I created a little table in each popup for a marker that contained a link to the source article, and in the case of the San Diego marker, included daily figures for three days so it was possible to track the state of that marker over time.In addition, I scaled the marker images so that they were more proportional to the number of cases – a marker for each infection quickly produced an unreadable map, hence it seemed a better approach is to produce summary markers for each location, with the size of the marker indicating the numbers.

The real trick is going to be to produce a web application to track and manage this information, that can then export it in a suitable form to display the information as discussed above. This is clearly something we should look at for Sahana.

Well, it has been a lot of work for the admins, the mentors, and the students, but it has paid off. The Sahana has been awarded 10 projects in the 2009 Summer of Code. We have some great projects lined up! The include:

• Person Registry for Sahana
• Warehouse Management
• Disaster Victim Identification
• J2ME clients for form data collection in the field
• Optical Character Recognition for scanning forms
• Peer to peer synchronisation of Sahana servers
• CAP Aggregation and Firefox CAP plugin
• CAP Editing and Publishing
• Mashup/Aggregation Dashboard
• Theme Manager

Having been neck deep in the process; working with others to set up our assessment process, coming up with ideas (I’m stoked to have two students working on CAP ideas that came out of my earlier suggestion), and reviewing each and every of the 45 proposals we recieved, it has been exciting to get so many projects accepted.

I think that by the end of the year, we are going to have some great new functionality available in Sahana. Even more, I hope we’ll attract more open source developers to our ever growing community!

I’ve just uploaded the presentation I gave on Sahana at the Sahana 2009 Conference in Colombo, Sri Lanka on the 25th of March, 2009. I’ll put a link up to the associated paper soon as well.

Here’s another CAP idea I wanted to get out before I read a document I’ve been sent that may cover the same topic (just to make sure I don’t potentially draw on someone else’s idea). This concept came to me, again, last year whilst I was working on the CAENZ Public Alerting research report last year (I’m still waiting for this to be publicly released so I can link to it). My recent post proposing a browser plugin for CAP alerts is part of this bigger picture I am outlining today.

The background for it came from the realisation that there are a significant number of organisations in New Zealand that are responsible for the publication of alerts – whether to a secure group, or the general public. For example, there are 16 CDEMG Groups, 70 odd local authorities, GeoNet, MetService, Police and those responsible for infrastructure such as roads, and the Centre for Critical Infrastructure Protection.

Each of these agencies would need some means of hosting a CAP server, and incorporating some means of resilience into their CAP server(s). Given that there are potentially such a large number of CAP servers required, there are some aspects that could provide a strong and robust CAP network without seeing a proliferation of potentially fragile CAP servers. This is all built on the concept of a secure peer-to-peer network of CAP servers.

Federation
It should be possible to federate a group of CAP servers into a cluster. If we take a CDEM Group as an example, the group members may elect to deploy say 4 or 5 CAP servers to create a peer-to-peer network providing CAP alert hosting for the CDEM Group. Any authorised CAP message posted to one of the federated CAP servers would automatically distributed the CAP message to the other CAP servers in the federation. In this manner, the CAP message is instantly distributed and made available to other servers in the federation.

Peer-to-peer
I believe that the more robust approach to developing a CAP network is to base it upon peer-to-peer network technology, although tweaked to provide a secure means of publishing messages to the network These servers could of course be deployed any way to provide maximum resilience, and may be located close to major New Zealand Internet backbones, and quite possibly well outside of their geographic region. This has two potential benefits for resilience. Firstly, the message is available from multiple servers, so that the load (particularly for publicly accessible CAP servers) can be distributed across the multiple servers automatically. Secondly, should any particular server fail, the messages will still automatically be provided from the other CAP servers in the federation.

One example means by which this could be deployed is the following.

Provide a national CAP server network of federated CAP servers at key points – a nationally managed set of strategically located CAP servers. For example, Government internal CAP servers would be most likely located on the Government Shared Network (GSN) or whatever comes out of the recent restructure of this service. Public servers may be spread around both by geography and ISP (e.g. key ISPs may host a CAP server for their customers). In all circumstances these would fallback to other CAP servers in the federation in case of their failure.

Naturally, the open approach applied to peer-to-peer file sharing is not appropriate for a trusted network CAP service. To create a more secure network, something like a two-tier approach may be necessary.

CAP Publishing Servers
Private CAP publishing servers may be utilised to act as the publishing gateway to the public read-only peer-to-peer network provided by the CAP Read-only servers. Authentication, encryption and/or digital signing should be used as the basis to authorise the publication of a CAP message via the publishing server. The publishing server is responsible for verifying the digitally signed CAP alert, as well as the authentication details to verify the user is authorised to post the alert. Once authorised, the CAP publishing server publishes the alert to the road-only servers. This is the only channel to publishing CAP alerts to the network. Some form of CAP writing software (or service) may be useful for creating CAP messages and then publishing them to the servers. One protocol that may be useful for publishing is Atom as suggested by this IBM article.

CAP Read-only Servers
These are the user-facing servers that provide CAP messages to their end users. Only the CAP publishing servers are authorised to publish CAP messages to the peer-to-peer network for dissemination.

Naturally, this concept is part of a larger plan to build a CAP framework, and the circle would be able to be partly completed by designing web browser plugins that are capable of connecting to the peer-to-peer CAP read-only servers.

Widespread deployment of CAP browser plugins may mean that traditional servers may not be capable of supporting tens or hundreds of thousands of CAP clients regularly checking for new alerts. A peer-to-peer approach will probably provide the most scaleable and robust approach to disseminating CAP alerts via the Internet.

I haven’t blogged about Sahana for a long time, and I’ve got plenty to write. So much that I can’t decide where to start, so I’m going to pick a nice small piece to start with.

The Concept

Last year, I was involved in a project in New Zealand to produce an investigative report on Public Alerting Systems with the New Zealand Centre for Advanced Engineering. This report will hopefully soon go public, and I’ll provide a link when it does.

This report was looking at the different technological solutions for getting alerts out to people in as timely a manner as possible. At one point in the search for different systems, we started discussing means of injecting HTML in web pages via an ISP, so that a public alert could be sent out to anyone on the Internet. I’ll talk about this and other options later. Let me get to the point of this post.

After starting at the HTML injection idea, and progressing through a few others, I reached a kind of natural conclusion that a more suitable means of alerting users via a web browser would be a browser plugin that can subscribe to Common Alerting Protocol (CAP) feeds, and when a relevant alert comes in via CAP, this is displayed to the user in their browser using the XUL:notificationbox at the top of the webpage.

Draft Requirements

Anyway, a possible idea for a Google Summer of Code 2009 project is that of constructing a browser plugin for Firefox that implements this alerting capability, and expanding Sahana to support full publishing of CAP alerts. Here are some features it could/should support.

Firefox Plugin

• Bundle publicly available CAP feeds (ideally listed in a nice Country/State taxonomy – this will make it easy to discover and utilise existing CAP services.
• Allow users to optionally register location in some manner, so as the plugin can identify relevant alerts (by location) and give them higher status than say remote alerts. Users should be able to register multiple locations – whether it has home & work, or multiple cities. Privacy is of course king and this information must be protected.
• Provide a means of adding additional user provided CAP feeds to the plugin.
• Provide the ability to open the alert in a new tab and format in a human-readable manner, including niceties such as embedding Google Maps to show geospatial information and links back to the source website of the alert for verification.
• Implement means of verifying messages that are digital signed, and decrypting encrypted messages.

Sahana

• Implement a CAP feed in Sahana so that Sahana can act as both a producer (in terms of creating a CAP message) and a publisher (in terms of making it available via a CAP RSS/Atom feed).
• Implement a CAP proxy or similar, so that say all users of a Sahana server can obtain CAP alerts directly from the Sahana server – rather than going to an external website. This may be useful for distribution of alerts within an organisation or centre without having every client browser connecting to an external server.

What would be very nice, but may be beyond the capabilities of Sahana servers currently, is making the CAP service on a Sahana server easily discoverable on a LAN via zero-conf services such as Bonjour.

Draft Outcomes for Assessment

The outcome of such a project would be to produce a working solution whereby a Firefox Browser plugin is capable of working with public CAP alerts and that CAP within Sahana is capable of fully acting as a CAP server via RSS/Atom feeds to the CAP alerting plugin.

Compulsory

• Implement the specified requirements
• The browser plugin works as expected with publicly available CAP feeds.
• The browser plugin works as expected against the Sahana demo server. (Yes, this means that your modifications to CAP on SahanaPHP need to be implemented).

Optional

• Implement the Sahana CAP server in SahanaPY
• Provide one or more standalone CAP clients for a mobile platform e.g. Google Android, Apple iPhone/iPod Touch etc
• Write an Internet Explorer plugin with similar functionality – it is important that this functionality is also provided for IE given its widespread usage and deployment.

Whilst the plug-in can and should operate completely independently of Sahana, it should also be designed to work well with Sahana servers (e.g. SahanaPHP and SahanaPY).

Anyway, this is just an idea I wanted to float and get out in the community for discussion. I’d welcome any further comment or ideas to build upon this!

This time it is BusinessWeek promoting the “Do-Good Imperative” – including free and open source software. There is also a sister article on collaborative map-making during emergencies using the likes of OpenStreetMap. Naturally, this is an area that we are working hard on building these geospatial capabilities into Sahana as well.

Open source and collaborative approachs are certainly starting to get the mainstream attention that they deserve. Now we just need funding to support these developments.