Google ambivalence

My good friend Kevin loves Google. For Kevin Google embodies many of the virtues that make IT such an interesting and fascinating place in which to work. Kevin appreciates Google’s creativity. He admires its business model. He harnesses its accessibility and, above all, he supports its attempts to break the traditional software licensing stranglehold that has underpinned the behemoths of the software industry for so long. Kevin’s admiration of Google is so high anytime I raise any complaint or problem that I have with the company his standard retort to me is an incredulous “Goooooogle!”

Over the last couple of months I have had a number of reasons to complain about Google. As such, I am now much more circumspect about whether Google is quite the force for good that it would have us believe. I can see that Google is helping make key software much more affordable and much more accessible. I also recognise its software offerings are much more versatile. However, I have come to have serious doubts about whether Google is really enterprise ready and I am deeply troubled by its apparent lack of transparency.

My Google tangles began in August this year. Increasingly finding Microsoft Outlook clunky and cumbersome Kevin suggested I give Gmail a go. I did and, to Kevin’s credit, I liked what I saw. I was impressed by the neater integration of email with calendars and contacts. I liked the ability to harness powerful search facilities to find past emails. I thought the option to tag emails with labels much more helpful than Outlook’s use of folders. In short I was sold. However, that’s when my problems began.

I have a number of email accounts. Moving the Coalface Community account from Outlook to Gmail was straightforward as Kevin had laid the groundwork with Google from the outset when we set up the company two years before. However, this was not the case with my Coalface Research domain which had been set up several years before. Apparently unable to fathom that I had been running this email account for some time, Gmail wanted to assign me an email address that would have read “phind@coalfaceresearch.com@gmail.com”. This was clearly not what I wanted but when I deleted this option I believe I also simultaneously deleted my Coalface Research blog. Unbeknown to me there appeared to be an association between this and my Coalface Research account which was apparently established when Google acquired Blogger. I suspect when I deleted the Coalface Research email address I, by association, inadvertently deleted my blog. I assume this because my blog disappeared at the same time. Anyway, the upshot was that I needed to ask Google’s help to get my blog back.

I cannot find adequate words to articulate the challenge I encountered in just trying to talk to Google about this problem. It was impossible. I have no doubt that it would have been easier for me to have got Kim Jong-il on the phone in his lair in Pyongyang than it would have been to talk to a technician at Google about my problem! There is simply no way to do it. All you can do is communicate through a self-help support Wiki. However, even this presents problems. You cannot just outline your problem and ask for help. You have to put it in one of a select group of pigeon-hole options. Woe betides you if none of these options adequately reflects the problems you are experiencing.

Even getting to this self-help Wiki was confronting. Firstly, I had to battle my way through a condescendingly written missive on the Blogger Help forum which hectored me on the reasons why Google may have elected to remove my blog. I clearly fitted none of these circumstances. However, before being able to proceed I had to state that I had read these points and understood them. After this I documented my problem and waited. Fortuitously for me, and I’m not sure why he did so, a contributor called “Martins” came to my rescue. As I was to discover on another incident this is not always the case.

There then entailed a six week toing and froing of emails between me and Martins. Periodically, I was assured that my blog had been restored but nothing eventuated. There was an enormous temptation to take my frustration out on Martins but I realised that he was the only person around Google that was offering me any encouragement. I suspected letting fly at him would only be counter-productive.

My problems during this time were exacerbated by the fact that I was discovering intricacies to Google about which I had been blissfully unaware. I found that even though I had used Google as my search engine for many years that I had Google account names and passwords and that it was even possible to sign in to Google. These issues were inconsequential to me when I used Google as a web browser. However, they became critical considerations when I wanted to use Google as an email system because my email account and my blog were dependent on my Google account name. Failing to appreciate this critical linkage for some time I succeeded in acquiring a myriad of passwords and logins that only complicated my path to recovery.

Then miraculously my blog reappeared. Mysteriously, but appropriately, it had even been assigned under my Coalface Community account. I wrote to thank Martins for his help. He stressed that my circumstances were complicated and he credited the help of a Google technician called Brett who had done a lot of work behind the scenes on my behalf. Mind you, I still have no idea what Brett did and what challenges he had to overcome.

As you can see I’m happily blogging again. Furthermore, I am using Gmail for both my email accounts and I do think they are a significant improvement on Outlook. However, my experiences with Google have left me very troubled. Kevin stresses that I need to recognise that I’m getting Gmail for free and this implies I can’t expect Enterprise level support. He says my situation would have been different if I had licensed the software. However, I wonder if that is a valid argument. Google’s business model centres on advertising and the more people use its products the bigger is the reach of that advertising. Moreover, experience has also told me that good customer service is a culture and embracing a cavalier service attitude in one part of the business will manifest itself elsewhere.

My biggest concern though is the lack of transparency that I encountered. This is especially troubling when you consider the amount of personal information that Google is acquiring about people as they use its products. When the only way you can communicate with someone is on their terms, which excludes being able to talk to them on the phone, how can you elevate your concerns with them. I’m a small business owner and I was frustrated. Imagine if I had been the CIO in a business owned by a self-made billionaire. How, in such circumstances, could you advise this type of boss with any certainty when a problem was likely to be resolved? With IT, and especially email, being integral to any modern company, we’re sitting and waiting for something to happen would, I’m sure, be a career ending response to proffer.

Nevertheless, there is always tomorrow. Google may have an element about it of what my mother used to describe as the “does the King know about me” syndrome. However, I’ve come to realise that pride always proceeds a fall. If the IT industry has taught me one thing it is that dominance is always a temporary illusion. Just ask IBM and Microsoft. As such, I’ve no doubt that on the horizon there lurks some amazing company with an incredible product set that will eventually challenge Google. I’m also sure that when it arrives my good friend Kevin will tell me all about it.

The need for agility

John Foster Dulles is the man after whom Washington DC airport was named. It was probably an appropriate place for a bureaucrat to be acknowledged. He served as the American Secretary of State for President Eisenhower at the height of the Cold War in the 1950s. I give him to you today for a memorable quote of his. Foster Dulles remarked that the measure of success is not whether you have a tough problem to solve but whether it’s the same problem you had last year!

Last year IBM undertook a survey of CEOs to ask them how they perceived the current economic circumstances. As you can see in this slide CEOs described the new economic environment as substantially more volatile, much more uncertain, increasingly complex and structurally different. Furthermore, the consensus among those interviewed was that this was unlikely to change in the short term. The biggest concern CEOs had was volatility and the challenge when things are volatile is to be able to be agile and fast enough to be able to respond to these changing circumstances.

McKinsey did a study of business executives in 2006 where they explored the importance of speed and agility. As you can see on this slide agility was rated as either very or extremely important by nearly 90% of the respondents. Speed wasn’t far behind. 80% of the same respondents saw this as highly important. Moreover, the respondents reported that if their businesses could increase their speed and agility then they believed that it would enable them to grow their revenues, respond better to clients to assist with customer satisfaction and provide their companies with greater operational efficiencies.

The McKinsey study then looked at what these executives thought were holding back their organisations from being quick and agile. By far the most common response, from half of all respondents, was the complexity of their companies. People spoke of being impeded by slow, centralised or complex decision making. Often these structures have been introduced to improve governance. However, smaller, more nimble, new entrants to a market are not so encumbered. The challenge then for an established organisation is how do you eliminate complexity in order to respond to these new, nimble competitors?

The same can be said of corporate IT. Larger, well-established, organisations tend to possess more complex IT departments which are often a reflection of how their business has grown over time. Ron Ahkenas is a writer for the Harvard Business Review. In a recent interview he was specifically asked how CIOs could reduce complexity in the IT department. Ashkenas identified four areas where he thought an individual IT department created unnecessary complexity. These were: inefficient organisational designs; product and service proliferation; unmanaged process evolution and unintentional managerial behaviours.

In analysing these Ashkenas argued that CIOs first needed to determine where they had opportunities to consolidate the number of systems & functions they supported. Then Ashkenas advocated the need for a greater rigour in culling the products and services that IT supported. Next Ashkenas advised that CIOs needed to have defined the core processes by which the IT department fulfilled its mandate. Finally, Ashkenas recommended that CIOs needed to review their own behaviour to see how this might add to the complexity emanating from their department. Did the IT staff have clear role definitions so people recognised who was accountable for what? If such definitions were not in place he felt that this was a clear recipe for potential confusion.

One of the things that Ashkenas recommends is rationalising the number of applications that their business supports. I feel many CIOs would very much like to do this but see this would meet resistance from the end users of these applications. One of the things I hear regularly in my discussions with CIOs is the burden they carry in managing this plethora of applications. In putting together this presentation I went looking for some evidence of what the cost was for IT departments in doing this work. This slide show some findings from a UK research paper which set out to explore the impact for CIOs of having to maintain a plethora of different software solutions from different vendors in today’s cash-strapped corporate environment.

As you can see their findings showed that over 60% saw software maintenance as a burden on their IT budget. In fact, 40% describe this burden as significant. In addition, the research revealed that nearly a quarter of the average IT budget was spent on software maintenance and for some companies it was 100%!. Moreover, the study showed that 71% of European IT Managers expected that the proportion of their IT budget allocated to the upkeep and maintenance of software packages would increase next year. In effect, these findings highlight the opportunity cost of software maintenance. If you are too busy patching up what you already have there’s insufficient time to help the business with where it wants to go.

Perhaps though there is another way of skinning the cat. One of the attractions that we had to today’s presentation was the fact that NAB took such a radical way to dealing with its quest for agility. In many ways they thought outside the box to the challenge of maintaining their key banking applications. Moreover, they used a separate private cloud to support this new system. As this Gartner research from earlier this year shows, private cloud is still in its infancy. However, perhaps it does offer us the ability to free ourselves from supporting the quagmire of legacy systems that are endemic in more organisations. Harking back to the words of Foster Dulles, they are unlikely to enable us to do away with our problems. However, it might be rewarding for us to start to tackle some new problems for a change.

At the centre of things

They say the world is cyclical. What goes around comes around, so to speak. Most of us are taught this from an early age. We study history to learn from the past and when it comes to trends to understand how better to deal with current challenges. I make mention of this because I believe the IT industry is witnessing a new cycle in its evolution. I think there is renewed faith in IT by the business. Moreover, I reckon that accompanying this is a growing status for CIOs.

In my opinion the IT industry spent much of the last decade in the doldrums. The technology largesse that accompanied Y2K subsequently acted as a disincentive to major investment in IT for many years afterwards. The dot com bust and the unfulfilled prophesies of the early Internet darlings of the stock market fostered a cynicism that IT people were somewhat ignorant of the real world of business. Then along came Nicholas Carr and his treaty that IT Didn’t Matter. For many years after I don’t think it did. It seemed that for most of the last decade CIOs were primarily tasked with doing more with less. Yet now I think the wheel has turned. Everywhere I go I seem to be encountering a growing recognition by many executives that IT is something their businesses need more of not less. They know they cannot ignore it.

One of the great pleasures I get from running The Coalface Community is the opportunity it presents me to have regular dialogue with senior IT executives in a variety of industries. Right now many of these conversations have a consistent theme to them. More and more CIOs are telling me that they are undertaking major projects in their businesses that entail some form of business transformation. Moreover, these are not changes to an application set. They are not just tinkering with the financial systems. Instead they seem to be intimately involved in a complete overhaul of their organisation, usually as a response to major challenges in the marketplace.

Changes of this nature are not for the faint hearted. They usually require a significant investment of both time and money. Yet the impression I get is that senior executives in the business are right behind these initiatives. This appears to reflect something of a renaissance in enthusiasm for IT by many such business executives. For a long time IT has been seen by many of those at the top as something of a necessary evil and with due reason. Research continues to show that the success rate for major IT projects is dire. A number of organisations have got their fingers burnt supporting major IT projects that have failed to deliver. However, many executives now seem to be appreciating that if they don’t watch out their industry sector could be turned upside down by someone much more technologically savvy than them.

There is a lot of evidence of the capacity for technology to disrupt a marketplace. The turmoil in the newspaper industry around the world is one classic example of this. Rupert Murdoch’s animosity to Google is not without reason. For centuries the newspaper operating model has been underpinned by classified advertising. However, in recent times there has been a flight of classified advertising from newspapers to the Internet. Murdoch fully appreciates that the future of his organisation is tied up in how well his business can respond to these threats. Moreover, he knows the response he takes must be IT centric. If it is going to get harder and harder to secure classified advertising then News Corporation has to develop new, alternative revenue streams. Moves towards paid content, placing web sites behind pay walls and creating multi-media newspapers are all examples of how Murdoch’s IT department is central to helping his business respond to these threats and opportunities.

Newspapers are not alone. Commercial television, the recording industry and the travel industry have all been turned on their head recently by challenges posed by the Internet. Even behemoth industries like banking are not immune from these threats. More and more people are comfortable paying their bills and doing their banking on line. These people don’t have the same need for a bank branch that their parents may have had. They are comfortable dealing online and anonymously with a Bank where they don’t know the Manager or the teller. As such, creating a new financial services organisation is much simpler. As such, the cost of entry for new financial service providers has fallen rapidly. The next Coalface Community session looks at UBank, a brand new alternative bank created by the National Australia Bank in order to respond to these very challenges posed by these new emerging entrants in to the local Australian banking sector.

There is another characteristic that comes out of my dialogue with IT executives dealing with major business transformation projects. They all talk about these change management initiatives as being lengthy journeys. The focus is not on crashing something through at breakneck speed. It’s on doing it right. The business knows that if it wants any change of this magnitude to be effective then this will not be an overnight occurrence. Radical change of this nature takes years to realise and it requires effective planning, sufficient resources and ongoing commitment.

This is why I believe there is a growing status for CIOs in the corporate world. Leading CIOs are agents of change. They are tasked with overseeing fundamental business transformations that are going to be integral to their organisation for years to come. To achieve this they must reside within the inner executive circle. This is where they will be privy to confidential, strategic discussions. This is where they will be arguing for the monies for substantial IT projects often in competition with others on the executive team who may be advocating important and scarce capital should be spent elsewhere.

These are clearly exciting times for CIOs. No one talks today about IT not mattering. No serious business executive thinks that IT is a cost to be minimised and that the CIO should just focus on keeping everything running. The long aspired entrance to the inner sanctum of the corporate executive team seems to be at hand. The task now is to seize the opportunity. If we follow the fact that business is cyclical it is probable that sometime in the future the wheel could turn in the opposite direction. However, if CIOs perform when they are in the spotlight like now it might be a long time before that happens.

Harnessing mobility

Winston Churchill once remarked “however beautiful the strategy, you should occasionally look at the results”. His remarks seem particularly pertinent for CIOs equipping their organisation with mobility technology. The IT industry has long heralded the potential of mobility to transform our businesses. Users can clearly get enthralled with each new release of mobile product sets. Yet when you examine where the functionality is applied it is clear that for many mobility applications are still confined to telephony and email. When Gartner asked CIOs around the world in 2006 to identify their technology priorities for the year ahead they ranked mobility as number three. Unfortunately, when they ran the same survey this year mobility was still in the same position!

This is despite the fact that there have been many significant advances in mobility technology since 2006. The first smart phones appeared in 2007. Last year we witnessed the iPad phenomenon. This year a global study by the UK research company Freeform Dynamics revealed that over half the organisations gave 50% of their employees a laptop while over a quarter of the organisations gave a smart phone to the same percentage of staff. Nevertheless, despite wide interest in it, and I suspect many protests, it was also clear from this study that all bar the privileged few were being allocated an iPad.

Moreover, there was also strong evidence that the laptop would continue to play an important role in the enterprise for at least the foreseeable future. Over the next three years, around 55% of respondents regarded it as remaining a highly important device, a figure that was noticeably higher than even the smart phone. On the other hand, even allowing for two or three more subsequent releases, iPad type devices were only seen as being of marginal importance by then for the majority of respondents. However, there was also strong evidence that employees will increasingly be equipped with a number of devices. These figures are supported by IBRS’s own research in Australia which has found a growing trend towards key executives carrying up to three distinct mobility products.

Despite this proliferation of mobility functionality, Freeform reported that IT executives felt that the products were still immature in a number of areas. There were particular concerns around the lack of device management tools, especially those capable of supporting multiple mobile products. It seems that while systems management tools on the desktop have matured considerably in recent years they are still almost nascent on devices like the smart phone and the iPad.

This issue is reflected in the strong call for improved data synchronisation functionality across mobility devices. It is common to hear IS executives complain that Apple, for example, which has created such phenomenally popular consumer devices as the iPhone and the iPad, has a naivety, or perhaps even a disinterest, in satisfying the requirements of the enterprise. Many IT executives lament the fact that they appear to be bringing a consumer mindset to the requirements of a corporation. Data synchronisation might be a “nice to have” for a consumer but it is essential for an executive.

Aside from the systems management aspect of mobility, it is apparent that the increasing consumerisation of IT has confronted CIOs with other challenges. Another piece of research by Freeform Dynamics in 2010 examined what internal practicalities around mobility presented the most challenges to them. Top of this list, and nominated by over 60% of respondents, was the challenge of dealing with user expectations. One only has to look at the queues of consumers, and in particular Gen Y people, who are prepared to line up outside a store overnight to get the latest release of a smart phone or a tablet to appreciate the strong demands that are likely to be made on corporate IT budgets to acquire these products.

The other major challenge highlighted were issues with integrating mobility equipment to corporate applications. Nearly 60% of respondents reported difficulties in this area. It seems that while mobility advocates have long trumpeted the ability to work anywhere or anytime it seems that for many this will only apply if you using the technology for email and telephony. Perhaps this difficulty is heightened by the inadequacy of synchronisation functionality.

Nevertheless, it is security that has long been perceived as the Achilles heel with mobility. How do you safeguard corporate data when you leave the organisation’s premises? The proliferation of mobile devices and form factors obviously has the potential to exacerbate this problem. The Freeform study asked CIOs to identify where they thought security needed to be enhanced in mobility. Top of the list was a call for better security to be built in to the devices. This includes features such as stronger access control and data encryption capabilities. There was also an appreciation that security policies needed to be consistent between mobile and static devices while others highlighted the need to consider what is safe when a device attempts to connect to a corporate network, especially when this is done via a public network.

However, perhaps the biggest issue about security is the one that seems to be looming on the horizon. With the increasing consumerisation of IT many staff are arguing for what is termed “bring your own computing” or BYO. Their argument is that the devices they use at home are superior to what is provided to them in the office. Moreover, having to handle both a personal and business device is likely to lead to a fragmentation of data between the various pieces of equipment making it harder to locate vital information. As such, they argue it will make more sense all round to subsidise staff for the use of their own equipment.

The Freeform study also examined this question. 25% of respondents reported half of their employees were using personal equipment which probably shows that BYO computing is still in its infancy. However, the findings also recorded a number of concerns from IT executives with this trend. In particular, they were worried about important corporate information residing on insecure machines that may be left lying around at home. Nevertheless, most acknowledged their concerns reflected an immaturity in the BYO approach and expected this trend to gain weight as better work practices and disciplines were established around it.

BYO though does go to the heart of the mobility promise. This is the opportunity to work anywhere, anytime. It is clear that more and more employees see such a possibility offers them the real ability to strike an effective work life balance while also enabling them to be judged more on the outputs they deliver rather than their visibility around the office. As such, demands for improved mobility functionality are probably only going to increase. Who knows then how far mobility may take us. Will most of us end up as totally mobile employees? If so, could the concept of the office as a central working place be relegated to a thing of the past?

Ducks talking to chickens

One of my wife’s favourite sayings is “ducks talking to chickens”. She uses it to describe the inability of one group of people to communicate with another. My wife is Chinese and I think the origins of the saying were in the 1980s when China was opening up to the world. Misunderstandings between the Chinese and foreign tourists were frequent as each realised the other had a different view of the world to their own. The conversation could be in English but the interpretation made by each party could be wildly different.

The first time I heard my wife use this saying was when she gave it to her boss. She works in the finance department of a large global retailer and she was describing the inability of some in the finance department to communicate with executives in other parts of the business. She realised that some of her less experienced colleagues had an inability to explain their actions and requests in a way that could be understood by someone who was not working in finance.

There is no doubt IT is an industry that certainly possesses quite a few people who could be described as “uncommunicative ducks”. Initially, I think this was done by design. These were the times when IT was overseen by people in white lab coats who worked in a place called the data centre where only a privileged few were allowed to tread. The mystery was compounded by the presence of a glass window which allowed people to see in to a world where they could never go. By talking about things that were not understandable to others, IT built up an illusion that it was something complex that was best left to the experts. The fact that these experts were quite well paid only added to the desire of the IT industry to keep itself above the hoi-polloi of the rest of the business.

However, as IT has become increasingly mainstream and consumerised it is clear that many in the industry have failed to understand the impact of the transition that IT has made. With IT being increasingly at the epicentre of the operations of most organisations any business change always has an IT consequence. Executives need to understand the impact of these changes and getting back incomprehensible or irrelevant answers from staff in the IT department is clearly exasperating for them.

This frustration is, I believe, one of the main drivers towards the current moves to cloud computing. Moreover, I believe this has been part of a long term trend. It follows a pursuit that began with outsourcing in the nineties and offshoring in the noughties. I contend that many business executives have long held a desire to rid themselves of the baggage of an operationally focused IT department. It costs a motza and the returns have been patchy. In fact, a COO at a large Australian insurance company once told me that after real estate and wages IT was the third major expense on the corporate balance sheet. The business is utterly dependent on IT. Yet many of those working in it seem to have no appreciation of what they can do to enhance the business that pays their wages.

It stands to reason that any executive worth their salt needs to know how the organisation can better leverage its investment in IT. They clearly want to talk to people who understand the challenges the organisation faces and who are proactive in promoting ideas and suggestions of how these challenges can be better addressed. What they don’t want is to talk to people who see their sole purpose in life is to keep the IT trains in the business running.

Several CIOs have recently told me that they think this problem is getting worse. In particular, they lament the business naivety of the recent graduates they hire. They find them obsessed with technology but unable to engage with others in the business to determine their requirements. One CIO though wondered if this had always been the case. He thought most of us usually only learn on the job. How then can a CIO help the business literacy of their IT department?

For starters, I believe there are some small things that a CIO can do in this regard. In particular, the CIO could begin by improving the level of communication of their IT staff. A simple first step would be to name equipment by the function that it performs rather than letters or numbers. The email server is down highlights the impact of the problem. Server 25 is offline doesn’t quite make the same impression. In addition, I think new recruits to the IT department should be put through some rudimentary course in public speaking. This would give them insights in how to better communicate with others. Speaking as someone who has been a Toastmaster for over 25 years, I would suggest a CIO encourages their reports to join a local Toastmasters club. I am sure many would find it an inexpensive way for their staff to become competent communicators.

In addition, I think it is important for the CIO to walk the talk. The CIO should be the person who gathers data on business challenges, strategies and objectives and who feeds this information back to their team. Their business focus will be seen by others in the language they speak, the magazines they read and the seminars they attend. All of these can be, and should be, a reflection of what they see as important for the IT department. Not only will this focus enhance their dialogue with their business counterparts but also it will help elevate the overall thinking of their IT department.

However, I think the best thing a CIO can do to elevate their IT department is to free it from the operational baggage. There has been a lot of discussion in the IT press about cloud computing, though it is quite hard to find actual IT departments that have made serious commitments to going down this path. Yet while it might not be happening now in earnest I can see something inevitable about cloud computing as the IT delivery model for the future. It delegates responsibility for IT infrastructure away from the internal IT department and ensures the remaining staff need not concern themselves with the business’s technology plumbing. Instead they must think about how IT can deliver competitive advantage to their organisation. In other words I think cloud computing has the capacity to free the IT department from its operational chains to enable it to take a more strategic perspective.

This, I believe, is the language that business executives want to hear from their IT department. IT can be a huge catalyst for change. It has the potential to disrupt and destroy long established working practices. Business executives know they need to be on top or ahead of these trends. To do this they need an IT department that sees its role as strategic rather than operational. When this happens I have no doubt that the ducks and chickens of business will be talking the same language.

What is the ROI on a DR plan?

We are all lucky to be alive. According to the US based preacher Harold Camping the world was meant to have ended on May 21st 2011. Mercifully, the divine disaster recovery plan must have kicked in. However, could we be so lucky in IT? We all know that IT systems lie at the epicentre of our businesses. They are the conduit through which our company conducts its operations. Their availability is integral to the effectiveness of the workforce. Yet, despite this dependence what is the true financial impact if they are unavailable? In other words how can you cost justify an investment in DR and BCP functionality to ensure that the effects of any malfunction in the IT environment are minimised.

This is the question that the American based research company The Aberdeen Group attempted to answer in a study it conducted last year in over 100 organisations that had a formal Disaster Recovery (DR) program. Aberdeen’s conclusion was that the cost of any business interruption event was a direct correlation to the effectiveness of the DR/BCP environment. Best in class organisations had on average fewer disruptions which were cheaper to rectify. On the other hand, laggards had more disruptions which cost their business on average nearly $3 million a year.

Aberdeen ranked organisations by the number of disruptions they recorded each year, the time it took to recover from these disruptions and how close the IT department was to meeting its SLAs with the business over data availability. Best in class businesses recorded fewer than 1 business interruptions in the last 12 months from which they required less than an hour to recover. In so doing they were able to meet 95% of their organisation’s data availability SLAs over the previous 12 months. Laggards on the other hand didn’t know or measure how well they met their business data availability measures. Perhaps this was fortuitous as they averaged more than 2 business disruptions over this period which took five hours recovery time.

Aberdeen attributed the success of the best in class businesses to a number of attributes. Firstly, there was an appreciation of the impact of any business interruption. Next these organisations had a back up and recovery strategy for each critical business element and were able to replicate their infrastructure in remote locations. In addition, DR was the responsibility of a cross-functional team who utilised a documented DR plan. This team reported to an executive champion who was incentivised to reduce downtime. Furthermore, the DR plan was updated regularly to reflect any changes in the corporate environment. Aberdeen also identified some of the tactics and tools that best in class DR businesses used. Some that are noteworthy include: the use of virtualisation; the engagement of an external consultant to provide a broader perspective to DR needs, the ability to measure the cost of any downtime and, finally, testing regularly a number of DR scenarios.

In contrasting the three groups of businesses, (i.e. best-in-class, industry average and laggards), Aberdeen identified four of the biggest differences between them. The most noticeable of these was having a senior manager accountable for DR performance. Almost every one of the best-in-class organisations had such an executive assigned compared with only 27% for the laggards. Then it was interesting to note the difference between the best performers and the rest in terms of establishing cross-functional teams with responsibility for DR. The industry average group actually trailed the laggards in this regard. However, this was still quite a bit less than half of such respondents. In contrast 78% of the best-in-class businesses had set up such a team. Other areas where these best performers were distinguished from their rivals was in the area of staff training on DR policies and the regular testing of different DR scenarios.

For those wishing to aspire to the best performers in the area of DR and BCP Aberdeen make a number of recommendations in their report. In particular, they believe it is essential to measure the financial impact of any downtime in your organisation. In many ways this reminds me of the advice an earlier Coalface speaker gave at a session last August when he stressed the importance of never wasting a crisis. These crisis’s give you the ammunition to outline the potential ROI of a business case for better DR.

Aberdeen’s other suggestions are to formalise a plan that emanates from the executive down and which is regularly updated as the business changes. They also stress the need to test this plan regularly to avoid any unexpected surprise omissions in a real life crisis and to invest in duplicate equipment and image-based backup solutions to accelerate the recovery speed.

In conclusion, one of the learnings I take away from this case study is how the IT industry has grown in its capability in the area of Disaster Recovery & BCP. For a long time it seemed that these were just terms for back up and recovery. Then in the 1990s came the concept of mirrored disks, replication and data snapshots. However, while this satisfied the needs for back-up the challenge then became how quickly could you recover from these back-ups to become operational again. As such, this need led to developments with clustered data centres running in active/passive and, more progressively, active/active mode so applications ran simultaneously in different locations. Therefore, it became easy to fall back to one of these environments if there was a problem with the other one.

The goal though for many businesses remains one of ensuring continuous availability. In such an environment there is no downtime and no user is inconvenienced and unable to do their job. Recent high profile examples in Australia highlight the IT industry is not there yet. However, we should not forget how far we have come in the DR/BCP journey in a comparatively short period of time. With this level of progress the goal of being able to ensure continuous availability is something that we are probably likely to see attainable within the next decade.

Back up and running

I blame Virgin. Up until late last year everything in IT seemed honky dory. However, ever since the Virgin reservation system went belly-up for a few days late last year there seems to have been a succession of IT failures that have caught the attention of the mainstream press. ATM’s have either been offline in major banks or else they have permitted unauthorised withdrawals. Elsewhere a financial institution in Queensland was off air for several days in January when both its production and back up data centres were affected by the Queensland floods. Yet to the layman all this seems inconceivable.

IT is at the epicentre of most organisations today. If your IT is down then so is your business. It clearly is a matter that regulators have recognised. Organisations like the Australian Prudential Regulatory Authority (APRA) certify the operational procedures of Australian financial institutions and one area they pay particular attention to are the disaster recovery procedures. They do this because they recognise that the integrity of any finance company is closely aligned with the robustness of its IT operations. I suspect, for similar reasons, this is why disaster recovery arrangements are a key element of SOX compliance. Similarly, I know disaster recovery is an area covered by the emerging IT governance frameworks like ISO 38001. Yet, why, with all this focus on compliance in recent years, have these IT disasters eventuated?

My colleague Kevin McIsaac eloquently summed it up for me the other day. He regarded disaster recovery as a no win topic for most IT executives. Like other security expenditure such as insurance it is a grudge investment. You do it because you have to but you hope you never have to use it. As such, there can be a tendency in even the best businesses to under invest in this area. The business is not enthused about it because they question how expenditure here will help the bottom line. IT executives are loath to highlight deficiencies here because they know they will only get lukewarm support from the business and they have other things to worry about. As such, Kevin believed many organisations took a wing and a prayer attitude to disaster recovery.

Disaster recovery will be the topic for the next Coalface session. The presenter will be the General Manager of IT at a second tier bank. One Saturday afternoon four years ago an electrician who was working to increase the capacity of the company’s data centre inadvertently plugged the wrong device in to a socket. In so doing he fused the electrics and plunged the data centre in to darkness. Pretty soon the room was full of smoke. The IT executive now had a golden opportunity to activate the company’s rigorously tested, thoroughly documented, independently audited, regulatory compliant, disaster recovery (DR) and business continuity (BCP) plans. He confidently reported to his management that he expected things would be back to normal in about four hours. Seventeen hours later, as Saturday evening rolled in to Sunday morning, he began to understand the deficiencies in these arrangements.

While the IT team eventually got the production data centre back up and running it was clear that the episode could easily have been a disaster. The following Monday the executive reported his concerns to his CEO who immediately wrote him a cheque for several million dollars and gave him a mandate to fix it. The CEO had done a rough, back of the envelope, calculation of just how much the unavailability of the IT systems for several days would cost the organisation. He realised that it would be foolhardy not to address these deficiencies in the DR arrangements.

Since 2007 the IT executive has been diligently improving the IT infrastructure in the business to enhance the effectiveness of DR arrangements. Moreover, he has engaged external consultants to help him document the necessary processes, and the personal who are responsible in each of these processes, on an easily read A3 document which can act as a key reference document in an emergency. One of the lessons the company had learned from its disaster in 2007 was that a 250 page detailed disaster recovery plan is not much use in a real life crisis. More recently, he has evolved his operations to an active-active arrangement across two remote datacentres which includes the ability to cluster a production database across the two data centres and concurrently write transactional data at both sites while retaining full data consistency.

I think he has a great story to tell. Yet I have to say I am putting on this session with some trepidation. I have quite a bit of interest from a number of members. Some have told me that they realise their company needs to do more in the area of DR, something that is highlighted by the fact that it often takes them two weeks to prepare for A DR test. Yet several other members have been quite dismissive of this topic as a session. They have told me they think their disaster arrangements are tested frequently and are well understood. They regard them as a strong point in their IT operations. As such, they doubt whether they can learn anything from someone else. Moreover, they believe the problems experienced by the Bank in question are a reflection that its DR arrangements were inadequate in the first place.

Nevertheless, in the balance of things I have decided to use this case study as the next Coalface session. I believe the growing number of high profile and prestigious organisations that have experienced significant issues with their DR and BCP plans in recent months is evidence that more needs to be done in this area. As such, I felt that the chance to hear from a local counterpart who has had to address a near death experience with DR strikes me as an invaluable learning experience for others and one that is very much in keeping with the ethos behind The Coalface Community.

However, there was also a personal factor in this. Last weekend I had my own near death experience. Driving on the wide suburban streets of Charters Towers in outback Queensland I, inadvertently, missed a STOP sign and ploughed head long in to a car on the primary road. Thanks to modern safety standards in cars my brother and I and the lady I struck walked away unharmed from what was an horrific accident. For me the episode was both a humiliating experience, given my own incompetence put three people’s lives at risk, but, perhaps more importantly, a wake up call. Never again will I deride the importance of seat belts and air bags in cars. I’m sure they saved my life. Therefore, I suppose I’m living evidence that you need to live through a potential disaster before you truly appreciate the safeguards you need to apply.