October 6th, 2008

Improve your failed IT culture

Posted by Michael Krigsman @ 7:42 am

Categories: IT issues, CIO issues, Risk, Research and statistics, Cultural issues

Tags: Team, Culture, Information Technology, Risk Management, Enterprise Risk Management, Strategy, Business Security, Business Operations, Management, Michael Krigsman

The underpinnings of IT failure lie in culture, the unspoken rules governing an organization’s style and general priorities. Since most organizations pay little attention to project culture, it’s not surprising failure rates remain high.

New research by SAS sheds light on this issue. In a survey of 316 senior financial industry executives sponsored by SAS, the Economist Intelligence Unit explored the role of culture in enterprise risk management (ERM):

Creating a culture for risk management is a challenging proposition for most firms. One of the keys to successful risk management is embedding risk management within the company culture, but for surveyed executives this was the most widely encountered challenge, cited by almost one-half of respondents.

Although a bit hard to read, the following chart clearly shows most respondent’s organizations don’t have a high-priority culture around enterprise risk management. That’s a problem, given the importance of this topic:

Read the rest of this entry »

August 27th, 2008

MediaMax / The Linkup: When the cloud fails

Posted by Michael Krigsman @ 9:55 am

Categories: Project failures, IT issues, SaaS, PaaS, and SOA, CIO issues, Financial impact, Enterprise 2.0, Availability and reliability, Failure 2.0, End-user impact

Tags: Information Technology, Data, Failure, Storage, Hardware, Michael Krigsman

Online storage service MediaMax, also called The Linkup, went out of business following a system administration error that deleted active customer data. The defunct company leaves behind unhappy users and raises questions about the reliability of cloud computing.

According to Nirvanix, which along with MediaMax was spun out of a company called Streamload, a faulty script caused the problem:

Streamload offered unlimited and then 25 GB of free storage for quite some time. This resulted in a tremendous amount of data stored in a few million free, non-active accounts for [nine years]. Streamload was literally paying for former users to store 100’s of terabytes of old, inactive data for free. In preparation for the split of the two companies, and subsequent move of the MediaMax application to SAVVIS, it was determined that the inactive data from former users would be purged on the Streamload/MediaMax storage system, thus shrinking the overall storage needs and costs for the new MediaMax company. During this process, a system administrator ran a script that misidentified active account data and disassociated physical files from their owners.

Although The Linkup lost a ton of customer data, CEO Steve Iverson told Network World he’s unsure how much is gone:

Iverson says at least 55% of the data was safe. How much of the remaining 45% was saved is not clear, he says. “We know there was definitely a lot of customer problems, and when we looked at some individual accounts, some people didn’t have any files, and some people had all their files.”

THE PROJECT FAILURES ANALYSIS

As with most failures, this story is fraught with complications and contradictions. Besides finger pointing and back-biting, which I suppose is to be expected, confusing corporate relationships coupled with a seemingly bizarre level of process and technical carelessness lend a weird flavor to the whole mess.

The human drama is documented in links from this post; more importantly, two significant and highly connected issues were at play:

1. Business process failures. Apparently, the company allowed a lone system administrator to perform tasks affecting the company’s core business without sufficiently performing dry runs. I suppose this point is self-evident: scenario planning is critical whenever IT handles irreplaceable data. Management is responsible for establishing all operating plans and contingency procedures before IT executes data-threatening procedures.
2. Technical failures. Given the high stakes and the script’s intended goal, the company should have performed intensive testing ahead of time.

I asked Newsgator’s VP of Software as a Service (SaaS), Jeff Nolan, to comment. Jeff questioned why the company maintained so much historical data:

Beyond meeting legitimate business and regulatory requirements, retaining years of old, inactive data adds unnecessary risk and cost.

There was also a process failure. Newsgator takes active steps to isolate problems and prevent this type of damage. In addition to sandbox testing, which is computer science 101, we require two-key authorization: the sys admin can only run these types of scripts after a second person has given approval. A well-defined system of checks and balances prevents problems.

While this case is an interesting footnote in the history of IT failures, the larger implications relate to cloud computing. On this subject, Larry Dignan says:

[The cloud’s] growing pains, which are more evident each day that we rely more on service-based software efforts, indicate that you can’t really trust the cloud at this juncture. It’s too early and providers are learning as they go.

Despite being a cloud-based failure, the underlying problem is human error and poor judgment. This cloud failure is no different from any other IT problem, where immature process coupled with lax management oversight resulted in catastrophic meltdown.

[Via George Ou. Image from iStockphoto.]

August 19th, 2008

The triple sins that cause IT failure

Posted by Michael Krigsman @ 3:18 pm

Categories: IT issues, CIO issues

Tags: Information Technology, Failure, Bruce F. Webster, Bruce, Team Management, Strategy, Management, Michael Krigsman

Why don’t more organizations recognize potential IT project problems before they escalate into full-blown failures? Bruce F. Webster believes many companies reject good solutions to fix bad projects for three reasons: internal politics, budget, and fear/pride.

Bruce’s column in Baseline describes three sins that make failure almost inevitable in many organizations:

Internal politics. Large internal IT systems…usually involve several different groups, each of which may or may not be all that happy about having to work with some of the others, but are forced to do so for various budgetary, departmental, or business alignment reasons.

Budget. This may seem counter-intuitive, but management often finds it easier and safer to have a project drag on year after year, ultimately costing large sums of money, than to spend a relatively small (but still painful) portion of that amount up front and fix the problems now.

Fear/pride. Fear and pride can be closely related, particularly when the issue is admitting you made a mistake. This is particularly true if a key manager, architect, team leader, or developer has championed or defended a given approach that turns out not to have worked.

Organizational inertia, the decision-making gridlock that arises when conflicting personal agendas and viewpoints prevent team consensus, lies at the heart of many failures.

While experienced CIOs may recognize that politics and fear cause failure, simply wishing the problem away accomplishes nothing. Instead, wise leaders must take active steps to change organizational attitudes toward failure itself. In fact, it can be healthy for companies to prune back their project portfolio periodically, encouraging natural selection to leave only strong projects untouched.

Facing the inevitability of failure, what’s a responsible CIO to do? Aside from seeking new employment, transparency is the best weapon in the fight against corporate inertia. Exposing self-interested agendas to the harsh glare of daylight is the surest way to keep the system honest.

And that, my friends, is precisely what’s needed to improve IT project success.

[Image via http://home.att.net/~s.l.keim/Sermon.htm.]

August 8th, 2008

IT failures roundup: Emergency services around the world

Posted by Michael Krigsman @ 5:42 am

Categories: Project failures, Government projects, Availability and reliability, End-user impact, News roundup

Tags: emergency service, information technology, computer, productivity, michael krigsman

Today’s roundup focuses on public emergency services. Although these systems should never fail, here are several that did.

The impact of most IT failures is limited to inconvenience, delays, and higher costs. However, computer problems affecting police, fire, and medical response can directly cause loss of life and property.

Maryland 911. A boy almost drowned when the fire department didn’t respond following a faulty Verizon system upgrade. From the Washington Post:

Peter Lucht, a Verizon spokesman, said the disturbances in Prince William “stemmed from an unusual combination of factors” and were “not something that we usually see.” The company has worked closely with the county “to stabilize the system and ensure that [problems] won’t be repeated,” he said.

The county purchased its 911 system from Verizon in 2002, and it was installed in 2003, and fire officials said there were no problems until the May 28 upgrade. There have been no problems since July 12, they said.

Maine 911. Seven “system shutdowns” prevented callers from reaching emergency services. From the Kennebec Journal:

Between April and June, emergency communications systems at the Cumberland County regional communications center, the state dispatch center in Gray and the Penobscot County regional center had problems receiving 911 calls. The most severe problems were at Cumberland County’s system in Windham where a series of seven system shutdowns in April and May left callers getting no answer, in one case for as much as an hour.

In response, FairPoint installed a switch allowing dispatch centers to immediately transfer calls to a backup center when there is a problem, and a designated a separate telephone line to alert staff when there is a problem. The equipment was installed in the six dispatching centers in the state that had the type of equipment affected by the malfunctions.

London ambulance services. Failed computers caused London ambulance workers to fall back on pencil and paper call tracking. From the Evening Standard:

Ambulance staff were forced to record emergency calls with pen and paper and find addresses using A to Z after their computerised system crashed early yesterday morning and a computer failure at the Royal Free Hampstead NHS Trust means patients face delays and records have been lost.

The Royal Free introduced the system in June to reduce paperwork but since then it has crashed, leaving those waiting for operations and blood tests badly affected. When the same system was introduced at Bart’s and The London NHS Trust, cancer patients missed critical appointments because their records were lost.

A spokesman for the Royal Free said: “With change on this scale, it is inevitable that it will take time for staff to familiarise themselves fully with all the functions that they need to use but overall we are pleased with the progress being made.

“It is recognised that while staff are getting used to the system a small number of our patients may have to wait longer than expected in clinic.”

Australian ambulance services. System failure forced Queensland, Australia emergency services to record and track calls using whiteboards. From IT Failures blog:

Thorough investigations have been conducted as to the cause of each, with the following outcomes:

• Two of the outages have been attributed directly to human error. The first was an SQL update which had unexpected results, the second was a server which was inadvertently shut down.
• The third outage has been traced back to a fault with the replication software ‘Replistor’, which resulted in the primary database server rebooting.

[No humorous picture today, because there’s absolutely nothing funny about these failures.]

July 22nd, 2008

Project portfolio mgt: “Draconian enforcement doesn’t work” [podcast]

Posted by Michael Krigsman @ 6:32 am

Categories: Project management, Tools, IT issues, CIO issues, Project portfolio management

Tags: Podcast, Information Technology, Project Portfolio Management, Project Management, Tools & Techniques, Strategy, It Operations, It service Management, Management, Michael Krigsman

Achieving successful technology implementations requires IT and business to establish shared expectations around project goals, objectives, and process. Too often, the business doesn’t understand real technology constraints while IT remains blissfully unaware of why some projects were even started.

Project portfolio management (PPM) software attempts to bridge these gaps by explicitly linking IT activities to business goals.

To learn more, I spoke with Ken Cheney, HP’s director of product marketing for project and portfolio management, asset management, and IT financial management products. Ken was in Boston to discuss the latest release of HP’s Project Portfolio Management Center.

Listen to the podcast to hear an expert discuss project portfolio management. I’ve combined Ken’s comments from the podcast with points he made during a separate, off-mic interview:

On IT challenges:

Despite flat budgets and increasing costs, organizations ask IT to drive innovation or return money back to the business. Everyone in IT must understand what the business trying to accomplish; IT plays a critical role helping an organization achieve those business goals.

Infrastructure costs have been stable, but resource costs have skyrocketed; addressing this means improving automation and connecting systems.

IT must coordinate across a number of moving parts, including strategy, applications, and operations. Each of these groups has become its own disconnected island, throwing work over the fence from one team to another.

The challenge of coordinating across these disparate teams lies at the heart of high IT failure rates.

I’ve illustrated this point in the following diagram (click the picture to see a larger version):

On PPM software:

Project portfolio management software shows where IT resources are being used, the risks associated with projects in the queue, and whether projects have been prioritized correctly. Increasing the visibility of activities performed by stakeholders based on their role helps everyone get the information they need to make better decisions.

On information hiding and other organizational issues:

That’s definitely been an issue with IT.

Most successful organizations have top-down, executive level support. Although Draconian enforcement doesn’t work, everyone must understand issues the organization is trying to address.

On implementing PPM software:

Organizations can possess different levels of project management maturity, ranging from PMP-certified folks to people that know absolutely nothing about project management. Getting baselines in place is very critical, so everyone in the organization works from the same playbook. Projects succeed when organizations adopt standard baselines and methodologies for getting work done.

[Thanks to Erin Muhlhan from Burson-Marsteller for arranging this interview. For more posts related to project portfolio management, click here.]

June 17th, 2008

Graphing the triple constraints of IT failure

Posted by Michael Krigsman @ 2:50 pm

Categories: Project management, Project portfolio management

Tags: Information Technology, Project Management, Tools & Techniques, Strategy, It Operations, It service Management, Management, Michael Krigsman

Every project management trainee is (or should be) taught the triple constraints of time, scope, and cost. These constraints represent trade-offs; you can’t change one without affecting the other two. For example, increasing scope has a corresponding effect on both project time and cost.

In this classic diagram, changing one leg of the triangle affects the other two:

The mysticMundane blog offers an unusual presentation of this fundamental set of project management relationships:

Looking at this graph, I mentioned to project failures guru, Ed Yourdon, that it all seems so obvious. His comment:

It may be obvious, but as Voltaire (and Will Rogers) have said, “Common sense isn’t common”

If you’re managing a project, it’s imperative to understand the relationship between all sides of the triple constraint triangle. The seemingly obvious triple constraint laws are ignored every day, much to the chagrin of those responsible for the failed projects that result.

May 27th, 2008

Moody’s software bug screws investors

Posted by Michael Krigsman @ 6:28 am

Categories: Project failures, IT issues, End-user impact

Tags: Software, Moody's Corp., Investor, Software Bug, Financial Accounting, Finance, Michael Krigsman

Moody’s, a major credit rating agency, incorrectly awarded its top rating to $4 billion of risky debt due to a software bug in the company’s mathematical models. The incorrect ratings misled investors into believing they could earn “very high returns with little risk,” according to the Financial Times. Some investors lost 60% of their investment as a result.

The Financial Times elaborated:

In a time of ever shrinking returns from investments in credit at the height of a raging bull market, early versions of these highly structured and complex deals promised to pay 200 basis points – that is 2 percentage points – over Libor, or the “risk-free” rate at which banks lend to each other. And that spread came with the top-notch triple A ratings that indicate an incredibly low probability that investors could lose their money.

Charles Shumer, Senator from New York, has called for an SEC investigation into Moody’s handling of the situation:

“The ratings inaccuracies that were disclosed are deeply troubling,” Mr Schumer wrote in a letter sent Wednesday. “However, the fact that Moody’s only downgraded these incorrectly rated products in January of 2008, nearly a full year after they became aware of the problem, is much worse, and is indicative of a culture of shirking responsibility that must end.”

Computerworld UK quotes Ralph Silva, senior analyst at financial services advisory firm Tower Group, regarding rating agencies’ lackadaisical attitude toward technology management:

Ratings agencies never put sufficient emphasis on their technology resources,” he said. In spite of technology playing a key part in ratings decisions, “they simply haven’t felt getting technology right was important enough to business processes, unlike banks”.

The end-user impact of computer failures in cases such as this cannot be overstated. Anyone who thinks otherwise should ask the affected investors. Computerworld UK blogger, Mike Simons, summarized the situation well:

It shows both the power of the systems IT professionals build and manage and the stupidity of anyone in suspending disbelief because a computer system tells them black is white. It also highlights the governance issues involved in rectifying mistakes.

Technology problems, whether in the form of unanticipated hardware failures or software bugs, can be notoriously hard to predict and prevent. Since preventing technology failure isn’t always achievable, responsible management policies and governance are critical tools to minimize failure’s impact on users.

Moody’s mathematical bug, exacerbated by management’s delay acknowledging the problem, caused the company’s stock price to drop substantially, may spark a federal investigation, and hurt innocent investors. This situation reminds us that even obscure software bugs can have enormous impact in the real world.

May 26th, 2008

Twitter: An IT failures management perspective

Posted by Michael Krigsman @ 2:44 pm

Categories: Project failures, Project management, IT issues, SaaS, PaaS, and SOA, Enterprise 2.0, Failure 2.0, End-user impact

Tags: Information Technology, Twitter, Groupware, Enterprise Software, Software, Michael Krigsman

Twitter, the well-known social messaging service, has finally acknowledged the depth and severity of technical problems causing downtime and disruption to users. While such candor is refreshing, it also offers a glimpse into the kinds of management issues that underlie virtually all IT failures.

The end-user problem. Twitter has raised about $20 million of funding and garnered tremendous publicity, giving users the expectation this high profile company should offer consistent and reliable service. Despite Twitter’s substantial resources, it has acknowledged the service is not sufficiently reliable. From the Twitter blog:

[This graph] should be flat.

We’ve gone through our various databases, caches, web servers, daemons, and despite some increased traffic activity across the board, all systems are running nominally. The truth is we’re not sure what’s happening. It seems to be occurring in-between these parts.

The technical problem. As typical of many Web 2.0 companies, Twitter built its service to meet short-term objectives; in this case, that meant choosing an unsuitable technical architecture, as another post on the Twitter blog describes:

Twitter is, fundamentally, a messaging system. Twitter was not architected as a messaging system, however. For expediency’s sake, Twitter was built with technologies and practices that are more appropriate to a content management system. [This has] introduced a great deal of complexity and unpredictability….This is, clearly, not optimal.

THE PROJECT FAILURES ANALYSIS

When technology fails, management oversight and skill usually determine the impact on a business and its users. Twitter acknowledged they “aren’t sure” where the technical problems lie and admitted they built the basic architecture for “expediency” rather than suitability to task.

Lack of sufficient management experience and judgment ultimately created the difficult situation where Twitter must rip-and-replace foundation technical components to resolve severe performance and reliability issues. These issues are substantial enough to threaten users’ confidence in both the company and the Twitter service.

Management experience and judgment. I asked Steve Mann, social media strategist at SAP, for comment on the experience issue:

In customer-centric organizations one of the most critical factors which these enterprises focus on is the customer expectation of high availability. Now I don’t know the Twitter management or technology teams so I won’t presume to armchair Quarterback on their behalf but as both an outsider looking in and an avid Twitter user, the recent outages suggest a degree of inexperience. Now maybe they’ve done this but I would have expected the Twitter team to project out usage patterns at least six months ago and based on those projections, would have begun re-architecting their infrastructure in order to scale to meet anticipated volumes and the availability expectations we are seeing today.

Steve also blogged about Twitter’s credibility in the face of poor reliability:

Once trust is blown it doesn’t matter if Twitter fixes its availability issues in a week or a year. Its already lost the opportunity it currently has. One might say, its already lost that opportunity for good.

Zoliblog shares similar concerns about Twitter’s management judgment:

On second thought, I am less forgiving. Twitter already raised $5M before this round, that should have allowed them to bring in expertise they clearly lack. If only their priorities were on fixing the service instead of chasing more money.

Legitimate technology challenges. In fairness, the challenges facing Twitter are substantial and push the limits of current technologies. Hueniverse put those issues in context:

The idea that building a large scale web application is trivial or a solved problem is simply ridiculous….The social web is creating demand for new scaling tools and technologies. Current databases and caching solutions are simply unable to handle a complex network of multiple relationship between objects.

Nonetheless, Twitter’s rollout planing process is clearly flawed. In contrast, Facebook has demonstrated professionalism in large-scale rollout planning:

The secret for going from zero to seventy million users overnight is to avoid doing it all in one fell swoop. We chose to simulate the impact of many real users hitting many machines by means of a “dark launch” period in which Facebook pages would make connections to the chat servers, query for presence information and simulate message sends without a single UI element drawn on the page. With the “dark launch” bugs fixed, we hope that you enjoy Facebook Chat now that the UI lights have been turned on.

The same post says: “scalability has to be baked in from the start,” a lesson that Twitter is learning slowly, painfully, and very much publicly. Facebook is a far larger and more mature organization than Twitter and you can see the difference in their respective approaches toward managing technology.

My take. Twitter is a great service and I love it when it works. In addition, the Twitter folks are friendly and accessible, so it feels somewhat mean-spirited to apply the usual IT failures expectations to them.

Twitter co-founder and Creative Director, Biz Stone, offered these comments by email:

I continue to be inspired by both Jack [Dorsey, CEO] and Ev [Williams, Chief Product Officer] and I’d argue that their talent and judgment is precisely what will navigate us through these growing pains and help us reach the vision of Twitter’s future we all share. You’ll be interested in knowing that we are actively seeking talented managers and we are spending significant resources on recruiting.

Despite the obvious good will, Twitter now has an $80 million valuation and provides a communications infrastructure upon which many people depend. From that perspective, users are completely justified in expecting a robust, reliable service with no explanations and no excuses.

March 20th, 2008

Billion-dollar IT failure at Census Bureau

Posted by Michael Krigsman @ 7:51 pm

Categories: Vendor relationships, Project failures, Government projects, Project management, Implementation, IT issues, CIO issues, Financial impact, Politics

Tags: Harris Corp., Government Accountability Office, Census Bureau, Bureau, Handhelds, Strategy, Hardware, Management, Michael Krigsman

The US Census Bureau faces cost overruns up to $2 billion on an IT initiative replacing paper-based data collection methods with specialized handheld devices for the upcoming 2010 census. The Bureau has not implemented longstanding Government Accountability Office (GAO) recommendations and may therefore be forced to scrap the program. Harris Corp., the contractor associated with this incompetently managed initiative, was awarded a $600 million contract to develop the handhelds and related software.

In March 5, 2008 testimony before the Senate, Commerce Secretary Carlos M. Gutierrez said: “There is no question that both the Census Bureau and Harris could have done things differently and better over the past couple of years.”

On the same date, Census Bureau Director, Steve H. Murdock, added:

I cannot over-emphasize the seriousness of this problem. My colleagues and I recognize that we must move quickly to address this problem, and implement solutions. While we still have an enormous challenge in front of us, I am confident that we are close to defining and implementing a strategy that will ensure a successful 2010 Census.

The GAO characterized the handheld initiative, known as the Field Data Collection Automation (FDCA) program, as follows:

Of the $11 billion total estimated cost of the 2010 Census, the Census Bureau planned (as of 2007) to spend about $3 billion on automation and information technology in order to improve census coverage, accuracy, and efficiency. Among other things, the Bureau is planning to automate many of its planned field data collection activities as a way to reduce costs and improve data quality and operational efficiency.

The GAO report, dated March 8, 2008, added:

In October 2007, GAO concluded that without effective management of key risks, the Field Data Collection Automation (FDCA) program responsible for the devices faced an increased probability that the system would not be delivered on schedule and within budget or perform as expected. The magnitude of these problems is not clear…. [T]he Bureau has not performed recommended analysis or provided sufficient information to provide a level of confidence in its $11.5 billion life-cycle cost estimate of the decennial census. The Bureau has not itemized the estimated costs of each component operation, conducted sensitivity analysis on cost drivers, or provided an explanation of significant changes in the assumptions on which these costs are based. Together, these weaknesses and actions raise serious questions about the Bureau’s preparations for conducting the 2010 Census.

Computer World blogger, Frank Hayes, summarized the situation succinctly, “The fancy custom handhelds might work. But if they don’t, the Census Bureau will use paper instead.”

THE IT PROJECT FAILURES ANALYSIS

Managing an $11 billion initiative is a daunting task and unforeseen problems are inevitable. Nonetheless, the GAO, going back to January, 2005, repeatedly identified significant procurement, management, and operational risks associated with this project. For reasons unknown, the Census Bureau chose not to follow these recommendations.

The following table summarizes significant project issues identified by the GAO:

How does a failure of this magnitude arise? Clearly, Census Bureau management is ineffective at properly and efficiently executing the organization’s basic mandate. A detailed analysis would probably reveal hidden agendas; conflicts of interest; good intentions gone bad; inexperienced, lazy, and incompetent management; lack of controls; and plain old poor judgment. I believe these deeply ingrained issues are symptomatic of fundamental problems shared by both Bureau leadership and line management.

My recommendation: The GAO must conduct a formal inquiry into two specific areas:

1. It should investigate and analyze the management policies and procedures that allowed this situation to develop and persist over the course of several years. We must understand why program controls didn’t prevent this huge waste of dollars.
2. It should perform a detailed (and I mean exhaustive) investigation of Harris Corp.’s role. Let an unbiased panel determine what percentage of the billion-dollar waste Harris caused and force the company to pay direct restitution for that amount.

Until the government holds contractors and their agency sponsors accountable, massive failures will continue and more money will be flushed down the drain.

March 1st, 2008

Was Heathrow 777 crash caused by poltergeists?

Posted by Michael Krigsman @ 8:24 am

Categories: Project failures, IT issues

Tags: Software, Aircraft, Accident, Analysis, Policies And Procedures, Aerospace & Defense, Tools & Techniques, Human Resources, Manufacturing, Management

Despite speculation (also here and here) that the January 17 crash of a BA airplane at Heathrow Airport was caused by defective software, the most recent Air Accidents Investigation Branch (AAIB) report does not draw this conclusion. Although the investigation has not yet unconvered the cause of the crash, preliminary analysis points toward the fuel system:

Investigations are now underway in an attempt to replicate the damage seen to the engine high pressure fuel pumps, and to match this to the data recorded on the accident flight. In addition, comprehensive examination and analysis is to be conducted on the entire aircraft and engine fuel system; including the modelling of fuel flows taking account of the environmental and aerodynamic effects.

Speculation that faulty software caused the accident arose because AAIB preliminary statements did not offer concrete findings:

The investigation is now focused on more detailed analysis of the Flight Recorder information, collecting further recorded information from various system modules and examining the range of aircraft systems that could influence engine operation.

THE PROJECT FAILURES ANALYSIS

Anyone involved with IT systems is keenly aware of the frequency with which software failures occur. In addition, many observers feel almost viscerally uncomfortable when a technical problem cannot be properly explained after some research.

This drive toward closure pushes some writers to accept a kind of “ghosts in the machine” logic, where they place unexplained technical phenomenon into a black box called “software failure.” Such logic represents little more than poor thinking and, even worse, failure-mongering.

Instead of jumping to conclusions, I suggest we wait for the AAIB to complete it’s investigation.