Tuesday, October 29, 2013

Why Do Integration Projects Fail?

Over the years I have been involved in hundreds, if not thousands, of enterprise application integration projects and it is no secret that some fail. They fail for a very simple reason: once the best tools and methodologies have been selected, the really difficult challenge begins, and that is to effect major change without upsetting anyone. Here are the key reasons integration projects fail and six tips to help them succeed.

It looks so easy

On paper, it seems incredible that so many integration projects would fail. After all, the project is only begun after a diligent and often lengthy search for the best tools and expertise. In most cases, the task is clear: make data or functionality from System X available in Application Y, or link them with a workflow, and there is usually a clear project plan and methodology. Everything would seem to be in place for a smooth, rapid transition, so why do so many projects fail?

The importance of management sponsorship

Top management holds the key to making any project succeed, by taking a strategic view and ensuring the right resources are dedicated to it. Enterprise integration projects must be understood to be a management task rather than a business unit or IT responsibility, because only with clear management guidance can these projects succeed. This may seem strange, after all many integration projects are tactical in order to make workflows link better into various systems; but these projects affect so many areas, and can have such a significant impact on the performance of the business, that change should be derived from overarching strategy.

The problems emerge

The real problem emerges when you look at who is affected first by any integration project. Integration affects systems, which requires changes to the support team, and their cheese has been moved (along with that of any subcontractors they may use) long before the integration project really touches the business. “Why”, you may hear in such an organisation, “does it take two days for a server authentication?”

Simple: as the IT department has been affected first, they have the first motivation to change the scope of the work. Because no one else is really concerned at this early stage, they also have plenty of power to derail the project. IT must not only be consulted but be seen as having a pivotal, key role in providing access to systems and ensuring that technologies work. IT must be seen as more than just executors of decisions; management needs their input and they must be brought into the project from the very beginning and given the big picture view of the integration project as a significant partner.

How can we fix this?

Of course, the problem goes a lot deeper than an IT department that is resisting change because their input hasn’t been sought: they are just a canary in the coal mine of the enterprise, the first sign that all is not well. Fortunately, with effective strategic planning by management these problems can be mitigated.

1. Use champions to sell the project

Individuals in enterprise departments don’t become polarised against a project purely because it represents a change; they become polarised because no member of the department understands the benefits of the project well enough to overcome the broad-based and possibly groundless fears of the group as a whole. Therefore, finding influential people within affected departments and empowering them to champion the project to their colleagues can dramatically reduce hostility to change.

This isn’t just about getting a cheery message to their colleagues; these champions should also be used as a way for management to understand the concerns of staff or other stakeholders such as suppliers who will be affected and try to mitigate these concerns. No project plan, however well defined, can really stand up to reality and sometimes the concerns of these staff or stakeholders will provide early warning of a flaw in the project plan that could have cost the enterprise valuable time and money to fix later on. These champions must have a deep understanding of multiple enterprise functions and be seen to be unbiased, and of course they need clear sponsorship and empowerment from top management. This is the only way to make sure that they are trusted and effective.

2. Avoid blame culture

When project schedules slip, it’s very easy to fall into the habit of assigning blame rather than fixing the problem and avoiding a repeat. This may be easy and tempting, but it doesn’t help the project progress and creates a risk-aversion in your key project leadership which can lead to unwillingness to cooperate. Left unchecked, these attitudes will recreate the organisational silos that the integration project is trying to remove in the first place.

There is no easy way to avoid this: the entire management team needs to adopt a strategic view and foster an attitude of cooperation, openness and problem-solving across all departments working on the project. Seek to understand why something has gone wrong rather than simply assigning blame, and also understand why elements proceed quickly and effectively. Encourage different areas of the business to learn from each others’ experiences and to anticipate where problems could potentially occur.

3. Focus on processes, not data

Integration projects don’t only fail because of people, but also because of data: specifically, data locked up in silos that cannot be accessed and which are jealously guarded. This is where a focus on process is vital, because while focusing on data leads to protectionism and small-scale thinking, processes require an end-to-end view that encompasses the entire enterprise.

The key to a process-led view is considering what users of the integrated system are actually trying to achieve when they begin any task, and what information flows are needed to complete these tasks. This helps reticent departments understand that the integration project is not asking for all data to be made accessible to all staff or suppliers, all the time; instead it is seeking to automate access to certain data as and when it is needed by appropriate staff. It is also harder to argue that data must not be accessible when it is clear that this data is key to quickly generating sales, for example.

4. Remove negativity

A problem I touched upon earlier is that in any integration project, staff and other stakeholders can feel that their cheese has been moved; this is the start of a downward spiral because people feel disempowered, helpless and unvalued. While this is a major challenge and a perfectly natural human reaction, it is possible to make these staff feel empowered by the exact same change.

Again, the key is focusing on the processes as an empowering development. No one likes to feel insecure and nothing builds insecurity like seeing the way they carry out their daily routine being changed. Focus on the benefits the new system will bring, in particular how the affected department will actually be able to work far more effectively and drive more value for the business.

5. Continuous change

The result of this is very similar to the Japanese concept of “Kaizen” or continuous change: essentially, do not try to fix your enterprise all in one go. Have a strategy for what you eventually want the enterprise to look like and break this down into components, where each component is an individual project that can be implemented independently of each of the others. This allows you to carry out, develop, test and make live the first component, then wait and see what happens. Once you have identified the impacts this first component has had on the business and analysed how and why this is different to what you expected, redesign your second component with these lessons in mind, develop it, go live, and again wait and see what happens. Chances are there will be very few changes between the first and second components, but because of this piece by piece approach there will probably be much larger changes between the first piece and the tenth.

This system provides you the time to take a big picture view of the impacts wrought by each component without feeling rushed to move on, and also allows you to really analyse why the impacts were different to what you expected and make appropriate changes to the next component. As a result, your integration projects will be more flexible and responsive to the reality of your enterprise.

It is impossible to predict how the changes made in one project will impact other parts of the business, because as your users learn the new system further opportunities will become visible. The biggest cause of failure is failing to anticipate the impact of change on the enterprise, and the bigger the project the greater the chance of creating a white elephant. So have a clear project scope, complete that project and see what happens before embarking on further projects. After all, if you have a clear strategy it is relatively simple to build on a successful project. However, this does mean that you need to choose an integration tool which allows this kind of flexible, process-driven integration where future changes are not just a possibility but the norm. You need a tool which doesn’t require all your connections to be rewritten every time a system is changed.

6. Dealing with suppliers

Large enterprise systems are supported by their vendors, and many of these vendors will have their own preferred implementation partners. It is important that you bring on board not only the vendors but their subcontracted partners, explain your vision and engage with them long before the project starts. This way you can resolve any potential issues before they become roadblocks to your project, and the importance of this further affirms the importance of top management’s involvement.

Ultimately, enterprise application integration projects fail because of people, but effective management can help those same people see the positives and become drivers of change. Successful integration projects require management and leadership, so relying on subcontractors and hidden agendas will create issues. Finally, IT needs to be seen as a driver of change and value, not as a support function, and needs to be brought in with a leadership role.

David Akka is the Managing Director of Magic Software (UK) Ltd and have worked with Magic Software for over 16 years.
Visit his blog at: http://www.davidakka.com/