Lean Integration and ICC

Integration Competency Center is a shared service governing all the integration work, platform, tools, processes, design patterns and best practices. In this post I’ll concentrate on best practices and exactly how ICC may realize them. As we know, describing how something should be done is lot easier than making it really happen.

In my idea of Pragmatic ICC (PICC), we are not just group of architects locked up into ivory tower musing about best practices together without anyone really listening and without a trace of realization. In Pragmatic ICC, first step is to assign competent people to predefined roles. Architect is a role among other roles.

You can read more about ICC roles and tasks assigned to them from my ICC Handbook. Note, that Pragmatic ICC is not usually the party that implements of integration work – PICC is the on that transforms business user stories to stories that developers understand. PICC is the party that enriches user stories with artifacts like mapping tables and rules, non-functional technical requirements and so on. PICC is the organization that either implements or orders implementation from several integration suppliers.

I think John Schmidt’s Lean Integration¹ is a great set of best practices to adopt in processes that ICC first design in establishment phase and then manage in ICC execution phase.

Focus on the customer and eliminate waste

Waste elimination from ICC point of view means that scrap everything that do not add value to internal customer like project, business unit or source of sponsorship for the ICC. ICC’s mission is to lean processes of ICC itself, integration suppliers as well as production management team. For example, if there is a step in development process that a) frustrates developers and b) does not really add value to solution being developed – kill it.

In addition to processes, business user stories should be cleansed from features that do not really add value. PICC should contain enough understanding of the business processes and needs to at least identify possible features that are not mandatory but would need an extensive amount of development work.

As you read this chapter, you will see that there are lots of tempting changes to overdo e.g. automation or quality measurement.

Continuously improve

PICC roles include architects of different specialization areas and ICC managers, who are more like project managers or scrum masters with experience on multi-vendor integration projects and continuous integration work. As ICC may not necessarily be the party that is implementing integration solutions, it sure should take care about how integration is developed and maintained. Architecture, best practices and processes, that ICC has dismounted, should have an organized way of being analyzed and improved. Again, it is easier to find what should be improved than make it really happen. This is the part where ICC needs managers more than anything. Whatever project or development model is, a good retrospective with people capable of taking lessons learned to field is crucial.



Picture above is anything but complete list of work where ICC’s Architects and ICC Manager should influence to the work of Development Team (own or supplier) and Production Team. If a person from Development Team and Production Team is present at PICC weekly meetings, it is easy to cut down the boundaries between separate Teams.

Empower the team

Creating cross-functional teams and sharing commitments across individuals empower the teams and individuals who have a clear understanding of their roles and the needs of their customers.“²

This is actually what PICC is all about: roles and clear tasks assigned to people on those roles. ICC Handbook has a sample set of roles and their assignments. The crucial role is ICC Manager – the one who orchestrates the PICC cross-functional team. He or she should – could even say must – have enough empowerment to make decisions concerning supplier management, backlog prioritization, schedules and other managerial stuff related to development and maintenance or integration.

Optimize the whole

PICC must adopt a big-picture perspective of the end-to-end process, which may run on several enterprise system and have stakeholders in separate business units, customers and business partners. One project, managed by PICC, may not see the added value of steps like taking care of the re-usability, ease of maintenance and change management. Instead, the whole organization around PICC will have benefits of these as faster development, change management and lower costs.

Plan for change

Common pitfall of EAI is pace and quantum of changes. PICC must ensure the re-usability of delivered integration solutions as well as PICC development processes must adapt to changes during implementation. This requires ability of seeing the needs of business units wider than the first project that needs the integration. For an example, if a process need a customer data from CRM, don’t just create web service that returns exact data fields that process at hand needs, create a service that returns customer information as a whole concept that could be used without a change by next process and project that needs it. Use composite and component architecture if required. This way you can plan for change by eliminating the need for change the service.

Other typical challenge is changes during development phase of integration. Planning for this challenge starts already at contract phase. Negotiate agile contract where customer or business unit understands what is a change in integration project or work and what kind of impact changes have in development process offered to customer. Read more about Agile Integration Contracts in my earlier post on that topic. Amount of changes during development phase are decreased with good user story design and artifact management.

Automate processes

Automating of tasks is one responsibility area of integration. Good goal for PICC would be to go for Continuous integration (CI) practice with integration development. Good way of starting this is requesting Test-driven development (TDD) from party developing integration user stories PICC manage. With automated testing and deployment processes, PICC add quality to development processes as well as cut costs of small changes that would cause a lot of manual testing and lots of deployment work compared to work amount required for that small change.

Build quality in

As advised in previous Automate processes chapter, implement aspect. TDD and maybe even Acceptance test-driven development (ATDD) is great way of increasing quality aspect to development process itself. In ATDD acceptance test cases are agreed and automated beforehand. Using ATDD requires PICC to define acceptance test cases with business people in a manner that test case can be easily automated by developers running ATDD.

Agreeing acceptance test cases before hand ensures that user story is really understood right. It is an opportunity to PICC’s Functional designer (a role defined in ICC Handbook) to get answers to common questions like what to do when process does not go forward as excepted in a user story (exceptions). One could think ATDD is a way of communication between Pragmatic ICC, business people and developers.

Be aware that running TDD in integration is a little trickier than in basic software development. Gregor Hohpe and Wendy Istvanick have released great article on Test-driven EAI. It is released back in 2002 and tools presented in it might be old or obsoleted, but the challenges and aspects to take care in TDD and integration are described extremely well and concepts are valid as they we’re written yesterday.

There are also other as crucial processes as development to add quality aspects to. One of these are deployment process of integration. What kind of quality gates there should be when integration solutions flows from development environment to production environment via one or more test platforms.

As we build quality in processes, we must as well set up (automated) measurements for quality. A key metrics could be:

  • Changes before accepted number, which is a number of times an integration case (independently implemented part of the user story) must be changed or modified in separate sprints unintentionally before it is accepted for production use.
  • First time through percentage, which is a number of times and end-to-end process is executed without having to do changes or manual actions like repeat some steps.

[¹] John Schmidt, http://blogs.informatica.com/perspectives/2009/01/14/10-weeks-to-lean-integration/

[²] Wikipedia, http://en.wikipedia.org/wiki/Lean_integration

One thought on “Lean Integration and ICC

  1. Pingback: Establishing Integration Competency Center, part 4 | Integration War Stories

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s