Monday, October 3, 2011

Emergency = Value

One of the biggest challenges faced by organizations is trying to separate the Value from the Waste.  Even in my own firm we are so focused on "what we've always done" that it takes tremendous discipline to really listen to the voice of the customer.  Part of this I believe is a basic, protective instinct that keeps us from facing the cold, hard reality that most of what we do is in fact Waste.

But I recently rediscovered a technique for distinguishing Value from Waste that I have stumbled across from time to time throughout my career; The Emergency.  The premis is simple; people in a burning building very quickly figure out what matters and what doesn't.  All of a sudden when your house is on fire the dirty dishes in the sink, the dust on the mantle and the clothes in the dryer become much less important than getting out alive.  And the same is true in any organization.  When nothing is metaphorically burning there is all the time in the world for things like proper documentation, documentation reviews, documentation modifications, more documentation reviews, more documentation modifications, final documentation reviews, documentation approvals and thorough documentation archiving.  You get the picture.

But when there is a crisis equivalent to the building being on fire all that goes out the window and people tend to focus on what really matters (Value).  A crisis can take many forms, but the ones that are most useful are the ones caused by customers.  These can include a sudden rush order, an unexpected change in delivery date or order specification, or even a critical quality issue.  When this happens it's all hands on deck, the documentation reviews are forgotten and everyone focuses on resolving the emergency.

As a management consultant working in a new organization such emergencies make great sources for improvement ideas.  Once the dust settles and the crisis has been averted it is very useful to start from the critical tasks performed during the emergency and work outward.  That means asking the question "why do we do this?" for anything that was not done during the crisis.  There will always be certain "risks" that were taken that probably should not have been.  But if you really challenge much of the non-crisis tasks you will find that most of these fall into one of the following categories: "Weve always done it that way" or my favourite "One time five years ago we had a problem so we put this fix in place."

Over my career I have had the opportunity to work in a number of software development environments.  Any of you familiar with this industry in general and ITIL in particular will appreciate the incredible number of checks and balances that go into developing, testing and deploying software.  And there are good reasons for this, especially when you are dealing with critical applications that can not go down.  But one of my pet peves with ITIL has always been the seeming lack of customer awareness from the Change Management group.  There is not a project manager anywhere in IT that does not cringe at the thought of getting a Request For Change (RFC) approved by Change Management.

Now I am not saying that all Change Managers are sadistic egomaniacs who's sole purpose in life is to compensate for their shattered dreams of becoming customs agents.  Because I would never say that.  But like any good auditor they all believe if they do not find at least one reason to reject an RFC the first and second time it is submitted then they have not done their job properly.  Just sayin'

To make matters worse most 'effective' Change Management departments take great pride in their rigorous RFC approval cadence.  "Cadence" by the way is a fancy way of saying "it's my way or the highway."  The idea of a "cadence" is that in order to get a Change approved one needs to comply with a rigid series of deadlines for approval.  And if at any point your Change gets out of cadence, say because you forgot to dot an "i" or cross a "t", then you have to go back to the beginning and start over, usually the following week.

All of this is put in place under the banner of protecting the client or User from reckless software developers who unfortunately do enjoy finding and fixing bugs far more than they do writing perfect code in the first place.

So how do we separate the Value from the Waste in such a critical process?  Enter the eRFC.  The "e" in this case stands for "emergency."  And as the name implies this is a change that is needed immediately if not before.  Usually either something has gone wrong with a piece of hardware and the Users are being impacted.  Or a defect with the software has been discovered and it has to be fixed right away.

In this case a few amazing things happen.  The first is the usual testing procedures are usually tossed out the window and the modified software is tested in just one environment called "EBF" which stands for Emergency Break Fix.  This simplified environment is kept just for emergencies and is supposed to match the Production environment to allow changes to be fully tested before exposing them to any Users.  This is in contrast to the normal testing process that can involve as many as eight lower environments in which the code must be tested (don't ask - the explanation is very complicated and highly technical).

But by far the most amazing thing that happens is that Change Management's precious cadence gets thrown out the window.  The owner of the Change is still required to provide all the necessary proof that the Change has been properly documented and tested.  But the approvals are done on an 'as needed' basis.  That means all the approval meetings normally scheduled on a weekly basis now get an "e" put in front of them and they are scheduled as soon as everything is ready for review and approval.  Yes this would be what Lean practicioners call "Pull Scheduling."

And for those of you worried that the level of scrutiny is lower for eRFCs I can assure you just the opposite is true.  Following the usual Change Management cadence there can be as many as 20, 30, 40 or more highly complex RFCs batched for approval.  Even with two hours scheduled these meetings are a free-for-all at best.  In fact I once had two RFCs approved at the same meeting where the titles of the Changes and the descriptions had been accidentally interchanged by Change Management.  But because everything else seemed to be in order no one even noticed.  shhhhh

On the other hand for an eRFC when the meeting is called there is only one Change to review.  And in a relatively short period of time everyone at the meeting can focus on what is involved, ask meaningful question, and ensure with confidence that everything is in order.  In this way I have seen eRFCs go from  initial submission to executive approval in a matter of hours.

So I ask you, why not make all Changes go through the eRFC process?  With today's mobile technology it is possible to convene meetings and get approval almost instantly.  The level of scrutiny is much more focused and accurate.  And the customers end up getting what they want when they want it.

Think about the last customer crisis you had in your organization.  What could you learn from that?