Tip:
Highlight text to annotate it
X
I was involved in a consulting engagement
from November 2009 until March 2010
to help identify ways to improve testing
at an organization that felt that testing
could be improved dramatically.
This is an organization that you probably all know
so I won’t name names, I would not going to hint you any hints about it
or even crazy puns that may lead to it... or hope I don't anyway.
But when I do the analysis, I like to look at three projects
I want to take one project that really went well, an example of excellence
I want to take one project that really screwed up, an example of something that was a dismal failure
And another project that people consider a typical project
We do task analysis of this, we interview many people,
it takes months and months,
I do this with assistance, research assistance,
we audio tape the interviews, we look very deeply and attentively at this whole thing
In the end I came up with certain recommendations
on some practices that they're doing
and because they had so many projects that are going on
they had practices that were working great and failing in another
there are a lot of things, because the company was built up by amalgamation
the story really is of the worst project
the one that everybody said "Rob, if you want to see
an example of failed project, pick this one"
So, let's say, it is project GS
I looked at it
And this is all implemented with their flavor of scrum
(a very anti-agile scrum but scrum nonetheless)
and I went from the beginning to the end through the turmoil
they had a metrics system
because they were so many different projects from different teams
they had a metrics committee
the metrics committee had decided on a series of metrics that
they would use to measure these projects
and the metrics committee would follow these on a weekly basis.
This project was red every single week
except when, occasionally, it was yellow.
Now, they had a fixed delivery day, end of year,
and they had beta testing, but basically this project was failed.
They actually had, during this project, fired people,
reorganized departments, dramatically had to change the way
things were done to try to bring this project out of the red.
It was truly a failed project by every measure that they had
and this project, basically, was something that I studied very carefully
and I found some practices there that I found a bit absurd,
and I suggested changes to all this sort of stuff
I made my recommendations and made my reports
And the job was done, from my perspective, except some follow up that I promised to do
A subsequent date, couple of months after my mandate, I'm called in
they're going to present my stuff to the exec
but they're presenting it, not me, so I have to coach the people presenting it
When in there, a mysterious person came in running into the meeting room
and he's excited and excited and excited
and I did not even know "why"...
and he said: "Project GS just won the industry award for the product of the year"
Product of the Year in their industry!
The customers loved it; they basically broke grounds in every technological field possible with this thing
and the product that was the product in red, this humpty-dumptying product
the one that was a failure by all their values
the one that people lost their livings because the were fired
was the product of the year in their industry sector.
Written about. Incredible accolades. By the user community, by the industry community.
What is this? What is this story about? I'll tell you what is this.
This is the story of ABSURD and FAILED metrics.
It was absurd. Not one of their metrics paid any consideration to what actual users cared about
what people valued in their industry was innovation, not nuance
they wanted breakthroughs, they wanted the next generation of the universe
and that's what the developers where trying to do!
and because what they were doing was so foreign from the paradigm
of these obtuse metrics program
they basically were like, dare I say it, a "Rebel Alliance"
a group of people fighting against the establishment and trying to survive.
Now, this story is not over.
I'm trying to figure out ways. I have the metrics information.
I have the data, I know the people.
There are lessons to be learned
I haven't finish learning them
because when I did my analysis, I treated this as the failed project
and quite frankly, I think I should've treated this as the excellent project
End of story