The Unnecessary Tension between Process and
Programmer
Some of My Best Friends Use an Agile Software Process
Dave Thomas, Bedarra Research Labs
|
 |
ARTICLE
PDF Version |
1 THE PROCESS BORG
Today all organizations are required to achieve process compliance,
be it ISO 9000, Six Sigma, CMM, Sarbanes-Oxley, Balanced Score Cards
etc. Each of these compliance activities seeks to ensure that the
company has the appropriate processes in place to ensure that the
company can serve the needs of their customers and shareholders.
Most mandate some form of continuous improvement measurement, dare
I say metrics, so that the organization can monitor its improvement.
Many of these efforts were originally targeted at improving manufacturing,
operations or finance, but they are having an increasing impact on
the IT and SE. Also, while these efforts were initially applied to
larger companies, they have trickled down to the smaller ones who supply
these larger firms.
Having a documented process has therefore become a mandatory requirement
for software development. It isn’t sufficient to say “we
are doing Waterfall, Iterative or Agile; go away and let us get our
work done”. Like it or not, everyone needs to be able to articulate
the process they use for software development. It is time for developers
to take ownership of the process side of things so that they can stop
fighting it. Process is like democracy: if you don’t participate,
you get the process you deserve. However, there is no need for process
tension to obstruct productive development!
2 SOFTWARE CULTURE YES - SOFTWARE PROCESS NO
Ask any software developer and they will tell you if they feel they
are working in a good software culture, and most who answer no will
say that they would like to be. However, ask most developers how they
feel about their company’s software process and they will complain
bitterly. Yet the best software cultures always have very disciplined
processes which developers follow almost to a fault. Indeed the culture
is defined by a unique combination of people and process. The open
source Apache foundation is an example of a successful software culture
that has very disciplined practices. In fact, all of the successful
software cultures I know of have disciplined and valued rites and rituals.
How can cultures be so respected while the software processes that
help drive them are so repugnant to developers?
Let Them Eat Process
When articulated by the CEO or VP of engineering, the process initiative
brings tears to your eyes: clearly we all want to improve. However,
too often these speeches take the form of political promises sounding
like free universal health care and a good life for all without seriously
looking at how they are funded or implemented in the organization.
My most recent irritation is the attempt by some to move to Agile development
without first making the investment in a continuous integration and
test environment.
Traditionally software processes have been dictated top down by the
Process or Quality department with little input from the analysts or
developers. Searching for the usual quick fix, many companies have
adopted commercial processes in the hopes that they could provide the
solution. Unfortunately this has created a perception gap which says
that process is bad and something which gets in the way. This persistent
adherence to a top-down approach is surprising when history has shown
that quality and change is best introduced bottom up sharing experiences
across teams so that the organization learns from experience. Instead
of evolving with the teams’ learning experience, the process
often remains a static pile of out-of-date manuals or a huge website
which is seldom referenced. Top-down processes emphasize artifacts,
roles and linear production where as bottom-up processes tend to focus
on skills, practices and incremental improvement.
Too often the appearance of process is more important than actually
having a process which works! Middle management people have become
experts at gaming any measurement system, and can often obtain their
quality bonus without actually doing anything to improve quality. The
measurement system is gamed to give the same good news results as before.
Consider for example the hotel room quality control card that says “if
you can’t give us at least 9 or 10 call me, the manager, at my
home number”. Clearly this hotel chain doesn’t really care
about quality! Nor does it care about improvement since reporting real
problems is actively discouraged.
Unfortunately many managers and executives often lack modern real
world development experience. Few, if any, have developed and deployed
a
new major application or product; hence they have maintenance processes
at best, which are designed to reduce risk through incremental fixes
and features. They believe in all best intensions that if there is
a documented process that development will improve. There is little
said about first-time software where expert education and mentoring
are so important to acquiring new practices. The problem is compounded
by process implementations by managers and project leaders that do
not provide for learning, proper time allocations for activities etc.
In many cases the process is simply a set of check boxes and artifacts
to be produced in addition to the software asset (with no additional
time to do so).
Culture – The Invisible Implicit Process
What one really wants is a learning organization with a culture that
is tightly aligned with an almost invisible process. In such a software
culture things just get done according to a seemingly implicit process
because that is the culture. In such organizations people follow the
process because they just do it, not because it is posted on the wall,
distributed in a large manual accompanied by draconian gate reviews,
piles of project charts etc. We need to move beyond large complex process
definitions to implicit practices which are intrinsic to a productive
culture.
Agile Development (www.agilealliance.org) is an increasingly successful
example of a bottom-up people/team center process which can be assimilated
quickly by small teams to improve software predictability and quality.
Agile Development quickly builds a common culture of simple, day-to-day
practices which developers live and breathe. Unfortunately, the lack
of Agile process descriptions and the disdain which Agile developers
have for process creates unjustified concerns about the lack of discipline
of developers. Agilists need to meet the need by providing the straightforward
process documentation and measurement which organizations need.
3 AGILE PROCESS DEVELOPMENT – MAKE IT BETTER EACH SPRINT
Process descriptions don’t need to be lengthy, they just need
to describe the vocabulary essence of what people do, what is produced,
the sequence in which things are done and how the process can be observed
by management or customers who are not actually inside the development
team. Since there are lots of publications available, there is no need
write a massive document; instead, the documentation needs only to
provide sufficient information and appropriate references, ideally
to online documents, to allow a newcomer to understand how it is that
things are done in your software culture.
Process doesn’t have to be built in a single day. It is really
just a case of Agile writing where pairing, incremental releases, and
collaborative reviews to find bugs can easily produce a simple website.
Many Agile teams do this already in one way or another in their team
wikis, blogs or websites. If you are concerned about the quality lingo
of Six Sigma, or the software metric police, take the time to ask them
what information they need and why they need it. Just asking them,
indicating you may even care if they can be successful at their job,
can often lead to a ground swell of cooperation that may have process
experts writing and editing your process for you as you speak.
Leverage Open Space Communities of Practice
Virtually everyone accepts that Agile works inside a small team,
but large organizations need to have common language across the organization
so that people can communicate using the same vocabulary and so that
people can be moved to from team to team and new employees can be brought
up to speed quickly. Building process consensus across teams is best
facilitated by communities of interest who meet informally in open
space to discuss how they can improve their practices, be they in developing
stories, acceptance tests, unit tests, interfaces, acting as scrum
master, measuring progress etc. Grow your process and culture bottom-up,
sharing what works and doesn’t for your team and your organization.
Measurements – Let Code Report Itself
Agile advocates’ providing constant visibility to both customers
and developers through collaborating as well as visible progress through
burn down and velocity charts. The Agile manifesto values working code
over voluminous documentation. Similarly, small releases and continuous
integration are essential practices. Since the code is so important,
make sure the code base is instrumented and stories’ implementations,
unit, and acceptance tests documented so that rather than using cumbersome
project reporting tools one can just produce accurate information from
the code base. “Luke, trust the code”. Why fake it with
tedious reporting tools if you can collect the information at every
build directly from the code base (feeding whatever inane PM tools
are still in use auto magically). Project information can best be obtained
from simple qualitative web based questionnaires filled out each iteration
by the scrum master, with quantitative measurements for the code base.
Towards Open Process Development
To be fair, the process community is very concerned about
the gap between process and programmer and they are working to find
ways to improve
the situation. Many process champions have tried everything they can
to publish processes on the web, wikis and other friendlier formats
which may appeal to developers. Recently, organizations have even been
situating process people with development teams to try to document
the processes actually used by the development teams. However, their
efforts are seldom supported by development organizations, which see
all process as evil overhead.
Recently Randy Miller (http://blogs.msdn.com/randymiller/archive/2005/05/10/416021.aspx)
and others at Microsoft (http://msdn.microsoft.com/vstudio/teamsystem/msf/msfagile/default.aspx)
have made efforts to incorporate processes integrated with development
tools.
In October Eclipse has announced the Beacon Eclipse Process Framework
(http://www.eclipse.org/proposals/beacon/proposal). Per Kroll, principle
author of the proposal, () promises EPF will provide open source process
definitions and tooling to define such processes. In doing so, the
process community is seeking to provide a common place repository for
process descriptions as well as tools for defining and publishing process
descriptions. The existence of open process documentation should reduce
the effort and expense needed to create process description and ideally
offer the opportunity for communities of interest to describe practices,
including typical pitfalls.
4 SUMMARY
It is time for developers to embrace and extend software processes
into a healthy description of your organization’s software culture.
All that is needed is the vocabulary, practices, measurements, lessons
learned and references to places to learn more. Agile software has
better ways of measuring and predicting software development than classic
PMI. Now we need to make the efforts to articulate these new ways to
others. We also need to automate our code bases so that the artifacts
themselves can provide the information that organizations need to manage
and plan their software.
About the author
|
|
Dave Thomas is cofounder/chairman
of Bedarra Research Labs (www.bedarra.com), www.Online-Learning.com
and the Open Augment Consortium (www.openaugment.org) and a founding
director of the Agile Alliance (www.agilealliance.com). He is
an adjunct research professor at Carleton University, Canada
and the University of Queensland, Australia. Dave is the founder
and past CEO of Object Technology International (www.oti.com)
creator of the Eclipse IDE Platform, IBM VisualAge for Smalltalk,
for Java, and MicroEdition for embedded systems. Contact him
at dave@bedarra.com or www.davethomas.net.
|
Cite this column as follows: Dave Thomas: “The Unnecessary Tension
between Process and Programmer”, in Journal of Object Technology,
vol. 5, no. 1, January-February 2006, pp. 7-11 http://www.jot.fm/issues/issue_2006_01/column1
|
