Scott Ambler, a consultant from Ronin International, Inc., talked about agile development which emphasizes incremental and iterative development in methodologies such as Extreme Programming (XP), Agile Modeling (AM), Dynamic Systems Development Method (DSDM), and SCRUM. These methods exist in stark contrast to the waterfall methods described in IEEE 12207 and the Rational Unified Process (RUP).
He first introduced the Manifesto for Agile Software Development:
1. Individuals and interactions over process and tools.
2. Working Software over comprehensive documentation. Indeed, documentation is the worst way to communicate and helps doom projects. He asks: Are we developing software or are we developing documentation? Scott often cited the analogy: If you were to purchase a car, and because of time or budgetary constraints, could only get the car or the user's manual, which would you choose?
3. Customer Collaboration over contract negotiation. Play to win, not play to lose.
4. Responding to Change over following a plan. If you write software based upon 6-month-old requirements, then your software is old and stale even as you just build it. "A requirement late in the cycle can be competitive if it can be acted upon." Mary Poppendieck.
Scott structured the rest of his talk as one might structure an agile project. He elicited topics from an audience of 75 and gave everyone three votes on each topic to determine the order in which he would discuss them. His tallies emphasized speed, not accuracy; the point was not to get bogged down in the process, but get to the meat of the talk.
It helps to think of the money applied to the project as a tap rather than a stack. The developer should be paid on a regular basis (weekly, bi-weekly, monthly) rewarding him for good work and penalizing him for bad. In this scenario, both the customer and the developer share risk--and rewards. This approach also ensures that your money is diverted to teams that can actually use it wisely.
While this might be more work for the customer, and you won't know in advance how much a project might cost, it is absolutely the wisest way to manage your money. A $2 million, 4-year-project does not guarantee results, and all you may have to show for it is a stack of useless documentation, in contrast to the agile project, which has working software after each iteration. Since the feedback loop is shortened with an agile project, you'll know sooner if the project has failed so that you can shut it down early, saving money.
Tongue in cheek, Scott suggested you could argue your case as follows: "If you're more comfortable with lots of documentation and a process proven to deliver a 65% failure rate..." While there are not yet studies that show the success of agile methods, one technique of XP, pair programming, is already a proven method.
A typical company will have folks who specialize in databases, modeling, Java, quality assurance and so on. This typically produces hand-offs from one specialist to the other, and these hand-offs produce lots of documentation (and no code). Because of this, the process is very slow and inefficient, and it is nearly impossible to generate a tight feedback loop.
In contrast, an organization would be better served to form a team of craftsmen or generalists with no real skills (laughter). They continue to learn new specialties over time and thereby gain a good understanding of the entire system. This makes things more interesting for the generalizing specialist, and makes them more employable as well. They are ready and willing to move on to the Next New Thing. Finally, they communicate with each other with interactions and code resulting in a tremendous amount of cohesion as everything that relates to the project is in one place.
Existing modeling produces lots of artifacts: flowcharts, UML, use cases, business rules, etc. One should constantly learn new tools, use different tools at the same time, iterate back and forth between the models, and do several models at the same time. Use the simplest tool and the right tool for the job: Scott used a flip-chart instead of Powerpoint since it is easier to write on during the presentation. Your users and customers don't know the Rational Rose CASE tools, so use a white board when talking with them. Keep in mind that UML is insufficient for user interfaces and data modeling.
At the beginning of a project, you and the customer will generate a stack of requirements. The developer has the responsibility to estimate each requirement and the customer has the responsibility to prioritize them. Once this is done, pull approximately two weeks of requirements out of the stack. Your project may require iterations of slightly longer duration, but two weeks is a good starting point. How many requirements can be implemented in an iteration? In the XP Planning Game, you (quickly) assign points to each task. At the end of an iteration, you count up the points in the completed tasks to calculate your velocity. This can be used to determine how many tasks should be chosen the next iteration. This velocity should improve over time as you get better at working together.
New requirements are absolutely no problem. They are estimated, prioritized and stuck in the stack with the rest.
Scott asked the audience whether the members could make trivial changes to their databases on the fly such as inserting or changing the name of a column. Many folks in the audience laughed as if to say, "Yeah, right."
Why can't people modify databases, Scott wondered. Incompetence? Databases need to be iterative and incremental too. Typically, the reason that databases can't change is due to coupling, that is, all of the applications talk directly to the database. The solution to this problem is to use an encapsulation layer. Then, if you add or modify a field, you only need to make your application changes in a single, centralized point. One very large organization in the audience did precisely this, which enabled them to change their customer table, test it, and roll it into production that day. It is possible to be agile with respect to data.
The two golden rules of database development are: 1) New applications must have a full regression test suite, and 2) There must be a full regression test suite for the database and every application that hits it.
That way you can make a change, even a trivial one, push a button, and test it throughout and know immediately if the change broke anything or not.
Legacy code that you need to modify must have regression tests put in place. Over time you'll slowly build up a full regression test suite for your legacy assets.
A refactoring is a change that improves the design of your code without changing its semantics. If you have a new requirement that can't be easily added, refactor the code until it can. Thus, your architecture emerges over time. If you follow the rules, you know that your design is perfect.
So you want to develop software with XP techniques. At present, you'll either have to find agile-friendly environments in the startups--early adopters in the acceptance life cycle--or wait 10-20 years until the laggards are using XP.
If you want to try XP in the big, stodgy, companies today, introduce techniques slowly, float some ideas, and let them sink in. "There is nothing more annoying than evangelists going door to door, but you might have a beer with them and pretty soon you're drinking their Koolaid." Getting management to buy into XP is all psychological. Scott recommended the books "Navigating Change" by Donald C. Hambrick, and "Who Moved My Cheese?" by Spencer Johnson.
In closing, Scott thinks that XP could deliver faster, higher quality software than a CMM level 5 project and could also be used in life and death projects.
You can contact Scott via email at firstname.lastname@example.org and read more at agilealliance.org, agilemodeling.com, agiledata.org, xprogramming.com, pairprogramming.com, and www.refactoring.com.
You can reach Bill Wohler via email at email@example.com or visit the website at http://www.newt.com/wohler.
Copyright © 2003 Bill Wohler
Last modified: 2006-02-06 15:34:51 -0800