Author Archive

Three Surprises in Software Development in 2012

‘Tis the season to make unsupportable predictions for the future.  Despite my prior record (and I remain surprised that we don’t yet have personal jet packs) I’d still like to share a long-range weather forecast for the software industry.

You’ve been warned.  From here on, you’re on your own.

Prediction 1: Everyone Will Claim They Are Agile

And 50% of them will be wrong, just based on the Nokia test.  And of the rest, half won’t get any value from it.

There are a lot, and here I really need to underline a lot, of bad development practices out there.  For every organization that is killing it with Agile, there are five (my agilesta friends say ten) organizations that are limping along, delivering buggy code to their customers, late, and missing committed functionality.  And often all three.

This “Going Agile Without Knowing How” problem is probably an inevitable result of the success the early-adopter teams had with Agile methods.  For instance, when I watch The Olympics, figure skaters make skating look effortless.  When I do it, I look like a drunken hippo and hurt my butt.  It’s hard to stop and remember that these athletes, in addition to good genetics, spent years at the rink with their coaches learning, trying, failing, and improving, before they got in front of the TV cameras.

Agile has crossed the chasm, and the great majority of organizations have too few people, with too little coaching, and hardly any tooling.  Sure, your boss doesn’t realize how useless your stand-up meetings are, or that your code isn’t fully tested at the end of a sprint, but she’ll eventually see that your customers are not happy.

Prediction 2: Development for Mobile Devices Will Still be Small

Yes, Mobile is really big, and moving fast.  It’s just that the great majority of the work to support useful mobile apps remains in the back office.  When we’re finished inventing new ways to swipe our coffee-stained fingers across our screens, the value of the great majority of our apps is back in the glass house, running Java and C++ on big ‘ole honking (virtualized) servers.

The development problem in the era of proliferating small platforms, remains the problem of dealing with large, complex, data and it’s interactions.

Sleep well, Larry Ellison.

Prediction 3: The Gap Between Pros And Amateurs Will Grow

Every new software technology “spike” rewards the early adopters with higher productivity, which can level the playing field for the “newbie” or occasional software developer.  But as application complexity grows, as the platforms become more complex and development environments become richer, the professional advantage becomes more significant.

There isn’t much of a disadvantage in time-to-market for the young developer, maybe working on his laptop with open source tools and no identifiable process. The difference between them and a team of experienced professionals, working with industrial strength tools and procedures, and building apps that run businesses on virtualized hardware in a web connected world, is in “value created.”

Created Value is a concept that we’ve all been learning during the past ten years of Agile evangelism.  Created Value is measured at the customer side, and primarily by the classic metrics of software: how does the software help me get my job done better and faster?

There was a time when the lowest cost labour source for a software project was the key criteria.  Over the past two years, software projects have been revisiting their decisions as they’ve seen the crippling effects of buggy, unmaintainable, badly architected products.  We’ve seen the evidence with a hiring boom in the US for developers and QA alike.

In short, in 2012, we’ll see a renewed focus on quality of development, over quantity.  And a better appreciation for the talent, tools, and techniques, that create it.


Posted in: Agile, Best Practices, News

Leave a Comment (4) →

Agile Kids Say the Darndest Things

I hope I don’t end up with a seized engine on the side of the road, but if I do, I’ll know I should have had that oil change. I hope I don’t end up on the Worst Dressed List, but if I do, at least I’ll know I should have given away those old shirts.  I feel sorry for those on the “Worst Agile Implementation” list who don’t even know they’re there.

In the past few months I’ve had the privilege of talking to approximately fifty organizations about their Agile implementation.  Most of them are doing well, and many of them have great insights about how they customized Agile to fit their process requirements.  But some of them really Say the Darndest Things.

“We do Scrum, it’s just the rest of the company doesn’t.”

“So first we break the requirements specification into pieces and call each of the pieces a story.  Then we do our iterations and pass them off to the release team.  We’d sure like to get Product Management, QA, and the customer involved, but they don’t want to.”

There are a lot of places an Agile approach can add value, and I’d hate to adopt a “waterfall approach to going Agile”, but you’re really not doing Scrum.  The biggest chunks of value, the incremental use of customer feedback, and going from “completed state” to “completed state” in each iteration are lost if you can’t get more support.

“We’re Agile until the development is done.”

More than once I’ve been speaking with an earnest development leader who’s describing the Scrum process.  They’ll launch in, with obvious pride, and tell me how they’ve gone to two week iterations, do standup meetings, and work from a backlog.  “Terrific!  And how do you do QA?”

Oh, yes, of course they do QA, silly!  In fact, they demo the completed development to the QA team every sprint review and send it off to get tested.  Sometimes, unfortunately, QA actually finds some bugs that need fixing.  So that’s why they put the sprint on hold for a while to fix the bugs and loop them back into QA “’cause we don’t want to wait an entire sprint before they can restart the testing.”

The other side of this one is the guys that take the old “Release Tail” loophole for all it’s worth.  “Yes, Lorne, we’ve been agile for three years now.  We do Scrum, unit testing, standups, and play in the World Series of ‘Planning Poker’.  We do that for about six weeks, or until the release.  Then we have a three month release testing tail, which follows a ‘modified Scrum process’ … the project leader estimates the amount of work on each bug QA finds, and assigns it to a developer.  Sure, sometimes we have to work on new functionality during the “release testing tail” … you can’t expect the customer to stop needing improvements for three months!”

Folks, I don’t think I’m sharing any great trade secret when I tell you the QA process needs to be completed before the story is considered “done.”  I don’t want to be Klaus Fuchs of Scrum, but here’s the secret: you’re going to have to invest more in testing up front.

“We do continuous integration every night.”

I blame the education system: how’s an engineer supposed to know what “Continuous” means when we have “social promotion!”  Now some people understand the idea of continuous integration, and made a conscious effort to make it more “Discrete”.  Some companies I talked to had broken builds that lasted for a week.  You’d rather have a child repeating “Mummy” every 30th of a second before you’d like to get an email every five minutes saying the “Build Failed.”  I get it.  And if the email was going to your boss too, well, you don’t have to be Dogbert to know that’s a bad idea.

Builds are going to fail.  Get used to it.  The problem is not that the build failed, but that you couldn’t fix it.  Good practices are to have the team drop what they’re doing when the build fails and hop on fixing it.  If they can’t fix it, it needs to get escalated *pronto*.  Better is to have the team do local builds and unit testing before they check in.  Best Practices are to divide up the build process by team and stage of development, so your team only pollutes itself, not the rest of the development org.

“We don’t need training since we can use the internet.”

Uh huh. So I guess the schools will be shutting down any day now.  Not that the Internet might not turn out to be a useful aid someday, but the software development process is a hands-on activity.  And similar to other hands on activities, like dancing or carpentry, you can’t learn to do it by reading a book.  You’re going to need to get some experience with the process before you understand how to run a sprint review or a stand up, how to estimate stories, and how to work with your QA partner.

Now if you’re a hobbyist and working for free, your time is cheap, and there’s no reason not to use trial and error as a learning method.  But if you’re getting paid, and your work is important, you really don’t want to waste four sprints figuring out what someone can help you get right in sprint two.

I’m hoping my surgeon, pilot, and barber got a few lessons before it was my turn.


No one has to pass a test to call themselves “Agile,” nor should they. Agilistas don’t have a monopoly on the right way to develop software.  But when people believe they’ve made it to Agile without using critical Agile concepts like time boxing development or getting to “done”, they’re missing the real value.


Posted in: Agile, Best Practices, continuous integration, Enterprise Agile, Humor, Scrum, Tips and Tricks

Leave a Comment (1) →

When Scrum is Not Enough: Doing Waterfall with Scrum

To paraphrase Harry Truman, if you’re looking for love in the software tools business, buy a dog.  If you’re looking for areas to improve, ask your customers.

For some time now I’ve noticed that those customers of ours developing both software and hardware find benefits in being primarily Agile in their software teams, but primarily Waterfall in their hardware teams.  It’s hard to fit “parsing a Sonet Frame Header” into a story, and it’s expensive to do a feature add iteration on the Mars Rover.  Especially when it’s already on Mars.

Interestingly, a large number of our customers’ software development groups have a mix of Agile and Non-Agile projects.  Sometimes there are tactical issues keeping them from being successful in Scrum at the current time (the group is too junior, they don’t have enough trained Scrum Product Owners to keep up, etc.), but some groups have especially difficult challenges making Scrum adoption difficult.

Where Scrum is Easiest to Adopt

I have come to the conclusion that Scrum works best when there is a linear relationship between team effort and customer benefit.  For projects that are highly non-linear, a process that invests more in up-front planning and architecture and then uses Scrum later in the process, works well and is easier to adopt.

Let’s look at a classic Scrum-benefit project.  You have an application in production use with a set of customers.  Customer feedback has resulted in a list of feature adds, each of which is in the one- to six-weeks of development effort range.

In this situation, plus a little training and tool support, the team will leverage Agile techniques like time-boxing, continual customer feedback, integrated QA and development, etc. and rapidly start to deliver real customer value.  Pretty soon, the customer will start asking, “Why don’t all your development teams use this new fangled process?”

There’s a nice linear relationship between the amount of work done during each iteration and the amount of feedback the customer gets.

Here are two curves, demonstrating the concept:

LorneGraph When Scrum is Not Enough: Doing Waterfall with Scrum

The customer feedback in the linear case is used to guide the product as it is developed, which overcomes the errors found in a traditional requirements documentation exercise.  This is the great win in the Scrum process: too many requirements of development turn out to add negligible customer value in the final delivery.

In the non-linear project, by the time the customer has gotten to week 10, they’re disengaged and putting pressure on the team to work on something else.

Agile engineering practices, like test driven design, unit tests, continuous integration, etc,.  add value whether  or not you end up at Scrum: better engineering practices are always useful. Is the development cadence produced by regular iterations more valuable than the executable, milestones in a good project plan?  In a linear problem, yes, there’s a lot of evidence that it is.  In a non-linear process, some teams see the benefits all the way along the process, and others benefits later in the project, after some of the basics of the architecture have been mapped out.

Non-linear Returns

Unfortunately, not all situations are linear.  There are several factors that produce non-linear returns.  Amongst them are such typical software challenges as:

-          Tent-pole like problems. This refers to the situation where there is a tent held up by a lot of tent poles.  Taking out a few tent poles doesn’t decrease the height of the tent. But when you take them all out, the tent lies on the ground.  They typically require you to develop a clear architecture and strategy, up front, and invest in working to fill out the architecture.  Examples of this kind of problem include supporting a new language (e.g. Japanese, not C#), greatly improving performance or porting to a new platform.

-          Implementing something of technical risk. If you don’t know *how* to do something, the Agile process of “doing Spikes” does not necessarily converge on a solution in a fixed amount of time.  Be advised that in this case, in Waterfall as in Scrum, you may end up going down multiple blind alleys, so the outcome better be worth the uncertain cost.

-          Developments with high business risk relative to development effort. Software controlling the Mars Rover is an example, but so is software embedded in a cardiac pacemaker, software that runs your e-Commerce site, or software that can make real-time stock market trading systems.

Success in non-linear projects often leverages the best practices of Agile in ways familiar to project leaders of Waterfall processes.  Two week iterations might not make a lot of sense when a customer‘s input isn’t of much use, but continuous integration, focus on upfront test, and meaningful, executable, milestones every week or two that contain customer value sure do.

Finally, a note about Pollution, or “technical debt.”  Each time we implement something of customer value, we either increase or decrease the technical debt load in the application.  Usually, when we focus exclusively on the customer story, we increase technical debt (“pollute”), and when we invest in decreasing technical debt, we deliver less customer benefit in the short term, in the interest of having better customer return in the future.  This is not a case of non-linear returns, but rather a symptom of a weak Scrum implementation.  Avoiding the build-up of technical debt is fundamental to a good implementation, and is one of the challenges of Agile management.

Most companies that build mission-critical software find they have both types of problems.  Their requirements on us as a tool and process partner are difficult but straightforward: support both processes with the same tool stack, and integrate development to enable code sharing.

Agile may be the single biggest advance in software development practice since The Mythical Man-Month, but it’s not The Magic Bullet.  Smart companies are learning how to utilize the best procedures for the problem at hand.

Posted in: AccuRev, Agile, Best Practices, Integrations, Scrum, Tips and Tricks

Leave a Comment (0) →
Page 1 of 5 12345