I was browsing reddit.com the other day and ran into this post:

Yup. It’s true. The tried and true development approach of Waterfall that we’ve been using for years was an example of what NOT to do for software development.

From the Wikipedia article: The first formal description of the waterfall model is often cited as a 1970 article by Winston W. Royce,[3] though Royce did not use the term “waterfall” in this article. Royce presented this model as an example of a flawed, non-working model (Royce 1970). This, in fact, is how the term is generally used in writing about software development—to describe a critical view of a commonly used software practice.

That’s what’s amazing about waterfall, and the agile transformations that seem to be taking the industry by storm. Maybe we all know deep down there is a better way to develop software.

I hope someday we don’t look back on Agile the same way we look back on Waterfall. I don’t think it will happen for the simple reason that Agile doesn’t have one methodology tied to it. Agile can mean a simple set of practices to help with software development, but it’s more like a mission statement as opposed to a plan.

I hope I don’t end up with a seized engine on the side of the road, but if I do, I’ll know I should have had that oil change. I hope I don’t end up on the Worst Dressed List, but if I do, at least I’ll know I should have given away those old shirts.  I feel sorry for those on the “Worst Agile Implementation” list who don’t even know they’re there.

In the past few months I’ve had the privilege of talking to approximately fifty organizations about their Agile implementation.  Most of them are doing well, and many of them have great insights about how they customized Agile to fit their process requirements.  But some of them really Say the Darndest Things.

“We do Scrum, it’s just the rest of the company doesn’t.”

“So first we break the requirements specification into pieces and call each of the pieces a story.  Then we do our iterations and pass them off to the release team.  We’d sure like to get Product Management, QA, and the customer involved, but they don’t want to.”

There are a lot of places an Agile approach can add value, and I’d hate to adopt a “waterfall approach to going Agile”, but you’re really not doing Scrum.  The biggest chunks of value, the incremental use of customer feedback, and going from “completed state” to “completed state” in each iteration are lost if you can’t get more support.

“We’re Agile until the development is done.”

More than once I’ve been speaking with an earnest development leader who’s describing the Scrum process.  They’ll launch in, with obvious pride, and tell me how they’ve gone to two week iterations, do standup meetings, and work from a backlog.  “Terrific!  And how do you do QA?”

Oh, yes, of course they do QA, silly!  In fact, they demo the completed development to the QA team every sprint review and send it off to get tested.  Sometimes, unfortunately, QA actually finds some bugs that need fixing.  So that’s why they put the sprint on hold for a while to fix the bugs and loop them back into QA “’cause we don’t want to wait an entire sprint before they can restart the testing.”

The other side of this one is the guys that take the old “Release Tail” loophole for all it’s worth.  “Yes, Lorne, we’ve been agile for three years now.  We do Scrum, unit testing, standups, and play in the World Series of ‘Planning Poker’.  We do that for about six weeks, or until the release.  Then we have a three month release testing tail, which follows a ‘modified Scrum process’ … the project leader estimates the amount of work on each bug QA finds, and assigns it to a developer.  Sure, sometimes we have to work on new functionality during the “release testing tail” … you can’t expect the customer to stop needing improvements for three months!”

Folks, I don’t think I’m sharing any great trade secret when I tell you the QA process needs to be completed before the story is considered “done.”  I don’t want to be Klaus Fuchs of Scrum, but here’s the secret: you’re going to have to invest more in testing up front.

“We do continuous integration every night.”

I blame the education system: how’s an engineer supposed to know what “Continuous” means when we have “social promotion!”  Now some people understand the idea of continuous integration, and made a conscious effort to make it more “Discrete”.  Some companies I talked to had broken builds that lasted for a week.  You’d rather have a child repeating “Mummy” every 30^th of a second before you’d like to get an email every five minutes saying the “Build Failed.”  I get it.  And if the email was going to your boss too, well, you don’t have to be Dogbert to know that’s a bad idea.

Builds are going to fail.  Get used to it.  The problem is not that the build failed, but that you couldn’t fix it.  Good practices are to have the team drop what they’re doing when the build fails and hop on fixing it.  If they can’t fix it, it needs to get escalated *pronto*.  Better is to have the team do local builds and unit testing before they check in.  Best Practices are to divide up the build process by team and stage of development, so your team only pollutes itself, not the rest of the development org.

“We don’t need training since we can use the internet.”

Uh huh. So I guess the schools will be shutting down any day now.  Not that the Internet might not turn out to be a useful aid someday, but the software development process is a hands-on activity.  And similar to other hands on activities, like dancing or carpentry, you can’t learn to do it by reading a book.  You’re going to need to get some experience with the process before you understand how to run a sprint review or a stand up, how to estimate stories, and how to work with your QA partner.

Now if you’re a hobbyist and working for free, your time is cheap, and there’s no reason not to use trial and error as a learning method.  But if you’re getting paid, and your work is important, you really don’t want to waste four sprints figuring out what someone can help you get right in sprint two.

I’m hoping my surgeon, pilot, and barber got a few lessons before it was my turn.

Finally…

No one has to pass a test to call themselves “Agile,” nor should they. Agilistas don’t have a monopoly on the right way to develop software.  But when people believe they’ve made it to Agile without using critical Agile concepts like time boxing development or getting to “done”, they’re missing the real value.

Did you know that CM systems back in the day were basically people? This is where the term “check-in” & “check-out” comes from- it refers to the days when there where actual software librarians would record peoples changes and check them in and out like books on disk or punch cards. It’s mind boggling to think of software this way.

If I was to ask software developers today what “software configuration management” was, they would probably say “SCM? Like Subversion?” Incorrect! You need to trust me on this one, SCM is not the same as a version control system. Yes, your version control system is an SCM tool (confusing?) but SCM is a broader discipline and technique that encompasses the management of change in software.

The introduction to the IEEE “Standard for Software Configuration begins with:

SCM constitutes good engineering practice for all software projects, whether phased development, rapid prototyping, or ongoing maintenance. It enhances the reliability and quality of software by:

• Providing a structure for identifying and controlling documentation, code, interfaces, and databases to support all life cycle phases
• Supporting a chosen development/maintenance methodology that fits the requirements, standards, policies, organization, and management philosophy
• Producing management and product information concerning the status of baselines, change control, tests, releases, audits, etc.

Let’s be clear- all of the things on this list do not fit under the heading of your version control system. Many of them will require practices and policies to maximize your development efforts and methodologies. With version control, release engineers will still have to perform some of these SCM related functions:

• Merge early and often
• Enforce a workflow for development teams to follow
• Record and have full visibility into all of the changes that were made
• Write build and compiler scripts
• Automate builds, deploys and tests
• Understand the dependencies between projects and code
• Maintain the development environment for a team
• Be responsible for the final product going out the door

That’s just the tip of the iceberg. A talented release engineer or SCM expert can do all of those things independently, but his or her job would be a lot easier with SCM tools that can automate and facilitate the necessary practices and processes. (This includes version control, compilers, debuggers, editors, continuous integration machines, automated deploy, and the ITS system.)

At it’s core, SCM answers the question “Somebody did something, how can one reproduce it?” In addition it’s about understanding and establishing relationships among items that are likely to change. It’s a tricky job, not one that’s easily understood. We have to understand the relationships between versioned artifacts, like code, hardware, documents, design models and even directory structures. In addition we have to do all of the necessary things to make those versions valuable to our organization. We have to design process, workflow, automation, build automation, reports and security.

With all of this, don’t tell me that SCM is the same as version control. Trust me on this one!

A funny thing has happened over the last couple of years…. we started calling waterfall software development… er… “waterfall”. By this I mean, we never had a name for the process… waterfall was just called “software development”. There was no distinction or name for what we were doing- there was only one way. This is a draw of agile, it’s something different.. and it’s the only new development methodology to actually get attention in the last 10 years.

So, what’s the deal?

#1) Agile is an umbrella term for several methodologies.
Agile encompasses a lot of different things; it can mean different things to different people. This might be why people have such a hard time understanding it. So comparing “waterfall” to Agile isn’t entirely accurate, or possible, since it’s like comparing one NBA team to all of MLB. Agile encompasses several methodologies (such as XP, Scrum, Kanban), which are all iterative in nature… that brings us to…

#2) Agile is iterative.
Yes, agile is an umbrella term, but all of the methods in agile share common core values: The fundamentals are to incorporate iterative development and to have continuous feedback so that you can always improve. This means you continuously plan, continuously test, and continuously integrate so you can adapt when needed.

#3) Agile is adaptive, not predictive.
Do you remember what “waterfall” was like back in the day? We spent months gathering business requirements, writing specs, and designing, and then spent the next 10 months coding. Since we spent the first few months trying to predict what the next 10 months would entail, we could never accurately estimate how much work a task was supposed to be, and heaven forbid the requirement changed half way through! Agile is an attempt to shorten that cycle so we don’t have to waste 10 months before find out something was wrong.

#4) You can pick and choose what methods you want to implement.
It’s funny. I ask people all the time, “How agile are you?” They typically say “Well, we’re somewhat agile, but not fully agile.” People tend to measure some sort of agile “zen” in their head, and that doesn’t exist! If you’re practicing some agile methodologies, you’ve won half the battle.
You’ve won half the battle if you are practicing:
·         Continuous Integration
·         Agile Workflows
·         Test Driven Development
·         Short Iterations
·         User Stories
There’s no out of the box way to do this, but if these methods work for you… then you’re there.

#5) The genius Of Agile is in the name.
Since the word “Agile” can’t be traced back to a specific methodology like “waterfall” we probably won’t ever think of it as “development”. In addition since it’s not a prescribed method of doing things (ex: Watefall.. requirements->design->implementation->verification->maintenance) it just can’t fail… whatever methods we use can always be improved and adapted to best suit our needs.

Mammoth retailer Target must have had very high hopes for big sales when a new designer collection went on sale last week. Unfortunately it ended up being a bust. (The site quickly became overwhelmed and crashed several times throughout the day.) Online retail experts say losses for the retailer could run into the millions as a result.

Scenarios like this are not new, though with a large retailer like Target, it was somewhat surprising. The painful reality of eCommerce is when transactional websites experience dramatically increased traffic, there is always a chance that, without well maintained software development measures in place, crash scenarios like Target’s will occur.

When an eCommerce crisis like this occurs, developers go into fire drill mode to make rapid changes so the site is back up and running as quickly as possible. Software change and configuration management tools play a crucial role in both ongoing updates and crisis management updates, like the Target example.

It’s also important to note that traditional SCM tools won’t cut it when it comes to high volume transaction web sites like Target, Amazon or any major retailer. Most retail and eCommerce sites need enterprise SCM tools that can scale to accommodate large, geographically distributed teams of developers working in parallel. More advanced SCM tools are integral to the success of transactional web sites, both from a day-to-day aspect, and when there are spikes in traffic and transactions for special sales events, promotions, and holiday shopping.

While last week’s incident demonstrated what can go wrong when an eCommerce site experiences significant problems, there are some great stories out there as well.

One such example is that of a very large and well known travel eCommerce site that receives an average of ten million visitors a month.

As with most of the top of travel eCommerce sites, this one is very dynamic. Users can make reservations for airlines, car rental, hotels, sign up for text message alerts about delayed flights, rate their own experiences and much more.

While these features are popular with consumers, these transactional environments are highly complex to manage. At this particular website, there are as many as 40 different development teams working on 40 different projects. One “project” might be car rentals, another hotel reservations, etc.

When development environments have this much complexity, when teams are working in parallel on a wide variety of projects, only the most powerful SCM tools can help minimize version control problems that lead to bugs slipping through. What may appear at first to be a minor issue can, in the worst case scenario, cause an array of site problems and crashes that have a direct impact on the consumer and revenue. This travel site, which uses AccuRev’s SCM solution, is able to do everything possible to ensure the site and the software behind it stay up and running smoothly, 24/7.

The web storefront is a business critical tool for any type of eCommerce business.  For these reasons, enterprise level SCM tools under the hood are essential to success.

What other similar success and web crisis/failure stories have you experienced directly or heard about?

AccuRev’s security has advanced to the next level with the release of version 5.2. One of the great new features that I’m happy to introduce is Access Control Lists, or ACLs.

AccuRev’s ACLs

Have you ever wanted to fine tune your security for files or streams based on role, group or user? Now you can using our new built in security commands. Here’s how it works:

In addition to locks, each stream can have one or more permissions. This also applies to files and directories. This means that you can control who can modify/see certain streams, and who can modify/see specific elements.

Setting ACLs are easy, this can be done via the CLI:

C:\Users\clucca>accurev setacl stream phoenix_Client chris all

The ACL has been set successfully.

This will now lock me out of all the files under the phoenix_Client stream…

If I try to access that stream in the GUI, I will receive this message:
This also works for files and directories, which is great for teams that want to have secure outsourcing and security conscious development initiatives.

In addition you can use the lsacl command to list permissions. You can see that for the stream phoenix_Client chris has no rights to access it, but the permission doesn’t inherit to the rest of the streams, so I can access everything underneath it.

At AccuRev, we’ve noticed a new practice emerging that we call “Issue Based Development.” You may have heard of this term or something like it such as Work Item Based Development or even “requirements traceability.” Whatever you call it, more and more teams are gravitating towards it, perhaps as a result of adopting some of the Agile practices. In this post I will give an overview of Issue Based Development as well as talk about how to maximize your success with the practice.

Issue Based Development is the practice of breaking all work down into individual issues; associating all work products such as code, tests, and documentation with those issues; and managing the progression of work through the various stages of your software lifecycle via those issues.

The benefits of Issue Based Development include requirements traceability, better progress and status tracking, and simpler application of patches from one stream of development to another. These benefits and more will be covered as we cover the individual practices that make Issue Based Development possible.

Track All Work by Issues

Whether you have requirements, enhancement requests, escalations, defects, bugs, tasks, user stories, RFE’s or some other way to describe work, it all goes into issues. You may use a different term to generically describe tracked work, such as work items, but in this post I will use the term issues.

Today, many companies use some combination of Excel, Word, a Requirements Management system, a bug tracking system, MS Project, and an Agile Project Management tool to track “different” kinds of work. In the end though, somebody has to be assigned the work and that work needs to be explained to them in a way that makes sense to them. Why not use issues as the final unit of describing work, whether you continue to use the other systems or not?

Make All Issues Available in a Single Place

Using issues to describe and track all work implies that all issues for a particular project eventually end up in a single Issue Tracking System (ITS). The obvious way to achieve this is to simply move everybody to use the same ITS and to use it for all work-related artifacts, from defects to requirements to whatever else it is that you use. In reality this is difficult to achieve, but is the right goal to strive for, to the extent that you can.

Synchronize Multiple Work Tracking Systems

A good complement to moving as much work tracking to a single ITS system as possible is to synchronize issues from other work tracking systems to a single ITS system. For instance, AccuRev provides AccuBridge and AccuSync to enable you to have all of your work in a single system.

Link Issues to the Work Performed

Once you have all of your work tracked via issues in a single ITS, you are now ready to take the next step, linking all of your issues to the work performed to implement those issues. I’m assuming that you are keeping all of your work artifacts such as code, test, and documentation in your Software Configuration Management (SCM) system. The best way to link your issues to the work performed is to use an SCM object called a Change Package. Some systems may have a similar object called a change list or change set. AccuRev uses change packages.

By linking issues to work artifacts, you can now manage by issues instead of by files. For instance, instead of trying to figure out which files correspond to a set of hotfixes that you want to apply to new development, you can instead refer to the issues that represent those hotfixes and have the SCM system do all the hard work of determining the exact set of changes and promoting those changes from one environment to another.

Use a Stream Per Stage

Having a set of promotion stages is a great complement to Issue Based Development. By having a set of stages corresponding to your development workflow, you can easily see what stage of development various issues are really in simply by querying your SCM system. Doing reports against your ITS are good, but those are generally hand updated. Your SCM system reflects the real state of your code.

A typical set of stages might be “Todo,” “WIP,” “Coded,” “Tested,” and “Done.” You can represent each of these promotion stages with Streams in your SCM system. This allows you to associate Continuous Integration servers which each stage and build versions of your product corresponding to those stages. Imagine for instance being able to build a version of your product that has only tested stories in it.

Another benefit of this arrangement is that it enables “One Piece Flow.” For more on this topic, see my related (http://damonpoole.blogspot.com/2009/09/one-piece-flow-transitioning-from-scrum.html) DIY Agile blog post.

Keep Issues as Small as Possible

Most organizations are good at tracking all bugs in a bug-tracking system, but usually track new work via Word documents which describe very large chunks of work. When moving to Issue Based Development it is important to break these large chunks down into as many and as small issues as you possibly can. This is important for two reasons.

First, it is next to impossible to measure real progress against very large chunks of work. By breaking work down into small issues, it is easier to measure progress by seeing how many issues are fully coded, tested, documented, and ready to ship and measuring the amount of effort associated with those issues.

Second, working on small issues reduces the chances of “issue entanglement.” Entanglement occurs when two issues have changes that are dependent on each other in such a way that they become impossible to promote independently. The ideal is to have one person (or programming pair) work on a single issue at a time and for that issue to be finished in days rather than weeks. The more people working on an issue an the longer they are working on that issue, the more likely it is that the issue will become entangled with multiple other issues.

I’ve often heard people wonder why the SCM system can’t make this simpler. At AccuRev, we are always striving to make Change Packages simpler to use, but the simple fact is that there are limits to how much can be done to simplify entanglement issues. It is generally much more productive to apply the practices in this post which have benefits in and of themselves as described throughout.

Apply the Open/Closed Principle

Large files are another factor that contributes to issue entanglement. Often, frequently used and/or complex classes create large source files. This is similar to the problem of large issues. The larger a file is, the more likely it is that multiple people will be working on it within the same timeframe and thus the more likely it is to lead to issue entanglement.

There is a very useful principle that comes from Objected Oriented programming called the “Open/Closed” principle. This is the idea that software should be open to addition, but closed to change. In its pure form this means that you only add and extend classes and change them as a matter of last resort. A slightly modified version of this would be to only add new methods and to avoid changing them. Also, changing methods should be restricted to rewriting the code such that it adheres better to the principle by being easier to add new methods to and requiring less change to existing methods. This is a very common refactoring pattern.

The side-effect of the Open/Closed principle is that you end up with smaller source code files. The smaller your files are, the less likely you are to run into entanglement and merge issues.

Start Towards Issue Based Development Today

Whether you follow these principles and practices to the letter or not, the more you follow them, the easier it will be to practice Issue Based Development. The better you are at Issue Based Development, the more success you will have with requirements traceability, progress and status tracking, and the application of patches from one stream of development to another.

I’m very excited about our 5.2 release! We’ve completed the move to PostgreSQL on the back end, fully internationalized our products, and added a slew of new features, like per-element security so that you can lock down certain files or directories to specific groups or users.

In addition to moving to PostgreSQL, we’ve taken advantage of its capabilities to increase performance in a few key areas such as update, especially when using cross-links, populate, and more. And as a first step towards fully embracing 64bit servers, we now support 64bit CPUs on Linux servers. You can look forward to additional performance boosts as we exploit the capabilities of PostgreSQL in the future.

Haven’t upgraded in a while? Check out what else is new! To get a full flavor of all of the new functionality we’ve added recently, check out recent webinar which highlights what’s new.

If you haven’t tried our plug-ins in a while, we’ve put out new releases of our Eclipse and Visual Studio plug-ins for all versions of AccuRev. Eclipse and Visual Studio plug-ins will work with older releases, so no need to wait for 5.2 to start benefiting from the new features in the plug-ins.

We’ve fully integrated the Web UI into the plug-ins so you can now use all of the following functionality: version browser, annotate, stream browser, and all of AccuWork. Also, in 5.2 you can now access the Web UI with a single click from most screens. For instance, if you want to send somebody a URL of a file or diff, you can just click the “copy to clipboard” button in the toolbar and then compose an e-mail to somebody and paste in the URL. When they click the URL, they will go right to that file or diff. And the Web UI now allows you to print any table that it supports.

While we are in the process of making the new functionality in the Web UI available in all of our UIs, this is a great first step to quickly link to the new functionality.

A Few More Details on What’s Available Via the Web UI
If you haven’t used the Web UI or are using an older version, you may not have heard that there is an entirely new version browser in the Web UI and for AccuWork users- AccuWork on the Web has been completely revamped. For instance, it is easy to search by keyword and you can do drag and drop query editing.

5.3 and Beyond… Here We Come!
The database and internationalization are two very important infrastructure changes which will accelerate both our engineering velocity as well as our market penetration. With our upcoming 5.3 release we will be re-introducing our quarterly pace of releases as we had with our 4.8 and 4.9 releases. With the switch to PostgreSQL and an internationalized code base complete, you will see the release of new functionality faster than ever before. In preparation for 5.3 and beyond, we’ve just launched a new product survey as one of the many ways we collect product priorities. Make sure to look for it and fill it out. It is one of the best ways to have a significant impact on our product roadmap. If you haven’t gotten an e-mail yet, you may not be in our survey database. Just ask your AccuRev administrator for a link.

In a recent post I talked about branching and merging, and offered some tips on how to avoid development nightmares involved with these procedures.

An integral part of the solution I cited is stream branching, which easily accepts parent code while allowing developers to then push code out to other streams. This makes merging easier because the entire file change history is available and automatically tracked.

Streams have been an important step in advancing software configuration management (SCM) technologies and processes. Now, with software development growing more complex seemingly by the day, the time has come for the next phase of more robust SCM solutions.

Between globally distributed teams, emerging methodologies such as Agile, and market demands for faster, better and more cost-effective software, there is great demand for enterprise developers to better manage all of their activities. In addition, industries such as financial services and healthcare face the added complexity of compliance with regulations such as SOX and HIPAA.

I’m excited about some big news we’ll be announcing next week around these challenges. What AccuRev is announcing will be a major step forward both for our clients and the software development industry – it really is the next leap forward for SCM solutions.

I can’t reveal too many details at this point, but stay tuned for more details coming soon – on this blog and the news wires!

Goals of Continuous Integration

If a primary goal of continuous integration is to improve software quality and reduce downstream build and test failures, then perhaps the single most important SCM best practice is to enable automated local developer builds. To this end, developers should be provided with access to a local build environment that mirrors the production build environment, including:

• Database schema
• Configuration files
• Environment variables
• Shared libraries and files
• Known compiler/linker/runtime environments

This minimizes the “it compiled fine on my machine” syndrome that plagues many development efforts. Once the build environment is available, every effort should be made to automate the local build, so that each developer can build easily and quickly without having to perform a series of manual steps.

Ideally, the developers should have access to a private build area that uses the same tools and configurations as the production builds. As part of configuring a local build, developers should be trained to take several steps to ensure that local changes will have minimal negative effects once shared with other developers and teams. These steps are as follows:

• Updating their private workspace to obtain the latest shared configuration code
• Merging conflicting changes from the shared configuration to their private workspace

By ensuring that developers have known, good build environments, and that they have the latest code with all conflicts resolved, there is high probability that the local build will succeed. If the local build is successful, a continuous integration build that includes the developer’s changes should also succeed.

Shifting to Continuous Integration and Frequently Updating Code

Traditionally, developers tend to put off sharing their changes, sometimes for days, because they don’t want to affect other people too early, or don’t want to get blamed for breaking the build. Unfortunately, this strategy tends to backfire and typically leads to more problems in debugging larger sets of changes after a build breaks. Independent of whether continuous integration is being employed, encouraging developers to build, test, and share their code in small “chunks” is a simple and effective way to improve team collaboration and reduce costly broken builds.

The importance of frequently updating the code in the SCM system is amplified when organizations move towards a continuous integration model. Continuous integration represents a paradigm shift for software development, emphasizing communication between developers about what changes have been made. By breaking down tasks into small chunks that take several hours to complete, developers can “commit” or “promote” their changes frequently and receive immediate feedback about the quality of the changes through the continuous integration build and unit test results. As teams get accustomed to this new approach, developers also gain a sense of progress by seeing not only their own changes build successfully, but also seeing the changes that other developers have made available.