There’s no denying that this has certainly been the Agile decade for the software development industry.  It’s evident all around us in this tenth year since the Agile Manifesto was created. Most companies and development organizations today have implemented some form or aspect of Agile methodology into their software development processes. Whether you’re aiming for pure Agile or a mixed/hybrid approach, proven best practices in all phases of the software development lifecycle are crucial to success.

This is especially true in the case of continuous integration, one of the foundational aspects of the Agile methodology. The concept of continuous integration, as defined by Martin Fowler, is “a fully automated and reproducible build, including testing, that runs many times a day.  This allows each developer to integrate daily, thus reducing integration problems.”

With this approach, developers can work more closely in parallel while identify problems and debugging on the fly, accelerating the development process and improving the quality of the finished product.  The benefits of continuous integration are tremendous, but can quickly be eradicated if software configuration management (SCM) best practices are not carefully followed.

There are a handful of SCM best practices that can optimize continuous integration.   Let’s start with a quick look at the first two:

• Using an SCM system to store and version all source code
• Utilizing private developer workspaces

Best Practice: Using an SCM System to Store and Version all Source Code

Parallel development and distributed software teams can make tracking changes a daunting task, especially with the frequent changes that occur when using continuous integration methods.

For this reason, it is important to employ a software configuration management (SCM) system to strictly version changes to the code base. In addition to versioning source code, everything needed to build the system should be placed under version control, including the following:

• Third-party libraries
• Properties files
• Database schema
• Test scripts
• Install scripts

All developers should have at least read-only access to all files needed for the build and should obtain all such files directly from the SCM system. This approach ensures that developers are working with the latest build environment, and is preferable to the common but error-prone practice of placing such files on a shared file server.

To effectively implement continuous integration, all development groups should work from the same central source code repository so that the latest changes from other developers are easily and immediately available.

Best Practice: Utilizing Private Developer Workspaces

In order to fully realize the benefits of continuous integration, software development organizations need to ensure that developers can remain productive regardless of the overall state and stability of the project source code. To achieve this, private workspaces that give developers full SCM capability should be used. Private workspaces enable developers to

• work in isolation
• revert to known “good” states when needed
• checkpoint their changes
• share only mature, well-tested code with other team members

The benefits of isolation are bidirectional—it protects developers from incoming changes, and protects the shared code configuration from incomplete or incorrect changes from any one developer. By creating private workspaces, developers receive all the benefits of SCM for their personal use, including the ability to revert to a previous state, viewing and tracking of changes between software configurations, and setting aside changes to begin work on a different task.

Once a new known good state is reached (for example, when a developer completes engineering and testing work on a feature), developers should checkpoint their work, typically by “checking in” or “keeping” the local changes in the SCM system. The checkpoint ensures that the developer’s work is safe on the SCM server and that the checkpoint can be revisited at any time. However, since the changes have not been shared, other developers and teams are not affected.

When a developer breaks isolation and decides to share a code change, he or she is essentially making an assertion that the change has reached a higher level of maturity. This, coupled with the use of local developer builds, helps to ensure that only mature, well-tested code is passed on to the rest of the development team, a primary benefit of continuous integration.

It occurred to me recently how much emphasis is put on the need for an Agile Project Management tool as you begin to adopt and expand the use of Agile in your development environment.  While many, including myself, would argue that using post-it notes and physical story boards are the best ways to start and Agile pilot team (since I believe it instills good habits, like circling the team around a common goal) there is certainly a place for Agile Project Management as you begin to expand Agile within a larger organization.  Agile Project Management provides visibility across the development lifecycle and into teams spanning multiple time zones, allowing for a big picture view of the project.

User Stories are the Key

Driving development through the use of User Stories is often considered a determining characteristic of whether a shop is considered Agile or not.  And while I won’t go into the details here, it is important to have a ‘place’ to manage those user stories in a backlog.   It’s also important to be able to see and report which iteration the user story is assigned to, and the current status of a particular story.  Has it been scheduled for an iteration?  Have we started development on it?  Who’s working on it? Etc…

Understanding the state and status of user stories that are in development is crucial to accurately reporting the current state of an iteration as a whole.  If you are going to use burndown charts to know how many hours of work are left for an iteration, or a burnup chart to determine how many points are left to be accepted for an iteration, you are going to need accurate information.  Accurate data comes from updating user stories and tasks on a timely basis so you always have the freshest information.

How Accurate is Your Information?

One aspect of user stories that is often overlooked is the linkage between the user stories within an active iteration and the location of the actual code changes for those particular stories.  If you only have an accurate view of the stories from a development standpoint, but you have no idea where the code for that story is, then all of the information in your project management tool contains incomplete data.

Under more traditional development methodologies, the time to ‘check’ the linkage between closed issues and code that was changed in order to complete said closed issues happens on an infrequent basis.  Checking the linkage usually occurs toward the end of a long development cycle, when teams are getting close to a release point.  If we are executing on a 6 month release cycle, it probably won’t occur until 4 months in.

Identifying the status and locating the code for user stories happens weekly on an Agile team, regardless of the iteration length.  This requires you to have quick access to accurate information to track those stories from a development perspective.

Accurate Information… Quickly

In many traditional SCM systems, developers must manually indicate the linkage between the code and the story it is associated with as the code is checked in. At the end of the iteration, teams must determine what stories are fully completed and what stories are only partially done, and need to be retargeted to the next sprint.

Traceability and visibility into each story is necessary to see where code changes are located. The ability to easily associate code modifications with a user story will eliminate error prone manual linkage problems. In AccuRev, this information is stored in a Change Package, or a deep linkage between the user story that is related to project management and code. Change Packages can be tracked through the development cycle.  This gives developers visibility into the status of an individual user story without additional overhead, while providing traceability to Scrum Masters and Product Owners into the status of an iteration with increased accuracy.

Dr. Dobb’s announced this year’s Jolt Productivity Awards, and AccuRev’s AgileCycle, the Application Lifecycle Management suite, was named a winner!  AccuRev’s AgileCycle won the Jolt Productivity Award #2 for Change and Configuration Management tools.

Jolt judge Mike Riley says “AgileCycle embodies the depth of expertise and understanding of how developers working on projects, large or small, need to effectively communicate and manage change.”

Thanks Dr. Dobb’s, and way to go team AgileCycle!

AccuRev is sponsoring a brand new Agile event with Eliassen Group, Agile Journal and Rally Software- Incorporating Agile into Your Organization- a breakfast panel discussion that focuses on maximizing the ROI of Agile methodologies throughout your company.

We invite you to join us in answering the common questions:

‘What is Agile?,’ ‘How can you help Agile permeate through your organization?‘ and ‘Where is the ROI within Agile development?‘

Date & Time:
Tuesday, December 7th, 2010
7:30AM – 10:30AM

Location:
Boston Marriott Burlington Hotel
One Burlington Mall Road
Burlington, Massachusetts 01803 USA

Cost:
$50/person registration fee – each attendee will receive a $50 Amazon gift card at the event.

Agile Event Agenda

7:30 AM   Registration and Breakfast

8:00 AM   Panel Discussion- What is Agile?

8:15 AM   Panel Discussion- How can you help Agile permeate through your organization?

8:45 AM   Networking Break

9:15  AM   Panel Discussion- Where is the ROI?

10:00 AM  Q&A

10:30 AM  Closing and Rafflle

It’s going to be a great event, we hope to see you there!

This event is over. For more events visit: http://www.accurev.com/events.html

Our Agile Comes to You seminar tour with AnthillPro and Rally is starting to wind down, with only 1 more seminar to go in this round!  (Check out our events page for more information.)   It’s been a very successful tour thus far, we have answered a lot of your questions about Agile adoption and Agile pain points, worked with some great presenters, and met a lot of Agile enthusiasts.

Check out what attendees are saying about Agile Comes to You, read a presenter’s point of view about a recent seminar, or follow #Agile2U on Twitter.

Agile Comes to You is brought to you by:



Missed the seminars?  Here are a few more events you might be interested in: http://www.accurev.com/events.html

My job here at AccuRev involves working as an “Agile Evangelist,” and along with the other Evangelists on my team, we have appropriately named ourselves “Team AgileCycle.”  Prior to our AgileCycle product launch, AccuRev took a company initiative to bring Agile into every part of the business.  The idea was to bring an educational awareness of Agile process to all of our teams by implementing basic Agile practices.  ”Team AgileCycle” was responsible for bringing Agile to the sales team, so our salespeople could have a taste of what Agile development was really all about.

(I should point out that we do realize sales organizations and development organizations are vastly different, and certain Agile practices can’t be applied to a sales cycle. But we did see great opportunities to pick up Scrum methodologies and usefully apply them to help within our sales organization.  Some of the changes we made do not qualify as not “pure” Agile, or even best practices, but the point of this exercise was to expose our team to some of the things software developers are doing in the real world.)

Implementing Agile Step 1: Sales Scrum Training

At AccuRev, we subjected our sales organization to Certified Scrum Training. In this training we walked our team through the different phases of Scrum: planning sessions, standups, and retrospectives.  We even exposed the sales team to planning poker when walking them through typical development cycle.

Implementing Agile Step 2: Implement Sales Standups

The next step was to take what we learned, and actually implement it.  At AccuRev, we now have multiple standups with our sales team, in order to obtain feedback quickly and learn what our customers are saying in the field about AgileCycle.

Implementing Agile Step 3: Mark Out Sprint and Retrospectives.

In the sales team, this is simple. Our iteration is once a quarter.  I would never suggest a development team implement this long of a sprint, but for sales it works. At the end of the sprint we got together and performed a retrospective, which discussed results for each territory, reviews of our processes, and brainstorming for the next quarter.

Implementing Agile Step 4: The Task Board

In the “Team AgileCycle headquarters,” we maintain a task board. Here we take all of our goals and tasks for the quarter, and mark them out as “backlog,” “in progress,” and “complete.”  (We’re still working on how to measure our story points, but the basic process is that we plan our backlog with our quarterly goals. When something else comes up, we fill the backlog with those tasks.)

And even though this task board seams simple, it actually wields a lot of power and has become a great tool in organizing our work.

What has surprised me the most during the whole implementation process is just how well the sales cycle seems to match specific Agile methodologies already. Think about this:

We already built in an iteration time: 1 quarter

We had planned velocity already: Sales to make this quarter

We inspected and adapted: If the numbers were not met we wanted to understand why. If we weren’t on velocity we changed course.

We had Scrum meetings before it was called “Scrum”: Weekly status and impediment meetings.

Burnup chart: Heck, the sales meter in Salesforce could even be compared to a burn up.

So after all of this, my question is:  Are sales teams “naturally” Agile because of their business? How similar is a highly functioning sales organization to a highly functioning Agile Development Organization? What do you think?

Our Agile seminar tour, “Agile Comes to You,” is back in action for the fall.  With a new line-up of cities, accomplished keynotes, our favorite partners and our favorite sponsors Agile Journal and SQE, we know this round of seminars could be our best yet.

“Agile Comes to You” Seminar Fall Line-up:

Agile Comes to You, Minneapolis Seminar

9/14 in Austin

9/15 in Dallas

9/28 in London

9/30 in Amsterdam

10/5 in Santa Clara

10/6 in Orange County

10/19 in Minneapolis

10/20 in Detroit

10/26 in New York City

If we are coming to your area, keep an eye out for registration on the AccuRev Events Page, you won’t want to miss it.  At “Agile Comes to You,” you will:

• Learn the core fundamentals of Agile development practices and what they mean to the various roles within your development team.
• Understand how you can apply software configuration management, continuous integration, static analysis, code review and quality management best practices within your Agile team.
• See how an integrated set of best-of-breed tools can help to enable quality, collaboration and visibility for development teams, their managers and executives.
• Hear examples from industry leaders of how development organizations like yours have reduced risk, boosted productivity and cut development costs with Agile.

For Twitter updates on “Agile Comes to You,” use #agile2u.

“Agile Comes to You” is brought to you by the following:

AccuRev has been down in Orlando at Agile 2010 for almost a full week now, and if you haven’t stopped by the booth to say hello to us yet, I encourage you to do so!

Damon Poole, AccuRev CTO, wrapped up a series of three Agile sessions today, and was pleased with the outcome.

“I think the sessions went really well, people were excited for them, I was excited for them, but ultimately I hope everybody learned something valuable” said Damon.

His presentations were titled “Scrum and Kanban- Like Chocolate and Peanut Butter,” “Getting Managers and Agile Teams Out of Each Others Hair,” and “Managing Growth Pains on the Way to 40 Scrum Teams.” (See “Three Days with Damon Poole on Agile Development and its Components” for more info).

Damon’s first session at Agile 2010, “Scrum and Kanban- Like Chocolate and Peanut Butter,” attracted a large crowd, and unfortunately several people were turned away.  But as he promised, we are posting all of his presentations in the AccuRev blog.  You can download Damon’s presentation of  “Scrum and KanBan- Like Chocolate and Peanut Butter” here.  Enjoy!

It seems like everyone loves continuous integration.  I’ll come out and say it- I love continuous integration! When we talk about the most widely adopted Agile practices, this one comes up the most.  Its positive benefits as a feedback mechanism provide a quantum leap forward in how development organizations think about their code.  I find it very difficult to see any downside in doing continuous integration, seriously what’s not to love?

Modern day continuous integration servers have 3 functions: Detect if a new build is needed, execute build, and notify people of the results. This is a great way of facilitating feedback to developers and allowing them to adapt and resolve problems.

But there’s something lurking in the shadows that nobody is talking about, maybe because people aren’t even aware that it’s a problem. Maybe it’s because we don’t want to ruin our love affair with CI and we’re all in denial.

What happens when those builds are done? What about the rest of “it”?

When I say “it” what I really mean is:

Most development environments include such things as complex application servers, automatic testing, release processes, compliance, audits, databases, 3^rd party libraries, build dependencies, code analysis, unit tests… and another 1,000 other things I don’t have enough space to list here. How can you deal with this? Getting feedback to my dev team is a great targeted way to let people know code is broken, but isn’t this feedback useless if you can’t get the product out the door?

If I take an example build lifecycle of an application which is:

1.)    Build Application

2.)    Test

3.)    Deploy to environments (DB) (APP SERVER) <- by hand

4.)    Test

5.)    Redeploy (DB) (APP SERVER) (PROD) <-by hand

This may seem like easy in this example, but if we took all of these steps in the real world, this could represent hundreds of servers.  And this process will have to be repeated for every version of the product.

The crux of this issue is that if any of these steps are not performed 100% correctly, it translates to real dollar$ lost for the organization. These operations have to perform like clockwork.

Taking Continuous Integration to the Next Level

This is why I believe bringing continuous integration to the next level starts with the concept that the build produced from the CI server is just the beginning. Setting up a simple CI server and producing a build is easy, but managing it through the rest of its life is the real trick.

In a real example of this, we could take

1.)    Build

2.)    Run automated tests

• If test succeeds: Deploy to to environments (DB) (APP SERVER)

• If test fails: Notify dev team

3.)    Manual Testing in QA environment

4.)    Approval process

5.)    Redeploy (DB) (APP SERVER) (PROD)

Using AgileCycle RM

In this scenario I’m taking a version of the results from a CI build, run automated tests on them, monitor the test output and wait for success or failure. If there is success then deploy all components of the application, this includes a database components and a java war component. The build will then sit in QA for manual testing until its marked approved by the QA team, managers and operations team for deployment to production.

The idea here is if we automate these processes and decisions based on build, automated tests and approval process, you can produce code quickly and at a more rapid pace. If you can produce a clockwork-like automation around your build/test/deploy related processes, your team can spend time on what’s most important: Getting code out the door.

AccuRev hosts educational Agile methodology seminars called “Agile Comes to You,” which reach audiences nationwide and focus on teaching best practices of Agile software development.  The seminars have been quite successful, and regardless of their organization’s level of Agile adoption, I know attendees have learned some great information from these sessions.

AccuRev doesn’t host the Agile methodology seminars alone, and generally presents in conjunction with partners Rally Software, Urbancode (the makers of AnthillPro), and Coverity. The seminars consist of a keynote with extensive Agile experience, educational presentations, and a short tools demonstration. The format has been so successful that we have used it for over a year, and you might even notice similarly-formatted seminars from other organizations as well. (Imitation is the most sincere form of flattery, right?)

The Agile Methodology Workshop

We try to focus on making our seminars as educational and relevant as possible by giving attendees access to the real life Agile experiences that presenters bring to the table.  So in addition to presentations focused on benefits of the Agile methodology and best practices, we came up with the concept of an “Agile Workshop.”

The Agile Workshop  allows each attendee to discuss their most difficult challenge in transitioning to Agile with other attendees in small groups, as well as with our Agile experts.  We do this for two reasons:

1) It gives the attendees a chance to exchange thoughts and solutions regarding their Agile migration.

2) It allows the attendees to interact with the panel of experts on how to solve these difficult challenges.

Once the group has discussed the challenges each individual faced during a transition to Agile, they then agree upon a top challenge that they ask the panel of Agile experts to comment on and offer advice.

For example, at a recent seminar in Toronto, this was the attendees list of top challenges:

We have been seeing this same pattern across most of our seminars, and I believe it gives us good insight into the state of Agile adoption.  It is amazing to see that even across very different organizations, the challenges that arise with Agile adoption are remarkably consistent from seminar to seminar.  It seems that no matter who you are, or what stage of Agile adoption you are in, many are facing the same challenges when moving towards Agile development. There is some comfort in numbers, knowing that you are not alone in facing hurdles.

While I won’t take the time to answer every one of these challenges here today, I plan on commenting on each one of these issues in the coming months, in hopes that sharing my experiences and alternatives help you in solving these difficult problems.  I would also like to invite some of our Agile experts, as well as our attendees, that are internal to AccuRev or our partners to comment or blog on some of these topics to share some of their experiences.

While the “Agile Comes to You” tour is taking a short break for the summer months, be sure to look for us in your city this September or stop by and visit us at Agile 2010 Conference in Orlando.  Have a great summer!