Posts Tagged ‘continuous integration’

SCM Best Practices and Continuous Integration Go Hand-in-Hand

June 15th, 2011 by AccuRev

There’s no denying that this has certainly been the Agile decade for the software development industry.  It’s evident all around us in this tenth year since the Agile Manifesto was created. Most companies and development organizations today have implemented some form or aspect of Agile methodology into their software development processes. Whether you’re aiming for pure Agile or a mixed/hybrid approach, proven best practices in all phases of the software development lifecycle are crucial to success.

This is especially true in the case of continuous integration, one of the foundational aspects of the Agile methodology. The concept of continuous integration, as defined by Martin Fowler, is “a fully automated and reproducible build, including testing, that runs many times a day.  This allows each developer to integrate daily, thus reducing integration problems.”

With this approach, developers can work more closely in parallel while identify problems and debugging on the fly, accelerating the development process and improving the quality of the finished product.  The benefits of continuous integration are tremendous, but can quickly be eradicated if software configuration management (SCM) best practices are not carefully followed.

There are a handful of SCM best practices that can optimize continuous integration.   Let’s start with a quick look at the first two:

  • Using an SCM system to store and version all source code
  • Utilizing private developer workspaces

Best Practice: Using an SCM System to Store and Version all Source Code

Parallel development and distributed software teams can make tracking changes a daunting task, especially with the frequent changes that occur when using continuous integration methods.

For this reason, it is important to employ a software configuration management (SCM) system to strictly version changes to the code base. In addition to versioning source code, everything needed to build the system should be placed under version control, including the following:

  • Third-party libraries
  • Properties files
  • Database schema
  • Test scripts
  • Install scripts

All developers should have at least read-only access to all files needed for the build and should obtain all such files directly from the SCM system. This approach ensures that developers are working with the latest build environment, and is preferable to the common but error-prone practice of placing such files on a shared file server.

To effectively implement continuous integration, all development groups should work from the same central source code repository so that the latest changes from other developers are easily and immediately available.

Best Practice: Utilizing Private Developer Workspaces

In order to fully realize the benefits of continuous integration, software development organizations need to ensure that developers can remain productive regardless of the overall state and stability of the project source code. To achieve this, private workspaces that give developers full SCM capability should be used. Private workspaces enable developers to

  • work in isolation
  • revert to known “good” states when needed
  • checkpoint their changes
  • share only mature, well-tested code with other team members

The benefits of isolation are bidirectional—it protects developers from incoming changes, and protects the shared code configuration from incomplete or incorrect changes from any one developer. By creating private workspaces, developers receive all the benefits of SCM for their personal use, including the ability to revert to a previous state, viewing and tracking of changes between software configurations, and setting aside changes to begin work on a different task.

Once a new known good state is reached (for example, when a developer completes engineering and testing work on a feature), developers should checkpoint their work, typically by “checking in” or “keeping” the local changes in the SCM system. The checkpoint ensures that the developer’s work is safe on the SCM server and that the checkpoint can be revisited at any time. However, since the changes have not been shared, other developers and teams are not affected.

When a developer breaks isolation and decides to share a code change, he or she is essentially making an assertion that the change has reached a higher level of maturity. This, coupled with the use of local developer builds, helps to ensure that only mature, well-tested code is passed on to the rest of the development team, a primary benefit of continuous integration.

More Agile Methods and Practices Defined

August 30th, 2010 by damonpoole

In the last few posts I have discussed some of, what I consider to be, the most valuable Agile methods for development.  The list is pretty long, so breaking the list up allows me to define each practice and include the individual benefit of each Agile method.  This post defines some hot terms right now- continuous integration, multistage continuous integration, and story points.  Enjoy!

Agile Method: Continuous Integration

How frequently have you merged your code with changes from the mainline, only to find that the result doesn’t build, or it builds but it doesn’t work? Monthly? Weekly? Daily? Hourly? Or worse, how often have you made changes that broke the build, requiring you to quickly correct the problem while getting flames from your team members?

A practice that has emerged to address the problems of integration is called Continuous Integration. The basic idea is that if integrating changes together and getting build and test results on a regular basis is a good idea, integrating and getting build and test results on every change is even better.

With Continuous Integration, all work from all teams is integrated into a single codeline as frequently as possible. Every check-in automatically triggers a build and usually a subsequent run of the test suite. This provides instant feedback about problems to all interested parties and helps to keep the code base free of build and test failures. It also reduces the integration headaches just prior to release.

Agile Method: Multi-stage Continuous Integration

Integration is tough enough when you are just integrating your work with the work of other folks in your small team, or the whole effort is being done by a small team, but when you are part of a large team there is also something called “Big-Bang” integration. That’s the integration of the work that multiple teams have been working on for long periods of time. In a typical project, this integration is done in a phase toward the end of the project. During integration, many problems are discovered for the first time which leads to delays and/or changes in scope.

The real question is, what is a good way to structure this integration so that it will scale smoothly as you add more people to the equation? A good starting place is to look around for a pattern to follow. What are some similar situations? I have found that everything your organization needs to do in order to produce the best possible development organization can be entirely derived from the patterns and practices at the individual level. This approach makes it much easier to understand and much more likely that it will be successfully followed.

As individuals we work in transient isolation to reduce the impact of work in progress on each other. Organizations isolate WIP by using only official versions of 3pty sources and by producing official releases for customers.

Multi-stage continuous integration (MSCI) scales CI to large distributed environments by isolating work in progress at the team level. Changes move from individual to team to mainline as fast as CI allows, but stop on failure. MSCI is particularly important in a distributed environment where fixes to problems exposed by CI can be delayed by a full day

Agile Method: Using Story Points For Estimation, Instead of Units of Time

In my experience, the best unit to use for estimates is story points. Two different people with two different skill sets or levels of ability in an area may take different amounts of time to perform a particular task. Estimating in hours mixes together the scope of the work that needs to be done with the speed at which a particular individual can do that work.

On the other hand, story points are a relative measure of the scope of a user story. Story points separates out the “what” from the “who.” For instance, if you have one individual that is stronger with .Net than with Java, they will estimate a Java story as taking more hours than somebody that is stronger with Java. But they will probably both agree that something that is twice as easy to implement will take half as long to do.

To use story points, you need to create a relative scale of scope. A simple approach is to find a simple and straightforward story that you use to represent a single story point. Then think of stories that are 2, 3, 5, and 8 times larger in scope. You should have a couple of examples for each story point value to take into account that some stories have more test than coding, more documentation than test, etc.

Story points are primarily used for planning, not for implementation. Story points are used to help determine the contents of release by calculating a velocity.

Next up: Backlog, velocity, planning poker and burnup charts.

Three Days with Damon Poole on Agile Development and its Components

August 3rd, 2010 by kenglert

Agile 2010 is fast approaching, and AccuRev is excited to sponsor andDamon headshot Three Days with Damon Poole on Agile Development and its Components support the Agile community involved with this conference.  However, our excitement is partly due to the fact that Damon Poole, AccuRev’s CTO, was chosen as a featured speaker for not one, not two, but three sessions during the conference week.  I sat down with Damon to chat about his Agile 2010 plans, the ideas behind his session topics, and ultimately, the unrelated topic of my recent exposure to Star Wars.

Star Wars aside, Damon has some great talks about Agile development and components planned for Agile 2010.

Damon’s Discussions on Agile Development and its Various Components

Damon’s first Agile 2010 session will take place on Tuesday, August 10th at 1:30 PM and is titledScrum and Kanban- Like Chocolate and Peanut Butter.”  Here Damon proves Scrum and Kanban are not in fact mutually exclusive, but play well together, much like chocolate and peanut butter.

KE: “So Damon, how did you come up with the chocolate and peanut butter concept?”

DAMON: “I have witnessed infighting within the Agile community, between Scrum advocates and Kanban advocates.  Agile is a way of thinking, and the community wins when there is a synergy between camps.  I want people to recognize that not only can Kanban and Scrum co-exist, they can actually be a very beneficial combination to development teams.”

KE: “Who might this session be best suited for?”

DAMON: “I would say this session is for folks that are already doing Scrum and are curious about Kanban.  I will address Kanban basics, how Kanban can help with real-world process problems, how to apply one-piece-flow to Scrum, and the value of work-in-progress limits applied to Scrum.”

KE: “Since it’s about chocolate and peanut butter, will there be Reese’s?”

DAMON: “You bet.”

__________________________________________________________________________________________________________

The following afternoon, Wednesday, August 11th at 1:30 PM Damon will present “Getting Managers and Agile Teams Out of Each Other’s Hair.”

KE: “This sounds like an interesting Agile pain point that lots of us can relate to.  How do you approach this topic?”

DAMON: “Well, one of the most talked about, but least understood components of Agile is the ‘self- organizing team.’ There is little research published on this concept and I spent a lot of time looking outside of software development for information and advice on self-managing teams.  I came up with new perspective on this concept by examining external roots of the practice.  What it is, what the benefits are, how it works.  I will share my advice on manager roles and responsibilities, aspects of self-organization enabled by multiple Agile development components, and challenges that teams face.  It should be a good session, I have given it before and it’s always well-received.”

_______________________________________________________________________________________________________________________________________________________________________________________________________

The third day with Damon is on Thursday, August 12th at 9:00 AM. He will present “Managing Growth Pains on the Way to 40 Scrum Teams

KE: “Forty Scrum teams is getting up there.  How would you recommend managing such large amounts of teams?”

DAMON: “You’re right, 40 Scrum teams is the sign of a large organization.  I have interacted with lots of large Agile shops that operate with many more than 40 Scrum teams, and noticed issues with Agile weren’t all that different from shops with smaller teams.  By recognizing trends and patterns sooner than later as the organization becomes deeper involved with Agile, teams can start following certain practices to eliminate issues.  When it comes to addressing issues, the sooner the better.

Agile 2010 Badge Template.jpeg Three Days with Damon Poole on Agile Development and its Components KE: “What best practices have you recommended to larger-sized Scrum teams in the past?”

DAMON: “Multi-stage Continuous Integration, small story size, collocation, cross-functional teams… a few more.  This is a good session even if your organization doesn’t have 40-something Scrum teams today.  It teaches you about growing pains and prepares you for future growth.

Well, Damon sure sounds like he has a busy week lined up at Agile 2010.  Make sure to check out his sessions- they are featured on the Agile 2010 schedule and under the “What’s Hot” tab in the Agile 2010 app for iphone and Droid, so don’t forget to add Damon’s sessions to your schedule via these apps!

Follow AccuRev on Twitter @accurev for Damon’s latest updates from Agile 2010!