Archive for July, 2008

What Can Developers Learn from their Surgeon?

July 30th, 2008 by jsherwood

I recently read a story that the World Health Organization (W.H.O.) issued a new safety checklist for surgical teams. Checklists sound great and frankly it was kind of surprising they weren’t already in practice. Here at our company we go through a number of checklists, and I can only assume you do as well. There are testing checklists to determine what test coverage exists, process checklists when products are about to ship, and integration points themselves could be considered a checklist of activities that need to occur for success.

What I found particularly interesting were the topics covered in the checklist. There are fundamental items like marking the location of the surgery, but I found the following to be the most topical:

*    Confirm that all team members have introduced themselves both by name and by their role on the surgical team.

*   The surgeon, anesthesia professional, and nurse should verbally confirm the patient’s identity, surgical site, and procedure to be performed.

*    Anticipated critical events to be reviewed by the surgeon are any critical or unexpected steps, estimated operative duration, and anticipated blood loss.

Keep it High-Level

Blood loss aside, what I find interesting about these items is how high level they are. You have team members participating as a team, identifying the areas of focus and activities. Staff executing risk analysis. Providing estimates. This is great.

SCRUM

We’ve been using Scrum. This means we have a 15 minute meeting once a day to go over our activities. We talk about what we did yesterday, what we are doing now, how long it should take and any risks to success. Sound familiar?

SCRUM HURDLES to HURDLE

Although Agile has been going well here for a while, there may be a number of initial hurdles to watch out for if you make this kind of change. Having to coordinate a day and time for all team members, then trying to keep the meeting to 15 minutes can be a challenge. It’s far too easy for development staff to lapse into traditional practices. When we first went to Scrum some of us would begin to discuss specific issues during the meeting (a Scrum no-no), and people would obviously start tuning out. This then translates into not knowing the more high-level team goals and activities.

Our first attempt to fix this, having everyone send a daily status email, at first just went to the team lead. The team lead knew what was going on, but it still didn’t solve the larger goal of having people be aware of their role and its impact on the team. We then tried to fix this, by group consensus, by everyone emailing the group with their daily status. This had the advantage of getting the format of the data consistent amongst the team, and getting information to everybody in the team. But it doesn’t get people to read the email. I now have a filter that automatically drops these into a directory that makes them easier to find, open, and read, or scan depending on my level of caffeine.

NOT LOSING SIGHT OF THE GOAL

So what I see as part of the goal, as well as what the W.H.O. was trying to achieve could be lost. Without the physical meeting everyone remains caught up in their own particular goals.

These personal goals are necessary, but they are not always conducive to team productivity.

Here we use Continuous Integration. We’ve been progressively converting more of our attended tests to unattended, increasing our product coverage and productivity. At the same time, staff members have been changing the test infrastructure to allow us to test varying configurations. Both are great goals, but in the beginning they ended up in conflict. One engineer was making more configuration settings required while another staffer was trying to make more of the configuration settings dynamic for portablity. The newly unattended tests were hanging or exiting early because configuration items unexpectedly became required.

Without a common goal, or at the least an understanding of everyones goals we run into conflict.

What else can be done?

When developers are used to being siloed, and their work feels isolated, they are more likely to tune out as to what is being done around them. But integration has to happen. And it has to work. Integration should no longer be thought of as somebody else’s problem, or as something that happens after the fact. Yes I was one of the people who didn’t like yet another meeting. I was one of the people who sometimes drifted into the mundane details of my work that I thought others would want to know, eating up meeting time. Now I champion the 15 minute meeting and keep people quick and to the point. And if everyone can start feeling like integration is the goal, then people will start thinking about how their work impacts others. Sharing that information in a timely manner will smooth out the bumps, and ultimately get us more focused on the larger goals. This isn’t “brain surgery”…or is it??

How We Integrated our Offshore Development Team

July 18th, 2008 by AccuRev

by Tahir Hussein, Alterian

This diagram shows one key aspect of our development process i.e. 3rd Line Support. It is critical to our operations because we support anywhere from the last 3 to 5 release steams. This includes monthly patch releases which address key customer defects and enhancements.

3rd Line Support

3rd Line Support

Prior to 2006 our development operation was UK based (in a single location). Everything was straight forward! However, when we opened up a development centre in Bangalore, that situation changed. Once we had recruited and trained the developers it was time to get them using our SCM tool, Serena Dimensions. This has served us well since 2000 but the performance of the connection to Bangalore meant that the PC and Web clients were totally unusable for shared development work e.g.

* It was taking over 5 minutes for them to browse change documents

* Editing the attributes e.g. to specify the fix times etc was taking around 10 minutes

* Making any kind of source code changes on a change document, especially one with a lot of additional related items or change documents was even longer.

* To fetch all 9000 items from the repository for a sandbox development and build was an overnight job (if it worked at all!)

Hence we had to put our thinking caps on and come up with another solution.

We tried making use of Subversion and associated tools and initially this looked very promising. However, when we started putting our Development processes on top of Subversion, in particular “Branching and Merging”, it became apparent that the underlying SVN functionality was not going to be mature enough for the tools to be available to cater for this. The solution we were looking at was Subversion running in UK and Bangalore, with the replication of data taken care of by a product called WANdisco and the Application Lifecycle management by Polarion. We had a number of meetings and discussions with all of the above parties, including the Subversion developers, who gave us an insight into the future plans. In the end the overall solution was not going to be elegant and satisfy all our needs. We carried on looking and eventually came across AccuRev.

At first we were reticent to go down this route as it was a proprietary system. However, when we were demoed the system by the AccuRev guys it seemed to fit in exactly with what we were looking for. Even with the addition of merge tracking in SVN, AccuRev is still head and shoulders above with its merge, change and namespace tracking.

We then obtained a demo license and spent a month evaluating the product against our list of requirements. The sort of checks we carried out were:

* How long does it take to create something e.g. stream, workspace in the UK and see it reflected in Bangalore and vice versa

* Can you create/delete hundreds or steams without affecting the system performance

* The critical one of can a number of developers on the UK and Bangalore work together on a set of changes in parallel and easily have those merged back together.

The whole thing ran so smoothly that we could no longer delay the decision to purchase. Last year around this time we purchased the required licences and then started the process of moving the development over to AccuRev.

So, where are we currently?

The intention has been to move over to using AccuRev and Jira combination (as Serena Dimensions covers both Issue Tracking and Source Control). However, as we have a limited number of resources available, and the fact that we require quite a lot of work to implement the appropriate workflow and associated build process, we have not had the time to do this. We are currently using AccuRev for ALL development and 3rd Line support work but still use Dimensions for the Issue tracking and build process. Given that none of the developers in Bangalore or UK are impacted by this and can get on with their day-day work then I am cool with this. The only person affected is me as I have an additional step to perform by moving changes between AccuRev and Dimensions and vice-versa. I am confident that in the near future we will fully move onto the AccuRev/JIRA system and gain even more benefits.

I hope this will be of help to others and I would be more than happy to answer specific queries.

Why Traceability Matters

July 16th, 2008 by matthew d. laudato

In a recent blog post on CodeSqueeze (http://www.codesqueeze.com/the-blame-game-how-necessary-is-traceability/) the subject of traceability and its utility in software project management was discussed. This post is a response to the author’s points on traceability. While I agree with the author on some points (especially that traceability can be used proactively), I believe that traceability has much greater utility in the development process and in the management of software projects.

Traceability, both at the code and issue levels, provides you with fundamental process metrics. By removing these metrics, you lose objective insight into the state of the project. Using such metrics for a blame game may indeed be a sign of poor project management, but not having the metrics at your disposal is a sign that you are not controlling the process, and that puts your project at risk. I can’t imagine other engineering disciplines such as chemical engineering even having this conversation – you measure (almost) everything that can be measured, and then determine which metrics are most useful in improving the process. The best engineers on your team will welcome objective measurements into their code – like having the fastest time at the racetrack, doing well by a set a metrics that are not easily manipulated is something to be proud of, and something for more junior engineers to aspire to.

As for feature traceability, some of the comments on the blog point in the right direction - for example, that in regulated industries, such traceability is a requirement. ‘We all trust each other’ doesn’t fly well with the FDA or DOD. But even beyond the regulated industries, traceability and related metrics answer questions that are too basic to leave unanswered:

- what requirement did this code satisfy?

- what ancillary issues (bugs, related requirements) is this code associated with?

- how long did it take compared to the initial estimate?

- how often since the fix was marked as ‘complete’ did the code change?

I could (and just might) write another blog post on why these metrics are interesting and essential to software process management, but for now I leave it to commenters to elaborate.

To the specific points that the author makes, lets look at them in turn. The author’s orginal points and counterpoints are in italics.

  • Provides visibility of per person velocity – Visibility can occur during daily stand ups and weekly powerdowns.Sure it can, but that implies among other things that you are doing such standups. What if you are using an alternate process to coordinate the efforts of large, widely distributed teams? At some level, independent of your specific process, you need to be able to look into your SCM system and answer questions of who is doing what and why. Also, (and this touches on the second issue of accountability) not all developers communicate in the same way, so even if you are doing standups, as a manager you need objective facts to correlate with the verbal and written communications of your team.
  • Creates a sense of accountability – Why the hell do you have people on your team that you don’t trust?  Trust doesn’t happen all at once. The basic principle of managing people, whether they are shipping clerks or software engineers is ‘trust but verify’. Once a person has established a track record of reliable communications that are consistent with the objective (and hopefully, automated) metrics, you might reduce the frequency of how often you check them against the metrics. But periodic checking is essential – if a previously reliable engineer has for some reason fallen off the tracks, they may be reluctant to admit it, or they may not even be aware of it. A properly constructed metric that is trending in the wrong direction will give you clue that there is an issue to be addressed with this engineer.
  • Allows for possible learning opportunities – Either the bug is a bug (and no real lesson is to be learned), or senior developers did not properly guide less experienced devs through a design problem. All bugs teach you something. The latter case that the author mentions is certainly one lesson, but it implies that senior developers don’t write buggy code or are somehow immune to design problems (or design-to-implementation translation problems) of their own. This seems unlikely. Most of the best engineers that I’ve worked with will admit to some whopper bugs – many written well after they had earned the title of ‘senior engineer’ or ‘chief cook and bottlewasher’, or whatever. What a bug tells you is that you have code that needs attention, independent of whether it dropped out of a build process or was discovered by the janitor. The overhead for recording a bug, commenting on it and connecting it to the source code change that resolves it is minimal in a well-designed and well-tooled development process, so for a small amount of effort you get the benefit of a window into your code, its failings, and your process. Seems like a low price to pay for such potentially valuable information.

I’ll admit that traceability and metrics are on my mind often (see for example, my prior post on the AccuRev-Rally integration and how it assists in agile requirements traceability). Maybe it’s because my favorite toys as a kid were ruler, compass and slide rule and I just like measuring things for the heck of it. Or maybe it’s because I’ve seen too many engineering projects fail – projects that if the manager (myself included) had a bit less hubris and a few more objective facts at their disposal, could have been squarely in the success column.