Chris has cajoled me into posting this, its been in draft for ages. I've been trying to work out if this is relevant and useful to the Lotus community, or am I the only one interested in this stuff.
I've been learning as much as I can about Agile and, in particular, Scrum. I'm a certified project manager (based on PMBok), so I was interested to understand the implications of managing software projects in an Agile way. Did I need to throw away all I knew about the PMBok. How do you manage the same things that the stakeholders need to know about risks, schedule and budget but in the Agile way.
I am also interested to find out how others tackle software development in the context of the Lotus platform. It doesn't seem to be a topic that is talked about much within our community.
There is a rising ground swell of interest and adoption of Agile software development, which is gaining credibility. Agile is now a widely accepted and valid method for managing software development. IBM Lotus Software Group and the Lotus Notes Development Team are using Agile and the fourth edition of the PMBok guide is being influenced by agile projects.
Should the Lotus community be interested ?
Is Agile (and Scrum) something that, as professional software developers, we should be interested in ? or interested enough to find out more ? We take toolkits and ideas from other technology areas and reuse them for our own purpose, so why not a development method ?
I don't need Agile, I'm using a RAD tool.
Lotus Notes has always been on the agile side of the spectrum and is often referred to a rapid application development (RAD) environment. Of course, just because you are using a RAD tool does not necessarily mean that you are developing software in a RAD way. In the same way that you can still write Java code in a procedural way and shun any form of Object Oriented techniques. I've seen the waterfall, big document up front, approach for Notes development and you end up spending more time on the document than actually writing the software.
For me developing solutions in Notes has been somewhat Agile. I started with Joint Application Design (JAD) and then Accelerate Value Method (AVM). AVM, for those who have not heard of it, was the Lotus consulting methodology from 1995 which embraced a collaborative development style with stakeholders and team. The aim was also to deliver value to the customer at the earliest opportunity through short iterations.
In fact if you did business with Lotus Consulting in Australia you would have come across a project flexibility matrix in our statements of work. I saw the same matrix recently while reading 'The Software Project Managers bridge to Agility'. We even used a documentation formatting methodology that favoured brevity and efficiency in communications.
So why should I care if I'm using Agile or Scrum ?
Simply, it gives you a common language that you can talk to other developers, clients and managers when they ask what is it that you are doing. You can point to a wealth of credible experience online and in books and say, yes we are using a software development methodology. A methodology favoured by highly respected experts, and by the way, they have produced high quality software that the users want quicker. Make no mistake, there are pit falls and traps which you will need to look out for. You will also need to assess if your team and organistion is ready and willing to try Agile.
There are also lot if misconceptions around Agile methods that have the potential to strike fear into managers and the PMO alike. "No documentation", "you'll get it when you get it" and "Agile doesn't scale" are some of the common ones. If you take the time to understand, like I have recently, you will realise that Agile has structure for dealing with the processes and disciplines that the PMBok describe. The PMBok is very general and, as the title describes, and only provides a guide to manage projects from building bridges to software development. Agile fills in the 'how' for software development and so the two can exist happily together.
I really would like to use Agile, how can I explain the benefits between Agile and Waterfall in a simple way ?
I recently attended the Sydney Scrum User Group and we worked through an exercise, which was an analogy for one of the difference between a waterfall approach and a scrum/agile approach. Rather than pinch the idea, here is my contrived story to explain the difference. Think of sandwiches in terms of software features, and a tray as a release or increment, and it might all make sense - then again maybe not.
Scenario 1. Making sandwiches the waterfall way.
Requirements.
Customer sends a fax at 9:00 am in the morning he needs vegemite sandwiches for 100 people for an event at lunch. Deliver to the office at 11:30.
Development.
You get a production line started, one person buttering and spreading the vegemite, the next person slicing and arranging on a platter. By 10:30 you've spread 75 sandwiches with vegemite, when the customer calls in to check on progress.
The customer suddenly remembers that, not everyone likes vegemite. So ask for half to be roast beef. Rather than 25 vegemite sandwiches end up in the bin the client agree to increase his order by 25 sandwiches to get the 50 roast beef sandwiches.
User Acceptance.
You turn up at the customers office, he looks at the platter and it looks a little plain then remembers that he has some vegetarians turning up who don't like vegemite! You quickly call the guys at the cafe and get them to quickly make another 25 sandwiches, this time cucumber.
Sign Off.
The customer is happy, but it cost him 150 sandwiches and you've expended 150 sandwich making effort. There is only 100 people so there is a lot of sandwiches left over that are not used.
Scenario 2. Making sandwiches the agile way.
Requirements.
Sometime later the same customer faxes another order, this time you try a different approach.
You invite him to come to the cafe and tell him he can use the wi-fi and he'll get a coffee on the house while he waits. You sit down with him and ask your usual catering questions. How many people ? are there any vegetarians ? The barista overhears - and suggests that to cater for vegans and wheat intolerance and through your many years of experience you estimate that 75 sandwiches should be enough for 100 people.
Development.
You start making sandwiches one at a time. As each tray of sandwiches is made you show the customer. You have made 50 sandwiches (two trays). When you overhear the customer exclaim - bugger!
He's just got an email. The meeting has been rescheduled until the afternoon. You suggest to him that for an afternoon meeting cakes might be more appropriate and so decide that 50 sandwiches and 25 cakes would be plenty. You stop making any more sandwiches and the and start slicing cake.
User Acceptance
The customer has seen the first tray of sandwiches as they have been made. He quickly checks the cakes as they are being arrange - all is looking good.
Sign Off. The customer is happy and pays. You suggest that he takes one tray of sandwiches to put in the fridge on the off chance that some of the attendees didn't get the rescheduled meeting notice. After all they are already made - he might as well take them.
Let me explain the difference
Scenario 1 - the customer didn't really understand what he wanted until he started thinking more about his needs when he saw the entire finished platter. He ended up with more food than he actually needed and there was a certain amount of waste. The sandwiches were all produced in great big block so there is little flexibility in changing the fillings during the making of the platter.
Scenario 2 - the customer also did not understand what he wanted, but as he was available in the coffee shop he could actively participate and give the sandwich makes feedback as his requirements changed, like when the meeting was rescheduled. There was little to no waste in the construction and as the platter was made in smaller, fully complete, chunks. In fact, so complete that the customer could actually take part of the order with him and realise that value earlier.
Summary.
You can see for the examples that when the customer is involved more closely with the production of the sandwiches that the whole team is more able to quickly adapt to the changing situation. Have smaller, fully complete, increments allowed for the customer to start using the end product quicker. The customer only paid for what he actually needed.
The trade off.
The customer/product owner has to commit to spend time working with the team. If the product owner, the person with ultimate responsibility to steer the results of the software development effort, doesn't have the time or inclination to work in an agile way then waterfall my still be the most appropriate method for your situation.
There is a lot more to agile than one post. Hopefully this post has given you a small insight and taster into Agile. Let me leave you with one last thought.
Image you have two project, one waterfall and one agile. Imagine a time of economic uncertainty. Imagine both projects are canceled half way through. Which project will be able deliver value to the customer in the form of working software.
Answers on a postcard...(or comments)
Hi Tony,
ReplyDeleteGreat post!
Thanks
JYR
Tony, I like your scenario's to explain the differences, perhaps a little too simple though. I haven't read alot about agile, so bear with me. It sounds like, what you have described connects strongly with requirements management techniques,JAD and incremental delivery. I can see that Agile formalises this process plus other stuff. I use agile inadvertently on a regular basis for small projects, and when budgets are tight.
ReplyDeleteAgile depends on people, and over the long term with complex systems, I think you end up in a poorer strategic position. I think Agile needs to be used with caution, and only in circumstances where it's benefits outweigh the risks, or you can manage the risks.
Agile gives you the quick hit now, but you will (potentially) pay a hefty price later. I have seen a few large projects using agile, and all ending in a situation that produces a functioning system, but is extremely costly to enhance and maintain.
Perhaps what I see is that Agile does not help eliminate the weaknesses within people doing the work, so agile gets abused quite a bit. This is usually in the form of 3 significant risks which are over-looked in most cases.
Knowledge attrition, system robustness, and cognitive ability (ie people forget over time). Typically, I have seen a bunch people race off and do stuff, then the "sandwich maker" finds another job, and leaves the kitchen in a mess, the next sandwich maker spends ages looking for the butter, and is churning through all the previously made sandwiches to work out how to make more sandwiches. In the meantime, the customer has asked you to make some new types of sandwiches, only that you had made those same sandwiches a few months ago and forgot and it was a disaster.
As you can see this is getting expensive, and you can't see how you got to "here".
Other conditions also work against you for agile development. For instance, Agile is most productive in a supportive environment, politics quickly destroy this and you can end up with alot of finger pointing. Another problem is if a project drags on for more than a few months you will most likely have people re-cycling ideas, and repeating work without seeing that they are going around in circles and wasting effort.
So, I am not saying Agile shouldn't be used, but as a PM, you need to consider when to use it, and be aware of the risks and manage it.
Giulio
Giulio,
ReplyDeleteThanks for the length comments. You are right, the example is fairly simple. I did want to highlight two of the differences between waterfall and agile. Small complete increments and customer participation. These are some of the differences which do appear in the start of a project - so as you say, this is more like requirements. However, this is not the whole agile story.
There is nothing in agile that negatively impacts, knowledge attrition, robustness or forgetting. There is nothing in Agile that says you should not focus on keeping the information (referenced or tacit) in the team - in fact if the team is using high bandwidth communication (face-to-face) a staff member leaving is less likely to impact the overall team. In any case, succession planning should be part of the PM process regardless of agile or waterfall. Robustness. Again agiles focus isn't just on speed, but also quality. To me that also means robustness. The team should be doing code reviews, unit tests and adhering to standards regardless of Agile or Waterfall.
You are right, Agile does not eliminate the weakness in the people on the team. In fact weaknesses in the team and other part of the software development process are amplified in agile(like missing code reviews). The good part is that it's likely that you'll discover those early and then as a team in the retrospective decided how best to proceed - with everyone committed to improvements.
You are also right, I wouldn't recommend attempting an Agile transformation without senior approval (the finger point could be messy for your career) - but then again, any transformation initiative should be treated the same - with senior approval.
Another of the differences is the concept of 'definition of done'. In Agile everyone agrees what that means, so that when you mark something as complete (done), it could mean that the story is working functionally, user documentation completed, unit test running with no errors, design document/wiki updated and code reviews completed. If the team decides that the 'definition of done' doesn't include any code reviews, then fine - the team and product owner is responsible for that and the outcome associated with that.
There are failures in Agile but there is also big increases in productivity too - and there are reasons for both.
Finally, having a small budget shouldn't be the deciding factor whether to do agile or waterfall. Agile isn't an excuse to skip the things to reduce the cost, such as documentation, code reviews and the like.
It sounds like there is still a few gaps in your understanding of agile. I would encourage you to get involved with the local user groups or delve into one of the many good books.
Once again thanks for the comments - keep watching I might just post about some more of the differences.
Tony,
ReplyDeleteThanks for the response. It makes sense that budget should not dictate the use of agile. Again, I might put that down to techies, and sponsors alike abusing the concepts and mis-construing "need to save money" with employing agile concepts and missing the whole point. One observation you pointed out that I over-looked, was how to manage the definition of deliverables. You covered it in your response as empowering responsibility with the relevant parties.
Giulio