Problems with agile practices

The following are some practices used on agile projects and the problems that can arise.  Click on each link for more information.

  1. Frequent changes to the requirements are welcome
  2. No up front design
  3. Short iterations
  4. Developers work in pairs
  5. Developers move around to different areas of the system
  6. Collective team ownership of the code
  7. The unit test is written first
  8. All acceptance tests must be run before checking code into the repository
  9. Incremental integration

The following links may be of interest to developers or managers considering using extreme programming or another agile methodology:

The case against extreme programming by Matt Stevens

Extreme programming considered harmful for reliable software development 2.0 by Gerold Keefer

A report released in July, 2012 calls “agile” a scam designed to sell consulting services, certification, training, and books.  In summary, they found:

  • Out of over 200 survey participants, we received only four detailed comments describing success with Agile.
  • Overwhelmingly, 40% of participants that use Agile did not identify a benefit.
  • Survey participants report that developers use the guise of Agile to avoid planning and to avoid creating documentation required for future maintenance.
  • Be aware that the Agile movement might very well just be either a developer rebellion against unwanted tasks and schedules or just an opportunity to sell Agile services including certification and training.
  • We received some unprecedented scathing and shocking comments about the level of competence, professionalism, and attitudes of some members of the Agile movement.

In summary, not all agile methods use all of the practices described above.  But, the main practices that defines a project as agile are short iterations and incremental deliveries.  The problem with these practices is that it turns the project into a maintenance project early on and requires one or more developers to revisit the code multiple times as each new feature is implemented.  This is very inefficient because there is too much overhead.

There is a better way to run a software project with Efficient Software Development Methodology Practices.


Prev          Next          Back to efficient software development home

Copyright 2011 by Robert G. Bryan

2 Responses to Problems with agile practices

  1. Jim Johnstone says:

    I get your point Robert, but I have a different set of experiences. While you claim welcoming requirements changes is a problem with agile practices, I would counter with two points. 1) I welcome more complete and validated understanding of the requirements that have always been there but may have been inaccurately or incompletely documented. 2) I welcome an understanding of a change in market demands as soon as that change can be made known to me. The longer either of these two truths are kept from me, the more code I write under an incorrect understanding of the requirements.

    That is to say I do not want to spend $100.00 per hour multiplied by 8+ hours per day for six months or more multiplied by10-20 developers just to find out either I have misunderstood the requirements, or that I built the exact thing that was requested but the market demands have now changed. In either case, I have a substantial refactoring effort. The efficiency I gained by not welcoming the changes in requirements suddenly is just an illusion. What would have cost me a few hours early in the project now may cost a few weeks (depending on the nature of the requirements change).

    These concerns are real and are, in my opinion, addressed by an iterative (not incremental) development process. Many years ago, a gentleman named Grady Booch wrote of these problems and suggested an iterative approach with each iteration beginning with a statement of its goal, and ending in a test of the work, proving or disproving whether or not the goal of the iteration was met. If not, one or more new iterations are spawned based on the new information gathered. Using an iterative development strategy allows filtering out requirements thrash (aka the good idea fairy) while allowing valuable real world changes to the requirements to come through as quickly as possible.

    Bottom line, I want to minimize the amount of time I have my head buried in the sand thinking everything is fine when in fact it is not.

  2. Bob Bryan says:

    Jim – thank you for your comment. You have put your finger on the crux of the problem. When you say:

    “I welcome more complete and validated understanding of the requirements that have always been there but may have been inaccurately or incompletely documented.”

    So, how come they were inaccurately or incompletely documented? How come the requirements can not, for the most part, be accurately analyzed and documented correctly? Who is responsible for this? Is it the customer’s fault? Is it the fault of the development team, or perhaps the methodology of the development team? In truth, it is a combination of all the above. Sometimes, the customer is not very sophisticated and does not have a good understanding of what his true needs are. This is where business analysts, managers, and some developers can play a major role by researching and analyzing his requirements and then advising how to proceed.

    It is only when developers start designing that they begin to discover all of the implied or hidden little requirements. These hidden requirements can explode to between 50 to 100 times the original general requirements. One of the marks of a great developer is that he can see hidden requirements and all of the implications for a requirement. This usually involves identifying how the system should handle key pieces of data. This is probably one of the most underestimated and overlooked skills when hiring a developer. Understanding how strong your developers are at finding and evaluating the implied requirements can be critical to scheduling and manpower estimates.

    Under ESDM, the development team is not sticking its proverbial head in the sand during any stage of development. During the requirements analysis portion of each phase, the customer should be heavily involved with the requirements, features, and specifications. One of the key insights of agile methodology is that frequent interaction with the customer compared to more traditional projects in the past leads to better software. ESDM does not dispute that – at least up to the point of design. However, when it comes to implementation or even design, the customer is often not sophisticated enough and/or has no interest to be involved.

    In ESDM, more requirements analysis and design work is done up front for each phase than in most agile projects. This means that it is far more likely that more of the true and hidden requirements can be discovered early. This saves time and is actually more efficient, not less. Developers always know where they are going and how their components work with the rest of the system. Unlike some agile projects, they do not start off with a random feature card and then just start coding with no up front design or any idea of how their component will interface with the rest of the system. What this means is that developers don’t have to spend nearly as much time re-designing and re-factoring as they do on many agile projects.

    You said “I do not want to spend $100.00 per hour multiplied by 8+ hours per day for six months or more multiplied by10-20 developers just to find out either I have misunderstood the requirements, or that I built the exact thing that was requested but the market demands have now changed.”

    There are some things that can be done to make sure this does not happen:

    1. Create a throw-away user-interface prototype that shows how the system will work from the point of view of the user.
    2. Create activity, sequence, component, and use case diagrams that show how major parts of the system will work and interact before the user signs off on the specification document.
    3. Give the customer plenty of demos of key functionality as they are completed in order to show progress and to determine if the software meets the customer’s needs. During these demos, the customer will usually point out any missing or extra desired functionality; you should certainly ask if the functionality is what he is looking for and if there is anything else he would like to see the software do if he does not say anything. Inviting the customer’s key users for the demo is an important part of determining if the software meets their needs.
    4. ESDM is recommended for medium to very large projects, but it can also be used for smaller projects. The delivery phase length should be negotiated with the customer. The minimum recommended time period is 3 months, but I believe that most large projects would be better off with at least 6-month phases. The idea behind this is to reduce overhead and over-impacting the customer with functionality that can not really be used until most of the other components have been written.

    In ESDM, the specification document is a very important document. For the development team to proceed, the customer must sign off on that document, which is why the customer needs to be very heavily involved in its creation. The specification document describes the software to be delivered and how it works. The high level design or architecture for that phase should also be completed before the specification document is written. The schedule should provide some padding so that unexpected and/or unforeseen problems can be handled, including “do-able” requirements modifications or additions during the early part of the phase. At some point though, any requested requirements or features not in the specification document will have to be locked out of that phase and implemented in a future phase. This approach is not uncommon with most agile projects, since they will move a requirement into a future iteration when there is not enough time to implement it in the current iteration. But, in ESDM if the requirement is critical, then the customer can also choose to slip the schedule so that it can be implemented for that phase.

    When beginning a phase, the development team’s business analysts, managers, team leads, and even developers should be researching the customer’s business and look at all of the software products used by or written by the customer’s competitors. This should help reduce the possibility of changes in market demands. But, if after signing off on the specification document, the customer comes back at some point and says that the market demands have changed and the software needs to significantly change to accommodate this, then there should be a clause in the contract that provides for this possibility. There should be an early termination clause (and perhaps fee) that the customer can exercise and it should also specify that if an agreement on the new requirements can not be reached, then either party may elect to terminate the contract. Regardless of the methodology being used on the project – new requirements and changes in market demands will always need to be addressed.

    In conclusion, ESDM can be just as responsive to the customer as many agile projects. But, because more work is done up front with requirements analysis and design, it results in less missing requirements, less requirement changes, less redesigning, less re-factoring, less revisiting of the code, and better code written the first time out. Other recommended practices like design and code reviews help improve the quality of the code and reduces the chances of failing to implement requirements. ESDM also emphasizes strong testing practices starting with line-by-line testing, unit tests, and interface tests; acceptance tests are only run by the developer when wholesale changes are made or to debug a problem. All of this increases the chances of the project succeeding, increases productivity, and thus reduces both risks and costs.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s