Agile Vs. Waterfall | An Intro into Development Methodologies

There are tons of opinions out there when it comes to which methodology is best. I have been working in various positions (product manager, project manager, technical sales, technical support, etc.) for over 20 years with a few different companies and have had extensive experience with each development method. Some people have a strong preference for Agile or Waterfall and expound all the benefits of their preferred method. I do not have a preferred method and I think each has its place and advantages/disadvantages. Some might say Waterfall is the old way of doing software development and Agile is a more modern way. I say that Waterfall may be older, but it is a proven method to get a predictable result.

What is Waterfall?

Waterfall is a linear or sequential development process. When using a Waterfall process, there are several phases of the project that are dependent on each other.

• A Waterfall process starts with requirements gathering and documentation. This phase can include several document types including a Marketing Requirements Document, Product Requirements Document, Functional Requirements Document, or several types of specifications documents such as a functional specification or technical specification.
• The next step is the design phase. This phase focuses on User Experience (UX) and User Interface (UI) design. The deliverables are wireframes, UI designs, and design specifications.
• Next up is development. Software engineers can go to work on building exactly what has been described thus far.
• Quality Assurance testing (or QA), comes next. This includes an Alpha release (an early mostly untested release for stakeholders to get a first look at the application) and then a Beta release (something that has been tested and is ready for feedback from the end users).
• Finally, it is time for production deployment. Typically, this means setting up servers (or cloud resources) and deploying to the web or an app store (iTunes or Play).
• Remember, software development has no true end.  There is always maintenance or the next phase of development.

The crux of the Waterfall method is that each of the above phases is very reliant upon the preceding phase. You cannot start one phase until the preceding is complete. Each phase is also very dependent on the preceding for success. If you do not have a complete requirements document, you will not get the feature you need in the UI designs. If the feature is not in the UI design the developers will not build it.

The name Waterfall refers to the importance of each step of the process. It is critical to carefully review and complete each step prior to moving the next stage. It is very difficult and can be both time consuming and costly if you must go backward during the process. For example, it may be difficult to add in functionality missing from the scope after the app is coded.

Imagine a waterfall containing several pools (requirements, design, develop, test), each overflowing into the next downstream. It would be difficult for a fish to swim upstream from the lowest pool to the top. So, if you miss something in requirements gathering it can be difficult to add if it is not caught until the QA process (Alpha Build).

What is Agile?

Agile is a term to describe a more iterative approach to software development. Contrasting Agile to the more monolithic Waterfall approach, with Agile you build an application in small, usable chunks instead of building the whole application at once. Agile projects are developed in small iterations often called sprints. You can think of these small iterations almost like mini-Waterfall projects. During an iteration, requirements are gathered, estimations developed, UI/UX design and development performed and QA tests the output.

At the end of an iteration, you have a small piece of the application that functions. Stakeholders can then test and give feedback on each piece. The feedback gathered at the end of the iteration drive product direction for future iterations.

The Agile Process

An agile process also begins with requirements gathering and planning. However, this phase is a little more flexible. The product is expected to change throughout the development cycle. It is less critical to detail all requirements and specifications in advance. The process is flexible by design. Making changes to the application as it is developed is part of the plan.

Once you have a high-level plan for the application the iterative process starts. Each iteration can last from 1 to 4 weeks until the next begins.

• Iterations start with a planning phase to decide what the iteration will contain. In this phase, the idea is to choose features/functions that can be completed together most efficiently. Each feature or function must also be estimated to understand the level of effort to complete. Based on these estimations, features are selected that can be completed in the allotted time.
• After planning the iteration starts and design, development, and testing all occur as part of the iteration.
• Next comes a review of the iteration and feedback from the stakeholders or end-users. This feedback feeds the planning process for the next iteration.
• The cycle continues in this way until the product is ready to be released. The interesting thing about an Agile process is that release milestones can be flexible. It can be determined that the product is ready for release when the stakeholders agree that enough functionality has been developed.

The main idea behind Agile methods is flexibility. In contrast to Waterfall, Agile incorporates user feedback into the development cycle. User and stakeholder feedback given during the development process helps form the application.  It is expected that additional requirements will be uncovered during the development process and these changes can be incorporated into the application. There is no penalty for making these changes as you go.

Which Development Methodology is Better?

Well, in my opinion neither is better. They are different and each has their place. Each methodology has its own strengths and weaknesses, which fit them to certain types of projects.

Benefits of Agile

Flexibility

You can make changes all throughout the process and every feature/requirement need not be set in stone from the start. Also, the release date can be more flexible. Instead of waiting for the planned “finish” date, you can release the application when it has enough functionality to be valuable.

End-User Driven

The needs of the stakeholders and end-users drive the features and functions of the application. Each iteration builds off of the feedback provided at the end of every sprint.

Transparency

During the process, all stakeholders know exactly what has been built and what remains.

Quality

Testing is continuous through each iteration of the application. Because of this, it is easier to catch bugs and increase the quality of the end product.

Motivation

Most Agile methods use self-organization and regular, short meetings the teams tend to be motivated to produce quality work in a timely fashion.

Project Size

Works well for large projects.

Benefits of Waterfall

Predictability

From the start, you know the outcome of the Waterfall process. You can constrain the project to a predictable cost and timeline.

Well-Established

Waterfall has a well-documented process, proven results, and avoidable ‘gotchas’ because it has been around for a long time.

Easily Managed

Using standard project management techniques such as milestones and dependencies, the process can be easily managed for a predictable result.

Boxed-In

Waterfall works well for projects with known and boxed-in requirements.

Efficient & Fast

The process has little overhead and can deliver a product quickly.

Project Size

Works well with small/short projects.

What is the Downside?

Both methods have their disadvantages as well. Interestingly, most of the disadvantages of one method are strengths for the other.

Disadvantages of Agile

Less Predictable

This is in terms of budget and release timeline. Because the method provides the opportunity to define the application as you build, it is difficult to constrain the project to a fixed budget. When you don’t have a fixed set of features and functions defined, it’s hard to predict when the application will be ready for release.

More Overhead

With each iteration there is planning, estimating, testing, reviewing, etc. This adds some managerial overhead to the process as the development team is shifting gears more frequently instead of being “head down” coding for a longer period. Each iteration involves more meetings. Because of this, it also does not work well for smaller projects.

Newer Methodology

There are several forms of Agile development. Some are more established and defined, others are less defined. Often a “by the numbers” SCRUM process does not work well (especially for remote teams). Your organization would need to modify the process to meet your requirements.

Difficult to Manage

Standard project management techniques do not work as well in Agile. It takes a strong understanding of the process by the project manager to successfully manage. It also demands a strong product owner and can be problematic if you have several stakeholders acting as a product owner.

Disadvantages of Waterfall

Lack of Flexibility

Waterfall locks you into the product requirements from the beginning. It becomes costly to make changes to the application after designs are complete.

Lack of Transparency (During Dev)

When a waterfall project goes into the development phase it is a bit of a black box. Once the application is in development, stakeholders do not really see the application until the first alpha release.

Perceived Quality 

The Alpha phase is typically full of bugs. This is not really a problem because the QA stage will find and resolve most issues before a production release. But when the stakeholders first get to see the product it can appear to be of poor quality because of the lack of testing.

Not for Large Projects

It does not work well for applications that are not well defined or boxed in. This can be difficult, especially with larger projects.

As you can see the benefits and detriments of each method can help make it clear which types of projects are best suited for each method. There is one more important consideration when it comes to starting a new development project, that is the type of development agreement you would like.

Because of the way each of these development methodologies works, when you choose a method you will also need to consider the type of development agreement you would like as well.

Fixed Cost or Time & Materials?

When entering an agreement with a development organization you will have two primary types of agreements. A fixed cost agreement is based on a scope of work. The scope specifically states exactly what will be built and a cost (not to exceed) for the development. A time and materials (or hourly) agreement will also typically provide a scope of work, but the cost will be an estimate or an estimated range. Both agreement types are suitable for software development projects and have their own set of advantages and disadvantages. A fixed cost agreement works well if you are going to use a Waterfall development methodology, however, it probably won’t work for an Agile project.

It’s all about risk. When a development organization estimates any software project there is an element of risk. Estimation is a bit of an art and there is no way to guarantee that an estimate for a project will be exact. A well-defined project, fixed set of requirements (Waterfall), and few integrations with other applications reduces risk. If you still have to determine the requirements for a project, or they will be determined during the project (Agile), you increase the risk.

Additionally, when an application requires integration with a third-party application via an API or some other method (ETL for example), risk increases. This is because the development team doesn’t have complete control over the API or secondary application.

Who Assumes The Risk?

With an hourly or time and materials agreement, you assume this risk. If the project takes longer, the developer bills you for the time spent on additional features or troubleshooting API issues. With a fixed cost agreement, the developer assumes all the risk, if the project goes over the estimated cost, they will have to absorb the cost. For this reason, development organizations will often build this risk into their estimated cost when proposing a fixed cost project. For a fixed cost agreement, the cost will most likely be higher than a pure time and materials agreement for the same scope.

Fixed-Cost = Developer Assumes Risk

If a project is well defined and easily boxed in, then it is a good candidate for a fixed cost agreement (and Waterfall). The advantage of a fixed cost agreement is that you are guaranteed to stay within budget. The downside to fixed cost is that you may end up paying a little more because the developer is assuming the risk. Fixed cost projects are ridged in their structure. They lend themselves to a waterfall style development process as opposed to a more flexible Agile approach. You also must understand that because a fixed cost project is reliant upon the scope of work, any changes to the scope (additional features or changes in functionality) will need to be addressed via the dreaded CHANGE ORDER.

Hourly = You Assume Risk

Conversely, hourly or time and materials agreements are more flexible, but you must be a little more flexible with your budget as well. With time and materials, you have the option to use a more agile development methodology and which provides more freedom to make changes to the application (add features you might not have considered or change functionality) during the development process. The scope of work does not bind you as tightly.  Because of this, it is often difficult to constrain the budget. It is easy to get carried away adding cool features.

It is possible with self-control to complete a project at a reduced cost compared to a fixed cost project. The idea is to stick to a minimum viable product and not gold plate your application. You will only be billed for the work performed as you are assuming the risk (versus a fixed-cost project, where the project cost reflects the risk).

Fixed + Agile Does Not Play Well

You can do a fixed cost agreement for an agile project, but the likelihood of hitting your budget is slim. During this type of engagement, you can expect that their will be several change orders along the way which will increase the cost of the project. Additionally, these types of projects can foster a very contentious relationship between the development team and the stakeholders. Each change will require a change order. Because of this, the stakeholders may feel like they are getting “nickel and dimed” for every little change. Fixed cost just does not play well with Agile.

Which is Right for You?

In the end, both development methodologies can work well, and each type of development agreement has its advantages. It really depends on your project and your priorities which will be best for you.

A fixed-cost waterfall project works well with smaller, easily box-in projects. However, a time and materials agreement may save you money if you are willing to take some risk.

For larger projects, a time and materials agreement for an Agile project may be the best way to go. Especially if the requirements are not fully formed or are difficult to constrain.

These are just a couple of examples on each end of the spectrum. Chances are your project may fall somewhere within this range. In the end, it all comes down to what is most important for you and how much risk you are willing to assume.