TSM - From Waterfall to Agile in SAP Landscape

Oana-Maria Bocârnea - Senior SAP ABAP Consultant @ Siemens

One of the first decisions most projects face, regardless of the development landscape or technology, is "Which development methodology should be used? What fits better for our application and landscape?" This topic gets more and more attention and it's currently a subject of debate. There are many articles debating if the developers creativity is not restricted by adopting standard steps to be followed. This comes up as since the chosen methodology determines how the project's development is organized. The two most known methodologies are Waterfall and Agile. Although SAP has ASAP for ERP implementation, in SAP software development Waterfall and Agile are both famous. The article will draw some ideas around the way of choosing the best fit for the developed software in SAP world.

SAP Landscape : let's describe the world we live in

Due to the fact that the SAP world is not so familiar among software developers, let's cover some basics. In the SAP world, a landscape is a set of servers on which the SAP software is running on. This landscape can be divided into one or more environments, which are separate, independently-operating instances of your SAP software system. Though these environments are not dependent on each other, the content can be moved from one to another using "transports". A transport contains the applications and configurations you wish to move, together with some instructions for how to pack and unpack those contents. All this might seem complicated, but it's a safety method. SAP is used to automate the functions of a business, so if any part of that automation stops working, so do the dependent business processes. To prevent this from happening, SAP recommends that you put your system through a sequence of at least three environments: Development (DEV), Quality Assurance (QAS), and Production (PRD).

These environments act like a filter, so that problems can be avoided in productive life. The first filter is the DEV environment, where the applications and configurations are first created. These objects are transported in the QAS environment for testing. In the QAS environment, thebusiness functions are simulated in order to find the existing errors before the solution gets productive. The encountered bugs are fixed in DEV system and the solutions are transported again in QAS system for retest. The third environment is the production environment (PRD.) PRD is what the customer uses and the final business solution. The described model should help to avoid big problems during applications Go-Live. Unforeseen problems that will pop up during the lifespan of the system will be fixed through the same cycle.

SAP developed the ASAP (Accelerated SAP) methodology for project implementation to fit to the described landscape. The Project implementation phases of ASAP are: Project Preparation, Blueprinting, Realization, Final Preparation, Go- Live support and operate. It is important to know that the ASAP phases generally follow each other and 70% of the tasks in each phase are dependent on completion of the tasks in the previous phase, however some tasks are carried out through all the phases. Since SAP has a big growth in cloud development they have also introduced SAP Launch methodology and SAP Active Implementation for their Cloud Portfolio. As any other landscape it can seem complicated but once you get it, you get to enjoy it.

Waterfall and Agile Approaches

**Waterfall Model

Waterfall is a sequential, non-iterative software development model. It represents the most structured implementation method, stepping through requirements, design, coding and testing, steadily downwards.

Each of these steps represent a distinct stage of the software development and each one finishes before the next one starts. This approach, as any other, offers advantages and disadvantages.

Pros:

Developers and customers agree on what will be delivered early in the development lifecycle. This makes planning and design more straightforward.
Due to the static nature and predictable scope, the progress is more easily measured.
Throughout the development effort, it's possible for various members of the team to be involved or to continue with other work, depending on the active phase of the project. For example, business analysts can learn about and document what needs to be done, while the developers are working on other projects. Testers can prepare test scripts from requirements documentation while coding is underway.
Except for reviews, approvals, status meetings, etc., a customer presence is not strictly required after the requirements phase.
Because design is completed early in the development lifecycle, this approach lends itself to projects where multiple software components must be designed (sometimes in parallel) for integration with external systems.
Finally, the software can be designed completely and more carefully, based on a more complete understanding of all software deliverables.

Cons:

Gathering requirements at the beginning is a risk. One of the most difficult parts of software development can be to understand and document the customer requirements. Customers often have difficulties to detail the specific of what they need. It is also a challenge to make the customer visualize the application from the requirements documentation. If this fails, the whole model fails, since change requests are hard to handle.
The biggest disadvantage is that waterfall is not change-friendly. The customer may not see what will be delivered until it's almost finished. Therefore, there is a possibility that the customer will be dissatisfied with the delivered software product. By that time, changes can be difficult and costly to implement.

Agile Model

Agile is an iterative, team-based approach that emphasizes the rapid delivery of an application in complete functional components.

The tasks and schedules are broken into small implements called sprints or iterations. Iterations are short time frames ('time-boxed"). Each sprint has a defined duration, with a running list of deliverables, planned at the beginning of it. If the planned work for the sprint is not completed, then it is rescheduled for the next iteration planning.

The completed work can be reviewed and evaluated by the project team and customer, through daily builds and end-of-sprint demos. Two of the most widely-used and known Agile approaches are SCRUM and KANBAN. A Scrum process is distinguished from other Agile processes by specific concepts and practices, divided into the three categories of Roles, Artifacts and Time Boxes.

Some Advantages of the Agile approach are:

Progress is measured in terms of working functions or products which can be reviewed by the customer, so immediate feedback is available
Customer ownership: by being involved in the project development as well as team work.
Changing requirements are welcome and don't make the development and delivery harder
Quality Improvement: by breaking down the project into manageable units, the project team can focus on high-quality development, testing, and collaboration.
Minimal Documentation: efforts for this are kept to a minimum, so more can be involved in development
Because each Sprint is a fixed duration, the cost is predictable and limited to the amount of work that can be performed by the team in the fixed-schedule time box.

And, there are also some disadvantages:

Agile works best when members of the development team are committed to the project.
The working relationships in an Agile project are easier to manage when the team members are proficient in their role and also if located in the same physical space, which is not always possible.
Because Agile methodologies try to enforce a close communication between team members it can lead to escalation of suppressed conflicts.

Learning by doing or "For the things we have to learn before we can do them, we learn by doing them" (Aristotle).

For a better understanding of the decisions taken in the project, in which we have experienced crossing from Waterfall to Agile, some history information is needed.

The ABAP Language (Advanced Business Application Programming, originally Allgemeiner Berichts-Aufbereitungs-Prozessor, German for "general report creation processor") was developed in 1980 as the report language for SAP. Until 1999, the language was a procedural programming language and the SAP applications were procedural developed apps (Reports, Function Groups and Transactions), which worked perfectly with the Waterfall method. In the early and mid '90 the object oriented programming developed as the dominant programming methodology, supported by the most known and powerful programming languages like C++, Objective-C, Pascal, C# and many others. So, as the market requested, SAP decided to extend the ABAP language to object orientation. This extension was released in 1999 and it was called ABAP Objects. The focus of procedural programming is to break down a programming task into a collection of variables, data structures, and subroutines, whereas in object-oriented programming it is to break down a programming task into objects that expose behavior (methods) and data (members or attributes) using interfaces. The most important distinction is that while procedural programming uses procedures to operate on data structures, object-oriented programming bundles the two together, so an "object", which is an instance of a class, operates on its "own" data structure. During the same period the software development methods evolved to alternatives for the waterfall approach. For example in 1991 the rapid application development (RAD) and in 1995 Agile-Scrum. We can say that the evolution of the programming language exposed the necessity of new methodologies.

SAP started developing new applications in an object oriented landscape using also Agile methods to organize the development, but they noticed that old applications needed also to catch up to the new trend.

We have experienced such a change in 2011. A 20 year-old application developed in old ABAP language using the Waterfall method was brought up to date while new features and extensions were also added. Because the effort estimation and the following steps were hard to document and predict, it was decided to use Agile-Scrum as the organization method for this project. A lot of preparation work was needed in order to facilitate the migration. There were many difficulties: starting from how to reuse what was already developed, how to change the mindset of people who were reluctant to the new approach, reduced costs, and so on. Another challenge was to bring teams together, since staff members were located in 3 countries: Romania, Austria and Germany.

The application migrated and extended, is a healthcare information app and, therefore, it needed to have long term maintenance. The old maintenance process did not fit to Agile-Scrum since bugs and problems could not be split into sprints. Also the effort needed for bug fixing is difficult to estimate since you first have to find the actual problem before actually fixing it. Since the maintenance tickets had to be prioritized and an evidence of their number was needed, parts of the Kanban approach were used.

Developers from Scrum teams have the following opinion about this experience: "On the new development front, Scrum was a better fit. In most cases user stories were based on ten year old functionality and in some cases continuing fresh development.

Since Scrum was new for everyone, including the Scrum Master, in the first six months we adjusted the process at every sprint retrospective and finally settled on four week long sprint and a week reserved for sprint review, planning and retrospective with stand ups on Tuesday and Thursday. We also had to learn to calibrate our commitment to our actual velocity and to calibrate the refinement of user stories so that any developer of the team could begin to work on them. In short it took the team almost a year to achieve its potential velocity but it also meant that developers specialized on parts of the project. The most helpful tool was the so called Product Canvas introduced after the first release. It was a blueprint of the next release that gave an overview of the task ahead and a target for release. After the first release our team felt the need to have a big picture for the next one and we took a page from the 'Waterfall' book and made a high level blueprint in a day or two before starting the release plan." (Peter Tako)

The project is big and still continues to grow; it contains a lot of changes and modifications and thus needs constant customer involvement. The Agile-Scrum methodology is currently still in use, although we have to admit that Waterfall advantages are sometimes missed. Luckily scrum allows tweaking the process if the team considers it helpful therefore the development methodology evolves together with the project. This approach is a great improvement over the rigid Waterfall model which greatly limits development teams in their creativity. Still Agile-Scrum needs to be improved as big projects, such as ours, proves that no methodology is perfect although it might seem so at first.