Why am I talking about this, right now? Because I've downloaded from Amazon and read the book titled
“Why Agile is Failing at Large Companies,
” by
Ms. Geri Schneider Winters. My first thought was: “this is another book that tries to demonstrate that things cannot be done, that changes are not possible.” Cautiously and respectfully, I reminded myself that even though reading those who think like us can be pleasant, it may probably be useless; on the other hand, reading those who think differently is always useful.
Reading this book was a gratifying experience because it looks deeply and seriously into the subject. Instead of emphasizing the impossibility (as I had assumed after reading the title) of applying the
Agile paradigm, it encourages readers to apply it in a responsible, sound and measured way, taking into account all the necessary elements for success. But I won't talk about this any longer here: I strongly recommend reading the book!
Waterfall Paradigm
The Waterfall paradigm implies a monolithic, deep and extensive analysis of the problem to solve, as well as its complete resolution at the project level, to then move on to its implementation. Does it work? It focuses on predictability (costs in time and money) and, ideally, everything works well as long as the original project is realistic and doesn't contain any serious errors. Also, this reality must not change significantly during the project.
This paradigm was designed for different circumstances, where both the systems' size, and the speed of changes, were much smaller than today.
In the early 80's, when our services began to be increasingly required for the implementation of large corporate systems, we were soon faced with major challenges. Even though we eventually managed to overcome these difficulties, we became more and more convinced that the paradigm we were using wasn't the right one.
In my case, in 1984, a large Brazilian company requested a very large corporate system based on a single central database around which everything had to be built. After an initial analysis, we realized that this database would involve at least 500 entities, and it was clear to us that the traditional
Waterfall paradigm would lead us to failure. It was also clear that there wasn't an alternative paradigm available, so we started to do research on the subject, looking for new solutions.
Agile Paradigm
Sooner or later, many people faced similar problems. And, most likely, we all asked ourselves the same questions: with the
Waterfall paradigm we give priority to cost predictability but, what happens with product quality? How do the systems obtained meet actual needs? How can we keep them valid and up to date over time? Is it reasonable to believe that costs (time and money) will be smaller than with the
Agile paradigm? In fact, they tend to be much higher!
The
Agile paradigm promotes incremental development. What characteristics does it have? Continuous interaction, user involvement, timely prototyping
. With an enough high abstraction level, all this can be conceptually summarized as
“feedback.
”
What fundamental problems do we face when using the
Agile Paradigm? Above all, unexpected situations arise as we move forward. Changing or adding processes to handle the same data can be easily done. However, this is not the case when we need to make structural changes to the database: the degree of independence between data and programs provided by DBMS is very limited. For this reason, major changes to data structures require a set of complex operations, for instance, to determine which programs will continue to be correct and which ones will have to be changed. As a result, this brings significant costs in time, money and risks of errors.
Does the same happen with the
Waterfall paradigm? Of course, but the
Waterfall paradigm has been designed for different situations, where little or no changes are expected.
Need for structural changes to the database
Must we inevitably face this problem? Isn't there a way to solve it automatically?
Many attempts have been made: for example, in the end of the 70's, ANSI-SPARC issued a recommendation oriented at facilitating high independence between data and programs. At that time, Cincom Systems launched a DBMS (SUPRA) that reasonably followed this recommendation. Later on, Microsoft's Framework.net included a mechanism (Datasets / Dataviews) that had the same objective. For various reasons, these initiatives were not successful.
In 1985, as a result of our own research, we arrived at the conclusion that the best way to solve the problem was to increase the abstraction level in order to work with pure knowledge.
Over time, we have achieved outstanding automatic management of the knowledge of business systems, and its by-products include the possibility for us to automate the following:
- The project, generation, maintenance and reorganization of the database.
- The project, generation and maintenance of the programs.
All this has been incorporated into our
GeneXus product, which is widely used in more than 45 countries, mainly for mission-critical systems in a wide range of companies.
IN SUMMARY:
The
Waterfall paradigm has become obsolete, but still exists because it is used by almost all legacy systems.
Today, it doesn't make sense to undertake the development of new systems under the
Waterfall paradigm.
On the other hand, the
Agile paradigm offers clear advantages for new applications.
The use of technology that provides significant independence between the data and the programs, in order to make true incremental development possible, is a great help.
* Breogán Gonda is a Computer Engineer, Researcher, Professor, Consultant, and Chairman of the Board at GeneXus, a company that specializes in building software tools based on the automatic processing of knowledge. bgv@genexus.com; http://www.genexus.com.