VUCE: The platform that enhanced quality and achieved a five-fold increase in delivery speed
Uruguay’s Single Window for Foreign Trade (VUCE, for its acronym in Spanish) is a platform connected to 91% of government and other agencies in Uruguay. It is used from Argentina, Brazil, Chile, Paraguay, Russia, Mexico, Vietnam, and Israel.
VUCE is accelerating progress in the region by promoting best practices and cross-border interoperability to achieve paperless foreign trade. The technology platform currently implements 186 different processes from 29 agencies, dealing with 400,000 procedures per year. It has eliminated the use of paper, as well as unnecessary transfers and waiting times, while reducing costs and increasing competitiveness. This article describes the changes that allowed VUCE to increase the quality and frequency of releases of the platform used by 54,000 users.
Uruguay's VUCE platform came about in 2010 as an initiative of the National Customs Department’s modernization process and the Inter-Ministerial Commission for Foreign Trade Affairs (CIA-CEX). It aims at promoting the use of information technologies with an eye on process optimization and stakeholder cooperation.
After an international public bidding process, the system started to be implemented in 2013 by Concepto, a supplier of the National Customs Department, and GeneXus Consulting. The VUCE team was partially integrated into the platform development process from the beginning and by 2018 it was fully in charge of the development process. Manual testing of each new release used to take 120 hours in the week before release, plus 40 hours of release stabilization during the following week. Users who found errors in their processes during this last phase would have a negative experience. In turn, the problem negatively impacted the support and development teams because the development of fixes and new requirements would be stopped. Then VUCE identified the need to optimize the testing process and improve the platform's quality by means of automated testing with GXtest v4.
«Our main problem was that when checking the system we had to reallocate collaborators from different teams, and there wasn’t enough time to do regression tests. Without this type of testing, we could not guarantee the application's quality and would be unsure about the releases, which overburdened the support, process, and development teams.»
Juan Juncal, Technology Manager - VUCE
Main drivers of change at VUCE to bring a five-fold reduction in delivery times and improve quality for its 54,000 users
«GXtest led us to make changes in methodology that enabled us to adopt a DevOps culture. We had to redefine processes in all work teams to make automation sustainable over time. We realized that the problem was not the people or the tools, but how people managed the tasks.»
Juan Juncal, Technology Manager - VUCE
Impact on quality during implementation
VUCE defined the regression test suite according to high business risk, complex, and/or time-consuming cases. The speed of implementation of the 82 flows was increased by generating GeneXus code obtained with the GXtest Recorder extension. In particular, VUCE flows had an average of 30 screens each, and included multi-user sessions, electronic signatures, and multiple file uploads on the same screen. The following graph shows the 88% of automated flows implemented in 440 hours, flows in progress, and flows pending automation for each system module.
During the implementation phase, the system’s response time and usability were improved; in particular, the start time of procedures was reduced from 50 seconds to less than 10 seconds.
Better quality of test execution in VUCE
VUCE ran the UI test suite on Chrome and Firefox in a test environment (production mirror). In each release, the test suite was manually triggered from the GeneXus IDE for a 15-hour overnight run to reduce waiting times. After each execution, 20 hours were devoted to the analysis of results, review with development team, test maintenance, and documentation. This led to an 86% reduction in testing hours. From the fourth version onwards, the return on investment began to be noticeable. As a result, an average of 2 bugs per version were identified in the test environment, preventing them from reaching end users. In addition, production bug reporting per release was significantly reduced from 8 to 1 bug. The following table shows the average cost in hours and bugs reported in production per version, as well as the estimated delivery time, before and after automation.
«Automation allows us to run tests more than once and in more than one environment per sprint, without assigned collaborators. Testing helps us to make sure that infrastructure changes do not affect any functionality.»
Juan Juncal, Technology Manager - VUCE
The new methodology had a major impact on the development team, allowing them to release 5 times faster and more securely, without introducing known bugs in the platform or changes that affect the most critical system flows. It also had a positive impact on the support team as end users reported fewer bugs and were given more accurate delivery time estimates for new requirements and/or fixes.
VUCE's technology team currently aims to achieve full coverage of system UI testing. In the short term, VUCE will implement a continuous integration scheme, where changes will be integrated, code will be compiled, and tests will periodically run unattended. In the longer term, VUCE will incorporate automated unit tests from the development stage, to ensure that no known or critical bugs are introduced in the logical layer of the application.
«I like to measure improvements in delivery quality by the number of bugs that make it to production; with automation, the number of bugs decreased considerably. Our plan going forward is to achieve a continuous delivery scheme with a fully automated development cycle. Our vision is to implement testing as early as possible in the development process.»
Juan Juncal, Technology Manager - VUCE
Uruguay's VUCE platform came about in 2010 as an initiative of the National Customs Department’s modernization process and the Inter-Ministerial Commission for Foreign Trade Affairs (CIA-CEX). It aims at promoting the use of information technologies with an eye on process optimization and stakeholder cooperation.
After an international public bidding process, the system started to be implemented in 2013 by Concepto, a supplier of the National Customs Department, and GeneXus Consulting. The VUCE team was partially integrated into the platform development process from the beginning and by 2018 it was fully in charge of the development process. Manual testing of each new release used to take 120 hours in the week before release, plus 40 hours of release stabilization during the following week. Users who found errors in their processes during this last phase would have a negative experience. In turn, the problem negatively impacted the support and development teams because the development of fixes and new requirements would be stopped. Then VUCE identified the need to optimize the testing process and improve the platform's quality by means of automated testing with GXtest v4.
«Our main problem was that when checking the system we had to reallocate collaborators from different teams, and there wasn’t enough time to do regression tests. Without this type of testing, we could not guarantee the application's quality and would be unsure about the releases, which overburdened the support, process, and development teams.»
Juan Juncal, Technology Manager - VUCE
Main drivers of change at VUCE to bring a five-fold reduction in delivery times and improve quality for its 54,000 users
- Implementation of automated testing:
VUCE automated a set of 82 UI flows with GXtest v4 in GeneXus 16, in a separate KB from that of the application as the platform was being migrated to the same version. Tests were implemented using GXtest Recorder, a Chrome extension that records test cases and generates GeneXus code. Among the main strengths of the toolkit chosen for automation, the following stood out:
- Simplicity and power of integration and management.
- Test objects in the same KB of the application.
- Methodology:
The execution of automated regression tests implied a methodological change. VUCE was trained in agile methodologies and, in particular, defined its methodology based on
- Daily and weekly coordination meetings.
- Visibility of the work defined.
- Execution of automated tests in each version.
- Fixed release dates.
«GXtest led us to make changes in methodology that enabled us to adopt a DevOps culture. We had to redefine processes in all work teams to make automation sustainable over time. We realized that the problem was not the people or the tools, but how people managed the tasks.»
Juan Juncal, Technology Manager - VUCE
Impact on quality during implementation
VUCE defined the regression test suite according to high business risk, complex, and/or time-consuming cases. The speed of implementation of the 82 flows was increased by generating GeneXus code obtained with the GXtest Recorder extension. In particular, VUCE flows had an average of 30 screens each, and included multi-user sessions, electronic signatures, and multiple file uploads on the same screen. The following graph shows the 88% of automated flows implemented in 440 hours, flows in progress, and flows pending automation for each system module.
During the implementation phase, the system’s response time and usability were improved; in particular, the start time of procedures was reduced from 50 seconds to less than 10 seconds.
Better quality of test execution in VUCE
VUCE ran the UI test suite on Chrome and Firefox in a test environment (production mirror). In each release, the test suite was manually triggered from the GeneXus IDE for a 15-hour overnight run to reduce waiting times. After each execution, 20 hours were devoted to the analysis of results, review with development team, test maintenance, and documentation. This led to an 86% reduction in testing hours. From the fourth version onwards, the return on investment began to be noticeable. As a result, an average of 2 bugs per version were identified in the test environment, preventing them from reaching end users. In addition, production bug reporting per release was significantly reduced from 8 to 1 bug. The following table shows the average cost in hours and bugs reported in production per version, as well as the estimated delivery time, before and after automation.
«Automation allows us to run tests more than once and in more than one environment per sprint, without assigned collaborators. Testing helps us to make sure that infrastructure changes do not affect any functionality.»
Juan Juncal, Technology Manager - VUCE
The new methodology had a major impact on the development team, allowing them to release 5 times faster and more securely, without introducing known bugs in the platform or changes that affect the most critical system flows. It also had a positive impact on the support team as end users reported fewer bugs and were given more accurate delivery time estimates for new requirements and/or fixes.
VUCE's technology team currently aims to achieve full coverage of system UI testing. In the short term, VUCE will implement a continuous integration scheme, where changes will be integrated, code will be compiled, and tests will periodically run unattended. In the longer term, VUCE will incorporate automated unit tests from the development stage, to ensure that no known or critical bugs are introduced in the logical layer of the application.
«I like to measure improvements in delivery quality by the number of bugs that make it to production; with automation, the number of bugs decreased considerably. Our plan going forward is to achieve a continuous delivery scheme with a fully automated development cycle. Our vision is to implement testing as early as possible in the development process.»
Juan Juncal, Technology Manager - VUCE