Continuous Integration & Automated Release Management with AWS

Our client came with the challenge

As an innovator in the fields of mapping and navigation, our client runs a chain of research and development centers that continuously develop its product portfolio and viable market-ready prototypes. Daily, tens of thousands of tests, code integrations, and builds are performed in their AWS continuous integration solutions and continuous delivery (CI/CD) pipeline.

In 2017, after two years of collaboration, BitsByteSoft took ownership of a large share of the product portfolio pertaining to our client’s map production and automotive solutions. At that time, our DevOps team responsible for CI/CD monitoring, support, and automation was still working at the client’s site in Germany. After six months, our BitsByteSoft teams started experiencing frequent obstacles due to delays in incident response and communication. Given the sheer number of commits, tests, and builds, our client’s team was physically unable to promptly respond to emergencies.

Another challenge lay in reconfiguring the execution of tens of thousands of automated tests that could contain numerous implicit and explicit dependencies and were run multiple times a day. To avoid large buildups of failed tests, they had to be executed early in the software development lifecycle.

BitsByteSoft developed the solution

Since BitsByteSoft was performing all related development, testing, and release planning activities, our client decided to also build a mature DevOps team with BitsByteSoft . In view of the large volume of tests to handle, we hired four highly skilled DevOps practitioners with proven experience in provisioning, automating, and managing complex end-to-end deployment pipelines on Amazon Web Services.

The DevOps team set off as an extension of our client’s team. We started by analyzing the CI/CD environment and identified key directions for improvements and getting benefits of continuous integration. Initially, we implemented the shift left approach to allow error-prone tests to fail as early as possible by configuring their execution sequence. This practice freed up more time to fix defects, facilitated debugging, shortened test cycles, and decreased the number of unpredictable issues at the end of the development cycle and in production.

Delays in incident response was another issue that required our attention. Our team redesigned the test execution schedule to enable more test runs during the day. This technique helped prevent large queues of failed tests from piling up throughout the night when no one was monitoring them.

Additionally, we enhanced the cooperation among BitsByteSoft and our client’s project teams by optimizing the feedback system and reorganizing the recipient list of affected parties. Developers and other stakeholders were sent timely reports on test execution metrics, deployment failures, and code integration issues. Optimizing feedback helped speed up issue detection and code modifications.

We’ve achieved great results together

Since the start of our collaboration, our BitsByteSoft DevOps team has been progressively enhancing the efficiency and performance of our client’s cloud-based deployment environment and automated release management. We’ve not only extended and re-engineered the execution of 40,000+ automated tests but also ramped up to 2.5 million test runs per day. Building deployment-ready artifacts that have undergone rigorous testing procedures have become faster, less costly, and hassle-free. This enhances continuous integration release management and increases release velocity and will eventually improve business results thanks to cloud-based continuous integration.

We’ve completely replicated and complemented the expertise of our client’s DevOps team, taking over their responsibilities for automated release management and AWS continuous integration and web development. Apart from managing our client’s deployment pipeline, our team also monitors and maintains the health of a proprietary testing framework that validates the integrity and compatibility of NDS maps prior to release.

BitsByteSoft’ team is now shifting focus toward a newer, more challenging project – a cloud-based collaborative location ecosystem. Our team will grow as we expect the workload to at least double from what we’re handling now.

Our continuous integration services have translated into:

• Lower test execution costs
• Faster build generation and code releases to production
• Fewer bottlenecks stalling project teams
• Fewer incidents during execution of multiple dependent tests
• Faster incident response
• Prompt and meaningful feedback on incidents and related risks for target audiences