Lead a Cross-Organization Test Automation Project Fearlessly!

Last year, Imperva initiated a cross-organizational software project focused on creating a new management console and I had the privilege to lead its test automation effort. In this blog, I will explain the challenges we faced as technical leaders in executing the project and reveal the solutions we developed to bring this test automation project to successful completion.

Getting started

Since Imperva’s mission is to protect customers’ data and all paths to it, we initiated this project because we believe it is critical to offer our complete suite of cybersecurity services via one unified management console. The successful completion of this project allows Imperva to provide industry-leading cybersecurity technology and to deliver significant UX and SW architecture benefits to our customers.

The overall goals of the project were to:

• Create a Unified Management Console offering Imperva’s entire cloud services portfolio via enhanced UX
• Revamp the UI for part of our cybersecurity services
• Gain a rich library of services for authentication, authorization, API Security, UI improvements, and more

Project constraints and challenges

Of course, a strategic test automation project that has a cross-impact on product development has constraints. As a test automation leader, I needed to maximize test coverage and gain the most early feedback while avoiding disruptions to other projects.

A large-scale software project that changes both existing architecture and UX brings technological challenges. Here are some of the notable challenges we faced during the project development lifecycle:

• UX changes “broke” our E2E UI tests that were coded to test the former management console which decreased our test coverage of our complete suite of cybersecurity services we offer
• We had to test the new software architecture and new UX first as single, stand-alone systems and then later with the whole system
• We had to execute the project in harmony along with other projects, ensuring that we took other project commitments and ad hoc obligations into consideration
• We needed to involve test engineering teams in the project plan so we could cut their learning curve and optimize their efforts invested in the project
• We had to manage ongoing changes in project content and dates and contending with delays in project milestones
• We had to ensure efficient project execution during the work environment changes and uncertainty brought on by the COVID-19 pandemic

Technical project considerations

We determined that allowing test engineers to fix their own tests was not an efficient use of project resources. This approach can result in diminished quality and commitments overall. We also discovered that the new UX we developed changed “only” the navigation to our cloud services. We needed to devise a “hack” to the UX to navigate to the “correct” menu item or URL. Once we discovered the “hack”, we were able to get our test coverage and test engineers back on track and start making good project decisions. These decisions included involving test engineers early in the project to produce effective communication, developing applicative infra to “save” existing E2E UI tests and keeping the same test coverage, sharing a cross-organizational test automation project plan, and setting clear roles and responsibilities.

Good decisions translate into good solutions

Once we had overcome the initial challenges and determined the most appropriate plan of action, we started to build out the solutions that would make the project successful. These included:

• Saving E2E UI tests. Once we developed an applicative infra that allowed our existing E2E UI tests to adapt new UX changes, we saved them. Also, we developed tools and training material to make sample tests creation and analysis easier and we set specific score results as rollout criteria.
• A productive testing plan. We designed tests to verify our new system. We set milestones based on selected test samples and our growing coverage capacity. We enabled developers to conduct a gradual rollout in a test environment and made the new UX testable via Selenium.
• Creating smoke tests for new build sanity. In addition, we also increased test regression coverage to secure more quality feedback.

Collaboration and project tracking

From the outset, we shared our high-level plan with all R&D level managers and asked that they assign a technical test engineer as a point person with whom we would work on an ongoing basis. We also ensured that the test engineers were “Left shifted”, meaning they were involved in the project at the design phase so they could provide feedback early on in the process. And once the project development path was clear, we designed and conducted training and update sessions throughout. Along the way we were able to follow up on project progress, get feedback, raise red flags, suggest solutions and update the project plan as needed.

Four take-home lessons

While I have described a specific test automation software development project in this post, there are lessons for everyone who needs to optimize the way they run projects. Here are four things you should consider and account for the next time you build a test automation project plan:

1. Get feedback early and avoid disrupting other projects. This can be accomplished more effectively when you have a “point person” from other stakeholder groups with whom you can communicate to avoid creeping in on other projects.
2. Establish the best way forward first, then execute the project. Seems simple enough, but not planning for basic challenges and technical considerations early on will likely affect the quality of your project-related decisions.
3. Roll out gradually. Empower your team to roll out projects in a thoughtful way so they can address issues in a smaller, less critical environment. The time you spend will likely save you from bigger headaches later.
4. Collaborate and communicate. Get as much feedback and input as possible from stakeholders as early in the project as you can. Ensure that your project is not a “black box” and that everyone who wants and needs to know is in the loop and can help keep it on track.

In the end, we successfully created a Unified Management Console offering Imperva’s entire cloud services portfolio via enhanced UX. Good luck with your next project!