This article is based on a press announcement originally reported by softwaretestingnews.co.uk in October 2017.
The SAP Regression Testing Problem
SAP environments must continuously evolve to keep pace with business requirements. But every update — whether a configuration change, a patch, or a major cloud migration — carries risk. Regression testing exists precisely to catch the unintended consequences of those changes before they reach production. The problem is that traditional regression testing in SAP is slow, expensive, and often delivers incomplete coverage.
Manual regression testing demands significant human effort: end-user recording sessions, business process discovery workshops, and the ongoing maintenance of test scripts. Automated testing tools have historically offered only partial relief, typically focusing on user interface (UI) interactions and still requiring considerable setup time. The result is that many organisations face an uncomfortable trade-off between test coverage and delivery speed — a trade-off that becomes acute during high-stakes projects like S/4HANA migrations or cloud re-platforming.
According to research cited by Basis Technologies at the time of this announcement, 66% of organisations using SAP software expected their regression testing burden to increase in the short term, and were actively seeking a cost-efficient solution.
Introducing Testimony and Robotic Test Automation (RTA)
Basis Technologies responded to this challenge with Testimony, a product built around a concept the company calls Robotic Test Automation (RTA). Rather than requiring manual test script creation or end-user recording sessions, Testimony automatically learns what needs to be tested and how — by observing real system usage.
The core RTA workflow works as follows:
- Automatic learning: Testimony monitors live SAP usage to understand actual business processes and execution paths, without requiring manual input from testers or end users.
- Automatic configuration: Based on what it has learned, the tool builds and configures a test suite that reflects real-world system behaviour.
- Automatic execution: Tests are run without manual intervention, validating new releases against the captured baseline.
- Automatic updates: As the system evolves, the test suite updates itself, reducing the maintenance burden that plagues traditional automated testing tools.
The practical outcome is that system-wide regression testing can reportedly be initiated within days rather than weeks or months, with no manual scripting phase required upfront.
Going Beyond the UI Layer
One of the most significant technical differentiators Basis Technologies highlighted is that Testimony does not limit its monitoring to the user interface. Many SAP regression test tools operate solely at the UI layer — an “outside-in” approach that misses what happens inside the system when a transaction is executed.
Testimony monitors SAP at both the UI layer and at the code-and-system level, capturing a broader range of execution paths and technical interactions, including:
- Remote Function Calls (RFCs)
- BAPIs (Business Application Programming Interfaces)
- Other internal technical interactions that UI-only tools would not observe
This deeper instrumentation means the resulting test plan covers significantly more of the system’s actual behaviour, reducing the risk of regressions that slip through a UI-only safety net.
Service Virtualisation for Integration Testing
Modern SAP landscapes rarely exist in isolation. They integrate with external systems — other SAP instances, third-party applications, and cloud services. Testing business processes that span these integrations has traditionally been difficult: either you require all connected systems to be available in the test environment (expensive and logistically complex), or you rely on incomplete stubs and estimated data (inaccurate and risky).
Testimony incorporates service virtualisation to address this challenge. It automatically isolates the SAP system under test from its external dependencies, capturing representative inputs and outputs from real interactions. During regression testing, these virtualised services stand in for the real external systems, allowing full end-to-end business process testing without requiring every connected system to be available or replicated in the test environment.
Enabling Shift-Left Testing and DevOps for SAP
A recurring theme in modern software delivery is shift-left testing — moving testing activities earlier in the development pipeline so that defects are caught closer to their point of introduction, where they are cheaper and faster to fix. For SAP environments, this has historically been difficult because of the overhead associated with test preparation.
By dramatically reducing the time and manual effort required to set up and maintain a regression test suite, Testimony is positioned to make shift-left testing practical for SAP. This aligns directly with DevOps principles: faster feedback loops, more frequent releases, and greater confidence in the quality of each deployment.
Early Adopter Perspective
Gavin Thornton, IT Senior Application SME – SAP at CBH Group, was among the first users of Testimony. He offered the following perspective at the time of the announcement:
“Testimony really is revolutionary. We were one of the first to use the product and the output immediately challenged what we thought we knew about the impact of change in our systems. It has already fundamentally altered the way we think about testing SAP software. We’ve evaluated automated testing tools before but until now none of them have successfully addressed our basic challenges. Testimony is different. It gives us a way to massively increase the speed and effectiveness of testing, so that we can deliver the changes that the business needs with much greater confidence.”
— Gavin Thornton, IT Senior Application SME – SAP, CBH Group
Key Capabilities at a Glance
| Capability | Traditional SAP Testing | Testimony / RTA |
|---|---|---|
| Test configuration | Manual scripting required | Automatic, learned from live usage |
| Test maintenance | High ongoing effort | Automatic updates |
| Coverage scope | Typically UI layer only | UI + code/system level (RFCs, BAPIs) |
| Integration testing | Requires real or manually stubbed systems | Service virtualisation built in |
| Time to initiate | Weeks to months | Days |
| DevOps compatibility | Limited — slow feedback cycle | Supports shift-left and frequent releases |
About Basis Technologies
Basis Technologies is a software and services company specialising in DevOps tooling for SAP environments. Testimony was announced as an expansion of their DevOps Toolset in 2017. For current product information, capabilities, and availability, visit the Basis Technologies website directly, as product features and positioning may have evolved since this announcement.