SOA test strategies

SOA is more than simple web services that provide interfaces to existing applications. An SOA implementation is likely to involve multiple services and vendors and is considered a complex computing problem. Many companies try implementing automated testing for GUI solutions. Due to the changing nature of GUIs, they find it difficult to achieve ROI. That is not to say that GUI testing cannot be achieved, but it must be done during the project lifecycle by applying automated interface testing. Focusing on programmatic interfaces and services in automated testing offers significant ROI, as the design contracts for these interfaces change less often, changing business processes in a distributed environment is costly. When it comes to SOA, a different approach, skill-set, and mind-set are required for testing.

The main factors of a successful test methodology are as follows:

  • Testing must start early in the project once business processes (for example, use cases) and interface definitions are agreed upon. If the SOA is deployed and does not have any documentation, then regression tests can be used to determine compatibility with an earlier version.
  • QA tests can be run at key stages throughout the project, and these tests can be shared with other groups as early as possible.
  • A test plan can outline the high-level test suites that need to be executed along with the acceptance criteria.
  • When you are designing tests, a data-driven approach to test generation can save time, particularly if an agile approach is being implemented.
  • To aid test design, a focus on test coverage analysis is also needed along with requirements traceability.
  • The following test phases are likely to be required when you are testing an SOA:
    • Policy testing to ensure, for example, continued compliance with an XML schema. This type of testing is required throughout the lifetime of an SOA ensuring there is always compliance to the interface definitions.
    • Service-component level testing (or unit testing) ensuring the individual components that make up a service work correctly. This level of testing is typically performed by a software engineer.
    • Service-level testing that focuses on functional, and security testing of individual services within a controlled environment. This is an important level of testing that can help ensure that the quality of the system components is sufficient.
    • Integration-level testing focuses on testing the service interfaces. This level of testing aims to determine if the interface behavior and information sharing between linked services works as specified (interoperability, functional, regression, and security). The interfaces must comply with the defined interface definitions (that is, standards, format, and data validation). Integration tests can also push the different layers of communications and network protocols. Stubs are likely to be required to simulate services that are not yet available.
    • Process-level testing ensures that services are operating collectively as specified. This phase of testing would cover business logic, sequencing, and exception handling.
    • System-level testing (or acceptance testing) focuses on testing services that are linked together to deliver a business process. This level of testing focuses on the key business scenarios and requires test coverage from previous test runs to be compared so that quality can be assessed.
  • Test execution is then achieved by executing the test plan with each test run associated with specific versions of the components under test and saved for future reference.
  • Simulation (stubs) enable project agility by crossing test phase boundaries (that is, development, unit testing, integration testing). To achieve successful simulation requires proper control of test data so that stubs can be created and use consistent data across the test environments. Stubs require maintenance across the different environments as both the data and endpoints change, meaning the deployment method for the stub might need to change. Basic stubs can be created when you are building a test project. These form the basis for more focused stubs that might be needed as the project evolves.
  • Once an SOA is deployed then regression testing is needed ensuring continued compliance, and compatibility with an earlier version.
  • A risk-based approach to testing can be taken where business impact, the likelihood of failure, and frequency of use are considered during test design and execution. This approach can help IT management and business users to make informed decisions quickly.