How to improve Testability

The aim of this blog is to stress the importance of testability for software architectures – especially with regards to frameworks or other reuse components.

What is Testability

Testability is the degree to which a system or component / module

• is itself testable with unit tests, component tests, integration tests and system tests

• enables testability of consuming systems

This is especially important for reuse components, programming language, utilities and frameworks. These systems on the one hand should have a very high testability for ensuring high quality and on the other hand provide means for efficient testability of the consuming applications.

Why is Testability so important

Being able to deliver sustainable high-quality features, you can only achieve with a very high automation and high quality. Testability is a main factor, which allows engineers to achieve that. The effort of considering testability as one goal of the architecture is low and the impact very quickly pays off in terms of improved productivity and quality.

Main benefits:

• Higher quality of application: As an application developer I depend on the architecture of my application and the used reuse systems enabling me to write automated tests efficiently. These need to be fast, stable and allow to detect potential bugs.

• Higher quality of reuse system: A system which is mockable for consumers can also tested by the developers of that system, leading to a higher quality.

• Higher productivity of developers: Testability is probably the cheapest way to improve developer productivity, since every developer is affected by bad testability. And ensuring quality or fixing bugs consumes a huge amount of overall productivity

• Executable documentation: Well written tests can be used by consuming developers as executable documentation, which can help to understand how an API works

• Detect design and architecture flaws early: It forces architects and engineers to look on how someone is going to use your code or API. Besides, you need to deal with principles like separation of concerns and single responsiblity principle early.

Problem Categories

For many of the examples described in the following chapters, solutions have been implemented. The examples have been taken based on relevance for a broad developer community and ensure that the rules are understood. As a provider of a reuse system, you should consider testability from the beginning in your architecture. It does not only simplify the life of the consumer of you system, but also your own life!

Test Isolation not possible or difficult

In order to write fast, stable and maintainable tests, a developer needs to be able to isolate from dependencies efficiently in order to control what are the inputs and observe the outputs for the system under test. In the following sub chapters we describe several categories of problems with regards to test isolation. The provider of a reuse system needs to provide means to overcome those.

Test Isolation on integration test (isolated) or component test level

Test isolation should not only be as efficiently possible for unit tests, but also for component and integration tests. This chapter refers to isolated integration tests. In our terminology integration test are testing some components and isolated from other components. For application depending strongly on frameworks (like Spring Boot applications) these kind of tests can include the framework in the tests.

Examples are:

• SAPUI5 integration testing: The MockSever for OData allows to isolate the UI5 application from the backend. And by that test the UI5 applications in a robust and fast way.

• Integration Tests using in memory Databases. In Spring and other frameworks it is common for integration tests to replace a heavyweight database with an in-memory database to speed up test execution and simplify test setup.

Static Dependencies

Static Dependencies to foreign components should be avoided.

• ABAP static classes: Many ABAP classes only provide static methods and handle their state using class attributes instead of instance attributes. Not only is this a surefire way to create classes which are hard to maintain, the call of a static method of a class is a hard dependency which cannot be easily mocked; a wrapper is needed around each call. An approach to simplify testing of the application is to define clear interfaces and factories for components, together with an injection mechanism.

Builder pattern in APIs

The builder pattern can greatly simplify writing code, but is usually a bad choice in outward facing code because it forces detailed knowledge of a foreign domain’s structure onto its consumers and thus ultimately contradicts the Law of Demeter.

You usually notice this when attempting to write unit tests for this pattern, which quickly becomes unbearable because you have to mock a large amount of objects and methods.

Encountered for example in Spring’s HttpSecurity – try writing a unit test for the @EnableWebSecurity configuration:

http.authorizeRequests()
  .antMatchers(GET, HealthCheckController.ENDPOINT)
    .anonymous()
  .antMatchers(POST, SomeOtherController.ENDPOINT)
    .authenticated()
  {tens of lines of code}
  .anyRequest()
    .denyAll()
  .and()
    .sessionManagement()
				.sessionCreationPolicy(SessionCreationPolicy.STATELESS);

Instable and fragile Tests

Instable tests can cause a huge loss in productivity. Therefore the writer of the reuse system needs to provide means for avoiding instable tests. The following sub-chapter lists some of the aspects, which can lead to instable tests.

Asynchronity is handled in a bad way

The tests should not directly deal with asynchronity. This should be hidden from the tests by providing abstractions for testing.

• Example from SAPUI5 application testing: Before the test tool OPA the application developer needed to consider the asynchronity in their tests.

Instable Tests: Testing against implementation details

The tests should not be implemented against private attributes or functions / methods. If the tests are defined against such private details, the danger is that the tests won’t work anymore with a future release. If the provider of is changing some of these details.

Verifying the outcome: Observability

In order to be able to test the outcome, the test needs to be able to observe state changes or other results. The reuse system needs to consider how to make this efficiently observable for tests on different levels.

Observability determines how easy it is to observe the behavior of a system in terms of its outputs, effects on the environment, and other software components. It focuses on the ease of observing outputs. If user of the system cannot observe the output, they cannot be sure how a given input has been processed.

• For example if a system under test has dependencies to other components. It should be possible to inject for those dependencies spys or mocks.

Fast Feedback

Fast Feedback: Tests should be executed often to detect bugs immediately and lower the effort to find them. Therefore it essential that the tests run very fast. Otherwise it has a very detrimental effect on productivity.

Summary

Therefore consider testability from the beginning in the software architecture to boost developer productivity and quality of your products or reuse components. More on patterns for improving the testability can also be found on xunitpatterns.

Why: More on my motivation to start the newsletter can be found in Collaboration on Improving: Why I’m starting the Engineering Ecosystem.

About me: I am Klaus Haeuptle an engineer and architect at SAP, the author of the books Clean ABAP and Clean SAPUI5, a coach for agile software engineering and a community servant leader for a large SAP internal grass roots community on improving tools, technologies, practices and culture, with more than 3000 participants from all locations and departments. Views are my own - the content published on this channel reflect my opinion and engineering principles.

Subscription: If you want to get updates, you can subscribe to the free newsletter:

Mark as not spam: : When you subscribe to the newsletter please do not forget to check your spam / junk folder. Make sure to "mark as not spam" in your email client and move it to your Inbox. Add the publication's Substack email address to your contact list. All posts will be sent from this address: ecosystem4engineering@substack.com.

Past Editions: The past editions of the engineering ecosystem newsletter can be found in the archive. You can also provide feedback in the discussion thread.

Sharing: Thanks for subscribing to the Engineering Ecosystem. This post is public, so feel free to share it: