Testing Auto-Configuration with ApplicationContextRunner

Auto-configuration classes have combinatorial behavior. The audit starter has four conditions: class presence, bean presence, property value, and tenant mode. That produces sixteen possible combinations. Testing each one by starting a full Spring Boot application context is not viable. It takes seconds per test. A full suite takes minutes. The feedback loop breaks.

ApplicationContextRunner exists to solve this. It creates a minimal application context in memory, runs it, and tears it down in milliseconds. No embedded server. No component scanning. No classpath noise. You configure exactly what you need and assert exactly what you expect.

The Wrong Tool

// BROKEN: Testing auto-configuration with @SpringBootTest
@SpringBootTest
class SaasAuditAutoConfigurationTest {

    @Autowired
    private ApplicationContext context;

    @Test
    void auditServiceIsRegistered() {
        assertThat(context.getBean(AuditService.class)).isNotNull();
    }
}

Four problems:

Full context startup. @SpringBootTest loads every auto-configuration class on the classpath, every component in the scan path, and starts the embedded server if webEnvironment is not set to NONE. For a test that checks one bean, this is a ten-second penalty minimum.
Classpath pollution. The test classpath includes all test dependencies. @ConditionalOnClass always passes because the class is always present. You cannot test the negative case: what happens when DataSource is not on the classpath.
Order dependency. @SpringBootTest loads everything. If another auto-configuration class registers a DataSource, your test passes for the wrong reason. You are testing the aggregate behavior of the entire application, not the behavior of your auto-configuration class in isolation.
No property isolation. Changing properties requires @TestPropertySource or @DynamicPropertySource, both of which apply to the entire test class. Testing multiple property combinations requires multiple test classes or @Nested with repeated annotations.

@SpringBootTest is the right tool for integration tests (CH23). It is the wrong tool for auto-configuration unit tests.

ApplicationContextRunner

ApplicationContextRunner is in org.springframework.boot.test.context.runner. It provides a builder API for constructing a minimal context and a callback API for asserting against it:

// CORRECT: Testing with ApplicationContextRunner
import org.springframework.boot.test.context.runner.ApplicationContextRunner;
import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration;
import static org.assertj.core.api.Assertions.assertThat;

class SaasAuditAutoConfigurationTest {

    private final ApplicationContextRunner contextRunner = new ApplicationContextRunner()
            .withConfiguration(AutoConfigurations.of(
                    SaasAuditAutoConfiguration.class,
                    DataSourceAutoConfiguration.class
            ));

    @Test
    void auditServiceIsCreatedWhenDataSourceExists() {
        contextRunner
                .withPropertyValues(
                        "spring.datasource.url=jdbc:h2:mem:testdb",
                        "spring.datasource.driver-class-name=org.h2.Driver"
                )
                .run(context -> {
                    assertThat(context).hasSingleBean(AuditService.class);
                    assertThat(context).hasSingleBean(TenantAwareAuditInterceptor.class);
                    assertThat(context.getBean(AuditService.class))
                            .isInstanceOf(JdbcAuditService.class);
                });
    }
}

The contextRunner is a template. Each test calls .run(), which creates a fresh ApplicationContext, executes the assertion lambda, and closes the context. No state leaks between tests. No context caching complications.

The .withConfiguration(AutoConfigurations.of(...)) method registers auto-configuration classes in the same way @EnableAutoConfiguration would. The classes go through ConditionEvaluator, respect @AutoConfiguration(after = ...) ordering, and participate in the condition evaluation report. This is not a shortcut. It is the same code path as production.

Testing Negative Conditions

The most important tests for auto-configuration are the ones that verify beans are not registered when conditions are not met:

@Test
void auditServiceIsNotCreatedWhenDataSourceIsMissing() {
    // No DataSourceAutoConfiguration, no DataSource bean
    new ApplicationContextRunner()
            .withConfiguration(AutoConfigurations.of(
                    SaasAuditAutoConfiguration.class
            ))
            .run(context -> {
                assertThat(context).doesNotHaveBean(AuditService.class);
                assertThat(context).doesNotHaveBean(TenantAwareAuditInterceptor.class);
            });
}

This test uses a fresh ApplicationContextRunner without DataSourceAutoConfiguration. No DataSource bean is registered, so @ConditionalOnBean(DataSource.class) on SaasAuditAutoConfiguration returns false, and the entire configuration class is skipped.

Note: testing @ConditionalOnClass for the negative case (class not on classpath) is harder with ApplicationContextRunner because the class is always on the test classpath. For true classpath isolation, use FilteredClassLoader:

@Test
void auditServiceIsNotCreatedWhenJdbcIsNotOnClasspath() {
    contextRunner
            .withClassLoader(new FilteredClassLoader(DataSource.class))
            .run(context -> {
                assertThat(context).doesNotHaveBean(AuditService.class);
            });
}

FilteredClassLoader is a Spring Boot test utility that creates a classloader that hides specific classes. When SaasAuditAutoConfiguration is evaluated, @ConditionalOnClass(DataSource.class) checks against this filtered classloader and returns false. The configuration is skipped entirely.

Testing Property Toggles

The audit starter supports saas.audit.enabled=false to disable it entirely:

@Test
void auditServiceIsNotCreatedWhenDisabledByProperty() {
    contextRunner
            .withPropertyValues(
                    "spring.datasource.url=jdbc:h2:mem:testdb",
                    "spring.datasource.driver-class-name=org.h2.Driver",
                    "saas.audit.enabled=false"
            )
            .run(context -> {
                assertThat(context).doesNotHaveBean(AuditService.class);
            });
}

@Test
void auditServiceIsCreatedByDefaultWhenPropertyIsAbsent() {
    contextRunner
            .withPropertyValues(
                    "spring.datasource.url=jdbc:h2:mem:testdb",
                    "spring.datasource.driver-class-name=org.h2.Driver"
            )
            .run(context -> {
                assertThat(context).hasSingleBean(AuditService.class);
            });
}

The second test verifies matchIfMissing = true on the @ConditionalOnProperty. When saas.audit.enabled is not set, the starter activates. This is the expected default for most starters: present on the classpath means enabled unless explicitly disabled.

Testing User Overrides

The @ConditionalOnMissingBean annotation on each @Bean method allows consuming applications to provide their own implementation. This is testable:

@Test
void userDefinedAuditServiceTakesPrecedence() {
    contextRunner
            .withPropertyValues(
                    "spring.datasource.url=jdbc:h2:mem:testdb",
                    "spring.datasource.driver-class-name=org.h2.Driver"
            )
            .withUserConfiguration(CustomAuditServiceConfig.class)
            .run(context -> {
                assertThat(context).hasSingleBean(AuditService.class);
                assertThat(context.getBean(AuditService.class))
                        .isInstanceOf(CustomAuditService.class);
                // The auto-configured JdbcAuditService is NOT registered
                assertThat(context).doesNotHaveBean(JdbcAuditService.class);
            });
}

@Configuration(proxyBeanMethods = false)
static class CustomAuditServiceConfig {

    @Bean
    AuditService auditService() {
        return new CustomAuditService();
    }
}

static class CustomAuditService implements AuditService {
    @Override
    public void record(String tenantId, String action, String entityType, String entityId) {
        // Custom implementation: send to external audit system
    }
}

.withUserConfiguration() registers a @Configuration class as user configuration, which is processed before auto-configuration classes. This mirrors production behavior: user-defined beans are registered first, then auto-configuration runs, and @ConditionalOnMissingBean sees the user’s bean and skips the auto-configured one.

Testing Custom Properties

Verify that properties are bound correctly to the AuditProperties class:

@Test
void customPropertiesAreBound() {
    contextRunner
            .withPropertyValues(
                    "spring.datasource.url=jdbc:h2:mem:testdb",
                    "spring.datasource.driver-class-name=org.h2.Driver",
                    "saas.audit.table-name=custom_audit_log",
                    "saas.audit.retention-days=30",
                    "saas.audit.async=true"
            )
            .run(context -> {
                AuditProperties properties = context.getBean(AuditProperties.class);
                assertThat(properties.getTableName()).isEqualTo("custom_audit_log");
                assertThat(properties.getRetentionDays()).isEqualTo(30);
                assertThat(properties.isAsync()).isTrue();
            });
}

@Test
void defaultPropertiesAreApplied() {
    contextRunner
            .withPropertyValues(
                    "spring.datasource.url=jdbc:h2:mem:testdb",
                    "spring.datasource.driver-class-name=org.h2.Driver"
            )
            .run(context -> {
                AuditProperties properties = context.getBean(AuditProperties.class);
                assertThat(properties.getTableName()).isEqualTo("audit_events");
                assertThat(properties.getRetentionDays()).isEqualTo(90);
                assertThat(properties.isAsync()).isFalse();
            });
}

Testing Custom Conditions

The multi-tenant condition from CH6 requires a separate test:

@Test
void tenantInterceptorIsRegisteredInMultiTenantMode() {
    contextRunner
            .withPropertyValues(
                    "spring.datasource.url=jdbc:h2:mem:testdb",
                    "spring.datasource.driver-class-name=org.h2.Driver",
                    "saas.tenant.mode=multi"
            )
            .run(context -> {
                assertThat(context).hasSingleBean(TenantAwareAuditInterceptor.class);
            });
}

@Test
void tenantInterceptorIsNotRegisteredInSingleTenantMode() {
    contextRunner
            .withPropertyValues(
                    "spring.datasource.url=jdbc:h2:mem:testdb",
                    "spring.datasource.driver-class-name=org.h2.Driver",
                    "saas.tenant.mode=single"
            )
            .run(context -> {
                assertThat(context).doesNotHaveBean(TenantAwareAuditInterceptor.class);
            });
}

WebApplicationContextRunner

If the auto-configuration registers web-specific beans (filters, interceptors that need ServletContext, reactive handlers), use WebApplicationContextRunner or ReactiveWebApplicationContextRunner:

private final WebApplicationContextRunner webContextRunner = new WebApplicationContextRunner()
        .withConfiguration(AutoConfigurations.of(
                SaasAuditAutoConfiguration.class,
                SaasAuditWebAutoConfiguration.class,
                DataSourceAutoConfiguration.class
        ));

@Test
void auditFilterIsRegisteredInWebContext() {
    webContextRunner
            .withPropertyValues(
                    "spring.datasource.url=jdbc:h2:mem:testdb",
                    "spring.datasource.driver-class-name=org.h2.Driver"
            )
            .run(context -> {
                assertThat(context).hasSingleBean(AuditFilter.class);
            });
}

WebApplicationContextRunner creates a GenericWebApplicationContext instead of a GenericApplicationContext. This makes @ConditionalOnWebApplication pass and provides a mock ServletContext for beans that need it. The reactive variant does the same for reactive contexts.

Testing Failure Scenarios

Auto-configuration should fail gracefully, not catastrophically. Test that misconfiguration produces clear errors:

@Test
void contextFailsWithMeaningfulErrorWhenTableNameIsBlank() {
    contextRunner
            .withPropertyValues(
                    "spring.datasource.url=jdbc:h2:mem:testdb",
                    "spring.datasource.driver-class-name=org.h2.Driver",
                    "saas.audit.table-name="
            )
            .run(context -> {
                assertThat(context).hasFailed();
                assertThat(context.getStartupFailure())
                        .rootCause()
                        .isInstanceOf(IllegalArgumentException.class)
                        .hasMessageContaining("table-name");
            });
}

The .hasFailed() assertion checks that the context failed to start. The .getStartupFailure() gives you the exception. This pattern verifies that your validation logic (in the JdbcAuditService constructor, for example) produces actionable error messages instead of obscure NullPointerExceptions at query time.

The Complete Test Class

Putting it all together:

package com.saas.audit.autoconfigure;

import org.junit.jupiter.api.Test;
import org.springframework.boot.autoconfigure.AutoConfigurations;
import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration;
import org.springframework.boot.test.context.FilteredClassLoader;
import org.springframework.boot.test.context.runner.ApplicationContextRunner;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import javax.sql.DataSource;

import static org.assertj.core.api.Assertions.assertThat;

class SaasAuditAutoConfigurationTest {

    private final ApplicationContextRunner contextRunner = new ApplicationContextRunner()
            .withConfiguration(AutoConfigurations.of(
                    SaasAuditAutoConfiguration.class,
                    DataSourceAutoConfiguration.class
            ));

    // Positive: bean is registered when all conditions pass
    @Test
    void auditServiceIsCreatedWhenDataSourceExists() {
        contextRunner
                .withPropertyValues("spring.datasource.url=jdbc:h2:mem:testdb")
                .run(context -> {
                    assertThat(context).hasSingleBean(AuditService.class);
                    assertThat(context.getBean(AuditService.class))
                            .isInstanceOf(JdbcAuditService.class);
                });
    }

    // Negative: no DataSource bean
    @Test
    void auditServiceIsNotCreatedWithoutDataSource() {
        new ApplicationContextRunner()
                .withConfiguration(AutoConfigurations.of(SaasAuditAutoConfiguration.class))
                .run(context -> assertThat(context).doesNotHaveBean(AuditService.class));
    }

    // Negative: DataSource class not on classpath
    @Test
    void auditServiceIsNotCreatedWithoutJdbcOnClasspath() {
        contextRunner
                .withClassLoader(new FilteredClassLoader(DataSource.class))
                .run(context -> assertThat(context).doesNotHaveBean(AuditService.class));
    }

    // Negative: disabled by property
    @Test
    void auditServiceIsNotCreatedWhenDisabled() {
        contextRunner
                .withPropertyValues(
                        "spring.datasource.url=jdbc:h2:mem:testdb",
                        "saas.audit.enabled=false"
                )
                .run(context -> assertThat(context).doesNotHaveBean(AuditService.class));
    }

    // Override: user bean wins
    @Test
    void userDefinedAuditServiceTakesPrecedence() {
        contextRunner
                .withPropertyValues("spring.datasource.url=jdbc:h2:mem:testdb")
                .withUserConfiguration(CustomAuditConfig.class)
                .run(context -> {
                    assertThat(context).hasSingleBean(AuditService.class);
                    assertThat(context.getBean(AuditService.class))
                            .isInstanceOf(CustomAuditService.class);
                });
    }

    // Properties: custom values bound correctly
    @Test
    void customPropertiesAreBound() {
        contextRunner
                .withPropertyValues(
                        "spring.datasource.url=jdbc:h2:mem:testdb",
                        "saas.audit.table-name=my_audit",
                        "saas.audit.retention-days=7"
                )
                .run(context -> {
                    AuditProperties props = context.getBean(AuditProperties.class);
                    assertThat(props.getTableName()).isEqualTo("my_audit");
                    assertThat(props.getRetentionDays()).isEqualTo(7);
                });
    }

    // Default: enabled when property is absent
    @Test
    void enabledByDefaultWhenPropertyAbsent() {
        contextRunner
                .withPropertyValues("spring.datasource.url=jdbc:h2:mem:testdb")
                .run(context -> assertThat(context).hasSingleBean(AuditService.class));
    }

    @Configuration(proxyBeanMethods = false)
    static class CustomAuditConfig {
        @Bean
        AuditService auditService() {
            return new CustomAuditService();
        }
    }

    static class CustomAuditService implements AuditService {
        @Override
        public void record(String tenantId, String action, String entityType, String entityId) {
        }
    }
}

Seven tests. Each runs in milliseconds. Each tests one specific condition or combination. Each can be run independently. Each failure points to exactly one thing: a condition that matched when it should not have, or a condition that did not match when it should have.

This is the correct granularity for auto-configuration testing. @SpringBootTest for integration. ApplicationContextRunner for auto-configuration logic. Mix them and you get slow tests that pass for the wrong reasons.

Debugging Test Failures

When an ApplicationContextRunner test fails, the assertion tells you what happened but not why. To see the condition evaluation report inside a test:

@Test
void debugConditionEvaluation() {
    contextRunner
            .withPropertyValues("spring.datasource.url=jdbc:h2:mem:testdb")
            .run(context -> {
                ConditionEvaluationReport report = ConditionEvaluationReport.get(
                        (ConfigurableListableBeanFactory) context.getBeanFactory()
                );
                report.getConditionAndOutcomesBySource().forEach((source, outcomes) -> {
                    if (source.contains("SaasAudit")) {
                        System.out.println(source + ": " + outcomes.isFullMatch());
                        outcomes.forEach(outcome ->
                                System.out.println("  " + outcome.getConditionOutcome()));
                    }
                });
            });
}

This prints the same information as --debug but scoped to your auto-configuration class, inside the test. Use it when a test fails and the assertion message does not make the cause obvious. The condition evaluation report will tell you exactly which @Conditional annotation returned false and why.