22 Types of Software Testing Every Developer Should Know in 2026

A pull request adds currency conversion to a checkout service. Line coverage is 94%, every test is green, and the first invoice in Japanese yen comes out a hundred times too large, because the tests only checked that a price came back. Each type of software testing catches a different kind of failure, and this suite had plenty of tests of one kind and none of the kind that would have caught it. With AI assistants now writing a share of both the code and the tests, choosing which tests to write has become the part a human has to own.

The 22 types of software testing at a glance

Most types of software testing fit into two families. Functional testing checks that the system does the right thing: given this input, that output. Non-functional testing checks how well it does it: how fast, under how much load, how safely and for which users. The map below sorts 22 types into those families. How a test is run (by hand, automated or continuously) and techniques such as mutation testing apply across all of them, so they sit underneath rather than in the count.

No project needs all 22 types of software testing with the same weight. A hotel metasearch engine answering hundreds of millions of queries a day needs load testing and soak tests from the first month. An internal document workflow with forty users gets more from access-control tests and a careful acceptance round than from spike tests. The useful question for each type is which failure it would catch in your system, and what that failure would cost. Answered for every type, that question is your testing strategy.

Manual vs automated vs continuous testing

These three software testing approaches describe how a test is run, not what it checks. A usability test is almost always manual; regression testing is almost always automated; and a load test can be a one-off manual exercise or a nightly pipeline job. The table compares manual testing, automation testing and continuous testing on the questions teams usually argue about.

Continuous testing is automated testing wired into every stage of the development process: unit and integration tests on each push, contract checks before a deploy, smoke tests right after it, and synthetic checks against production afterwards. It depends on the pipeline staying fast. Once a build takes forty minutes, developers batch their changes and the feedback arrives too late to point at a single commit.

Automation testing has not made manual testing obsolete: it keeps its place in exploratory sessions, usability studies and acceptance by the people who will use the system. A person notices that a flow is confusing; an assertion only checks what someone thought to write down.

Functional vs non-functional testing: what each one checks

A functional requirement says what the system does: an order over 200 euros gets free shipping, a locked account cannot log in. A test passes or fails against it with no interpretation needed.

A non-functional requirement describes an aspect of software quality, such as speed or availability, and it is only testable once someone writes it as a number. “The checkout must be fast” cannot fail. “The checkout answers 95% of requests in under 300 ms at 200 requests a second” can, and a load test can check it on every release. Write non-functional requirements as numbers before the first performance or availability test, or the tests have nothing to assert.

Functional testing types, from one class to real users

From unit testing to system testing, the functional types differ mostly in how much of the system they run for real, which sets both what they can catch and how long they take.

1. Unit testing

Unit testing checks one piece of logic, usually a class or a small group of classes, in milliseconds and without a database, network or file system. A unit test is the test you run hundreds of times a day, so when performing unit testing, test behaviour through a public method so the test survives a refactoring of the internals.

The Java toolkit in 2026 is JUnit 6, released on 30 September 2025, which raises the baseline to Java 17 and gives the Platform, Jupiter and Vintage modules a single version number; AssertJ for readable assertions; and Mockito for the collaborators you replace. Domain code written with records and sealed types is easy to test, because every case is a value you can construct:

sealed interface Discount {
    record None() implements Discount {}
    record Percent(int value) implements Discount {
        Percent {
            if (value < 0 || value > 100) throw new IllegalArgumentException("percent: " + value);
        }
    }
    record Fixed(BigDecimal amount) implements Discount {}
}

final class Pricing {
    static BigDecimal apply(BigDecimal total, Discount discount) {
        return switch (discount) {
            case Discount.None _ -> total;
            case Discount.Percent(int p) -> total.multiply(BigDecimal.valueOf(100 - p)).movePointLeft(2);
            case Discount.Fixed(BigDecimal amount) -> total.subtract(amount).max(BigDecimal.ZERO);
        };
    }
}

class PricingTest {

    @ParameterizedTest
    @MethodSource("discounts")
    void appliesDiscount(Discount discount, String expected) {
        assertThat(Pricing.apply(new BigDecimal("200.00"), discount))
                .isEqualByComparingTo(expected);
    }

    static Stream<Arguments> discounts() {
        return Stream.of(
                arguments(new Discount.None(), "200.00"),
                arguments(new Discount.Percent(10), "180.00"),
                arguments(new Discount.Fixed(new BigDecimal("250.00")), "0.00"));
    }

    @Test
    void rejectsPercentOutsideRange() {
        assertThatIllegalArgumentException().isThrownBy(() -> new Discount.Percent(120));
    }
}

The parameterized test reads as a table of cases, and isEqualByComparingTo compares BigDecimal values by amount, so 180.00 equals 180.0000. Two practical notes on Mockito: mock the interfaces you own (a PriceFeed port) rather than library types, and on Java 21 and later attach Mockito as a Java agent in the Surefire configuration, since the JDK now warns about agents loaded at runtime. In Spring Boot tests, use @MockitoBean from Spring Framework; @MockBean has been deprecated since Spring Boot 3.4, and the Spring Boot 4 migration guide covers what else moves in an upgrade.

2. Integration testing

Integration testing checks that your code works with the things it talks to: a real database, a message broker, a cache. These are the tests that catch a mistyped column, a missing index that turns a query into a table scan, a transaction boundary in the wrong place, or JSON that a real Kafka consumer cannot read.

Integration tests with Testcontainers and Spring Boot

The old compromise was an in-memory H2 database standing in for PostgreSQL or Oracle, which hid every difference in SQL dialect, locking and JSON support. Testcontainers starts the real database in Docker for the test run, and Spring Boot’s @ServiceConnection points the application at it with no properties to set:

@SpringBootTest
@Testcontainers
class OrderRepositoryIT {

    @Container
    @ServiceConnection
    static PostgreSQLContainer postgres = new PostgreSQLContainer("postgres:17-alpine");

    @Autowired
    OrderRepository orders;

    @Test
    void findsOnlyOpenOrdersOfOneCustomer() {
        orders.save(Order.open("c-42"));
        orders.save(Order.open("c-42").ship());
        orders.save(Order.open("c-7"));

        assertThat(orders.findOpenByCustomer("c-42"))
                .singleElement()
                .extracting(Order::customerId)
                .isEqualTo("c-42");
    }
}

The query runs against the same PostgreSQL major version as production, with your schema migrations applied at startup. Expect seconds per test class rather than milliseconds; reusing one container across test classes keeps a suite of a few hundred integration tests within a few minutes.

What changed in Testcontainers 2.0

Testcontainers 2.0 removed JUnit 4 support, prefixed every module artifact with testcontainers- (org.testcontainers:postgresql became org.testcontainers:testcontainers-postgresql) and moved container classes into per-module packages such as org.testcontainers.postgresql, where PostgreSQLContainer is no longer generic. Spring Boot 4.0 manages Testcontainers 2.0, so a Boot upgrade brings the change with it. Moving from 1.x is mostly imports and artifact coordinates, which automated refactoring with OpenRewrite handles well.

3. Contract testing

In a system of services owned by different teams, the costly failures happen at the boundaries: the pricing team renames a field, their tests pass, and checkout breaks in production. End-to-end tests across ten services would catch it, slowly and unreliably. Consumer-driven contract testing catches it in each team’s own build.

With Pact, the consumer writes a test against a mock provider that records what it sends and what it relies on in the response:

@ExtendWith(PactConsumerTestExt.class)
@PactTestFor(providerName = "pricing-service")
class PricingClientPactTest {

    @Pact(consumer = "checkout-service")
    V4Pact priceOfKnownSku(PactDslWithProvider builder) {
        return builder
                .given("SKU A-100 exists")
                .uponReceiving("a price request for A-100")
                    .path("/prices/A-100")
                    .method("GET")
                .willRespondWith()
                    .status(200)
                    .body(new PactDslJsonBody()
                            .stringType("sku", "A-100")
                            .decimalType("amount", 19.99)
                            .stringMatcher("currency", "[A-Z]{3}", "EUR"))
                .toPact(V4Pact.class);
    }

    @Test
    void readsPriceAndCurrency(MockServer mockServer) {
        var client = new PricingClient(URI.create(mockServer.getUrl()));

        assertThat(client.priceOf("A-100").currency()).isEqualTo("EUR");
    }
}

The test produces a contract file, which goes to a Pact Broker. The provider’s pipeline replays every consumer’s contract against the real pricing service and fails if a field the checkout relies on has gone. Before a deploy, the broker’s can-i-deploy check answers whether this version is compatible with what is running in the target environment. The matchers matter: stringType asserts that sku is a string, not that it equals “A-100”, so the contract pins the shape the consumer needs and nothing more. Spring Cloud Contract is the provider-driven alternative for teams that are all on Spring.

4. End-to-end (system) testing

An end-to-end test drives the deployed system the way a user would, through the browser or the public API, with every service, database and queue running. System testing is the older name for the same level, and many a test plan still uses it. It is the only test that proves a journey works with all the real configuration in place, and it is also the slowest and flakiest test you own.

Keep end-to-end suites to the journeys that make money or carry legal risk: sign-up, checkout, payment, a claim submission. Playwright for Java is a sound default for new suites: it waits for elements automatically and runs Chromium, Firefox and WebKit from one API. When an end-to-end test fails, write the narrower test that would have caught the same bug, so the next occurrence fails in seconds rather than at the end of the pipeline. For services without a user interface, API testing with REST Assured fills the same role at a lower cost.

5. Smoke testing (and sanity testing)

Smoke testing means a handful of quick checks run right after a deploy: the application starts, the health endpoint reports its database and broker as up, a user can log in, the main page renders. If smoke tests fail, the pipeline rolls back before anyone runs the longer suites. A Spring Boot Actuator health check plus three or four API calls is enough for most services.

Sanity testing is the narrower check after a small fix: confirm the bug is gone and the existing functionality around it still works, before spending time on full regression testing.

6. Regression testing

Regression testing reruns the existing suite to confirm that a change did not break existing functionality. In a continuous pipeline it is less a separate activity than a discipline: every bug fix starts with a test that reproduces the bug, and that test stays in the suite.

Regression tests matter most before a large change. When we moved a London wine merchant’s platform from Java 17 and Spring 5 to Java 25, we first added regression tests around PDF invoices, spreadsheet imports, product search, orders and billing, plus browser tests through both applications. During the upgrade they caught two breaks before release: one would have stopped all PDF generation, the other would have broken spreadsheet imports. For a legacy system modernization, that safety net comes before the first framework moves.

Acceptance testing: UAT, alpha and beta, operational

Acceptance testing answers a different question from the functional testing above: does the system meet user expectations and the needs of the people who operate it, rather than what the specification says?

7. User acceptance testing (UAT)

In user acceptance testing, the people who will use the system run real scenarios on production-like data and sign off. UAT finds the gap between the requirement as written and the job as done, such as a claims handler who needs three fields on one screen. Book the testers’ time in advance; acceptance testing competes with everyone’s day job and slips when nobody has hours set aside for it.

8. Alpha and beta testing

Alpha testing puts the product in front of internal users; beta testing widens it to a group of real customers. In 2026 most web products do this with feature flags and staged rollouts rather than separate beta builds: the new flow goes to staff, then to 5% of users, with error rates and support tickets watched at each step.

9. Operational acceptance testing

Operational acceptance testing (OAT) checks that the system can be run, not just used: backups restore, failover works, alerts fire and reach someone, logs carry a correlation ID, and the runbook matches the system. It is the test most often skipped and the one an incident at 3 a.m. depends on.

Test pyramid or testing trophy: how many of each test to write

Once the types are clear, the argument moves to proportions, and to the testing strategy behind them. The test pyramid, which Mike Cohn introduced and Martin Fowler’s site popularized, says to write many fast unit tests, fewer integration tests and a few end-to-end tests. The testing trophy from Kent C. Dodds moves the weight to integration tests, with static analysis as the base.

Much of the disagreement is about words. Fowler’s essay on test shapes points out that people mean different things by “unit test”: a solitary test with every collaborator mocked, or a sociable test that runs a few real classes together. A sociable unit test in one team’s pyramid is an integration test in another team’s trophy.

Our position for Java services: let the cost of feedback decide. Testcontainers made a real-database test cost seconds, so for a Spring Boot service whose logic is mostly persistence and calls to other services, the trophy fits: most tests should run against real infrastructure, because that is where the bugs are. For code where the logic is the product, such as pricing rules, ledger postings or schedule calculations, the pyramid still wins: thousands of millisecond unit and property tests cover combinations no integration suite could reach. Most systems contain both kinds of code, so most need both shapes in different modules.

The shape to avoid is the ice-cream cone: few unit tests, some integration tests and a large, slow, manual or end-to-end layer on top. It usually grows when tests are added only after bugs reach production.

Non-functional testing types: performance, security, experience and reliability

Non-functional tests rarely belong on every pull request; a soak test takes hours and a penetration test takes days. The matrix shows a cadence that works for most Java services, and the tools we would reach for first.

Performance testing

Performance testing measures response time, throughput and resource use under a defined load. The five types below differ mostly in the shape of that load over time.

Report percentiles, not averages: a 50 ms average can hide a p99 of four seconds, and the p99 is what your heaviest users experience. Use an open workload model (new users arrive at a set rate whether or not earlier requests have finished) when real traffic behaves that way; a closed model with a fixed number of virtual users slows its own arrival rate when the system slows down, which flatters the results. Gatling’s Java DSL keeps the load test in the same language and repository as the service, and its assertions turn the run into a pass or a fail for the pipeline:

import static io.gatling.javaapi.core.CoreDsl.*;
import static io.gatling.javaapi.http.HttpDsl.*;

public class CheckoutSimulation extends Simulation {

    HttpProtocolBuilder protocol = http.baseUrl("https://staging.example.com");

    ScenarioBuilder checkout = scenario("Checkout")
            .exec(http("view cart").get("/api/cart"))
            .pause(1)
            .exec(http("pay").post("/api/checkout")
                    .body(StringBody("{\"cartId\": \"c-1\"}")).asJson());

    {
        setUp(checkout.injectOpen(rampUsersPerSec(10).to(200).during(Duration.ofMinutes(10))))
                .protocols(protocol)
                .assertions(
                        global().responseTime().percentile(95.0).lt(300),
                        global().failedRequests().percent().lt(1.0));
    }
}

k6 is the main alternative, with scripts in JavaScript or TypeScript and close integration with Grafana; Apache JMeter remains common in organizations with existing test plans. Whatever the performance testing tool, the result is only as good as the workload model. On the UK hotel metasearch engine we redesigned, the production figures after the redesign were over 300 million queries a day, 95% of them answered in under a second. A load test that cannot reproduce that query mix says little about either number. When a test fails a target, a profiler tells you why; see Java profilers and common Java performance problems.

10. Load testing

Load testing runs the expected peak, for example the Monday-morning or Black Friday traffic, and checks latency and error rates against the targets. It is the one performance test worth running on every release candidate.

11. Stress testing

Stress testing raises load in steps past the peak until something gives, to learn which component fails first (often a connection pool or a downstream service rather than your own code) and whether the system recovers once the load drops. A system that sheds load with fast errors is in better shape than one that queues requests until every caller times out.

12. Spike testing

Spike testing jumps from normal traffic to several times the peak in seconds: a marketing email, a flash sale, a retry storm after an outage elsewhere. It tests autoscaling delays, cold caches and JIT warm-up, which a slow ramp never reaches.

13. Soak (endurance) testing

Soak testing, also called endurance testing, holds a realistic load for hours, sometimes days. It finds what only time reveals: a memory leak, a growing cache with no eviction, connections that leak from the pool one at a time, log files that fill a disk. Record a Java Flight Recorder session for the whole run so the heap and GC trend is there when you need it.

14. Scalability testing

Scalability testing adds nodes or resources and measures whether throughput grows with them. Perfectly linear scaling is rare; the useful output is the curve and the point where it flattens, which shows where the shared bottleneck is (often the database) and what the next tenfold growth in traffic will cost. It is also where a performance engineering engagement should start, because it shows which component to fix first.

Security testing

Security testing finds weaknesses before an attacker does. Four types cover most of the ground, from cheap and automatic to expensive and human.

15. Static analysis (SAST)

Static application security testing reads the source code without running it, looking for injection, unsafe deserialization, hard-coded secrets and the like. SonarQube (Server or Cloud), SpotBugs with the Find Security Bugs plugin and GitHub CodeQL all handle Java. Run it on every pull request and fail the build on new critical findings only, or the backlog of old warnings teaches everyone to ignore the tool. The secure Java coding guide covers the code patterns these tools flag.

16. Dependency scanning (SCA)

Software composition analysis checks every library in the dependency tree against vulnerability databases. Most of the code in a Spring Boot application is someone else’s, and new advisories appear without any change on your side, so scan nightly as well as on each build. OWASP Dependency-Check, GitHub Dependabot and the OSV scanner all read Maven and Gradle builds. On the wine merchant’s platform, a dependency scan found 12 critical and 35 high-severity advisories; after the upgrade the same OSV scan found none. Advisories that need a framework upgrade to fix are what our Java vulnerability remediation work handles.

17. Dynamic scanning (DAST)

Dynamic application security testing attacks the running application from the outside, the way a scanner on the internet would: missing security headers, injectable parameters, exposed admin endpoints. ZAP, the open-source scanner now published as ZAP by Checkmarx, has a baseline scan that fits in a pipeline stage against a staging environment, and a full scan for nightly runs.

18. Penetration testing

A penetration test is a time-boxed attack by specialists who chain small weaknesses into a real breach, which no scanner does. Book one before a major release, after an architecture change and at least yearly for systems that hold payment or personal data. Financial entities in the EU face a stricter version: the Digital Operational Resilience Act, applied since January 2025, requires threat-led penetration testing for the firms it designates.

Usability, accessibility and compatibility testing

19. Usability testing

Usability testing watches real users try to complete real tasks, and measures where they hesitate, misread or give up. Short rounds of moderated sessions, repeated as the design changes, show where people get stuck. It is manual by nature and works best early, on a prototype, while changing a screen still takes an afternoon.

20. Accessibility testing

Accessibility testing checks that people using screen readers, keyboards or magnification can use the product, usually against WCAG 2.2 level AA. Since June 2025 the European Accessibility Act has made this a legal requirement for many consumer-facing digital services sold in the EU. Automated checks with axe-core, which has a Playwright integration for Java, catch missing labels and contrast failures on every build; a manual audit with a screen reader catches the rest before a release.

21. Compatibility testing

On the front end, compatibility testing means running the main journeys on the browsers, devices and operating systems your users have, which Playwright’s Chromium, Firefox and WebKit engines cover for most web applications. On the back end, it means a build matrix: if a library supports Java 21 and 25, CI should run its tests on both, and an application should be tested on the JDK and operating systems it will run on in production. The Java end-of-life calendar tells you which versions are worth keeping in the matrix.

22. Resilience (chaos) testing

Resilience testing injects failures on purpose: a dependency that answers in five seconds instead of fifty milliseconds, a dropped connection, a node that disappears. It checks the timeouts, retries, circuit breakers and fallbacks that only run when something else goes wrong, which is exactly when nobody has tested them. Toxiproxy, which has a Testcontainers module, adds latency and cuts connections inside an ordinary integration test; Chaos Mesh does the same at cluster level on Kubernetes. Start small, with one dependency made slow in a test environment, before any experiment in production.

Techniques that work across test types: exploratory, property-based and mutation testing

These are testing methods rather than types in their own right: ways of designing or checking tests that apply at any level.

Exploratory, black-box and white-box testing

Exploratory testing is unscripted: a tester learns the system, designs tests and runs them in the same session, following whatever looks wrong. Black-box testing works from inputs and outputs only; white-box testing uses knowledge of the code, for example to reach every branch. Most good test suites mix both, and random “monkey” testing survives today in its automated form, fuzzing.

Property-based testing

An example-based test checks one input. A property-based test states a rule that must hold for every input, and the framework generates hundreds of inputs to try to break it, then shrinks any failure to the smallest case. On the JVM, jqwik runs on the JUnit Platform and tries 1,000 inputs per property by default:

class PricingProperties {

    @Property
    void discountedTotalStaysBetweenZeroAndTotal(
            @ForAll @BigRange(min = "0", max = "1000000") @Scale(2) BigDecimal total,
            @ForAll @IntRange(min = 0, max = 100) int percent) {

        var result = Pricing.apply(total, new Discount.Percent(percent));

        assertThat(result).isBetween(BigDecimal.ZERO, total);
    }
}

Properties work well for parsers, serializers (decode after encode returns the input), money and date arithmetic, and any code with an invariant you can state in one sentence. They also counter the pesticide paradox, where a suite that always feeds the same inputs stops finding new bugs. One thing to know before adopting it: since version 1.10, jqwik ships with an anti-AI usage clause and prints a line addressed to AI coding agents on every test run, which matters if agents run your test suite.

Mutation testing

Coverage tells you which lines a test executed, not whether it checked anything. Mutation testing measures that directly: PIT makes small changes to the compiled code, such as turning a - into a + or a < into a <=, and runs your tests against each mutant. A mutant that no test notices has survived, and points at a line that is covered but not tested. This test covers the percent branch fully and kills almost nothing:

@Test
void appliesPercentDiscount() {
    var result = Pricing.apply(new BigDecimal("200.00"), new Discount.Percent(10));
    assertThat(result).isNotNull();   // line covered, behaviour unchecked
}

PIT changes 100 - p to 100 + p, the method returns 420.00 instead of 180.00, and the test still passes. The parameterized test earlier kills that mutant, and so does the jqwik property. PIT runs JUnit Platform suites through its pitest-junit5-plugin, placed on the plugin’s classpath rather than the project’s; its README lists the platform versions each release was built against, so check it before moving to JUnit 6.

<plugin>
    <groupId>org.pitest</groupId>
    <artifactId>pitest-maven</artifactId>
    <version>${pitest.version}</version>
    <dependencies>
        <dependency>
            <groupId>org.pitest</groupId>
            <artifactId>pitest-junit5-plugin</artifactId>
            <version>${pitest-junit5.version}</version>
        </dependency>
    </dependencies>
    <configuration>
        <targetClasses>
            <param>com.example.pricing.*</param>
        </targetClasses>
        <mutationThreshold>80</mutationThreshold>
    </configuration>
</plugin>

Run mvn test-compile org.pitest:pitest-maven:mutationCoverage for a full report, or the scmMutationCoverage goal to mutate only the classes changed in the working copy, which keeps the run short enough for a pull request. The mutationThreshold fails the build when the share of killed mutants drops below 80%.

Six rules for testing AI-generated and vibe-coded software

AI coding assistants are now ordinary tools: in the Stack Overflow Developer Survey 2025, 84% of respondents use or plan to use them and 51% of professional developers use them daily. The same survey found that 46% distrust the accuracy of AI output, against 33% who trust it. Google’s DORA research describes AI as an amplifier that magnifies a team’s existing strengths and weaknesses (State of AI-assisted Software Development, 2025). For software testing, that means a weak suite lets the defects in generated code through at the same higher speed as the code itself.

Vibe coding, the term Andrej Karpathy coined in early 2025 for building software by prompting and accepting the output without reading it closely, takes this to its limit: the tests become the only part of the system anyone has reasoned about. The six rules below build on the classic principles of good testing (start early, avoid the pesticide paradox, automate, test in production-like conditions, keep testing continuous) and apply them to generated code. For the assistants themselves, see our comparison of the best AI coding tools.

1. Write or approve the tests before you accept generated code

In an AI-assisted development process, treat the tests as the specification. Before an assistant writes the implementation, a developer writes the key test cases, or reviews and corrects the ones the assistant proposes, including the edge cases: empty input, the currency with no minor unit, the leap day, the customer with two open orders. If the tests were generated alongside the code, passing them shows only that the code agrees with the model’s own reading of the task.

2. Never let one model write both the code and its only tests

When the same model writes a function and its tests in one pass, both carry the same misunderstanding of the requirement, and the tests confirm it. Keep authorship of code and tests separate: a human writes one side, or at minimum reviews the tests as carefully as the code. Watch for tests that mock the very class under test, assert on values copied from the implementation, or check only that no exception was thrown.

3. Prove the tests can fail with mutation and property-based testing

Generated tests are often long and green and assert very little, like the isNotNull() example above. Run PIT on the classes a pull request changes and look at the surviving mutants before accepting the tests. Add property-based tests for anything with an invariant; they explore inputs that neither the model nor the reviewer thought of, which is where generated code tends to be wrong.

4. Test the boundaries against real systems

Assistants write plausible code against APIs and schemas they have only partly seen: a column name from an older migration, a response field the provider never sent, a library method from a different major version. Unit tests with mocks reproduce the same guess. Integration testing against a real database in Testcontainers, together with contract tests verified by the provider, is where those guesses fail. Run them with data shaped like production (volumes, encodings, nulls, the long tail of odd records), since a test database of ten tidy rows tells you little.

5. Scan generated code and every dependency it adds

Run SAST on generated code like any other code, and review every new dependency an assistant suggests. Models invent package names, and attackers register them: the practice is called slopsquatting. In a study of 576,000 code samples generated by 16 models, Spracklen and colleagues found that at least 5.2% of packages suggested by commercial models and 21.7% by open-source models did not exist, with 205,474 unique invented names (USENIX Security 2025 paper). The study covered Python and JavaScript. Maven Central’s verified group IDs make it harder to publish under an established company’s namespace, but not under a plausible new one, so check that each new artifact exists, is the one you meant, and is maintained before it reaches the build.

6. Gate merges on new code in CI

Put the rules into the pipeline rather than relying on reviewers’ memory. A quality gate on new code, such as SonarQube’s, can require a coverage level on changed lines, a mutation score from PIT, no new critical security findings and no new dependency with a known vulnerability. Gating on new code instead of the whole codebase keeps the gate passable on a legacy system while ensuring that nothing new arrives without tests. Keep the pipeline fast enough that nobody is tempted to bypass it; continuous testing only works while people trust the signal.

Checking a Java test suite you inherited

The rules are easiest to apply on a new codebase. On an existing one, five checks show within a day how far the current suite can be trusted:

1. Run PIT on the module that holds the core business logic. A mutation score far below the line coverage means the tests execute code without checking it.
2. List what the integration tests replace with mocks or in-memory substitutes. Every mocked database, queue or external API is a boundary that no test has exercised for real.
3. Time the suite and count the tests that were retried or disabled in the last month. Flaky tests that everyone re-runs are tests nobody believes.
4. Find the non-functional targets. If response time, throughput and availability are not written down as numbers, no performance test can pass or fail.
5. Run a dependency scan and count the critical and high advisories, and how many need a framework upgrade to fix.

The answers usually point to a few weeks of work: a safety net of regression tests around what the business depends on, real-infrastructure integration tests in place of mocks, and gates in CI. We have kept business-critical Java systems running and changing since 2013, and we build that safety net before anything else moves. If your suite is green while production keeps finding bugs, tell us about the system and we will tell you where we would start.