SoapUI testrunner assertion failed / non-zero exit in CI

By Kaveh Alemi·Latchkey

SoapUI's command-line testrunner runs a project and fails when any assertion (status, XPath, JSONPath, SLA) does not pass. It prints the failed test case and step, then exits non-zero so CI marks the job failed.

What this error means

testrunner logs "assertion failed" (or "Assertion Failed") for a step and ends with a summary showing failed test cases and a non-zero exit code.

assertion failed [Valid HTTP Status Codes] Response status code:500 is not in acceptable list of status codes
Total failed: 1
com.eviware.soapui.tools.SoapUITestCaseRunner - ...FAILED

Common causes

The service returned an unexpected response

A status, XPath, or content assertion failed because the API returned something the assertion did not accept. Often a real service error.

Environment or property values wrong for CI

An endpoint property, credential, or custom property set for another environment makes the request hit the wrong target or fail auth in CI.

How to fix it

Read the failed assertion and step

1. Find the named test case and assertion in the testrunner output.
2. Reproduce the request against the CI endpoint to see the response.
3. Fix the service, or override the endpoint and properties for CI.

testrunner.sh -e http://localhost:8080 \
  -PapiToken=$API_TOKEN api-tests.xml

Fail the build on assertion errors

Ensure the workflow does not mask the testrunner exit code so a failed assertion fails the job.

- run: ./bin/testrunner.sh -r -a soapui-project.xml

How to prevent it

• Override endpoints and credentials per environment with -e and -P.
• Keep assertions specific to the expected response.
• Do not swallow the testrunner exit code in the workflow.

Frequently asked questions

Related guides

References

• GitHub Actions documentation