Skip to content

Import automated test results

The web client supports importing JUnit XML, TRX and NUnit 3 XML. It maps automated test results to ordinary planned test case variants. Manual and imported results appear together in existing test-run and release reports, while the import origin remains identifiable.

Prepare

Create the required test cases, variants and test run first. A dedicated run for an automated build makes mapping easier and protects manual work. You need the normal read and result-editing permissions for the selected run and its variants.

The import does not start programs or contact a CI server. It processes only the selected XML result file. The format is detected from its content; TRX files may use the .trx extension. Existing Windows scripts and MCP result actions remain available.

Preview and import

  1. Open the target run and its automated-results entry in the web client.
  2. Choose the result file and enter a meaningful source and a build identifier. Do not upload credentials or confidential logs.
  3. Explicitly map each external test you want to import to a planned variant and an appropriate result definition in the project. Similar titles are not automatically treated as identical.
  4. Review the preview. Unmapped, skipped and blocked rows are not successful tests.
  5. Confirm the import and inspect every row's outcome. A partial failure may mean that some other rows have already been imported.

Files are limited to 2 MiB and 1000 test cases. Duplicate external identities, contradictory statuses, invalid durations and XML with external entities are rejected. A skipped external test does not create an executed or successful result. Quick-test rows are not supported by this import.

Supported formats and limits

  • JUnit XML: including exports from pytest, PHPUnit and Playwright. Generated examples from these three reporters are covered by permanent regression tests. JUnit is an exchange format here, not a restriction to Java or unit tests.
  • TRX: Visual Studio reports using the TeamTest 2010 XML namespace. Passed maps to success; Failed, Error and Timeout map to failure; NotExecuted maps to skipped. Nested data rows (InnerResults) are not supported yet.
  • NUnit 3 XML: NUnit reports rooted at test-run. Parameterized cases remain separate by their full names. Passed, Failed and Skipped are supported.

Inconclusive, aborted or unsupported states are rejected. Recognized retry/flaky markers, duplicate identities and contradictory success evidence never silently become success. If a reporter exports only the final result without retry evidence, earlier failures cannot be reconstructed. Attachments, console output and execution environments are not imported automatically; bounded failure diagnostics become the result description. The source report does not replace a completeness check of the test scope.

Protect existing work

Permissions, run state, prerequisites and target results are checked again before import. Completed runs and results already edited manually are not overwritten. Notes, duration and attachments also count as existing work. Required additional fields and test case dependencies retain their normal validation.

Repeating the same import identifier with identical content and mapping does not create duplicate results. Reusing an identifier with changed content, or encountering results changed in the meantime, produces a conflict. Inspect affected rows instead of treating a failed import as fully completed.

After a partial failure, create a new preview with the same import identifier and mapping to retry. Confirming the same preview again returns its existing receipt.

Origin and limitations

Source, build identifier, importer and import time describe recording the result. They do not prove who actually ran the automated test or when it ran. Unknown execution time remains unknown. An imported result also does not automatically prove which version of the test specification the external test exercised.

Therefore, release comparison and risk assessment can still show evidence gaps. A successful imported result alone replaces neither complete test scope nor human acceptance. After a manual edit, the original import origin is not evidence for the current result content.

Windows and MCP

Windows reads ordinary results and provides a web link at the relevant test run. The shared web launch passes the authenticated Windows session; sign-in is required only if that handoff fails. The import interface itself is in the web client.

MCP uses the same checks through ztm_automation_import_read, ztm_automation_import_preview and ztm_automation_import_commit. Ordinary project permissions apply; committing additionally requires the project.automation-import.commit write capability. MCP also retains its existing 1 MiB limit for the entire JSON request body, including XML and JSON escaping; use the web interface for larger files up to 2 MiB. Consult the connected server's tool descriptions and retain import identifiers for safe retries.