Release comparison and acceptance¶

Compare two release contexts in the same project and record a domain decision with its evidence at that time. These are releases of your tested product, such as “Portal 2.3” and “Portal 2.4”. No AI provider is required.

Release overview¶

The current development version adds Release overview: a compact report for one release with an optional baseline. Open the overview in Web, select the target release and optionally a baseline. Load test runs, select one result run per plan and environment, choose the expected scope from a reference run or the current template, then choose Create overview. Loading the selection does not capture a report. Without a baseline, no comparison figures appear.

Counts represent expected test case variants per selected plan and environment; quick-test rows do not increase the total. The report shows scope, executed tests, results and evidence gaps. Missing results or evidence are not success. Selected findings are neither a complete issue list nor a business priority ranking; use the detailed release review for source context. The actual test period remains unknown without reliable execution timestamps. Release dates or the capture time do not replace that evidence.

Download PDF exports the same captured data. The report gives its capture time in UTC; later result changes are not automatically included. Create a new overview for current figures. The temporary report is bound to its project and user and available for at most 15 minutes; expiry, a server restart or cache eviction may require a new capture. Read and source permissions are checked again when opening or exporting it.

A live overview does not create acceptance, a responsible person or a human assessment. An overview of an existing saved comparison snapshot can show its recorded decision; a subsequent decision change is flagged. The PDF includes selected findings with their total count and a link to the detailed review, not a complete test case list. Very long decision text can prevent export. Downloaded PDFs remain portable copies: later permission changes or withdrawal of acceptance do not remove them. Share confidential project data only with authorized recipients.

Overview through MCP¶

ztm_release_overview_read captures the selected scope and returns counts with a temporary reportId. ztm_release_overview_export creates a protected PDF resource from it, retrieved through resources/read. Use the same project, user and API key for both calls and resource retrieval. The report and its resource expire after at most 15 minutes; source permissions are checked again. These tools read or export and do not create an acceptance decision. No AI provider is required.

Select the baseline, target and scope¶

In Web, open Release comparison and acceptance. If there are fewer than two contexts, use Manage releases to create them with their test plans or runs.
Select Baseline and Target. Under Review state, choose Current data or a saved review state for that context.
Choose Load test runs. For current data, select one run per plan and environment, then select its Expected review scope: an appropriate reference run or Current template.
Choose Compare and expand Review scope and notes to check sources and gaps.

The baseline → target direction determines whether a result change is an improvement or a regression. Current template describes today's expected scope; it does not reconstruct a historical release state.

Explicitly select the reference runs and environments. Results from different runs are not automatically combined into complete release evidence. A partial retest only covers its included tests; omitted tests remain a gap.

Assess differences¶

Review scope and missing evidence alongside result changes:

Improved / regressed: Comparable results have changed in quality.
Newly executed: A test already in scope now has a result.
Untested in both: Neither side has a result.
Unchanged: Comparable results are equal.
Not comparable: Evidence needed for reliable matching or assessment is missing.

New or removed tests require evidence of a scope difference. Matching titles alone do not identify the same test. Review changed test content separately from result changes. A missing result never counts as success. Different custom results do not automatically mean an improvement.

Filters affect the displayed rows, not the underlying review scope. Always consider counts together with their test population and environment. With different test populations, a higher pass rate alone does not establish better release quality.

Save evidence and a decision¶

Under Acceptance, enter a name and choose Save review state. Open the saved state, select the decision and responsible person, enter a comment and open points, then choose Record decision. Enter each open point on a separate line and review the confirmation.

A saved review snapshot preserves the comparison at that time; it is not itself an acceptance decision. A decision refers to that evidence and records the responsible person, actual decision maker, timestamp, comment and open points.

Available decisions are accepted, accepted with conditions, rejected and withdrawn. Without a decision, the state remains open. Conditional acceptance requires at least one open point; rejection and withdrawal require a comment. Record later completion as a new decision, preserving the original entry.

Review the selected sources and complete the required test runs before saving a new acceptance decision. If the current evidence has changed since the preview, review it again. Completing a test run or freezing a release context does not itself constitute domain acceptance.

Very large review snapshots can exceed the storage limit. A message explains this; no incomplete snapshot is saved. Select fewer test plans or environments for the comparison. Snapshot size also depends on the amount of test description and evidence content.

Later changes and permissions¶

Tests and results can still change, including in Windows. A saved decision retains its historical evidence. Check the notice about changed evidence before treating a decision as a statement about the current data.

Reading requires the corresponding object permissions. Saving and deciding are available to administrators and project managers allowed to edit project settings. A restricted partial view cannot establish full acceptance. Historical evidence is also checked for permissions on access; deleted or inaccessible sources may prevent nonadministrators from opening it. A release context with stored history cannot simply be deleted.

Open in Web¶

Open Reporting and choose Release overview in the report catalog. You can also find it using the catalog search. In Planning & execution, the Reports menu of the selected tree item provides another entry. Check the release and test run selection on the destination page before creating the overview.

Open from Windows¶

In the project tree, right-click a test plan folder, test plan or test run and choose Reports → Release overview (web). Like the ribbon command for opening Web, the link uses your authenticated Windows session to sign you in automatically in the external browser and open the destination directly. Only if the handoff fails (for example after the session expires or the server restarts) does the sign-in page appear; signing in then opens the selected report. A selected test run is preselected; the release is inferred only when its assignment is unambiguous. Review the selection in Web before creating the overview.

Open a test run's properties and choose View release comparison and acceptance in Web. The existing Web launch retains the project context. Use the browser to compare releases and record decisions. If Web access is unavailable, ask your administrator to check its configuration.