Skip to main content
Impact Analysis in Insights runs the recorded inputs of a release’s production runs again on a new version of your project, and compares the output parameters. Use it before you release a change, to see which real cases it affects.

​
Before you start

  • The project has a release with production runs on a runtime cluster of your workspace.
  • You can sign in to that runtime cluster and may read runs in its environment.
  • The version you want to check is on main or on a draft. See Change Workflow.

​
Choose what to compare

  1. In the Blueprint Editor, click Insights in the top toolbar, then Impact Analysis in the rail.
  2. Under Sign in to your Runtime Cluster, choose the cluster and click Sign in. Finish the sign-in in the window that opens.
  3. If Leapter asks Which environment’s runs should be replayed?, click the environment, then Continue.
  4. In Test version (1), choose the stream (main or a draft) and its version.
  5. In Recorded runs, choose the Release (2) and the time window: 24h, 7d, or 30d.
  6. Click Analyze impact (3).
The header of Impact Analysis for the Blueprint Insurance Claim Triage. Test version shows main and v4 (latest), with a green ring labeled 1. Recorded runs shows Release v4 with a green ring labeled 2, then from my-cluster · production, Filter, and the time windows 24h, 7d, and 30d with 7d selected. The Analyze impact button has a green ring labeled 3.

Choose the version to test (1) and the release whose runs are replayed (2), then click Analyze impact (3).

To replay fewer runs, set Max runs or Correlation ID contains under Filter.

​
Read the result

The result shows how many runs Leapter replayed, split into three tiles:
  • Changed: at least one output parameter has another value on the tested version, or the run now fails, no longer fails, or fails differently.
  • Unchanged: both versions give the same output parameters, or fail with the same error.
  • Not run: runs without a complete recording, for example ones that timed out. There is nothing to compare.
On the Changed tab, Shifts lists each output parameter that changed and in how many runs. Open a parameter to see each change from the recorded value to the tested one, and how often it happened.
The result of Impact Analysis: 204 runs, Compare logic with release v4, and the tiles Changed 34 (16.7%), Unchanged 170 (83.3%), and Not run 0. Below, Show All changes 34, and Shifts with a green ring. Shifts lists decision with 34 runs changed, opened to the row Recorded · release v4 approve, Tested · main v5 review at 100%, and reason with 34 runs changed.

34 of 204 runs changed. Shifts (ringed) shows that decision changed from approve to review in all 34.

​
Keep a case as a test case

  1. On the Changed tab, click a run under Changed cases (1). The case shows each changed output parameter with its recorded value (Recorded · release) and its new value (Tested ·).
  2. Click Keep as test case (2).
  3. The dialog Create test case from run is prefilled with the run’s inputs, and with the tested version’s output parameters as the expected outputs. Adjust them if needed.
  4. Click Add test case. Leapter adds it to the test suite the dialog names.
Changed cases with the run portal-CLM-30196 opened and ringed with the label 1: 2 outputs changed, decision and reason. The open case compares decision, approve in Recorded · release v4 and review in Tested · main v5, and reason, Claim amount is within the policy limit and Claim amount exceeds policy limit and requires manual review. Below are Recorded inputs (4) and the Keep as test case button, ringed with the label 2.

Open a changed case (1), then click Keep as test case (2).

Leapter keeps the result only while the browser tab is open. After a reload, sign in to the runtime cluster and click Analyze impact again.

​
Troubleshooting

​
What to do next

  • Test Suites: run the test cases you kept
  • Releases: release the version once its output parameters are right