Test Gap Analysis via Pseudo-Mutation

Workflow

Step 1: Gather production and test code

Read both the production code and its corresponding test files. If the user points to a directory, identify production/test pairs by convention (e.g., Calculator.cs tested by CalculatorTests.cs).

Establish which production methods are exercised by which test methods — trace this through method calls in test code, setup, and helper methods.

Step 2: Identify mutation points

Scan the production code and annotate every location where a mutation could reveal a test gap. Use the mutation catalog below.

#### Boundary Mutations

| Original | Mutation | What it tests | |----------|----------|---------------| | < | <= | Off-by-one at upper bound | | > | >= | Off-by-one at lower bound | | <= | < | Boundary inclusion | | >= | > | Boundary inclusion | | == 0 | == 1 or <= 0 | Zero-boundary handling | | i < length | i < length - 1 or i <= length | Loop boundary | | index + 1 | index or index + 2 | Index arithmetic |

#### Boolean and Logic Mutations

| Original | Mutation | What it tests | |----------|----------|---------------| | && | \|\| | Condition independence | | \|\| | && | Condition necessity | | !condition | condition | Negation correctness | | if (x) | if (!x) | Branch selection | | true (constant) | false | Hardcoded assumption | | flag \|\| other | other | Short-circuit first operand |

#### Return Value Mutations

| Original | Mutation | What it tests | |----------|----------|---------------| | return result | return null | Null handling downstream | | return result | return default | Default value handling | | return true | return false | Boolean return verification | | return list | return new List<T>() | Empty collection handling | | return count | return 0 or return count + 1 | Numeric return verification | | return string | return "" or return null | String return verification |

#### Exception Removal Mutations

| Original | Mutation | What it tests | |----------|----------|---------------| | throw new ArgumentNullException(...) | _(remove entire throw)_ | Guard clause verification | | throw new InvalidOperationException(...) | _(remove entire throw)_ | State validation testing | | if (x == null) throw ... | _(remove entire guard)_ | Null guard testing | | if (!IsValid()) throw ... | _(remove entire check)_ | Validation testing |

#### Arithmetic Mutations

| Original | Mutation | What it tests | |----------|----------|---------------| | a + b | a - b | Addition correctness | | a - b | a + b | Subtraction correctness | | a * b | a / b | Multiplication correctness | | a / b | a * b | Division correctness | | a % b | a / b | Modulo correctness | | x++ | x-- | Increment direction | | -value | value | Sign flip |

#### Null-Check Removal Mutations

| Original | Mutation | What it tests | |----------|----------|---------------| | if (x == null) return ... | _(remove null check)_ | Null path coverage | | if (x != null) { ... } | _(always enter block)_ | Null guard necessity | | x ?? defaultValue | x | Null coalescing coverage | | x?.Method() | x.Method() | Null-conditional coverage | | x! | x | Null-forgiving operator necessity |

Step 3: Evaluate each mutation against tests

For each identified mutation point, reason about whether existing tests would detect the change:

1. Find covering tests — Which test methods exercise the mutated line? Follow call chains through helpers and setup methods.
2. Check assertion relevance — Do those tests assert something that would change if the mutation were applied? A test that calls the method but only asserts an unrelated property would NOT catch the mutation.
3. Classify the mutation as:

| Verdict | Meaning | Action | |---------|---------|--------| | Killed | At least one test would fail if this mutation were applied | No action needed — tests are effective here | | Survived | No test would fail — the mutation would go undetected | This is a test gap — recommend a test improvement | | No coverage | No test exercises this code path at all | Worse than survived — the code is untested | | Equivalent | The mutation produces identical behavior (e.g., x * 1 → x / 1) | Skip — not a real mutation |

Step 4: Calibrate findings

Before reporting, apply these calibration rules:

• Don't flag trivial code. Simple property getters (return _name;), auto-properties, and boilerplate don't need mutation analysis. Focus on logic, conditions, calculations, and error handling.
• Consider defensive depth. If a null guard has a survived mutation but the caller also checks for null, note the redundancy but rate it lower priority.
• Equivalent mutations are not gaps. If changing >= to > doesn't alter behavior because the == case is impossible given the domain, mark it Equivalent and skip.
• Private methods reached through public API are valid targets. Trace through the call chain — a private method called from a tested public method may still have survived mutations if the test doesn't assert the specific behavior affected.
• Rate by risk, not count. A single survived mutation in payment calculation logic is more important than five survived mutations in logging code.

Step 5: Report findings

Present the analysis in this structure:

1. Summary — Overall mutation score and key findings:

`` | Metric | Value | |---------------------|----------| | Mutation points | 42 | | Killed | 28 (67%) | | Survived | 10 (24%) | | No coverage | 2 (5%) | | Equivalent (skipped) | 2 (5%) | ``

1. Survived Mutations (Test Gaps) — For each survived mutation, report:

- Location: File, method, line - Mutation category: Boundary / Boolean / Return value / Exception / Arithmetic / Null-check - Original code: The current code - Hypothetical mutation: What would change - Why it survives: Which tests cover this code and why their assertions miss it - Recommended fix: A concrete test assertion or new test case that would kill this mutation

Group by priority: high-risk survived mutations first (business logic, calculations, security checks), lower-risk last (logging, formatting).

1. No-Coverage Zones — Code paths that no test reaches at all. These are worse than survived mutations.

1. Killed Mutations (Strengths) — Briefly note areas where tests are effective. Highlight well-tested methods and strong assertion patterns. Don't enumerate every killed mutation — summarize.

1. Recommendations — Prioritized list:

- Which survived mutations to address first (by risk) - Specific test methods to add or strengthen - Patterns the team can adopt to prevent future gaps (e.g., always test boundary values, always assert exception types)

Related skills