PR Reviews - Detailed Guide

CodeStax reviews a pull request using multiple evidence sources. Diff-focused checks and AI-assisted analysis run alongside scanners that evaluate the checked-out PR head, so results are broader than changed lines alone.

Detection layers

Deterministic and AI-assisted diff analysis

These checks look for security-sensitive patterns, leaked credentials, unsafe data handling, error-handling gaps, and architecture or code-quality concerns in the pull-request changes.

Repository-head scanners

Security, secret, dependency/container, and infrastructure-as-code analyzers run against the resolved PR head. Their findings can therefore point to files or lines outside the diff. The review’s Analysis integrity panel shows whether required analyzer coverage was complete, degraded, incomplete, or unavailable.

AI-risk pattern signals

CodeStax can surface patterns such as placeholder implementations, broad error handling, disabled controls, suspicious imports, copy-paste repetition, and unfinished code. The 0–100 value summarizes implementation-pattern evidence for reviewer verification.

Use these signals as review prompts. Inspect the cited evidence before making a decision.

Scoring and gate decisions

The numeric risk score is normalized from deterministic diff findings, accepted AI-assisted findings, and repository-scanner findings. Scanner severity influences the final score and category totals. The final value is the greater of the weighted category composition and the accepted-evidence severity floor: high evidence floors the score at 75 and critical evidence at 95.

The canonical quality gate evaluates the configured maximum risk score, critical-finding count, and configured high-finding limit. Settings → Policies and Reviews → Settings edit the canonical organization or repository policy. Each completed review stores an immutable versioned gate snapshot, so later policy edits do not rewrite its historical decision.

Gate and analyzer coverage are independent:

• Passed: policy passed and required analyzer coverage is complete.
• Failed: a configured gate condition failed.
• Pending: analysis or the gate decision is still running.
• Inconclusive: a pass cannot be asserted because analysis is incomplete, degraded, unverified, or a completed review has no gate result.

Provider-side branch protection or merge-check configuration is required to turn the published signal into merge blocking.

Impact analysis

When impact data is available, the review detail page can visualize changed functions, callers, downstream dependencies, entry points, hotspots, and blast radius. This interactive graph is a dashboard feature. Provider comments contain the review summary and supported annotations; they do not currently receive the dashboard’s full interactive or Mermaid visualization.

AI chat

Completed reviews can open an AI chat using the review summary, issue count, risk score, and selected category context. Conversation continuity is maintained in the current browser session; it is not a durable project knowledge base. Treat suggestions as guidance and verify them against the repository and tests.

Finding delivery

Delivery is provider-specific:

Fix text may be included with a finding. It is reviewer guidance that must be checked against repository context and tests; the provider delivery contract does not automatically apply code changes.

What the review comment says

The summary comment posted on the pull request is built from the stored review, top to bottom:

1. Risk score and, when any finding carries exploit signal, an Exploit signal line: how many are on the CISA KEV list, how many are exposed secrets, how many are critical. Those findings are listed first.
2. Analyzer coverage: every analyzer that did not fully run on the exact pull-request head, with the reason (timed out, unavailable, skipped, partial). A degraded review never reads as a pass. This section also names the repository configuration in effect (see Repository Configuration) and what the pull request merges into: the target branch’s latest scan, its age and coverage state, open critical and high findings, known-exploited findings and findings past the organisation’s SLA. A target branch that has never been scanned is reported as unknown, never as clean. The same sentence appears in the GitHub check-run summary, the Bitbucket Code Insights report and the review page in the dashboard.
3. Summary and a collapsed Walkthrough: each changed file with its change size, finding count and highest-signal finding.
4. Issues found, ordered by exploit signal (KEV, EPSS, exposed secrets), then severity. KEV and secret findings are tagged.
5. Deterministic checks: secrets and dangerous patterns from the diff itself.

The gate signal (check run, commit status or Code Insights report) leads with the exploitable count when there is one, ahead of risk and severity counts. The target branch’s state is reported there too; it never blocks the review on its own. A target branch with no scan in the last seven days is scanned automatically, at most once a day and only within the organisation’s plan limit, so this posture does not go stale.

Suppression and feedback

Suppression preserves the original analyzer severity, description, and suggestion. It records structured suppression state, the human actor, timestamp, and required reason in the audit trail. Suppressed findings are hidden by default, can be included in the review-detail findings view, and can be restored through the audited restore action.

Accept, reject, and false-positive actions are aggregated on Feedback Insights. Category totals and rates cover the organization’s full recorded feedback history; the event table is a bounded list of the most frequent category/severity/action groups. These aggregates are reporting only: they do not retrain the model, tune prompts, or automatically suppress similar future findings.

Saved PR Review settings

The automatic-review master switch, open/update/reopen event switches, and target-branch patterns are enforced before webhook dispatch. Path exclusions are enforced only by the review worker after it obtains the complete authoritative provider file list. If the worker cannot establish a complete list, the review fails closed; if every changed file is excluded, it records a durable no-scope result instead of silently skipping the review. Diff and repository context are independently bounded; file, part, and byte-budget omissions are stored with bounded identities plus authoritative record/unique-file totals, and make coverage partial. Enabled organization custom rules are loaded for AI analysis. The current shared AI response records submitted and response-accepted counts, but does not claim individual rules were evaluated until a strict per-rule outcome exists. A custom-rule finding is accepted only when it points to an added line and uses the exact enabled custom:<id> identifier; it remains candidate evidence rather than proof of every rule’s execution. Resolution failures fail closed, and unavailable or unverified outcomes are explicitly recorded. The merge-block preference records intent; actual merge blocking requires the CodeStax signal to be required in provider branch protection or merge checks.