Custom Rules

Custom rules add organization-specific policy checks to CodeStax’s AI pull-request analysis. The implemented object is deliberately small: an organization-scoped rule_text, a configured severity, an enabled state, and a generated numeric ID. These shipped natural-language rules are AI-only; loading an enabled rule from the database does not mean it was evaluated.

Create a rule

Go to Dashboard → PR Reviews → Custom Rule Library, then select Add Rule.

Example:

Rule description: API handlers must validate authentication before reading customer data
Severity: high

Write one observable requirement per rule. Rule text is treated as untrusted policy data; it does not become a system instruction.

Evaluation contract

For each pull-request review:

1. The worker resolves the enabled rules for the review’s organization in a stable ID order.
2. If rule resolution, validation, or bounded-context checks fail, the review fails closed instead of claiming that the rules were evaluated.
3. The AI analysis receives each rule’s text, generated identifier (custom:<id>), and configured severity.
4. A custom-rule finding is accepted only when its file and line point to an added line in the authoritative pull-request diff and its rule identifier exactly matches an enabled custom:<id> value.

Each review stores separate bounded counts for rules loaded, eligible, evaluated, matched, and skipped, plus explicit skip reasons. An unavailable or timed-out AI provider records every eligible AI-only rule as skipped and evaluated count 0; it never reports those rules as evaluated. A no-scope review records loaded rules as skipped with no_in_scope_changes. The current schema has no deterministic custom-rule type. If one is added later, it must record positive execution evidence before it can increment the evaluated count. 5. The configured severity is authoritative; model-supplied severity cannot override it. 6. The review stores immutable, non-sensitive snapshot evidence: IDs, custom:<id> identifiers, severities, hashes of rule text, total count, completeness, snapshot version, and snapshot hash. Raw rule text is excluded from that evidence.

This grounding applies to custom-rule findings specifically. Repository-head scanners can still report findings outside changed lines, as described in PR Reviews.

Enable, disable, and delete

• Enabled: included in future AI pull-request analyses.
• Disabled: retained in the organization library but excluded from future analyses.
• Deleted: removed from the library. Historical review evidence remains an immutable hash-based snapshot and is not rewritten.

Only principals with the server-provided manage_rules capability can create, toggle, or delete rules. Other users receive read-only access.

Current boundaries

The current contract does not include:

• a separate rule name or category;
• language, path, repository, or file-pattern scope;
• organization/repository inheritance or overrides;
• editing a saved rule;
• testing a rule against a selected repository;
• importing rule packs; or
• automatically applying remediation.

To change rule text or severity, delete the rule and create a replacement. Finding remediation is text guidance for a reviewer to validate against repository context and tests.

Safety limits

The create API accepts rule text from 10 to 1,000 characters. The review worker evaluates at most 50 enabled rules and at most 20,000 total rule-text characters in one review context. Exceeding an enabled-rule bound causes policy resolution to fail closed; rules are not silently dropped.

There is no documented per-plan custom-rule entitlement in this contract. Confirm product entitlements separately before presenting a plan-specific limit.