Writing Tests
Writing Good Conditions
The condition field is the most important part of a test. The core principle: assert behavior, not implementation.
A good condition describes a property that should be true about your codebase, not how that property is implemented. The test: would this condition still be valid after a complete refactor that preserves the same functionality?
Behavioral vs. Implementation-Focused
| Implementation-focused (fragile) | Behavioral (robust) |
|---|---|
Functions must wrap errors using fmt.Errorf("desc: %w", err) | Error messages should include enough context to diagnose where the problem originated |
| Check that each exported func has a comment on the line before it | All public API surfaces should be documented |
| The CLI should wrap errors in a SetupError type and check using errors.As | Setup errors should produce a different exit code than test failures |
| There should be a shared path validation function that checks the repo root prefix | Agent tools that accept file paths must prevent path traversal outside the repo |
Guidelines
Be specific. Instead of "code should be clean," write "no function should exceed 50 lines of code, excluding comments and blank lines."
Reference concrete patterns. Name the functions, decorators, or patterns you expect. "All route handlers must call requireAuth(ctx) before accessing ctx.user" is better than "routes should be authenticated."
Name files or directories. If the invariant applies to a specific part of the codebase, mention it. "Files in src/db/ should only export functions, not raw database client instances."
State exemptions explicitly. "No any type assertions except in src/generated/ files and test fixtures."
Make conditions falsifiable. A condition is falsifiable when the agent can find concrete evidence that it does not hold. Ask yourself: "What would a failing test look like? Can I picture the agent pointing at a specific file and saying 'this violates the condition'?"
Anti-Patterns: Conditions to Avoid
Vague and subjective:
# BAD: What does "clean" mean? The agent will always pass this.
condition: "The code should be clean and well-organized"
# BAD: No objective measure of "descriptive"
condition: "Variable names should be descriptive"
# BAD: Not measurable from source code
condition: "The application should be fast"Over-specified implementation details:
# BAD: Breaks when you change the error wrapping pattern
condition: >
Functions must wrap errors using fmt.Errorf("description: %w", err).
Check that all error return paths use the %w verb.
# BAD: Requires manual updates when packages are added
condition: >
The agent package must not import internal/cli, internal/runner,
internal/output, internal/discovery, or internal/cache.Always-passing conditions:
# BAD: Some error handling exists everywhere, so this always passes.
condition: "The codebase should handle errors"
# GOOD: Specific enough that the agent can identify violations.
condition: >
Functions calling external APIs must handle timeout errors and return
them to the caller, not panic or swallow them silently.Testing existence over behavior:
# BAD: Passes if the middleware file exists, even if no route uses it.
condition: "There should be auth middleware"
# GOOD: Tests actual enforcement.
condition: >
All data-modifying route handlers must authenticate the user before
executing any database mutations.What NOT to Test with Axiomatic
Some properties are better served by specialized tools:
- Code formatting -- use
gofmt,prettier,black - Recognized code patterns -- use Semgrep or existing linter rules
- Type correctness -- use the compiler / type checker
- Single-function behavior -- use unit tests
- Circular import detection -- caught at build time
Axiomatic is best for cross-cutting behavioral properties that span multiple files and require understanding intent.
Examples by Category
Security
# axiomatic/auth-required.yml
condition: >
Every API route in app/api/ that performs a mutation (POST, PUT,
PATCH, DELETE) must verify the user session using getServerSession()
or the withAuth() wrapper before executing any business logic.
Read-only GET endpoints serving public data are exempt.
on:
- "app/api/**/*.ts"
severity: error
tags: [security, api]# axiomatic/no-secrets-in-code.yml
condition: >
No source files should contain hardcoded secrets, API keys,
passwords, or tokens. Values like "sk-", "ghp_", "password=",
and Base64-encoded credentials should not appear in source.
Environment variable references (process.env.X) are acceptable.
on:
- "src/**/*.{ts,tsx,js,jsx}"
severity: error
tags: [security]# axiomatic/sql-injection.yml
condition: >
All SQL queries must use parameterized statements or an ORM's
query builder. String concatenation or template literals to build
SQL queries with user input are not acceptable.
on:
- "src/db/**/*.ts"
severity: error
tags: [security]# axiomatic/path-traversal.yml
condition: >
Any function that accepts file paths from user input must validate
that the resolved path does not escape the allowed root directory.
Attempts to access files outside the root (e.g., "../../../etc/passwd")
must be rejected.
on:
- "src/api/**/*.ts"
severity: error
tags: [security]Architecture
# axiomatic/data-layer-boundary.yml
condition: >
No files outside of src/data/ should import from the Prisma
client directly or use prisma.* calls. All database access must
go through the repository functions exported from src/data/.
on:
- "src/**/*.ts"
severity: error
tags: [architecture]# axiomatic/no-circular-imports.yml
condition: >
Modules in src/features/ should not have circular import
dependencies. Each feature directory should depend on shared/
and lib/ but not on other feature directories.
on:
- "src/features/**/*.ts"
severity: warning
tags: [architecture]# axiomatic/cli-isolation.yml
condition: >
The CLI layer should not directly import the agent package.
All orchestration should be routed through the runner, keeping
the CLI as a thin presentation layer.
on:
- "src/cli/**/*.ts"
severity: error
tags: [architecture]Code Quality
# axiomatic/jsdoc-exports.yml
condition: >
All exported functions and classes in src/ must have JSDoc
comments. Internal (non-exported) functions are exempt.
Type-only exports (interfaces, types) do not require JSDoc.
on:
- "src/**/*.ts"
severity: warning
tags: [code-quality, documentation]# axiomatic/no-console-log.yml
condition: >
Source files should not contain console.log() calls.
console.warn() and console.error() are acceptable for error
handling. Test files and scripts/ are exempt.
on:
- "src/**/*.{ts,tsx}"
severity: warning
tags: [code-quality]# axiomatic/error-context.yml
condition: >
Error messages returned to users should include enough context
to diagnose where the problem originated. Bare "Something went wrong"
or generic error strings without request-specific details are not
acceptable in production error handlers.
on:
- "src/api/**/*.ts"
severity: warning
tags: [code-quality, error-handling]Testing
# axiomatic/component-tests.yml
condition: >
Every React component file in src/components/ should have a
corresponding test file (ComponentName.test.tsx) in the same
directory or in __tests__/. Utility components under 20 lines
are exempt.
on:
- "src/components/**/*.tsx"
severity: warning
tags: [testing]Cross-Language Examples
Python (Django/Flask)
# axiomatic/django-auth.yml
condition: >
All Django view functions that modify data must use the
@login_required decorator or check request.user.is_authenticated
before performing any write operations. Views decorated with
@csrf_exempt must have a comment explaining why.
on:
- "app/views/**/*.py"
severity: error
tags: [security, python]# axiomatic/async-cleanup.yml
condition: >
All async functions that open resources (database connections,
file handles, HTTP sessions) must use async context managers
(async with) to ensure cleanup on exceptions.
on:
- "src/**/*.py"
severity: warning
tags: [code-quality, python]Java (Spring)
# axiomatic/spring-auth.yml
condition: >
All Spring controller methods that modify state must have
@PreAuthorize or @Secured annotations. GET endpoints returning
public data are exempt.
on:
- "src/main/java/**/controller/**/*.java"
severity: error
tags: [security, java]# axiomatic/constructor-injection.yml
condition: >
Spring components should use constructor injection, not field
injection with @Autowired. Constructor injection is preferred
for testability and explicit dependencies.
on:
- "src/main/java/**/*.java"
severity: warning
tags: [code-quality, java]Rust
# axiomatic/unsafe-containment.yml
condition: >
Unsafe blocks must only appear in modules explicitly designated
for unsafe operations. Each unsafe block must have a SAFETY comment
explaining why it is necessary and why it is sound.
on:
- "src/**/*.rs"
severity: error
tags: [security, rust]Common Mistakes
Compound conditions hiding failures. "All API routes require auth AND all queries are parameterized" combines two separate properties. If one fails but the agent exhausts its budget on the first, the second might pass undetected. Split compound properties into separate tests.
Overly broad on patterns. on: ["**/*"] invalidates the cache on every file change, causing unnecessary re-runs. Scope to the relevant directories.
Not specifying exemptions. If some files are legitimately exempt (generated code, test fixtures, migration files), say so in the condition. Otherwise the agent may flag them as violations or waste budget inspecting them.