🧠 AI Assistant
Test Generation

Test Generation — nat ai generate-tests

Quick use:

nat ai generate-tests --endpoint /users/{id} --spec openapi.yaml

What it does

nat ai generate-tests takes a specific endpoint (or a full spec) and generates ready-to-run test cases for it. Each test case includes:

  • A description of what it tests
  • The exact request to send (method, path, headers, body)
  • Expected results and failure conditions
  • Severity classification

This is different from nat ai plan — planning tells you what to test, generation gives you how to test it.

Command options

FlagDescriptionDefault
--endpointTarget endpoint path (e.g. /users/{id})—
--specPath or URL to OpenAPI/Swagger spec—
--typeTest type: security, functional, bothboth
--countNumber of test cases to generate10
--outputOutput file pathstdout
--contextExtra context string to guide generation—

Example output

{
  "endpoint": "GET /api/v1/users/{id}",
  "generated_at": "2024-01-15T10:30:00Z",
  "test_cases": [
    {
      "id": "tc_001",
      "name": "BOLA — access another user's record",
      "type": "security",
      "severity": "critical",
      "description": "Attempt to access a different user's record by manipulating the id parameter",
      "request": {
        "method": "GET",
        "path": "/api/v1/users/99999",
        "headers": { "Authorization": "Bearer <token_for_user_1>" }
      },
      "expected": {
        "status": 403,
        "body_must_not_contain": ["email", "phone", "address"]
      },
      "failure_condition": "Returns 200 with another user's data"
    },
    {
      "id": "tc_002",
      "name": "Auth bypass — unauthenticated access",
      "type": "security",
      "severity": "critical",
      "description": "Access the endpoint without any Authorization header",
      "request": {
        "method": "GET",
        "path": "/api/v1/users/1"
      },
      "expected": { "status": 401 },
      "failure_condition": "Returns 200 or 403 without proper 401"
    },
    {
      "id": "tc_003",
      "name": "Functional — valid user retrieval",
      "type": "functional",
      "severity": "info",
      "description": "Retrieve the authenticated user's own record",
      "request": {
        "method": "GET",
        "path": "/api/v1/users/1",
        "headers": { "Authorization": "Bearer <token_for_user_1>" }
      },
      "expected": {
        "status": 200,
        "body_must_contain": ["id", "email"]
      }
    }
  ]
}

Custom test scenarios

Pass additional context with --context to guide the AI toward your specific threat model or business logic:

nat ai generate-tests \
  --endpoint /api/v1/payments \
  --spec openapi.yaml \
  --type security \
  --context "Payment amounts are integers in cents. Negative values should be rejected. Users cannot initiate payments on behalf of other users."

The more context you provide about your business rules, the more targeted the generated test cases will be. Include information about user roles, expected input ranges, and known edge cases.

Want to just scan? Quick Scan guide →

Was this helpful?
Test PlanningScan Interpretation