Functional Testing API

The Functional Testing API lets you start functional test runs (visual regression, accessibility, and performance), retrieve detailed results for each test type, and integrate automated quality gates into your CI/CD pipeline.

Authentication

All endpoints require your API key. The tenant is resolved automatically from the key.

X-API-Key: nat_pk_your_api_key_here

Base URL

https://api.nat-testing.io/api/v1

Functional testing is available on Pro and Team plans. Visual regression, accessibility, and performance sub-reports are only populated when "mode": "functional" (or "mode": "full") is included in the scan request.

Endpoints

Start a functional test scan

Start a new functional test scan against the specified target. Pass "mode": "functional" in the request body to enable functional testing checks, or "mode": "full" to run both security and functional tests together.

POST /api/v1/scan

Request body:

Field	Type	Required	Description
`url`	string	✅	Target application URL
`mode`	string		`"security"` (default), `"functional"`, or `"full"`
`spec_url`	string		OpenAPI/Swagger spec URL (recommended for security mode)
`auth.type`	string		`bearer`, `header`, `basic`, `oauth2`, `none`
`auth.token`	string		Bearer token value
`options.concurrency`	integer		Parallel browser/request count (default: 5)
`options.timeout`	integer		Per-page timeout in seconds (default: 30)
`options.fail_on`	string		Severity threshold for pipeline failure
`options.exclude`	string[]		URL patterns to exclude from testing

# Functional tests only
curl -X POST https://api.nat-testing.io/api/v1/scan \
  -H "X-API-Key: $NAT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://app.example.com",
    "mode": "functional",
    "options": {
      "concurrency": 3,
      "timeout": 30
    }
  }'
 
# Security + functional tests together
curl -X POST https://api.nat-testing.io/api/v1/scan \
  -H "X-API-Key: $NAT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://app.example.com",
    "mode": "full",
    "spec_url": "https://api.example.com/openapi.json"
  }'

Response 202 Accepted:

{
  "scan_id": "scan_ft_abc123xyz",
  "status": "running",
  "mode": "functional",
  "created_at": "2025-01-15T10:30:00Z"
}

Error responses:

Status	Code	Description
`402`	`QUOTA_EXCEEDED`	Monthly scan quota exhausted
`402`	`FEATURE_NOT_AVAILABLE`	Functional testing requires Pro or Team plan
`429`	`CONCURRENT_LIMIT`	Too many scans running simultaneously
`401`	`UNAUTHORIZED`	Missing or invalid API key
`400`	`INVALID_REQUEST`	Malformed request body

Get functional test results

Returns the complete functional test results for a scan. Poll until status is completed.

GET /api/v1/scan/{id}/functional-results

Path parameters:

Parameter	Type	Description
`id`	string	Scan ID returned by `POST /api/v1/scan`

curl https://api.nat-testing.io/api/v1/scan/scan_ft_abc123xyz/functional-results \
  -H "X-API-Key: $NAT_API_KEY"

Response 200 OK (completed):

{
  "scan_id": "scan_ft_abc123xyz",
  "status": "completed",
  "mode": "functional",
  "passed": 24,
  "failed": 3,
  "total": 27,
  "elapsed_seconds": 142.5,
  "agent_count": 4,
  "tasks": [
    {
      "task_id": "task_001",
      "name": "Login flow — successful authentication",
      "status": "passed",
      "duration_ms": 1240
    },
    {
      "task_id": "task_002",
      "name": "Checkout flow — payment submission",
      "status": "failed",
      "error": "Expected redirect to /confirmation, got 500 Internal Server Error",
      "duration_ms": 3200
    }
  ],
  "visual_regression": {
    "comparisons": 18,
    "diffs_detected": 2,
    "diff_threshold_percent": 0.5
  },
  "accessibility": {
    "pages_scanned": 18,
    "violations": 5,
    "compliance_score": 91.2
  },
  "performance": {
    "pages_tested": 18,
    "performance_score": 78.4
  }
}

Result fields:

Field	Type	Description
`passed`	integer	Number of functional test tasks that passed
`failed`	integer	Number of test tasks that failed
`total`	integer	Total number of test tasks executed
`elapsed_seconds`	float	Total scan duration in seconds
`agent_count`	integer	Number of AI agents used
`tasks`	array	Individual functional test task results
`visual_regression`	object	Summary of visual diff results
`accessibility`	object	Summary of accessibility audit results
`performance`	object	Summary of Core Web Vitals results

Get visual regression report

Returns the visual regression report including before/after screenshots and pixel-diff analysis for each page tested.

GET /api/v1/scan/{id}/visual-regression

Path parameters:

Parameter	Type	Description
`id`	string	Scan ID

curl https://api.nat-testing.io/api/v1/scan/scan_ft_abc123xyz/visual-regression \
  -H "X-API-Key: $NAT_API_KEY"

Response 200 OK:

{
  "scan_id": "scan_ft_abc123xyz",
  "comparisons": 18,
  "diffs_detected": 2,
  "diff_threshold_percent": 0.5,
  "pages": [
    {
      "page_url": "https://app.example.com/dashboard",
      "diff_detected": true,
      "diff_percent": 1.8,
      "screenshot_before": "https://cdn.nat-testing.io/screenshots/scan_ft_abc123xyz/dashboard_before.png",
      "screenshot_after": "https://cdn.nat-testing.io/screenshots/scan_ft_abc123xyz/dashboard_after.png",
      "diff_image": "https://cdn.nat-testing.io/screenshots/scan_ft_abc123xyz/dashboard_diff.png"
    },
    {
      "page_url": "https://app.example.com/login",
      "diff_detected": false,
      "diff_percent": 0.0,
      "screenshot_before": "https://cdn.nat-testing.io/screenshots/scan_ft_abc123xyz/login_before.png",
      "screenshot_after": "https://cdn.nat-testing.io/screenshots/scan_ft_abc123xyz/login_after.png",
      "diff_image": null
    }
  ]
}

Response fields:

Field	Type	Description
`comparisons`	integer	Total number of page comparisons performed
`diffs_detected`	integer	Number of pages where visual differences exceeded the threshold
`diff_threshold_percent`	float	Pixel difference threshold used (configurable per scan)
`pages[].diff_detected`	boolean	Whether this page exceeded the diff threshold
`pages[].diff_percent`	float	Percentage of pixels that changed
`pages[].screenshot_before`	string	URL of the baseline screenshot
`pages[].screenshot_after`	string	URL of the current run screenshot
`pages[].diff_image`	string \| null	URL of the highlighted diff image (null if no diff)

Visual regression compares each page against a baseline captured on the first scan. To update the baseline, re-run with "reset_baseline": true in the request body.

Get accessibility scan results

Returns a detailed accessibility audit including WCAG violation details and per-page compliance scores.

GET /api/v1/scan/{id}/accessibility

Path parameters:

Parameter	Type	Description
`id`	string	Scan ID

curl https://api.nat-testing.io/api/v1/scan/scan_ft_abc123xyz/accessibility \
  -H "X-API-Key: $NAT_API_KEY"

Response 200 OK:

{
  "scan_id": "scan_ft_abc123xyz",
  "pages_scanned": 18,
  "total_violations": 5,
  "compliance_score": 91.2,
  "wcag_level": "AA",
  "violations": [
    {
      "rule": "color-contrast",
      "impact": "serious",
      "wcag": "1.4.3",
      "page_url": "https://app.example.com/dashboard",
      "elements_affected": 3,
      "description": "Elements must have sufficient color contrast",
      "help_url": "https://dequeuniversity.com/rules/axe/4.8/color-contrast"
    },
    {
      "rule": "image-alt",
      "impact": "critical",
      "wcag": "1.1.1",
      "page_url": "https://app.example.com/reports",
      "elements_affected": 2,
      "description": "Images must have alternate text",
      "help_url": "https://dequeuniversity.com/rules/axe/4.8/image-alt"
    }
  ],
  "pages": [
    {
      "page_url": "https://app.example.com/dashboard",
      "violations": 3,
      "compliance_score": 88.5
    },
    {
      "page_url": "https://app.example.com/login",
      "violations": 0,
      "compliance_score": 100.0
    }
  ]
}

Violation impact levels:

Impact	Description
`critical`	Causes complete inaccessibility for some users — must fix
`serious`	Causes significant barriers — should fix
`moderate`	Causes some difficulty — consider fixing
`minor`	Best practice improvement

NAT checks against WCAG 2.1 AA by default. WCAG 2.2 and Section 508 profiles are available on the Team plan. See Functional Testing for a full list of accessibility rules.

Get performance metrics

Returns Core Web Vitals and performance scores for each page tested. Metrics are measured using real browser instrumentation.

GET /api/v1/scan/{id}/performance

Path parameters:

Parameter	Type	Description
`id`	string	Scan ID

curl https://api.nat-testing.io/api/v1/scan/scan_ft_abc123xyz/performance \
  -H "X-API-Key: $NAT_API_KEY"

Response 200 OK:

{
  "scan_id": "scan_ft_abc123xyz",
  "pages_tested": 18,
  "performance_score": 78.4,
  "pages": [
    {
      "page_url": "https://app.example.com/dashboard",
      "performance_score": 72.1,
      "lcp_ms": 2850,
      "fid_ms": 85,
      "cls": 0.12,
      "ttfb_ms": 380,
      "fcp_ms": 1200,
      "tbt_ms": 210,
      "si_ms": 1850,
      "tti_ms": 4200,
      "rating": "needs-improvement"
    },
    {
      "page_url": "https://app.example.com/login",
      "performance_score": 94.3,
      "lcp_ms": 980,
      "fid_ms": 12,
      "cls": 0.02,
      "ttfb_ms": 95,
      "fcp_ms": 620,
      "tbt_ms": 45,
      "si_ms": 780,
      "tti_ms": 1100,
      "rating": "good"
    }
  ]
}

Core Web Vitals fields:

Metric	Field	Good	Needs improvement	Poor
Largest Contentful Paint	`lcp_ms`	≤ 2500ms	≤ 4000ms	> 4000ms
First Input Delay	`fid_ms`	≤ 100ms	≤ 300ms	> 300ms
Cumulative Layout Shift	`cls`	≤ 0.1	≤ 0.25	> 0.25
Time to First Byte	`ttfb_ms`	≤ 800ms	≤ 1800ms	> 1800ms
First Contentful Paint	`fcp_ms`	≤ 1800ms	≤ 3000ms	> 3000ms
Total Blocking Time	`tbt_ms`	≤ 200ms	≤ 600ms	> 600ms
Speed Index	`si_ms`	≤ 3400ms	≤ 5800ms	> 5800ms
Time to Interactive	`tti_ms`	≤ 3800ms	≤ 7300ms	> 7300ms

rating values: "good", "needs-improvement", "poor"

Rate limiting

All functional testing endpoints share the same rate limits as the core scan API.

Plan	Requests / minute
Free	10
Pro	60
Team	200
Enterprise	1,000

Functional Testing API

Authentication

Base URL

Endpoints

Start a functional test scan

Get functional test results

Get visual regression report

Get accessibility scan results

Get performance metrics

Rate limiting

See also