Functional Testing, Reimagined

NAT brings the same autonomous, multi-agent intelligence that powers its API security testing to the world of functional testing. A single run() call orchestrates a team of BDI (Belief-Desire-Intention) agents that execute real browser interactions, detect visual regressions, scan for accessibility violations, and measure Core Web Vitals — producing a unified report across all four dimensions.

Four Capabilities, One Test Run

🧪 Browser-Based Functional Testing

The BrowserExecutorAgent launches a headless Chromium browser via Playwright and executes your test scenarios as real user interactions. At each step it captures:

• DOM snapshots — full serialized document state for structural assertions
• Screenshots — pixel-perfect renders saved as PNG baselines or comparison targets
• Console logs & network traffic — surfaced in the unified report for debugging failures
• Interaction traces — click, fill, navigate, and wait actions are logged with timestamps

This gives you the same fidelity as a human tester without the manual overhead.

👁️ Visual Regression Detection

The VisualRegressionAgent compares each screenshot captured during the run against a stored baseline using pixel-diff analysis. It reports:

• Exact pixel difference counts and percentages per page/component
• Side-by-side diff images highlighting changed regions
• Configurable thresholds — set a tolerance in pixels or percentage before a diff becomes a failure
• Automatic baseline creation on the first run; subsequent runs compare against the stored baseline

No more "it looks fine on my machine" — every deploy is checked against a visual contract.

♿ WCAG Accessibility Scanning

The AccessibilityScannerAgent audits every page against the following WCAG 2.1 rules, covering key Success Criteria:

Violations are reported with element selectors, WCAG level (A/AA), and remediation guidance.

⚡ Core Web Vitals Performance Testing

The PerformanceTestingAgent measures five key performance metrics against Google's recommended thresholds:

Each metric receives a color-coded score (🟢 Good / 🟡 Needs Improvement / 🔴 Poor) that rolls up into a weighted performance score in the unified report.

How It Works

The FunctionalTestOrchestrator coordinates all four agents in a single, sequential pipeline. You call run() once and the orchestrator handles everything:

# nat-engine is the PyPI package; the Python import uses the `mannf` namespace
from mannf.core.functional_orchestrator import FunctionalTestOrchestrator
 
orchestrator = FunctionalTestOrchestrator(
    base_url="https://your-app.example.com",
    scenarios=[
        {"name": "Homepage load", "url": "/"},
        {"name": "Login flow", "url": "/login", "actions": [
            {"type": "fill", "selector": "#email", "value": "user@example.com"},
            {"type": "fill", "selector": "#password", "value": "secret"},
            {"type": "click", "selector": "button[type=submit]"},
            {"type": "wait_for", "selector": ".dashboard"},
        ]},
    ]
)

results = orchestrator.run()

print(results.summary())
# Tests passed:    12 / 14
# Visual diffs:    1 (homepage: 23px change in header)
# A11Y violations: 3 (2× missing alt text, 1× missing label)
# Perf score:      87 / 100  (LCP: 2.1s ✅  CLS: 0.04 ✅  TBT: 310ms ⚠️)

Unified Reporting

Every test run produces two output files:

The UnifiedReportGenerator produces both files from the same result object:

from mannf.core.reporting.unified_report import UnifiedReportGenerator
 
generator = UnifiedReportGenerator(results)
generator.write_html("report.html")
generator.write_json("report.json")

The HTML report includes:

• ✅ Passed / ❌ Failed status for each functional scenario
• Side-by-side baseline vs. current screenshots for visual diffs
• Accessibility violation table with WCAG references and element selectors
• Core Web Vitals gauge charts with Google threshold overlays
• A top-level pass/fail badge suitable for embedding in CI/CD summaries

NAT vs. Other Testing Tools

Quick Start

Python SDK

# nat-engine is the PyPI package name; the Python import uses the `mannf` namespace
pip install nat-engine

from mannf.core.functional_orchestrator import FunctionalTestOrchestrator
from mannf.core.reporting.unified_report import UnifiedReportGenerator
 
# Configure
orchestrator = FunctionalTestOrchestrator(
    base_url="https://your-app.example.com",
    scenarios=[{"name": "Homepage", "url": "/"}]
)
 
# Run all four agents in one call
results = orchestrator.run()
 
# Generate reports
generator = UnifiedReportGenerator(results)
generator.write_html("report.html")
generator.write_json("report.json")

REST API

curl -X POST https://api.nat-testing.io/api/v1/functional/run \
  -H "X-API-Key: $NAT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "base_url": "https://your-app.example.com",
    "scenarios": [{"name": "Homepage", "url": "/"}]
  }'

CLI

nat functional --url https://your-app.example.com --report html

Get Started

Questions? See the FAQ or email hello@nat-testing.io.