Coverage and troubleshooting

What Respan tests

Current black-box coverage includes:

Prompt injection and goal hijacking
System prompt leakage
Secret and credential disclosure
Cross-customer data exposure
Unsafe or excessive agent actions
Indirect prompt injection
Data exfiltration through links
Server-side request forgery through agent tools
Misinformation and fabricated policy
Toxic or demeaning output
Competitor endorsement and brand-safety risks

Tool-dependent tests run only when reconnaissance detects the relevant capability. For example, refund-abuse testing requires a refund capability, while cross-customer data testing requires customer lookup behavior.

Some risks cannot be fully verified from a black-box chat endpoint. Categories involving training data, response-side handling, or the retrieval and embedding layer may appear as Deeper integration required in the report.

Usage limits

Hosted-sandbox assessments are subject to your organization’s allowance, which resets with its usage period. Failed assessments do not count toward usage. If the allowance is exhausted, Respan prevents another hosted-sandbox launch until the period resets or the organization’s plan changes.

Troubleshooting

The endpoint cannot be reached

Check that:

The Base URL is correct.
The endpoint is reachable from your browser.
CORS permits the Respan application’s origin.
An HTTPS Respan page is not calling an HTTP endpoint.
Your firewall, VPN, or private network permits the request.

The endpoint returns an authentication error

Confirm that the API key is valid and that the endpoint accepts it as:

Authorization: Bearer <api-key>

The current connection form does not support custom authentication headers or request signing.

The response format is rejected

Confirm that your endpoint returns assistant text in choices[0].message.content. A URL alone is not enough; the endpoint must implement the OpenAI-compatible Chat Completions format.

The assessment stopped before completion

For a connected endpoint, keep the Respan tab open and avoid refreshing it during the run. Also confirm that individual target responses complete before the request timeout.

A category says deeper integration is required

This does not mean the category passed or failed. It means a black-box chat connection does not expose enough information to test that risk completely.

Security considerations

Reports can contain attack prompts, target responses, reconstructed instructions, secrets, or personal data. Restrict report access accordingly.
Use a narrowly scoped target credential whenever possible.
Test non-production environments before assessing a production agent.
Consider the systems and tools available to the agent, not only the language model itself, when granting authorization.