Skip to main content
Respan provides LLM observability service: keeping track of all the inputs to and outputs of LLM inferences, along with any additional metrics that are calculated during the inference, such as token usage, generation time etc.

Architecture and Data Flow Overview

Cloud Infrastructure

  • Amazon Web Services (AWS) as the primary cloud service provider
  • Application hosted on Amazon Elastic Container Service (ECS)
  • Redis for event queue management
  • PostgreSQL for persistent data storage
  • ClickHouse for high-performance analytics and observability data warehousing

Data Flow

  1. Client requests are sent to our API server hosted on AWS ECS
  2. During LLM inference operations, events are generated and pushed to Redis queue
  3. Celery workers consume these events from Redis
  4. Data is batch inserted into PostgreSQL and ClickHouse

Security and Encryption Standards

  • All API communications secured via TLS 1.2+ (HTTPS)
  • Authentication credentials and API keys are hashed using SHA-256 (SHA-2 family) before storage
  • Data at rest is encrypted using AWS-managed encryption (AES-256)
  • Inter-service communication within AWS infrastructure is secured through AWS security groups
We implement multiple layers of encryption:
  • TLS 1.2+ for all data in transit
  • SHA-256 hashing for data integrity verification
  • AES-256 encryption for data at rest in our PostgreSQL databases
  • All API communications are encrypted end-to-end
All data transmission between your applications and Respan uses:
  • TLS 1.2 or higher encryption protocols
  • Certificate pinning for additional security
  • Encrypted API keys for authentication
  • Secure WebSocket connections for real-time data
Our data storage follows enterprise security standards:
  • AES-256 encryption for all data at rest
  • Regular automated backups with encryption
  • Database access controls and audit logging
  • Network isolation and VPC security groups

Security Operations

  • Regular internal security audits (monthly)
  • Weekly security testing of applications
  • Continuous monitoring via AWS CloudWatch
  • Regular code reviews (weekly)
  • Vulnerability scanning and penetration testing planned for next security roadmap phase
We maintain rigorous security testing practices:
  • Internal security audits conducted regularly
  • Weekly penetration testing by our security team
  • Automated vulnerability scanning of all infrastructure
  • Code security reviews for all deployments
Yes, we undergo regular external security assessments:
  • Third-party penetration testing annually
  • Security architecture reviews by external experts
  • Compliance audits for industry standards
  • Vulnerability assessments by certified security firms

Access Controls

  • Multi-factor authentication (MFA) required
  • Role-based access control (RBAC) with least privilege
  • Just-in-time (JIT) access for administrative functions
  • Regular access reviews and deprovisioning
We implement strict access controls:
  • Role-based access control (RBAC) for all team members
  • Multi-factor authentication (MFA) required for all accounts
  • Principle of least privilege access
  • Regular access reviews and deprovisioning
Employee access is strictly controlled:
  • No default access to customer data
  • Access only granted for specific support requests with customer approval
  • All access is logged and audited
  • Time-limited access tokens for support activities

Data Protection

Our data retention policies are designed for security and compliance:
  • Configurable retention periods based on your requirements
  • Automatic data purging after retention period expires
  • Secure data deletion using DoD 5220.22-M standards
  • Data export capabilities before deletion
Yes, you have full control over data collection:
  • Configurable logging levels and data types
  • Option to exclude sensitive data from logging
  • Custom metadata filtering capabilities
  • Real-time data masking for PII protection
Data storage locations are configurable:
  • Primary data centers in US East (Virginia) and US West (Oregon)
  • EU data residency options available
  • Data never leaves your specified geographic region
  • Compliance with local data sovereignty requirements

Incident Response

  • Dedicated incident response team with defined roles
  • Customer notification within 24 hours of any security incident
  • Detailed incident reports and remediation plans
  • Post-incident reviews and continuous improvement
We have a comprehensive incident response plan:
  • 24/7 monitoring and alerting systems
  • Dedicated security incident response team
  • Automated threat detection and response
  • Customer notification within 24 hours of confirmed incidents
Our vulnerability management process includes:
  • Continuous vulnerability scanning and assessment
  • Prioritized patching based on risk assessment
  • Coordinated disclosure for security researchers
  • Regular security updates and patches

Business Continuity

  • Recovery Time Objective (RTO): 4 hours
  • Recovery Point Objective (RPO): 1 hour
  • Automated daily backups with cross-region replication
  • Regular disaster recovery testing

Compliance & Certifications

  • SOC 2 Type II - Security, Availability, Confidentiality (Certified)
  • HIPAA - Healthcare data protection compliance
  • GDPR - European data protection compliance
  • AWS and GCP security frameworks utilized

Monitoring & Analytics

We use comprehensive monitoring and analytics across our infrastructure:
  • PostHog for product analytics and user behavior tracking
  • ClickHouse for high-performance data warehousing and analytics
  • AWS CloudWatch for infrastructure monitoring
  • Custom alerting for security events
  • Real-time dashboards for system health
  • Automated incident escalation for critical issues
Our threat detection includes:
  • Machine learning-based anomaly detection
  • Real-time log analysis and correlation
  • Network traffic monitoring and analysis
  • Behavioral analysis for unusual access patterns

Security Reports

Detailed security architecture documentation, penetration test reports, and compliance certificates are available to enterprise customers under NDA. Contact: security@respan.ai