Integrate LlamaIndex with the Respan gateway
Overview
The Respan LlamaIndex integration leverages LlamaIndex's OpenAI LLM interface to route calls through the Respan gateway. Access multiple providers through a unified API with automatic logging and observability.
Switch between models like GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro by changing the model parameter. All standard LlamaIndex features - queries, retrievals, and agent steps - work natively.
How it works
Install llama-index-llms-openai and configure the OpenAI LLM with api_base set to https://api.keywordsai.co/api/ and your Respan API key.
Use extra_body to pass Respan-specific parameters for user tracking, fallback models, and custom metadata. Every request is logged with full token usage and cost information.
python
from llama_index.llms.openai import OpenAI
llm = OpenAI(
api_base="https://api.keywordsai.co/api/",
api_key="YOUR_RESPAN_API_KEY",
model="gpt-4o",
)
response = llm.complete("Explain the RAG pattern in 3 sentences.")