DSPy - Helicone OSS LLM Observability

Introduction#

DSPy is a declarative framework for building modular AI software with structured code instead of brittle prompts, offering algorithms that compile AI programs into effective prompts and weights for language models across classifiers, RAG pipelines, and agent loops.

Integration Steps#

Create an account and generate an API key

Log into Helicone or create an account. Once you have an account, you can generate an API key here.

Set up your Helicone API key in your .env file

Create a .env file in your project.

HELICONE_API_KEY=sk-helicone-...

Install DSPy

Install required dependencies

Python

pip install dspy

Start using DSPy with Helicone

View requests in the Helicone dashboard

Python

import dspyimport osfrom dotenv import load_dotenvload_dotenv()# Configure DSPy to use Helicone AI Gatewaylm = dspy.LM(  'gpt-4o-mini',  # or any other model from the Helicone model registry  api_key=os.getenv('HELICONE_API_KEY'),  api_base='https://ai-gateway.helicone.ai/')dspy.configure(lm=lm)print(lm("Hello, world!"))

You can find all 100+ supported models at helicone.ai/models.

Verify your requests in Helicone

With the above setup, any calls to DSPy will automatically be logged and monitored by Helicone. Review them in your Helicone dashboard.

Tip

While you're here, why not give us a star on GitHub? It helps us a lot!

Complete Working Examples#

Basic Chain of Thought#

Python

import dspyimport osfrom dotenv import load_dotenvload_dotenv()# Configure Helicone AI Gatewaylm = dspy.LM(    'gpt-4o-mini',    api_key=os.getenv('HELICONE_API_KEY'),    api_base='https://ai-gateway.helicone.ai/v1')dspy.configure(lm=lm)# Define a moduleqa = dspy.ChainOfThought('question -> answer')# Run inferenceresponse = qa(question="How many floors are in the castle David Gregory inherited?")print('Answer:', response.answer)print('Reasoning:', response.reasoning)

Custom Generation Configuration#

Configure temperature, max_tokens, and other parameters:

Python

import dspyimport osfrom dotenv import load_dotenvload_dotenv()# Configure with custom generation parameterslm = dspy.LM(    'gpt-4o-mini',    api_key=os.getenv('HELICONE_API_KEY'),    api_base='https://ai-gateway.helicone.ai/v1',    temperature=0.9,    max_tokens=2000)dspy.configure(lm=lm)# Use with any DSPy modulepredict = dspy.Predict("question -> creative_answer")response = predict(question="Write a creative story about AI")print(response.creative_answer)

Tracking with Custom Properties#

Add custom properties to track and filter your requests in the Helicone dashboard:

Python

import dspyimport osfrom dotenv import load_dotenvload_dotenv()# Configure with custom Helicone headerslm = dspy.LM(    'gpt-4o-mini',    api_key=os.getenv('HELICONE_API_KEY'),    api_base='https://ai-gateway.helicone.ai/v1',    extra_headers={        # Session tracking        'Helicone-Session-Id': 'dspy-example-session',        'Helicone-Session-Name': 'Question Answering',        # User tracking        'Helicone-User-Id': 'user-123',        # Custom properties for filtering        'Helicone-Property-Environment': 'production',        'Helicone-Property-Module': 'chain-of-thought',        'Helicone-Property-Version': '1.0.0'    })dspy.configure(lm=lm)# Use normallyqa = dspy.ChainOfThought('question -> answer')response = qa(question="What is DSPy?")print(response.answer)

Helicone Prompts Integration#

Use Helicone Prompts for centralized prompt management with DSPy signatures:

Python

import dspyimport osfrom dotenv import load_dotenvload_dotenv()# Configure with prompt parameterslm = dspy.LM(    'gpt-4o-mini',    api_key=os.getenv('HELICONE_API_KEY'),    api_base='https://ai-gateway.helicone.ai/v1',    extra_body={        'prompt_id': 'customer-support-prompt-id',        'version_id': 'version-uuid',        'environment': 'production',        'inputs': {            'customer_name': 'Sarah',            'issue_type': 'technical'        }    })dspy.configure(lm=lm)

Note

Learn more about Prompts with AI Gateway.

Advanced Features#

Rate Limiting#

Configure rate limits for your DSPy applications:

Python

lm = dspy.LM(    'gpt-4o-mini',    api_key=os.getenv('HELICONE_API_KEY'),    api_base='https://ai-gateway.helicone.ai/v1',    extra_headers={        'Helicone-Rate-Limit-Policy': 'basic-100'    })

Caching#

Enable intelligent caching to reduce costs:

Python

lm = dspy.LM(    'gpt-4o-mini',    api_key=os.getenv('HELICONE_API_KEY'),    api_base='https://ai-gateway.helicone.ai/v1',    cache=True  # DSPy's built-in caching works with Helicone)

Session Tracking for Multi-Turn Conversations#

Track entire conversation flows in DSPy programs:

Python

import uuidsession_id = str(uuid.uuid4())lm = dspy.LM(    'gpt-4o-mini',    api_key=os.getenv('HELICONE_API_KEY'),    api_base='https://ai-gateway.helicone.ai/v1',    extra_headers={        'Helicone-Session-Id': session_id,        'Helicone-Session-Name': 'Customer Support',        'Helicone-Session-Path': '/support/chat'    })dspy.configure(lm=lm)# All calls in this session will be grouped togetherqa = dspy.ChainOfThought('question -> answer')# Multiple turnsresponse1 = qa(question="What is your return policy?")response2 = qa(question="How long does shipping take?")response3 = qa(question="Do you ship internationally?")# View the full conversation in Helicone Sessions