OpenAI Realtime API - Helicone OSS LLM Observability

Warning

This integration has been removed. The Helicone Realtime WebSocket proxy at wss://api.helicone.ai/v1/gateway/oai/realtime is no longer served, and the instructions below will not work. Use the AI Gateway for LLM proxying and observability. This page is kept for reference only.

OpenAI's Realtime API enables low-latency, multi-modal conversational experiences with support for text and audio as both input and output.

By integrating with Helicone, you can monitor performance, analyze interactions, and gain valuable insights into your real-time conversations.

How to Integrate#

Create an account and generate an API key

Log into Helicone or create an account. Once you have an account, you can generate an API key here.

Set up your Helicone API key in your .env file

 // For OpenAI OPENAI_API_KEY=<your-openai-api-key> HELICONE_API_KEY=<your-helicone-api-key> // For Azure AZURE_API_KEY=<your-azure-api-key> AZURE_RESOURCE=<your-azure-resource> AZURE_DEPLOYMENT=<your-azure-deployment> HELICONE_API_KEY=<your-helicone-api-key>

Configure WebSocket connection

You can connect to the Realtime API through Helicone using either OpenAI or Azure as your provider.

import WebSocket from "ws";import { config } from "dotenv";config();const url = "wss://api.helicone.ai/v1/gateway/oai/realtime?model=[MODEL_NAME]"; // gpt-4o-realtime-preview-2024-12-17const ws = new WebSocket(url, {  headers: {    "Authorization": `Bearer ${process.env.OPENAI_API_KEY}`,    "Helicone-Auth": `Bearer ${process.env.HELICONE_API_KEY}`,    // Optional Helicone properties    "Helicone-Session-Id": `session_${Date.now()}`,    "Helicone-User-Id": "user_123"  },});ws.on("open", function open() {  console.log("Connected to server");  ws.send(JSON.stringify({    type: "session.update",    session: {      modalities: ["text", "audio"],      instructions: "You are a helpful AI assistant...",      voice: "alloy",      input_audio_format: "pcm16",      output_audio_format: "pcm16",    }  }));});

Handle WebSocket events

ws.on("message", function incoming(message) {  try {    const response = JSON.parse(message.toString());    console.log("Received:", response);    // Handle specific event types    switch (response.type) {      case "input_audio_buffer.speech_started":        console.log("Speech detected!");        break;      case "input_audio_buffer.speech_stopped":        console.log("Speech ended. Processing...");        break;      case "conversation.item.input_audio_transcription.completed":        console.log("Transcription:", response.transcript);        break;      case "error":        console.error("Error:", response.error.message);        break;    }  } catch (error) {    console.error("Error parsing message:", error);  }});ws.on("error", function error(err) {  console.error("WebSocket error:", err);});// Handle cleanupprocess.on("SIGINT", () => {  console.log("\nClosing connection...");  ws.close();  process.exit(0);});

Verify your requests in Helicone

With the above setup, any calls to OpenAI Realtime API will automatically be logged and monitored by Helicone. Review them in your Helicone dashboard.

This step-by-step guide covers function calling, response formatting and monitoring with Helicone.

Trace, replay, and debug LLM sessions

Learn how to replay and modify LLM sessions using Helicone to optimize your AI agents and improve their performance.

How to Integrate#

Related Guides#