Skip to content

LangChain Adapter — Quick-Start

Not verified by current release evidence. The first-class integrations are Claude Code and Codex, and the tested adapters are the OpenAI SDK, Anthropic SDK and LiteLLM; this page describes an expected setup for LangChain.

Route your LangChain applications through the TokenPak proxy to try request records and cost tracking.

Why Route Through TokenPak?

  • Cost tracking: Per model, per session and per agent, in a local SQLite store
  • Explicit context tools: The default proxy preserves conversation turns. Explicit context tools can reduce eligible content; routing a request through the proxy does not by itself compress it.
  • Usage and cache attribution: Inspect recorded usage with tokenpak savings and provider-cache attribution with tokenpak status --tip-cache

Prerequisites

  1. TokenPak proxy running locally 

    tokenpak serve &
# Proxy starts on http://localhost:8766 by default

  2. LangChain and anthropic adapter installed 

    pip install langchain langchain-anthropic anthropic

  3. ANTHROPIC_API_KEY set 

    export ANTHROPIC_API_KEY="sk-ant-..."

Quick Start (10 lines)

from langchain_anthropic import ChatAnthropic

# Point ChatAnthropic at the TokenPak proxy instead of Anthropic directly
llm = ChatAnthropic(
    model="claude-sonnet-4-6",
    base_url="http://localhost:8766/v1",  # TokenPak proxy endpoint
    api_key="sk-ant-..."  # Proxy forwards this to Anthropic
)

# Use it normally — proxy handles everything behind the scenes
response = llm.invoke("What is Python good for?")
print(response.content)

That's it. All traffic flows through the proxy automatically.

Configuration

Environment Variables

Variable Default Purpose
ANTHROPIC_BASE_URL https://api.anthropic.com/v1 Overrides LLM endpoint (set to proxy)
ANTHROPIC_API_KEY none API key (proxy forwards this)
TOKENPAK_PROXY_URL http://localhost:8766 Proxy address if not default
TOKENPAK_VAULT_PATH none Path to vault blocks for injection

Passing Config Directly

from langchain_anthropic import ChatAnthropic
import os

llm = ChatAnthropic(
    model="claude-sonnet-4-6",
    base_url=os.getenv("TOKENPAK_PROXY_URL", "http://localhost:8766/v1"),
    api_key=os.getenv("ANTHROPIC_API_KEY"),
    timeout=30,  # request timeout in seconds
)

Verification — Check the Proxy

After making a few requests, verify they went through TokenPak:

# View proxy stats (per-session counters + cache attribution)
curl http://localhost:8766/stats | python3 -m json.tool

# View cache-hit metrics
curl http://localhost:8766/cache-stats | python3 -m json.tool

# The cache-stats endpoint reports keys such as:
# {
#   "total_requests": 5,
#   "cache_hits": 1,
#   "hit_rate": 0.2
# }

The exact fields returned depend on your TokenPak version; run the commands above to see the current shape. If you see request counts climbing, traffic is flowing correctly through the proxy.

Common Errors & Fixes

❌ Connection refused / Cannot connect to proxy

Cause: Proxy not running or listening on wrong port.

Fix: 

# Start proxy in background
tokenpak serve &

# Verify it's listening
curl http://localhost:8766/health
# Should return: {"status": "ok"}

❌ Authentication failed / Invalid API key

Cause: API key not set or proxy can't forward it.

Fix: 

# Check key is exported
echo $ANTHROPIC_API_KEY  # Should print your key, not be empty

# Or pass explicitly
llm = ChatAnthropic(
    model="claude-sonnet-4-6",
    base_url="http://localhost:8766/v1",
    api_key="sk-ant-YOUR_KEY_HERE"  # Explicit > env var for debugging
)

❌ 404 Not Found when calling proxy

Cause: Endpoint path missing /v1 suffix or proxy address wrong.

Fix: 

# ✅ CORRECT
base_url="http://localhost:8766/v1"

# ❌ WRONG
base_url="http://localhost:8766"  # Missing /v1

❌ Requests not showing up in proxy stats

Cause: Traffic bypassing proxy or using different endpoint.

Fix: 

# Check what URL LangChain is actually calling
import logging
logging.basicConfig(level=logging.DEBUG)
logging.getLogger("httpx").setLevel(logging.DEBUG)

# Now make a request — debug logs will show the URL being called
llm.invoke("test")

# Should see requests to http://localhost:8766/...

Full Example Application

"""
Complete example: chat app routing through TokenPak proxy
"""
import os
from langchain_anthropic import ChatAnthropic
from langchain_core.messages import HumanMessage, SystemMessage

# Initialize with proxy endpoint
llm = ChatAnthropic(
    model="claude-sonnet-4-6",
    base_url=os.getenv("TOKENPAK_PROXY_URL", "http://localhost:8766/v1"),
    api_key=os.getenv("ANTHROPIC_API_KEY"),
    timeout=60,
)

# Build conversation
messages = [
    SystemMessage(content="You are a helpful Python assistant."),
    HumanMessage(content="Write a function that checks if a number is prime."),
]

# Call through proxy
response = llm.invoke(messages)

print("Assistant:", response.content)

# Check proxy recorded it
import requests
stats = requests.get("http://localhost:8766/cache-stats").json()
print(f"\nProxy stats: {stats['total_requests']} requests processed")

Context tools

The default proxy preserves conversation turns. Explicit context tools can reduce eligible content; routing a request through the proxy does not by itself compress it.

Troubleshooting

  • Proxy won't start: Check port 8766 isn't in use (lsof -i :8766)
  • Can't import langchain_anthropic: Run pip install langchain-anthropic --upgrade
  • Requests timing out: Increase timeout in ChatAnthropic(timeout=120)
  • Proxy crashes with errors: Check logs: tail -f /tmp/tokenpak-proxy.log

Next Steps