SDK Reference

Privacy-first SDK

The official @tallyify/sdk package wraps any Promise-based AI call, extracts usage when the provider returns it, and sends aggregated spend metrics to Tallyify.

Tallyify SDK does not read, store or transmit provider API keys. Your OpenAI, Anthropic, Google, DeepSeek or Mistral keys remain inside your application. The SDK only sends aggregated usage metrics such as provider, model, token counts, latency, cost estimate and optional metadata.

Installation

Shell
npm install @tallyify/sdk

Quick Start

app.ts
import { Tracker } from '@tallyify/sdk';
import OpenAI from 'openai';

const tracker = new Tracker({ apiKey: process.env.TALLYIFY_API_KEY });
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const response = await tracker.track(
  openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'Hello!' }]
  }),
  {
    provider: 'openai',
    model: 'gpt-4o-mini',
    endpoint: 'chat.completions',
    metadata: { feature: 'chatbot', environment: 'production' }
  }
);

Provider Adapters

Provider adapters are explicit wrappers around known SDK methods. They do not monkey-patch global fetch, http, https, axios, or provider clients outside the object you wrap.

openai-adapter.ts
import OpenAI from 'openai';
import { trackOpenAI } from '@tallyify/sdk';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const trackedOpenAI = trackOpenAI(openai, {
  trackerApiKey: process.env.TALLYIFY_API_KEY!,
  metadata: { feature: 'chatbot', environment: 'production' }
});

const response = await trackedOpenAI.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [{ role: 'user', content: 'Hello!' }]
});

Available adapters: trackOpenAI, trackAnthropic, trackGemini, trackDeepSeek, trackMistral, trackGroq, trackAzureOpenAI, and trackBedrock. The generic tracker.track() API remains the most portable option for custom clients and unsupported methods.

advanced-adapters.ts
import { trackGroq, trackAzureOpenAI, trackBedrock } from '@tallyify/sdk';

// Groq (OpenAI-compatible)
const groq = trackGroq(new Groq({ apiKey: process.env.GROQ_API_KEY }), {
  trackerApiKey: process.env.TALLYIFY_API_KEY!
});

// Azure OpenAI (OpenAI SDK shape, Azure-hosted)
const azure = trackAzureOpenAI(azureClient, { trackerApiKey: process.env.TALLYIFY_API_KEY! });

// AWS Bedrock — wraps client.send(command); reads modelId from the command
const bedrock = trackBedrock(new BedrockRuntimeClient({ region: 'us-east-1' }), {
  trackerApiKey: process.env.TALLYIFY_API_KEY!
});

Streaming

Streaming responses are tracked automatically. When the wrapped call returns a stream, the SDK hands you back a pass-through async iterable: you consume chunks exactly as before, while the SDK accumulates token usage as the stream is read and dispatches the telemetry event when it ends. This works for both the final-usage-chunk pattern (OpenAI) and the cumulative pattern (Anthropic), via the generic track() and every adapter.

streaming.ts
// Works with the generic tracker AND every adapter — no extra config.
const stream = await tracker.track(
  openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'Write a haiku about the sea' }],
    stream: true,
    stream_options: { include_usage: true } // OpenAI: required for token counts
  }),
  { provider: 'openai', model: 'gpt-4o-mini' }
);

// Consume chunks exactly as you normally would:
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
}
// Usage is accumulated from the chunks; the telemetry event (tokens + cost)
// is dispatched automatically once the stream finishes.

Live Cost Estimation

By default the SDK fetches live prices from the Tallyify catalog (cached per process) so cost_estimate is accurate for every model — not just the small built-in fallback table. It is best-effort: if the catalog is unreachable or the key lacks the pricing:read scope, it silently falls back to bundled prices. Set dynamicPricing: false to disable it.

dynamic-pricing.ts
const tracker = new Tracker({
  apiKey: process.env.TALLYIFY_API_KEY!,
  dynamicPricing: true // default: pull live catalog prices for accurate cost estimates
});

Budget Guardrails

Add a budget to flag — or hard-stop — runaway spend. Because a call's cost is only known after it returns, this is a post-hoc tripwire: with enforce: true the SDK throws TallyifyBudgetError once a per-call or cumulative limit is crossed, so an overspend can't pass silently; otherwise it logs a warning and tags the event.

budgets.ts
import { Tracker, TallyifyBudgetError } from '@tallyify/sdk';

const tracker = new Tracker({
  apiKey: process.env.TALLYIFY_API_KEY!,
  budget: {
    maxCostPerCall: 0.50,  // USD: flag a single call above this
    maxTotalCost: 100,     // USD: cumulative cap for this Tracker instance
    enforce: true,         // throw instead of only warning
    onExceeded: (info) => console.error('Budget exceeded', info)
  }
});

try {
  await tracker.track(providerPromise, { provider: 'openai', model: 'gpt-4o' });
} catch (err) {
  if (err instanceof TallyifyBudgetError) {
    // overspend tripwire — info has { kind, limit, cost, totalCost, provider, model }
  }
}

Retry Tracking

trackWithRetries() retries a call factory with exponential backoff and records every attempt, including the failures. The successful event carries a retry_count so you can see how much retries are costing you.

retries.ts
// Pass a factory (not a promise) so each attempt is a fresh call.
const response = await tracker.trackWithRetries(
  () => openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'Hello!' }]
  }),
  {
    provider: 'openai',
    model: 'gpt-4o-mini',
    retries: 3,          // max attempts (default 3)
    retryDelayMs: 250    // exponential backoff base
  }
);
// Each attempt is tracked; events carry retry_count so you can see retry cost.

Configuration

TypeScript
const tracker = new Tracker({
  apiKey: process.env.TALLYIFY_API_KEY,
  baseUrl: 'https://api.tallyify.com',
  service: 'api-server',
  debug: false,
  dynamicPricing: true,
  budget: { maxCostPerCall: 0.5, enforce: true },
  privacy: {
    capturePrompts: false,
    captureResponses: false,
    redactSecrets: true
  }
});

Option	Type	Required	Description
`apiKey`	`string`	Yes	Your Tallyify key: `tly_live_...` or `tly_test_...`
`baseUrl`	`string`	No	Override the default API endpoint.
`service`	`string`	No	Optional service label added to metadata.
`debug`	`boolean`	No	Logs local, non-fatal telemetry dispatch errors.
`dynamicPricing`	`boolean`	No	Pull live catalog prices for accurate cost estimates. Default `true`.
`budget`	`BudgetConfig`	No	Spend guardrails: `maxCostPerCall`, `maxTotalCost`, `enforce`, `onExceeded`.

OpenAI Example

openai-example.ts
import { Tracker } from '@tallyify/sdk';
import OpenAI from 'openai';

const tracker = new Tracker({ apiKey: process.env.TALLYIFY_API_KEY });
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

const response = await tracker.track(
  openai.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'Hello!' }]
  }),
  {
    provider: 'openai',
    model: 'gpt-4o-mini',
    endpoint: 'chat.completions',
    metadata: { feature: 'chatbot', environment: 'production' }
  }
);

Anthropic Example

anthropic-example.ts
import { Tracker } from '@tallyify/sdk';
import Anthropic from '@anthropic-ai/sdk';

const tracker = new Tracker({ apiKey: process.env.TALLYIFY_API_KEY });
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

const response = await tracker.track(
  anthropic.messages.create({
    model: 'claude-3-5-sonnet-latest',
    max_tokens: 1024,
    messages: [{ role: 'user', content: 'What is 2 + 2?' }]
  }),
  { provider: 'anthropic', model: 'claude-3-5-sonnet-latest' }
);

Google Gemini Example

gemini-example.ts
import { Tracker } from '@tallyify/sdk';
import { GoogleGenerativeAI } from '@google/generative-ai';

const tracker = new Tracker({ apiKey: process.env.TALLYIFY_API_KEY });
const genAI = new GoogleGenerativeAI(process.env.GOOGLE_API_KEY);
const model = genAI.getGenerativeModel({ model: 'gemini-1.5-pro' });

const response = await tracker.track(
  model.generateContent('Tell me a fun fact about space'),
  { provider: 'google', model: 'gemini-1.5-pro' }
);

Generic Tracking

TypeScript
const response = await tracker.track(providerPromise, {
  provider: 'mistral',
  model: 'mistral-large',
  endpoint: 'chat',
  metadata: {
    feature: 'customer-support',
    environment: 'production'
  }
});

Privacy & Security

The ingestion API rejects provider-key-looking strings, and deprecated externalKeyHint values are ignored. The SDK only ever sends aggregated usage metrics — never a real provider key.

JSON
{
  "provider": "openai",
  "model": "gpt-4o-mini",
  "input_tokens": 1200,
  "output_tokens": 300,
  "total_tokens": 1500,
  "cached_tokens": 0,
  "reasoning_tokens": 0,
  "cost_estimate": 0.0023,
  "endpoint": "chat.completions",
  "latency_ms": 842,
  "status": "success",
  "metadata": {
    "feature": "customer-support",
    "environment": "production"
  }
}

The SDK does not send prompts, messages, responses, provider API keys, provider Authorization headers, or provider environment variables.

TypeScript Types

TypeScript
export interface TrackerConfig {
  apiKey: string;
  baseUrl?: string;
  service?: string;
  debug?: boolean;
  dynamicPricing?: boolean; // default true
  budget?: BudgetConfig;
  privacy?: {
    capturePrompts?: false;
    captureResponses?: false;
    redactSecrets?: true;
  };
}

export interface BudgetConfig {
  maxCostPerCall?: number;
  maxTotalCost?: number;
  enforce?: boolean;
  onExceeded?: (info: BudgetExceededInfo) => void;
}

export class TallyifyBudgetError extends Error {
  info: BudgetExceededInfo;
}

export interface TrackOptions {
  provider?: string;
  model?: string;
  endpoint?: string;
  metadata?: Record<string, string | number | boolean>;
  retryCount?: number;
}

export interface RetryOptions extends TrackOptions {
  retries?: number;       // default 3
  retryDelayMs?: number;  // default 250
  shouldRetry?: (error: unknown, attempt: number) => boolean;
}

export interface TrackEventPayload {
  provider: string;
  model?: string;
  input_tokens?: number;
  output_tokens?: number;
  total_tokens?: number;
  cached_tokens?: number;
  reasoning_tokens?: number;
  cost_estimate?: number;
  endpoint?: string;
  latency_ms?: number;
  status?: 'success' | 'error';
  retry_count?: number;
  streamed?: boolean;
  metadata?: Record<string, string | number | boolean>;
}

export interface ProviderAdapterConfig {
  trackerApiKey: string;
  baseUrl?: string;
  service?: string;
  metadata?: Record<string, string | number | boolean>;
  debug?: boolean;
  dynamicPricing?: boolean;
  budget?: BudgetConfig;
}

// Provider adapters:
// trackOpenAI, trackAnthropic, trackGemini, trackDeepSeek,
// trackMistral, trackGroq, trackAzureOpenAI, trackBedrock