BootstrapFewShot

Name: PromptC
Rating: 4.8 (50 reviews)
Author: Osamudiamen E. Osazuwa

Compiler that finds optimal few-shot examples through evaluation.

Constructor

class BootstrapFewShot<O> {
  constructor(evaluator: Evaluator<O>)
}

Parameter	Type	Description
evaluator	Evaluator<O>	Function to score predictions against ground truth

compile()

async compile<I, O>(
  program: Program<I, O>,
  trainset: Example<I, O>[],
  options?: CompileOptions
): Promise<CompilationResult<I, O>>

Options

Option	Type	Default	Description
candidates	number	10	Number of candidate prompts to try
concurrency	number	5	Parallel evaluations
examplesPerCandidate	number	3	Few-shot examples per prompt
validationSplit	number	0.3	Fraction of data for validation
seed	number	undefined	Random seed for reproducibility
earlyStopThreshold	number	0	Skip candidates below this score
budget.maxTokens	number	undefined	Token budget limit
budget.onBudgetWarning	function	undefined	Called when nearing budget
onProgress	function	undefined	Progress callback

Return Value

interface CompilationResult<I, O> {
  meta: {
    score: number;
    compiledAt: string;
    strategy: "BootstrapFewShot";
    tokenUsage: {
      inputTokens: number;
      outputTokens: number;
      totalTokens: number;
      calls: number;
    };
  };
  config: {
    instructions: string;
    fewShotExamples: Example<I, O>[];
  };
}

estimateCost()

estimateCost(
  trainsetSize: number,
  options?: Partial<CompileOptions>
): { estimatedCalls: number; estimatedTokens: number }

Example

import { BootstrapFewShot, exactMatch } from "@mzhub/promptc";

const compiler = new BootstrapFewShot(exactMatch());

// Estimate first
const estimate = compiler.estimateCost(trainset.length, { candidates: 20 });
console.log("Estimated tokens:", estimate.estimatedTokens);

// Compile
const result = await compiler.compile(program, trainset, {
  candidates: 20,
  concurrency: 5,
  earlyStopThreshold: 0.3,
  budget: {
    maxTokens: 50000,
    onBudgetWarning: (used, max) => console.warn(`Budget: ${used}/${max}`)
  },
  onProgress: ({ candidatesEvaluated, totalCandidates, currentBestScore }) => {
    console.log(`${candidatesEvaluated}/${totalCandidates} - Best: ${currentBestScore}`);
  }
});

console.log("Best score:", result.meta.score);
console.log("Total tokens:", result.meta.tokenUsage.totalTokens);

Early Stopping

Use earlyStopThreshold to skip candidates that are performing poorly early, saving tokens and time.