InstructionRewrite

Name: PromptC
Rating: 4.8 (50 reviews)
Author: Osamudiamen E. Osazuwa

Compiler that uses LLM to generate and test instruction variations.

Constructor

class InstructionRewrite<O> {
  constructor(evaluator: Evaluator<O>)
}

compile()

async compile<I, O>(
  program: Program<I, O>,
  trainset: Example<I, O>[],
  options: InstructionRewriteOptions
): Promise<CompilationResult<I, O>>

Options

Option	Type	Required	Description
provider	LLMProvider	Yes	LLM to generate instruction variations
instructionVariations	number	No (5)	Number of instruction rewrites
candidates	number	No (5)	Few-shot combinations per instruction
concurrency	number	No (5)	Parallel evaluations
examplesPerCandidate	number	No (3)	Examples per prompt
validationSplit	number	No (0.3)	Validation data fraction
seed	number	No	Random seed
budget	object	No	Token budget settings
onProgress	function	No	Progress callback

Provider Required

InstructionRewrite requires a provider to generate variations. This adds one extra API call at the start of compilation.

How It Works

InstructionRewrite:

Takes your schema description as the base instruction
Uses an LLM to generate N variations (different wording, tone, structure)
Tests each variation with different few-shot example combinations
Returns the best-performing instruction + examples

Example

import { 
  InstructionRewrite, 
  exactMatch, 
  createProvider 
} from "@mzhub/promptc";

// Provider for generating variations
const optimizerProvider = createProvider("openai", {
  apiKey: process.env.OPENAI_API_KEY,
  defaultModel: "gpt-4o-mini"
});

const compiler = new InstructionRewrite(exactMatch());

const result = await compiler.compile(program, trainset, {
  provider: optimizerProvider,
  instructionVariations: 5,  // Generate 5 instruction variations
  candidates: 5,             // 5 few-shot combos per variation = 25 total
  concurrency: 3,
  onProgress: ({ currentBestScore }) => {
    console.log("Best:", currentBestScore);
  }
});

// The optimized instruction
console.log("Instructions:", result.config.instructions);

// Might be different from your original schema.description!
// e.g., "You are an expert at extracting names. Given text, 
//        identify all proper names of people mentioned..."

estimateCost()

estimateCost(
  trainsetSize: number,
  options?: Partial<InstructionRewriteOptions>
): { estimatedCalls: number; estimatedTokens: number }

const estimate = compiler.estimateCost(trainset.length, {
  instructionVariations: 5,
  candidates: 5
});

// Note: This includes the instruction generation call
console.log("Estimated calls:", estimate.estimatedCalls);