Important: modern LLMs are trained with an instruction hierarchy. OpenAI’s IH-Challenge and model spec make explicit the priority ordering: System > Developer > User > Tool. Design system and developer instructions accordingly—put safety-critical and policy constraints in the system or developer layer so they are highest priority and resilient to lower-priority inputs (including tool outputs).

Prompt strategies

• Zero-shot: simple tasks, low token cost
• Few-shot: consistent format or tricky edge-cases; include diverse examples
• Chain-of-thought: multi-step reasoning or proofs — use sparingly in high-cost paths
• Self-consistency: sample multiple reasoning traces and take the consensus for high-stakes tasks
• ReAct / tool-guided: when the model must propose actions (tool calls); implement an orchestrator to execute and verify tool outputs
• Structured output: use schema validation (e.g., zod, JSON Schema) rather than relying on free-form parsing

Temperature guide

• 0: deterministic-style behavior for classification/extraction (note: greedy decoding still permits small variance)
• 0.3–0.5: balanced
• 0.7–1.0: creative

Workflow

Step 1: Define the task contract

Before writing any prompt, answer these questions:

type PromptContract = {
  input: string;       // What does the model receive?
  output: string;      // What should it produce?
  format: string;      // JSON, markdown, plain text?
  constraints: string[]; // What must it avoid?
  edgeCases: string[];   // How should it handle ambiguity?
  examples: Array<{ input: string; output: string }>;
};

Step 2: Write the system prompt

Focus system-level text on what must always hold (safety, privacy, legal constraints). Put product-level preferences in developer instructions. Remember: models trained on instruction-hierarchy datasets are more likely to honor these distinctions.

const systemPrompt = `You are a senior code reviewer specializing in TypeScript and React.

## Task
Review the provided code diff and return structured feedback.

## Output format
Return a JSON array of findings:
[
  {
    "severity": "critical" | "warning" | "info",
    "line": <number>,
    "message": "<concise description>",
    "suggestion": "<specific fix>"
  }
]

## Rules
- Focus on bugs, security issues, and performance problems
- Do not comment on style preferences unless they cause bugs
- If the code is correct and well-written, return an empty array []
- Never suggest changes that would break existing tests
- Limit findings to the top 5 most important issues

## Examples

Input: \`const x = data.map(d => d.name)\`
Output:
[
  {
    "severity": "warning",
    "line": 1,
    "message": "No null check on data before .map()",
    "suggestion": "Use optional chaining: data?.map(d => d.name) ?? []"
  }
]`;

Step 3: Implement structured output with OpenAI (Responses API)

• Prefer the Responses API when you need agentic primitives (tools) or an orchestrator loop; the Responses API supports custom tools and the shell/container environment for workflows where the model proposes actions and the platform executes them.
• Treat tool outputs as lower-priority content: do not place tool-provided text into system messages without sanitization and verification (OpenAI’s instruction hierarchy treats tools as lower priority than system/developer instructions).
• Use schema helpers (like zodResponseFormat) to produce machine-parseable responses and validate them server-side.

Example (OpenAI Responses + zod):

import OpenAI from "openai";
import { z } from "zod";
import { zodResponseFormat } from "openai/helpers/zod";

const FindingSchema = z.object({
  severity: z.enum(["critical", "warning", "info"]),
  line: z.number(),
  message: z.string(),
  suggestion: z.string(),
});

const ReviewSchema = z.object({
  findings: z.array(FindingSchema),
});

const client = new OpenAI();

async function reviewCode(diff: string) {
  const response = await client.responses.create({
    model: "gpt-4o",
    instructions: systemPrompt,
    input: diff,
    text: {
      format: zodResponseFormat(ReviewSchema, "code_review"),
    },
  });

  // Validate and parse on the server to guard against format drift
  return ReviewSchema.parse(JSON.parse(response.output_text));
}

Step 4: Implement with Anthropic

• Anthropic’s docs recommend explicit roles and XML-like tags to structure prompts. Use tags like <instructions>, <examples>, and <input> to reduce ambiguity in multi-component prompts.
• Include 3–5 diverse examples wrapped in <examples> / <example> tags to improve format fidelity and edge-case handling for Claude models.

Example (Anthropic pattern):

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

const system = `
<instructions>
You are a senior code reviewer specializing in TypeScript and React.
</instructions>
<examples>
  <example>
    <input>const x = data.map(d => d.name)</input>
    <output>[{"severity":"warning","line":1,"message":"No null check","suggestion":"Use optional chaining"}]</output>
  </example>
</examples>
`;

async function reviewCodeClaude(diff: string) {
  const response = await client.messages.create({
    model: "claude-opus-4-6",
    max_tokens: 2048,
    system,
    messages: [{ role: "user", content: diff }],
  });

  const text = response.content
    .filter(block => block.type === "text")
    .map(block => block.text)
    .join("");

  return JSON.parse(text);
}

Step 5: Agents and orchestrators

• When building agents, do not rely on the model to execute actions — the model should only propose tool calls. Use an orchestrator that executes the calls in a sandbox (e.g., Responses API + shell tool + hosted container workspace) and returns sanitized outputs for the next step.
• Design a tight execution loop: propose → execute → return result → propose next step. This reduces hallucination and gives you a place to enforce policy, rate limits, and retries.
• Treat tool outputs as untrusted: validate, sanitize, and re-check them against higher-priority system/developer instructions before using them to make decisions.

Chain-of-thought and reasoning

• Chain-of-thought remains a powerful technique for multi-step reasoning, but it increases token costs and can reveal intermediate reasoning that might be sensitive.
• OpenAI and other vendors use chain-of-thought monitoring internally to detect misalignment. Prefer concise, auditable reasoning blocks and consider separating the "thinking" trace from the answer (e.g., <thinking> vs <answer> blocks) so you can store or purge traces depending on privacy needs.

Example template:

<thinking>
1. Restate the problem
2. Break into subproblems
3. Solve each subproblem with short steps
4. Verify constraints
</thinking>

<answer>
[Concise final answer here]
</answer>

Examples

Example 1: Classification with few-shot

const classificationPrompt = `Classify the support ticket into exactly one category.

Categories: billing, technical, account, feature-request, other

## Examples

Ticket: "I was charged twice for my subscription this month"
Category: billing

Ticket: "The API returns 500 errors when I send more than 10 requests"
Category: technical

Ticket: "Can you add dark mode to the dashboard?"
Category: feature-request

Ticket: "I can't log in after resetting my password"
Category: account

## Rules
- Return only the category name, nothing else
- If genuinely ambiguous, choose the most actionable category
- "other" is the last resort — use it only when no category fits`;

Example 2: Data extraction with structured output

const extractionPrompt = `Extract structured event information from the text.

Return JSON matching this schema exactly:
{
  "event_name": "string",
  "date": "YYYY-MM-DD or null",
  "time": "HH:MM or null",
  "location": "string or null",
  "attendees": ["string"],
  "confidence": 0.0-1.0
}

## Rules
- If a field is not mentioned, use null (not a guess)
- Parse relative dates against today's date provided in the user message
- List only explicitly named attendees, not implied ones
- Confidence reflects how clearly the information was stated`;

Example 3: Multi-turn agent instructions

const agentSystemPrompt = `You are a research assistant that helps users
find and synthesize information.

## Available tools
- web_search(query: string) — search the web for information
- read_url(url: string) — read the content of a web page
- save_note(title: string, content: string) — save a research note

## Behavior
1. When the user asks a question, search for relevant sources first
2. Read the top 2-3 results to gather information
3. Synthesize findings into a concise answer with citations
4. Save important findings as notes for future reference

## Citation format
Use inline citations: "The API supports 100 req/s [1]"
List sources at the end:
[1] https://docs.example.com/rate-limits

## Constraints
- Never fabricate information — if you can't find it, say so
- Always cite sources for factual claims
- Prefer official documentation over blog posts
- If search returns no results, suggest alternative queries`;

Decision tree

What type of prompt do you need?
├── Classification / Routing
│   ├── < 5 categories → Zero-shot with category list
│   └── > 5 categories or subtle distinctions → Few-shot with examples
├── Data extraction
│   ├── Fixed schema → Structured output (JSON mode / zodResponseFormat)
│   └── Variable schema → Describe output format in prompt
├── Reasoning / Analysis
│   ├── Single-step → Zero-shot with clear instructions
│   └── Multi-step → Chain-of-thought with <thinking> blocks
├── Generation / Writing
│   ├── Consistent style → Few-shot with 3+ examples
│   └── Creative → Higher temperature, fewer constraints
└── Agent / Tool use
    ├── Simple tool routing → ReAct pattern in system prompt
    └── Complex orchestration → Agent orchestration skill

Edge cases and gotchas (UPDATED)

• Prompt injection: instruction-hierarchy training reduces some classes of prompt-injection attacks, but never assume immunity — keep safety-critical rules in system/developer messages and sanitize tool outputs.
• Tool outputs are lower-priority: do not treat tool-proposed text as authoritative commands. Validate before using.
• Token budget & context: system prompts count against the context window — measure and compact context using retrieval, summarization, or the Responses API’s hosted workspace when you need file access.
• Model drift and portability: prompts that work for one provider can fail on another. Test across providers and include provider-specific tags or examples when needed.
• Few-shot ordering: the last example often has the most influence — order examples intentionally.
• Negative instructions vs positive rules: prefer "Always do Y" over "Don't do X" where possible.
• Output format compliance: models may drift — always validate parsed outputs server-side and fall back to a retry or clarification flow.
• Temperature 0 is not true determinism: small variance can remain. For strict determinism use programmatic checks and validation.
• Long prompts degrade: distill system prompts and use external context stores for large documents.
• Anthropic XML tips: wrap examples in <examples>/<example> and use descriptive tags (<instructions>, <input>, <output>) for more reliable parsing.
• Gemini inference tiers: for high-throughput background tasks use cost-optimized tiers (Flex); for interactive user-facing actions use higher-reliability tiers (Priority). Choose an inference tier as part of your cost/latency planning.

Evaluation criteria

• Accuracy: % of outputs matching ground-truth labels
• Format compliance: % of outputs parseable as the requested format
• Consistency: variance across multiple identical requests (temperature 0)
• Cost efficiency: tokens consumed per successful completion (include inference tier cost in estimates)
• Latency: time-to-first-token and total generation time
• Robustness: accuracy on adversarial / edge-case inputs
• Provider portability: cross-provider test coverage and provider-specific validation

Research-backed changes included in this update

• Instruction hierarchy priority (System > Developer > User > Tool) and its practical implications (OpenAI IH-Challenge)
• Guidance on building agents using Responses API, shell tool, and hosted container workspace (OpenAI engineering post)
• Anthropic prompt structuring and XML-tag recommendations (Anthropic docs)
• Gemini API inference tiers (Flex and Priority) and how to treat them in production design (Google AI blog)