Important: modern LLMs are trained with an instruction hierarchy. OpenAI’s IH‑Challenge (Mar 2026) and the Model Spec emphasize a clear priority ordering: System > Developer > User > Tool. Place safety‑critical and policy constraints in the system or developer layer so they remain highest priority and are resilient to lower‑priority inputs (including tool outputs and web content). See OpenAI IH‑Challenge for details: https://openai.com/index/instruction-hierarchy-challenge/ (Mar 10, 2026).

Prompt strategies

• Zero-shot: simple tasks, low token cost
• Few-shot: consistent format or tricky edge-cases; include diverse examples and order them intentionally (the last example often has outsized influence)
• Chain-of-thought: multi-step reasoning or proofs — isolate thinking traces when possible and avoid leaking sensitive intermediate content
• Self-consistency: sample multiple reasoning traces and take the consensus for high-stakes tasks
• ReAct / tool-guided: when the model must propose actions (tool calls); implement an orchestrator to execute, verify, and sanitize tool outputs
• Structured output: use schema validation (e.g., zod, JSON Schema) rather than relying on free-form parsing

Temperature guide

• 0: deterministic-style behavior for classification/extraction (note: greedy decoding still permits small variance)
• 0.3–0.5: balanced
• 0.7–1.0: creative

Workflow

Step 1: Define the task contract

Before writing any prompt, answer these questions:

type PromptContract = {
  input: string;       // What does the model receive?
  output: string;      // What should it produce?
  format: string;      // JSON, markdown, plain text?
  constraints: string[]; // What must it avoid?
  edgeCases: string[];   // How should it handle ambiguity?
  examples: Array<{ input: string; output: string }>;
};

Step 2: Write the system prompt

Focus system-level text on what must always hold (safety, privacy, legal constraints). Put product-level preferences in developer instructions. Remember: models trained with instruction-hierarchy data are more likely to honor these distinctions.

(Example omitted here for brevity — keep a compact system prompt that lists role, task, output format, rules, and examples.)

Step 3: Implement structured output with OpenAI (Responses API)

• Prefer the Responses API when you need agentic primitives (tools) or an orchestrator; OpenAI’s engineering posts describe using the Responses API together with a shell tool and hosted containers to execute sandboxed actions and keep execution auditable (see OpenAI engineering posts on agent runtimes).
• Treat tool outputs as lower-priority content in the instruction hierarchy: do not place tool-provided text into system messages without sanitization and verification. Validate and re-check tool outputs server-side before acting on them.
• Use schema helpers (like zodResponseFormat) to produce machine-parseable responses and validate them server-side to guard against format drift.

Example (OpenAI Responses + zod):

import OpenAI from "openai";
import { z } from "zod";
import { zodResponseFormat } from "openai/helpers/zod";

const FindingSchema = z.object({
  severity: z.enum(["critical", "warning", "info"]),
  line: z.number(),
  message: z.string(),
  suggestion: z.string(),
});

const ReviewSchema = z.object({
  findings: z.array(FindingSchema),
});

const client = new OpenAI();

async function reviewCode(diff: string) {
  const response = await client.responses.create({
    model: "gpt-4o",
    instructions: systemPrompt,
    input: diff,
    text: {
      format: zodResponseFormat(ReviewSchema, "code_review"),
    },
  });

  // Validate and parse on the server to guard against format drift
  return ReviewSchema.parse(JSON.parse(response.output_text));
}

Prompt versioning & rollout (production pattern)

Adopt explicit versioning and staged rollout for system/developer prompts:

• Keep prompts in version control alongside integration tests (prompts/ directory in the repo).
• Use semantic versioning for prompts (e.g., prompt v1.2.0) and include a short changelog entry describing behavioral intent and test coverage.
• Automate regression tests: unit-style prompts that assert final output, format compliance, and performance budgets (tokens, latency).
• Canary and rollback: roll a new prompt to a small % of traffic, monitor key metrics (accuracy, format compliance, error rates), then promote or rollback automatically.
• Runtime feature flags: allow switching between prompt versions at runtime and maintain per-version telemetry so you can compare behavior.

Rationale: treat prompts like code — the OpenAI Model Spec and iterative-deployment guidance encourage incremental rollout and monitoring of model/agent behavior.

Step 4: Implement with Anthropic

• Anthropic’s docs recommend explicit roles and XML-like tags to structure prompts. Use tags like <instructions>, <examples>, and <input> to reduce ambiguity in multi-component prompts.
• Include 3–5 diverse examples wrapped in <examples> / <example> tags to improve format fidelity and edge-case handling for Claude models.

Step 5: Agents and orchestrators

• When building agents, the model should only propose tool calls — an orchestrator must execute them in a sandbox and return sanitized outputs for the next step. OpenAI’s agent runtime examples demonstrate combining Responses API, a shell tool, and hosted containers to limit blast radius and keep execution auditable (see: OpenAI agent runtime post, Mar 2026).
• Design a tight execution loop: propose → execute → return result → propose next step. This reduces hallucination and gives you a place to enforce policy, rate limits, retries, and identity checks.
• Treat tool outputs as untrusted: validate, sanitize, and re-check them against higher-priority system/developer instructions before using them to make decisions.

Prompt safety checklist (operational)

• Minimize authority: give the agent the least privilege needed (allowlist endpoints, limit API scopes, and avoid broad filesystem or network write permissions).
• Verification steps: require explicit verification for sensitive or irreversible actions (human approval, re-authentication, or second-factor confirmation).
• Sanitize external content: canonicalize and filter inputs from web pages, emails, or tools; avoid passing raw HTML or scraped text into the model without normalization.
• Constrain impact: default to read-only or simulated-run modes and require an explicit intent switch for irreversible operations.
• Audit & short-lived credentials: log proposals and tool outputs, and use time-limited credentials for downstream calls so compromised outputs cannot be reused.

Sources: OpenAI "Designing AI agents to resist prompt injection" (Mar 11, 2026) and related engineering posts on agent runtimes: https://openai.com/index/designing-agents-to-resist-prompt-injection/ and https://openai.com/index/equip-responses-api-computer-environment/ (Mar 11, 2026).

Chain-of-thought and reasoning

• Chain-of-thought remains a powerful technique for multi-step reasoning, but it increases token costs and can reveal intermediate reasoning that might be sensitive.
• OpenAI’s internal monitoring work shows chain-of-thought traces can be useful for detecting misalignment and failure modes. Make thinking traces auditable and optionally separable from final answers so you can store, inspect, or purge them according to privacy policy. (See OpenAI monitoring posts, Mar 2026.)
• Prefer concise, auditable reasoning blocks and consider separating the "thinking" trace from the answer (e.g., <thinking> vs <answer> blocks) so you can store or purge traces depending on privacy needs.

Example template:

<thinking>
1. Restate the problem
2. Break into subproblems
3. Solve each subproblem with short steps
4. Verify constraints
</thinking>

<answer>
[Concise final answer here]
</answer>

Examples

Example 1: Classification with few-shot

(unchanged — keep compact few-shot pattern; enforce return-only rule and server-side validation)

Example 2: Data extraction with structured output

(unchanged — use structured JSON schema, return null for missing fields, parse relative dates against provided reference date)

Example 3: Multi-turn agent instructions

(unchanged — agent system prompt that lists available tools, behavior loop, citation rules, and constraints; ensure orchestrator validates tool output)

Decision tree

(unchanged — retain mapping from problem type to prompt pattern: classification, extraction, reasoning, generation, agent/tool use)

Edge cases and gotchas (UPDATED)

• Prompt injection: instruction-hierarchy training (IH‑Challenge) reduces some classes of prompt-injection attacks, but never assume immunity. Modern attacks increasingly resemble social engineering; architectural mitigations (least privilege, verification, sandboxing, short-lived tokens) remain vital. See: https://openai.com/index/designing-agents-to-resist-prompt-injection/ (Mar 11, 2026).
• Tool outputs are lower-priority: do not treat tool-proposed text as authoritative commands. Validate before using.
• Token budget & context: system prompts count against the context window — measure and compact context using retrieval, summarization, or the Responses API’s hosted workspace when you need file access.
• Model drift and portability: prompts that work for one provider can fail on another. Test across providers and include provider-specific tags or examples when needed.
• Few-shot ordering: the last example often has the most influence — order examples intentionally.
• Negative instructions vs positive rules: prefer "Always do Y" over "Don't do X" where possible.
• Output format compliance: models may drift — always validate parsed outputs server-side and fall back to a retry or clarification flow.
• Temperature 0 is not true determinism: small variance can remain. For strict determinism use programmatic checks and validation.
• Long prompts degrade: distill system prompts and use external context stores for large documents.
• Anthropic XML tips: wrap examples in <examples>/<example> and use descriptive tags (<instructions>, <input>, <output>) for more reliable parsing.
• Gemini inference tiers (Flex vs Priority): use Flex for cost-sensitive, latency-tolerant workloads and Priority for interactive, user-facing workloads. Set the service_tier field to priority to request Priority behavior; the default is standard. Implement graceful-downgrade logic that retries or queues work to Flex/standard when Priority is unavailable and monitor rate limits and cost tradeoffs (see Google Gemini API docs: https://ai.google.dev/gemini-api/docs/priority-inference/).

Evaluation criteria

• Accuracy: % of outputs matching ground-truth labels
• Format compliance: % of outputs parseable as the requested format
• Consistency: variance across multiple identical requests (temperature 0)
• Cost efficiency: tokens consumed per successful completion (include inference tier cost in estimates)
• Latency: time-to-first-token and total generation time
• Robustness: accuracy on adversarial / edge-case inputs
• Provider portability: cross-provider test coverage and provider-specific validation

Research-backed changes included in this update

• Instruction hierarchy priority (System > Developer > User > Tool) reinforced by IH‑Challenge (OpenAI, Mar 10, 2026)
• Concrete agent prompt-injection mitigations and architectural controls (OpenAI, Mar 11, 2026)
• Recommendations for agent runtimes that combine the Responses API with sandboxed tools and hosted containers (OpenAI engineering posts, Mar 2026)
• Gemini API inference-tier guidance (Flex vs Priority) and client-side downgrade strategies (Google Gemini API docs, Apr 2026)