Prompt Engineering

Prompt engineering is the practice of designing inputs to language models to reliably elicit high-quality, accurate, and appropriately formatted outputs. It covers everything from writing system instructions to multi-step reasoning pipelines and retrieval-augmented generation. Effective prompting reduces hallucinations, improves consistency, and unlocks capabilities the model already has but needs guidance to apply. The techniques here apply across providers (OpenAI, Anthropic, Google) with minor syntactic differences.

When to use this skill

Trigger this skill when the task involves:

• Writing or refining a system prompt for an agent or chatbot
• Implementing chain-of-thought reasoning to improve accuracy on hard tasks
• Designing few-shot examples to steer model behavior
• Building a RAG pipeline (retrieval + context injection + generation)
• Getting structured JSON/schema output from a model reliably
• Chaining multiple LLM calls (decomposition, routing, verification)
• Evaluating or benchmarking prompt quality across dimensions
• Choosing between zero-shot, few-shot, fine-tuning, or RAG approaches
• Debugging inconsistent or hallucinated model outputs

Do NOT trigger this skill for:

• Model training, fine-tuning infrastructure, or RLHF pipelines (those are ML engineering)
• Framework-specific agent wiring (use the mastra or relevant framework skill instead)

Key principles

1. Be specific and explicit - Vague instructions produce vague outputs. State the audience, format, length, tone, and constraints in every prompt.
2. Provide context before instruction - Background and examples before the task reduces ambiguity. The model reads top-to-bottom; front-load what matters.
3. Use structured output - Request JSON, markdown tables, or a fixed schema when downstream code will consume the response. Pair with schema validation and retries.
4. Iterate and evaluate - Treat prompts as code. Version them, test against a golden eval set, and measure regressions before deploying changes.
5. Decompose complex tasks - A single prompt asking the model to research, reason, and format simultaneously degrades quality. Break into sequential or parallel calls.

Core concepts

System / user / assistant roles

Temperature and sampling

• temperature: 0 - Deterministic, best for factual extraction and structured output
• temperature: 0.3-0.7 - Balanced creativity and coherence; good for most tasks
• temperature: 1.0+ - High diversity; useful for brainstorming, risky for factual tasks
• top_p (nucleus sampling) - Alternative to temperature; values 0.9-0.95 are common
• Never set both temperature and top_p to non-default at the same time

Token economics

• Input tokens cost less than output tokens on most providers - keep outputs focused
• Longer context = slower TTFT (time to first token) and higher cost
• Few-shot examples consume significant tokens; choose examples carefully
• Use max_tokens to cap runaway responses

Context window management

• Modern models: 128K-1M token windows, but quality degrades near limits ("lost in the middle")
• Place critical instructions at the start and end of long prompts
• For RAG: inject only top-K retrieved chunks, not entire documents
• Summarize long conversation history rather than passing raw transcripts

Prompt vs fine-tuning decision

Common tasks

Write effective system prompts

Template:

You are [PERSONA] helping [AUDIENCE] with [DOMAIN].

Your responsibilities:
- [CORE TASK 1]
- [CORE TASK 2]

Constraints:
- [HARD RULE 1 - what to never do]
- [HARD RULE 2]

Output format: [FORMAT DESCRIPTION]

Concrete example:

You are a senior code reviewer helping software engineers improve TypeScript code quality.

Your responsibilities:
- Identify bugs, logic errors, and type safety issues
- Suggest idiomatic improvements with brief reasoning
- Flag security vulnerabilities explicitly

Constraints:
- Never rewrite the entire file unprompted; focus on the diff
- Do not praise code unless it exemplifies a non-obvious pattern worth reinforcing

Output format: Return a markdown list of findings. Each item: [SEVERITY] - description.

Anti-patterns:

• "Be helpful, harmless, and honest" (too generic - the model already knows this)
• Contradictory constraints ("be concise" and "explain everything in detail")
• No output format specification when downstream parsing is required

Implement chain-of-thought

Zero-shot CoT - append "Let's think step by step." to trigger reasoning:

User: A store has 3 boxes of apples, each containing 12 apples. They sell 15 apples.
      How many remain? Let's think step by step.

Structured CoT - define explicit reasoning steps:

System: When solving math or logic problems, follow this structure:
  1. UNDERSTAND: Restate what is being asked
  2. PLAN: List the operations needed
  3. EXECUTE: Work through each step
  4. ANSWER: State the final answer clearly

User: [problem]

Self-consistency (sample multiple reasoning paths, majority-vote the answer):

answers = []
for _ in range(5):
    response = llm.complete(cot_prompt, temperature=0.7)
    answers.append(extract_answer(response))
final_answer = Counter(answers).most_common(1)[0][0]

Use CoT for arithmetic, logic, multi-step planning, and ambiguous classification. Skip CoT for simple lookup tasks - it adds tokens without benefit.

Design few-shot examples

Selection criteria:

• Cover the most common input patterns (not edge cases for initial shot selection)
• Include at least one negative/refusal example if the model should decline certain inputs
• Keep formatting identical across all examples - models learn from structural patterns

Ordering:

• Most representative examples first; most recent (closest to the query) last
• For classification: interleave classes rather than grouping them

Formatting template:

System: Classify the sentiment of customer reviews as POSITIVE, NEGATIVE, or NEUTRAL.

User: Review: "The product arrived on time but the packaging was damaged."
Assistant: NEGATIVE

User: Review: "Exactly as described, fast shipping. Very happy!"
Assistant: POSITIVE

User: Review: "It works."
Assistant: NEUTRAL

User: Review: "{actual_review}"

3-8 examples typically saturate few-shot gains. More examples rarely help and consume context budget that could be used for the actual input.

Build a RAG prompt pipeline

Step 1 - Retrieval: embed the query and fetch top-K chunks from a vector store.

Step 2 - Context injection:

System: You are a documentation assistant. Answer questions using ONLY the provided
        context. If the answer is not in the context, say "I don't have that information."

Context:
---
{retrieved_chunk_1}
---
{retrieved_chunk_2}
---

User: {user_question}

Step 3 - Generation with citation:

System: [...as above...]
        After your answer, list sources as: Sources: [chunk title or ID]

User: How do I configure authentication?

Key decisions:

• Chunk size: 256-512 tokens for precision; 1024 for broader context
• Overlap: 10-20% of chunk size to avoid cutting mid-sentence
• Reranking: use a cross-encoder reranker after initial retrieval to improve top-K quality
• Query rewriting: expand ambiguous queries before embedding for better recall

Never inject raw retrieved text without a clear delimiter. Models need structural separation to distinguish context from instructions.

Get structured JSON output

Schema enforcement via function calling / structured output (preferred):

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Extract person info from: Alice Smith, 32, engineer"}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person",
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "age": {"type": "integer"},
                    "role": {"type": "string"}
                },
                "required": ["name", "age", "role"]
            }
        }
    }
)

Prompt-based fallback with retry:

def extract_json(prompt: str, schema: dict, max_retries=3) -> dict:
    for attempt in range(max_retries):
        raw = llm.complete(f"{prompt}\n\nRespond with valid JSON matching: {schema}")
        try:
            data = json.loads(raw)
            validate(data, schema)  # jsonschema
            return data
        except (json.JSONDecodeError, ValidationError) as e:
            prompt += f"\n\nPrevious response was invalid: {e}. Fix and retry."
    raise RuntimeError("Failed to get valid JSON after retries")

Always validate parsed JSON against a schema - do not trust model-generated structure blindly. Use response_format: json_object as a minimum guardrail.

Implement prompt chaining

Decomposition pattern - split a complex task into sequential LLM calls:

# Step 1: Research
research = llm.complete(f"List key facts about: {topic}")

# Step 2: Outline
outline = llm.complete(f"Given these facts:\n{research}\n\nCreate a structured outline.")

# Step 3: Write
article = llm.complete(f"Outline:\n{outline}\n\nWrite the full article.")

Routing pattern - use a classifier call to select the right downstream prompt:

intent = llm.complete(
    f"Classify this request as one of [refund, technical, billing, other]: {user_message}"
)
handler_prompt = PROMPTS[intent.strip().lower()]
response = llm.complete(handler_prompt.format(message=user_message))

Verification pattern - add a critic call after generation:

draft = llm.complete(task_prompt)
critique = llm.complete(
    f"Review this output for accuracy and completeness:\n{draft}\n\n"
    "List any errors or missing information. If none, respond 'APPROVED'."
)
if "APPROVED" not in critique:
    final = llm.complete(f"Revise based on this critique:\n{critique}\n\nDraft:\n{draft}")

Evaluate prompt quality

Eval harness pattern:

results = []
for case in eval_set:
    output = llm.complete(prompt.format(**case["inputs"]))
    results.append({
        "id": case["id"],
        "pass": case["expected"] in output,
        "output": output,
    })
print(f"Pass rate: {sum(r['pass'] for r in results) / len(results):.1%}")

Anti-patterns / common mistakes

Gotchas

1. Temperature > 0 for structured extraction - Even temperature: 0.1 meaningfully increases JSON parse failure rates. Always use temperature: 0 when the output must be parsed programmatically. This is the single highest-yield change for reliability.

2. RAG context injected without delimiters - When retrieved chunks are concatenated directly into the prompt without separators (--- or XML-style tags), models confuse retrieved content with instructions. Always use explicit structural delimiters around each retrieved chunk.

3. Verification pattern creates hallucination loops - The critic-and-revise pattern can cause a model to confidently generate new hallucinations to "fix" non-existent errors. If the draft is factually grounded, set a high bar for what triggers revision - don't revise unless there's a concrete, checkable error.

4. Few-shot examples grouped by class - In classification prompts, showing all POSITIVE examples first then all NEGATIVE examples trains the model to pattern-match on recency rather than semantic content. Interleave classes in few-shot examples.

5. System prompt changes not tracked against an eval set - Prompt changes that feel like improvements often degrade performance on edge cases. Maintain a golden eval set of 20+ cases before any production prompt is modified, and measure pass rate before and after every change.

References

For a comprehensive catalog of 15+ individual prompting techniques with examples and effectiveness notes, load:

• references/techniques-catalog.md - zero-shot, CoT, self-consistency, ToT, ReAct, meta-prompting, role prompting, and more

Only load the references file when selecting or comparing specific techniques - it is long and will consume context.