GPT-4.1 Prompting Guide

Comprehensive guide for prompting GPT-4.1, including agentic workflows, long context handling, and instruction following best practices.

Overview

GPT-4.1 represents a significant advancement in instruction following and agentic capabilities. This guide covers the specific prompting techniques that maximize its potential.

┌─────────────────────────────────────────────────────────────────────┐
│                    GPT-4.1 KEY CAPABILITIES                          │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  ✅ STRENGTHS:                                                       │
│  ├─► Superior instruction following                                  │
│  ├─► Extended context window (1M tokens)                             │
│  ├─► Improved agentic task completion                               │
│  ├─► Better structured output generation                            │
│  └─► Enhanced tool/function calling                                 │
│                                                                      │
│  🎯 OPTIMAL USE CASES:                                               │
│  ├─► Software engineering agents                                    │
│  ├─► Multi-step autonomous workflows                                │
│  ├─► Long document processing                                       │
│  └─► Complex reasoning tasks                                        │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

Agentic System Prompts

The SWE-bench System Prompt

This exact prompt achieved top scores on SWE-bench, the software engineering benchmark:

SYS_PROMPT_SWEBENCH = """You will be tasked to fix an issue from an open-source repository.

Your thinking should be thorough and so it's fine if it's very long. You can think step by step before and after each action you decide to take.

You MUST iterate and keep going until the problem is solved.

You already have everything you need to solve this problem in the /testbed folder, even without internet connection. I want you to fully solve this autonomously before coming back to me.

The repository is already cloned in /testbed. DO NOT clone it again. DO NOT checkout a different branch.

DO NOT run any testing command. Only check whether the issue is fixed on a unit level, with something like a unit test script or a simple python script. NEVER run `pytest`, `tox`, `pre-commit`, `make`, or anything similar.

BEFORE YOU START, use `find_file` to first understand the general structure of the codebase.

Take your time and explore the repository structure to understand where the relevant files are.

You MUST plan extensively before each function call, and reflect extensively on the outcomes of the previous function calls. DO NOT do this entire process by trial and error, take time to think about the problem and your attempted solutions.

You have a lot of autonomy in this task, so please use your best judgement.

Start by exploring the repository structure to understand the codebase.
"""

Key Prompting Principles

1. Encourage thorough thinking: "Your thinking should be thorough and so it's fine if it's very long"
2. Explicit persistence: "You MUST iterate and keep going until the problem is solved"
3. Prevent premature termination: "I want you to fully solve this autonomously before coming back to me"
4. Clear constraints: "DO NOT run any testing command... NEVER run pytest, tox..."
5. Structured workflow: "BEFORE YOU START, use find_file to first understand the general structure"
6. Reflection requirement: "You MUST plan extensively before each function call, and reflect extensively on the outcomes"

Agentic Workflow Patterns

The PLAN → EXECUTE → REFLECT Loop

AGENTIC_SYSTEM_PROMPT = """You are an autonomous agent. Follow this workflow:

1. PLAN: Before taking any action, explicitly state:
   - What you're trying to accomplish
   - Why this action is necessary
   - What you expect to happen

2. EXECUTE: Perform the action using available tools

3. REFLECT: After each action, analyze:
   - Did the outcome match your expectations?
   - What did you learn?
   - What should you do next?

Continue this loop until the task is complete.

IMPORTANT: Do not give up after encountering errors. Analyze the error, form a hypothesis, and try a different approach.
"""

Tool Persistence Pattern

TOOL_PERSISTENCE_PROMPT = """When using tools:

1. If a tool call fails, DO NOT immediately give up
2. Analyze the error message
3. Adjust your parameters or approach
4. Try again with the corrected input

Common recovery patterns:
- File not found → Use find_file to locate correct path
- Permission denied → Check for alternative access methods
- Invalid syntax → Fix the syntax and retry
- Timeout → Break into smaller operations

You have unlimited retries. Keep trying until you succeed.
"""

Long Context Handling

Document Position Strategy

┌─────────────────────────────────────────────────────────────────────┐
│                    CONTEXT POSITIONING                               │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  OPTIMAL PLACEMENT:                                                  │
│  ┌─────────────────────────────────────────────────────────────┐    │
│  │ System Prompt (instructions, persona)                        │    │
│  ├─────────────────────────────────────────────────────────────┤    │
│  │ IMPORTANT CONTEXT (documents, data) ← Put critical info     │    │
│  │                                        at START of context  │    │
│  ├─────────────────────────────────────────────────────────────┤    │
│  │ Supporting context (less critical)                           │    │
│  ├─────────────────────────────────────────────────────────────┤    │
│  │ Recent conversation history                                  │    │
│  ├─────────────────────────────────────────────────────────────┤    │
│  │ User query (most recent)                                     │    │
│  └─────────────────────────────────────────────────────────────┘    │
│                                                                      │
│  ⚠️ Avoid "lost in the middle" - information in the center of      │
│     very long contexts may receive less attention                   │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

Chunking Strategy

def chunk_for_context(document: str, max_chunk_tokens: int = 8000) -> list[str]:
    """Chunk document with overlap for context continuity."""
    chunks = []
    overlap = max_chunk_tokens // 10  # 10% overlap

    # Use tiktoken for accurate token counting
    import tiktoken
    enc = tiktoken.encoding_for_model("gpt-4")
    tokens = enc.encode(document)

    start = 0
    while start < len(tokens):
        end = min(start + max_chunk_tokens, len(tokens))
        chunk_tokens = tokens[start:end]
        chunks.append(enc.decode(chunk_tokens))
        start = end - overlap if end < len(tokens) else end

    return chunks

Chain-of-Thought Prompting

Explicit CoT Triggers

COT_TRIGGERS = [
    "Let's think step by step",
    "Let me work through this carefully",
    "I'll break this down into steps",
    "First, let me understand the problem...",
    "Let me analyze this systematically",
]

# Use in system prompt
COT_SYSTEM_PROMPT = """When solving problems:

1. Always show your reasoning
2. Break complex problems into steps
3. Validate each step before proceeding
4. State assumptions explicitly
5. Consider edge cases

Format your response as:
UNDERSTANDING: [Restate the problem]
APPROACH: [Your planned steps]
EXECUTION: [Step-by-step work]
VERIFICATION: [Check your answer]
ANSWER: [Final response]
"""

Mathematical Reasoning Example

MATH_SYSTEM_PROMPT = """You are a mathematical reasoning assistant.

For every problem:
1. Parse the problem statement carefully
2. Identify known quantities and unknowns
3. Select appropriate formulas/methods
4. Show ALL intermediate steps
5. Verify the answer makes sense

IMPORTANT: Never skip steps. Even if a step seems obvious, write it out.

Example format:
Given: [List all given information]
Find: [State what we're solving for]
Solution:
  Step 1: [First operation with explanation]
  Step 2: [Second operation with explanation]
  ...
Verification: [Check the answer]
Answer: [Final answer with units]
"""

Instruction Following Best Practices

Explicit Format Specification

# BAD: Vague instruction
prompt_bad = "Summarize this document"

# GOOD: Explicit format
prompt_good = """Summarize this document following these requirements:

FORMAT:
- Start with a one-sentence TL;DR
- Follow with 3-5 bullet points of key findings
- End with a "Next Steps" section

CONSTRAINTS:
- Total length: 150-200 words
- Use simple language (8th grade reading level)
- No jargon without definitions
- Include specific numbers/statistics from the document

OUTPUT STRUCTURE:
## TL;DR
[One sentence]

## Key Findings
- [Finding 1]
- [Finding 2]
...

## Next Steps
[Recommended actions]
"""

Constraint Enforcement

CONSTRAINT_PROMPT = """MANDATORY CONSTRAINTS (violation = task failure):

1. LENGTH: Response must be 100-150 words. Count carefully.
2. FORMAT: Must be valid JSON. Use proper escaping.
3. CONTENT: Only use information from the provided context.
4. TONE: Professional and neutral. No opinions.
5. CITATIONS: Every claim must cite a source.

SELF-CHECK before responding:
- [ ] Word count within range?
- [ ] Valid JSON syntax?
- [ ] All claims sourced?
- [ ] No personal opinions?
- [ ] Professional tone?

If any check fails, revise before submitting.
"""

Diff-Based Output Formats

The Search/Replace Block Format

For code modifications, use explicit search/replace blocks:

DIFF_INSTRUCTION = """When making code changes, use this format:

<<<<<<< SEARCH
[exact code to find - must match file exactly]
=======
[replacement code]
>>>>>>> REPLACE

Rules:
1. SEARCH block must match existing code EXACTLY (including whitespace)
2. Only include enough context to uniquely identify the location
3. One change per SEARCH/REPLACE block
4. For multiple changes, use multiple blocks

Example:
<<<<<<< SEARCH
def calculate_total(items):
    return sum(items)
=======
def calculate_total(items):
    \"\"\"Calculate total with validation.\"\"\"
    if not items:
        return 0
    return sum(items)
>>>>>>> REPLACE
"""

Unified Diff Format

UNIFIED_DIFF_INSTRUCTION = """Output changes as unified diff:

```diff
--- a/path/to/file.py
+++ b/path/to/file.py
@@ -10,7 +10,9 @@ def existing_function():
     existing_line_1
     existing_line_2
-    line_to_remove
+    replacement_line_1
+    replacement_line_2
     existing_line_3

Rules:

• Lines starting with - are removed
• Lines starting with + are added
• Context lines (no prefix) help locate changes
• Include 3 lines of context around changes """

## Model-Specific Optimization

### GPT-4.1 vs Other Models

| Feature | GPT-4.1 | GPT-4o | GPT-3.5 |
|---------|---------|--------|---------|
| Context Window | 1M tokens | 128K tokens | 16K tokens |
| Instruction Following | Excellent | Good | Moderate |
| Agentic Tasks | Optimized | Good | Limited |
| Structured Output | Native support | Good | Requires guidance |
| Tool Calling | Advanced | Standard | Basic |

### Temperature Guidelines

```python
# Task-specific temperature settings
TEMPERATURE_GUIDE = {
    "code_generation": 0.0,      # Deterministic, precise
    "code_review": 0.2,          # Slight variation in phrasing
    "creative_writing": 0.7,     # More creative freedom
    "brainstorming": 0.9,        # Maximum diversity
    "factual_qa": 0.0,           # Accurate retrieval
    "summarization": 0.3,        # Consistent but natural
    "agentic_tasks": 0.0,        # Reliable execution
}

Complete Agentic Example

from openai import OpenAI

client = OpenAI()

AGENTIC_SYSTEM_PROMPT = """You are an autonomous software engineering agent.

CAPABILITIES:
- Read files using read_file(path)
- Write files using write_file(path, content)
- Execute commands using run_command(cmd)
- Search codebase using search(query)

WORKFLOW:
1. UNDERSTAND: Analyze the task requirements completely
2. EXPLORE: Investigate the codebase structure
3. PLAN: Create a detailed implementation plan
4. IMPLEMENT: Make changes incrementally
5. VERIFY: Test that changes work correctly
6. ITERATE: Fix any issues found

RULES:
- Think step by step before each action
- Reflect on outcomes after each action
- Never give up - try alternative approaches
- Ask for clarification only if truly blocked

Your goal is complete autonomous task resolution.
"""

tools = [
    {
        "type": "function",
        "function": {
            "name": "read_file",
            "description": "Read contents of a file",
            "parameters": {
                "type": "object",
                "properties": {
                    "path": {"type": "string", "description": "File path to read"}
                },
                "required": ["path"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "write_file",
            "description": "Write content to a file",
            "parameters": {
                "type": "object",
                "properties": {
                    "path": {"type": "string"},
                    "content": {"type": "string"}
                },
                "required": ["path", "content"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "run_command",
            "description": "Execute a shell command",
            "parameters": {
                "type": "object",
                "properties": {
                    "cmd": {"type": "string", "description": "Command to execute"}
                },
                "required": ["cmd"]
            }
        }
    }
]

def run_agent(task: str):
    """Run the agentic loop until task completion."""
    messages = [
        {"role": "system", "content": AGENTIC_SYSTEM_PROMPT},
        {"role": "user", "content": task}
    ]

    max_iterations = 50
    for i in range(max_iterations):
        response = client.chat.completions.create(
            model="gpt-4.1",
            messages=messages,
            tools=tools,
            temperature=0
        )

        message = response.choices[0].message
        messages.append(message)

        # Check if agent wants to use tools
        if message.tool_calls:
            for tool_call in message.tool_calls:
                # Execute tool and add result
                result = execute_tool(tool_call)
                messages.append({
                    "role": "tool",
                    "tool_call_id": tool_call.id,
                    "content": result
                })
        else:
            # Agent finished (no more tool calls)
            if "TASK COMPLETE" in message.content:
                return message.content

    return "Max iterations reached"

Key Takeaways

1. Be explicit: GPT-4.1 follows instructions precisely - make them detailed
2. Encourage persistence: Explicitly tell it not to give up
3. Require reflection: Force planning before action and analysis after
4. Use structure: Format constraints produce consistent outputs
5. Position matters: Put critical information at the start of long contexts
6. Temperature 0 for agents: Deterministic behavior for reliable execution

See Also

• Structured Outputs
• Orchestrating Agents
• LLM Guardrails