System, User, Assistant Roles

Have you ever sent a prompt to an LLM and gotten a response that was technically correct but completely useless — wrong tone, ignored your format, or answered a question you didn't ask? The culprit is usually a single, flat prompt that tries to do everything at once. By learning to use system, user, and assistant roles, you separate who the AI is, what you want, and how it should respond — turning chaotic one-shot prompts into structured conversations that produce consistent, controllable outputs.

The problem this lesson solves

Most beginners write prompts like a message to a friend: one giant paragraph mixing instructions, context, and examples. The LLM has to guess which parts are commands and which are background noise. This leads to several frustrating failure modes:

• Inconsistent tone — the model swings from corporate jargon to casual slang based on the last few words.
• Ignored formatting — you asked for a JSON array, but got a Markdown list.
• Context confusion — the model treats a statement about the weather as an instruction to change its behavior.

These issues multiply in production. A customer-support bot that answers with the wrong persona, or a data-extraction pipeline that returns prose instead of structured data, can break an entire workflow. The single-role approach simply doesn't scale.

When you use system, user, and assistant roles, you give each part of the conversation a clear job. The system message defines the model's behavior and constraints. The user message carries the new input. The assistant message holds the model's prior response — letting you build multi-turn state. This separation is the foundation for reliable, maintainable prompts.

Core concept / mental model

Think of an LLM chat as a three-person play: the director, the prompter, and the actor.

• System role = director: sets the stage, the rules, and the character. "You are a helpful assistant that always speaks in pirate dialect."
• User role = prompter: gives the actor their lines and cues. "What's the capital of France?"
• Assistant role = actor: delivers the spoken response. "Arrr! 'Tis Paris, matey!"

The assistant's previous lines are fed back as assistant messages. This is how the model remembers what it said and can build on it — like an actor who remembers their earlier dialogue.

Key definitions:

• System message (also called system prompt) — sets the scene. It's the highest-priority instruction; it defines the assistant's identity, tone, and constraints.
• User message — the actual query or task. This is where you put new information, the question, or the data to process.
• Assistant message — the model's output. You include it only to provide context from a previous turn.

Here's a verbal diagram of the conversation flow:

[System] You are a meticulous data analyst.
        Always respond in JSON.

[User]   Here is the sales data: [data]
        Summarize the quarterly trend.

[Assistant] {...valid JSON summary...}

[User]   Now compare with last year's numbers.

[Assistant] {...compares, still JSON...}

This mental model clarifies who is speaking, which is the single biggest lever for prompt control.

How it works step by step

Let's walk through the mechanics of assembling a proper multi-role prompt.

Step 1: Define the system role

Start with the system message. Ask: What should the assistant be? Give it a persona, a goal, and any hard rules. The more specific you are, the better the model obeys. For example:

• "You are a legal expert focusing on data privacy."
• "You are a code reviewer. Only suggest changes that fix security vulnerabilities."

Step 2: Place the user request

Next, write the user message. This is your task — the question, the data, the instruction. Keep it separate from the system to avoid ambiguity. If the model needs background info, put that in the user turn.

Step 3: Feed back assistant turns

For multi-turn tasks, append the previous assistant response as an assistant message. This gives the model its own history, which it can use to keep context and consistency.

Step 4: Iterate with new user turns

Each new user message continues the conversation. The system prompt stays the same throughout, so behavior remains stable even as the topic changes.

A typical sequence, in pseudo-code:

1. Set system: identity/rules
2. Set user: initial task + context
3. Model returns assistant message
4. If continuing, set user: follow-up question
5. Model returns new assistant message (using history)
6. Repeat

When to use each role

Hands-on walkthrough

Let's build a real example using the OpenAI API. We'll create a system prompt that forces JSON output, and then have a conversation.

Example 1: Single-turn with system role

Here's a minimal Python script:

from openai import OpenAI

client = OpenAI()

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "You are a helpful assistant. Always respond with a JSON object containing a 'answer' key."},
        {"role": "user", "content": "What is the capital of France?"}
    ]
)

print(response.choices[0].message.content)

Expected output:

{"answer": "Paris"}

Example 2: Multi-turn with assistant memory

from openai import OpenAI

client = OpenAI()

messages = [
    {"role": "system", "content": "You are a concise programming tutor. Keep answers under 50 words."},
    {"role": "user", "content": "What is a list comprehension in Python?"}
]

# First turn
resp1 = client.chat.completions.create(model="gpt-4o-mini", messages=messages)
assistant_reply1 = resp1.choices[0].message.content
print("Assistant 1:", assistant_reply1)

# Append assistant reply and add a follow-up user message
messages.append({"role": "assistant", "content": assistant_reply1})
messages.append({"role": "user", "content": "Give me one example."})

resp2 = client.chat.completions.create(model="gpt-4o-mini", messages=messages)
print("Assistant 2:", resp2.choices[0].message.content)

Expected output (approximately):

Assistant 1: A list comprehension is a concise way to create lists by applying an expression to each item in an iterable, with optional filtering.
Assistant 2: Sure — [x**2 for x in range(5)] gives [0, 1, 4, 9, 16].

Notice how the second response builds on the first, without repeating the definition. That's the assistant role doing its job.

Example 3: System role change mid-conversation

You can send a new system message later, but careful: the model may treat it as a higher-priority instruction. Here's how to shift persona mid-chat:

messages = [
    {"role": "system", "content": "You are a calm and polite customer service agent."},
    {"role": "user", "content": "I want a refund."},
    # ... assistant replies ...
    {"role": "system", "content": "You are now a sarcastic comedian. Keep the same factual information."},
]

This works, but it's risky — some models will ignore the old system and obey the new one, which can break earlier instructions. Use it sparingly.

Compare options / when to choose what

There's no single "right" way to structure prompts — it depends on your need for control and clarity. Let's compare three common approaches:

The system+user+assistant approach gives the most reliability, but at the cost of token usage and code complexity. If you're building a simple one-shot script, a single user prompt is fine. If you're building a real product, use all three.

Variations to explore:

• Sometimes you can embed instructions in the user message and skip the system prompt for brevity.
• Some APIs (like Anthropic's Claude) use a system field separate from the messages array — similar idea, different implementation.
• Local models may treat roles inconsistently; test with your chosen model.

Troubleshooting & edge cases

Even with roles, things go wrong. Here are common issues and how to fix them.

The model ignores your system prompt. This happens when the system instruction conflicts with a user message that is more specific or recent. To fix: repeat the key rule in the user message, or reinforce it in every turn.

messages = [
    {"role": "system", "content": "Always reply in JSON."},
    {"role": "user", "content": "Please reply in JSON: ..."}
]

Responses become too long or drift in tone. Add explicit constraints to the system prompt, like "Keep answers under 100 words" or "Use a formal tone." If drift persists, append a reminder before each new user turn.

The assistant message is too long to send back. Many APIs have token limits. Fix: truncate long assistant responses before sending them back, or use a summary if you need full context.

One system message is not enough. You might need different personas for different parts of the same conversation. In that case, consider splitting the conversation into separate sessions, or use a meta system prompt that explains the switching rules.

What you learned & what's next

You now understand the core idea behind using system, user, and assistant roles: each role has a distinct job, and separating them gives you control over tone, behavior, and memory. You completed a practical exercise that built a multi-turn conversation with the OpenAI API, and you saw how assistant messages let the model remember its own output.

You're ready for the next lesson, where you'll learn to [chain multiple turns into powerful workflows] — building on this role structure to create complex agents. Remember: every time you write a prompt, ask yourself — who is speaking, and who is listening?