Learn › Local AI Agents: Hermes 3 and the Model Context Protocol › Hermes 3 Tool Calling: ChatML, tool_call, and tool_response

Hermes 3 Tool Calling: ChatML, tool_call, and tool_response

Advanced 🕐 12 min Lesson 8 of 12
What you'll learn
  • Write the Hermes 3 system prompt preamble and tools block correctly to enable tool calling
  • Explain the structure of a tool_call response and how to extract the tool name and arguments from it
  • Inject a tool result back into the conversation using the tool role and tool_response tags
  • Run a direct test of the tool calling format against Ollama and verify the model produces a tool_call block

ChatML: The Conversation Format

Hermes 3 uses ChatML as its conversation format. Each turn is delimited by special tokens that mark the role and the start and end of the content:

<|im_start|>system
You are a helpful assistant.
<|im_end|>
<|im_start|>user
What is the weather in London?
<|im_end|>
<|im_start|>assistant

When you use the Ollama chat API, you do not write these tokens manually — you pass a list of message dicts with role and content fields, and Ollama formats them into ChatML before passing to the model. The tokens are listed here so you understand what the model actually receives.

Valid roles in a Hermes 3 conversation: system, user, assistant, and tool. The tool role is used to inject tool results back into the conversation — this is not a standard ChatML role but was added by Hermes during fine-tuning.

The System Prompt for Tool Calling

To enable tool calling, the system prompt must do two things: declare that tools are available and provide their schemas. Hermes 3 expects a specific structure:

You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Do not make assumptions about what values to plug into functions.

<tools>
[
  {
    "type": "function",
    "function": {
      "name": "add_note",
      "description": "Save a new research note with a title and content.",
      "parameters": {
        "type": "object",
        "properties": {
          "title": {"type": "string"},
          "content": {"type": "string"}
        },
        "required": ["title", "content"]
      }
    }
  }
]
</tools>

For each function call return a JSON object with function name and arguments within <tool_call></tool_call> XML tags as follows:
<tool_call>
{"name": "function_name", "arguments": {"arg1": "value1"}}
</tool_call>

If no tool is relevant, respond in natural language.

Three elements are essential:

  1. The preamble sentence: "You are a function calling AI model..." — without this, Hermes does not activate its tool-calling behaviour reliably
  2. The <tools> block containing a JSON array of tool definitions in OpenAI function-calling format
  3. The instruction to return calls inside <tool_call> tags

The agent code you build in Lesson 9 constructs this system prompt automatically from the tool list returned by the MCP servers. You will not write it by hand for each session.

The Tool Call Output Format

When Hermes 3 decides to call a tool, it outputs a <tool_call> block:

<tool_call>
{"name": "add_note", "arguments": {"title": "MCP Architecture", "content": "Three participants: host, client, server. Two transports: stdio and Streamable HTTP."}}
</tool_call>

Your agent code detects this by checking whether the response contains <tool_call>. If it does, the agent extracts the JSON, parses it, identifies which MCP server provides that tool, and calls it.

Hermes 3 sometimes generates text before the <tool_call> block, especially with the scratch pad reasoning feature. When parsing, search for the tag anywhere in the response rather than assuming the entire response is the tool call.

Injecting the Tool Result

After executing the tool call via MCP, inject the result back into the conversation as a tool role message wrapped in <tool_response> tags:

{
  "role": "tool",
  "content": "<tool_response>
{"name": "add_note", "content": "Note 1 saved: MCP Architecture"}
</tool_response>"
}

Then call the Ollama API again with the updated conversation history. Hermes 3 reads the tool response and either generates a final answer or issues another tool call if more information is needed.

Testing the Format Directly

Verify the format works before building the full agent loop. The following Python script sends a tool-enabled request to Ollama and prints the raw response:

import json, requests

SYSTEM_PROMPT = """You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags.

<tools>
[
  {
    "type": "function",
    "function": {
      "name": "get_current_time",
      "description": "Return the current date and time.",
      "parameters": {"type": "object", "additionalProperties": false}
    }
  }
]
</tools>

For each function call return a JSON object within <tool_call></tool_call> tags:
<tool_call>
{"name": "function_name", "arguments": {}}
</tool_call>

If no tool is relevant, respond in natural language."""

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": "What time is it right now?"},
]

resp = requests.post(
    "http://localhost:11434/v1/chat/completions",
    json={"model": "hermes3", "messages": messages, "temperature": 0.1, "stream": False},
)
reply = resp.json()["choices"][0]["message"]["content"]
print(repr(reply))

Run this and confirm the response contains a <tool_call> block with "name": "get_current_time". If you see the model responding in natural language about the time without a tool call, the system prompt was not formatted correctly — check for missing angle brackets or truncated JSON in the <tools> block.

The Optional Scratch Pad

Hermes 3 supports an optional reasoning mode called Goal Oriented Action Planning (GOAP). When enabled, the model outputs a <scratch_pad> block before the tool call containing its reasoning: the goal it is trying to achieve, the actions it plans to take, what it observed from previous tool calls, and a reflection on whether it is on track.

To enable it, add this to the system prompt after the tool call format instruction:

Before each tool call, reason through the task in <scratch_pad></scratch_pad> tags covering: Goal, Actions, Observation, and Reflection.

The scratch pad makes the model's planning visible and can improve accuracy on complex multi-step tasks. The tradeoff is higher token usage — roughly 100–300 extra tokens per tool call step. For the agent in this track, leaving it disabled is a sensible default and you can enable it if you find the model making poor tool selection decisions.

What the Agent Code Needs to Parse

To summarise, your agent loop needs to handle three response shapes from Hermes 3:

  • Tool call response: contains <tool_call>...</tool_call> — parse the JSON, execute the tool, inject the result, continue
  • Final answer: natural language response with no <tool_call> tag — return this to the user
  • Scratch pad + tool call: <scratch_pad>...</scratch_pad> followed by a tool call — ignore the scratch pad content, parse only the tool call

Lesson 10 builds this parsing logic into the full agent loop. Lesson 9 covers the other half: connecting to the MCP servers and translating their tool schemas into the Hermes system prompt format.

Key takeaways
  • The Hermes system prompt needs three elements: the preamble sentence, the tools JSON array inside tool_call tags, and the format instruction — missing any one of them causes unreliable tool activation
  • The tool role message wraps the result in tool_response tags — the JSON payload contains the function name and content; Hermes reads this before generating its next turn
  • Hermes sometimes outputs text before the tool_call block — parse by searching for the tag anywhere in the response, not by assuming the response IS the tool call
  • Temperature 0.1 is important for tool calling — higher temperatures cause the model to vary the JSON structure, which breaks parsing
Learn more every week Get 5 hand-picked prompts + new AI tools every Monday. Free.