wangtao
/
autogen
Not watched
1
0
Fork 0
Code
Releases 127 Wiki evaluate Activity
Issues 0 Pull Requests 0 Datasets Model Cloudbrain HPC
Browse Source

Add tool_agent_caller_loop and group chat notebook. (#405) 

* Add tool_agent_caller_loop and group chat notebook.

* Fix types

* fix ref

---------

Co-authored-by: Jack Gerrits <jackgerrits@users.noreply.github.com>
tags/v0.4.0.dev0
Eric Zhu GitHub 1 year ago
parent
c8f6f3bb38
commit
12cf331e71
9 changed files with 924 additions and 354 deletions
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. +1
    -1
      python/docs/src/core-concepts/overview.md
  2. +399
    -0
      python/docs/src/getting-started/group-chat.ipynb
  3. +18
    -0
      python/docs/src/getting-started/multi-agent-design-patterns.md
  4. +5
    -28
      python/docs/src/getting-started/reflection.ipynb
  5. +318
    -322
      python/docs/src/getting-started/tools.ipynb
  6. +2
    -0
      python/docs/src/index.rst
  7. +2
    -0
      python/src/agnext/components/tool_agent/__init__.py
  8. +77
    -0
      python/src/agnext/components/tool_agent/_caller_loop.py
  9. +102
    -3
      python/tests/test_tool_agent.py

+ 1
- 1
python/docs/src/core-concepts/overview.md View File

@@ -116,7 +116,7 @@ implementation of the contracts determines how agents handle messages.
The behavior contract is sometimes referred to as the message protocol.
It is the developer's responsibility to implement the behavior contract.
Multi-agent patterns are design patterns that emerge from behavior contracts
(see [Multi-Agent Design Patterns](../getting-started/multi-agent-design-patterns.ipynb)).
(see [Multi-Agent Design Patterns](../getting-started/multi-agent-design-patterns.md)).

### An Example Application



+ 399
- 0
python/docs/src/getting-started/group-chat.ipynb
File diff suppressed because it is too large
View File


+ 18
- 0
python/docs/src/getting-started/multi-agent-design-patterns.md View File

@@ -0,0 +1,18 @@
# Multi-Agent Design Patterns

Agents can work together in a variety of ways to solve problems.
Research works like [AutoGen](https://aka.ms/autogen-paper),
[MetaGPT](https://arxiv.org/abs/2308.00352)
and [ChatDev](https://arxiv.org/abs/2307.07924) have shown
multi-agent systems out-performing single agent systems at complex tasks
like software development.

A multi-agent design pattern is a structure that emerges from message protocols:
it describes how agents interact with each other to solve problems.
For example, the [tool-equiped agent](./tools.ipynb#tool-equipped-agent) in
the previous section employs a design pattern called ReAct,
which involves an agent interacting with tools.

You can implement any multi-agent design pattern using AGNext agents.
In the next two sections, we will discuss two common design patterns:
group chat for task decomposition, and reflection for robustness.

python/docs/src/getting-started/multi-agent-design-patterns.ipynb → python/docs/src/getting-started/reflection.ipynb View File

@@ -4,30 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Multi-Agent Design Patterns\n",
"\n",
"Agents can work together in a variety of ways to solve problems.\n",
"Research works like [AutoGen](https://aka.ms/autogen-paper),\n",
"[MetaGPT](https://arxiv.org/abs/2308.00352)\n",
"and [ChatDev](https://arxiv.org/abs/2307.07924) have shown\n",
"multi-agent systems out-performing single agent systems at complex tasks\n",
"like software development.\n",
"\n",
"A multi-agent design pattern is a structure that emerges from message protocols:\n",
"it describes how agents interact with each other to solve problems.\n",
"For example, the [tool-equiped agent](./tools.ipynb#tool-equipped-agent) in\n",
"the previous section employs a design pattern called ReAct,\n",
"which involves an agent interacting with tools.\n",
"\n",
"You can implement any multi-agent design pattern using AGNext agents.\n",
"In this section, we use the reflection pattern as an example."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Reflection\n",
"# Reflection\n",
"\n",
"Reflection is a design pattern where an LLM generation is followed by a reflection,\n",
"which in itself is another LLM generation conditioned on the output of the first one.\n",
@@ -50,7 +27,7 @@
"will generate a code snippet, and the reviewer agent will generate a critique\n",
"of the code snippet.\n",
"\n",
"### Message Protocol\n",
"## Message Protocol\n",
"\n",
"Before we define the agents, we need to first define the message protocol for the agents."
]
@@ -107,7 +84,7 @@
"\n",
"![coder-reviewer data flow](coder-reviewer-data-flow.svg)\n",
"\n",
"### Agents\n",
"## Agents\n",
"\n",
"Now, let's define the agents for the reflection design pattern."
]
@@ -376,7 +353,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Logging\n",
"## Logging\n",
"\n",
"Turn on logging to see the messages exchanged between the agents."
]
@@ -397,7 +374,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Running the Design Pattern\n",
"## Running the Design Pattern\n",
"\n",
"Let's test the design pattern with a coding task."
]

+ 318
- 322
python/docs/src/getting-started/tools.ipynb View File

@@ -1,324 +1,320 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tools\n",
"\n",
"Tools are code that can be executed by an agent to perform actions. A tool\n",
"can be a simple function such as a calculator, or an API call to a third-party service\n",
"such as stock price lookup and weather forecast.\n",
"In the context of AI agents, tools are designed to be executed by agents in\n",
"response to model-generated function calls.\n",
"\n",
"AGNext provides the {py:mod}`agnext.components.tools` module with a suite of built-in\n",
"tools and utilities for creating and running custom tools."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Built-in Tools\n",
"\n",
"One of the built-in tools is the {py:class}`agnext.components.tools.PythonCodeExecutionTool`,\n",
"which allows agents to execute Python code snippets.\n",
"\n",
"Here is how you create the tool and use it."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from agnext.components.code_executor import LocalCommandLineCodeExecutor\n",
"from agnext.components.tools import PythonCodeExecutionTool\n",
"from agnext.core import CancellationToken\n",
"\n",
"# Create the tool.\n",
"code_executor = LocalCommandLineCodeExecutor()\n",
"code_execution_tool = PythonCodeExecutionTool(code_executor)\n",
"cancellation_token = CancellationToken()\n",
"\n",
"# Use the tool directly without an agent.\n",
"code = \"print('Hello, world!')\"\n",
"result = await code_execution_tool.run_json({\"code\": code}, cancellation_token)\n",
"print(code_execution_tool.return_value_as_string(result))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The {py:class}`~agnext.components.code_executor.LocalCommandLineCodeExecutor`\n",
"class is a built-in code executor that runs Python code snippets in a subprocess\n",
"in the local command line environment.\n",
"The {py:class}`~agnext.components.tools.PythonCodeExecutionTool` class wraps the code executor\n",
"and provides a simple interface to execute Python code snippets.\n",
"\n",
"Other built-in tools will be added in the future."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Custom Function Tools\n",
"\n",
"A tool can also be a simple Python function that performs a specific action.\n",
"To create a custom function tool, you just need to create a Python function\n",
"and use the {py:class}`agnext.components.tools.FunctionTool` class to wrap it.\n",
"\n",
"For example, a simple tool to obtain the stock price of a company might look like this:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"138.75280591295171\n"
]
}
],
"source": [
"import random\n",
"\n",
"from agnext.components.tools import FunctionTool\n",
"from agnext.core import CancellationToken\n",
"from typing_extensions import Annotated\n",
"\n",
"\n",
"async def get_stock_price(ticker: str, date: Annotated[str, \"Date in YYYY/MM/DD\"]) -> float:\n",
" # Returns a random stock price for demonstration purposes.\n",
" return random.uniform(10, 200)\n",
"\n",
"\n",
"# Create a function tool.\n",
"stock_price_tool = FunctionTool(get_stock_price, description=\"Get the stock price.\")\n",
"\n",
"# Run the tool.\n",
"cancellation_token = CancellationToken()\n",
"result = await stock_price_tool.run_json({\"ticker\": \"AAPL\", \"date\": \"2021/01/01\"}, cancellation_token)\n",
"\n",
"# Print the result.\n",
"print(stock_price_tool.return_value_as_string(result))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Tool-Equipped Agent\n",
"\n",
"To use tools with an agent, you can use {py:class}`agnext.components.tool_agent.ToolAgent`,\n",
"by using it in a composition pattern.\n",
"Here is an example tool-use agent that uses {py:class}`~agnext.components.tool_agent.ToolAgent`\n",
"as an inner agent for executing tools."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"import asyncio\n",
"from dataclasses import dataclass\n",
"from typing import List\n",
"\n",
"from agnext.application import SingleThreadedAgentRuntime\n",
"from agnext.components import FunctionCall, RoutedAgent, message_handler\n",
"from agnext.components.models import (\n",
" AssistantMessage,\n",
" ChatCompletionClient,\n",
" FunctionExecutionResult,\n",
" FunctionExecutionResultMessage,\n",
" LLMMessage,\n",
" OpenAIChatCompletionClient,\n",
" SystemMessage,\n",
" UserMessage,\n",
")\n",
"from agnext.components.tool_agent import ToolAgent, ToolException\n",
"from agnext.components.tools import FunctionTool, Tool, ToolSchema\n",
"from agnext.core import AgentId, AgentInstantiationContext, MessageContext\n",
"\n",
"\n",
"@dataclass\n",
"class Message:\n",
" content: str\n",
"\n",
"\n",
"class ToolUseAgent(RoutedAgent):\n",
" def __init__(self, model_client: ChatCompletionClient, tool_schema: List[ToolSchema], tool_agent: AgentId) -> None:\n",
" super().__init__(\"An agent with tools\")\n",
" self._system_messages: List[LLMMessage] = [SystemMessage(\"You are a helpful AI assistant.\")]\n",
" self._model_client = model_client\n",
" self._tool_schema = tool_schema\n",
" self._tool_agent = tool_agent\n",
"\n",
" @message_handler\n",
" async def handle_user_message(self, message: Message, ctx: MessageContext) -> Message:\n",
" # Create a session of messages.\n",
" session: List[LLMMessage] = [UserMessage(content=message.content, source=\"user\")]\n",
" # Get a response from the model.\n",
" response = await self._model_client.create(\n",
" self._system_messages + session, tools=self._tool_schema, cancellation_token=cancellation_token\n",
" )\n",
" # Add the response to the session.\n",
" session.append(AssistantMessage(content=response.content, source=\"assistant\"))\n",
"\n",
" # Keep iterating until the model stops generating tool calls.\n",
" while isinstance(response.content, list) and all(isinstance(item, FunctionCall) for item in response.content):\n",
" # Execute functions called by the model by sending messages to itself.\n",
" results: List[FunctionExecutionResult | BaseException] = await asyncio.gather(\n",
" *[self.send_message(call, self._tool_agent) for call in response.content],\n",
" return_exceptions=True,\n",
" )\n",
" # Combine the results into a single response and handle exceptions.\n",
" function_results: List[FunctionExecutionResult] = []\n",
" for result in results:\n",
" if isinstance(result, FunctionExecutionResult):\n",
" function_results.append(result)\n",
" elif isinstance(result, ToolException):\n",
" function_results.append(FunctionExecutionResult(content=f\"Error: {result}\", call_id=result.call_id))\n",
" elif isinstance(result, BaseException):\n",
" raise result # Unexpected exception.\n",
" session.append(FunctionExecutionResultMessage(content=function_results))\n",
" # Query the model again with the new response.\n",
" response = await self._model_client.create(\n",
" self._system_messages + session, tools=self._tool_schema, cancellation_token=cancellation_token\n",
" )\n",
" session.append(AssistantMessage(content=response.content, source=self.metadata[\"type\"]))\n",
"\n",
" # Return the final response.\n",
" assert isinstance(response.content, str)\n",
" return Message(content=response.content)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `ToolUseAgent` class is a bit involved, however,\n",
"the core idea can be described using a simple control flow graph:\n",
"\n",
"![ToolUseAgent control flow graph](tool-use-agent-cfg.svg)\n",
"\n",
"The `ToolUseAgent`'s `handle_user_message` handler handles messages from the user,\n",
"and determines whether the model has generated a tool call.\n",
"If the model has generated tool calls, then the handler sends a function call\n",
"message to the {py:class}`~agnext.components.tool_agent.ToolAgent` agent\n",
"to execute the tools,\n",
"and then queries the model again with the results of the tool calls.\n",
"This process continues until the model stops generating tool calls,\n",
"at which point the final response is returned to the user.\n",
"\n",
"By having the tool execution logic in a separate agent,\n",
"we expose the model-tool interactions to the agent runtime as messages, so the tool executions\n",
"can be observed externally and intercepted if necessary.\n",
"\n",
"To run the agent, we need to create a runtime and register the agent."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"# Create a runtime.\n",
"runtime = SingleThreadedAgentRuntime()\n",
"# Create the tools.\n",
"tools: List[Tool] = [FunctionTool(get_stock_price, description=\"Get the stock price.\")]\n",
"# Register the agents.\n",
"await runtime.register(\n",
" \"tool-executor-agent\",\n",
" lambda: ToolAgent(\n",
" description=\"Tool Executor Agent\",\n",
" tools=tools,\n",
" ),\n",
")\n",
"await runtime.register(\n",
" \"tool-use-agent\",\n",
" lambda: ToolUseAgent(\n",
" OpenAIChatCompletionClient(model=\"gpt-4o-mini\"),\n",
" tool_schema=[tool.schema for tool in tools],\n",
" tool_agent=AgentId(\"tool-executor-agent\", AgentInstantiationContext.current_agent_id().key),\n",
" ),\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This example uses the {py:class}`agnext.components.models.OpenAIChatCompletionClient`,\n",
"for Azure OpenAI and other clients, see [Model Clients](./model-clients.ipynb).\n",
"Let's test the agent with a question about stock price."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The stock price of NVDA on June 1, 2024, is approximately $49.28.\n"
]
}
],
"source": [
"# Start processing messages.\n",
"runtime.start()\n",
"# Send a direct message to the tool agent.\n",
"tool_use_agent = AgentId(\"tool-use-agent\", \"default\")\n",
"response = await runtime.send_message(Message(\"What is the stock price of NVDA on 2024/06/01?\"), tool_use_agent)\n",
"print(response.content)\n",
"# Stop processing messages.\n",
"await runtime.stop()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"See [samples](https://github.com/microsoft/agnext/tree/main/python/samples#tool-use-examples)\n",
"for more examples of using tools with agents, including how to use\n",
"broadcast communication model for tool execution, and how to intercept tool\n",
"execution for human-in-the-loop approval."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "agnext",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
}
},
"nbformat": 4,
"nbformat_minor": 2
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Tools\n",
"\n",
"Tools are code that can be executed by an agent to perform actions. A tool\n",
"can be a simple function such as a calculator, or an API call to a third-party service\n",
"such as stock price lookup and weather forecast.\n",
"In the context of AI agents, tools are designed to be executed by agents in\n",
"response to model-generated function calls.\n",
"\n",
"AGNext provides the {py:mod}`agnext.components.tools` module with a suite of built-in\n",
"tools and utilities for creating and running custom tools."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Built-in Tools\n",
"\n",
"One of the built-in tools is the {py:class}`agnext.components.tools.PythonCodeExecutionTool`,\n",
"which allows agents to execute Python code snippets.\n",
"\n",
"Here is how you create the tool and use it."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Hello, world!\n",
"\n"
]
}
],
"source": [
"from agnext.components.code_executor import LocalCommandLineCodeExecutor\n",
"from agnext.components.tools import PythonCodeExecutionTool\n",
"from agnext.core import CancellationToken\n",
"\n",
"# Create the tool.\n",
"code_executor = LocalCommandLineCodeExecutor()\n",
"code_execution_tool = PythonCodeExecutionTool(code_executor)\n",
"cancellation_token = CancellationToken()\n",
"\n",
"# Use the tool directly without an agent.\n",
"code = \"print('Hello, world!')\"\n",
"result = await code_execution_tool.run_json({\"code\": code}, cancellation_token)\n",
"print(code_execution_tool.return_value_as_string(result))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The {py:class}`~agnext.components.code_executor.LocalCommandLineCodeExecutor`\n",
"class is a built-in code executor that runs Python code snippets in a subprocess\n",
"in the local command line environment.\n",
"The {py:class}`~agnext.components.tools.PythonCodeExecutionTool` class wraps the code executor\n",
"and provides a simple interface to execute Python code snippets.\n",
"\n",
"Other built-in tools will be added in the future."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Custom Function Tools\n",
"\n",
"A tool can also be a simple Python function that performs a specific action.\n",
"To create a custom function tool, you just need to create a Python function\n",
"and use the {py:class}`agnext.components.tools.FunctionTool` class to wrap it.\n",
"\n",
"For example, a simple tool to obtain the stock price of a company might look like this:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"194.71306528148511\n"
]
}
],
"source": [
"import random\n",
"\n",
"from agnext.components.tools import FunctionTool\n",
"from agnext.core import CancellationToken\n",
"from typing_extensions import Annotated\n",
"\n",
"\n",
"async def get_stock_price(ticker: str, date: Annotated[str, \"Date in YYYY/MM/DD\"]) -> float:\n",
" # Returns a random stock price for demonstration purposes.\n",
" return random.uniform(10, 200)\n",
"\n",
"\n",
"# Create a function tool.\n",
"stock_price_tool = FunctionTool(get_stock_price, description=\"Get the stock price.\")\n",
"\n",
"# Run the tool.\n",
"cancellation_token = CancellationToken()\n",
"result = await stock_price_tool.run_json({\"ticker\": \"AAPL\", \"date\": \"2021/01/01\"}, cancellation_token)\n",
"\n",
"# Print the result.\n",
"print(stock_price_tool.return_value_as_string(result))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Tool-Equipped Agent\n",
"\n",
"To use tools with an agent, you can use {py:class}`agnext.components.tool_agent.ToolAgent`,\n",
"by using it in a composition pattern.\n",
"Here is an example tool-use agent that uses {py:class}`~agnext.components.tool_agent.ToolAgent`\n",
"as an inner agent for executing tools."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"from dataclasses import dataclass\n",
"from typing import List\n",
"\n",
"from agnext.application import SingleThreadedAgentRuntime\n",
"from agnext.components import RoutedAgent, message_handler\n",
"from agnext.components.models import (\n",
" ChatCompletionClient,\n",
" LLMMessage,\n",
" OpenAIChatCompletionClient,\n",
" SystemMessage,\n",
" UserMessage,\n",
")\n",
"from agnext.components.tool_agent import ToolAgent, tool_agent_caller_loop\n",
"from agnext.components.tools import FunctionTool, Tool, ToolSchema\n",
"from agnext.core import AgentId, AgentInstantiationContext, MessageContext\n",
"\n",
"\n",
"@dataclass\n",
"class Message:\n",
" content: str\n",
"\n",
"\n",
"class ToolUseAgent(RoutedAgent):\n",
" def __init__(self, model_client: ChatCompletionClient, tool_schema: List[ToolSchema], tool_agent: AgentId) -> None:\n",
" super().__init__(\"An agent with tools\")\n",
" self._system_messages: List[LLMMessage] = [SystemMessage(\"You are a helpful AI assistant.\")]\n",
" self._model_client = model_client\n",
" self._tool_schema = tool_schema\n",
" self._tool_agent = tool_agent\n",
"\n",
" @message_handler\n",
" async def handle_user_message(self, message: Message, ctx: MessageContext) -> Message:\n",
" # Create a session of messages.\n",
" session: List[LLMMessage] = [UserMessage(content=message.content, source=\"user\")]\n",
" # Run the caller loop to handle tool calls.\n",
" messages = await tool_agent_caller_loop(\n",
" self,\n",
" tool_agent_id=self._tool_agent,\n",
" model_client=self._model_client,\n",
" input_messages=session,\n",
" tool_schema=self._tool_schema,\n",
" cancellation_token=ctx.cancellation_token,\n",
" )\n",
" # Return the final response.\n",
" assert isinstance(messages[-1].content, str)\n",
" return Message(content=messages[-1].content)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `ToolUseAgent` class uses a convenience function {py:meth}`agnext.components.tool_agent.tool_agent_caller_loop`, \n",
"to handle the interaction between the model and the tool agent.\n",
"The core idea can be described using a simple control flow graph:\n",
"\n",
"![ToolUseAgent control flow graph](tool-use-agent-cfg.svg)\n",
"\n",
"The `ToolUseAgent`'s `handle_user_message` handler handles messages from the user,\n",
"and determines whether the model has generated a tool call.\n",
"If the model has generated tool calls, then the handler sends a function call\n",
"message to the {py:class}`~agnext.components.tool_agent.ToolAgent` agent\n",
"to execute the tools,\n",
"and then queries the model again with the results of the tool calls.\n",
"This process continues until the model stops generating tool calls,\n",
"at which point the final response is returned to the user.\n",
"\n",
"By having the tool execution logic in a separate agent,\n",
"we expose the model-tool interactions to the agent runtime as messages, so the tool executions\n",
"can be observed externally and intercepted if necessary.\n",
"\n",
"To run the agent, we need to create a runtime and register the agent."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"AgentType(type='tool_use_agent')"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Create a runtime.\n",
"runtime = SingleThreadedAgentRuntime()\n",
"# Create the tools.\n",
"tools: List[Tool] = [FunctionTool(get_stock_price, description=\"Get the stock price.\")]\n",
"# Register the agents.\n",
"await runtime.register(\n",
" \"tool_executor_agent\",\n",
" lambda: ToolAgent(\n",
" description=\"Tool Executor Agent\",\n",
" tools=tools,\n",
" ),\n",
")\n",
"await runtime.register(\n",
" \"tool_use_agent\",\n",
" lambda: ToolUseAgent(\n",
" OpenAIChatCompletionClient(model=\"gpt-4o-mini\"),\n",
" tool_schema=[tool.schema for tool in tools],\n",
" tool_agent=AgentId(\"tool_executor_agent\", AgentInstantiationContext.current_agent_id().key),\n",
" ),\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This example uses the {py:class}`agnext.components.models.OpenAIChatCompletionClient`,\n",
"for Azure OpenAI and other clients, see [Model Clients](./model-clients.ipynb).\n",
"Let's test the agent with a question about stock price."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The stock price of NVIDIA (NVDA) on June 1, 2024, was approximately $148.86.\n"
]
}
],
"source": [
"# Start processing messages.\n",
"runtime.start()\n",
"# Send a direct message to the tool agent.\n",
"tool_use_agent = AgentId(\"tool_use_agent\", \"default\")\n",
"response = await runtime.send_message(Message(\"What is the stock price of NVDA on 2024/06/01?\"), tool_use_agent)\n",
"print(response.content)\n",
"# Stop processing messages.\n",
"await runtime.stop()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"See [samples](https://github.com/microsoft/agnext/tree/main/python/samples#tool-use-examples)\n",
"for more examples of using tools with agents, including how to use\n",
"broadcast communication model for tool execution, and how to intercept tool\n",
"execution for human-in-the-loop approval."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "agnext",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

+ 2
- 0
python/docs/src/index.rst View File

@@ -30,6 +30,8 @@ To learn about the core concepts of AGNext, read the `overview <core-concepts/ov
getting-started/model-clients
getting-started/tools
getting-started/multi-agent-design-patterns
getting-started/group-chat
getting-started/reflection

.. toctree::
:caption: Guides


+ 2
- 0
python/src/agnext/components/tool_agent/__init__.py View File

@@ -1,3 +1,4 @@
from ._caller_loop import tool_agent_caller_loop
from ._tool_agent import (
InvalidToolArgumentsException,
ToolAgent,
@@ -12,4 +13,5 @@ __all__ = [
"ToolNotFoundException",
"InvalidToolArgumentsException",
"ToolExecutionException",
"tool_agent_caller_loop",
]

+ 77
- 0
python/src/agnext/components/tool_agent/_caller_loop.py View File

@@ -0,0 +1,77 @@
import asyncio
from typing import List

from ...components import FunctionCall
from ...core import AgentId, AgentRuntime, BaseAgent, CancellationToken
from ..models import (
AssistantMessage,
ChatCompletionClient,
FunctionExecutionResult,
FunctionExecutionResultMessage,
LLMMessage,
)
from ..tools import Tool, ToolSchema
from ._tool_agent import ToolException


async def tool_agent_caller_loop(
caller: BaseAgent | AgentRuntime,
tool_agent_id: AgentId,
model_client: ChatCompletionClient,
input_messages: List[LLMMessage],
tool_schema: List[ToolSchema] | List[Tool],
cancellation_token: CancellationToken | None = None,
caller_source: str = "assistant",
) -> List[LLMMessage]:
"""Start a caller loop for a tool agent. This function sends messages to the tool agent
and the model client in an alternating fashion until the model client stops generating tool calls.

Args:
tool_agent_id (AgentId): The Agent ID of the tool agent.
input_messages (List[LLMMessage]): The list of input messages.
model_client (ChatCompletionClient): The model client to use for the model API.
tool_schema (List[Tool | ToolSchema]): The list of tools that the model can use.

Returns:
List[LLMMessage]: The list of output messages created in the caller loop.
"""

generated_messages: List[LLMMessage] = []

# Get a response from the model.
response = await model_client.create(input_messages, tools=tool_schema, cancellation_token=cancellation_token)
# Add the response to the generated messages.
generated_messages.append(AssistantMessage(content=response.content, source=caller_source))

# Keep iterating until the model stops generating tool calls.
while isinstance(response.content, list) and all(isinstance(item, FunctionCall) for item in response.content):
# Execute functions called by the model by sending messages to tool agent.
results: List[FunctionExecutionResult | BaseException] = await asyncio.gather(
*[
caller.send_message(
message=call,
recipient=tool_agent_id,
cancellation_token=cancellation_token,
)
for call in response.content
],
return_exceptions=True,
)
# Combine the results into a single response and handle exceptions.
function_results: List[FunctionExecutionResult] = []
for result in results:
if isinstance(result, FunctionExecutionResult):
function_results.append(result)
elif isinstance(result, ToolException):
function_results.append(FunctionExecutionResult(content=f"Error: {result}", call_id=result.call_id))
elif isinstance(result, BaseException):
raise result # Unexpected exception.
generated_messages.append(FunctionExecutionResultMessage(content=function_results))
# Query the model again with the new response.
response = await model_client.create(
input_messages + generated_messages, tools=tool_schema, cancellation_token=cancellation_token
)
generated_messages.append(AssistantMessage(content=response.content, source=caller_source))

# Return the generated messages.
return generated_messages

+ 102
- 3
python/tests/test_tool_agent.py View File

@@ -1,19 +1,34 @@
import asyncio
import json
from typing import Any, AsyncGenerator, List

import pytest
from openai.resources.chat.completions import AsyncCompletions
from openai.types.chat.chat_completion import ChatCompletion, Choice
from openai.types.chat.chat_completion_chunk import ChatCompletionChunk
from openai.types.chat.chat_completion_message import ChatCompletionMessage
from openai.types.chat.chat_completion_message_tool_call import ChatCompletionMessageToolCall, Function
from openai.types.completion_usage import CompletionUsage

from agnext.application import SingleThreadedAgentRuntime
from agnext.components import FunctionCall
from agnext.components.models import FunctionExecutionResult
from agnext.components.tool_agent import (
InvalidToolArgumentsException,
ToolAgent,
ToolExecutionException,
ToolNotFoundException,
tool_agent_caller_loop,
)
from agnext.components.tools import FunctionTool, Tool
from agnext.core import CancellationToken, AgentId
from agnext.components.models import (
AssistantMessage,
FunctionExecutionResult,
FunctionExecutionResultMessage,
OpenAIChatCompletionClient,
UserMessage,
)
from agnext.components.tools import FunctionTool
from agnext.core import CancellationToken
from agnext.core import AgentId


def _pass_function(input: str) -> str:
@@ -29,6 +44,60 @@ async def _async_sleep_function(input: str) -> str:
return "pass"


class _MockChatCompletion:
def __init__(self, model: str = "gpt-4o") -> None:
self._saved_chat_completions: List[ChatCompletion] = [
ChatCompletion(
id="id1",
choices=[
Choice(
finish_reason="tool_calls",
index=0,
message=ChatCompletionMessage(
content=None,
tool_calls=[
ChatCompletionMessageToolCall(
id="1",
type="function",
function=Function(
name="pass",
arguments=json.dumps({"input": "pass"}),
),
)
],
role="assistant",
),
)
],
created=0,
model=model,
object="chat.completion",
usage=CompletionUsage(prompt_tokens=0, completion_tokens=0, total_tokens=0),
),
ChatCompletion(
id="id2",
choices=[
Choice(
finish_reason="stop", index=0, message=ChatCompletionMessage(content="Hello", role="assistant")
)
],
created=0,
model=model,
object="chat.completion",
usage=CompletionUsage(prompt_tokens=0, completion_tokens=0, total_tokens=0),
),
]
self._curr_index = 0

async def mock_create(
self, *args: Any, **kwargs: Any
) -> ChatCompletion | AsyncGenerator[ChatCompletionChunk, None]:
await asyncio.sleep(0.1)
completion = self._saved_chat_completions[self._curr_index]
self._curr_index += 1
return completion


@pytest.mark.asyncio
async def test_tool_agent() -> None:
runtime = SingleThreadedAgentRuntime()
@@ -74,3 +143,33 @@ async def test_tool_agent() -> None:
await result_future

await runtime.stop()


@pytest.mark.asyncio
async def test_caller_loop(monkeypatch: pytest.MonkeyPatch) -> None:
mock = _MockChatCompletion(model="gpt-4o-2024-05-13")
monkeypatch.setattr(AsyncCompletions, "create", mock.mock_create)
client = OpenAIChatCompletionClient(model="gpt-4o-2024-05-13", api_key="api_key")
tools : List[Tool] = [FunctionTool(_pass_function, name="pass", description="Pass function")]
runtime = SingleThreadedAgentRuntime()
await runtime.register(
"tool_agent",
lambda: ToolAgent(
description="Tool agent",
tools=tools,
),
)
agent = AgentId("tool_agent", "default")
runtime.start()
messages = await tool_agent_caller_loop(
runtime,
agent,
client,
[UserMessage(content="Hello", source="user")],
tool_schema=tools
)
assert len(messages) == 3
assert isinstance(messages[0], AssistantMessage)
assert isinstance(messages[1], FunctionExecutionResultMessage)
assert isinstance(messages[2], AssistantMessage)
await runtime.stop()

Write Preview
Loading…
Cancel
Save