You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.

teammanager.py 6.1 kB

Enable LLM Call Observability in AGS (#5457) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> It is often helpful to inspect the raw request and response to/from an LLM as agents act. This PR, does the following: - Updates TeamManager to yield LLMCallEvents from core library - Run in an async background task to listen for LLMCallEvent JIT style when a team is run - Add events to an async queue and - yield those events in addition to whatever actual agentchat team.run_stream yields. - Update the AGS UI to show those LLMCallEvents in the messages section as a team runs - Add settings panel to show/hide llm call events in messages. - Minor updates to default team <img width="1539" alt="image" src="https://github.com/user-attachments/assets/bfbb19fe-3560-4faa-b600-7dd244e0e974" /> <img width="1554" alt="image" src="https://github.com/user-attachments/assets/775624f5-ba83-46e8-81ff-512bfeed6bab" /> <img width="1538" alt="image" src="https://github.com/user-attachments/assets/3becbf85-b75a-4506-adb7-e5575ebbaec4" /> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5440 ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Add Token Streaming in AGS , Support Env variables (#5659) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> This PR has 3 main improvements. - Token streaming - Adds support for environment variables in the app settings - Updates AGS to persist Gallery entry in db. ## Adds Token Streaming in AGS. Agentchat now supports streaming of tokens via `ModelClientStreamingChunkEvent `. This PR is to track progress on supporting that in the AutoGen Studio UI. If `model_client_stream` is enabled in an assitant agent, then token will be streamed in UI. ```python streaming_assistant = AssistantAgent( name="assistant", model_client=model_client, system_message="You are a helpful assistant.", model_client_stream=True, # Enable streaming tokens. ) ``` https://github.com/user-attachments/assets/74d43d78-6359-40c3-a78e-c84dcb5e02a1 ## Env Variables Also adds support for env variables in AGS Settings You can set env variables that are loaded just before a team is run. Handy to set variable to be used by tools etc. <img width="1291" alt="image" src="https://github.com/user-attachments/assets/437b9d90-ccee-42f7-be5d-94ab191afd67" /> > Note: the set variables are available to the server process. <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5627 Closes #5662 Closes #5619 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Enable LLM Call Observability in AGS (#5457) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> It is often helpful to inspect the raw request and response to/from an LLM as agents act. This PR, does the following: - Updates TeamManager to yield LLMCallEvents from core library - Run in an async background task to listen for LLMCallEvent JIT style when a team is run - Add events to an async queue and - yield those events in addition to whatever actual agentchat team.run_stream yields. - Update the AGS UI to show those LLMCallEvents in the messages section as a team runs - Add settings panel to show/hide llm call events in messages. - Minor updates to default team <img width="1539" alt="image" src="https://github.com/user-attachments/assets/bfbb19fe-3560-4faa-b600-7dd244e0e974" /> <img width="1554" alt="image" src="https://github.com/user-attachments/assets/775624f5-ba83-46e8-81ff-512bfeed6bab" /> <img width="1538" alt="image" src="https://github.com/user-attachments/assets/3becbf85-b75a-4506-adb7-e5575ebbaec4" /> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5440 ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Add Token Streaming in AGS , Support Env variables (#5659) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> This PR has 3 main improvements. - Token streaming - Adds support for environment variables in the app settings - Updates AGS to persist Gallery entry in db. ## Adds Token Streaming in AGS. Agentchat now supports streaming of tokens via `ModelClientStreamingChunkEvent `. This PR is to track progress on supporting that in the AutoGen Studio UI. If `model_client_stream` is enabled in an assitant agent, then token will be streamed in UI. ```python streaming_assistant = AssistantAgent( name="assistant", model_client=model_client, system_message="You are a helpful assistant.", model_client_stream=True, # Enable streaming tokens. ) ``` https://github.com/user-attachments/assets/74d43d78-6359-40c3-a78e-c84dcb5e02a1 ## Env Variables Also adds support for env variables in AGS Settings You can set env variables that are loaded just before a team is run. Handy to set variable to be used by tools etc. <img width="1291" alt="image" src="https://github.com/user-attachments/assets/437b9d90-ccee-42f7-be5d-94ab191afd67" /> > Note: the set variables are available to the server process. <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5627 Closes #5662 Closes #5619 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Enable LLM Call Observability in AGS (#5457) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> It is often helpful to inspect the raw request and response to/from an LLM as agents act. This PR, does the following: - Updates TeamManager to yield LLMCallEvents from core library - Run in an async background task to listen for LLMCallEvent JIT style when a team is run - Add events to an async queue and - yield those events in addition to whatever actual agentchat team.run_stream yields. - Update the AGS UI to show those LLMCallEvents in the messages section as a team runs - Add settings panel to show/hide llm call events in messages. - Minor updates to default team <img width="1539" alt="image" src="https://github.com/user-attachments/assets/bfbb19fe-3560-4faa-b600-7dd244e0e974" /> <img width="1554" alt="image" src="https://github.com/user-attachments/assets/775624f5-ba83-46e8-81ff-512bfeed6bab" /> <img width="1538" alt="image" src="https://github.com/user-attachments/assets/3becbf85-b75a-4506-adb7-e5575ebbaec4" /> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5440 ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Enable LLM Call Observability in AGS (#5457) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> It is often helpful to inspect the raw request and response to/from an LLM as agents act. This PR, does the following: - Updates TeamManager to yield LLMCallEvents from core library - Run in an async background task to listen for LLMCallEvent JIT style when a team is run - Add events to an async queue and - yield those events in addition to whatever actual agentchat team.run_stream yields. - Update the AGS UI to show those LLMCallEvents in the messages section as a team runs - Add settings panel to show/hide llm call events in messages. - Minor updates to default team <img width="1539" alt="image" src="https://github.com/user-attachments/assets/bfbb19fe-3560-4faa-b600-7dd244e0e974" /> <img width="1554" alt="image" src="https://github.com/user-attachments/assets/775624f5-ba83-46e8-81ff-512bfeed6bab" /> <img width="1538" alt="image" src="https://github.com/user-attachments/assets/3becbf85-b75a-4506-adb7-e5575ebbaec4" /> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5440 ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Add Token Streaming in AGS , Support Env variables (#5659) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> This PR has 3 main improvements. - Token streaming - Adds support for environment variables in the app settings - Updates AGS to persist Gallery entry in db. ## Adds Token Streaming in AGS. Agentchat now supports streaming of tokens via `ModelClientStreamingChunkEvent `. This PR is to track progress on supporting that in the AutoGen Studio UI. If `model_client_stream` is enabled in an assitant agent, then token will be streamed in UI. ```python streaming_assistant = AssistantAgent( name="assistant", model_client=model_client, system_message="You are a helpful assistant.", model_client_stream=True, # Enable streaming tokens. ) ``` https://github.com/user-attachments/assets/74d43d78-6359-40c3-a78e-c84dcb5e02a1 ## Env Variables Also adds support for env variables in AGS Settings You can set env variables that are loaded just before a team is run. Handy to set variable to be used by tools etc. <img width="1291" alt="image" src="https://github.com/user-attachments/assets/437b9d90-ccee-42f7-be5d-94ab191afd67" /> > Note: the set variables are available to the server process. <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5627 Closes #5662 Closes #5619 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Add Token Streaming in AGS , Support Env variables (#5659) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> This PR has 3 main improvements. - Token streaming - Adds support for environment variables in the app settings - Updates AGS to persist Gallery entry in db. ## Adds Token Streaming in AGS. Agentchat now supports streaming of tokens via `ModelClientStreamingChunkEvent `. This PR is to track progress on supporting that in the AutoGen Studio UI. If `model_client_stream` is enabled in an assitant agent, then token will be streamed in UI. ```python streaming_assistant = AssistantAgent( name="assistant", model_client=model_client, system_message="You are a helpful assistant.", model_client_stream=True, # Enable streaming tokens. ) ``` https://github.com/user-attachments/assets/74d43d78-6359-40c3-a78e-c84dcb5e02a1 ## Env Variables Also adds support for env variables in AGS Settings You can set env variables that are loaded just before a team is run. Handy to set variable to be used by tools etc. <img width="1291" alt="image" src="https://github.com/user-attachments/assets/437b9d90-ccee-42f7-be5d-94ab191afd67" /> > Note: the set variables are available to the server process. <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5627 Closes #5662 Closes #5619 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Add Token Streaming in AGS , Support Env variables (#5659) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> This PR has 3 main improvements. - Token streaming - Adds support for environment variables in the app settings - Updates AGS to persist Gallery entry in db. ## Adds Token Streaming in AGS. Agentchat now supports streaming of tokens via `ModelClientStreamingChunkEvent `. This PR is to track progress on supporting that in the AutoGen Studio UI. If `model_client_stream` is enabled in an assitant agent, then token will be streamed in UI. ```python streaming_assistant = AssistantAgent( name="assistant", model_client=model_client, system_message="You are a helpful assistant.", model_client_stream=True, # Enable streaming tokens. ) ``` https://github.com/user-attachments/assets/74d43d78-6359-40c3-a78e-c84dcb5e02a1 ## Env Variables Also adds support for env variables in AGS Settings You can set env variables that are loaded just before a team is run. Handy to set variable to be used by tools etc. <img width="1291" alt="image" src="https://github.com/user-attachments/assets/437b9d90-ccee-42f7-be5d-94ab191afd67" /> > Note: the set variables are available to the server process. <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5627 Closes #5662 Closes #5619 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Enable LLM Call Observability in AGS (#5457) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> It is often helpful to inspect the raw request and response to/from an LLM as agents act. This PR, does the following: - Updates TeamManager to yield LLMCallEvents from core library - Run in an async background task to listen for LLMCallEvent JIT style when a team is run - Add events to an async queue and - yield those events in addition to whatever actual agentchat team.run_stream yields. - Update the AGS UI to show those LLMCallEvents in the messages section as a team runs - Add settings panel to show/hide llm call events in messages. - Minor updates to default team <img width="1539" alt="image" src="https://github.com/user-attachments/assets/bfbb19fe-3560-4faa-b600-7dd244e0e974" /> <img width="1554" alt="image" src="https://github.com/user-attachments/assets/775624f5-ba83-46e8-81ff-512bfeed6bab" /> <img width="1538" alt="image" src="https://github.com/user-attachments/assets/3becbf85-b75a-4506-adb7-e5575ebbaec4" /> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5440 ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Enable LLM Call Observability in AGS (#5457) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> It is often helpful to inspect the raw request and response to/from an LLM as agents act. This PR, does the following: - Updates TeamManager to yield LLMCallEvents from core library - Run in an async background task to listen for LLMCallEvent JIT style when a team is run - Add events to an async queue and - yield those events in addition to whatever actual agentchat team.run_stream yields. - Update the AGS UI to show those LLMCallEvents in the messages section as a team runs - Add settings panel to show/hide llm call events in messages. - Minor updates to default team <img width="1539" alt="image" src="https://github.com/user-attachments/assets/bfbb19fe-3560-4faa-b600-7dd244e0e974" /> <img width="1554" alt="image" src="https://github.com/user-attachments/assets/775624f5-ba83-46e8-81ff-512bfeed6bab" /> <img width="1538" alt="image" src="https://github.com/user-attachments/assets/3becbf85-b75a-4506-adb7-e5575ebbaec4" /> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5440 ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Add Token Streaming in AGS , Support Env variables (#5659) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> This PR has 3 main improvements. - Token streaming - Adds support for environment variables in the app settings - Updates AGS to persist Gallery entry in db. ## Adds Token Streaming in AGS. Agentchat now supports streaming of tokens via `ModelClientStreamingChunkEvent `. This PR is to track progress on supporting that in the AutoGen Studio UI. If `model_client_stream` is enabled in an assitant agent, then token will be streamed in UI. ```python streaming_assistant = AssistantAgent( name="assistant", model_client=model_client, system_message="You are a helpful assistant.", model_client_stream=True, # Enable streaming tokens. ) ``` https://github.com/user-attachments/assets/74d43d78-6359-40c3-a78e-c84dcb5e02a1 ## Env Variables Also adds support for env variables in AGS Settings You can set env variables that are loaded just before a team is run. Handy to set variable to be used by tools etc. <img width="1291" alt="image" src="https://github.com/user-attachments/assets/437b9d90-ccee-42f7-be5d-94ab191afd67" /> > Note: the set variables are available to the server process. <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5627 Closes #5662 Closes #5619 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Enable LLM Call Observability in AGS (#5457) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> It is often helpful to inspect the raw request and response to/from an LLM as agents act. This PR, does the following: - Updates TeamManager to yield LLMCallEvents from core library - Run in an async background task to listen for LLMCallEvent JIT style when a team is run - Add events to an async queue and - yield those events in addition to whatever actual agentchat team.run_stream yields. - Update the AGS UI to show those LLMCallEvents in the messages section as a team runs - Add settings panel to show/hide llm call events in messages. - Minor updates to default team <img width="1539" alt="image" src="https://github.com/user-attachments/assets/bfbb19fe-3560-4faa-b600-7dd244e0e974" /> <img width="1554" alt="image" src="https://github.com/user-attachments/assets/775624f5-ba83-46e8-81ff-512bfeed6bab" /> <img width="1538" alt="image" src="https://github.com/user-attachments/assets/3becbf85-b75a-4506-adb7-e5575ebbaec4" /> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5440 ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Enable LLM Call Observability in AGS (#5457) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> It is often helpful to inspect the raw request and response to/from an LLM as agents act. This PR, does the following: - Updates TeamManager to yield LLMCallEvents from core library - Run in an async background task to listen for LLMCallEvent JIT style when a team is run - Add events to an async queue and - yield those events in addition to whatever actual agentchat team.run_stream yields. - Update the AGS UI to show those LLMCallEvents in the messages section as a team runs - Add settings panel to show/hide llm call events in messages. - Minor updates to default team <img width="1539" alt="image" src="https://github.com/user-attachments/assets/bfbb19fe-3560-4faa-b600-7dd244e0e974" /> <img width="1554" alt="image" src="https://github.com/user-attachments/assets/775624f5-ba83-46e8-81ff-512bfeed6bab" /> <img width="1538" alt="image" src="https://github.com/user-attachments/assets/3becbf85-b75a-4506-adb7-e5575ebbaec4" /> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5440 ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Enable LLM Call Observability in AGS (#5457) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> It is often helpful to inspect the raw request and response to/from an LLM as agents act. This PR, does the following: - Updates TeamManager to yield LLMCallEvents from core library - Run in an async background task to listen for LLMCallEvent JIT style when a team is run - Add events to an async queue and - yield those events in addition to whatever actual agentchat team.run_stream yields. - Update the AGS UI to show those LLMCallEvents in the messages section as a team runs - Add settings panel to show/hide llm call events in messages. - Minor updates to default team <img width="1539" alt="image" src="https://github.com/user-attachments/assets/bfbb19fe-3560-4faa-b600-7dd244e0e974" /> <img width="1554" alt="image" src="https://github.com/user-attachments/assets/775624f5-ba83-46e8-81ff-512bfeed6bab" /> <img width="1538" alt="image" src="https://github.com/user-attachments/assets/3becbf85-b75a-4506-adb7-e5575ebbaec4" /> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5440 ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Add Token Streaming in AGS , Support Env variables (#5659) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> This PR has 3 main improvements. - Token streaming - Adds support for environment variables in the app settings - Updates AGS to persist Gallery entry in db. ## Adds Token Streaming in AGS. Agentchat now supports streaming of tokens via `ModelClientStreamingChunkEvent `. This PR is to track progress on supporting that in the AutoGen Studio UI. If `model_client_stream` is enabled in an assitant agent, then token will be streamed in UI. ```python streaming_assistant = AssistantAgent( name="assistant", model_client=model_client, system_message="You are a helpful assistant.", model_client_stream=True, # Enable streaming tokens. ) ``` https://github.com/user-attachments/assets/74d43d78-6359-40c3-a78e-c84dcb5e02a1 ## Env Variables Also adds support for env variables in AGS Settings You can set env variables that are loaded just before a team is run. Handy to set variable to be used by tools etc. <img width="1291" alt="image" src="https://github.com/user-attachments/assets/437b9d90-ccee-42f7-be5d-94ab191afd67" /> > Note: the set variables are available to the server process. <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5627 Closes #5662 Closes #5619 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Add Token Streaming in AGS , Support Env variables (#5659) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> This PR has 3 main improvements. - Token streaming - Adds support for environment variables in the app settings - Updates AGS to persist Gallery entry in db. ## Adds Token Streaming in AGS. Agentchat now supports streaming of tokens via `ModelClientStreamingChunkEvent `. This PR is to track progress on supporting that in the AutoGen Studio UI. If `model_client_stream` is enabled in an assitant agent, then token will be streamed in UI. ```python streaming_assistant = AssistantAgent( name="assistant", model_client=model_client, system_message="You are a helpful assistant.", model_client_stream=True, # Enable streaming tokens. ) ``` https://github.com/user-attachments/assets/74d43d78-6359-40c3-a78e-c84dcb5e02a1 ## Env Variables Also adds support for env variables in AGS Settings You can set env variables that are loaded just before a team is run. Handy to set variable to be used by tools etc. <img width="1291" alt="image" src="https://github.com/user-attachments/assets/437b9d90-ccee-42f7-be5d-94ab191afd67" /> > Note: the set variables are available to the server process. <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5627 Closes #5662 Closes #5619 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165
  1. import asyncio
  2. import json
  3. import logging
  4. import os
  5. import time
  6. from pathlib import Path
  7. from typing import AsyncGenerator, Callable, List, Optional, Union
  8. import aiofiles
  9. import yaml
  10. from autogen_agentchat.agents import UserProxyAgent
  11. from autogen_agentchat.base import TaskResult, Team
  12. from autogen_agentchat.messages import AgentEvent, ChatMessage
  13. from autogen_agentchat.teams import BaseGroupChat
  14. from autogen_core import EVENT_LOGGER_NAME, CancellationToken, Component, ComponentModel
  15. from autogen_core.logging import LLMCallEvent
  16. from ..datamodel.types import EnvironmentVariable, LLMCallEventMessage, TeamResult
  17. logger = logging.getLogger(__name__)
  18. class RunEventLogger(logging.Handler):
  19. """Event logger that queues LLMCallEvents for streaming"""
  20. def __init__(self):
  21. super().__init__()
  22. self.events = asyncio.Queue()
  23. def emit(self, record: logging.LogRecord):
  24. if isinstance(record.msg, LLMCallEvent):
  25. self.events.put_nowait(LLMCallEventMessage(content=str(record.msg)))
  26. class TeamManager:
  27. """Manages team operations including loading configs and running teams"""
  28. @staticmethod
  29. async def load_from_file(path: Union[str, Path]) -> dict:
  30. """Load team configuration from JSON/YAML file"""
  31. path = Path(path)
  32. if not path.exists():
  33. raise FileNotFoundError(f"Config file not found: {path}")
  34. async with aiofiles.open(path) as f:
  35. content = await f.read()
  36. if path.suffix == ".json":
  37. return json.loads(content)
  38. elif path.suffix in (".yml", ".yaml"):
  39. return yaml.safe_load(content)
  40. raise ValueError(f"Unsupported file format: {path.suffix}")
  41. @staticmethod
  42. async def load_from_directory(directory: Union[str, Path]) -> List[dict]:
  43. """Load all team configurations from a directory"""
  44. directory = Path(directory)
  45. configs = []
  46. valid_extensions = {".json", ".yaml", ".yml"}
  47. for path in directory.iterdir():
  48. if path.is_file() and path.suffix.lower() in valid_extensions:
  49. try:
  50. config = await TeamManager.load_from_file(path)
  51. configs.append(config)
  52. except Exception as e:
  53. logger.error(f"Failed to load {path}: {e}")
  54. return configs
  55. async def _create_team(
  56. self,
  57. team_config: Union[str, Path, dict, ComponentModel],
  58. input_func: Optional[Callable] = None,
  59. env_vars: Optional[List[EnvironmentVariable]] = None,
  60. ) -> BaseGroupChat:
  61. """Create team instance from config"""
  62. if isinstance(team_config, (str, Path)):
  63. config = await self.load_from_file(team_config)
  64. elif isinstance(team_config, dict):
  65. config = team_config
  66. else:
  67. config = team_config.model_dump()
  68. # Load env vars into environment if provided
  69. if env_vars:
  70. logger.info("Loading environment variables")
  71. for var in env_vars:
  72. os.environ[var.name] = var.value
  73. team: BaseGroupChat = BaseGroupChat.load_component(config)
  74. for agent in team._participants:
  75. if hasattr(agent, "input_func") and isinstance(agent, UserProxyAgent) and input_func:
  76. agent.input_func = input_func
  77. return team
  78. async def run_stream(
  79. self,
  80. task: str,
  81. team_config: Union[str, Path, dict, ComponentModel],
  82. input_func: Optional[Callable] = None,
  83. cancellation_token: Optional[CancellationToken] = None,
  84. env_vars: Optional[List[EnvironmentVariable]] = None,
  85. ) -> AsyncGenerator[Union[AgentEvent | ChatMessage | LLMCallEvent, ChatMessage, TeamResult], None]:
  86. """Stream team execution results"""
  87. start_time = time.time()
  88. team = None
  89. # Setup logger correctly
  90. logger = logging.getLogger(EVENT_LOGGER_NAME)
  91. logger.setLevel(logging.INFO)
  92. llm_event_logger = RunEventLogger()
  93. logger.handlers = [llm_event_logger] # Replace all handlers
  94. try:
  95. team = await self._create_team(team_config, input_func, env_vars)
  96. async for message in team.run_stream(task=task, cancellation_token=cancellation_token):
  97. if cancellation_token and cancellation_token.is_cancelled():
  98. break
  99. if isinstance(message, TaskResult):
  100. yield TeamResult(task_result=message, usage="", duration=time.time() - start_time)
  101. else:
  102. yield message
  103. # Check for any LLM events
  104. while not llm_event_logger.events.empty():
  105. event = await llm_event_logger.events.get()
  106. yield event
  107. finally:
  108. # Cleanup - remove our handler
  109. if llm_event_logger in logger.handlers:
  110. logger.handlers.remove(llm_event_logger)
  111. # Ensure cleanup happens
  112. if team and hasattr(team, "_participants"):
  113. for agent in team._participants:
  114. if hasattr(agent, "close"):
  115. await agent.close()
  116. async def run(
  117. self,
  118. task: str,
  119. team_config: Union[str, Path, dict, ComponentModel],
  120. input_func: Optional[Callable] = None,
  121. cancellation_token: Optional[CancellationToken] = None,
  122. env_vars: Optional[List[EnvironmentVariable]] = None,
  123. ) -> TeamResult:
  124. """Run team synchronously"""
  125. start_time = time.time()
  126. team = None
  127. try:
  128. team = await self._create_team(team_config, input_func, env_vars)
  129. result = await team.run(task=task, cancellation_token=cancellation_token)
  130. return TeamResult(task_result=result, usage="", duration=time.time() - start_time)
  131. finally:
  132. if team and hasattr(team, "_participants"):
  133. for agent in team._participants:
  134. if hasattr(agent, "close"):
  135. await agent.close()