You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.

db_manager.py 14 kB

Add Token Streaming in AGS , Support Env variables (#5659) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> This PR has 3 main improvements. - Token streaming - Adds support for environment variables in the app settings - Updates AGS to persist Gallery entry in db. ## Adds Token Streaming in AGS. Agentchat now supports streaming of tokens via `ModelClientStreamingChunkEvent `. This PR is to track progress on supporting that in the AutoGen Studio UI. If `model_client_stream` is enabled in an assitant agent, then token will be streamed in UI. ```python streaming_assistant = AssistantAgent( name="assistant", model_client=model_client, system_message="You are a helpful assistant.", model_client_stream=True, # Enable streaming tokens. ) ``` https://github.com/user-attachments/assets/74d43d78-6359-40c3-a78e-c84dcb5e02a1 ## Env Variables Also adds support for env variables in AGS Settings You can set env variables that are loaded just before a team is run. Handy to set variable to be used by tools etc. <img width="1291" alt="image" src="https://github.com/user-attachments/assets/437b9d90-ccee-42f7-be5d-94ab191afd67" /> > Note: the set variables are available to the server process. <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5627 Closes #5662 Closes #5619 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Add Token Streaming in AGS , Support Env variables (#5659) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> This PR has 3 main improvements. - Token streaming - Adds support for environment variables in the app settings - Updates AGS to persist Gallery entry in db. ## Adds Token Streaming in AGS. Agentchat now supports streaming of tokens via `ModelClientStreamingChunkEvent `. This PR is to track progress on supporting that in the AutoGen Studio UI. If `model_client_stream` is enabled in an assitant agent, then token will be streamed in UI. ```python streaming_assistant = AssistantAgent( name="assistant", model_client=model_client, system_message="You are a helpful assistant.", model_client_stream=True, # Enable streaming tokens. ) ``` https://github.com/user-attachments/assets/74d43d78-6359-40c3-a78e-c84dcb5e02a1 ## Env Variables Also adds support for env variables in AGS Settings You can set env variables that are loaded just before a team is run. Handy to set variable to be used by tools etc. <img width="1291" alt="image" src="https://github.com/user-attachments/assets/437b9d90-ccee-42f7-be5d-94ab191afd67" /> > Note: the set variables are available to the server process. <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5627 Closes #5662 Closes #5619 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Add Token Streaming in AGS , Support Env variables (#5659) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> This PR has 3 main improvements. - Token streaming - Adds support for environment variables in the app settings - Updates AGS to persist Gallery entry in db. ## Adds Token Streaming in AGS. Agentchat now supports streaming of tokens via `ModelClientStreamingChunkEvent `. This PR is to track progress on supporting that in the AutoGen Studio UI. If `model_client_stream` is enabled in an assitant agent, then token will be streamed in UI. ```python streaming_assistant = AssistantAgent( name="assistant", model_client=model_client, system_message="You are a helpful assistant.", model_client_stream=True, # Enable streaming tokens. ) ``` https://github.com/user-attachments/assets/74d43d78-6359-40c3-a78e-c84dcb5e02a1 ## Env Variables Also adds support for env variables in AGS Settings You can set env variables that are loaded just before a team is run. Handy to set variable to be used by tools etc. <img width="1291" alt="image" src="https://github.com/user-attachments/assets/437b9d90-ccee-42f7-be5d-94ab191afd67" /> > Note: the set variables are available to the server process. <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5627 Closes #5662 Closes #5619 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Enable Auth in AGS (#5928) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? https://github.com/user-attachments/assets/b649053b-c377-40c7-aa51-ee64af766fc2 <img width="100%" alt="image" src="https://github.com/user-attachments/assets/03ba1df5-c9a2-4734-b6a2-0eb97ec0b0e0" /> ## Authentication This PR implements an experimental authentication feature to enable personalized experiences (multiple users). Currently, only GitHub authentication is supported. You can extend the base authentication class to add support for other authentication methods. By default authenticatio is disabled and only enabled when you pass in the `--auth-config` argument when running the application. ### Enable GitHub Authentication To enable GitHub authentication, create a `auth.yaml` file in your app directory: ```yaml type: github jwt_secret: "your-secret-key" token_expiry_minutes: 60 github: client_id: "your-github-client-id" client_secret: "your-github-client-secret" callback_url: "http://localhost:8081/api/auth/callback" scopes: ["user:email"] ``` Please see the documentation on [GitHub OAuth](https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/authenticating-to-the-rest-api-with-an-oauth-app) for more details on obtaining the `client_id` and `client_secret`. To pass in this configuration you can use the `--auth-config` argument when running the application: ```bash autogenstudio ui --auth-config /path/to/auth.yaml ``` Or set the environment variable: ```bash export AUTOGENSTUDIO_AUTH_CONFIG="/path/to/auth.yaml" ``` ```{note} - Authentication is currently experimental and may change in future releases - User data is stored in your configured database - When enabled, all API endpoints require authentication except for the authentication endpoints - WebSocket connections require the token to be passed as a query parameter (`?token=your-jwt-token`) ``` ## Related issue number <!-- For example: "Closes #1234" --> Closes #4350 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed. --------- Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
1 year ago
Enable Auth in AGS (#5928) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? https://github.com/user-attachments/assets/b649053b-c377-40c7-aa51-ee64af766fc2 <img width="100%" alt="image" src="https://github.com/user-attachments/assets/03ba1df5-c9a2-4734-b6a2-0eb97ec0b0e0" /> ## Authentication This PR implements an experimental authentication feature to enable personalized experiences (multiple users). Currently, only GitHub authentication is supported. You can extend the base authentication class to add support for other authentication methods. By default authenticatio is disabled and only enabled when you pass in the `--auth-config` argument when running the application. ### Enable GitHub Authentication To enable GitHub authentication, create a `auth.yaml` file in your app directory: ```yaml type: github jwt_secret: "your-secret-key" token_expiry_minutes: 60 github: client_id: "your-github-client-id" client_secret: "your-github-client-secret" callback_url: "http://localhost:8081/api/auth/callback" scopes: ["user:email"] ``` Please see the documentation on [GitHub OAuth](https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/authenticating-to-the-rest-api-with-an-oauth-app) for more details on obtaining the `client_id` and `client_secret`. To pass in this configuration you can use the `--auth-config` argument when running the application: ```bash autogenstudio ui --auth-config /path/to/auth.yaml ``` Or set the environment variable: ```bash export AUTOGENSTUDIO_AUTH_CONFIG="/path/to/auth.yaml" ``` ```{note} - Authentication is currently experimental and may change in future releases - User data is stored in your configured database - When enabled, all API endpoints require authentication except for the authentication endpoints - WebSocket connections require the token to be passed as a query parameter (`?token=your-jwt-token`) ``` ## Related issue number <!-- For example: "Closes #1234" --> Closes #4350 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed. --------- Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
1 year ago
Add Token Streaming in AGS , Support Env variables (#5659) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> This PR has 3 main improvements. - Token streaming - Adds support for environment variables in the app settings - Updates AGS to persist Gallery entry in db. ## Adds Token Streaming in AGS. Agentchat now supports streaming of tokens via `ModelClientStreamingChunkEvent `. This PR is to track progress on supporting that in the AutoGen Studio UI. If `model_client_stream` is enabled in an assitant agent, then token will be streamed in UI. ```python streaming_assistant = AssistantAgent( name="assistant", model_client=model_client, system_message="You are a helpful assistant.", model_client_stream=True, # Enable streaming tokens. ) ``` https://github.com/user-attachments/assets/74d43d78-6359-40c3-a78e-c84dcb5e02a1 ## Env Variables Also adds support for env variables in AGS Settings You can set env variables that are loaded just before a team is run. Handy to set variable to be used by tools etc. <img width="1291" alt="image" src="https://github.com/user-attachments/assets/437b9d90-ccee-42f7-be5d-94ab191afd67" /> > Note: the set variables are available to the server process. <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5627 Closes #5662 Closes #5619 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
Add Token Streaming in AGS , Support Env variables (#5659) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> This PR has 3 main improvements. - Token streaming - Adds support for environment variables in the app settings - Updates AGS to persist Gallery entry in db. ## Adds Token Streaming in AGS. Agentchat now supports streaming of tokens via `ModelClientStreamingChunkEvent `. This PR is to track progress on supporting that in the AutoGen Studio UI. If `model_client_stream` is enabled in an assitant agent, then token will be streamed in UI. ```python streaming_assistant = AssistantAgent( name="assistant", model_client=model_client, system_message="You are a helpful assistant.", model_client_stream=True, # Enable streaming tokens. ) ``` https://github.com/user-attachments/assets/74d43d78-6359-40c3-a78e-c84dcb5e02a1 ## Env Variables Also adds support for env variables in AGS Settings You can set env variables that are loaded just before a team is run. Handy to set variable to be used by tools etc. <img width="1291" alt="image" src="https://github.com/user-attachments/assets/437b9d90-ccee-42f7-be5d-94ab191afd67" /> > Note: the set variables are available to the server process. <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number <!-- For example: "Closes #1234" --> Closes #5627 Closes #5662 Closes #5619 ## Checks - [ ] I've included any doc changes needed for <https://microsoft.github.io/autogen/>. See <https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
1 year ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352
  1. import threading
  2. from datetime import datetime
  3. from pathlib import Path
  4. from typing import Optional, Union
  5. from loguru import logger
  6. from sqlalchemy import exc, inspect, text
  7. from sqlmodel import Session, SQLModel, and_, create_engine, select
  8. from ..datamodel import Response, Team
  9. from ..teammanager import TeamManager
  10. from .schema_manager import SchemaManager
  11. class DatabaseManager:
  12. _init_lock = threading.Lock()
  13. def __init__(self, engine_uri: str, base_dir: Optional[Union[str, Path]] = None) -> None:
  14. """
  15. Initialize DatabaseManager with database connection settings.
  16. Does not perform any database operations.
  17. Args:
  18. engine_uri: Database connection URI (e.g. sqlite:///db.sqlite3)
  19. base_dir: Base directory for migration files. If None, uses current directory
  20. """
  21. connection_args = {"check_same_thread": True} if "sqlite" in engine_uri else {}
  22. if base_dir is not None and isinstance(base_dir, str):
  23. base_dir = Path(base_dir)
  24. self.engine = create_engine(engine_uri, connect_args=connection_args)
  25. self.schema_manager = SchemaManager(
  26. engine=self.engine,
  27. base_dir=base_dir,
  28. )
  29. def _should_auto_upgrade(self) -> bool:
  30. """
  31. Check if auto upgrade should run based on schema differences
  32. """
  33. needs_upgrade, _ = self.schema_manager.check_schema_status()
  34. return needs_upgrade
  35. def initialize_database(self, auto_upgrade: bool = False, force_init_alembic: bool = True) -> Response:
  36. """
  37. Initialize database and migrations in the correct order.
  38. Args:
  39. auto_upgrade: If True, automatically generate and apply migrations for schema changes
  40. force_init_alembic: If True, reinitialize alembic configuration even if it exists
  41. """
  42. if not self._init_lock.acquire(blocking=False):
  43. return Response(message="Database initialization already in progress", status=False)
  44. try:
  45. # Enable foreign key constraints for SQLite
  46. if "sqlite" in str(self.engine.url):
  47. with self.engine.connect() as conn:
  48. conn.execute(text("PRAGMA foreign_keys=ON"))
  49. inspector = inspect(self.engine)
  50. tables_exist = inspector.get_table_names()
  51. if not tables_exist:
  52. logger.info("Creating database tables...")
  53. SQLModel.metadata.create_all(self.engine)
  54. if self.schema_manager.initialize_migrations(force=force_init_alembic):
  55. return Response(message="Database initialized successfully", status=True)
  56. return Response(message="Failed to initialize migrations", status=False)
  57. # Handle existing database
  58. if auto_upgrade or self._should_auto_upgrade():
  59. logger.info("Checking database schema...")
  60. if self.schema_manager.ensure_schema_up_to_date():
  61. return Response(message="Database schema is up to date", status=True)
  62. return Response(message="Database upgrade failed", status=False)
  63. return Response(message="Database is ready", status=True)
  64. except Exception as e:
  65. error_msg = f"Database initialization failed: {str(e)}"
  66. logger.error(error_msg)
  67. return Response(message=error_msg, status=False)
  68. finally:
  69. self._init_lock.release()
  70. def reset_db(self, recreate_tables: bool = True):
  71. """
  72. Reset the database by dropping all tables and optionally recreating them.
  73. Args:
  74. recreate_tables (bool): If True, recreates the tables after dropping them.
  75. Set to False if you want to call create_db_and_tables() separately.
  76. """
  77. if not self._init_lock.acquire(blocking=False):
  78. logger.warning("Database reset already in progress")
  79. return Response(message="Database reset already in progress", status=False, data=None)
  80. try:
  81. # Dispose existing connections
  82. self.engine.dispose()
  83. with Session(self.engine) as session:
  84. try:
  85. # Disable foreign key checks for SQLite
  86. if "sqlite" in str(self.engine.url):
  87. session.exec(text("PRAGMA foreign_keys=OFF")) # type: ignore
  88. # Drop all tables
  89. SQLModel.metadata.drop_all(self.engine)
  90. logger.info("All tables dropped successfully")
  91. # Re-enable foreign key checks for SQLite
  92. if "sqlite" in str(self.engine.url):
  93. session.exec(text("PRAGMA foreign_keys=ON")) # type: ignore
  94. session.commit()
  95. except Exception as e:
  96. session.rollback()
  97. raise e
  98. finally:
  99. session.close()
  100. self._init_lock.release()
  101. if recreate_tables:
  102. logger.info("Recreating tables...")
  103. self.initialize_database(auto_upgrade=False, force_init_alembic=True)
  104. return Response(
  105. message="Database reset successfully" if recreate_tables else "Database tables dropped successfully",
  106. status=True,
  107. data=None,
  108. )
  109. except Exception as e:
  110. error_msg = f"Error while resetting database: {str(e)}"
  111. logger.error(error_msg)
  112. return Response(message=error_msg, status=False, data=None)
  113. finally:
  114. if self._init_lock.locked():
  115. self._init_lock.release()
  116. logger.info("Database reset lock released")
  117. def upsert(self, model: SQLModel, return_json: bool = True) -> Response:
  118. """Create or update an entity
  119. Args:
  120. model (SQLModel): The model instance to create or update
  121. return_json (bool, optional): If True, returns the model as a dictionary.
  122. If False, returns the SQLModel instance. Defaults to True.
  123. Returns:
  124. Response: Contains status, message and data (either dict or SQLModel based on return_json)
  125. """
  126. status = True
  127. model_class = type(model)
  128. existing_model = None
  129. with Session(self.engine) as session:
  130. try:
  131. existing_model = session.exec(select(model_class).where(model_class.id == model.id)).first()
  132. if existing_model:
  133. model.updated_at = datetime.now()
  134. for key, value in model.model_dump().items():
  135. setattr(existing_model, key, value)
  136. model = existing_model # Use the updated existing model
  137. session.add(model)
  138. else:
  139. session.add(model)
  140. session.commit()
  141. session.refresh(model)
  142. except Exception as e:
  143. session.rollback()
  144. logger.error("Error while updating/creating " + str(model_class.__name__) + ": " + str(e))
  145. status = False
  146. return Response(
  147. message=(
  148. f"{model_class.__name__} Updated Successfully"
  149. if existing_model
  150. else f"{model_class.__name__} Created Successfully"
  151. ),
  152. status=status,
  153. data=model.model_dump() if return_json else model,
  154. )
  155. def _model_to_dict(self, model_obj):
  156. return {col.name: getattr(model_obj, col.name) for col in model_obj.__table__.columns}
  157. def get(
  158. self,
  159. model_class: SQLModel,
  160. filters: dict | None = None,
  161. return_json: bool = False,
  162. order: str = "desc",
  163. ):
  164. """List entities"""
  165. with Session(self.engine) as session:
  166. result = []
  167. status = True
  168. status_message = ""
  169. try:
  170. statement = select(model_class)
  171. if filters:
  172. conditions = [getattr(model_class, col) == value for col, value in filters.items()]
  173. statement = statement.where(and_(*conditions))
  174. if hasattr(model_class, "created_at") and order:
  175. order_by_clause = getattr(model_class.created_at, order)() # Dynamically apply asc/desc
  176. statement = statement.order_by(order_by_clause)
  177. items = session.exec(statement).all()
  178. result = [self._model_to_dict(item) if return_json else item for item in items]
  179. status_message = f"{model_class.__name__} Retrieved Successfully"
  180. except Exception as e:
  181. session.rollback()
  182. status = False
  183. status_message = f"Error while fetching {model_class.__name__}"
  184. logger.error("Error while getting items: " + str(model_class.__name__) + " " + str(e))
  185. return Response(message=status_message, status=status, data=result)
  186. def delete(self, model_class: SQLModel, filters: dict = None) -> Response:
  187. """Delete an entity"""
  188. status_message = ""
  189. status = True
  190. with Session(self.engine) as session:
  191. try:
  192. if "sqlite" in str(self.engine.url):
  193. session.exec(text("PRAGMA foreign_keys=ON"))
  194. statement = select(model_class)
  195. if filters:
  196. conditions = [getattr(model_class, col) == value for col, value in filters.items()]
  197. statement = statement.where(and_(*conditions))
  198. rows = session.exec(statement).all()
  199. if rows:
  200. for row in rows:
  201. session.delete(row)
  202. session.commit()
  203. status_message = f"{model_class.__name__} Deleted Successfully"
  204. else:
  205. status_message = "Row not found"
  206. logger.info(f"Row with filters {filters} not found")
  207. except exc.IntegrityError as e:
  208. session.rollback()
  209. status = False
  210. status_message = f"Integrity error: The {model_class.__name__} is linked to another entity and cannot be deleted. {e}"
  211. # Log the specific integrity error
  212. logger.error(status_message)
  213. except Exception as e:
  214. session.rollback()
  215. status = False
  216. status_message = f"Error while deleting: {e}"
  217. logger.error(status_message)
  218. return Response(message=status_message, status=status, data=None)
  219. async def import_team(
  220. self, team_config: Union[str, Path, dict], user_id: str, check_exists: bool = False
  221. ) -> Response:
  222. try:
  223. # Load config if path provided
  224. if isinstance(team_config, (str, Path)):
  225. config = await TeamManager.load_from_file(team_config)
  226. else:
  227. config = team_config
  228. # Check existence if requested
  229. if check_exists:
  230. existing = await self._check_team_exists(config, user_id)
  231. if existing:
  232. return Response(
  233. message="Identical team configuration already exists", status=True, data={"id": existing.id}
  234. )
  235. # Store in database
  236. team_db = Team(user_id=user_id, component=config)
  237. result = self.upsert(team_db)
  238. return result
  239. except Exception as e:
  240. logger.error(f"Failed to import team: {str(e)}")
  241. return Response(message=str(e), status=False)
  242. async def import_teams_from_directory(
  243. self, directory: Union[str, Path], user_id: str, check_exists: bool = False
  244. ) -> Response:
  245. """
  246. Import all team configurations from a directory.
  247. Args:
  248. directory: Path to directory containing team configs
  249. user_id: User ID to associate with imported teams
  250. check_exists: Whether to check for existing teams
  251. Returns:
  252. Response containing import results for all files
  253. """
  254. try:
  255. # Load all configs from directory
  256. configs = await TeamManager.load_from_directory(directory)
  257. results = []
  258. for config in configs:
  259. try:
  260. result = await self.import_team(team_config=config, user_id=user_id, check_exists=check_exists)
  261. # Add result info
  262. results.append(
  263. {
  264. "status": result.status,
  265. "message": result.message,
  266. "id": result.data.get("id") if result.status else None,
  267. }
  268. )
  269. except Exception as e:
  270. logger.error(f"Failed to import team config: {str(e)}")
  271. results.append({"status": False, "message": str(e), "id": None})
  272. return Response(message="Directory import complete", status=True, data=results)
  273. except Exception as e:
  274. logger.error(f"Failed to import directory: {str(e)}")
  275. return Response(message=str(e), status=False)
  276. async def _check_team_exists(self, config: dict, user_id: str) -> Optional[Team]:
  277. """Check if identical team config already exists"""
  278. teams = self.get(Team, {"user_id": user_id}).data
  279. for team in teams:
  280. if team.component == config:
  281. return team
  282. return None
  283. async def close(self):
  284. """Close database connections and cleanup resources"""
  285. logger.info("Closing database connections...")
  286. try:
  287. # Dispose of the SQLAlchemy engine
  288. self.engine.dispose()
  289. logger.info("Database connections closed successfully")
  290. except Exception as e:
  291. logger.error(f"Error closing database connections: {str(e)}")
  292. raise