* Prints the version of AutoGenBench from the command line, closing i1458
* Added autogenbench version to timestamp.txt
* Attempting to fix formatting.
* Add a gitignore for autogenbench
* Generalize to read all template dirs from Templates
* AutoGenBench logs telemetry when available.
* Remove spaces if present from template names.
* Bump version.
* Fixed formatting.
* Allow native warning to be skipped. Mount autogen repo in Docker if it can be found (experimental).
* Native execution now occurs in a venv.
* Bump version.
* Fixed a prompt escaping bug evident in GAIA task '6f37996b-2ac7-44b0-8e68-6d28256631b4'
* Updated all scenarios to use template discovery.
* Update with main version of runtime_logging.
* Better handling of Ctrl-C and cleanup of unused containers.
* Even stronger hinting that containers should be removed.
---------
Co-authored-by: gagb <gagb@users.noreply.github.com>
* Prints the version of AutoGenBench from the command line, closing i1458
* Added autogenbench version to timestamp.txt
* Attempting to fix formatting.
* Add a gitignore for autogenbench
* Generalize to read all template dirs from Templates
* AutoGenBench logs telemetry when available.
* Remove spaces if present from template names.
* Bump version.
* Fixed formatting.
* Allow native warning to be skipped. Mount autogen repo in Docker if it can be found (experimental).
* Native execution now occurs in a venv.
* Bump version.
* Fixed a prompt escaping bug evident in GAIA task '6f37996b-2ac7-44b0-8e68-6d28256631b4'
* Updated all scenarios to use template discovery.
* Update with main version of runtime_logging.
---------
Co-authored-by: gagb <gagb@users.noreply.github.com>
* Initial commit of AutoGenBench
* wording
* typo
* pre-commit reformulation
* Updated README to point to contributor's guide earlier.
* Simplified the description of the JSON format.
* Added print statements to indicate when run.sh and scenario.py are starting.
* Added SocietyOfMind scenario to GAIA.
* Pointing autogenbench clone command to the latest branch.
* Temporarily disable subsample option.
* Updated the GAIA readme to specify how to define a BING API key.
* Fixed and re-enabled the subsample option.
* Added a draft of a blog post.
* Updated authors.
* Incorporating Gagan's feedback.
* Fixed code formatting.
* Updated the help string in the docs.
* Light editing of the AutoGenBench blogpost.
* Support filtering on model tags.
* Added websurfer dependencies to Dockerfile.
* Renamed testbed -> autogenbench
* Attempting to fix formatting.
* Added more gracefull handling of task timeouts (the script is allowed to terminate before Docker is stopped).
* Updated the blogpost based on Saleema's and Julia's feedback.
* Fixed formatting... again.
* Added a main MANIFEST to list available scenarios.
* Limit main manifest to directories.
* Manifests now use relative paths.
* All manifests are now relative.
* Updated the contributing guide, and address windows path issues.
* Updated the version. Fixed formatting.
* Fixed formatting.
* De-listing Examples, since it has no clear tabulate criteria.
* Updated email in pyproject
* typo in blogpost
* wording
---------
Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
Co-authored-by: Qingyun Wu <qingyun0327@gmail.com>
afourney
Eduardo Salinas
Gagan Bansal
gagb
Davor Runje
Yiran Wu