wangtao
/
autogen
Not watched
1
0
Fork 0
Code
Releases 127 Wiki evaluate Activity
Issues 0 Pull Requests 0 Datasets Model Cloudbrain HPC
You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
701 Commits
199 Branches
282 MB
21 Download
Tree: 0cb79dfdff
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from '0cb79dfdff'
${ noResults }
autogen/website/docs/Examples/AutoML-Rank.md

AutoML-Rank.md 8.3 kB
Raw Normal View History

Deploy a new doc website (#338) A new documentation website. And: * add actions for doc * update docstr * installation instructions for doc dev * unify README and Getting Started * rename notebook * doc about best_model_for_estimator #340 * docstr for keep_search_state #340 * DNN Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu> Co-authored-by: Z.sk <shaokunzhang@psu.edu>
4 years ago
​
create an automl option to remove unnecessary dependency for autogen and tune (#1007) * version update post release v1.2.2 * automl option * import pandas * remove automl.utils * default * test * type hint and version update * dependency update * link to open in colab * use packging.version to close #725 --------- Co-authored-by: Li Jiang <lijiang1@microsoft.com> Co-authored-by: Li Jiang <bnujli@gmail.com>
3 years ago
​
​
​
​
​
​
Deploy a new doc website (#338) A new documentation website. And: * add actions for doc * update docstr * installation instructions for doc dev * unify README and Getting Started * rename notebook * doc about best_model_for_estimator #340 * docstr for keep_search_state #340 * DNN Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu> Co-authored-by: Z.sk <shaokunzhang@psu.edu>
4 years ago
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
​
precommit: end-of-file-fixer (#929) * precommit: end-of-file-fixer * exclude .gitignore * apply --------- Co-authored-by: Shaokun <shaokunzhang529@gmail.com>
3 years ago
 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103 
  1. # AutoML - Rank

  2. 

  3. ### Prerequisites

  4. 

  5. Install the [automl] option.

  6. ```bash

  7. pip install "flaml[automl]"

  8. ```

  9. 

  10. ### A simple learning-to-rank example

  11. 

  12. ```python

  13. from sklearn.datasets import fetch_openml

  14. from flaml import AutoML

  15. 

  16. X_train, y_train = fetch_openml(name="credit-g", return_X_y=True, as_frame=False)

  17. y_train = y_train.cat.codes

  18. # not a real learning to rank dataaset

  19. groups = [200] * 4 + [100] * 2    # group counts

  20. automl = AutoML()

  21. automl.fit(

  22.     X_train, y_train, groups=groups,

  23.     task='rank', time_budget=10,    # in seconds

  24. )

  25. ```

  26. 

  27. #### Sample output

  28. 

  29. ```

  30. [flaml.automl: 11-15 07:14:30] {1485} INFO - Data split method: group

  31. [flaml.automl: 11-15 07:14:30] {1489} INFO - Evaluation method: holdout

  32. [flaml.automl: 11-15 07:14:30] {1540} INFO - Minimizing error metric: 1-ndcg

  33. [flaml.automl: 11-15 07:14:30] {1577} INFO - List of ML learners in AutoML Run: ['lgbm', 'xgboost']

  34. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 0, current learner lgbm

  35. [flaml.automl: 11-15 07:14:30] {1944} INFO - Estimated sufficient time budget=679s. Estimated necessary time budget=1s.

  36. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.1s,  estimator lgbm's best error=0.0248,     best estimator lgbm's best error=0.0248

  37. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 1, current learner lgbm

  38. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.1s,  estimator lgbm's best error=0.0248,     best estimator lgbm's best error=0.0248

  39. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 2, current learner lgbm

  40. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.2s,  estimator lgbm's best error=0.0248,     best estimator lgbm's best error=0.0248

  41. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 3, current learner lgbm

  42. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.2s,  estimator lgbm's best error=0.0248,     best estimator lgbm's best error=0.0248

  43. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 4, current learner xgboost

  44. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.2s,  estimator xgboost's best error=0.0315,  best estimator lgbm's best error=0.0248

  45. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 5, current learner xgboost

  46. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.2s,  estimator xgboost's best error=0.0315,  best estimator lgbm's best error=0.0248

  47. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 6, current learner lgbm

  48. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.3s,  estimator lgbm's best error=0.0248,     best estimator lgbm's best error=0.0248

  49. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 7, current learner lgbm

  50. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.3s,  estimator lgbm's best error=0.0248,     best estimator lgbm's best error=0.0248

  51. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 8, current learner xgboost

  52. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.4s,  estimator xgboost's best error=0.0315,  best estimator lgbm's best error=0.0248

  53. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 9, current learner xgboost

  54. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.4s,  estimator xgboost's best error=0.0315,  best estimator lgbm's best error=0.0248

  55. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 10, current learner xgboost

  56. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.4s,  estimator xgboost's best error=0.0233,  best estimator xgboost's best error=0.0233

  57. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 11, current learner xgboost

  58. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.4s,  estimator xgboost's best error=0.0233,  best estimator xgboost's best error=0.0233

  59. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 12, current learner xgboost

  60. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.4s,  estimator xgboost's best error=0.0233,  best estimator xgboost's best error=0.0233

  61. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 13, current learner xgboost

  62. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.4s,  estimator xgboost's best error=0.0233,  best estimator xgboost's best error=0.0233

  63. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 14, current learner lgbm

  64. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.5s,  estimator lgbm's best error=0.0225,     best estimator lgbm's best error=0.0225

  65. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 15, current learner xgboost

  66. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.5s,  estimator xgboost's best error=0.0233,  best estimator lgbm's best error=0.0225

  67. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 16, current learner lgbm

  68. [flaml.automl: 11-15 07:14:30] {2029} INFO -  at 0.5s,  estimator lgbm's best error=0.0225,     best estimator lgbm's best error=0.0225

  69. [flaml.automl: 11-15 07:14:30] {1826} INFO - iteration 17, current learner lgbm

  70. [flaml.automl: 11-15 07:14:31] {2029} INFO -  at 0.5s,  estimator lgbm's best error=0.0225,     best estimator lgbm's best error=0.0225

  71. [flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 18, current learner lgbm

  72. [flaml.automl: 11-15 07:14:31] {2029} INFO -  at 0.6s,  estimator lgbm's best error=0.0225,     best estimator lgbm's best error=0.0225

  73. [flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 19, current learner lgbm

  74. [flaml.automl: 11-15 07:14:31] {2029} INFO -  at 0.6s,  estimator lgbm's best error=0.0201,     best estimator lgbm's best error=0.0201

  75. [flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 20, current learner lgbm

  76. [flaml.automl: 11-15 07:14:31] {2029} INFO -  at 0.6s,  estimator lgbm's best error=0.0201,     best estimator lgbm's best error=0.0201

  77. [flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 21, current learner lgbm

  78. [flaml.automl: 11-15 07:14:31] {2029} INFO -  at 0.7s,  estimator lgbm's best error=0.0201,     best estimator lgbm's best error=0.0201

  79. [flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 22, current learner lgbm

  80. [flaml.automl: 11-15 07:14:31] {2029} INFO -  at 0.7s,  estimator lgbm's best error=0.0201,     best estimator lgbm's best error=0.0201

  81. [flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 23, current learner lgbm

  82. [flaml.automl: 11-15 07:14:31] {2029} INFO -  at 0.8s,  estimator lgbm's best error=0.0201,     best estimator lgbm's best error=0.0201

  83. [flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 24, current learner lgbm

  84. [flaml.automl: 11-15 07:14:31] {2029} INFO -  at 0.8s,  estimator lgbm's best error=0.0201,     best estimator lgbm's best error=0.0201

  85. [flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 25, current learner lgbm

  86. [flaml.automl: 11-15 07:14:31] {2029} INFO -  at 0.8s,  estimator lgbm's best error=0.0201,     best estimator lgbm's best error=0.0201

  87. [flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 26, current learner lgbm

  88. [flaml.automl: 11-15 07:14:31] {2029} INFO -  at 0.9s,  estimator lgbm's best error=0.0197,     best estimator lgbm's best error=0.0197

  89. [flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 27, current learner lgbm

  90. [flaml.automl: 11-15 07:14:31] {2029} INFO -  at 0.9s,  estimator lgbm's best error=0.0197,     best estimator lgbm's best error=0.0197

  91. [flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 28, current learner lgbm

  92. [flaml.automl: 11-15 07:14:31] {2029} INFO -  at 1.0s,  estimator lgbm's best error=0.0197,     best estimator lgbm's best error=0.0197

  93. [flaml.automl: 11-15 07:14:31] {1826} INFO - iteration 29, current learner lgbm

  94. [flaml.automl: 11-15 07:14:31] {2029} INFO -  at 1.0s,  estimator lgbm's best error=0.0197,     best estimator lgbm's best error=0.0197

  95. [flaml.automl: 11-15 07:14:31] {2242} INFO - retrain lgbm for 0.0s

  96. [flaml.automl: 11-15 07:14:31] {2247} INFO - retrained model: LGBMRanker(colsample_bytree=0.9852774042640857,

  97.            learning_rate=0.034918421933217675, max_bin=1023,

  98.            min_child_samples=22, n_estimators=6, num_leaves=23,

  99.            reg_alpha=0.0009765625, reg_lambda=21.505295697527654, verbose=-1)

  100. [flaml.automl: 11-15 07:14:31] {1608} INFO - fit succeeded

  101. [flaml.automl: 11-15 07:14:31] {1610} INFO - Time taken to find the best model: 0.8846545219421387

  102. [flaml.automl: 11-15 07:14:31] {1624} WARNING - Time taken to find the best model is 88% of the provided time budget and not all estimators' hyperparameter search converged. Consider increasing the time budget.

  103. ```

This is a mirror of AutoGen from GitHub. AutoGen is a framework that enables the development of LLM applications using multiple agents that can converse with each other to solve tasks.