You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.

README.md 4.2 kB

5 years ago
5 years ago
5 years ago
5 years ago
5 years ago
5 years ago
5 years ago
5 years ago
5 years ago
5 years ago
5 years ago
5 years ago
5 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596
  1. Recommendation Model
  2. ## Overview
  3. This is an implementation of WideDeep as described in the [Wide & Deep Learning for Recommender System](https://arxiv.org/pdf/1606.07792.pdf) paper.
  4. WideDeep model jointly trained wide linear models and deep neural network, which combined the benefits of memorization and generalization for recommender systems.
  5. ## Dataset
  6. The Criteo datasets are used for model training and evaluation.
  7. ## Running Code
  8. ### Code Structure
  9. The entire code structure is as following:
  10. ```
  11. |--- wide_and_deep/
  12. train_and_test.py "Entrance of Wide&Deep model training and evaluation"
  13. test.py "Entrance of Wide&Deep model evaluation"
  14. train.py "Entrance of Wide&Deep model training"
  15. train_and_test_multinpu.py "Entrance of Wide&Deep model data parallel training and evaluation"
  16. |--- src/ "entrance of training and evaluation"
  17. config.py "parameters configuration"
  18. dataset.py "Dataset loader class"
  19. process_data.py "process dataset"
  20. preprocess_data.py "pre_process dataset"
  21. WideDeep.py "Model structure"
  22. callbacks.py "Callback class for training and evaluation"
  23. metrics.py "Metric class"
  24. |--- script/ "run shell dir"
  25. run_multinpu_train.sh "run data parallel"
  26. ```
  27. ### Train and evaluate model
  28. To train and evaluate the model, command as follows:
  29. ```
  30. python train_and_test.py
  31. ```
  32. Arguments:
  33. * `--data_path`: This should be set to the same directory given to the data_download's data_dir argument.
  34. * `--epochs`: Total train epochs.
  35. * `--batch_size`: Training batch size.
  36. * `--eval_batch_size`: Eval batch size.
  37. * `--field_size`: The number of features.
  38. * `--vocab_size`: The total features of dataset.
  39. * `--emb_dim`: The dense embedding dimension of sparse feature.
  40. * `--deep_layers_dim`: The dimension of all deep layers.
  41. * `--deep_layers_act`: The activation of all deep layers.
  42. * `--keep_prob`: The rate to keep in dropout layer.
  43. * `--ckpt_path`:The location of the checkpoint file.
  44. * `--eval_file_name` : Eval output file.
  45. * `--loss_file_name` : Loss output file.
  46. To train the model in one device, command as follows:
  47. ```
  48. python train.py
  49. ```
  50. Arguments:
  51. * `--data_path`: This should be set to the same directory given to the data_download's data_dir argument.
  52. * `--epochs`: Total train epochs.
  53. * `--batch_size`: Training batch size.
  54. * `--eval_batch_size`: Eval batch size.
  55. * `--field_size`: The number of features.
  56. * `--vocab_size`: The total features of dataset.
  57. * `--emb_dim`: The dense embedding dimension of sparse feature.
  58. * `--deep_layers_dim`: The dimension of all deep layers.
  59. * `--deep_layers_act`: The activation of all deep layers.
  60. * `--keep_prob`: The rate to keep in dropout layer.
  61. * `--ckpt_path`:The location of the checkpoint file.
  62. * `--eval_file_name` : Eval output file.
  63. * `--loss_file_name` : Loss output file.
  64. To train the model in distributed, command as follows:
  65. ```
  66. # configure environment path, RANK_TABLE_FILE, RANK_SIZE, MINDSPORE_HCCL_CONFIG_PATH before training
  67. bash run_multinpu_train.sh
  68. ```
  69. To evaluate the model, command as follows:
  70. ```
  71. python test.py
  72. ```
  73. Arguments:
  74. * `--data_path`: This should be set to the same directory given to the data_download's data_dir argument.
  75. * `--epochs`: Total train epochs.
  76. * `--batch_size`: Training batch size.
  77. * `--eval_batch_size`: Eval batch size.
  78. * `--field_size`: The number of features.
  79. * `--vocab_size`: The total features of dataset.
  80. * `--emb_dim`: The dense embedding dimension of sparse feature.
  81. * `--deep_layers_dim`: The dimension of all deep layers.
  82. * `--deep_layers_act`: The activation of all deep layers.
  83. * `--keep_prob`: The rate to keep in dropout layer.
  84. * `--ckpt_path`:The location of the checkpoint file.
  85. * `--eval_file_name` : Eval output file.
  86. * `--loss_file_name` : Loss output file.
  87. There are other arguments about models and training process. Use the `--help` or `-h` flag to get a full list of possible arguments with detailed descriptions.