You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.

mindspore.DatasetHelper.rst 3.3 kB

4 years ago
4 years ago
4 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596
  1. mindspore.DatasetHelper
  2. ========================
  3. .. py:class:: mindspore.DatasetHelper(dataset, dataset_sink_mode=True, sink_size=-1, epoch_num=1)
  4. DatasetHelperһMindDataݼ࣬ṩݼϢ
  5. ݲͬģıݼĵڲͬʹͬĵ
  6. .. note::
  7. DatasetHelperĵṩһepochݡ
  8. ****
  9. - **dataset** (Dataset) - ѵݼݼݼAPI :class:`mindspore.dataset` ɣ :class:`mindspore.dataset.ImageFolderDataset`
  10. - **dataset_sink_mode** (bool) - ֵΪTrueʹ :class:`mindspore.ops.GetNext` 豸DeviceͨͨлȡݣHostֱӱݼȡݡĬֵTrue
  11. - **sink_size** (int) - ÿ³е `sink_size` Ϊ-1³ÿepochݼ `sink_size` 0³ÿepoch `sink_size` ݡĬֵ-1
  12. - **epoch_num** (int) - ƴ͵epochĬֵ1
  13. ****
  14. >>> from mindspore import DatasetHelper
  15. >>>
  16. >>> train_dataset = create_custom_dataset()
  17. >>> set_helper = DatasetHelper(train_dataset, dataset_sink_mode=False)
  18. >>> # DatasetHelperǿɵ
  19. >>> for next_element in set_helper:
  20. ... next_element
  21. .. py:method:: continue_send()
  22. epochʼʱ豸ݡ
  23. .. py:method:: dynamic_min_max_shapes()
  24. ض̬ݵ״(shape)ΧС״(shape)״(shape)
  25. ****
  26. >>>from mindspore import DatasetHelper
  27. >>>
  28. >>>train_dataset = create_custom_dataset()
  29. >>># config dynamic shape
  30. >>>dataset.set_dynamic_columns(columns={"data1": [16, None, 83], "data2": [None]})
  31. >>>dataset_helper = DatasetHelper(train_dataset, dataset_sink_mode=True)
  32. >>>
  33. >>>min_shapes, max_shapes = dataset_helper.dynamic_min_max_shapes()
  34. .. py:method:: get_data_info()
  35. ³ģʽ£ȡǰݵͺ״(shape)ͨ״(shape)̬仯ijʹá
  36. ****
  37. >>> from mindspore import DatasetHelper
  38. >>>
  39. >>> train_dataset = create_custom_dataset()
  40. >>> dataset_helper = DatasetHelper(train_dataset, dataset_sink_mode=True)
  41. >>>
  42. >>> types, shapes = dataset_helper.get_data_info()
  43. .. py:method:: release()
  44. ͷ³Դ
  45. .. py:method:: sink_size()
  46. ȡÿε `sink_size`
  47. ****
  48. >>>from mindspore import DatasetHelper
  49. >>>
  50. >>>train_dataset = create_custom_dataset()
  51. >>>dataset_helper = DatasetHelper(train_dataset, dataset_sink_mode=True, sink_size=-1)
  52. >>>
  53. >>># if sink_size==-1, then will return the full size of source dataset.
  54. >>>sink_size = dataset_helper.sink_size()
  55. .. py:method:: stop_send()
  56. ֹͣ³ݡ
  57. .. py:method:: types_shapes()
  58. ӵǰеݼȡͺ״(shape)
  59. ****
  60. >>>from mindspore import DatasetHelper
  61. >>>
  62. >>>train_dataset = create_custom_dataset()
  63. >>>dataset_helper = DatasetHelper(train_dataset, dataset_sink_mode=True)
  64. >>>
  65. >>>types, shapes = dataset_helper.types_shapes()