YangLuo
bf48400c10
unify include file of minddata
4 years ago
Cathy Wong
636809c255
dataset: C++ API ToNumber & Lookup: change data_type parm from string to DataType
4 years ago
Cathy Wong
a2ee0cb3e5
dataset UT: Update UTs to check for return Status
4 years ago
mindspore-ci-bot
0cfd8b7c9e
!13364 Add WordpieceTokenizer and AddDict to Jieba
From: @alexyuyue
Reviewed-by:
Signed-off-by:
5 years ago
alex-yuyue
2d6d608fae
Add WordpieceTokenizer to API and IR. Add AddDict to C++ API
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
5 years ago
Cathy Wong
1775c4e83d
dataset: Reinstate INFO logging and data verification - part 3
Updates: GetItemAt, CreateFromVector, CreateScalar.
Add TEST_MS_LOG_MSTENSOR.
Wrap ASSERT_OK around Tensor function calls.
5 years ago
Zirui Wu
8022001450
Remove datasets.h dependency on tensor.h and other related dependencies
Replace std::function<TensorRow(TensorRow)> with std::function<MSTensorVec(MSTensor)> by using std::bind
Replace pad_info with MSTensor
Switch TensorShape/dataset::DataType to vector<int64> /mindspore::DataType
Change TypeId to mindspore::DataType in SchemaObj
address review cmts, add CreateFromMSTensor() in tensor.h/.cc
5 years ago
TinaMengtingZhang
c5aa3eeef2
Add API class for data transform ops except Compose, RandomApply, RandomChoice
Add API class for text transforms ops
5 years ago
Eric
9953757ff4
Fixed tenor compile error for MD
5 years ago
Cathy Wong
3e6e35d749
dataset: delete minddata/dataset/include/status.h
5 years ago
lixian
7d2fd6e76c
cxx api refactor: tensor/status/model
5 years ago
alex-yuyue
a8675f8227
TextOp decoupling
Signed-off-by: alex-yuyue <yue.yu1@huawei.com>
5 years ago
mohammad
f48ab2b5c9
make MD public headers standalone
5 years ago
YangLuo
da6de519ae
Add ut for dataset to_number
5 years ago
Xiao Tianci
0659473535
add TruncateSequencePair, ToNumber C++ API and enable three test cases
5 years ago
mindspore-ci-bot
6b5626634c
!9530 add BasicTokenizer and BertTokenizer C++ API
From: @tiancixiao
Reviewed-by: @mikef,@nsyca
Signed-off-by: @nsyca
5 years ago
shenwei41
206ae1a7c3
Add Three text C++ API
5 years ago
Xiao Tianci
290cc72887
add BasicTokenizer and BertTokenizer C++ API
5 years ago
shenwei41
3bea84d0f7
add four new text API
5 years ago
mohammad
9cfeb08c4d
port name() to TensorOperation and its derived classes
5 years ago
shenwei41
4e56618d18
Add text C++ API
5 years ago