jiangshuqiang
147b531652
add ut for summary_parser
5 years ago
ougongchang
dea874e30b
Update level of crc failed message to error level
When CRC error occurs, it is generally caused by incomplete data in the file, so error information should be given to prompt the user.
5 years ago
jiangshuqiang
4a621688d3
fix log level
5 years ago
ougongchang
2ac87e663d
fix the pylint error in test_summary_watcher file
5 years ago
ougongchang
b5e21cf4ab
add some ut for list explain directories and reduct code function length
5 years ago
wangshuide2020
dd7e72e811
add notes for init files.
5 years ago
Li Hongzhang
da797e3e2f
parse pb files in executor
5 years ago
Li Hongzhang
c2210200fc
redefine reload interval
5 years ago
mindspore-ci-bot
5e6932f0f6
!601 Add the summary loading switch mechanism
Merge pull request !601 from LiHongzhang/fix_caching
5 years ago
yelihua
50e1400505
add debugger module
5 years ago
Li Hongzhang
990800239b
add summary loading switch mechanism
5 years ago
wangshuide2020
f847850382
add ut and st of tensor visualization
5 years ago
luopengting
1c73d20cc7
mainly to new a thread to load detail info
1. New a thread to load detail info. Loading detail info takes too much time,
so the summary list and lineage can not be loaded timely.
2. Add a status for DetailCacheManager to indicate it is INIT, LOADING or DONE.
3. Update UT/ST.
5 years ago
wenkai
26fabf4770
Refactor the mindinsight multiprocessing computing code to use a unified manager.
Main features:
1. Use the ComputingResourceManager to manage all computing workers.
2. Ensure fair worker number between summary directories at first. So every summary directories in cache will be loaded simultaneously.
3. When a summary directory is loaded, it's worker will be released, and other unfinished summary directories can use the released workers to speed up. This way we solved the slow worker problem.
Code changes:
1. Added computing_resource_mgr.py
2. Passed ComputingResourceManager instances instead of workers_count
3. Simplified the _load_single_fine() function a bit.
5 years ago
wangshuide2020
e8ffeb70ef
Support tensor visualization. 1.Tensor display in a table, it can support no more than two dimensions tensor visualization; 2.Tensor histogram visualization for all step in cache.
5 years ago
wenkai
c610544905
fix ZeroDivisionError when original bucket width is 0 by checking the width.
5 years ago
ougongchang
2281686922
modify the ms data loader and abstract the parser class
5 years ago
wenkai
7e17d6ffae
refactor data manager and unify cache and data access/reload
5 years ago
luopengting
ee07d8364d
fix pylint
5 years ago
luopengting
660a8dbfd4
optimize parsing for out-of-order events
5 years ago
mindspore-ci-bot
988aad751f
!58 support resampling buckets
Merge pull request !58 from wenkai/wk0422
5 years ago
wenkai
84a39a4e92
resample buckets
5 years ago
mindspore-ci-bot
fd6bb5e460
!53 fix parsing pb file bug and abstracting pb parsing
Merge pull request !53 from ougongchang/fixbug_pb_file
5 years ago
ougongchang
d0cd07904c
fix sorting pb file bug and abstracting pb parsing
5 years ago
mindspore-ci-bot
284f2d8027
!48 Enhance param checking and error msg in lineagemgr, set user-define's required as False
Merge pull request !48 from luopengting/lineage_lpt
5 years ago
wenkai
c4c74bd6b4
cross-step bucket unify
5 years ago
luopengting
670bc72017
enhance validation for cmp operation, set user-difined' required value as False
5 years ago
李鸿章
5b455143fa
resolve the crc32 c/c++ extension
5 years ago
ougongchang
a807c45a4a
add some exceptions, such as TrainJobNotExistError, GraphNotExistError and so on
6 years ago
李鸿章
79a615269a
fix pylint warnings
6 years ago
luopengting
2b670b156f
move log operations to tests.utils, change import method to relative path
6 years ago
gaocongli
e7a0496e87
initial version
6 years ago