PKU-DAIR
/
Hetu
Not watched
1
0
Fork 0
Code
Releases 0 Wiki evaluate Activity
Issues 0 Pull Requests 0 Datasets Model Cloudbrain HPC
You can not select more than 25 topics Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
25 Commits
1 Branch
116 MB
3 Download
 
 
 
 
 
 
Branch: master
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from 'master'
${ noResults }
Hetu/python/hetu/gpu_ops
Hsword 7fd3de80ce initial commit 4 years ago
..
AddConst.py initial commit 4 years ago
AddElewise.py initial commit 4 years ago
AllReduceCommunicate.py initial commit 4 years ago
AvgPool.py initial commit 4 years ago
BatchMatrixMult.py initial commit 4 years ago
BatchNorm.py initial commit 4 years ago
BinaryCrossEntropy.py initial commit 4 years ago
Broadcast.py initial commit 4 years ago
BroadcastShape.py initial commit 4 years ago
Concat.py initial commit 4 years ago
Conv2d.py initial commit 4 years ago
Conv2dBroadcast.py initial commit 4 years ago
Conv2dReduceSum.py initial commit 4 years ago
CuSparse.py initial commit 4 years ago
DataTransfer.py initial commit 4 years ago
Dispatch.py initial commit 4 years ago
DistGCN_15d.py initial commit 4 years ago
Division.py initial commit 4 years ago
Dropout.py initial commit 4 years ago
Dropout2d.py initial commit 4 years ago
EmbeddingLookUp.py initial commit 4 years ago
InstanceNorm2d.py initial commit 4 years ago
LayerNorm.py initial commit 4 years ago
LeakyRelu.py initial commit 4 years ago
MatrixDot.py initial commit 4 years ago
MatrixMult.py initial commit 4 years ago
MaxPool.py initial commit 4 years ago
MultiplyConst.py initial commit 4 years ago
MultiplyElewise.py initial commit 4 years ago
Node.py initial commit 4 years ago
OneHot.py initial commit 4 years ago
OnesLike.py initial commit 4 years ago
Opposite.py initial commit 4 years ago
Pad.py initial commit 4 years ago
ParameterServerCommunicate.py initial commit 4 years ago
PipelineReceive.py initial commit 4 years ago
PipelineSend.py initial commit 4 years ago
README.md initial commit 4 years ago
ReduceMean.py initial commit 4 years ago
ReduceSum.py initial commit 4 years ago
ReduceSumAxisZero.py initial commit 4 years ago
Relu.py initial commit 4 years ago
Reshape.py initial commit 4 years ago
Sigmoid.py initial commit 4 years ago
Slice.py initial commit 4 years ago
Softmax.py initial commit 4 years ago
SoftmaxCrossEntropy.py initial commit 4 years ago
Split.py initial commit 4 years ago
Sqrt.py initial commit 4 years ago
Tanh.py initial commit 4 years ago
Transpose.py initial commit 4 years ago
Variable.py initial commit 4 years ago
Where.py initial commit 4 years ago
ZerosLike.py initial commit 4 years ago
__init__.py initial commit 4 years ago
executor.py initial commit 4 years ago

README.md

gpu_ops

This directory contains executor and operators for computation and communication. Though the name of directory is "gpu_ops", in each operator we call different API for computation in NumPy(CPU), DNNL(CPU), CUDA(GPU) according to the context specified in executor and the environment.

Executor

  • Defined in executor.py, contains all the configurations and controls the training/inference process.

Operators

Computation

Operator NumPy(CPU) DNNL(CPU) CUDA(GPU) CUDA Backend
AddByConstOp ✔ ✔ ✔ /
AddOp ✔ ✔ ✔ /
Avg_Pool2dOp ✔ ✔ ✔ CuDNN
Avg_Pool2d_GradientOp ✔ ✔ ✔ CuDNN
BatchMatMulOp ✔ ✖ ✔ CuBLAS
Batch_NormalizationOp ✔ ✔ ✔ CuDNN
Batch_Normalization_GradientOp ✔ ✔ ✔ CuDNN
BinaryCrossEntropyOp ✔ ✖ ✔ /
BroadcastToOp ✔ ✖ ✔ /
BroadcastShapeOp ✔ ✖ ✔ /
ConcatOp ✔ ✔ ✔ /
Concat_gradientOP ✔ ✔ ✔ /
Conv2dOp ✔ ✔ ✔ /
Conv2d_Gradient_of_DataOp ✔ ✔ ✔ /
Conv2d_Gradient_of_FilterOp ✔ ✔ ✔ /
Conv2d_BroadcastToOp ✔ ✖ ✔ /
Conv2d_ReduceSumOp ✔ ✖ ✔ /
CsrmvOp ✔ ✖ ✔ /
CsrmmOp ✔ ✖ ✔ /
DistGCN_15dOp ✖ ✖ ✔ /
DivOp ✔ ✔ ✔ /
DivConstOp ✔ ✔ ✔ /
DropoutOp ✔ ✔ ✔ CuRAND
Dropout_GradientOp ✔ ✔ ✔ CuRAND
Dropout2dOp ✖ ✖ ✔ CuRAND
Dropout2d_GradientOp ✖ ✖ ✔ CuRAND
EmbeddingLookUp ✔ ✖ ✔ /
EmbeddingLookUp_Gradient ✔ ✖ ✔ /
Instance_Normalization2dOp ✖ ✖ ✔ CuDNN
Instance_Normalization2d_GradientOp ✖ ✖ ✔ CuDNN
Layer_NormalizationOp ✔ ✖ ✔ CuDNN
Layer_Normalization_GradientOp ✔ ✖ ✔ CuDNN
LeakyReluOp ✖ ✖ ✔ /
LeakyReluGradientOp ✖ ✖ ✔ /
MatrixDotOp ✔ ✖ ✔ /
MatMulOp ✔ ✔ ✔ CuBLAS
Max_Pool2dOp ✔ ✔ ✔ CuDNN
Max_Pool2d_GradientOp ✔ ✔ ✔ CuDNN
MulByConstOp ✔ ✔ ✔ /
MulOp ✔ ✔ ✔ /
OneHotOp ✔ ✖ ✔ /
OnesLikeOp ✔ ✔ ✔ /
OppositeOp ✔ ✔ ✔ /
PadOp ✔ ✔ ✔ /
Pad_GradientOp ✔ ✔ ✔ /
ReduceMeanOp ✔ ✖ ✔ CuDNN
ReduceSumOp ✔ ✖ ✔ CuDNN
ReduceSumAxisZeroOp ✔ ✔ ✔ /
ReluOp ✔ ✔ ✔ /
ReluGradientOp ✔ ✔ ✔ /
Array_ReshapeOp ✔ ✔ ✔ /
SigmoidOp ✔ ✔ ✔ /
SliceOp ✔ ✖ ✔ /
SliceGradientOp ✔ ✖ ✔ /
SoftmaxOp ✔ ✔ ✔ CuDNN
SoftmaxGradientOp ✔ ✖ ✔ CuDNN
SoftmaxCrossEntropyOp ✔ ✔ ✔ CuDNN (Optional)
SoftmaxCrossEntropyGradientOp ✔ ✖ ✔ CuDNN (Optional)
SplitOp ✔ ✖ ✔ /
SplitGradientOp ✔ ✖ ✔ /
SqrtOp ✔ ✔ ✔ /
ReciprocalSqrtOp ✔ ✔ ✔ /
TanhOp ✔ ✔ ✔ /
TransposeOp ✔ ✔ ✔ /
WhereOp ✔ ✖ ✔ /
ZerosLikeOp ✔ ✔ ✔ /
OptimizerOp ✔ ✔ ✔ /
OptimizerOp for sparse ✔ ✖ ✔ /
DataloaderOp ✔ ✔ / /

Communication

  • DataH2DOp
  • DataD2HOp
  • DataD2HSparseOp
  • AllReduceCommunicateOp
  • ParameterServerCommunicateOp
  • PipelineSendOp
  • PipelineReceiveOp
  • Dispatch
Delete task

Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.

cancel
confirm

分布式深度学习系统

Python C++ Cuda C CMake other