You can not select more than 25 topicsTopics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
1) extend the Liner cell for including BatchMatMul implementation, in
which the first dimension indicates the expert number;
2) implement a Switch (top1) router;
3) implement a MoE cell, which extends the FeedForward cell.