4 Commits (a468cd4c0c198a561284624a0f185da2369e857f)

Author SHA1 Message Date
  Niklas Gustafsson a468cd4c0c Revert "Generalized Tokenizer argument types." 4 years ago
  Niklas Gustafsson d9465d8ab0 Generalized Tokenizer argument types. 4 years ago
  Niklas Gustafsson 0413e5197a Adding more implementations to Tokenizer -- seq->text, pre-tokenized texts, etc. 4 years ago
  Niklas Gustafsson 606dba816e Started implementation of Tokenizer(), as well as pad_sequences(). 4 years ago