3 Commits (d9465d8ab01f1bb1f53977fa5fb23931382b4112)

Author SHA1 Message Date
  Niklas Gustafsson d9465d8ab0 Generalized Tokenizer argument types. 4 years ago
  Niklas Gustafsson 0413e5197a Adding more implementations to Tokenizer -- seq->text, pre-tokenized texts, etc. 4 years ago
  Niklas Gustafsson 606dba816e Started implementation of Tokenizer(), as well as pad_sequences(). 4 years ago