Aaron Miller 6624d7b2dd sampling: remove incorrect offset for n_vocab (#900) ...

no effect, but avoids a *potential* bug later if we use
actualVocabSize - which is for when a model has a larger
embedding tensor/# of output logits than actually trained token
to allow room for adding extras in finetuning - presently all of our
models have had "placeholder" tokens in the vocab so this hasn't broken
anything, but if the sizes did differ we want the equivalent of
`logits[actualVocabSize:]` (the start point is unchanged), not
`logits[-actualVocabSize:]` (this.)

2023-06-08 11:08:10 -07:00

9.7 KiB

Raw Blame History

View Raw