tokenizer_fineweb / bpe_32k.model
khalidalt's picture
add tokenizer files
c2598b6
This file is stored with Xet . It is too big to display, but you can still download it.

Large File Pointer Details

( Raw pointer file )
SHA256:
584ca031372f6c0dee4ad3275cf3104145b28a0979357087d7023cf830515292
Pointer size:
131 Bytes
·
Size of remote file:
829 kB
·
Xet hash:
ecf35a3e019f350a4d6987fc7c0f4a5d59e425684aa5ee6759acda9f22da5866

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.