Skip to content
OpenAdvisories

Product · huggingface

tokenizers

1 known vulnerabilities.

1 result · Clear filters · Feed

  • Medium 6.5

    tokenizers BpeBuilder Buffer Overflow via merge token

    tokenizers (Hugging Face) is affected by an out-of-bounds buffer access in BpeBuilder::build (tokenizers/src/models/bpe/model.rs). When loading a tokenizer.json via Tokenizer::from_file/from_str, the builder sizes a scratch buffer to the longest vocabulary ke…