hex_core decompresses registry and Erlang term API responses without size limits

Improper Handling of Highly Compressed Data (Data Amplification) vulnerability in hexpm hex_core allows an untrusted repository mirror, or a server answering hex_core's API requests, to exhaust the memory of the Erlang VM running hex_core. The same code is vendored in Hex (mix_hex_repo, mix_hex_api) and rebar3 (r3_hex_repo, r3_hex_api).

hex_repo:get_names/1, hex_repo:get_versions/1, hex_repo:get_package/2 and hex_repo:get_policy/2 decompress the registry response with zlib:gunzip/1 and decode the signed envelope before verifying its signature. A mirror doesn't have the repository's private key, but it chooses the compressed bytes, so a small response decompresses into an envelope of any size before the signature check rejects it.

hex_api decodes responses with the application/vnd.hex+erlang content type through hex_safe_binary_to_term:safe_binary_to_term/1, which calls binary_to_term(Binary, [safe]). The safe option prevents atom creation, but it accepts compressed terms and doesn't limit the size of the decoded term, and the check that rejects functions and ports runs after the term has been built. The fix for CVE-2026-21619 added the safe option and that check without bounding the decoded size.

This issue affects rebar3: from 3.9.1 before 3.27.3; hex: from 2.1.0 before 2.5.2 and from 2.6.0 onward; hex_core: from 0.1.0 before 0.20.0.