Description
Prior to Spark 2.3.3, in certain situations Spark would write user data to local disk unencrypted, even if spark.io.encryption.enabled=true. This includes cached blocks that are fetched to disk (controlled by spark.maxRemoteBlockSizeFetchToMem); in SparkR, using parallelize; in Pyspark, using broadcast and parallelize; and use of python udfs.
Affected products
| Vendor | Product | Affected | Fixed |
|---|---|---|---|
| Apache | Apache Spark | 2.3.2 and below | — |
Affected packages
| Ecosystem | Package | Affected | Fixed | Record |
|---|---|---|---|---|
| Maven | org.apache.spark:spark-core_2.11 | < 2.3.3 | 2.3.3 | GHSA-fp5j-3fpf-mhj5 |
| PyPI | pyspark | < 2.3.3 | 2.3.3 | PYSEC-2019-114 |
References
- https://lists.apache.org/thread.html/c2a39c207421797f82823a8aff488dcd332d9544038307bf69a2ba9e%40%3Cuser.spark.apache.org%3E
- [spark-issues] 20200318 [jira] [Commented] (SPARK-28626) Spark leaves unencrypted data on local disk, even with encryption turned on (CVE-2019-10099) mailing-list
- [spark-commits] 20200622 [spark-website] branch asf-site updated: CVE-2020-9480 details (#275) mailing-list