cover of episode 💥 Spark DataFrame Cache

💥 Spark DataFrame Cache

2023/1/11
logo of podcast Programmers Quickie

Programmers Quickie

Frequently requested episodes will be transcribed first

Shownotes Transcript

In Apache Spark, you can cache a DataFrame in memory using the cache() or persist() method. The cache() method is a shorthand for persist() with the default storage level of MEMORY_ONLY. You can use the persist() method to specify a different storage level if desired, such as MEMORY_AND_DISK or MEMORY_ONLY_SER.