Spark Application Garbage Collector

ContextCleaneris a Spark service that is responsible for application-wide cleanup of shuffles , RDDs , broadcasts , accumulators and checkpointed RDDs that is aimed at reducing the memory requirements of long-running data-heavy Spark applications.

ContextCleaner runs on the driver. It is created and immediately started when SparkContext starts (and spark.cleaner.referenceTracking Spark property is enabled, which it is by default). It is stopped when SparkContext is stopped

.