pyspark.SparkContext.addPyFile#

SparkContext.addPyFile(path)[source]#

Add a .py or .zip dependency for all tasks to be executed on thisSparkContext in the future. Thepath passed can be either a localfile, a file in HDFS (or other Hadoop-supported filesystems), or anHTTP, HTTPS or FTP URI.

New in version 0.7.0.

Parameters

pathstr

can be either a .py file or .zip dependency.

See also

SparkContext.addFile()

Notes

A path can be added only once. Subsequent additions of the same path are ignored.