PySpark provides the facility to upload your files using sc.addFile. We can also get the path of working directory using SparkFiles.get. Moreover, to resolve the path to the files added through SparkContext.addFile(), the following types of class method are available in SparkFiles, such as:
Note: SparkFiles only contain the class method that can be used as required. Users should not create SparkFiles instances.
Let's learn about the class method in detail.
Class Methods of PySpark SparkFiles
The get(filename) specifies the path of the file which is added through SparkContext.addFile().
This class method specifies the path to the root directory. Basically, it contains the whole file which is added through the SparkContext.addFile().
Next TopicPySpark StorageLevel