Javatpoint Logo
Javatpoint Logo

PySpark SparkFiles

PySpark provides the facility to upload your files using sc.addFile. We can also get the path of working directory using SparkFiles.get. Moreover, to resolve the path to the files added through SparkContext.addFile(), the following types of class method are available in SparkFiles, such as:

  • get(filename)
  • getrootdirectory()

Note: SparkFiles only contain the class method that can be used as required. Users should not create SparkFiles instances.

Let's learn about the class method in detail.

Class Methods of PySpark SparkFiles

  • get(filename)

The get(filename) specifies the path of the file which is added through SparkContext.addFile().

  • getrootdirectory()

This class method specifies the path to the root directory. Basically, it contains the whole file which is added through the SparkContext.addFile().

Youtube For Videos Join Our Youtube Channel: Join Now


Help Others, Please Share

facebook twitter pinterest

Learn Latest Tutorials


Trending Technologies

B.Tech / MCA