Useful Path Handling Functions in Python

Photo by Birgit Held from Pexels

Recently, I published an article about path handling functions in R which are built-in functions in base R.

Since I use both languages at my work — I work as a Bioinformatician (= a Data Scientist/Analyst working on biomedical datasets), I wanted to have the same list for Python.

Path handling functions in Python are not built-in into base Python. They are distributed across three standard packages: os, pathlib(for Python3; pathlib2 when using Python2), andshutil. Therefore, they are shipped with Python, so at least you don’t have to install them. Often, each of the packages offer solutions for different problems in path handling. None of the packages contains all necessary functions. Thus, the listing in Python looks heterogeneous and not so straight forward and smooth like in base R.

In this article, I give you my researched corresponding Python functions for each of the R commands given in the previous article “Useful Path Handling Functions in R” .

1. How to get/set the current directory in Python

Get the current working directory:

os.path.abspath(os.getcwd())  # abspath because to deal with Windows
os.path.realpath('.') # eliminates symbolic links
# and replace them by the real path
pathlib.Path.cwd()
pathlib.Path(__file__).resolve() # resolve to absolute path

Set the current working directory:

os.chdir("/to/dir") # no corresponding in pathlib or shutil

2. How to dissect and access Path Name Components in Python

Dir name (parent directory’s name of the path):

os.path.dirname(path)

Base name (file name of the path):

os.path.basename(path)

File extensions:

filename, file_extension = os.path.splitext(path)

3. How to join Path Components in Python

os.path.join("/dir", "dir", "file.txt")                                                
pathlib.Path("/dir") / "dir" / "file.txt"
pathlib.PurePath("/dir", "dir", "file.txt")
# make Path() objects to string using `str()`
# or their `.as_posix()` method
# they all create "/dir/dir/file.txt"

4. How to recursively list Files in Folders in Python

pathlib.Path("/dir").glob("**/*.csv") 
# recursivity through wildcards `**`!

5. How to test for Existence of Files and Folders in Python

# for both:
pathlib.Path("/to/dir/or/file.txt").exists()
# for folders
pathlib.Path("to/dir").is_dir()
os.path.isdir("to/dir")
# for files
pathlib.Path("to/file.txt").is_file()
os.path.isfile("to/dir")
# there is also
os.path.islink(path)

6. How to create new Folders and Files in R

pathlib.Path("/to/new/dir").mkdir(exist_ok=True, parents=True)
os.makedirs("/to/new/dir", exist_ok=True)
# create a file (touch)
pathlib.Path("/to/new/file.txt").touch()

7. How to (recursively) delete Files and Folders in Python

# for both
os.unlink("/to/dir/or/file.txt")
pathlib.Path("/to/dir/or/file.txt").unlink()
# rmdir (however, I never liked rmdir - it deletes only empty folders)
os.rmdir(dir_path, missing_ok=True)

8. How to copy Folders with contents and Files in R

# copy a folder recursively
shutil.copytree(src="/from/dir", dst="/to/dir",
symlinks=False,
ignore=None,
copy_function=shutil.copy2,
ignore_dangling_symlinks=False,
dirs_exist_ok=False)
# copy a file
shutil.copyfile("/from/file.txt",
"/to/file.txt", # must be a complete file
follow_symlinks=True)
shutil.copy("/to/file.txt", "/new/dir/file.txt") # only Python3.8+ # # in <Python3.8 put `str()` around Path() objects! or `.as_posix()`!

9. How to move and rename Folders and Files in in Python

# move a folder recursively
shutil.move(src="/from/dir", dst="/new/dir", copy_function=shutil.copy2)
# Files:
os.rename("/to/file.txt", "/new/dir")
pathlib.Path("/to/file.txt").rename("/new/dir/file.txt")
shutil.move("/to/file.txt", "/new/dir")

10. How to get the size and creation time of Folders and Files in Python

os.getsize(path)
os.getatime(path)
os.getctime(path)
os.getmtime(path)
# and one can use the setters tooos.path.isdir(path)# look for methods in
dir(os)
dir(os.path)
dir(pathlib.Path)

In general, path handling in Python is much more hetereogenous and messy than in R. Perhaps, one should streamline Python path handling functions to be R-like in a new package. This should be the topic of the next publication.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store