File and directory management is a crucial aspect of any software development process, whether you're building a simple script or a complex application. Python provides several built-in libraries that make it easier to interact with the file system, allowing you to create, read, write, and manipulate files and directories. The os, shutil, and pathlib modules are the core tools used for managing files and directories in Python.
In this blog post, we will cover:
shutil and osos.path and pathlibPython provides several modules for interacting with files and directories:
os module: This is the core module for interacting with the operating system. It provides functions for file and directory management, such as creating, deleting, and renaming files and directories.shutil module: The shutil module provides a higher-level interface to file operations, such as copying, moving, and removing files and directories.pathlib module: Introduced in Python 3.4, pathlib offers a modern and object-oriented approach to file system paths.Python’s os module provides functions to create and delete directories. Here's how you can do that:
To create a directory, you can use the os.mkdir() function. To create multiple directories (including parent directories), use os.makedirs().
import os
# Create a single directory
os.mkdir('my_directory')
# Create a directory and any necessary parent directories
os.makedirs('parent_directory/child_directory')
You can delete an empty directory with os.rmdir(). To delete non-empty directories, use shutil.rmtree().
import os
import shutil
# Remove an empty directory
os.rmdir('my_directory')
# Remove a non-empty directory
shutil.rmtree('parent_directory/child_directory')
To list all the files and directories in a given directory, you can use os.listdir(). It returns a list of filenames in the specified directory.
import os
# List files and directories in the current directory
print(os.listdir('.'))
If you want to list only files or directories, you can filter the results:
import os
# List only directories
dirs = [d for d in os.listdir('.') if os.path.isdir(d)]
print(dirs)
# List only files
files = [f for f in os.listdir('.') if os.path.isfile(f)]
print(files)
You can change the current working directory using os.chdir().
import os
# Change current working directory
os.chdir('/path/to/your/directory')
To get the current working directory, you can use os.getcwd().
import os
# Get current working directory
print(os.getcwd())
Python provides multiple ways to read files, such as using open(). Here's an example:
# Open a file for reading
with open('example.txt', 'r') as file:
content = file.read()
print(content)
'r' is the mode for reading the file (default).'w' is the mode for writing to a file (creates a new file or overwrites an existing file).'a' is for appending data to an existing file.
with open('example.txt', 'r') as file:
for line in file:
print(line.strip()) # strip removes leading/trailing whitespace
To write data to a file, you can use open() in write ('w') or append ('a') mode.
with open('example.txt', 'w') as file:
file.write("Hello, this is a new line of text!")
with open('example.txt', 'a') as file:
file.write("\nThis is an appended line.")
shutil and osThe shutil module provides several utilities for file management, such as copying and moving files. Here are some examples:
import shutil
# Copy a file to a new location
shutil.copy('source.txt', 'destination.txt')
import shutil
# Move a file to a new location
shutil.move('source.txt', 'new_directory/source.txt')
You can rename a file using os.rename():
import os
# Rename a file
os.rename('old_name.txt', 'new_name.txt')
os.path and pathlibPython provides two primary ways to work with file paths: os.path (traditional) and pathlib (modern and object-oriented).
os.pathos.path provides a variety of functions for manipulating file paths:
import os
# Get the absolute path of a file
abs_path = os.path.abspath('example.txt')
print(abs_path)
# Check if a file or directory exists
print(os.path.exists('example.txt'))
# Join paths in a platform-independent manner
path = os.path.join('folder', 'subfolder', 'example.txt')
print(path)
# Get the file name from a path
file_name = os.path.basename('/folder/subfolder/example.txt')
print(file_name)
# Get the directory name from a path
dir_name = os.path.dirname('/folder/subfolder/example.txt')
print(dir_name)
pathlibThe pathlib module provides a modern, object-oriented way to work with file paths.
from pathlib import Path
# Create a Path object
path = Path('example.txt')
# Get the absolute path
print(path.resolve())
# Check if the path exists
print(path.exists())
# Join paths
new_path = Path('folder') / 'subfolder' / 'example.txt'
print(new_path)
# Get file name and directory name
print(path.name) # File name
print(path.parent) # Directory name
Python’s shutil module makes it easy to copy, move, and rename files:
shutil.copy(src, dst) or shutil.copy2(src, dst) for preserving metadata.shutil.move(src, dst) moves or renames files.os.rename(src, dst).
import shutil
# Copy a file
shutil.copy('source.txt', 'destination.txt')
# Move or rename a file
shutil.move('old_name.txt', 'new_name.txt')
To delete files or directories, you can use os.remove() for files and os.rmdir() for empty directories. For non-empty directories, shutil.rmtree() is the recommended option.
import os
# Delete a file
os.remove('example.txt')
import os
# Delete an empty directory
os.rmdir('empty_directory')
# Delete a non-empty directory
import shutil
shutil.rmtree('non_empty_directory')
with Statements: Always open files using the with statement to ensure files are properly closed after operations.os.path.exists() or pathlib.Path.exists() to check if a file or directory exists before performing operations.pathlib for Path Manipulation: pathlib is more modern, cleaner, and more powerful for manipulating file paths than os.path.rmtree(): Deleting non-empty directories with shutil.rmtree() is irreversible, so make sure you are targeting the correct directory.