How to manage resources in Python using a context manager

Proper resource management is important when building applications to avoid memory leaks, ensure proper cleanup, and maintain application stability. Context managers provide an effective solution for this situation. It streamlines resource management by automating the process of acquiring and releasing resources.

Context manager

Essentially, a context manager is an object that defines methods for acquiring and releasing resources when needed. Context managers are useful because they can organize resource management into a clear, simple, and precise structure. Using a context manager can reduce code duplication and make it easier to read.

Think of a program that has to write data in a file. Whenever the application needs to log something, you have to manually open and close the log file because there is no context manager. However, if you use it, you easily streamline setup and deconstruct logging resources, ensuring proper handling of logging tasks.

The with command

The with command in Python provides a way to use a context manager. Even if exceptions occur while the code block is deployed, it ensures that the resulting resources are released appropriately after being used as intended.

with context_manager_expression as resource: # Khối code sử dụng tài nguyên này # Tài nguyên tự động được giải phóng khi khối code thoát

By using the with command , you give the context manager control over resource management, and better focus on the application logic.

Use the built-in context manager

Python provides built-in context managers for common cases. You'll see two examples: handling files with the open() function and managing network connections with the socket module .

Handling files with open()

The open() function is an inbuilt context manager used for file processing. It is frequently used for reading or writing files and returning a file object. When using a context manager for file management, it avoids potential data corruption by automatically closing files when no longer needed.

with open('file.txt', 'r') as file: content = file.read() # Xử lý nội dung # File tự động đóng sau khi thoát khối code

Connect to the network using socket()

The socket module provides a context manager for network sockets. The context manager can ensure proper setup and proper segmentation when working with network connections, preventing connection vulnerabilities.

import socket with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: s.connect(('localhost', 8080)) # Gửi/nhận dữ liệu trên socket # Socket tự động được đóng sau khi thoát khối

Implement a custom context manager

Custom context managers allow you to encapsulate specific resource management or behavior in code. Python provides different ways to create customized context managers for different situations. Here, you will explore class and function based methods.

Context manager using class-based approach

In the class-based approach, you define a class that implements __enter__ and __exit__ magic or dunder . The __enter__ method initializes and returns the resource you want to manage, while the __exit__ method ensures proper cleanup, even when an exception occurs.

class CustomContext: def __enter__(self): # Yêu cầu tài nguyên return resource def __exit__(self, exc_type, exc_value, traceback): # Giải phóng tài nguyên pass

Consider a task where you have to run some process. This task needs a context manager that will simplify the simultaneous execution of all processes, providing resource management, synchronization, and accurate error management.

import multiprocessing import queue class ProcessPool: def __init__(self, num_processes): self.num_processes = num_processes self.processes = [] def __enter__(self): self.queue = multiprocessing.Queue() for _ in range(self.num_processes): process = multiprocessing.Process(target=self._worker) self.processes.append(process) process.start() return self def __exit__(self, exc_type, exc_value, traceback): for process in self.processes: # Gửi giá trị dự phòng để báo hiệu quá trình worker nghỉ self.queue.put(None) for process in self.processes: process.join() def _worker(self): while True: number = self.queue.get() if number is None: break calculate_square(number) def calculate_square(number): result = number * number print(f"The square of {number} is {result}") if __name__ == "__main__": numbers = [1, 2, 3, 4, 5] # Sử dụng with ProcessPool(3) as pool: for num in numbers: pool.queue.put(num) # Tự động bắt đầu các quá trình và # được nối khi thoát khối "with"

The ProcessPool context manager manages a pool of workers, allocating tasks to these processes for concurrent execution. This parallelism can let you make more efficient use of available CPU cores and run tasks faster than sequentially deploying them in a single process.

Context manager using a function-based approach

The contextlib module provides the @contextmanager decorator to create a context manager using constructors. Decorators allow you to add functionality to a function without having to modify it.

In the 'decorate' constructor, you can use yield and final to indicate the location of the required resource and where it is released.

from contextlib import contextmanager @contextmanager def custom_context(): # Code để yêu cầu tài nguyên resource = . try: yield resource # Resource is provided to the with block finally: # Code để giải phóng tài nguyên pass

Suppose you want to develop a context manager that calculates the execution time of a block of code. You can do this by implementing a function-based strategy.

import time from contextlib import contextmanager @contextmanager def timing_context(): start_time = time.time() try: yield finally: end_time = time.time() elapsed_time = end_time - start_time print(f"Elapsed time: {elapsed_time} seconds") # Cách dùng with timing_context(): # Khối code đo thời gian thực thi time.sleep(2)

In this example, the timing_context context manager records the start and end times of the code block and calculates the elapsed time when the code block exits.

Using either of the above approaches, you can build custom context managers to encapsulate complex resource management logic and repetitive operations, improving arrangements and code maintenance.

Above is the basic information you need to know about how to manage resources in Python using the context manager . Hope the article is useful to you.

Lesley Montoya

Update 17 September 2023