The Celery project recently addressed several critical stability issues affecting distributed task execution and backend resource management. Recent updates focus on preventing worker process stalling during broker reconnections and fixing a silent memory leak in the Redis result backend.

Scrapy continues to evolve as the primary framework for large scale web scraping in the Python ecosystem. Recent updates show a clear focus on modernizing the underlying network stack and adopting contemporary features. By integrating support for HTTP/2 and refining how security protocols are handled, the framework ensures it can interact with modern web servers efficiently. These changes also include improvements to the developer experience through better type hints and the adoption of dataclasses for data modeling. The project maintainers are prioritizing stability and performance as the web moves toward more sophisticated protocols.

OpenSanctions recently released a large update to its data processing engine and PEP dataset collection. This week saw over 130 commits focusing on structural refactors to the occupancy data model and significant hardening of crawler reliability against source web application firewalls.

Scrapy recently introduced support for verbatim URLs to handle non standard web servers and fixed a resource leak in Google Cloud Storage feed exports. These updates improve the robustness of data extraction pipelines when dealing with unconventional URL encoding and cloud storage backends.