Decaf Blog In-depth updates on Decaf software, tooling, and open-source experiments
  • Aggregation
  • Airbyte
  • Analytics
  • Apache-Airflow
  • Apache-Beam
  • Apache-Camel
  • Apache-Flink
  • Apache-Nifi
  • Apache-Spark
  • Api
  • Approx
  • Argo-Workflows
  • Automation
  • Awesome-Pipeline
  • Bash
  • Batch
  • Browsers
  • Cdap
  • Cdn
  • Celery
  • Certbot
  • Ci
  • Cleanup
  • Cli
  • Cloudflare
  • Cloudquery
  • Cms
  • Crawler
  • Cron
  • Curl
  • Dagster
  • Dask
  • Data
  • Data-Engineering
  • Database
  • Datasets
  • Dbt-Core
  • Debezium
  • Dig
  • Dlt
  • Dns
  • Domains
  • Downloads
  • Drupal
  • Elt
  • Encoding
  • Etl
  • File-Manipulation
  • File-Transfer
  • Git
  • Go
  • Go-Elasticsearch
  • Go-Micro
  • Go-Sqlite3
  • Golang
  • Goose
  • Gopherjs
  • Great-Expectations
  • Hamilton
  • Hashicorp
  • Headless-Browser
  • Http
  • Ingestr
  • Java
  • Jenkins
  • Json
  • Json-Schema
  • K8s
  • Kafka
  • Keywords
  • Ksql
  • Lftp
  • Linux
  • Meltano
  • Message-Queue
  • Mongodb
  • Monitoring
  • N8n
  • Networking
  • Nlp
  • Nokogiri
  • Nosql
  • Open-Data
  • Opensanctions
  • Pandas
  • Parquet
  • Pentaho-Kettle
  • Performance
  • Php
  • Polars
  • Prefect
  • Programming
  • Python
  • Redis
  • Redpanda-Connect
  • Reference
  • Release
  • Releases
  • Rest-Api
  • Rulego
  • Scheduler
  • Scrapy
  • Script
  • Security
  • Sftp
  • Spring-Batch
  • Sql
  • Sqlalchemy
  • Ssh
  • Ssl
  • Streaming
  • Temporal
  • Testing
  • Text
  • Text-Processing
  • Tld
  • Toil
  • Traffic
  • Troubleshooting
  • Unison
  • User-Agents
  • Varmq
  • Web
  • Web-Development
  • Web-Scraping
  • Wget
  • Wgpu
  • Wordpress
  • Workflow-Orchestration
  • Search

Scrapy Adds Verbatim URL Support and Refines GCS Feed Exports

Scrapy recently introduced support for verbatim URLs to handle non standard web servers and fixed a resource leak in Google Cloud Storage feed exports. These updates improve the robustness of data extraction pipelines when dealing with unconventional URL encoding and cloud storage backends.

Read full post
Jun 7, 2026 5 min read crawler data-engineering etl python scrapy web-scraping
Page 1 of 1
Atom Feed
Built with Hugo and