Decaf Blog In-depth updates on Decaf software, tooling, and open-source experiments
  • Aggregation
  • Airbyte
  • Analytics
  • Apache-Airflow
  • Apache-Beam
  • Apache-Camel
  • Apache-Flink
  • Apache-Nifi
  • Apache-Spark
  • Api
  • Approx
  • Argo-Workflows
  • Automation
  • Awesome-Pipeline
  • Bash
  • Batch
  • Browsers
  • Cdap
  • Cdn
  • Celery
  • Certbot
  • Ci
  • Cleanup
  • Cli
  • Cloudflare
  • Cloudquery
  • Cms
  • Crawler
  • Cron
  • Curl
  • Dagster
  • Dask
  • Data
  • Data-Engineering
  • Database
  • Datasets
  • Dbt-Core
  • Debezium
  • Dig
  • Dlt
  • Dns
  • Domains
  • Downloads
  • Drupal
  • Elt
  • Encoding
  • Etl
  • File-Manipulation
  • File-Transfer
  • Git
  • Go
  • Go-Elasticsearch
  • Go-Micro
  • Go-Sqlite3
  • Golang
  • Goose
  • Gopherjs
  • Great-Expectations
  • Hamilton
  • Hashicorp
  • Headless-Browser
  • Http
  • Ingestr
  • Java
  • Jenkins
  • Json
  • Json-Schema
  • K8s
  • Kafka
  • Keywords
  • Ksql
  • Lftp
  • Linux
  • Meltano
  • Message-Queue
  • Mongodb
  • Monitoring
  • N8n
  • Networking
  • Nlp
  • Nokogiri
  • Nosql
  • Open-Data
  • Opensanctions
  • Pandas
  • Parquet
  • Pentaho-Kettle
  • Performance
  • Php
  • Polars
  • Prefect
  • Programming
  • Python
  • Redis
  • Redpanda-Connect
  • Reference
  • Release
  • Releases
  • Rest-Api
  • Rulego
  • Scheduler
  • Scrapy
  • Script
  • Security
  • Sftp
  • Spring-Batch
  • Sql
  • Sqlalchemy
  • Ssh
  • Ssl
  • Streaming
  • Temporal
  • Testing
  • Text
  • Text-Processing
  • Tld
  • Toil
  • Traffic
  • Troubleshooting
  • Unison
  • User-Agents
  • Varmq
  • Web
  • Web-Development
  • Web-Scraping
  • Wget
  • Wgpu
  • Wordpress
  • Workflow-Orchestration
  • Search

OpenSanctions Refines PEP Data Pipelines and Crawler Reliability

OpenSanctions recently released a large update to its data processing engine and PEP dataset collection. This week saw over 130 commits focusing on structural refactors to the occupancy data model and significant hardening of crawler reliability against source web application firewalls.

Read full post
Jun 9, 2026 4 min read data-engineering datasets etl open-data opensanctions python
Page 1 of 1
Atom Feed
Built with Hugo and