Data Pipelines

AgentQL

Tiny Fish, Inc.

Browser automation and web data extraction using natural language queries — connects AI agents to any website without fragile CSS selectors.

Airbyte

Airbyte

Open-source ELT platform with 600+ connectors for moving data from any source into your warehouse, lakehouse, or AI agent's context layer.

Apify

Apify

Cloud platform for web scraping, browser automation, and structured data extraction at scale — with 46,000+ ready-made actors, managed proxy infrastructure, and MCP connectors for AI agent workflows.

Automatisch

Automatisch

Open-source workflow automation that connects 80+ apps. Self-host for unlimited executions, or use their hosted cloud from €20/month (task-metered).

Bright Data

Bright Data Ltd.

Web data platform combining a 400M+ IP proxy network, AI-ready scraper APIs, a no-code Scraper Studio, and ready-to-use datasets for large-scale data collection and AI agents.

Databricks

Databricks, Inc.

Unified data intelligence platform that merges data engineering, analytics, and AI development in one lakehouse environment built for enterprise scale.

Firecrawl

Firecrawl

API service that turns any website into clean, structured data — markdown or JSON — ready to feed directly into an LLM or RAG pipeline.

Hookdeck

Hookdeck Technologies Inc.

Event gateway that reliably receives, queues, retries, and monitors webhooks and event streams so nothing gets dropped under load.

Kestra

Kestra (Kestra SAS)

Open-source declarative workflow orchestration — YAML-defined pipelines with 1,600+ integrations, visual DAGs, and event-based triggers.

Make

Celonis, Inc

Visual automation platform for connecting apps and building complex multi-step workflows without code.

Apache NiFi

Open-source data-flow tool with a browser canvas for moving, routing, and transforming data between systems in real time, with full lineage tracking.

n8n

N8N

Fair-code workflow automation platform built for technical teams — visual editor, 1,000+ integrations (native + community), AI agent builder, and a code escape hatch when the drag-and-drop nodes aren't enough.

Node-RED

OpenJS Foundation

Open-source visual flow editor for wiring together hardware devices, APIs, and online services with drag-and-drop nodes.

Octoparse

Octopus Data Inc. 2025

No-code web scraper that turns any website into structured, exportable data via a point-and-click interface.

PhantomBuster

PhantomBuster

Cloud automation platform for scraping leads and data from social networks and websites using 100+ pre-built no-code scripts.

Pipedream

Pipedream

Workflow automation platform that connects APIs, databases, and AI services with a visual builder and optional custom code in Node.js, Python, or Go.

Windmill.dev

Windmill

Open-source workflow engine that turns Python and TypeScript scripts into internal apps, REST APIs, and scheduled jobs without the infrastructure overhead.