Crawlab12.3k

crawlab-team/crawlab

Open sourceFreeAutomationSelf-hosted

What is Crawlab?

Distributed web crawler admin platform for managing spiders written in any language It has 12,295 stars on GitHub and ranks #48 in Automation. You can host it on your own server and keep your data.

GitHub description: Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

Platforms

WebDocker

Top alternatives for Crawlab

#220

EasySpider44.8k

Visual no code web crawler for extracting data from websites.

AutomationWindowsMacLinux

#863

Maxun18.2k

No-code tool that turns any website into a structured API and monitors the web.

AutomationSelf-hostedDocker

More Automation

n8n209.6k

Workflow automation platform with a visual editor and native AI features for connecting apps.

Self-hostedWebDocker

Firecrawl199k

Web scraping service that turns websites into clean text and structured data for AI.

Self-hostedWebDockerCommand line

Home Assistant92k

Open source home automation platform focused on local control and privacy.

Self-hostedWebDocker

MediaCrawler70.3k

Scraper that collects posts and comments from Xiaohongshu, Douyin and Kuaishou.

Command line

Freqtrade56.5k

Free crypto trading bot for building, backtesting and running strategies.

Self-hostedDockerCommand line

Huginn50.3k

Build agents that monitor the web and act on your behalf, such as sending alerts when pages change.

Self-hostedWebDocker

RSSHub46.8k

Self hosted tool that turns almost any site into an RSS feed.

Self-hostedWebDocker

Agent Browser46k

Browser automation CLI built for AI agents, letting them open pages, click elements and read content.

Command line