Crawls websites into clean Markdown for LLMs and vector stores, with page search and structured data extraction.
Setup
This project documents no single launch command; follow its repository instructions.
MarkCrawl
AIMLPM
Related
More- AApify MCP ServerApifyOfficialLets agents run ready-made Apify Actors to extract data from social media, search engines, maps, e-commerce sites and other websites.Search & Web DataLocal + remote12.5k Repository stars
- FfastCRWfastCRWSelf-hostable web scraper, crawler and search API in Rust with a Firecrawl-compatible API, returning Markdown or structured JSON.Search & Web DataLocal + remote1.2k Repository stars
- SSiteMCPryoppippiFetches an entire website and serves its content as an MCP server.Search & Web Data760 Repository stars
- WWeb Scout MCP Serverpinkpixel-devAn MCP server providing web search and content extraction capabilities. Integrates DuckDuckGo search functionality and URL content extraction into your MCP environment, enabling AI assistants to search the web and extract webpage content.Search & Web DataLocal + remote138 Repository stars
- DDoc ScraperSriram-PRCrawls documentation sites into local Markdown or JSONL and serves them with offline BM25 search, page reads and crawl diffs.Search & Web Data107 Repository stars
- 微微信公众号文章 MCPditingdapengChina让智能体抓取并分析微信公众号文章内容，基于 FastMCP 构建。Search & Web Data45 Repository stars