Web crawler for agents that fetches real-time web data and searches private knowledge bases.
Setup
This project documents no single launch command; follow its repository instructions.
WebDataSource
WebDataSourceOfficial
Related
More- WMCP Server WebcrawlpragmarSearches and retrieves web crawler data and archives from WARC, wget, Katana, SiteOne and InterroBot crawls.Search & Web Data46 Repository stars
- FFirecrawl MCP ServerFirecrawlOfficialFirecrawl's official server for web search, scraping and crawling that returns clean Markdown or structured data to agents.Search & Web DataLocal + remote7.7k Repository stars
- EExa MCP ServerExaOfficialExa's official hosted server for web search, content fetching and multi-step research; anonymous use is rate limited.Search & Web DataLocal + remote5.2k Repository stars
- TTavily MCP ServerTavilyOfficialTavily's official server offering real-time web search, page content extraction, site mapping and crawling tools.Search & Web DataLocal + remote2.5k Repository stars
- BBrave Search MCP ServerBraveOfficialBrave's official server for the Brave Search API, covering web, local business, image, video and news search plus AI summarization.Search & Web DataLocal + remote1.5k Repository stars
- BBocha Search MCP Server博查 (BochaAI)OfficialChinaOfficial Bocha MCP server offering web search and AI search with structured modal cards; run it from a cloned repository with a Bocha API key.Search & Web Data185 Repository stars