In-memory Bloom filter for URL and document hash deduplication preventing redundant web crawler ingestion
Setup
This project documents no single launch command; follow its repository instructions.
GenPark Bloom Filter URL Dedup Crawler
alphaparkinc
Related
More- KKiwi.comunknownOfficialOfficial Kiwi.com flight search MCP server. Search and book flights directly from your favorite AI assistant.Search & Web Data
- NNiche: Your niche, followed for younicheangleOfficialRead niche briefings, research news, and explore sourced story angles with your assistant.Search & Web DataRemote only
- SScrapling MCP ServerD4Vinci🕷️ A Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_devSearch & Web Data89.1k Repository stars
- MMetabasemetabaseLets AI clients search, explore, query, and visualize data in a Metabase instance.Search & Web Data49.8k Repository stars
- Smcp-server-searchbytedanceChinaMCP server for web search operationsSearch & Web Data39.4k Repository stars
- AAgent Skills Search ServercomSpecification and documentation for Agent SkillsSearch & Web DataRemote only26.4k Repository stars