SpideyX a multipurpose Web Penetration Testing tool with asynchronous concurrent performance with multiple mode and configurations.
-
Updated
Mar 18, 2025 - Python
#
crawler-engine
Here are 14 public repositories matching this topic...
Powerful Telegram bot for web scraping and crawling. Fast, easy, and loved by thousands!
crawler scraper telegram telegram-bot scraping selenium requests web-scraping webscraping hacktoberfest webscrapping beautifulsoup4 webscrapper crawler-engine pyrogram crawler-python webscrapping-python pyrogram-bot hacktoberfest-accepted hacktoberfest2023
-
Updated
Mar 19, 2026 - Python
Generic Interfaces to Addressable Objects
framework driver proxies iterators sessions filters protocols generators formats controller-manager crawler-engine
-
Updated
Oct 31, 2024 - Python
a fast and open source search engine
python search-engine flask crawler sqlite sqlite-database python3 flask-application sqlite3 persian flask-web flask-backend flask-server search-engine-optimization crawler-engine crawler-python search-engine-indexing
-
Updated
Jul 28, 2026 - Python
Research-grade focused crawler with event-driven pipelines, selective NLP/LLM scoring, and live crawl observability.
nlp information-retrieval etl reactjs sqlite aiohttp embeddings event-driven asynchronous-programming lxml d3-visualization crawler-engine huggingface llm aiengineering openrouter-api
-
Updated
Oct 2, 2026 - Python
A data gathering/trawling framework to search and get information from web sources like bing
-
Updated
Apr 13, 2026 - Python
Type a query → collect company emails/phones → export to Excel/CSV. SerpAPI-powered, polite crawler. Tiny GUI + CLI.
-
Updated
Dec 15, 2025 - Python
⚡ 《计算机网络与分布式》实战：基于 asyncio 的通用高并发异步任务调度引擎与平滑令牌桶限流器
-
Updated
Sep 3, 2026 - Python
⛔️ DEPRECATED. Please use LaSSI instead (see url below)
-
Updated
Sep 18, 2024 - Python
A fully functional, educational search engine demonstrating core concepts of web crawling, inverted indexing, and relevance-based ranking—built entirely with native Python libraries
-
Updated
Apr 21, 2026 - Python
Plan to use https://github.com/sxoxgxi/WebCrawler from the web
-
Updated
Jul 17, 2022 - Python
Functionality to Extract Social data.
-
Updated
Oct 28, 2022 - Python
🕷️ Crawl and mirror modern SPAs with optional login, capturing dynamic content and static assets for offline static site serving.
python laravel eloquent cache scraping crawling caddy php-crawler typer crawler-engine wx-code wx-be wx-pentest spatie-crawler cendertron crawler-cendertron laravel-crawler crawlee site-mirroring
-
Updated
Oct 3, 2026 - Python
Add this topic to your repo
To associate your repository with the crawler-engine topic, visit your repo's landing page and select "manage topics."