Paulpierre / Markdown Crawler
Description
A multithreaded 🕸️ web crawler that recursively crawls a website and creates a 🔽 markdown file for each page, designed for LLM RAG
Technical Specifications
| Core Language | |
| GitHub Authority | ⭐ 445 stars |
| Last Code Push | 2024-08-13 |
| Open Issues / Bugs | 🛠️ 14 bugs listed |
| License Type | Open-Source (Free to use) |
Get Source Code
This project is open-source and hosted on GitHub. Click below to explore the repository, deployment guides, or fork the code.
Go to Repository →