URL Scraper

A python script to scrape URLs from major search engines. Search dork is supported, dork multiple search engines at once.

Supported search engines

• Google
• Bing
• Yahoo
• DuckDuckGo

Requirements

• Python 3.10
• requests
• beautifulsoup4

Usage

1. Clone the repository.

2. Install the dependencies & activate virtual environemnt.

   $ pipenv install
   $ pipenv shell

3. Run the program.

   $ python scraper.py

4. Follow the prompt.

5. Obtain results from the output/ folder.

Note: Program is intentionally slowed (humanized) to prevent being locked-out from search engines.