🔨 List all IP ranges from: Google (Cloud & GoogleBot), Bing (Bingbot), Amazon (AWS), Microsoft, Oracle (Cloud), GitHub, Facebook (Meta), OpenAI (GPTBot) and other with daily updates.

• UpdatedApr 25, 2025
• Shell

Simple robots.txt template. Keep unwanted robots out (disallow). White lists (allow) legitimate user-agents. Useful for all websites.

• UpdatedFeb 16, 2025

Updated lists of IP addresses/whitelists of good bots and crawlers. Includes GoogleBot, BingBot, DuckDuckBot, etc.

• UpdatedApr 26, 2025

This project is a Python bot that automates the process of logging into Gmail, joining a Google Meet, recording the audio of the meeting, and then generating a summary, key points, action items, and sentiment analysis of the meeting.

• UpdatedApr 2, 2025
• Python

Crawl, match, parse IP or IP range, check if IP or range is in another range. Support IPv4, IPv6, IP Interval, Wildcard and CIDR. Check if IP is Cloudflare node IP, Google bot IP. 爬取，正则匹配，解析 IP 和 IP 范围，检测 IP 或范围是否在另一个范围中。支持 IPv4，IPv6，区间、通配符或 CIDR 表示的 IP 范围。检测 IP 是否是 Cloudflare 节点或 Google 漫游器 IP

• UpdatedSep 16, 2022
• PHP

google for discord, yo

• UpdatedOct 24, 2017
• JavaScript

🤔 Is this Web request from a real search engine🕷 or from an impersonating agent 🕵️‍♀️?

• UpdatedApr 19, 2025
• Ruby

Check-list para reunir as principais tags a serem adicionadas na criação de páginas HTML para que os motores de busca façam a indexação do site de forma orgânica.

• UpdatedSep 12, 2023

Scraping Amazon website using Proxies for extracting Mobile details

• UpdatedMay 30, 2019
• Python

Django middleware to allow googlebot to index logged-in accessible content.

• UpdatedJun 22, 2022
• Python

Bot Untuk Mencari Sesuatu Di Google Via Telegram. Maintaned by Rio

• UpdatedOct 9, 2021
• JavaScript

Validate search engine user agents and IP addresses.

• UpdatedFeb 3, 2022
• Python

Shell Script for Monitoring GoogleBot Crawls via Nginx log file and Sending notification to Telegram

• UpdatedOct 11, 2021
• Shell

Super simple Python3 website URL scraper/crawler. Multi-threaded.

• UpdatedMay 16, 2022
• Python

Uses the common crawl data to find sites that disallow Googlebot for crawling

• UpdatedFeb 3, 2022
• Python

Play Framework filter: allow only legitimate search bots to access the site

• UpdatedDec 13, 2016
• Scala

WordPress plugin to redirect 404 URLs to a specified URL.

• UpdatedJun 6, 2024
• PHP

Prerenders your website into a static, SEO friendly, HTML version.

• UpdatedJan 1, 2020
• JavaScript

Playstore auto ratings using selenium with python on localhost

• UpdatedDec 25, 2023
• Python

TDMRep: TDM Reservation Protocol integration for WordPress

• UpdatedMay 23, 2024
• PHP