Movatterモバイル変換

NotificationsYou must be signed in to change notification settings
Fork225
Star1.7k

The undetected self-hosted browser automation platform. Powered by Camoufox (Firefox) for 0% detection rates. Built for speed, privacy, and scalability.

headlessx.saify.me

License

MIT license

1.7k stars 225 forks Branches Tags Activity

Star

Notifications

You must be signed in to change notification settings

Branches Tags

Folders and files

Name		Name	Last commit message	Last commit date
Latest commit History 116 Commits
.clinic		.clinic
.github		.github
assets		assets
backend		backend
frontend		frontend
scripts		scripts
.env.example		.env.example
.eslintignore		.eslintignore
.gitignore		.gitignore
.npmrc		.npmrc
.prettierrc.json		.prettierrc.json
.semgrepignore		.semgrepignore
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
SECURITY.md		SECURITY.md
audit-results.json		audit-results.json
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml

Repository files navigation

The World's Most Advanced Anti-Detection Scraping Full Stack Web App

🎯0% Detection Rate on all major anti-bot systems

Powered byCamoufox — Firefox with C++ level fingerprint spoofing

📖 Documentation •🚀 Quick Start •🐛 Report Bug •✨ Request Feature

🆕 What's New in V2.0

🦊Camoufox Engine	🛡️0% Detection	⚡3x Faster
Firefox with binary-level stealth patches	Passes CreepJS, Sannysoft, & all major tests	Optimized context pooling

V2.0 Changelog

+ 🦊 NEW: Camoufox browser engine (replaces Chromium + stealth plugins)+ 🛡️ NEW: 0% headless detection rate (was 67%+)+ 🔒 NEW: Built-in WebRTC leak protection+ 🎭 NEW: C++ level Canvas/WebGL/AudioContext fingerprint spoofing+ ⚡ IMPROVED: Faster browser launches with persistent contexts- 🧹 REMOVED: playwright-extra (no longer needed)- 🧹 REMOVED: puppeteer-extra-plugin-stealth (no longer needed)

📊 Detection Benchmark Comparison

Tool	Headless Detection	Stealth Score	Engine
🦊 HeadlessX V2.0 (Camoufox)	✅0%	✅0%	Firefox (C++ patched)
HeadlessX V1 (Playwright Extra)	❌ 67%	⚠️ 50%	Chromium (JS patches)
Puppeteer Stealth	❌ 33%	❌ 80%	Chromium (JS patches)
Vanilla Playwright	❌ 100%	❌ 100%	Chromium
Vanilla Selenium	❌ 100%	❌ 100%	Chrome

Lower scores = better (0% = undetectable)

🤖 Scrapers

Google SERP	🌐Website Scraping
Extract search results with zero detection	Scrape any website with full JS rendering
✅Live	✅Live

Coming Soon

Amazon	LinkedIn	Twitter	Instagram
Product & Reviews	Job Listings & Profiles	Tweets & Trends	Posts & Stories

🖥️ UI Screenshots

Google SERP Scraper

Website Scraper

📸 Proof of Undetectability

BrowserScan (100% Passed)

Pixelscan (Human Detection)	Profile Tested with Proxy

📋 Table of Contents

⚡ Quick Start

Prerequisites:
Node.js 18+ (v22+ recommended)
pnpm 9+ (npm install -g pnpm)
PostgreSQL Database — either:
Supabase (recommended, free tier available)
Self-hosted PostgreSQL 14+

1️⃣ Clone & Configure Environment

git clone https://github.com/your-repo/HeadlessX.gitcd HeadlessX# Copy environment templatecp .env.example backend/.env

Editbackend/.env with your database connection:

# REQUIRED: Your PostgreSQL connection stringDATABASE_URL="postgresql://user:password@host:5432/database"# Optional: Server configurationPORT=3001NODE_ENV=development

💡Supabase Users: Find your connection string at:
Dashboard → Settings → Database → Connection String (URI)

2️⃣ Install Dependencies

# Install all packages (workspace: root, backend, frontend)pnpm install

⚠️ If you see a Prisma warning during install, that's normal!
It means you need to configureDATABASE_URL first, then runpnpm db:push.

3️⃣ Install AI Models (CAPTCHA Solving)

Windows	Linux / macOS
install.bat	chmod +x install.sh./install.sh

4️⃣ Setup Database

# Push Prisma schema to your databasepnpm db:push# Or use migrations for productionpnpm db:migrate

5️⃣ Start Development Server

# Start both frontend and backend concurrentlypnpm dev

Or start individually:

Windows	Linux / macOS
scripts\start.bat	./scripts/start.sh

6️⃣ Access the Application

Service	URL	Notes
🖥️Dashboard	http://localhost:3000	Frontend UI
🔗Backend API	http://localhost:3001	REST API

Custom Ports

You can customize ports via environment variables:

# Backend (from root)cross-env PORT=4001 pnpm dev:backend# Frontend (Linux/macOS)PORT=4000 pnpm --filter headlessx-frontend dev

🔥 Key Features

🦊 Camoufox Stealth Engine (V2.0)

Feature	Description
🔒Binary-Level Patches	Firefox modified at C++ level
🎭Fingerprint Spoofing	Canvas, WebGL, AudioContext
🌐WebRTC Protection	No IP leaks in headless mode
🖱️Human Behavior	Natural mouse movements & scrolling
📱Device Emulation	Realistic viewport & user agents
🚫Zero Detection	Passes all anti-bot tests

🖥️ Modern Dashboard

Feature	Description
⚛️Next.js 16	React 19 with Turbopack
🎨HeroUI	Beautiful dark/light mode UI
⚙️Live Config	Change settings without restart
📊Request Logs	Full history with screenshots
🧪Playground	Test scraping in real-time
🔑API Keys	Secure key management

🌐 API Endpoints

Website Scraping APIs

Endpoint	Method	Description
`/api/website/html`	`POST`	Get raw HTML (fast)
`/api/website/html-js`	`POST`	Get HTML with JS rendering
`/api/website/content`	`POST`	Get Markdown content
`/api/website/screenshot`	`POST`	Capture full-page Screenshot
`/api/website/stream`	`POST`	Real-time SSE Stream

Google SERP APIs

Endpoint	Method	Description
`/api/google-serp/search`	`POST`	Extract Google search results
`/api/google-serp/stream`	`GET`	Real-time SSE Stream

Example Request

curl -X POST http://localhost:3001/api/website/html \  -H"Content-Type: application/json" \  -H"X-API-Key: your-api-key" \  -d'{"url": "https://example.com", "stealth": true}'

Example Response

{"success":true,"data": {"url":"https://example.com","title":"Example Domain","html":"<!DOCTYPE html>...","statusCode":200  }}

Configuration

Environment Variables

⚠️Note: Most configuration (Browsers, Stealth, Proxy, Timeouts) is managed directly via theDashboard Settings.

Only the following core variables are required in.env:

Variable	Default	Description
`PORT`	`3001`	Backend API port
`DATABASE_URL`	-	PostgreSQL connection (Supabase or self-hosted)
`NEXT_PUBLIC_API_URL`	`http://localhost:3001`	Frontend API URL

Dashboard Settings

Configure these live at/settings:

General Configuration

Headless Mode: Run browser without UI (faster execution)
Browser Timeout: Max execution time per job (default: 60000ms)
Max Concurrent Jobs: Concurrent scraping jobs (recommended: 3-8)

Browser Engine (Camoufox)

Block WebRTC: Prevent IP leaks via WebRTC
Camoufox GeoIP: Spoof location based on IP
Enable Cache: Cache resources for speed

Proxies

Manage proxy configurations and rotation settings

🛠️ Manual Setup

Backend

cd backend# 1. Configure environmentcp ../.env.example .env# Edit .env with your DATABASE_URL# 2. Install dependenciespnpm install# 3. Download Camoufox browsernpx camoufox-js fetch# 4. Setup databasenpx prisma db push# 5. Start dev serverpnpm dev

Frontend

cd frontendpnpm installpnpm dev

🤝 Contributing

Contributions are welcome! Please read ourContributing Guide for details.

Fork the repository
Create your feature branch (git checkout -b feature/amazing-feature)
Commit your changes (git commit -m 'Add amazing feature')
Push to the branch (git push origin feature/amazing-feature)
Open a Pull Request

📝 License

This project is licensed under theMIT License - see theLICENSE file for details.

🌟 Star History

If you find HeadlessX useful, please consider giving it a star! ⭐

Built with ❤️ using cutting-edge technologies

🦊 HeadlessX V2.0 — Undetectable by Design

About

The undetected self-hosted browser automation platform. Powered by Camoufox (Firefox) for 0% detection rates. Built for speed, privacy, and scalability.

headlessx.saify.me