- Notifications
You must be signed in to change notification settings - Fork7
Ultra-low resource 4chan/altchan thread and board archiver
License
bbepis/Hayden
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Hayden is a 4chan / altchan archiver written in .NET Core for ultra-low resource usage and high performance.
It also doubles as imageboard software.
It was originally writen as a drop-in alternative toAsagi, however Asagi compatibility currently has no guarantees.
Developer documentation is inARCHITECTURE.md
.
README for running the scraper
README for running the webserver / imageboard
Software | API-less fallback | Example sites |
---|---|---|
Yotsuba | ❌ | 4chan.org |
LynxChan | ❌ | 8chan.moe endchan.org |
Vichan/Infinity (not OpenIB/8kun) | ❌ | sportschan.org smuglo.li |
InfinityNext | ❌ | 9chan.tw |
Meguca / shamichan | ✅ | 2chen.moe shamik.ooo |
FoolFuuka | ❌ | desuarchive.org |
Ponychan | ❌ | ponychan.net |
ASPNetChan | ✅ | mlpol.net |
Much smaller memory consumption than Asagi.
- For comparison, Hayden requires roughly 40MB of working memory to archive a single board (including all archived threads), while Asagi consumes several gigabytes to do the same.
Uses a much more efficient algorithm to perform API calls, reducing overall network calls made considerably and eliminates cloudflare rate limit issues.
Supports using multiple SOCKS proxies to distribute network load and allow parallel network operations.
Supports writing to multiple types of data stores.
- Thread ID-based scraping system. Currently the only logic for thread archival operates on a per-board basis
I wanted to archive 4chan threads and display them, but didn't like what was already offered. As per usual, this turned into making a very large project to do so.
Why C#? Doesn't it have a GC like Java and other managed languages, and consume a lot of memory as a result?
Yeah sure. Maybe you could get better performance / memory usage out of something like Rust.
But you could also just not be wasteful and be considerate of how you structure your data, and achieve very similar results?
I was listening to theDoom 2016 soundtrack as I was programming this.
I was not a fan of Eternal.
About
Ultra-low resource 4chan/altchan thread and board archiver