moeru-ai/airiPublic

NotificationsYou must be signed in to change notification settings
Fork14
Star131

💖 アイリ, ultimate Neuro-sama like LLM powered Live2D/VRM living character life pod, near by you.

License

MIT license

131 stars 14 forks Branches Tags Activity

Star

Notifications

You must be signed in to change notification settings

Branches Tags

Folders and files

Name		Name	Last commit message	Last commit date
Latest commit History 534 Commits
.github		.github
.vscode		.vscode
apps		apps
docs		docs
packages		packages
scripts		scripts
services		services
.editorconfig		.editorconfig
.gitignore		.gitignore
.npmrc		.npmrc
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
README.zh-CN.md		README.zh-CN.md
bump.config.ts		bump.config.ts
changelogithub.config.ts		changelogithub.config.ts
cspell.config.yaml		cspell.config.yaml
eslint.config.js		eslint.config.js
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
tsconfig.json		tsconfig.json
uno.config.ts		uno.config.ts
vitest.workspace.ts		vitest.workspace.ts

Repository files navigation

アイリ VTuber

[Try it] [简体中文文档]

Heavily inspired byNeuro-sama

Unlike the other AI driven VTuber open source projects, アイリ VTuber was built with many support of Web technologies such asWebGPU,WebAudio,Web Workers,WebAssembly,WebSocket, etc. from the first day.

This means thatアイリ VTuber is capable to run on modern browsers and devices, and even on mobile devices (already done with PWA support), this brought a lot of possibilities for us (the developers) to build and extend the power of アイリ VTuber to the next level, while still left the flexibilities for users to enable features that requires TCP connections or other non-Web technologies such as connect to voice channel to Discord, or playing Minecraft, Factorio with you and your friends.

Note

We are still in the early stage of development where we are seeking out talented developers to join us and help us to make アイリ VTuber a reality.

It's ok if you are not familiar with Vue.js, TypeScript, and devtools that required for this project, you can join us as an artist, designer, or even help us to launch our first live stream.

Even you are a big fan of React or Svelte, even Solid, we welcome you, you can open a sub-directory to add features that you want to see in アイリ VTuber, or would like to experiment with.

Fields (and related projects) that we are looking for:

Live2D modeller
VRM modeller
VRChat avatar designer
Computer Vision
Reinforcement Learning
Speech Recognition
Speech Synthesis
ONNX Runtime
Transformers.js
vLLM
WebGPU
Three.js
WebXR (checkout the another project we have under @moeru-ai organization)

If you are interested in, why not introduce yourself here?Would like to join part of us to build Airi?

Current progress

Capable of

Brain
- PlayMinecraft
- PlayFactorio (WIP, butPoC and demo available)
- Chat inTelegram
- Chat inDiscord
- Memory
  - Pure in-browser database support (DuckDB WASM |pglite)
  - Memory Alaya (WIP)
- Pure in-browser local (WebGPU) inference
Ears
- Audio input from browser
- Audio input fromDiscord
- Client side speech recognition
- Client side talking detection
Mouth
- ElevenLabs voice synthesis
Body
- VRM support
  - Control VRM model
- VRM model animations
  - Auto blink
  - Auto look at
  - Idle eye movement
- Live2D support
  - Control Live2D model
- Live2D model animations
  - Auto blink
  - Auto look at
  - Idle eye movement

Development

pnpm i

pnpm dev

Supported the following LLM API Providers (powered byxsai)

OpenRouter
vLLM
SGLang
Ollama
Google Gemini
OpenAI
- Azure OpenAI API (PR welcome)
Anthropic Claude
- AWS Claude (PR welcome)
DeepSeek
Qwen
xAI
Groq
Mistral
Cloudflare Workers AI
Together.ai
Fireworks.ai
Novita
Zhipu
SiliconFlow
Stepfun
Baichuan
Minimax
Moonshot AI
Tencent Cloud
Sparks (PR welcome)
Volcano Engine (PR welcome)

Sub-projects born from this project

unspeech: Universal endpoint proxy server for/audio/transcriptions and/audio/speech, like LiteLLM but for any ASR and TTS
hfup: tools to help on deploying, bundling to HuggingFace Spaces
@proj-airi/drizzle-duckdb-wasm: Drizzle ORM driver for DuckDB WASM
@proj-airi/duckdb-wasm: Easy to use wrapper for@duckdb/duckdb-wasm
@proj-airi/lobe-icons: Iconify JSON bundle for amazing AI & LLM icons fromlobe-icons, support Tailwind and UnoCSS
@proj-airi/elevenlabs: TypeScript definitions for ElevenLabs API
Airi Factorio: Allow Airi to play Factorio
Factorio RCON API: RESTful API wrapper for Factorio headless server console
autorio: Factorio automation library
`tstl-plugin-reload-factorio-mod: Reload Factorio mod when developing
🥺 SAD: Documentation and notes for self-host and browser running LLMs

%%{ init: { 'flowchart': { 'curve': 'catmullRom' } } }%%flowchart TD  Core("Core")  Unspeech["unspeech"]  DBDriver["@proj-airi/drizzle-duckdb-wasm"]  MemoryDriver["[WIP] Memory Alaya"]  DB1["@proj-airi/duckdb-wasm"]  ICONS["@proj-airi/lobe-icons"]  UI("@proj-airi/stage-ui")  Stage("Stage")  F_AGENT("Factorio Agent")  F_API["Factorio RCON API"]  F_MOD1["autorio"]  SVRT["@proj-airi/server-runtime"]  MC_AGENT("Minecraft Agent")  XSAI["xsai"]  subgraph Airi    DB1 --> DBDriver --> MemoryDriver --> Memory --> Core    ICONS --> UI --> Stage --> Core    Core --> STT    Core --> SVRT  end  STT --> |Speaking|Unspeech  SVRT --> |Playing Factorio|F_AGENT  SVRT --> |Playing Minecraft|MC_AGENT  subgraph Factorio Agent    F_AGENT --> F_API -..- factorio-server    subgraph factorio-server-wrapper      subgraph factorio-server        F_MOD1      end    end  end  subgraph Minecraft Agent    MC_AGENT --> Mineflayer -..- minecraft-server    subgraph factorio-server-wrapper      subgraph factorio-server        F_MOD1      end    end  end  XSAI --> Core  XSAI --> F_AGENT  XSAI --> MC_AGENT

%%{ init: { 'flowchart': { 'curve': 'catmullRom' } } }%%flowchart TD  subgraph deploy&bundle    direction LR    HFUP["hfup"]    HF[/"HuggingFace Spaces"\]    HFUP -...- UI -...-> HF    HFUP -...- whisper-webgpu -...-> HF    HFUP -...- moonshine-web -...-> HF  end