Movatterモバイル変換

Skip to content

#

rmsnorm

Here are 6 public repositories matching this topic...

Language:All

Filter by language

All6 Python4 Jupyter Notebook2

bzhangGo /rmsnorm

Root Mean Square Layer Normalization

layernorm rmsnorm

UpdatedMar 28, 2023
Python

knotgrass /Griffin

Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models

h3 linear attention language-model griffin mamba gelu conv1d rmsnorm rg-lru shift-ssm

UpdatedDec 23, 2024
Python

dtunai /Tri-RMSNorm

Efficient kernel for RMS normalization with fused operations, includes both forward and backward passes, compatibility with PyTorch.

machine-learning ai triton rmsnorm

UpdatedJun 5, 2024
Python

s-chh /PyTorch-Scratch-LLM

Simple and easy to understand PyTorch implementation of Large Language Model (LLM) GPT and LLAMA from scratch with detailed steps. Implemented: Byte-Pair Tokenizer, Rotational Positional Embedding (RoPe), SwishGLU, RMSNorm, Mixture of Experts (MOE). Tested on Taylor Swift song lyrics dataset.

moe mixture-of-experts kv-cache llm rmsnorm swiglu pytorch-llm byte-pair-tokenizer rotational-positional-embedding

UpdatedNov 18, 2024
Python

rmgogogo /nano-aigc

Generative models nano version for fun. No STOA here, nano first.

gan vae llama vit clip bert gemma rope diffusion variational-autoencoder conditional-vae vq-vae gpt2 huggingface ddim latent-diffusion rmsnorm llama2 multiqueryattention conditional-diffusion

UpdatedMay 20, 2024
Jupyter Notebook

sushantkumar23 /nano-gpt

Simple character level Transformer

transformers pytorch attention attention-mechanism rope self-attention multi-head-attention shakespeare-dataset transformer-architecture llm rmsnorm

UpdatedMay 27, 2024
Jupyter Notebook

Improve this page

Add a description, image, and links to thermsnorm topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with thermsnorm topic, visit your repo's landing page and select "manage topics."

[8]ページ先頭

©2009-2025 Movatter.jp