Movatterモバイル変換

dtunai/Ring-Attention-JaxPublic

generated fromdtunai/Python-Template

NotificationsYou must be signed in to change notification settings
Fork0
Star1

Packaged Ring Attention with Blockwise Transformers for Near-Infinite Context implemented in Jax + Flax.

License

Apache-2.0 license

1 star 0 forks Branches Tags Activity

Star

Notifications

You must be signed in to change notification settings

Branches Tags

Folders and files

Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
.github		.github
docs		docs
example		example
gitops		gitops
ring_attention_jax		ring_attention_jax
scripts		scripts
tests		tests
.dockerignore		.dockerignore
.gitignore		.gitignore
.pre-commit-config.yaml		.pre-commit-config.yaml
.readthedocs.yaml		.readthedocs.yaml
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
Makefile		Makefile
README.md		README.md
pyproject.toml		pyproject.toml
setup-requirements.txt		setup-requirements.txt
setup.py		setup.py

Repository files navigation

Ring Attention Sharded Jax

Packaged version of standard ring attention mechanism using JAX used inLWM.

About

Packaged Ring Attention with Blockwise Transformers for Near-Infinite Context implemented in Jax + Flax.

Releases

No releases published

Packages

No packages published

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

License

Folders and files

Latest commit

History

Repository files navigation

Ring Attention Sharded Jax

About

Topics

Resources

License

Stars

Watchers

Forks

Releases

Packages

Languages

Movatterモバイル変換

License

dtunai/Ring-Attention-Jax

Folders and files

Latest commit

History

Repository files navigation

Ring Attention Sharded Jax

About

Topics

Resources

License

Stars

Watchers

Forks

Releases

Packages0

Languages

Packages