July 7, 2025

• MobileNet-v5 backbone tweaks for improved Google Gemma 3n behaviour (to pair with updated official weights)
  • Add stem bias (zero'd in updated weights, compat break with old weights)
  • GELU -> GELU (tanh approx). A minor change to be closer to JAX
• Add two arguments to layer-decay support, a min scale clamp and 'no optimization' scale threshold
• Add 'Fp32' LayerNorm, RMSNorm, SimpleNorm variants that can be enabled to force computation of norm in float32
• Some typing, argument cleanup for norm, norm+act layers done with above
• Support Naver ROPE-ViT (https://github.com/naver-ai/rope-vit) ineva.py, add RotaryEmbeddingMixed module for mixed mode, weights on HuggingFace Hub

• Some cleanup of ROPE modules, helpers, and FX tracing leaf registration
• Preparing version 1.0.17 release

What's Changed

• Adding Naver rope-vit compatibility to EVA ViT by@rwightman in#2529
• Update no_grad usage to inference_mode if possible by@GuillaumeErhard in#2534
• Add a min layer-decay scale clamp, and no optimization threshold to exclude groups from optimization by@rwightman in#2537
• Add stem_bias option to MNV5. Resolve the norm layer so can pass string. by@rwightman in#2538
• Add flag to enable float32 computation for normalization (norm + affine) by@rwightman in#2536
• fix: mnv5 conv_stem bias and GELU with approximate=tanh by@RyanMullins in#2533
• Fixup casting issues for weights/bias in fp32 norm layers by@rwightman in#2539
• Fix H, W ordering for xy indexing in ROPE by@rwightman in#2541
• Fix 3 typos in README.md by@robin-ede in#2544

New Contributors

• @GuillaumeErhard made their first contribution in#2534
• @RyanMullins made their first contribution in#2533
• @robin-ede made their first contribution in#2544

Full Changelog:v1.0.16...v1.0.17

Contributors