• Added PyTorchBigBird-Pegasus to Transformers
• Added PyTorchBigBird-RoBERTa to Transformers
• Added Flax/JaxBigBird-RoBERTa to Transformers
• Added script for trainingFlaxBigBird on natural questions to Transformers
• IntegratedMicrosoft's DeepSpeed with Accelerate
• AddedModelHubMixin in Hub

Open Source Contributions to TensorFlow (Google):

• Exported fine-tuned Wav2Vec2 model to TFHub
• Exported pre-trained Wav2Vec2 model to TFHub
• Added notebook for demonstrating Wav2Vec2 fine-tuning to TFHub
• Implemented & trained Wav2Vec2 model in TensorFlow

Blogs/Posters:

• Optimizing adapters for NMT
• Understanding BigBird's Block Sparse Attention
• BioBigBird: Leveraging Entire Articles for Biomedical Language Understanding

Talks/Lectures:

• BigBird: Transformers on long sequences
• Deep Dive into Pre-trained Transformers
• CUDA Concepts YouTube Series

Tutorials:

• Flax BigBird evaluation on natural-questions dataset
• PyTorch BigBird evaluation on natural-questions dataset
• PyTorch BigBirdPegasus evaluation on PubMed dataset
• How to use BigBird (RoBERTa & Pegasus) for inference
• Template for fine-tuning a pre-trained Wav2Vec2 SavedModel
• Conversion of TF Wav2Vec2 model to ONNX and compares the latency of ONNX exported model & TF model on CPU
• Wav2Vec2 evaluation (without any padding) on LibriSpeech data
• Wav2Vec2 SavedModel evaluation (with constant padding upto 246000 length) on LibriSpeech data
• Small demo of how to use Wav2Vec2 for inference for ASR task

Please visit my webpage:https://thevasudevgupta.com/ to know more about my work.

PinnedLoading

1. bigbirdbigbirdPublic

   Google's BigBird (Jax/Flax & PyTorch) @ 🤗Transformers

   Jupyter Notebook 48 9

2. gsoc-wav2vec2gsoc-wav2vec2Public

   GSoC'2021 | TensorFlow implementation of Wav2Vec2

   Jupyter Notebook 91 29

3. speech-jaxspeech-jaxPublic

   Speech in Flax/JAX

   Python 15 2

4. transformers-adapterstransformers-adaptersPublic

   This repositary hosts my experiments for the project, I did with OffNote Labs.

   Python 10 3

5. biobigbirdbiobigbirdPublic

   BigBird for bio-medical domain

   Python 1 1

6. gpt-tritongpt-tritonPublic

   Triton implementation of GPT/LLAMA

   Python 16 2