Note

Gemma 3n Conversational Fine tuning 2B on a Free Colab Notebook:

Gemma 3n Conversational Fine tuning 4B on a Free Colab Notebook:

Gemma 3n Multimodal Finetuning 2B/4B on a Free Colab Notebook:

Multimodal inference using Gemma 3n via pipeline:

Getting Started

$ pip install -U -q transformers timm

Inference with pipeline

importtorchfromtransformersimportpipelinepipe=pipeline("image-text-to-text",model="google/gemma-3n-E4B-it",# "google/gemma-3n-E4B-it"device="cuda",torch_dtype=torch.bfloat16)messages= [   {"role":"user","content": [           {"type":"image","url":"https://huggingface.co/datasets/ariG23498/demo-data/resolve/main/airplane.jpg"},           {"type":"text","text":"Describe this image"}       ]   }]output=pipe(text=messages,max_new_tokens=32)print(output[0]["generated_text"][-1]["content"])

Detailed inference with transformers

fromtransformersimportAutoProcessor,AutoModelForImageTextToTextimporttorchmodel_id="google/gemma-3n-e4b-it"# google/gemma-3n-e2b-itprocessor=AutoProcessor.from_pretrained(model_id)model=AutoModelForImageTextToText.from_pretrained(model_id).to(device)defmodel_generation(model,messages):inputs=processor.apply_chat_template(messages,add_generation_prompt=True,tokenize=True,return_dict=True,return_tensors="pt",    )input_len=inputs["input_ids"].shape[-1]inputs=inputs.to(model.device,dtype=model.dtype)withtorch.inference_mode():generation=model.generate(**inputs,max_new_tokens=32,disable_compile=False)generation=generation[:,input_len:]decoded=processor.batch_decode(generation,skip_special_tokens=True)print(decoded[0])

Text only

# Text Onlymessages= [    {"role":"user","content": [            {"type":"text","text":"What is the capital of France?"}        ]    }]model_generation(model,messages)

Interleaved with Audio

# Interleaved with Audiomessages= [    {"role":"user","content": [            {"type":"text","text":"Transcribe the following speech segment in English:"},            {"type":"audio","audio":"https://huggingface.co/datasets/ariG23498/demo-data/resolve/main/speech.wav"},        ]    }]model_generation(model,messages)

Interleaved with Image/Video

# Interleaved with Imagemessages= [    {"role":"user","content": [            {"type":"image","image":"https://huggingface.co/datasets/ariG23498/demo-data/resolve/main/airplane.jpg"},            {"type":"text","text":"Describe this image."}        ]    }]model_generation(model,messages)

Inference

Gemma 3n

Notebooks

Function Calling

Gemma 3n

Notebooks

Fine Tuning

Gemma 3n

Notebooks

Scripts

Gemma 3

RAG

Gemma 3n

$ pip install -U -q -r requirements.txt