Include large files in multimodal requests and manage files using Cloud Storage for Firebase

Only available when using theVertex AI Gemini API as your API provider.

When calling theVertex AI Gemini API from your app using aFirebase AI Logic SDK, you can prompt a Gemini model to generate textbased on a multimodal input, like images, PDFs, video, and audio.

For the non-text parts of the input (like media files), you can optionally useCloud Storage for Firebase to include files in the request. At ahigh-level, here's what you need to know about this feature:

  • You can useCloud Storage for Firebase with any multimodal request (like bothtext generation and chat) if you're using theVertex AI Gemini API.Theexamples in this guide show a basictext-and-image input.

  • You specify the file's MIME type and itsCloud Storage for Firebase URL(which always begin withgs://) in the request input. These values aremetadata automatically assigned to any file uploaded to aCloud Storagebucket.

  • You need to use a supportedfile type and URL.

Important:ForFirebase AI Logic SDKs, the maximum request size is20 MB. You get an HTTP 413 error if a request is too large.

If a file's size will make the total request sizeexceed 20 MB,then you'll need toprovide the file using a URL(for example, by using aCloud Storage for Firebase URL, as described on thispage). However, if a file is small, you can often pass it directly asinline data (note though, that a file provided as inline data is encoded tobase64 in transit, which increases the size of the request).


This solution guide describes how to set upCloud Storage for Firebase, upload afile to aCloud Storage for Firebase bucket from your app, and then include thefile's MIME type andCloud Storage for Firebase URL in your multimodal request totheGemini API.

Do you want to see the code examples? Or have you already set upCloud Storage for Firebase and you're ready to start using it with yourmultimodal requests?

Jump to the code examples

Why useCloud Storage for Firebase with your app?

Cloud Storage for Firebase uses the same fast, secure, andscalable infrastructure asGoogle Cloud Storage to store blobs and files,and itsclient SDKs are specifically built for mobile and web apps.

ForFirebase AI Logic SDKs, the maximum request size is 20 MB.You get an HTTP 413 error if a request is too large.If a file's size willmake the total request size exceed 20 MB, then use aCloud Storage for Firebase URL to include the file in your multimodal request.However, if a file is small, you can often pass it directly as inline data(note though, that a file provided as inline data is encoded to base64 intransit, which increases the size of the request).

Here are some additional benefits of usingCloud Storage for Firebase:

  • You can have end users upload images directly from your app into aCloud Storage for Firebase bucket, and then you can include those images inyour multimodal prompts just by specifying the file's MIME type andCloud Storage for Firebase URL (which is an identifier for the file).

  • You can save your end users time and bandwidth if they need to provide images,especially if they have poor or flaky network quality.

    • If a file upload or download gets interrupted, theCloud Storage for FirebaseSDKs automatically restart the operation right where it left off.
    • The same uploaded file can be used multiple times without the end userhaving to upload the same file each time its needed in your app (like in anew multimodal request).

  • You can restrict end user access to files stored inCloud Storage for Firebase by usingFirebase Security Rules,which allow only an authorized user to upload, download, or delete files.

  • You can access the files in your bucket from Firebase or fromGoogle Cloud,giving you the flexibility to do server-side processing such as imagefiltering or video transcoding using theGoogle Cloud Storage APIs.

What types of files and URLs are supported?

Here are the requirements for files and URLs when you want to useCloud Storage for Firebase URLs with theFirebase AI Logic SDKs:

  • The file must meet therequirements of input files for multimodal requests.This includes requirements like MIME type and file size.

  • The file must be stored in aCloud Storage for Firebase bucket(which means the bucket is accessible to Firebase services, likeFirebase Security Rules).If you can view your bucket in theFirebase console,then it's aCloud Storage for Firebase bucket.

  • TheCloud Storage for Firebase bucket must be in the same Firebase project inwhich you registered your app.

  • The file'sCloud Storage for Firebase URL must begin withgs://, which is theway that allGoogle Cloud Storage URLs are constructed.

  • The file's URL cannot be a "browser" URL (for example, the URL of an imagethat you find on the internet).

Also, theFirebase Security Rules for your bucket must allowappropriate access to the file. For example:

  • If you havepublic rules,thenany user or client can access the file.

  • If you haverobust rules(strongly recommended), then Firebase will check that the signed in user orclient has sufficient access to the file before allowing the call to gothrough with the provided URL.

Caution:Public rules should only be used to get started and during earlyprototyping (unless the files are actually meant to be wholly publiclyaccessible files).

UseCloud Storage for Firebase URLs withFirebase AI Logic

Only available when using theVertex AI Gemini API as your API provider.

Step 1: Set upCloud Storage for Firebase

You can find detailed instructions for setting upCloud Storage for Firebase inits getting started guide:iOS+ |Android |Web |Flutter |Unity

Here are the high-level tasks that you'll need to do:

  1. Create or import aCloud Storage for Firebase bucket in your Firebaseproject.

  2. ApplyFirebase Security Rules to this bucket.Ruleshelp you secure your files by restricting access to authorized end users.

    Important: To get started or during early prototyping, you can considersetting theseRules forpublic access, but westrongly recommend that youapply more robustFirebase Security Rules once youstart seriously developing your app andespecially before going toproduction.

  3. Add the client library forCloud Storage for Firebase to your app.

    Note that you can skip this task, but you must thenalwaysexplicitly include the MIME type and URL values in your requests.

Step 2: Upload a file to a bucket

In theCloud Storage documentation, you can learn all the differentways to upload files to a bucket. For example, you can upload local files fromthe end-user's device, such as photos and videos from the camera.Learn more:iOS+ |Android |Web |Flutter |Unity

When you upload a file to a bucket,Cloud Storage automatically appliesthe following two pieces of information to the file. You'll need to includethese values in the request (as shown in the next step of this guide).

  • MIME type: This is the media type of the file (for example,image/png).We'll automatically try to detect the MIME type during upload and apply thatmetadata to the object in the bucket. However, you can optionally specify theMIME type during upload.

  • Cloud Storage for Firebase URL: This is a unique identifier for the file.The URL must start withgs://.

Step 3: Include the file's MIME type and URL in a multimodal request

Once you have a file stored in a bucket, you can include its MIME type and URLin a request. Note that these examples show a non-streaminggenerateContentrequest, but you can also use URLs with streaming and chat.

Important: Make sure to review whichtypes of files and URLs are supported byFirebase AI Logic SDKs.

To include the file in the request, you can use either of the following options:

Option 1: Include the MIME type and URL using a Storage reference

Before trying this example, make sure that you've completed thegetting started guide for theFirebase AI Logic SDKs.

Use this option if you've just uploaded the file to the bucket, and you want toimmediately include the file (via a Storage reference) in the request. The callrequires both the MIME type and theCloud Storage for Firebase URL.

Swift

// Upload an image file using Cloud Storage for Firebase.letstorageRef=Storage.storage().reference(withPath:"images/image.jpg")guardletimageURL=Bundle.main.url(forResource:"image",withExtension:"jpg")else{fatalError("File 'image.jpg' not found in main bundle.")}letmetadata=tryawaitstorageRef.putFileAsync(from:imageURL)// Get the MIME type and Cloud Storage for Firebase URL.guardletmimeType=metadata.contentTypeelse{fatalError("The MIME type of the uploaded image is nil.")}// Construct a URL in the required format.letstorageURL="gs://\(storageRef.bucket)/\(storageRef.fullPath)"letprompt="What's in this picture?"// Construct the imagePart with the MIME type and the URL.letimagePart=FileDataPart(uri:storageURL,mimeType:mimeType)// To generate text output, call generateContent with the prompt and the imagePart.letresult=tryawaitmodel.generateContent(prompt,imagePart)iflettext=result.text{print(text)}

Kotlin

For Kotlin, the methods in this SDK are suspend functions and need to be calledfrom aCoroutine scope.
// Upload an image file using Cloud Storage for Firebase.valstorageRef=Firebase.storage.reference.child("images/image.jpg")valfileUri=Uri.fromFile(File("image.jpg"))try{valtaskSnapshot=storageRef.putFile(fileUri).await()// Get the MIME type and Cloud Storage for Firebase file path.valmimeType=taskSnapshot.metadata?.contentTypevalbucket=taskSnapshot.metadata?.bucketvalfilePath=taskSnapshot.metadata?.pathif(mimeType!=null&&bucket!=null){// Construct a URL in the required format.valstorageUrl="gs://$bucket/$filePath"// Construct a prompt that includes text, the MIME type, and the URL.valprompt=content{fileData(mimeType=mimeType,uri=storageUrl)text("What's in this picture?")}// To generate text output, call generateContent with the prompt.valresponse=model.generateContent(prompt)println(response.text)}}catch(e:StorageException){// An error occurred while uploading the file.}catch(e:GoogleGenerativeAIException){// An error occurred while generating text.}

Java

For Java, the methods in this SDK return aListenableFuture.
// Upload an image file using Cloud Storage for Firebase.StorageReferencestorage=FirebaseStorage.getInstance().getReference("images/image.jpg");UrifileUri=Uri.fromFile(newFile("images/image.jpg"));storage.putFile(fileUri).addOnSuccessListener(taskSnapshot->{// Get the MIME type and Cloud Storage for Firebase file path.StringmimeType=taskSnapshot.getMetadata().getContentType();Stringbucket=taskSnapshot.getMetadata().getBucket();StringfilePath=taskSnapshot.getMetadata().getPath();if(mimeType!=null &&bucket!=null){// Construct a URL in the required format.StringstorageUrl="gs://"+bucket+"/"+filePath;// Create a prompt that includes text, the MIME type, and the URL.Contentprompt=newContent.Builder().addFileData(storageUrl,mimeType).addText("What's in this picture?").build();// To generate text output, call generateContent with the prompt.GenerativeModelFuturesmodelFutures=GenerativeModelFutures.from(model);ListenableFuture<GenerateContentResponse>response=modelFutures.generateContent(prompt);Futures.addCallback(response,newFutureCallback<>(){@OverridepublicvoidonSuccess(GenerateContentResponseresult){StringresultText=result.getText();System.out.println(resultText);}@OverridepublicvoidonFailure(@NonNullThrowablet){t.printStackTrace();}},executor);}}).addOnFailureListener(e->{// An error occurred while uploading the file.e.printStackTrace();});

Web

// Upload an image file using Cloud Storage for Firebase.conststorageRef=ref(storage,"image.jpg");constuploadResult=awaituploadBytes(storageRef,file);// Get the MIME type and Cloud Storage for Firebase URL.// toString() is the simplest way to construct the Cloud Storage for Firebase URL// in the required format.constmimeType=uploadResult.metadata.contentType;conststorageUrl=uploadResult.ref.toString();// Construct the imagePart with the MIME type and the URL.constimagePart={fileData:{mimeType,fileUri:storageUrl}};// To generate text output, call generateContent with the prompt and imagePart.constresult=awaitmodel.generateContent([prompt,imagePart]);console.log(result.response.text());

Dart

// Upload an image file using Cloud Storage for Firebase.finalstorageRef=FirebaseStorage.instance.ref();finalimageRef=storageRef.child("images/image.jpg");awaitimageRef.putData(data);// Get the MIME type and Cloud Storage for Firebase file path.finalmetadata=awaitimageRef.getMetadata();finalmimeType=metadata.contentType;finalbucket=imageRef.bucket;finalfullPath=imageRef.fullPath;finalprompt=TextPart("What's in the picture?");// Construct a URL in the required format.finalstorageUrl='gs://$bucket/$fullPath';// Construct the filePart with the MIME type and the URL.finalfilePart=FileData(mimeType,storageUrl);// To generate text output, call generateContent with the text and the filePart.finalresponse=awaitmodel.generateContent([Content.multi([prompt,filePart])]);print(response.text);

Unity

varstorageRef=FirebaseStorage.DefaultInstance.GetReference("images/image.jpg");varmetadata=awaitstorageRef.PutFileAsync(filePathToJpg);// Get the MIME type and Cloud Storage for Firebase URL.varmimeType=metadata.ContentType;// Construct a URL in the required format.varstorageURL=newUri($"gs://{storageRef.Bucket}/{storageRef.Path}");varprompt=ModelContent.Text("What's in this picture?");// Construct a FileData that explicitly includes the MIME type and// Cloud Storage for Firebase URL values.varfileData=ModelContent.FileData(mimeType,storageURL);// To generate text output, call GenerateContentAsync with the prompt and fileData.varresponse=awaitmodel.GenerateContentAsync(new[]{prompt,fileData});UnityEngine.Debug.Log(response.Text??"No text in response.");

Option 2: Include the MIME type and URL explicitly

Before trying this example, make sure that you've completed thegetting started guide for theFirebase AI Logic SDKs.

Use this option if you know the values for the MIME type andCloud Storage for Firebase URL, and you want to include them explicitly in themultimodal request. The call requires both the MIME type and the URL.

Swift

letprompt="What's in this picture?"// Construct an imagePart that explicitly includes the MIME type and// Cloud Storage for Firebase URL values.letimagePart=FileDataPart(uri:"gs://bucket-name/path/image.jpg",mimeType:"image/jpeg")// To generate text output, call generateContent with the prompt and imagePart.letresult=tryawaitmodel.generateContent(prompt,imagePart)iflettext=result.text{print(text)}

Kotlin

For Kotlin, the methods in this SDK are suspend functions and need to be calledfrom aCoroutine scope.
// Construct a prompt that explicitly includes the MIME type and Cloud Storage for Firebase URL values.valprompt=content{fileData(mimeType="image/jpeg",uri="gs://bucket-name/path/image.jpg")text("What's in this picture?")}// To generate text output, call generateContent with the prompt.valresponse=model.generateContent(prompt)println(response.text)

Java

For Java, the methods in this SDK return aListenableFuture.
// Construct a prompt that explicitly includes the MIME type and Cloud Storage for Firebase URL values.Contentprompt=newContent.Builder().addFilePart("gs://bucket-name/path/image.jpg","image/jpeg").addText("What's in this picture?").build();// To generate text output, call generateContent with the promptGenerativeModelFuturesmodelFutures=GenerativeModelFutures.from(model);ListenableFuture<GenerateContentResponse>response=modelFutures.generateContent(prompt);Futures.addCallback(response,newFutureCallback<>(){@OverridepublicvoidonSuccess(GenerateContentResponseresult){StringresultText=result.getText();System.out.println(resultText);}@OverridepublicvoidonFailure(@NonNullThrowablet){t.printStackTrace();}},executor);

Web

constprompt="What's in this picture?";// Construct an imagePart that explicitly includes the MIME type and Cloud Storage for Firebase URL values.constimagePart={fileData:{mimeType:"image/jpeg",fileUri:"gs://bucket-name/path/image.jpg"}};// To generate text output, call generateContent with the prompt and imagePart.constresult=awaitmodel.generateContent([prompt,imagePart]);console.log(result.response.text());

Dart

finalprompt=TextPart("What's in the picture?");// Construct a filePart that explicitly includes the MIME type and Cloud Storage for Firebase URL values.finalfilePart=FileData('image/jpeg','gs://bucket-name/path/image.jpg'),// To generate text output, call generateContent with the prompt and filePart.finalresponse=awaitmodel.generateContent([Content.multi([prompt,filePart])]);print(response.text);

Unity

varprompt=ModelContent.Text("What's in this picture?");// Construct a FileData that explicitly includes the MIME type and// Cloud Storage for Firebase URL values.varfileData=ModelContent.FileData(mimeType:"image/jpeg",uri:newUri("gs://bucket-name/path/image.jpg"));// To generate text output, call GenerateContentAsync with the prompt and fileData.varresponse=awaitmodel.GenerateContentAsync(new[]{prompt,fileData});UnityEngine.Debug.Log(response.Text??"No text in response.");

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2025-10-03 UTC.