Recognize Landmarks with Firebase ML on Android

This page describes an old version of recognizing well-known landmarks in images using the deprecatedFirebase ML Vision SDK. As an alternative, you may call Cloud Vision APIs using Firebase Auth and Callable Functions to allow only users logged into your app to access the API.

You can useFirebase ML to recognize well-known landmarks in an image.

Use of the Cloud Vision APIs is subject to theGoogle Cloud Platform LicenseAgreement andServiceSpecific Terms, and billed accordingly. For billing information, see thePricing page.

Before you begin

  1. If you haven't already,add Firebase to your Android project.
  2. In yourmodule (app-level) Gradle file(usually<project>/<app-module>/build.gradle.kts or<project>/<app-module>/build.gradle),add the dependency for theFirebase ML Vision library for Android. We recommend using theFirebase Android BoMto control library versioning.
    dependencies{// Import theBoM for the Firebase platformimplementation(platform("com.google.firebase:firebase-bom:34.9.0"))// Add the dependency for theFirebase ML Vision library// When using theBoM, you don't specify versions in Firebase library dependenciesimplementation'com.google.firebase:firebase-ml-vision'}

    By using theFirebase Android BoM, your app will always use compatible versions of Firebase Android libraries.

    (Alternative)  Add Firebase library dependencies without using theBoM

    If you choose not to use theFirebase BoM, you must specify each Firebase library version in its dependency line.

    Note that if you usemultiple Firebase libraries in your app, we strongly recommend using theBoM to manage library versions, which ensures that all versions are compatible.

    dependencies{// Add the dependency for theFirebase ML Vision library// When NOT using theBoM, you must specify versions in Firebase library dependenciesimplementation'com.google.firebase:firebase-ml-vision:24.1.0'}

  3. If you haven't already enabled Cloud-based APIs for your project, do so now:

    1. Open theFirebase ML APIs page in theFirebase console.

    2. If you haven't already upgraded your project to thepay-as-you-go Blaze pricing plan, clickUpgrade to do so. (You'll be prompted to upgrade only if your project isn't on the Blaze pricing plan.)

      Only projects on the Blaze pricing plan can use Cloud-based APIs.

    3. If Cloud-based APIs aren't already enabled, clickEnable Cloud-based APIs.
    Before you deploy to production an app that uses a Cloud API, you should takesome additional steps toprevent and mitigate theeffect of unauthorized API access.

Configure the landmark detector

By default, the Cloud detector uses theSTABLE version of themodel and returns up to 10 results. If you want to change either of thesesettings, specify them with aFirebaseVisionCloudDetectorOptionsobject.

For example, to change both of the default settings, build aFirebaseVisionCloudDetectorOptions object as in the followingexample:

Kotlin

valoptions=FirebaseVisionCloudDetectorOptions.Builder().setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL).setMaxResults(15).build()

Java

FirebaseVisionCloudDetectorOptionsoptions=newFirebaseVisionCloudDetectorOptions.Builder().setModelType(FirebaseVisionCloudDetectorOptions.LATEST_MODEL).setMaxResults(15).build();

To use the default settings, you can useFirebaseVisionCloudDetectorOptions.DEFAULT in the next step.

Run the landmark detector

To recognize landmarks in an image, create aFirebaseVisionImage objectfrom either aBitmap,media.Image,ByteBuffer, byte array, or a file onthe device. Then, pass theFirebaseVisionImage object to theFirebaseVisionCloudLandmarkDetector'sdetectInImage method.

  1. Create aFirebaseVisionImage object from your image.

    • To create aFirebaseVisionImage object from amedia.Image object, such as when capturing an image from a device's camera, pass themedia.Image object and the image's rotation toFirebaseVisionImage.fromMediaImage().

      If you use the CameraX library, theOnImageCapturedListener andImageAnalysis.Analyzer classes calculate the rotation value for you, so you just need to convert the rotation to one ofFirebase ML'sROTATION_ constants before callingFirebaseVisionImage.fromMediaImage():

      Kotlin

      privateclassYourImageAnalyzer:ImageAnalysis.Analyzer{privatefundegreesToFirebaseRotation(degrees:Int):Int=when(degrees){0->FirebaseVisionImageMetadata.ROTATION_090->FirebaseVisionImageMetadata.ROTATION_90180->FirebaseVisionImageMetadata.ROTATION_180270->FirebaseVisionImageMetadata.ROTATION_270else->throwException("Rotation must be 0, 90, 180, or 270.")}overridefunanalyze(imageProxy:ImageProxy?,degrees:Int){valmediaImage=imageProxy?.imagevalimageRotation=degreesToFirebaseRotation(degrees)if(mediaImage!=null){valimage=FirebaseVisionImage.fromMediaImage(mediaImage,imageRotation)// Pass image to an ML Vision API// ...}}}

      Java

      privateclassYourAnalyzerimplementsImageAnalysis.Analyzer{privateintdegreesToFirebaseRotation(intdegrees){switch(degrees){case0:returnFirebaseVisionImageMetadata.ROTATION_0;case90:returnFirebaseVisionImageMetadata.ROTATION_90;case180:returnFirebaseVisionImageMetadata.ROTATION_180;case270:returnFirebaseVisionImageMetadata.ROTATION_270;default:thrownewIllegalArgumentException("Rotation must be 0, 90, 180, or 270.");}}@Overridepublicvoidanalyze(ImageProxyimageProxy,intdegrees){if(imageProxy==null||imageProxy.getImage()==null){return;}ImagemediaImage=imageProxy.getImage();introtation=degreesToFirebaseRotation(degrees);FirebaseVisionImageimage=FirebaseVisionImage.fromMediaImage(mediaImage,rotation);// Pass image to an ML Vision API// ...}}

      If you don't use a camera library that gives you the image's rotation, you can calculate it from the device's rotation and the orientation of camera sensor in the device:

      Kotlin

      privatevalORIENTATIONS=SparseIntArray()init{ORIENTATIONS.append(Surface.ROTATION_0,90)ORIENTATIONS.append(Surface.ROTATION_90,0)ORIENTATIONS.append(Surface.ROTATION_180,270)ORIENTATIONS.append(Surface.ROTATION_270,180)}/** * Get the angle by which an image must be rotated given the device's current * orientation. */@RequiresApi(api=Build.VERSION_CODES.LOLLIPOP)@Throws(CameraAccessException::class)privatefungetRotationCompensation(cameraId:String,activity:Activity,context:Context):Int{// Get the device's current rotation relative to its "native" orientation.// Then, from the ORIENTATIONS table, look up the angle the image must be// rotated to compensate for the device's rotation.valdeviceRotation=activity.windowManager.defaultDisplay.rotationvarrotationCompensation=ORIENTATIONS.get(deviceRotation)// On most devices, the sensor orientation is 90 degrees, but for some// devices it is 270 degrees. For devices with a sensor orientation of// 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.valcameraManager=context.getSystemService(CAMERA_SERVICE)asCameraManagervalsensorOrientation=cameraManager.getCameraCharacteristics(cameraId).get(CameraCharacteristics.SENSOR_ORIENTATION)!!rotationCompensation=(rotationCompensation+sensorOrientation+270)%360// Return the corresponding FirebaseVisionImageMetadata rotation value.valresult:Intwhen(rotationCompensation){0->result=FirebaseVisionImageMetadata.ROTATION_090->result=FirebaseVisionImageMetadata.ROTATION_90180->result=FirebaseVisionImageMetadata.ROTATION_180270->result=FirebaseVisionImageMetadata.ROTATION_270else->{result=FirebaseVisionImageMetadata.ROTATION_0Log.e(TAG,"Bad rotation value:$rotationCompensation")}}returnresult}

      Java

      privatestaticfinalSparseIntArrayORIENTATIONS=newSparseIntArray();static{ORIENTATIONS.append(Surface.ROTATION_0,90);ORIENTATIONS.append(Surface.ROTATION_90,0);ORIENTATIONS.append(Surface.ROTATION_180,270);ORIENTATIONS.append(Surface.ROTATION_270,180);}/** * Get the angle by which an image must be rotated given the device's current * orientation. */@RequiresApi(api=Build.VERSION_CODES.LOLLIPOP)privateintgetRotationCompensation(StringcameraId,Activityactivity,Contextcontext)throwsCameraAccessException{// Get the device's current rotation relative to its "native" orientation.// Then, from the ORIENTATIONS table, look up the angle the image must be// rotated to compensate for the device's rotation.intdeviceRotation=activity.getWindowManager().getDefaultDisplay().getRotation();introtationCompensation=ORIENTATIONS.get(deviceRotation);// On most devices, the sensor orientation is 90 degrees, but for some// devices it is 270 degrees. For devices with a sensor orientation of// 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.CameraManagercameraManager=(CameraManager)context.getSystemService(CAMERA_SERVICE);intsensorOrientation=cameraManager.getCameraCharacteristics(cameraId).get(CameraCharacteristics.SENSOR_ORIENTATION);rotationCompensation=(rotationCompensation+sensorOrientation+270)%360;// Return the corresponding FirebaseVisionImageMetadata rotation value.intresult;switch(rotationCompensation){case0:result=FirebaseVisionImageMetadata.ROTATION_0;break;case90:result=FirebaseVisionImageMetadata.ROTATION_90;break;case180:result=FirebaseVisionImageMetadata.ROTATION_180;break;case270:result=FirebaseVisionImageMetadata.ROTATION_270;break;default:result=FirebaseVisionImageMetadata.ROTATION_0;Log.e(TAG,"Bad rotation value: "+rotationCompensation);}returnresult;}

      Then, pass themedia.Image object and the rotation value toFirebaseVisionImage.fromMediaImage():

      Kotlin

      valimage=FirebaseVisionImage.fromMediaImage(mediaImage,rotation)

      Java

      FirebaseVisionImageimage=FirebaseVisionImage.fromMediaImage(mediaImage,rotation);
    • To create aFirebaseVisionImage object from a file URI, pass the app context and file URI toFirebaseVisionImage.fromFilePath(). This is useful when you use anACTION_GET_CONTENT intent to prompt the user to select an image from their gallery app.

      Kotlin

      valimage:FirebaseVisionImagetry{image=FirebaseVisionImage.fromFilePath(context,uri)}catch(e:IOException){e.printStackTrace()}

      Java

      FirebaseVisionImageimage;try{image=FirebaseVisionImage.fromFilePath(context,uri);}catch(IOExceptione){e.printStackTrace();}
    • To create aFirebaseVisionImage object from aByteBuffer or a byte array, first calculate the image rotation as described above formedia.Image input.

      Then, create aFirebaseVisionImageMetadata object that contains the image's height, width, color encoding format, and rotation:

      Kotlin

      valmetadata=FirebaseVisionImageMetadata.Builder().setWidth(480)// 480x360 is typically sufficient for.setHeight(360)// image recognition.setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21).setRotation(rotation).build()

      Java

      FirebaseVisionImageMetadatametadata=newFirebaseVisionImageMetadata.Builder().setWidth(480)// 480x360 is typically sufficient for.setHeight(360)// image recognition.setFormat(FirebaseVisionImageMetadata.IMAGE_FORMAT_NV21).setRotation(rotation).build();

      Use the buffer or array, and the metadata object, to create aFirebaseVisionImage object:

      Kotlin

      valimage=FirebaseVisionImage.fromByteBuffer(buffer,metadata)// Or: val image = FirebaseVisionImage.fromByteArray(byteArray, metadata)

      Java

      FirebaseVisionImageimage=FirebaseVisionImage.fromByteBuffer(buffer,metadata);// Or: FirebaseVisionImage image = FirebaseVisionImage.fromByteArray(byteArray, metadata);
    • To create aFirebaseVisionImage object from aBitmap object:

      Kotlin

      valimage=FirebaseVisionImage.fromBitmap(bitmap)

      Java

      FirebaseVisionImageimage=FirebaseVisionImage.fromBitmap(bitmap);
      The image represented by theBitmap object must be upright, with no additional rotation required.

  2. Get an instance ofFirebaseVisionCloudLandmarkDetector:

    Kotlin

    valdetector=FirebaseVision.getInstance().visionCloudLandmarkDetector// Or, to change the default settings:// val detector = FirebaseVision.getInstance()//         .getVisionCloudLandmarkDetector(options)

    Java

    FirebaseVisionCloudLandmarkDetectordetector=FirebaseVision.getInstance().getVisionCloudLandmarkDetector();// Or, to change the default settings:// FirebaseVisionCloudLandmarkDetector detector = FirebaseVision.getInstance()//         .getVisionCloudLandmarkDetector(options);

  3. Finally, pass the image to thedetectInImage method:

    Kotlin

    valresult=detector.detectInImage(image).addOnSuccessListener{firebaseVisionCloudLandmarks->// Task completed successfully// ...}.addOnFailureListener{e->// Task failed with an exception// ...}

    Java

    Task<List<FirebaseVisionCloudLandmark>>result=detector.detectInImage(image).addOnSuccessListener(newOnSuccessListener<List<FirebaseVisionCloudLandmark>>(){@OverridepublicvoidonSuccess(List<FirebaseVisionCloudLandmark>firebaseVisionCloudLandmarks){// Task completed successfully// ...}}).addOnFailureListener(newOnFailureListener(){@OverridepublicvoidonFailure(@NonNullExceptione){// Task failed with an exception// ...}});

Get information about the recognized landmarks

If the landmark recognition operation succeeds, a list ofFirebaseVisionCloudLandmark objects will be passed to the success listener. EachFirebaseVisionCloudLandmark object represents a landmark that was recognized in theimage. For each landmark, you can get its bounding coordinates in the input image,the landmark's name, its latitude and longitude, itsKnowledge Graph entity ID(if available), and the confidence score of the match. For example:

Kotlin

for(landmarkinfirebaseVisionCloudLandmarks){valbounds=landmark.boundingBoxvallandmarkName=landmark.landmarkvalentityId=landmark.entityIdvalconfidence=landmark.confidence// Multiple locations are possible, e.g., the location of the depicted// landmark and the location the picture was taken.for(locinlandmark.locations){vallatitude=loc.latitudevallongitude=loc.longitude}}

Java

for(FirebaseVisionCloudLandmarklandmark:firebaseVisionCloudLandmarks){Rectbounds=landmark.getBoundingBox();StringlandmarkName=landmark.getLandmark();StringentityId=landmark.getEntityId();floatconfidence=landmark.getConfidence();// Multiple locations are possible, e.g., the location of the depicted// landmark and the location the picture was taken.for(FirebaseVisionLatLngloc:landmark.getLocations()){doublelatitude=loc.getLatitude();doublelongitude=loc.getLongitude();}}

Next steps

Except as otherwise noted, the content of this page is licensed under theCreative Commons Attribution 4.0 License, and code samples are licensed under theApache 2.0 License. For details, see theGoogle Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.

Last updated 2026-02-18 UTC.