Repository files navigation

imgrec provides an interface for image recognition using theGoogleVision API. It includes functions toconvert data for features such as object detection and optical characterrecognition to data frames. The package also includes functions foranalyzing image annotations.

You can download and install the latest development version of imgrecwith the devtools package by runningdevtools::install_github('cschwem2er/imgrec').

For Windows users installing from github requires proper setup ofRtools.

The package can also be installed from CRAN by runninginstall.packages('imgrec').

Before loadingimgrec you first need to initiate your authentificationcredentials. You need an API key from a Google Project with accesspermission for the Google Vision API. For this, you can first create aproject using the Google Cloud platform. The setup process is explainedin the APIdocumentation.You will probably need to enable billing, but depending on your featureselection up to 1000 requests per month are free (seepricing). Next following theinstructionsfor creating an API key. Finally, the API key needs to be set asenvironment variable before using the initialization functiongvision_init():

Sys.setenv(gvision_key="Your Google Vision API key")

library(imgrec)gvision_init()#> Succesfully initialized authentification credentials.

In order to avoid callingSys.setenv, you can permanently store theAPI key in your.Renviron. I recommendusethis::edit_r_environ() tofind and edit your environment file.

Google Vision accepts common file types such as JPG, PNG, or BMP. Imagescan be passed to the functionget_annotations, either as url stringsor file paths to local images. In the following example,get_annotations is used to retrieve annotations for a poster of theStar Wars movieThe ForceAwakens.

sw_image<-'https://upload.wikimedia.org/wikipedia/en/a/a2/Star_Wars_The_Force_Awakens_Theatrical_Poster.jpg'results<- get_annotations(images=sw_image,# image character vectorfeatures='all',# request all available featuresmax_res=5,# maximum number of results per featuremode='url')# determine image type#> [1] "Sending API request(s).."

The function returns a response object from the Google Vision API. Italso recognizes if a user passes a character vector with multipleimages. In this case, request batches are created automatically toreduce the number of required calls to the API. After retrievingannotations, raw data can be stored in an UTF-8 encodedJSON file:

temp_file_path<- tempfile(fileext='.json')save_json(results,temp_file_path)

While some users might prefer to work with raw.json data, whichincludes every single detail returned by the API, the structure is quitecomplex and deeply nested. To simplify the data,parse_annotationsconverts most of the features to data frames. For each feature, theoriginal identifier of each image is included asimg_id.

img_data<- parse_annotations(results)# returns list of data framesnames(img_data)# all available features#>  [1] "labels"            "web_labels"        "web_similar"#>  [4] "web_match_partial" "web_match_full"    "web_match_pages"#>  [7] "web_best_guess"    "faces"             "objects"#> [10] "logos"             "full_text"         "safe_search"#> [13] "colors"            "crop_hints"

Once the features are converted to data frames, other R packages can beused to analyze the data. For instance, thelabels data frame containsannotations about image content:

img_labels<-img_data$labelshead(img_labels)

The package also extracts bounding polygons for logos, objects, facesand landmarks. We can for instance visualize all recognized logos of theStar Wars movie poster withmagick andggplot2:

library(magick)library(ggplot2)img<- image_read(sw_image)

[!!] There is currently a bug when usingmagick andggplot2 whichleads to upside down annotations. A temporary work around is to subtractimage width height (y) values (see code below).

image_ggplot(img)+    geom_rect(data=img_data$logos,           aes(xmin=poly_x_min,xmax=poly_x_max,ymin=322-poly_y_min,ymax=322-poly_y_max),color='yellow',fill=NA,linetype='dashed',size=2,inherit.aes=FALSE)+   geom_text(data=img_data$logos,           aes(x=poly_x_max,y=322-poly_y_max,label=description),size=4,color="yellow",vjust=1)+  theme(legend.position="none")

Please note that forobject recognition data, bounding polygons arerelative to image dimensions. Therefore, you need to multiply them withimage width (x) and height (y). These attributes are not returned byGoogle Vision, but can for instance be identified withmagick::image_info():

img_info<- image_info(img)img_info#> # A tibble: 1 × 7#>   format width height colorspace matte filesize density#>   <chr>  <int>  <int> <chr>      <lgl>    <int> <chr>#> 1 JPEG     220    322 sRGB       FALSE   136703 28x28

Additional functions for feature analysis are currently in development.

Please citeimgrec if you use the package for publications:

Carsten Schwemmer (2024). imgrec: Image Recognition. R package version 0.1.3.https://CRAN.R-project.org/package=imgrec

A BibTeX entry for LaTeX users is:

@Manual{,  title = {imgrec: Image Recognition},  author = {Carsten Schwemmer},  year = {2024},  note = {R package version 0.1.4},  url = {https://CRAN.R-project.org/package=imgrec},}