Golang image search api

#gin #go #restapi #imagesearch

Building the REST API

I'll be using gin for this part of project.

Gin is a web framework written in Go (Golang). It features a martini-like API with performance that is up to 40 times faster thanks to httprouter. If you need performance and good productivity, you will love Gin.

TLDR

The colly code for scraping

First we create ourgo.mod file:

 go mod init Intersect_api

Then we'll create ourmain.go file and add the following code.

// main.gopackagemainimport("Intersect/server""log""os""time""github.com/getsentry/sentry-go")funcmain(){// Initialize Libraries// Initialize and defer Sentryiferr:=sentry.Init(sentry.ClientOptions{Dsn:os.Getenv("SENTRY_DSN"),});err!=nil{log.Fatalln("Sentry Init Error: ",err)}defersentry.Flush(2*time.Second)// Initialize Serverserver.Init()}

Create a server module and initialize the server

/// server.gopackageserverimport"github.com/getsentry/sentry-go"// Init : Initialize the routes and serverfuncInit(){r:=NewRouter()err:=r.Run()iferr!=nil{sentry.CaptureException(err)}}

Good rule of thumb to add some middleware with a token required

// middleware.gopackageserverimport("net/http""github.com/gin-gonic/gin")funcAuthMiddleware()gin.HandlerFunc{authHeader:="let-me-in"// os.Getenv("AUTHORIZATION_STRING")returnfunc(c*gin.Context){requiredAuth:=c.Request.Header.Get("Authorization")ifrequiredAuth!=authHeader{c.AbortWithStatusJSON(http.StatusUnauthorized,map[string]string{"message":"unauthorized peasant 😃"})}c.Next()// middleware}}

Then we'll create default cors, routes and also call our middleware inside the router.use() method

packageserverimport("Intersect/scraper""github.com/gin-contrib/cors""github.com/gin-gonic/gin")/// router.gofuncNewRouter()*gin.Engine{// gin.SetMode(gin.ReleaseMode)router:=gin.New()// Gin and CORS Middlewaresrouter.Use(gin.Logger())router.Use(gin.Recovery())/// Corsrouter.Use(setCors())/// Declare Middlewareauthorized:=router.Group("/")authorized.Use(AuthMiddleware()){authorized.GET("",scraper.Greet)searchRouter:=authorized.Group("search")searchRouter.GET("",scraper.GetImgs)}returnrouter}// CorsfuncsetCors()gin.HandlerFunc{returncors.New(cors.Config{AllowOrigins:[]string{"*"},AllowMethods:[]string{"GET","OPTIONS","PUT"},AllowHeaders:[]string{"Origin","Authorization"},ExposeHeaders:[]string{"Content-Length"},AllowCredentials:true,})}

Then we'll create a scraper directory/module and create two files

greeting.go for default route and
search.go for the request route.(Because the default route will be lonely).

packagescraperimport("net/http""github.com/gin-gonic/gin")funcGreet(c*gin.Context){c.JSON(http.StatusOK,map[string]string{"sup":"🤘🚀"})}

And now the main component of the server, we'll scrape the images from various websites for free and non-stock images.
- https://unsplash.com
- https://burst.shopify.com
- https://www.pexels.com/
- https://www.flickr.com
- https://stocksnap.io
- https://google.com ... (because why not)

packagescraperimport("fmt""net/http""strings""github.com/gin-gonic/gin""github.com/gocolly/colly")funcGetImgs(c*gin.Context){searchQuery:=c.Query("q")res:=getSearch(searchQuery)c.JSON(http.StatusOK,res)}funcgetSearch(searchQuerystring)Images{searchString:=strings.Replace(searchQuery," ","-",-1)c:=colly.NewCollector()c.UserAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X x.y; rv:42.0) Gecko/20100101 Firefox/42.0"c.AllowURLRevisit=truec.DisableCookies()array:=[]string{}// Find and visit all linksc.OnHTML("img[src]",func(e*colly.HTMLElement){src:=e.Attr("src")ifsrc!=""{array=append(array,e.Attr("src"))}})// Requesting a url for htmlc.OnRequest(func(r*colly.Request){fmt.Println("Visiting",r.URL)})// search querypexelsQuery:=strings.Replace(searchString,"-","%20",-1)stocSnapQuery:=strings.Replace(searchString,"-","+",-1)//c.Visit("https://unsplash.com/s/"+searchString)c.Visit("https://burst.shopify.com/photos/search?utf8=%E2%9C%93&q="+searchString+"&button=")c.Visit("https://www.pexels.com/search/"+pexelsQuery+"/")c.Visit("https://www.flickr.com/search/?text="+pexelsQuery)c.Visit("http://www.google.com/images?q="+stocSnapQuery)c.Visit("https://stocksnap.io/search/"+stocSnapQuery)//returnImages{Count:len(array),Data:array}}typeImagesstruct{Countint`json:"counts"`Data[]string`json:"data"`}