- Notifications
You must be signed in to change notification settings - Fork0
A Java library to access SerpApi
License
rumpelstiltzkin/serpapi4j
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
This Java package enables the scraping and parsing of search results usingSerpApi from varioussearch providers like Google, Bing, Baidu, etc. Feel free to fork this repository to add more backends.
The default language version for this package is Java 16, but if a lower version package is requiredfeel free to contact me at my email below. This code depends on GSON for efficient JSON processing.The HTTP responses are converted to JSON using GSON.
There are two native clients available to talk SerpApi. The asynchronous client provides callbackswhen search results are available. The synchronous client is written on top of the async client thatblocks the thread until the results are available.
For an example usage either see the demo app, which uses theParamsBasedSearchclass, at:src/main/java/com/anandganesh/code/serpapi4j/demo/App.javaor see the unit test, which uses the SerpApiSyncClient directly, at:src/test/java/com/anandganesh/code/serpapi4j/query/search/SearchParamsTest.java
For understanding how to useSerpApiAsyncClient, see the implementation ofSerpApiSyncClient.The APIs inSerpApiAsyncClient/SerpApiSyncClient are fairly self-explanatory.
The full documentation for SerpApi is available here.
Runtime:
- Java / JDK 16+ (https://www.java.com/en/download/)
For development:
- Maven 3.8.1 (on a Mac)
To get started with this project in Java.We provide a fully working example.
git clone git@github.com:rumpelstiltzkin/serpapi4j.gitcd serpapi4j/SERPAPI_KEY="<your private serpapi key>" mvn clean install
Note: You need an account with SerpApi to obtain this private key from:https://serpapi.com/dashboard
file:src/main/java/com/anandganesh/code/serpapi4j/demo/App.java
publicclassApp {publicstaticvoidmain(String[]args)throwsSerpApiException {if (args.length !=1) {System.out.println("Usage: app <serp api key>");System.exit(1); }// Create a search against a provider like Google using your API-key.ParamsBasedSearchparamsBasedSearch =newParamsBasedSearch(newApiKey(args[0]));GoogleSearchParamsBuildergoogleSearchParams =newGoogleSearchParamsBuilder();Stringlocation ="Austin,Texas";System.out.println("find the first Coffee in " +location);// parametersgoogleSearchParams.withSearchItem("Coffee").withLocation(location);try {// Execute searchSearchResponsesearchResponse =paramsBasedSearch.getResult(googleSearchParams);// Decode responseJsonArrayresults =searchResponse.getJsonObject() .get("local_results").getAsJsonObject() .get("places").getAsJsonArray();JsonObjectfirst_result =results.get(0).getAsJsonObject();System.out.println("first coffee shop: " +first_result.get("title").getAsString() +" found on Google in " +location);paramsBasedSearch.close(); }catch (SerpApiException |IOExceptionexception) {System.out.println(exception.getMessage() +" - while performing search");exception.printStackTrace(); }System.exit(0); }}
This example runs a search for "Coffee" using your SerpApi key.
The SerpApi service (running at serpapi.com)
- searches on Google using the query: q = "coffee"
- parses the HTML responses
- return a standardized JSON response
The classesParamsBasedSearch orSerpApi*Client:
- Format the request to the SerpApi server
- Execute a GET http request
- Parse the returned JSON response using Gsonto provide the results whose individual fields can be parsed out of the JSON response.
To use different search providers like Bing, DuckDuckGo, etc. simply use the appropriatesearch params builder (e.g.DuckDuckGoSearchParamsBuilder).
See the playground to generate your code.https://serpapi.com/playground
- How to use your SerpApi key
- Possible search params
- The Location API
- The Archive search API
- The Account API
The SerpApi key can be used with the client for the APIs that need the key
GoogleSearchParamsBuildersearchParams =newGoogleSearchParamsBuilder();searchParams.withSearchItem("Coffee");SearchResponsesearchResponse =searchClient.search(newApiKey("<your api key>"),searchParams);
These are all the possible search params to SerpApi. Not all are implemented.Feel free to fork this repo and add methods to AbstractSearchParamsBuilder.
withSearchItem(String searchItem)// e.g. "coffee"withOffsetLimit(int offset, int limit)// specify start offset and limit in resultswithJsonOutput()/ withHtmlOutput()// control output formatwithLocation(String location)// e.g. "Austin, TX"withLanguage(String language)// e.g. "en"withCountry(String country)// e.g. "us"withSearchType(SearchType searchType)// e.g. IMAGE, NEWS, SHOP, etc.withAdvancedSearchParams(AdvancedSearchParams advSearchParams)// e.g. IMGSZ_MEDIUM
SerpApiSyncClientclient =newSerpApiSyncClient();LocationsResponselocationsResponse =client.locations("San Jose",3);JsonArraylocations =locationsResponse.getJsonArray();Assert.assertEquals(3,locations.size());client.close();
This gets the first 3 locations matching "San Jose".
Let's run a search to get a search_id and then use that search_id to get results from thesearch archive maintained by serpapi.com.
// First run a params-based search to get an Id.GoogleSearchParamsBuilderparamsBuilder =newGoogleSearchParamsBuilder();// use google for thisparamsBuilder.withSearchItem("Coffee");SearchResponsesearchResponse =searchClient.search(getTestApiKey(),paramsBuilder);JsonObjectparamsBasedResults =searchResponse.getJsonObject();Assert.assertTrue(paramsBasedResults.getAsJsonArray("organic_results").size() >=1);// Then search again with the search id to get it from the archiveStringsearchId =paramsBasedResults.get("search_metadata").getAsJsonObject().get("id").getAsString();SearchResponsesearchResponse =searchClient.search(getTestApiKey(),searchId);JsonObjectarchivedResult =searchResponse.getJsonObject();Assert.assertEquals(searchId,archivedResult.get("search_metadata").getAsJsonObject().get("id").getAsString());Assert.assertEquals(paramsBasedResults,archivedResult);
This prints your account information.
SerpApiSyncClientclient =newSerpApiSyncClient();AccountResponseaccountResponse =client.account(getTestApiKey());JsonObjectaccount =accountResponse.getJsonObject();Assert.assertEquals(getTestApiKey(),newApiKey(account.get("api_key").getAsString()));client.close();
You must clone this repository.
$> git clone https://github.com/rumpelstiltzkin/serpapi4j.gitBuild the jar file.
$> SERPAPI_KEY="<your private serpapi key>" mvn clean install
Copy the jar to your project lib/ directory.
$> cp target/serpapi4j-2.0.1-SNAPSHOT.jar path/to/yourproject/lib$> SERPAPI_KEY="<your private serpapi key>" mvn clean install
This service supports searching of images, news articles, shopping results, etc.The full documentation and latest search parameters are listed on theSerpApi website.
None
- 2.0.6-SNAPSHOT Migrated package from com.serpapi to com.anandganesh.code.serpapi4j
- 2.0.5-SNAPSHOT Migrated groupId from com.serpapi to com.anandganesh.code
- 2.0.4-SNAPSHOT Bumped to diverge from an existing 2.0.3 version in the main repo
- 2.0.3-SNAPSHOT Updated this README to be more correct
- 2.0.2-SNAPSHOT serpapi4j
- 2.0.1 update gradle 6.7.1
- 2.0 refactor API : suffix SearchResults renamed Search
- 1.4 Add support for Yandex, Yahoo, Ebay
- 1.3 Add support for Bing and Baidu
- 1.2 Add support for location API, account API, search API
Anand Ganesh -rumpelgit@gmail.com.
Forked and modified from the work done by Victor Benarbia -victor@serpapi.com
About
A Java library to access SerpApi
Topics
Resources
License
Stars
Watchers
Forks
Languages
- Java87.0%
- HTML12.0%
- Other1.0%