In this article, we will discuss how to scrape data like Names, Ratings, Descriptions, Reviews, addresses, Contact numbers, etc. from google maps using Python.
Modules needed:
- Selenium: Usually, to automate testing, Selenium is used. We can do this for scraping also as the browser automation here helps with interacting javascript involved with clicks, scrolls, movement of data between multiple frames, etc.,
- Web/Chrome Driver:Selenium requires a web driver to interface with the chosen browser. Web drivers interact with a remote web server through a wire protocol which is common to all. You can check out and install the web drivers of your browser choice, to establish a connection with the web driver.
Download chrome Driverfrom https://chromedriver.chromium.org/home then copy/cut the file from your download and paste it at your C drive, i.eC:\chromedriver_win32\chromedriver.exe.Please make sure you are following this path otherwise it will through a path error.
Syntax:
For obtaining the title of the place:
review_titles=browser.find_element_by_class_name("x3AX1-LfntMc-header-title-title")
print(review_titles.text)
For obtaining the rating of the place:
stars=browser.find_element_by_class_name("aMPvhf-fI6EEc-KVuj8d")
print("The stars of restaurant are:",stars.text)
For obtaining the description of the place:
description=browser.find_element_by_class_name("uxOu9-sTGRBb-T3yXSc")
print(description.text)
For obtaining the address of the place:
address=browser.find_elements_by_class_name("CsEnBe")[0]
print("Address: ",address.text)
For obtaining the contact number of the place:
phone = browser.find_elements_by_class_name("CsEnBe")[-2]
print("Contact Number: ", phone.text)
For obtaining the reviews of the place:
review=browser.find_elements_by_class_name("OXD3gb")
print("------------------------ Reviews --------------------")
for j in review:
print(j.text)
Example 1:
Scrap Title and rating.
Python3# Import the library Seleniumfromseleniumimportwebdriverfromselenium.webdriver.common.action_chainsimportActionChains# Make browser open in backgroundoptions=webdriver.ChromeOptions()options.add_argument('headless')# Create the webdriver objectbrowser=webdriver.Chrome(executable_path="C:\chromedriver_win32\chromedriver.exe",options=options)# Obtain the Google Map URLurl=["https://www.google.com/maps/place/\Papa+John's+Pizza/@40.7936551,-74.0124687,17z/data=!3m1!4b1!\4m5!3m4!1s0x89c2580eaa74451b:0x15d743e4f841e5ed!8m2!3d40.\7936551!4d-74.0124687","https://www.google.com/maps/place/\Lucky+Dhaba/@30.653792,76.8165233,17z/data=!3m1!4b1!4m5!3m4!\1s0x390feb3e3de1a031:0x862036ab85567f75!8m2!3d30.653792!4d76.818712"]# Initialize variables and declare it 0i=0# Create a loop for obtaining data from URLsforiinrange(len(url)):# Open the Google Map URLbrowser.get(url[i])# Obtain the title of that placetitle=browser.find_element_by_class_name("x3AX1-LfntMc-header-title-title")print(i+1,"-",title.text)# Obtain the ratings of that placestars=browser.find_element_by_class_name("aMPvhf-fI6EEc-KVuj8d")print("The stars of restaurant are:",stars.text)print("\n")Output:
1 - Papa Johns PizzaThe stars of restaurant are: 3.62 - Lucky Da DhabaThe stars of restaurant are: 3.8
Example 2:
Scrap Title and address.
Python3# Import the library Seleniumfromseleniumimportwebdriverfromselenium.webdriver.common.action_chainsimportActionChains# Make browser open in backgroundoptions=webdriver.ChromeOptions()options.add_argument('headless')# Create the webdriver objectbrowser=webdriver.Chrome(executable_path="C:\chromedriver_win32\chromedriver.exe",options=options)# Obtain the Google Map URLurl=["https://www.google.com/maps/place/\Papa+John's+Pizza/@40.7936551,-74.0124687,17z/data=!3m1!4b1!\4m5!3m4!1s0x89c2580eaa74451b:0x15d743e4f841e5ed!8m2!3d40.\7936551!4d-74.0124687","https://www.google.com/maps/place/\Lucky+Dhaba/@30.653792,76.8165233,17z/data=!3m1!4b1!4m5!3m4!\1s0x390feb3e3de1a031:0x862036ab85567f75!8m2!3d30.653792!4d76.818712"]# Initialize variables and declare it 0i=0# Create a loop for obtaining data from URLsforiinrange(len(url)):# Open the Google Map URLbrowser.get(url[i])# Obtain the title of that placetitle=browser.find_element_by_class_name("x3AX1-LfntMc-header-title-title")print(i+1,"-",title.text)# Obtain the address of that placeaddress=browser.find_elements_by_class_name("CsEnBe")[0]print("Address: ",address.text)print("\n")Output:
1 - Papa Johns Pizza
Address: 6602 Bergenline Ave, West New York, NJ 07093, United States
2 - Lucky Da Dhaba
Address: shop no.2, Patiala Road, National Highway 64, Zirakpur, Punjab 140603
Example 3:
Scrap Title and contact number.
Python3# Import the library Seleniumfromseleniumimportwebdriverfromselenium.webdriver.common.action_chainsimportActionChains# Make browser open in backgroundoptions=webdriver.ChromeOptions()options.add_argument('headless')# Create the webdriver objectbrowser=webdriver.Chrome(executable_path="C:\chromedriver_win32\chromedriver.exe",options=options)# Obtain the Google Map URLurl=["https://www.google.com/maps/place/\Papa+John's+Pizza/@40.7936551,-74.0124687,17z/data=!3m1!4b1!\4m5!3m4!1s0x89c2580eaa74451b:0x15d743e4f841e5ed!8m2!3d40.\7936551!4d-74.0124687","https://www.google.com/maps/place/\Lucky+Dhaba/@30.653792,76.8165233,17z/data=!3m1!4b1!4m5!3m4!\1s0x390feb3e3de1a031:0x862036ab85567f75!8m2!3d30.653792!4d76.818712"]# Initialize variables and declare it 0i=0# Create a loop for obtaining data from URLsforiinrange(len(url)):# Open the Google Map URLbrowser.get(url[i])# Obtain the title of that placetitle=browser.find_element_by_class_name("x3AX1-LfntMc-header-title-title")print(i+1,"-",title.text)# Obtain the contact number of that placephone=browser.find_elements_by_class_name("CsEnBe")[-2]print("Contact Number: ",phone.text)print("\n")Output:
1 - Papa Johns PizzaContact Number: +1 201-662-72722 - Lucky Da DhabaContact Number: 095922 67185
Example 4:
Scrap Title and reviews.
Python3# Import the library Seleniumfromseleniumimportwebdriverfromselenium.webdriver.common.action_chainsimportActionChains# Make browser open in backgroundoptions=webdriver.ChromeOptions()options.add_argument('headless')# Create the webdriver objectbrowser=webdriver.Chrome(executable_path="C:\chromedriver_win32\chromedriver.exe",options=options)# Obtain the Google Map URLurl=["https://www.google.com/maps/place/\Papa+John's+Pizza/@40.7936551,-74.0124687,17z/data=!3m1!4b1!\4m5!3m4!1s0x89c2580eaa74451b:0x15d743e4f841e5ed!8m2!3d40.\7936551!4d-74.0124687","https://www.google.com/maps/place/\Lucky+Dhaba/@30.653792,76.8165233,17z/data=!3m1!4b1!4m5!3m4!\1s0x390feb3e3de1a031:0x862036ab85567f75!8m2!3d30.653792!4d76.818712"]# Initialize variables and declare it 0i=0# Create a loop for obtaining data from URLsforiinrange(len(url)):# Open the Google Map URLbrowser.get(url[i])# Obtain the title of that placetitle=browser.find_element_by_class_name("x3AX1-LfntMc-header-title-title")print(i+1,"-",title.text)# Obtain the reviews of that placereview=browser.find_elements_by_class_name("OXD3gb")print("------------------------ Reviews --------------------")forjinreview:print(j.text)print("\n")Output:
1 - Papa Johns Pizza
------------------------ Reviews --------------------
"The food is so good and they even make the pizza so fast, omg."
"He deals with the money also helps making the pizza without plastic gloves."
"This is my pizza place to go, no hassles at all!"
2 - Lucky Da Dhaba
------------------------ Reviews --------------------
"Best place for a small group of people, food quality is amazing"
"Nice staff and quick service good quantity and well cooked meal."
"I ordered chicken biryani they served me chicken pulao not Biryani."
Code Implementation:
Python3# Import the library Seleniumfromseleniumimportwebdriverfromselenium.webdriver.common.action_chainsimportActionChains# Make browser open in backgroundoptions=webdriver.ChromeOptions()options.add_argument('headless')# Create the webdriver objectbrowser=webdriver.Chrome(executable_path="C:\chromedriver_win32\chromedriver.exe",options=options)# Obtain the Google Map URLurl=["https://www.google.com/maps/place/\Papa+John's+Pizza/@40.7936551,-74.0124687,17z/data=!3m1!4b1!\4m5!3m4!1s0x89c2580eaa74451b:0x15d743e4f841e5ed!8m2!3d40.\7936551!4d-74.0124687","https://www.google.com/maps/place/\Lucky+Dhaba/@30.653792,76.8165233,17z/data=!3m1!4b1!4m5!3m4!\1s0x390feb3e3de1a031:0x862036ab85567f75!8m2!3d30.653792!4d76.818712"]# Initialize variables and declare it 0i=0# Create a loop for obtaining data from URLsforiinrange(len(url)):# Open the Google Map URLbrowser.get(url[i])# Obtain the title of that placetitle=browser.find_element_by_class_name("x3AX1-LfntMc-header-title-title")print(i+1,"-",title.text)# Obtain the ratings of that placestars=browser.find_element_by_class_name("aMPvhf-fI6EEc-KVuj8d")print("The stars of restaurant are:",stars.text)# Obtain the description of that placedescription=browser.find_element_by_class_name("uxOu9-sTGRBb-T3yXSc")print("Description: ",description.text)# Obtain the address of that placeaddress=browser.find_elements_by_class_name("CsEnBe")[0]print("Address: ",address.text)# Obtain the contact number of that placephone=browser.find_elements_by_class_name("CsEnBe")[-2]print("Contact Number: ",phone.text)# Obtain the reviews of that placereview=browser.find_elements_by_class_name("OXD3gb")print("------------------------ Reviews --------------------")forjinreview:print(j.text)print("\n")Output:
