Repository files navigation

A Happy and lightweight Python Package that Provides an API to search for articles on Google News and returns a usable JSON response! 🚀
If you like ❤️ GNews or find it useful 🌟, support the project by buying me a coffee ☕.


🚀 View Demo ·🐞 Report Bug ·🚀 Request Feature

🚩 GNews is A Happy and lightweight Python Package that searches Google News RSS Feed and returns a usable JSONresponse
🚩 As well as you can fetch full article (No need to write scrappers for articles fetching anymore)

Google News cover across141+ countries with41+ languages. On the bottom left side of the Google News page youmay find aLanguage & region section where you can find all of the supported combinations.

This section provides instructions for two different use cases:

1. Installing the GNews package for immediate use.
2. Setting up the GNews project for local development.

To install the package and start using it in your own projects, follow these steps:

pip install gnews

If you want to make modifications locally, follow these steps to set up the development environment.

1. Installdocker and docker-compose.
2. Configure the.env file by placing your MongoDB credentials.
3. Run the following command to build and start the Docker containers:

docker-compose up --build

1. Clone this repository:

git clone https://github.com/ranahaani/GNews.git

2. Set up a virtual environment:

virtualenv venvsource venv/bin/activate# MacOS/Linux.\venv\Scripts\activate# Windows

3. Install the required dependencies:

pip install -r requirements.txt

fromgnewsimportGNewsgoogle_news=GNews()pakistan_news=google_news.get_news('Pakistan')print(pakistan_news[0])

[{'publisher': 'Aljazeera.com', 'description': 'Pakistan accuses India of stoking conflict in Indian Ocean  '                'Aljazeera.com', 'published date': 'Tue, 16 Feb 2021 11:50:43 GMT', 'title': 'Pakistan accuses India of stoking conflict in Indian Ocean - '          'Aljazeera.com', 'url': 'https://www.aljazeera.com/news/2021/2/16/pakistan-accuses-india-of-nuclearizing-indian-ocean' }, ...]

• GNews.get_top_news()

• GNews.get_news(keyword)

• GNews.get_news_by_topic(topic)
• Available topics: WORLD, NATION, BUSINESS, TECHNOLOGY, ENTERTAINMENT, SPORTS, SCIENCE, HEALTH, POLITICS, CELEBRITIES, TV, MUSIC, MOVIES, THEATER, SOCCER, CYCLING, MOTOR SPORTS, TENNIS, COMBAT SPORTS, BASKETBALL, BASEBALL, FOOTBALL, SPORTS BETTING, WATER SPORTS, HOCKEY, GOLF, CRICKET, RUGBY, ECONOMY, PERSONAL FINANCE, FINANCE, DIGITAL CURRENCIES, MOBILE, ENERGY, GAMING, INTERNET SECURITY, GADGETS, VIRTUAL REALITY, ROBOTICS, NUTRITION, PUBLIC HEALTH, MENTAL HEALTH, MEDICINE, SPACE, WILDLIFE, ENVIRONMENT, NEUROSCIENCE, PHYSICS, GEOLOGY, PALEONTOLOGY, SOCIAL SCIENCES, EDUCATION, JOBS, ONLINE EDUCATION, HIGHER EDUCATION, VEHICLES, ARTS-DESIGN, BEAUTY, FOOD, TRAVEL, SHOPPING, HOME, OUTDOORS, FASHION.

• GNews.get_news_by_location(location)
• location can be name of city/state/country

• GNews.get_news_by_site(site)
• site should be in the format of:"cnn.com"

All parameters are optional and can be passed during initialization. Here’s a list of the available parameters:

• language: The language in which to return results (default: 'en').
• country: The country code for the headlines (default: 'US').
• period: The time period for which you want news.
• start_date: Date after which results must have been published.
• end_date: Date before which results must have been published.
• max_results: The maximum number of results to return (default: 100).
• exclude_websites: A list of websites to exclude from results.
• proxy: A dictionary specifying the proxy settings used to route requests. The dictionary should contain a single key-value pair where the key is the protocol (http orhttps) and the value is the proxy address. Example:

# Example with only HTTP proxyproxy= {'http':'http://your_proxy_address',  }# Example with only HTTPS proxyproxy= {'https':'http://your_proxy_address',  }

fromgnewsimportGNews# Initialize GNews with various parameters, including proxygoogle_news=GNews(language='en',country='US',period='7d',start_date=None,end_date=None,max_results=10,exclude_websites=['yahoo.com','cnn.com'],proxy={'https':'https://your_proxy_address'    })

• Or change it to an existing object

google_news.period='7d'# News from last 7 daysgoogle_news.max_results=10# number of responses across a keywordgoogle_news.country='United States'# News from a specific countrygoogle_news.language='english'# News in a specific languagegoogle_news.exclude_websites= ['yahoo.com','cnn.com']# Exclude news from specific website i.e Yahoo.com and CNN.comgoogle_news.start_date= (2020,1,1)# Search from 1st Jan 2020google_news.end_date= (2020,3,1)# Search until 1st March 2020

The format of the timeframe is a string comprised of a number, followed by a letter representing the time operator. Forexample 1y would signify 1 year. Full list of operators below:

 - h = hours (eg: 12h) - d = days (eg: 7d) - m = months (eg: 6m) - y = years (eg: 1y)

Setting the start and end dates can be done by passing in either a datetime or a tuple in the form (YYYY, MM, DD).

print(google_news.AVAILABLE_COUNTRIES){'Australia':'AU','Botswana':'BW','Canada ':'CA','Ethiopia':'ET','Ghana':'GH','India ':'IN','Indonesia':'ID','Ireland':'IE','Israel ':'IL','Kenya':'KE','Latvia':'LV','Malaysia':'MY','Namibia':'NA','New Zealand':'NZ','Nigeria':'NG','Pakistan':'PK','Philippines':'PH','Singapore':'SG','South Africa':'ZA','Tanzania':'TZ','Uganda':'UG','United Kingdom':'GB','United States':'US','Zimbabwe':'ZW','Czech Republic':'CZ','Germany':'DE','Austria':'AT','Switzerland':'CH','Argentina':'AR','Chile':'CL','Colombia':'CO','Cuba':'CU','Mexico':'MX','Peru':'PE','Venezuela':'VE','Belgium ':'BE','France':'FR','Morocco':'MA','Senegal':'SN','Italy':'IT','Lithuania':'LT','Hungary':'HU','Netherlands':'NL','Norway':'NO','Poland':'PL','Brazil':'BR','Portugal':'PT','Romania':'RO','Slovakia':'SK','Slovenia':'SI','Sweden':'SE','Vietnam':'VN','Turkey':'TR','Greece':'GR','Bulgaria':'BG','Russia':'RU','Ukraine ':'UA','Serbia':'RS','United Arab Emirates':'AE','Saudi Arabia':'SA','Lebanon':'LB','Egypt':'EG','Bangladesh':'BD','Thailand':'TH','China':'CN','Taiwan':'TW','Hong Kong':'HK','Japan':'JP','Republic of Korea':'KR'}

print(google_news.AVAILABLE_LANGUAGES){'english':'en','indonesian':'id','czech':'cs','german':'de','spanish':'es-419','french':'fr','italian':'it','latvian':'lv','lithuanian':'lt','hungarian':'hu','dutch':'nl','norwegian':'no','polish':'pl','portuguese brasil':'pt-419','portuguese portugal':'pt-150','romanian':'ro','slovak':'sk','slovenian':'sl','swedish':'sv','vietnamese':'vi','turkish':'tr','greek':'el','bulgarian':'bg','russian':'ru','serbian':'sr','ukrainian':'uk','hebrew':'he','arabic':'ar','marathi':'mr','hindi':'hi','bengali':'bn','tamil':'ta','telugu':'te','malyalam':'ml','thai':'th','chinese simplified':'zh-Hans','chinese traditional':'zh-Hant','japanese':'ja','korean':'ko'}

• Get news returns the list with following keys:title,published_date,description,url,publisher.

• To read a full article you can either:
  • Navigate to the url directly in your browser, or
  • Usenewspaper3k library to scrape the article
• The article url, needed for both methods, is accessed asarticle['url'].

1. Install the library -pip3 install newspaper3k.
2. Useget_full_article method fromGNews, that creates annewspaper.article.Article object from the url.

fromgnewsimportGNewsgoogle_news=GNews()json_resp=google_news.get_news('Pakistan')article=google_news.get_full_article(json_resp[0]['url'])# newspaper3k instance, you can access newspaper3k all attributes in article

This new object containstitle,text (full article) orimages attributes. Examples:

article.title

IMF Staff and Pakistan Reach Staff-Level Agreement on the Pending Reviews Under the Extended Fund Facility'

article.text

End-of-Mission press releases include statements of IMF staff teams that convey preliminary findings after a mission. The views expressed are those of the IMF staff and do not necessarily represent the views of the IMF’s Executive Board.\n\nIMF staff and the Pakistani authorities have reached an agreement on a package of measures to complete second to fifth reviews of the authorities’ reform program supported by the IMF Extended Fund Facility (EFF) ..... (full article)

article.images

{'https://www.imf.org/~/media/Images/IMF/Live-Page/imf-live-rgb-h.ashx?la=en', 'https://www.imf.org/-/media/Images/IMF/Data/imf-logo-eng-sep2019-update.ashx', 'https://www.imf.org/-/media/Images/IMF/Data/imf-seal-shadow-sep2019-update.ashx', 'https://www.imf.org/-/media/Images/IMF/Social/TW-Thumb/twitter-seal.ashx', 'https://www.imf.org/assets/imf/images/footer/IMF_seal.png'}

article.authors

[]

Read full documentation fornewspaper3knewspaper3k

• Save to MongoDB
• Save to SQLite
• Save to JSON
• Save to .CSV file
• More than 100 articles

See theopen issues for a list of proposed features (and known issues).

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Anycontributions you make aregreatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

Distributed under the MIT License. SeeLICENSE for more information.

Muhammad Abdullah -@ranahaani -ranahaani@gmail.com

Project Link:https://github.com/ranahaani/GNews