Diffbot is a suite of ML-based products that make it easy to structure and integrate web data.

Installation and Setup​

Get a free Diffbot API token andfollow these instructions to authenticate your requests.

Document Loader​

Diffbot'sExtract API is a service that structures and normalizes data from web pages.

Unlike traditional web scraping tools,Diffbot Extract doesn't require any rules to read the content on a page. It uses a computer vision model to classify a page into one of 20 possible types, and then transforms raw HTML markup into JSON. The resulting structured JSON follows a consistenttype-based ontology, which makes it easy to extract data from multiple different web sources with the same schema.

See ausage example.

from langchain_community.document_loadersimport DiffbotLoader

Graphs​

Diffbot'sNatural Language Processing API allows for the extraction of entities, relationships, and semantic meaning from unstructured text data.

See ausage example.

from langchain_experimental.graph_transformers.diffbotimport DiffbotGraphTransformer