- Notifications
You must be signed in to change notification settings - Fork26
An Elixir library for parsing and extracting data from HTML and XML with CSS or XPath selectors.
License
MIT, Apache-2.0 licenses found
Licenses found
mischov/meeseeks
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
Meeseeks is an Elixir library for parsing and extracting data from HTML and XML with CSS or XPath selectors.
importMeeseeks.CSShtml=HTTPoison.get!("https://news.ycombinator.com/").bodyforstory<-Meeseeks.all(html,css("tr.athing"))dotitle=Meeseeks.one(story,css(".title a"))%{title:Meeseeks.text(title),url:Meeseeks.attr(title,"href")}end#=> [%{title: "...", url: "..."}, %{title: "...", url: "..."}, ...]
- Friendly API
- Browser-grade HTML5 parser
- Permissive XML parser
- CSS and XPath selectors
- Supports custom selectors
- Helpers to extract data from selections
Meeseeks is tested with a a minimum combination of Elixir 1.16.0 and Erlang/OTP 26.0 and a maximum combination of Elixir 1.18.0 and Erlang/OTP 27.0.
Meeseeks depends on the Rust libraryhtml5ever viameeseeks_html5ever, but becausemeeseeks_html5ever provides pre-compiled NIFs viarustler_precompiledyou do not need to have Rust installed to use Meeseeks.
To install Meeseeks, add it to yourmix.exs:
defpdepsdo[{:meeseeks,"~> 0.18.0"}]end
Then runmix deps.get.
If you need to force compilation of the Rust NIF for some reason, see the instructionshere.
Start by parsing a source (HTML/XML string orMeeseeks.TupleTree) into aMeeseeks.Document so that it can be queried.
Meeseeks.parse/1 parses the source as HTML, butMeeseeks.parse/2 accepts a second argument of either:html,:xml, or:tuple_tree that specifies how the source is parsed.
document=Meeseeks.parse("<div id=main><p>1</p><p>2</p><p>3</p></div>")#=> #Meeseeks.Document<{...}>
The selection functions accept an unparsed source, parsing it as HTML, but parsing is expensive so parse ahead of time when running multiple selections on the same document.
Next, use one of Meeseeks's selection functions -fetch_all,all,fetch_one, orone - to search for nodes.
All these functions accept a queryable (a source, a document, or aMeeseeks.Result), one or moreMeeseeks.Selectors, and optionally an initial context.
all returns a (possibly empty) list of results representing every node matching one of the provided selectors, whileone returns a result representing the first node to match a selector (depth-first) or nil if there is no match.
fetch_all andfetch_one work likeall andone respectively, but wrap the result in{:ok, ...} if there is a match or return{:error, %Meeseeks.Error{type: :select, reason: :no_match}} if there is not.
To generate selectors, use thecss macro provided byMeeseeks.CSS or thexpath macro provided byMeeseeks.XPath.
importMeeseeks.CSSresult=Meeseeks.one(document,css("#main p"))#=> #Meeseeks.Result<{ <p>1</p> }>importMeeseeks.XPathresult=Meeseeks.one(document,xpath("//*[@id='main']//p"))#=> #Meeseeks.Result<{ <p>1</p> }>
Retrieve information from theMeeseeks.Result with an extractor.
The included extractors areattr,attrs,data,dataset,html,own_text,tag,text,tree.
Meeseeks.tag(result)#=> "p"Meeseeks.text(result)#=> "1"Meeseeks.tree(result)#=> {"p", [], ["1"]}
The extractorshtml andtree work onMeeseeks.Documents in addition toMeeseeks.Results.
Meeseeks.html(document)#=> "<html><head></head><body><div id=\"main\"><p>1</p><p>2</p><p>3</p></div></body></html>"
If you are interested in contributing please read thecontribution guidelines.
Meeseeks is licensed under theMIT license.
About
An Elixir library for parsing and extracting data from HTML and XML with CSS or XPath selectors.
Topics
Resources
License
MIT, Apache-2.0 licenses found
Licenses found
Contributing
Uh oh!
There was an error while loading.Please reload this page.
Stars
Watchers
Forks
Uh oh!
There was an error while loading.Please reload this page.
Contributors7
Uh oh!
There was an error while loading.Please reload this page.