Repository files navigation

The packagepublicsuffix provides a Go domain name parser based on thePublic Suffix List.

Currently,publicsuffix-go requires Go version 1.21 or greater. We do our best not to break older versions of Go if we don't have to, but due to tooling constraints, we don't always test older versions.

Clone the repositoryin your workspace and move into it:

mkdir -p$GOPATH/src/github.com/weppos&&cd$_git clone git@github.com:weppos/publicsuffix-go.gitcd publicsuffix-go

Fetch the dependencies:

go get ./...

Run the test suite.

gotest ./...

The following command runs the entire test suite.

gotest ./...

There are 3 different test suites built into this library:

• Acceptance: the acceptance test suite contains some high level tests to ensure the library behaves as expected
• PSL: the PSL test suite runs the library against theofficial Public Suffix test cases
• Unit: the unit test suite stresses the various single components of this package

go get github.com/weppos/publicsuffix-go

This is a simple example that demonstrates how to use the package with the default options and the default Public Suffix list packaged with the library.

package mainimport ("fmt""github.com/weppos/publicsuffix-go/publicsuffix")funcmain() {// Extract the domain from a string// using the default listfmt.Println(publicsuffix.Domain("example.com"))// example.comfmt.Println(publicsuffix.Domain("www.example.com"))// example.comfmt.Println(publicsuffix.Domain("example.co.uk"))// example.co.ukfmt.Println(publicsuffix.Domain("www.example.co.uk"))// example.co.uk// Parse the domain from a string// using the default listfmt.Println(publicsuffix.Parse("example.com"))// &DomainName{"com", "example", ""}fmt.Println(publicsuffix.Parse("www.example.com"))// &DomainName{"com", "example", "www"}fmt.Println(publicsuffix.Parse("example.co.uk"))// &DomainName{"co.uk", "example", ""}fmt.Println(publicsuffix.Parse("www.example.co.uk"))// &DomainName{"co.uk", "example", "www"}}

The PSL is composed by two list of suffixes: IANA suffixes, and Private Domains.

Private domains are submitted by private organizations. By default, private domains are not ignored.Sometimes, you want to ignore these domains and only query against the IANA suffixes. You have two options:

1. Ignore the domains at runtime
2. Create a custom list without the private domains

In the first case, the private domains are ignored at runtime: they will still be included in the lists but the lookup will skip them when found.

publicsuffix.DomainFromListWithOptions(publicsuffix.DefaultList(),"google.blogspot.com",nil)// google.blogspot.compublicsuffix.DomainFromListWithOptions(publicsuffix.DefaultList(),"google.blogspot.com",&publicsuffix.FindOptions{IgnorePrivate:true})// blogspot.com// Note that the DefaultFindOptions includes the private domains by defaultpublicsuffix.DomainFromListWithOptions(publicsuffix.DefaultList(),"google.blogspot.com",publicsuffix.DefaultFindOptions)// google.blogspot.com

This solution is easy, but slower. If you find yourself ignoring the private domains in all cases (or in most cases), you may want to create a custom list without the private domains.

list:=NewListFromFile("path/to/list.txt",&publicsuffix.ParserOption{PrivateDomains:false})publicsuffix.DomainFromListWithOptions(list,"google.blogspot.com",nil)// blogspot.com

A-label and U-label are two different ways to represent IDN domain names. These two encodings are also known as ASCII (A-label) or Pynucode vs Unicode (U-label). Conversions between U-labels and A-labels are performed according to the"Punycode" specification, adding or removing the ACE prefix as needed.

IDNA-aware applications generally use the A-label form for storing and manipulating data, whereas the U-labels can appear in presentation and user interface forms.

Although the PSL list has been traditionally U-label encoded, this library follows the common industry standards and stores the rules in their A-label form. Therefore, unless explicitly mentioned, any method call, comparison or internal representation is expected to be ASCII-compatible encoded (ACE).

Passing Unicode names to the library may either result in error or unexpected behaviors.

If you are interested in the details of this decision, you can read the full discussionhere.

Thegolang.org/x/net/publicsuffix is a package part of the Golangx/net package, that provides a public suffix list implementation.

The main difference is that thex/net package is optimized for speed, but it's less flexible. The list is compiled and embedded into the package itself. However, this is also the main downside.Thelist is not frequently refreshed, hence the results may be inaccurate, in particular if you heavily rely on the private domain section of the list. Changes in the IANA section are less frequent, whereas changes in the Private Domains section happens weekly.

This package provides the following extra features:

• Ability to load an arbitrary list at runtime (e.g. you can feed your own list, or create multiple lists)
• Ability to create multiple lists
• Ability to parse a domain using a previously defined list
• Ability to add custom rules to an existing list, or merge/load rules from other lists (provided as file or string)
• Advanced access to the list rules
• Ability to ignore private domains at runtime, or when the list is parsed

This package also aims for 100% compatibility with thex/net package. A special adapter is provided as a drop-in replacement. Simply change the include statement from

import ("golang.org/x/net/publicsuffix")

to

import ("github.com/weppos/publicsuffix-go/net/publicsuffix")

Thegithub.com/weppos/publicsuffix-go/net/publicsuffix package defines the same methods defined ingolang.org/x/net/publicsuffix, but these methods are implemented using thegithub.com/weppos/publicsuffix-go/publicsuffix package.

Note that the adapter doesn't offer the flexibility ofgithub.com/weppos/publicsuffix-go/publicsuffix, such as the ability to use multiple lists or disable private domains at runtime.

This package implements thecookiejar.PublicSuffixList interface. It means it can be used as a value for thePublicSuffixList option when creating anet/http/cookiejar.

import ("net/http/cookiejar""github.com/weppos/publicsuffix-go/publicsuffix")deliciousJar:=cookiejar.New(&cookiejar.Options{PublicSuffixList:publicsuffix.CookieJarList})

Copyright (c) 2016-2024 Simone Carletti. This is Free Software distributed under the MIT license.