Repository files navigation

Developers Italia providesa catalog of Free and Open Sourcesoftware aimed to Public Administrations.

Thiscrawler retrieves thepubliccode.yml files from therepositories of publishers found in theDevelopers Italia API.

The crawler can either run manually on the target machine or it can be deployedfrom a Docker container.

1. Renameconfig.toml.example toconfig.toml and set the variables

   NOTE: The application also supports environment variables in substitutionto config.toml file. Remember: "environment variables get higher priority thanthe ones in configuration file"

2. Build the binary withgo build

You can build the Docker image using

docker build .

or use the image published to DockerHub:

docker run -it italia/publiccode-crawler

Gets the list of publishers fromhttps://api.developers.italia.it/v1/publishersand starts to crawl their repositories.

Gets the list of publishers inpublishers*.yml and starts to crawltheir repositories.

• crawler download-publishers downloads organizations and repositories fromtheonboarding portal repositoryand saves them to a publishers YAML file.

• developers-italia-api: the APIused to store the results of the crawling
• publiccode-parser-go: the Gopackage for parsing publiccode.yml files

Developers Italia is a project byAgID and theItalian Digital Team, which developed thecrawler and maintains this repository.