- Notifications
You must be signed in to change notification settings - Fork2
cbadenes/r4r
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
If you have data described inRDF format (e.g. a knowledge base or an ontology) and you want to publish them on the web following the REST principles via API over HTTP, this is your site!
You only needDocker installed in your system.
Once Docker is installed, you should be able to run the R4R container by:
docker run -it --rm \ -p 8080:7777 \ -v "$(pwd)/resources:/resources" \ -e "SPARQL_ENDPOINT=https://dbpedia.org/sparql" \ -e "RESOURCE_NAMESPACE=http://dbpedia.org/resource/" \ cbadenes/r4r:latestThe-v parameter sets a local folder where the resources that are published in the API are defined.
The-e parameters will be seen below.
The-p parameter defines the port where the service will be listening. In this case we've set 8080, so let's go tohttp://localhost:8080/ from a web browser and see a welcome message like this:
Welcome to R4R ;)A folder namedresources should have been created in the same directory where you launched the container. This folder will contain the definition of the resources that will be published through the API.
In order to continue with the following steps, it is recommended to use a text editor such asAtom ( withvelocity,json andsparql plugins), to easily handle JSON and Sparql files.
Creates the fileresources/movies/get.json.vm to handle a HTTP_GET request by providing the following content:
[ {"name":"movie1","uri" :"uri1" }, {"name":"movie2","uri" :"uri2" }]The request tohttp://localhost:8080/movies returns that json.
You have easily created a MOCK server!
The-e parameters set the endpoint (SPARQL_ENDPOINT) and namespace (RESOURCE_NAMESPACE) where the service retrieves the data through Sparql queries.
To retrieve data from a HTTP_GET request, simply create the fileresources/movies/get.sparql with the following content:
PREFIX dbo: <http://dbpedia.org/ontology/>PREFIX dbp: <http://dbpedia.org/property/>PREFIX res: <http://dbpedia.org/resource/>PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>PREFIX foaf: <http://xmlns.com/foaf/0.1/>SELECT ?m_uri ?m_title ?m_directorWHERE { ?m_uri rdf:type dbo:Film ; dbp:title ?m_title ; dbo:director ?director . ?director dbp:name ?m_director . FILTER (lang(?m_title) = 'en') . FILTER (lang(?m_director) = 'en') .}R4R allows query results (e.g.m_title andm_director) to be available in the json template.
How? Easily, we can edit the fileget.json.vm to use the sparql query responses usingVelocity Template Language:
[ #foreach( $result in $results ) { "title" : "$result.m_title", "director" : "$result.m_director" } #if ( $velocityCount < ${results.size()} ) , #end #end]A new variable namedresults is always available from this template. It has all values retrieved in the sparql query so can be iterated to create a list of resources. In our example, a list of movies is create with two fields:title anddirector.
Now, a different json message is returned by doing the requesthttp://localhost:8080/movies
R4R allows you to make paginated queries by simply adding the query paramsize andoffset since they are special variables.
If we want the list of only 5 films, it will be enough to request it this way:http://localhost:8080/movies?size=5
and if we want the next page, enough with:http://localhost:8080/movies?size=5&offset=1
When considering paginated queries it is necessary to set theORDER option in the Sparql query.
Some fields may not always be available. For example, thebonusTracks field is accessible only in some movies.
Let's set it into the sparql query (resources/movies/get.sparql) by:
PREFIX dbo: <http://dbpedia.org/ontology/>PREFIX dbp: <http://dbpedia.org/property/>PREFIX res: <http://dbpedia.org/resource/>PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>PREFIX foaf: <http://xmlns.com/foaf/0.1/>SELECT ?m_uri ?m_title ?m_director ?m_budgetWHERE { ?m_uri rdf:type dbo:Film ; dbp:title ?m_title ; dbo:director ?director . ?director dbp:name ?m_director . FILTER (lang(?m_title) = 'en') . FILTER (lang(?m_director) = 'en') . OPTIONAL { ?m_uri dbo:budget ?m_budget } .}And we also load it into the JSON template (resources/movies/get.json.vm)to handle this conditional value:
[ #foreach( $result in $results ) { "title" : "$result.m_title", #if ($result.m_budget) "budget": "$result.m_budget", #end "director" : "$result.m_director" } #if ( $velocityCount < ${results.size()} ) , #end #end]The returned json only includes thebudget field when it has value by doing a request tohttp://localhost:8080/movies.
New fields can easily be generated without the need to request new information.
These fields are created from existing information, for example to indicate theID of a resource from its URI.
It would be enough to add the necessary operations in the JSON generation template (resources/movies/get.json.vm) as follows:
[ #foreach( $result in $results ) #set ( $index = $result.m_uri.lastIndexOf("/") ) #set ( $index = $index + 1) #set ( $id = $result.m_uri.substring($index, $result.m_uri.length())) { "id" : "$id", "title" : "$result.m_title", #if ($result.m_budget) "budget": "$result.m_budget", #end "director" : "$result.m_director" } #if ( $velocityCount < ${results.size()} ) , #end #end]These fields are available at:http://localhost:8080/movies
To filter by movie title, for example, just add the following condition to the sparql query (resources/movies/get.sparql):
PREFIX dbo: <http://dbpedia.org/ontology/>PREFIX dbp: <http://dbpedia.org/property/>PREFIX res: <http://dbpedia.org/resource/>PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>PREFIX foaf: <http://xmlns.com/foaf/0.1/>SELECT ?m_uri ?m_title ?m_director ?m_budgetWHERE { ?m_uri rdf:type dbo:Film ; dbp:title ?m_title ; dbo:director ?director . ?director dbp:name ?m_director . FILTER (lang(?m_title) = 'en') . FILTER (lang(?m_director) = 'en') . OPTIONAL { ?m_uri dbo:budget ?m_budget } . FILTER ( ?title = "_empty_" || regex(?m_title, ?title, "i") ) .}Now you can make requests like this:http://localhost:8080/movies?title=Games
Be careful when naming variables, because if you use the same name in the query field as the variable returned in the sparql query an error will occur.
The XSD type of the query parameter can be specified by adding the following suffixes to its name:
_dt: xsd:dateTime_d: xsd:double_i: xsd:integer_s: xsd:string_b: xsd:boolean
Thesort query param establishes the order of a solution sequence.
It contains a field name and an order modifier (either+ or-). Each ordering comparator is either ascending (indicated by the+ modifier or by no modifier) or descending (indicated by the-` modifier).
Internally, R4R adds an ORDER BY clause to the sparql query with the closest property (by using the Levenhstein distance) to the one specified in thesort field.
Now you can make requests like this:http://localhost:8080/movies?title=Games&sort=-name
In order to recover the information of a specific resource it is enough to add the following files:
Theresources/movies/getById.sparql file with the following content:
PREFIX dbo: <http://dbpedia.org/ontology/>PREFIX dbp: <http://dbpedia.org/property/>PREFIX res: <http://dbpedia.org/resource/>PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>PREFIX foaf: <http://xmlns.com/foaf/0.1/>SELECT ( ?id AS ?m_uri ) ?d_name ?m_country ?m_name ?m_budget ?m_releasedWHERE {?id dbo:director ?d_uri .?d_uri foaf:name ?d_name .OPTIONAL {?id dbp:country ?m_country} .OPTIONAL {?id dbo:budget ?m_budget} .OPTIONAL {?id foaf:name ?m_name} .OPTIONAL {?id dbp:released ?m_released} .}A variable?sid is also available with a short version of the id (i.e without the namespace)
And theresources/movies/getById.json.vm with this content:
{ "title": "$m_name", #if ($m_country) "country" : "$m_country", #end #if ($m_budget) "budget" : "$m_budget", #end #if ($m_released) "released" : "$m_released", #end "director" : "$d_name"}Now, you can get details about a movie by:http://localhost:8080/movies/WarGames
Sometimes the type of the resource is required to identify it, and adding the ID to the namespace is not enough:
https://eu.dbpedia.org/movies/WarGamesIn this scenario, R4R should be run with the environment variableRESOURCE_NESTED=True. In this way, the resource type is incorporated, together with the namespace and ID, to create its URI.
To request resources from a given one, simply add asubfolder with the template files.
For instance, to get the list of starring characters in the film it is enough to create the following files:
resources/movies/characters/get.sparql:
PREFIX dbo: <http://dbpedia.org/ontology/>PREFIX dbp: <http://dbpedia.org/property/>PREFIX res: <http://dbpedia.org/resource/>PREFIX dbr: <http://dbpedia.org/resource/>PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>PREFIX foaf: <http://xmlns.com/foaf/0.1/>PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>SELECT ?name ?birthDate ( ?starring AS ?uri )WHERE {?id dbo:starring ?starring .?starring foaf:name ?name .?starring dbo:birthDate ?birthDate .OPTIONAL {?name rdfs:label ?string . FILTER (lang(?string) = 'en') }}resources/movies/characters/get.json.vm:
[ #foreach( $result in $results ) { "name" : "$result.name", "birthDate" : "$result.birthDate" } #if ( $velocityCount < ${results.size()} ) , #end #end]That list can then be obtained byhttp://localhost:8080/movies/WarGames/characters
A HTTP basic authentication can be defined by only adding a (list of)user:password pair(s) in the environment variableAPI_USERS as follows:
docker run -it --rm \ -p 8080:7777 \ -v "$(pwd)/resources:/resources" \ -e "SPARQL_ENDPOINT=http://dbpedia.org/sparql" \ -e "RESOURCE_NAMESPACE=http://dbpedia.org/resource/" \ -e "API_USERS=user1:pwd1;user2:pwd2" \ cbadenes/r4r:latestNow, the request tohttp://localhost:8080/movies require a user name ( e.guser1) and a password (e.gpwd1) to be performed. These values have been defined in that environment variable.
curl -u user1:pwd1 http://localhost:8080/movies
Static HTML can be used to document API Rest. All files inresources/doc folder are available from a browser.
A Swagger interface can be created by describing our services in a YAML file, as follows:
swagger:'2.0'info:description:API documentation.version:1.0.0title:Swagger DBpedia MoviestermsOfService:'http://swagger.io/terms/'contact:email:cbadenes@fi.upm.eslicense:name:Apache 2.0url:'http://www.apache.org/licenses/LICENSE-2.0.html'host:localhost:8080basePath:/schemes: -httppaths:'/movies':get:summary:Gets a list of moviesoperationId:getMoviesparameters: -name:sizein:querydescription:number of movie to returnrequired:falsetype:integer -name:offsetin:querydescription:page of movies to returnrequired:falsetype:integerresponses:'200':description:OKschema:type:arrayitems:type:objectproperties:uri:type:stringid:type:stringbonus:type:stringname:type:string'/movies/{id}':get:summary:Find movie by IDdescription:Returns a single movieoperationId:getMovieByIdproduces: -application/jsonparameters: -name:idin:pathdescription:ID of movie to returnrequired:truetype:stringresponses:'200':description:successful operationschema:$ref:'#/definitions/Movie''400':description:Invalid ID supplied'404':description:Movie not founddefinitions:Movie:type:objectrequired: -uri -director -countryproperties:uri:type:stringdirector:type:stringtitle:type:stringbudget:type:integercountry:type:stringwiki:type:stringabstract:type:stringreleased:type:string
Then, a static html description can be created from that description inswagger editor by selectingGenerate Client > html2 option .
A new file (index.html) is created and would be placed into theresources/doc folder. In this way, our API is described in:http://localhost:8080/doc/index.html.
It can be extended withwebhook to easily create HTTP endpoints (hooks) on your server, which you can use to execute configured commands.
For example, if you're using Github, you can use it to set up a hook that updates the resources for your R4R project on your staging server, whenever you push changes to the master branch of your project.
It would be enough to create thehooks.json file:
[ {"id":"update","execute-command":"/home/cbadenes/project/hook-git.sh","command-working-directory":"" }]And a script to run it fromnohup:
nohup webhook -hooks hooks.json -verbose> nohup.log2>&1&echo$!> nohup.pidtail -f nohup.log
Then, the configured command can be something like thishook-git.sh:
#!/bin/bashecho"Updating content"git pull origin master
Docker-compose allows to start R4R and Virtuoso together in one command line.
Describe both services in adocker-compose.yml as follows:
version:'3'services:r4r:image:cbadenes/r4rcontainer_name:r4renvironment:SPARQL_ENDPOINT:"http://virtuoso:8890/sparql"RESOURCE_NAMESPACE:"http://www.example.com"volumes: -./resources:/resourcesports: -"8080:7777"depends_on: -"virtuoso"virtuoso:image:tenforce/virtuosocontainer_name:virtuosoenvironment:SPARQL_UPDATE:"true"DBA_PASSWORD:"my-pass"DEFAULT_GRAPH:"http://www.example.com/my-graph"volumes: -./data:/dataports: -"8890:8890"
Then, run it by:$ docker-compose up
Virtuoso will be available at:http://localhost:8890, and R4R will be listening at:http://localhost:8080
We kindly ask that any published research making use of the R4R framework cites our paper listed below:
Badenes-Olmedo, C., Espinoza-Arias, P., Corcho, O.: R4R: Template-based REST API Framework for RDF Knowledge Graphs. In: ISWC 2021: 20th International Semantic Web Conference. Demo Track. CEUR Workshop Proceedings.
This research was supported by the Spanish national project Datos 4.0, and by the European Union's Horizon 2020 research and innovation programme under grant agreement No 780247:TheyBuyForYou.