Download to read offline
JN
Uploaded byJoachim Neubert
184 views
Wikidata as a hub for the linked data cloud
The document provides an overview of a tutorial presented at the DCMI Conference in Seoul regarding the usage and querying of Wikidata as a linking hub in the linked data cloud. It covers key concepts such as external identifiers, mapping relations, and the process of linking external data to Wikidata through tools like Mix-n-Match, while also addressing quality control and community involvement in Wikidata. Additionally, it outlines challenges in performance for federated SPARQL queries and emphasizes the importance of community consensus for decision-making in Wikidata.
Recommended
More Related Content
What's hot
Similar to Wikidata as a hub for the linked data cloud
More from Joachim Neubert
Recently uploaded
Wikidata as a hub for the linked data cloud
- 1.Wikidata as ahub for the linked data cloudWikidata as a hub for the linked data cloudTUTORIAL AT DCMI CONFERENCE, SEOUL, 2019-09-25TUTORIAL AT DCMI CONFERENCE, SEOUL, 2019-09-25Tom Baker, Joachim Neubert, Andra WaagmeesterSlides (partitially) at https://jneubert.github.io/wd-dcmi2019/#/
- 2.OverviewOverviewPart 1: Usingand querying WikidataPart 2: Wikidata as a linking hubPart 3: Applications based on WikidataPart 4: Wikidata usage scenariosScenariosIntro and detailsHands-on: Mix-n-matchQuality control tools and proceduresWikidata community
- 3.Wikidata as linkinghubWikidata as linking hub
- 4.The idea oflinking hubsThe idea of linking hubsConnect concepts via identifiers/URLsExisting hubs: , , ...Image by Jakob VossVIAF sameAs.org
- 5.Different linking propertiesDifferentlinking properties1. (datatype URL)generic link to URL in the meaning of skos:exactMatch2. : more than 4000 specialized properties (datatype external identifier)exact matchPxxxx
- 6.Examples for externalidentifiersExamples for external identifiersGND / VIAF identifiersgeogaphical entitiesproteinsSwedish cultural heritage objectsAfrican plantsbaseball playersTED conference speakers
- 8.Property definitionsProperty definitionssubjectitem for the propertyexamplesconstraints on values, cardinality, etc.: creates a clickable link for the IDstart at the property page, e.g., for the ISSN:formatter URLhttps://www.wikidata.org/wiki/Property:P236
- 9.
- 11.Beyond sameness -mapping relationsBeyond sameness - mapping relationsWikidata external ids imply "sameness" of linked conceptseven with geographic names, other mapping relations are required in somecases.examples:close matches, e.g., "Yugoslavia" (1918-1992) (Wikidata) ≅ "Yugoslavia (until1990)" (STW)related matches, e.g. a company and its founder
- 12.Mapping relation type(P4390)Mapping relation type (P4390)introduced after a community discussion in October 2017to be used as qualifier for external id entriesfixed value set - SKOS mapping relations (exact, close, broad, narrow, relatedmatch)
- 13.EXAMPLE AT ITEMEXAMPLEAT ITEM ASSESSMENT CENTERASSESSMENT CENTER
- 14.How does thatrelate to the Linked Data model?How does that relate to the Linked Data model?Internal data model and storage (Wikibase) is transformed to RDF for:RDF dumpsQuery Service
- 15.RDF linking fromWikidataRDF linking from Wikidata: linked data URIe.g., , (vs. formatter URL)linked external RDF resourcesplus ~950.000 relations to individual URIsformatter URI for RDF resourcehttp://sws.geonames.org/$1/https://www.geonames.org/$1List of 130+ relationships to external RDF datasets26+ millionexact match
- 16.Links in theRDF dumpsLinks in the RDF dumpsOutput has full URLs to external resources, however with Wikidata-specificproperties:This creates a hurdle for generic Linked Data browsers and tools - not evenis translated to skos:exactMatchwd:Q123 wdt:P234 "External-ID" ;wdtn:P234 <http://example.com/reference/External-ID>exactmatch
- 17.Federated SPARQL queriesFederatedSPARQL queriesExample use case: GND authority has information about theprofessions/occupations of people which is not known in Wikidata.So get that information dynamically from a GND SPARQL endpoint.Here, we are interested in economists, in particular.
- 18.From Wikidata toa remote endpointFrom Wikidata to a remote endpointFrom a remote endpoint to WikidataFrom a remote endpoint to Wikidata<== not working currentlyquery to WDQSquery to GND endpoint
- 19.Several points forattentionSeveral points for attentionDirection and sequence of statements often matters for performanceTo reach out from Wikidata, endpoints have to be ( )In the other direction, access is normally not restrictedSome federated queries get extremely slow, when large sets of bindings exist before the remoteservice is invokedbe sure to exclude variables bound to blank nodes ('unknown value' in Wikidata)approved full list
- 20.Further reading onWikidata/RDFFurther reading on Wikidata/RDF( )Critical comments/suggestions:RDF dump format (documentation)Waagmeester: Integrating Wikidata and other linked data sources -Federated SPARQL queries more examplesMalyshev et al.: Semantic Technology Usage in Wikipedia’sKnowledge GraphFreire/Isaac: Technical usability ofWikidatas linked data
- 21.Application process fora new propertyApplication process for a new propertyDouble-check, that the property does not already existPrepare a property proposal in the according section, e.g., Wikidata:Propertyproposal/Authority control
- 23.Hints for gettingit approved smoothlyHints for getting it approved smoothlyClearly lay out the motivation and planned use for the propertyProvide working examples (with the formatter URI you are suggesting)Be responsive to comments
- 24.Wikidata as auniversal linking hubWikidata as a universal linking hubeasy extensibility with new properties for external identifiersimmense fund of existing items, with the full set of SKOS mapping relations formore or less exact mappings to theseimmediate extensibility with new items
- 25.Linking content viaMix-n-MatchLinking content via Mix-n-Match
- 26.Mix-n-match is awidely used tool (by Magnus Manske) to link external databases,catalogs, etc. to existing Wikidata items (or to create new ones).
- 27.Example list:Example list:Newspapersand journalsNewspapers and journalsfrom the 20th Century Press Archivefrom the 20th Century Press Archive
- 32.Please navigate toour example catalogMix-n-match manual
- 33.TasksTasksLogin through WidarIn"Automatically matched" list:Connect matching itemsRemove non-matching entriesIn "Unmatched" list:Search for existing itemsCreate missing items ( )suggested properties
- 34.
- 35.Item creation "onthe go"Item creation "on the go"With Mix-n-match "New item": rudimentary, no referencesCustom list of prepared QuickStatements insert blocks ( from STWThesaurus for Economics - please don't mess with it, this is work in progress)Workflow-wise, use same sequence for M-n-m input and prepared insert blocksexample
- 36.Recommendations for itemcreationRecommendations for item creationPay attention to (much more relaxed thenWikipedias)Explain your plan and ask for feedback in theto make mass edits ( )Source every statement ( )Create input inCheck with a few statements, verify resultRun as batch, document input and batch URLWikidata's notability criteriaWikidata project chatApply for a bot account examplehintsQuickStatements text format
- 37.Matching from WDto the external database entriesMatching from WD to the external database entries
- 39.Normally requires anendpoint for the external source, where you can search forthe labels, aliases or other data of Wikidata itemsInsert statement for external id into Wikidata can be prepared for cut&paste oreven semi-automatic execution in QuickStatementsSome hints and linked code here
- 40.Import catalog datato Mix-n-MatchImport catalog data to Mix-n-Match
- 41.Prepare dataPrepare data...as tab-separated table (one line per record) with three columns1. identitfier2. name3. descriptionInput file for the example used earlier
- 42.Pay attention toPayattention todescription column: include everything useful for intelectual identificationorder: the sequence may help structuring your workflow (e.g., most used entriesfirst)
- 43.Load data viaweb interfaceLoad data via web interface... at https://tools.wmflabs.org/mix-n-match/import.php
- 48.Sync existing idsfrom WikidataSync existing ids from Wikidata
- 52.Quality control toolsand proceduresQuality control tools and proceduresPerception: Anybody can edit anything - so Wikidata is no reliable source ofknowledgeSeen as a threat for information systems based on Wikidataparticularly by some large Wikipedias (e.g., the English one)Basic policy to address this: Statements should be referenced
- 53.QA support foreditorsQA support for editorsContraint definition for propertiesraise warnings during data input, when, e.g.a format definition (ISBN, DOI etc.) is violateda supposedly unique identifier is added to more than one itemgenerated lists of constraint violations (e.g. )Constraints can be very helpful, but do not cover complex casesZDB ID format
- 54.More QA supportfor editorsMore QA support for editorsAdditional reports can be created via SPARQL queriesShape Expressions (ShEx) allow to define complex constraints and conformancechecksShEx PrimerHow to get started with Shape Expressions in Wikidata?
- 55.Revision control andpatrolingRevision control and patrolingVersioned edits and version controlManual and tool supported vandalism preventionWatchlistsAutomated flagging of suspect edits (e.g., "new editor deleteing statements")Technically very easy to revert editsSemi-protection or protection of oftenly-vandalized itemsPatroling
- 57.Automated tools forvandalism detectionAutomated tools for vandalism detectionFighting to keep up with rate of human edits in Wikidata (multiple per second)... requires reducing the manual workload, e.g. viaObjective Revision Evaluation Service ( )and other rule-based and machine-learning toolsWikidata Abuse FilterORES
- 58.Ongoing researchOngoing researchHeindorfet al.: Vandalism Detection in WikidataSarabadani et al.: Building automated vandalism detection tools forWikidata
- 59.The Wikidata communityTheWikidata communityEverybody can participateNo central "committee" or decision structureDesisions are made via discussion and community consensus
- 60.Main entry pointfor all kind of discussionsResolved discussions archived after 7 days -Beginner's questions welcome (but please try to find the answer online before,particularly in the , which has a search link to the help pages)Compared to Wikipedia, the overall atmosphere is constructive (thoughexceptions exist sometimes in some sub-communities)English is the lingua franca, but a few questions show up in other languages,and receive comments, tooProject chatProject chatsearchableFAQ
- 61.User page anduser talk pageUser page and user talk pageIntroduce yourself - especially if you work in a professional context withWikidata ( )Activate notifications to your email addressBe responsive to comments on your talk pageYou can address other users on their talk page, tooexample
- 62.Talk pages ofpropertiesTalk pages of propertiesQuestions on the use of a certain propertySuggestions for changes or enhancements of the property definition ( )Consider adding properties you are interested in to your watchlistexample
- 63.
- 65.Often a greatsource to find documentation about the community consensus incertain fieldsMany WikiProjects pages contain data structuring recommendations - see, e.g.,forCurrent WikiProjects on Wikidataperiodicals
- 66.Thank you -questions welcome!Thank you - questions welcome!Joachim Neubertj.neubert@zbw.euJneubert on WD