{"id":71,"date":"2022-03-12T19:09:32","date_gmt":"2022-03-12T19:09:32","guid":{"rendered":"https:\/\/quxfarm.com\/?p=71"},"modified":"2022-03-12T19:09:39","modified_gmt":"2022-03-12T19:09:39","slug":"fulltext-searching-against-marcsparql-smarcql","status":"publish","type":"post","link":"https:\/\/quxfarm.com\/?p=71","title":{"rendered":"Fulltext Searching Against MARC+SPARQL (SMARCQL)"},"content":{"rendered":"\n

In my initial SMARCQL post<\/a>, I included a SPARQL query for the most frequently referenced author\/title<\/a> and pointed out how inconsistencies in MARC records “stick out like a sore thumb in SPARQL”. I also hinted that text searching, as opposed to querying, can help deal with some of those difficulties. To facilitate this in practice, Blazegraph supports a hybrid FullTextSearch extension<\/a> that is easily enabled<\/a>.<\/p>\n\n\n\n

To illustrate, here is a hybrid query that uses the text search extension to find references to the terms “lois chan”, regardless of order, adjacency, capitalization, and punctuation. The remainder of the query groups the resulting subfields by tag and code so the context and variances are a little more obvious.<\/p>\n\n\n\n

PREFIX tag: <https:\/\/w3id.org\/smarcql\/tag\/>\nPREFIX code: <https:\/\/w3id.org\/smarcql\/code\/>\nprefix bds: <http:\/\/www.bigdata.com\/rdf\/search#>\n\nSELECT ?tag ?code (COUNT(DISTINCT ?rec) AS ?numRecs) ?subfield\nWHERE {\n  ?rec ?tag ?field .\n  ?field ?code ?subfield .\n  ?subfield bds:search \"lois chan\" .\n  ?subfield bds:matchAllTerms \"true\" .\n  \n  FILTER(?code != rdfs:label)\n}\nGROUP BY ?tag ?code ?subfield\nORDER BY DESC(?numRecs)\nLIMIT 10<\/code><\/pre>\n\n\n\n
tag<\/th>code<\/th>numRecs<\/th>subfield<\/th><\/tr><\/thead>
tag:bd100<\/td>code:sa<\/td>176<\/td>Chan, Lois Mai.<\/td><\/tr>
tag:bd245<\/td>code:sc<\/td>75<\/td>Lois Mai Chan.<\/td><\/tr>
tag:bd700<\/td>code:sa<\/td>48<\/td>Chan, Lois Mai.<\/td><\/tr>
tag:bd776<\/td>code:sa<\/td>25<\/td>Chan, Lois Mai.<\/td><\/tr>
tag:bd245<\/td>code:sc<\/td>17<\/td>by Lois Mai Chan.<\/td><\/tr>
tag:bd245<\/td>code:sc<\/td>13<\/td>prepared by Lois Mai Chan for the Library of Congress.<\/td><\/tr>
tag:bd700<\/td>code:sa<\/td>12<\/td>Chan, Lois Mai<\/td><\/tr>
tag:bd245<\/td>code:sc<\/td>10<\/td>Lois Mai Chan and Richard Pollard.<\/td><\/tr>
tag:bd250<\/td>code:sb<\/td>9<\/td>by Lois Mai Chan.<\/td><\/tr>
tag:bd245<\/td>code:sc<\/td>9<\/td>Sharon Chien Lin ; forewords by Lois Mai Chan and Ching-Chih Chen.<\/td><\/tr><\/tbody><\/table>\n\n\n\n
The FullTextSearch extension also supports relevance (cosine similarity) and ranking (ordinal position) , which can be useful for reconciliation or autosuggestion purposes. Once the SMARCQL mapping<\/a> and ontology<\/a> are a little more fleshed out, I have some applications in mind to demonstrate more of these possibilities. <\/p>\n","protected":false},"excerpt":{"rendered":"
In my initial SMARCQL post, I included a SPARQL query for the most frequently referenced author\/title and pointed out how inconsistencies in MARC records “stick out like a sore thumb in SPARQL”. I also hinted that text searching, as opposed to querying, can help deal with some of those difficulties. To facilitate this in practice, […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"nf_dc_page":"","_crdt_document":"","om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[9,10,8],"tags":[],"class_list":["post-71","post","type-post","status-publish","format-standard","hentry","category-marc","category-smarcql","category-sparql"],"aioseo_notices":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/quxfarm.com\/index.php?rest_route=\/wp\/v2\/posts\/71","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/quxfarm.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/quxfarm.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/quxfarm.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/quxfarm.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=71"}],"version-history":[{"count":3,"href":"https:\/\/quxfarm.com\/index.php?rest_route=\/wp\/v2\/posts\/71\/revisions"}],"predecessor-version":[{"id":76,"href":"https:\/\/quxfarm.com\/index.php?rest_route=\/wp\/v2\/posts\/71\/revisions\/76"}],"wp:attachment":[{"href":"https:\/\/quxfarm.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=71"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/quxfarm.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=71"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/quxfarm.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=71"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}