Evaluating SIRF for Accessibility - Evaluating Accessibility

10.5 Evaluating Accessibility

10.5.3 Evaluating SIRF for Accessibility

The author performed initial tests on British Library (BL) dataset. British Library’s developers have provided a set of sample queries for the end users to use their SPARQL endpoint. The author has used these sample queries to validate the effectiveness of SIRF. The author compared the hard coded queries available on BL endpoint with the queries generated by the prototype to evaluate the quality and accuracy of results. The author further experimented with variations of individual queries to validate the accessibility of the data. The details of initial tests are below.

10. PROTOTYPE AND EVALUATION 10.5.3.1 Test No. 1

User Query:

“search for a book with ISBN 9780415435864” Sample SPARQL query from British Library

P R E F I X b i b o : < h t t p :// p u r l . org / o n t o l o g y / b i b o / > P R E F I X bio : < h t t p :// p u r l . org / v o c a b / bio /0.1/ >

P R E F I X blt : < h t t p :// www . bl . uk / s c h e m a s / b i b l i o g r a p h i c / b l t e r m s # > P R E F I X dct : < h t t p :// p u r l . org / dc / t e r m s / >

P R E F I X e v e n t : < h t t p :// p u r l . org / NET / c 4 d m / e v e n t . owl # > P R E F I X f o a f : < h t t p :// x m l n s . com / f o a f /0.1/ >

P R E F I X geo : < h t t p :// www . w3 . org / 2 0 0 3 / 0 1 / geo / w g s 8 4 _ p o s # > P R E F I X i s b d : < h t t p :// i f l a s t a n d a r d s . i n f o / ns / i s b d / e l e m e n t s / > P R E F I X org : < h t t p :// www . w3 . org / ns / org # >

P R E F I X owl : < h t t p :// www . w3 . org / 2 0 0 2 / 0 7 / owl # >

P R E F I X r d a u : < h t t p :// r d a r e g i s t r y . i n f o / E l e m e n t s / u / > P R E F I X m a d s r d f : < h t t p :// www . loc . gov / m a d s / rdf / v1 # > P R E F I X rdf : < h t t p :// www . w3 . org / 1 9 9 9 / 0 2 / 2 2 - rdf - syntax - ns # > P R E F I X r d f s : < h t t p :// www . w3 . org / 2 0 0 0 / 0 1 / rdf - s c h e m a # > P R E F I X s k o s : < h t t p :// www . w3 . org / 2 0 0 4 / 0 2 / s k o s / c o r e # > P R E F I X xsd : < h t t p :// www . w3 . org / 2 0 0 1 / X M L S c h e m a # > P R E F I X v o i d : < h t t p :// r d f s . org / ns / v o i d # > S E L E C T ? b o o k ? bnb ? t i t l e W H E R E { ? b o o k b i b o : i s b n 1 3 " 9 7 8 0 4 1 5 4 3 5 8 6 4 " ; blt : bnb ? bnb ; dct : t i t l e ? t i t l e . }

Figure 10.9: Accessibility Test No. 1A - Results from British Library (Source: Author, 2015)

10. PROTOTYPE AND EVALUATION

The author tested following variations of the user query: Variation 1: “search for a book with ISBN 9780415435864” Variation 2: “book with ISBN 9780415435864”

Variation 3: “book ISBN 9780415435864”

All three variations generated the same query and results as given below.

S E L E C T D I S T I N C T * W H E R E {

{? B o o k a < h t t p :// p u r l . org / o n t o l o g y / b i b o / Book >}

{? B o o k < h t t p :// p u r l . org / o n t o l o g y / b i b o / isbn13 > ‘ 9 7 8 0 4 1 5 4 3 5 8 6 4 ’ } {? B o o k < h t t p :// p u r l . org / dc / t e r m s / title > ? B o o k _ t i t l e }

}

Figure 10.10: Accessibility Test No. 1B - Results from SIRF (Source: Author, 2015) Discussion:

Figure 10.10 shows that the natural language query tested with the prototype gets similar results to the hard coded query on British Library SPARQL endpoint (Figure10.9). SIRF extracts the CPI concepts (i.e. Book, ISBN, 9780415435864) and identifies the relation among these concepts. Testing the same query with different text (as described above) returned same result, hence validated the query generation algorithm.

10. PROTOTYPE AND EVALUATION 10.5.3.2 Test No. 2

User Query:

“Find serial with ISSN 0955-6664”

Sample SPARQL query from British Library

P R E F I X b i b o : < h t t p :// p u r l . org / o n t o l o g y / b i b o / > P R E F I X bio : < h t t p :// p u r l . org / v o c a b / bio /0.1/ >

P R E F I X blt : < h t t p :// www . bl . uk / s c h e m a s / b i b l i o g r a p h i c / b l t e r m s # > P R E F I X dct : < h t t p :// p u r l . org / dc / t e r m s / >

P R E F I X e v e n t : < h t t p :// p u r l . org / NET / c 4 d m / e v e n t . owl # > P R E F I X f o a f : < h t t p :// x m l n s . com / f o a f /0.1/ >

P R E F I X owl : < h t t p :// www . w3 . org / 2 0 0 2 / 0 7 / owl # >

P R E F I X r d a u : < h t t p :// r d a r e g i s t r y . i n f o / E l e m e n t s / u / > P R E F I X m a d s r d f : < h t t p :// www . loc . gov / m a d s / rdf / v1 # > P R E F I X rdf : < h t t p :// www . w3 . org / 1 9 9 9 / 0 2 / 2 2 - rdf - syntax - ns # > P R E F I X r d f s : < h t t p :// www . w3 . org / 2 0 0 0 / 0 1 / rdf - s c h e m a # > P R E F I X s k o s : < h t t p :// www . w3 . org / 2 0 0 4 / 0 2 / s k o s / c o r e # > P R E F I X xsd : < h t t p :// www . w3 . org / 2 0 0 1 / X M L S c h e m a # > P R E F I X v o i d : < h t t p :// r d f s . org / ns / v o i d # > S E L E C T ? s e r i a l ? bnb ? t i t l e W H E R E { ? s e r i a l b i b o : i s s n " 0 9 5 5 - 6 6 6 4 " ; blt : bnb ? bnb ; dct : t i t l e ? t i t l e . }

Figure 10.11: Accessibility Test No. 2A - Results from British Library (Source: Author, 2015)

10. PROTOTYPE AND EVALUATION

SPARQL Query generated by the Prototype The author tested following variations of the user query: Variation 1: “Find book with ISSN 0955-6664”

Variation 2: “book with ISSN 0955-6664” Variation 3: “book ISSN 0955-6664”

All three variations generated the same query and results as given below.

S E L E C T D I S T I N C T * W H E R E {

{? P e r i o d i c a l a < h t t p :// p u r l . org / o n t o l o g y / b i b o / P e r i o d i c a l >} {? P e r i o d i c a l < h t t p :// p u r l . org / o n t o l o g y / b i b o / issn > ’0955 -6664 ’} {? P e r i o d i c a l < h t t p :// p u r l . org / dc / t e r m s / title > ? P e r i o d i c a l _ t i t l e } }

Figure 10.12: Accessibility Test No. 2B - Results from SIRF (Source: Author, 2015) Discussion:

The above test validated the effectiveness of the prototype to find qualified concepts. The author tested the above query with deliberate non-existing concepts. The query asks for a book with ISSN “0955-6664” whereas an ISSN is linked to periodicals or journals. Figure10.12shows that the natural language query tested with the prototype gets similar results to the hard coded query on British Library SPARQL endpoint (Figure 10.11) although the query was only partially matched. The Concept Mapper does not find any relation for “Book” hence ignores it. The Query Generator generates query based on the qualified concepts (i.e. ISSN and “0955-6664”).

10. PROTOTYPE AND EVALUATION 10.5.3.3 Test No. 3

User Query:

“Find books on a subject e.g. crystallography” Sample SPARQL query from British Library

P R E F I X b i b o : < h t t p :// p u r l . org / o n t o l o g y / b i b o / > P R E F I X bio : < h t t p :// p u r l . org / v o c a b / bio /0.1/ >

P R E F I X blt : < h t t p :// www . bl . uk / s c h e m a s / b i b l i o g r a p h i c / b l t e r m s # > P R E F I X dct : < h t t p :// p u r l . org / dc / t e r m s / >

P R E F I X e v e n t : < h t t p :// p u r l . org / NET / c 4 d m / e v e n t . owl # > P R E F I X f o a f : < h t t p :// x m l n s . com / f o a f /0.1/ >

P R E F I X owl : < h t t p :// www . w3 . org / 2 0 0 2 / 0 7 / owl # >

P R E F I X r d a u : < h t t p :// r d a r e g i s t r y . i n f o / E l e m e n t s / u / > P R E F I X m a d s r d f : < h t t p :// www . loc . gov / m a d s / rdf / v1 # > P R E F I X rdf : < h t t p :// www . w3 . org / 1 9 9 9 / 0 2 / 2 2 - rdf - syntax - ns # > P R E F I X r d f s : < h t t p :// www . w3 . org / 2 0 0 0 / 0 1 / rdf - s c h e m a # > P R E F I X s k o s : < h t t p :// www . w3 . org / 2 0 0 4 / 0 2 / s k o s / c o r e # > P R E F I X xsd : < h t t p :// www . w3 . org / 2 0 0 1 / X M L S c h e m a # > P R E F I X v o i d : < h t t p :// r d f s . org / ns / v o i d # > S E L E C T ? b o o k ? i s b n ? t i t l e W H E R E { ? b o o k dct : s u b j e c t < h t t p :// bnb . d a t a . bl . uk / id / c o n c e p t / l c s h / C r y s t a l l o g r a p h y >; b i b o : i s b n 1 3 ? i s b n ; dct : t i t l e ? t i t l e . }

Figure 10.13: Accessibility Test No. 3A - Portion of results from British Library (Source: Author, 2015)

10. PROTOTYPE AND EVALUATION

SPARQL Query generated by the Prototype System The author tested following variations of the user query: Variation 1: “Find books on Crystallography”

Variation 2: “books on Crystallography” Variation 3: “Crystallography”

All three variations generated the same query and results as given below.

S E L E C T D I S T I N C T * W H E R E { {? B o o k a < h t t p :// p u r l . org / o n t o l o g y / b i b o / Book >} {? S u b j e c t < h t t p :// www . w3 . org / 2 0 0 0 / 0 1 / rdf - s c h e m a # label > " C r y s t a l l o g r a p h y "} {? B o o k < h t t p :// p u r l . org / dc / t e r m s / subject > ? S u b j e c t } {? B o o k < h t t p :// p u r l . org / dc / t e r m s / title > ? B o o k _ t i t l e } }

10. PROTOTYPE AND EVALUATION

Figure 10.14: Accessibility Test No. 3B - Results from SIRF (Source: Author, 2015) Discussion:

The above query tests (Figure 10.14) the capacity of the CPI Binder to find missing concepts. The Concept Mapper finds CPI available concepts (i.e. book, crystallography) and calculates the missing concept (i.e. subject) that links “Book” with “crystallography”.

10.5.3.4 Test No. 4

User Query:

“titles by detective writer Ian Rankin”

Sample SPARQL query from British Library

10. PROTOTYPE AND EVALUATION

P R E F I X bio : < h t t p :// p u r l . org / v o c a b / bio /0.1/ >

P R E F I X blt : < h t t p :// www . bl . uk / s c h e m a s / b i b l i o g r a p h i c / b l t e r m s # > P R E F I X dct : < h t t p :// p u r l . org / dc / t e r m s / >

P R E F I X e v e n t : < h t t p :// p u r l . org / NET / c 4 d m / e v e n t . owl # > P R E F I X f o a f : < h t t p :// x m l n s . com / f o a f /0.1/ >

P R E F I X owl : < h t t p :// www . w3 . org / 2 0 0 2 / 0 7 / owl # >

In document An integrated architecture for semantic search (Page 148-160)