necesidades adicionales de información, buscando documentos relevantes en la Web u otras libre- rías electrónicas. Aprender mejores vocabularios es una manera de aumentar la percepción y la accesibilidad del material útil. Se propuso un método prometedor para identificar la necesidad de- trás de la consulta, lo cual es uno de los principales objetivos para muchos servicios y herramientas web actuales y futuras.
6.2.
Trabajo a futuro
Dentro de las limitaciones encontradas durante el desarrollo de esta tesis, la más importante resultó ser el tiempo de ejecución de los algoritmos presentados. La velocidad es un obstáculo muy grande a la hora de realizar una evaluación con usuarios y es un aspecto a tener en cuenta a futuro. Por otro lado, el tiempo límite de ejecución podría incluirse como un parámetro a ser definido por el usuario, indicando qué tanto está dispuesto a esperar por resultados o si en cambio, desea un determinado número de documentos novedosos sin importar el tiempo de espera. Otro aspecto que no fue abordado dentro de los objetivos y contribuciones de estas tesis es la determinación del contexto actual del usuario, que también es de especial interés al momento de realizar las evaluaciones con usuarios. En lugar de esto, en las evaluaciones presentadas, se utilizó un conjunto de términos extraídos de una página de un tópico dado o la descripción de un tópico realizada por un editor de una ontología temática. En la literatura existen diversos trabajos que abordan el tema del reconocimiento automático del contexto actual de un usuario [BSY95, Bha00, BL01, BSHB06].
Se está trabajando actualmente para aplicar el método propuesto para el aprendizaje de mejores vocabularios en otras tareas de IR, como la clasificación de texto. También se están analizando las distintas estrategias que ayudan a mantener al sistema enfocado en el contexto inicial, luego de que se han llevado a cabo varios pasos incrementales. Por otro lado, se espera adaptar la plataforma propuesta para evaluar otras aplicaciones de recuperación de información, tales como algoritmos de clasificación y clustering.
Se ampliará la plataforma de evaluación presentada en esta tesis con el propósito de ponerla a disponibilidad de la comunidad de IR, lo que resultará de gran utilidad a la comunidad científica del área, proveyéndola de una herramienta que permitirá analizar de manera objetiva la efectividad de nuevos métodos. Entonces, se diseñará un instrumento de evaluación para sistemas de IR basado en un gran número de tópicos y documentos obtenidos a partir de ontologías de tópicos, para luego
integrarlo con métodos de evaluación existentes y novedosos. En tal sentido será importante el uso de las nociones de similitud semántica y relevancia parcial incorporadas a partir de esta tesis. Como se mostró en los capítulos anteriores, la construcción de colecciones de prueba ha merecido especial atención del ámbito de la IR experimental, ya que analizar grandes colecciones de docu- mentos y juzgar su relevancia es una tarea sumamente costosa, especialmente cuando los documen- tos cubren tópicos diversos. A la luz de estas necesidades y dificultades, y a partir de ontologías de tópicos editadas por humanos, tales como ODP, hemos desarrollado, y esperamos seguir refinando, un marco de experimentación para la evaluación automática y semi-automática de sistemas de IR, aprovechando el número masivo de relaciones disponibles entre tópicos y documentos.
´Indice de figuras
2-1 Espacio de documentos ideal. . . 9
2-2 Componentes de una consulta booleana. . . 11
2-3 Una consulta y un documento representados en un espacio vectorial. . . 13
2-4 Matriz de asociación. . . 20
2-5 Descomposición en valores singulares de la matrizM~. . . 21
2-6 Descomposición en valores singulares reducida de la matrizM~. . . 21
2-7 Realimentación de relevancia. . . 26
2-8 Modificación de una consulta basada en la realimentación del usuario. . . 28
2-9 Porción de una taxonomía de tópicos. . . 40
2-10 Ejemplo de similitud semántica basada en un árbol. . . 42
2-11 Porción de una ontología de tópicos. . . 42
2-12 Ejemplo de una ontología. . . 43
3-1 Curva de Precisión-Cobertura. . . 64
3-2 Curva de rendimiento de un sistema para múltiples valores del parámetrok. . . 66
3-3 Curva interpolada promedio calculada utilizando la técnica de microevaluación. . . 68
3-4 Curva promedio calculada utilizando la técnica de macroevaluación. . . 68
5-1 Representación esquemática de la plataforma de evaluación. . . 98
5-2 Evolución de la similitud novedosa máxima. . . 104
5-3 Comparación de los tres métodos basada en Similitud Novedosa. . . 105
5-4 Comparación de los tres métodos basada en Precisión. . . 105
5-5 Comparación de los tres métodos basada en Precisión Semántica. . . 106
5-6 Medias e ICs del rendimiento de los tres métodos basados ensimS,P yPS. . . 107
5-7 Evaluación del AG mono-objetivo utilizando la métricasimcos. . . 108
Bibliograf´ıa
[ACR04] Giambattista Amati, Claudio Carpineto, and Giovanni Romano. Query Difficulty, Robustness and Selective Application of Query Expansion. In Advances in Information Retrieval, 26th European Conference on IR research, volume 2997 ofLecture Notes in Computer Science, pages 127–137, Berlin / Heidelberg, 2004. Springer.
[Adl86] Klaus-Peter Adlassnig. Fuzzy set theory in medical diagnosis.IEEE Transactions on Systems, Man and Cybernetics, 16(2):260–265, March 1986.
[AF77] Rony Attar and Aviezri S. Fraenkel. Local Feedback in Full-text Retrieval Systems. Journal of the ACM (JACM), 24(3):397–417, July 1977.
[AFJM95] Robert Armstrong, Dayne Freitag, Thorsten Joachims, and Tom Mitchell. WebWatcher: A Learning Apprentice for the World Wide Web. In AAAI Spring Symposium on Information Gathering from Heterogeneous, Distributed Environments, pages 6–12, Palo Alto, CA, USA, March 1995. AAAI Press.
[AMWW88] Helmut Alt, Kurt Mehlhorn, Hubert Wagener, and Emo Welzl. Congruence, similarity, and symmetries of geometric objects. Discrete and Computational Geometry, 3:237–256, 1988. [Aus63] David P. Ausubel. The psychology of meaningful verbal learning. Grune and Stratton, 1963. [AvR02] Giambattista Amati and Cornelis J. van Rijsbergen. Probabilistic models of information retrie- val based on measuring the divergence from randomness. ACM Transactions on Information Systems (TOIS), 20(4):357–389, October 2002.
[BCK08] Shyam Boriah, Varun Chandola, and Vipin Kumar. Similarity measures for categorical data: A comparative evaluation. InSIAM International Conference on Data Mining, pages 243–254, Atlanta, GA, USA, April 2008.
[Bel63] Nuel D. Belnap. An analysis of questions: Preliminary report. Scientific Report TM-1287, System Development Corporation, Santa Monica, CA, USA, June 1963.
[Bel00] Nicolas J. Belkin. Helping people find what they don’t know. Communications of the ACM, 43(8):58–61, 2000.
[BFJ96] Justin Boyan, Dayne Freitag, and Thorsten Joachims. A machine learning architecture for optimizing web search engines. InAAAI Workshop on Internet-based Information Systems, 1996.
[BGL+98] Charles H. Bennett, Péter Gács, Ming Li, Paul M. B. Vitányi, and Wojciech H. Zurek. Infor- mation distance. IEEE Transactions on Information Theory, 44(4):1407–1423, July 1998. [BH99a] Shannon Bradshaw and Kristian J. Hammond. Mining Citation Text as Indices for Docu-
ments. InProceedings of the ASIS 1999 Annual Conference, Washington, DC, USA, October 1999. Information Today, Inc.
[BH99b] Jay Budzik and Kristian J. Hammond. Watson: Anticipating and Contextualizing Information Needs. In62nd Annual Meeting of the American Society for Information Science, pages 727– 740, Medford, NJ, USA, 1999.
[Bha00] Krishna Bharat. Searchpad: explicit capture of search context to support web search. Com- puter Networks, 33(1-6):493–501, 2000.
[BHB01] Jay Budzik, Kristian J. Hammond, and Larry Birnbaum. Information Access in Context.
Knowledge-Based Systems, 14:37–53, 2001.
[BJCG03] Steven M. Beitzel, Eric C. Jensen, Abdur Chowdhury, and David Grossman. Using titles and category names from editor-driven taxonomies for automatic evaluation. InProceedings of the twelfth international conference on Information and knowledge management, CIKM ’03, pages 17–23, New York, NY, USA, 2003. ACM.
[BL01] Travis Bauer and David B. Leake. Wordsieve: A Method for Real-Time Context Extraction. In Varol Akman, Paolo Bouquet, Richmond Thomason, and Roger Young, editors,Modeling and Using Context, volume 2116 ofLecture Notes in Computer Science, pages 30–44. Springer Berlin / Heidelberg, London, UK, 2001.
[BLTZ03] Stefan Bleuler, Marco Laumanns, Lothar Thiele, and Eckart Zitzler. PISA – A Platform and Programming Language Independent Interface for Search Algorithms. In C. M. Fonseca, P. J. Fleming, E. Zitzler, K. Deb, and L. Thiele, editors,Evolutionary Multi-Criterion Optimiza- tion, volume 2632 of Lecture Notes in Computer Science, pages 494–508. Springer-Verlag, Berlin, 2003.
[BP98] Sergey Brin and Lawrence Page. The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems, 30(1–7):107–117, 1998.
[BP99] Daniel Billsus and Michael J. Pazzani. A personal news agent that talks, learns and explains. InProceedings of the third annual conference on Autonomous Agents, AGENTS ’99, pages 268–275, New York, NY, USA, 1999. ACM.
[Bra08] Roger B. Bradford. An empirical study of required dimensionality for large-scale latent se- mantic indexing applications. In CIKM ’08: Proceeding of the 17th ACM conference on Information and knowledge management, pages 153–162, New York, NY, USA, 2008. ACM. [BS76] Nuel D. Belnap and Thomas B. Steel. The Logic of Questions and Answers. Yale University
Press, New Haven and London, England, first edition, 1976.
[BS97] Marko Balabanovi´c and Yoav Shoham. Fab: content-based, collaborative recommendation.
Communications of the ACM, 40(3):66–72, 1997.
[BSHB06] Jay Budzik, Sanjay Sood, Kristian J. Hammond, and Larry Birnbaum. Context transforma- tions for just-in-time retrieval: Adapting the watson system to user needs. Technical Report NWU-EECS-06-21, Northwestern University, EECS Department, 2006.
[BSMS95] Chris Buckley, Amit Singhal, Mandar Mitra, and Gerard Salton. New Retrieval Approaches Using SMART: TREC 4. In Donna K. Harman, editor, Fourth Text REtrieval Conference (TREC-4), volume Special Publication 500-236, pages 25–48, Gaithersburg, MD, USA, 1995. National Institute of Standards and Technology (NIST).
[BSWZ03] Bodo Billerbeck, Falk Scholer, Hugh E. Williams, and Justin Zobel. Query expansion using associated queries. InProceedings of the twelfth international conference on Information and knowledge management, CIKM ’03, pages 2–9, New York, NY, USA, 2003. ACM.
BIBLIOGRAFÍA 119
[BSY95] Marko Balabanovi´c, Yoav Shoham, and Yeogirl Yun. An Adaptive Agent for Automated Web Browsing. Technical report, Stanford University, Palo Alto, CA, USA, 1995.
[BV00] Chris Buckley and Ellen M. Voorhees. Evaluating evaluation measure stability. InProcee- dings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ’00, pages 33–40, New York, NY, USA, 2000. ACM.
[BV04] Chris Buckley and Ellen M. Voorhees. Retrieval evaluation with incomplete information. In Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ’04, pages 25–32, New York, NY, USA, 2004. ACM.
[BV05] Chris Buckley and Ellen M. Voorhees. TREC: Experiment and Evaluation in Information Retrieval, chapter Retrieval System Evaluation, pages 53–78. MIT Press, 2005.
[BW97] Michelle Q. Wang Baldonado and Terry Winograd. SenseMaker: an information-exploration interface supporting the contextual evolution of a user’s interests. In Steven Pemberton, editor,
Proceedings of the SIGCHI conference on Human factors in computing systems, CHI ’97, pages 11–18, New York, NY, USA, March 1997. ACM.
[BYRN99] Ricardo A. Baeza-Yates and Berthier A. Ribeiro-Neto. Modern Information Retrieval. Addison-Wesley Longman Publishing Co. Inc., Reading, MA, USA, May 1999.
[CA05] Ben Carterette and James Allan. Incremental test collections. InCIKM ’05: Proceedings of the 14th ACM international conference on Information and knowledge management, pages 680–687, New York, NY, USA, 2005. ACM.
[CCB95] Charles L. A. Clarke, Gordon V. Cormack, and Forbes J. Burkowski. Shortest substring ranking (multitext experiments for trec-4). In Donna K. Harman, editor,Fourth Text REtrieval Conference (TREC-4), volume Special Publication 500-236, pages 295–304, Gaithersburg, MD, USA, November 1995. National Institute of Standards and Technology (NIST).
[CCT00] Charles L. A. Clarke, Gordon V. Cormack, and Elizabeth A. Tudhope. Relevance ranking for one to three term queries. Information Processing and Management, 36(2):291–311, 2000. [Cec10] Rocío L. Cecchini.Desarrollo de técnicas de computación evolutiva para soporte en minería
de datos y texto. PhD thesis, Universidad Nacional del Sur, Bahía Blanca, Argentina, Abril
2010. http://www.cs.uns.edu.ar/~cr/web/Thesis.pdf.
[CH79] William B. Croft and David J. Harper. Using probabilistic models of document retrieval without relevance information. Journal of Documentation, 35(4):285–295, 1979.
[CL01] Fabio Crestani and Mounia Lalmas. Logic and uncertainty in information retrieval. In Ma- ristella Agosti, Fabio Crestani, and Gabriella Pasi, editors,Lectures on information retrieval, volume 1980 of Lecture Notes In Computer Science, pages 179–206. Springer-Verlag, New York, NY, USA, first edition, May 2001. Third European Summer-School, ESSIR 2000, Re- vised Lectures.
[Cle62] Cyril W. Cleverdon. Report on the testing and analysis of an investigation into the comparative efficiency of indexing systems. Technical report, College of Aeronautics, Cranfield, England, 1962.
[Cle91] Cyril W. Cleverdon. The significance of the Cranfield tests on index languages. InProcee- dings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ’91, pages 3–12, New York, NY, USA, 1991. ACM.
[CLMB08] Rocío L. Cecchini, Carlos M. Lorenzetti, Ana G. Maguitman, and Nélida B. Brignole. Using genetic algorithms to evolve a population of topical queries. Information Processing and Management, 44(6):1863–1878, 2008.
[CLMB10] Rocío L. Cecchini, Carlos M. Lorenzetti, Ana G. Maguitman, and Nélida B. Brignole. Multi- objective Evolutionary Algorithms for Context-based Search.Journal of the American Society for Information Science and Technology, 61(6):1258–1274, June 2010.
[CMK66] Cyril W. Cleverdon, Jack Mills, and Michael Keen. Factors determining the performance of indexing systems, volume I, II. Aslib Cranfield Research Project, Cranfield, England, 1966. [CNGR08] Guihong Cao, Jian-Yun Nie, Jianfeng Gao, and Stephen Robertson. Selecting good expansion
terms for pseudo-relevance feedback. InProceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ’08, pages 243–250, New York, NY, USA, 2008. ACM.
[Coh96] William W. Cohen. Learning rules that classify e-mail. In Marti A. Hearst and Haym Hirsh, editors,AAAI Spring Symposium on Machine Learning in Information Access, pages 18–25, March 1996.
[Coo64] Clyde H. Coombs. A theory of data. John Wiley, New York, 1964.
[Coo71] William S. Cooper. A definition of relevance for information retrieval. Information Storage and Retrieval, 7(1):19 – 37, 1971.
[CPC98] Gordon V. Cormack, Christopher R. Palmer, and Charles L. A. Clarke. Efficient construction of large test collections. InProceedings of the 21st annual international ACM SIGIR confe- rence on Research and development in information retrieval, SIGIR ’98, pages 282–289, New York, NY, USA, 1998. ACM.
[CR96] Heting Chu and Marilyn Rosenthal. Search engines for the World Wide Web: A comparati- ve study and evaluation methodology. InAnnual Conference Proceedings (ASIS’96), pages 127–135, Medford, NJ, USA, October 1996. American Society for Information Science, In- formation Today, Inc.
[CS98] Liren Chen and Katia Sycara. WebMate: A Personal Agent for Browsing and Searching. InProceedings of the 2nd International Conference on Autonomous Agents and Multi Agent Systems, AGENTS ’98, pages 132–139. ACM Press, May 1998.
[CT91] Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley, 1991. [CvdBD99] Soumen Chakrabarti, Martin van den Berg, and Byron E. Dom. Focused crawling: a new
approach to topic-specific Web resource discovery.Computer Networks, 31:1623–1640, 1999. [CvR09] Di Cai and Cornelis J. van Rijsbergen. Learning semantic relatedness from term discrimina-
tion information. Expert Systems with Applications, 36(2):1860–1875, 2009.
[CW97] Isaac Cheng and Robert Wilensky. An Experiment in Enhancing Information Access by Na- tural Language. Technical Report CSD-97-963, EECS Department, University of California, Berkeley, CA, USA, July 1997.
[DAPM02] Kalyanmoy Deb, Samir Agrawal, Amrit Pratap, and T. Meyarivan. A fast and elitist mul- tiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation, 6(2):182–197, April 2002.
[DDL+90] Scott C. Deerwester, Susan T. Dumais, Thomas K. Landauer, George W. Furnas, and Ri- chard A. Harshman. Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science, 41(6):391–407, 1990.
[DFF03] Didier Dubois, Helene Fargier, and Philippe Fortemps. Fuzzy scheduling: Modelling fle- xible constraints vs. coping with incomplete knowledge. European Journal of Operational Research, 147(2):231–252, 2003.
[Dic45] Lee R. Dice. Measures of the amount of ecologic association between species. Ecology, 26(3):297–302, July 1945.
BIBLIOGRAFÍA 121
[Dum91] Susan T. Dumais. Improving the retrieval of information from external sources. Behavior, Research Methods, Instruments, and Computers, 23(2):229–236, 1991.
[EFHW93] David Ellis, Jonathan Furner-Hines, and Peter Willett. Measuring the degree of similarity between objects in text-retrieval systems.Perspectives in Information Management, 3(2):128– 149, 1993.
[EL93] David A. Evans and Robert G. Lefferts. Design and evaluation of the CLARIT-TREC-2 system. In Donna K. Harman, editor,Second Text REtrieval Conference (TREC-2), volume Special Publication 500-215, pages 137–150, Gaithersburg, MD, USA, 1993. National Insti- tute of Standards and Technology (NIST).
[EMTB02] Floriana Esposito, Donato Malerba, Valentina Tamma, and Hans-Hermann Bock. Analysis of Symbolic Data, Exploratory Methods for Extracting Statistical Information from Complex Data, chapter Classical resemblance measures, pages 139–152. Studies in Classification, Data Analysis, and Knowledge Organization. Springer-Verlag, 2002.
[FDD+88] George W. Furnas, Scott C. Deerwester, Susan T. Dumais, Thomas K. Landauer, Richard A. Harshman, Lynn A. Streeter, and Karen E. Lochbaum. Information retrieval using a singular value decomposition model of latent semantic structure. InProceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ’88, pages 465–480, New York, NY, USA, 1988. ACM.
[Fuh92] Norbert Fuhr. Probabilistic models in information retrieval. The Computer Journal, 35(3):243–255, June 1992.
[Gar84] Eugene Garfield. A Tribute to S. R. Ranganathan, the Father of Indian Library Science.
Current Contents, 7:3–7, February 1984.
[GGMW03] Prasanna Ganesan, Hector Garcia-Molina, and Jennifer Widom. Exploiting hierarchical do- main structure to compute similarity. ACM Transactions on Information Systems (TOIS), 21(1):64–93, 2003.
[GLLV05] Jade Goldstein, Alon Lavie, Chin-Yew Lin, and Clare Voss, editors. Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, Proceedings of the An- nual Meeting of the Association of Computational Linguistics, Ann Arbor, MI, USA, June 2005. The Association of Computational Linguistics.
[Gol89] David E. Goldberg. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Longman Publishing Co. Inc., Boston, MA, USA, 1989.
[Gru93] Thomas R. Gruber. A translation approach to portable ontology specifications. Knowledge Acquisition, 5(2):199–220, 1993.
[GSJ79] H. Gilbert and Karen Spärck Jones. Statistical bases of relevance assessment for the ‘ideal’ information retrieval test collection. Technical Report 5481, Computer Laboratory, University of Cambridge, British Library Research and Development, 1979.
[GVSSJ97] John S. Garofolo, Ellen M. Voorhees, Vincent M. Stanford, and Karen Spärck Jones. TREC- 6 1997 Spoken Document Retrieval Track Overview and Results. In Ellen M. Voorhees and Donna K. Harman, editors,Sixth Text REtrieval Conference (TREC-6), volume Special Publication 500-240, pages 83–91. National Institute of Standards and Technology (NIST), 1997.
[Ham50] Richard W. Hamming. Error detecting and error correcting codes. The Bell System Technical Journal, 29(2):147–160, 1950.
[Har92] Donna Harman. Relevance feedback revisited. InProceedings of the 15th annual interna- tional ACM SIGIR conference on Research and development in information retrieval, SIGIR ’92, pages 1–10, New York, NY, USA, 1992. ACM.
[Har95] Donna Harman, editor. The second Text Retrieval Conference (TREC-2), volume 31 ofInfor- mation Processing & Management. Elsevier, May-June 1995. Special Issue.
[HBC99] David Hawking, Peter Bailey, and Nick Craswell. ACSys TREC-8 Experiments. In Ellen M. Voorhees and Donna K. Harman, editors,Eighth Text REtrieval Conference (TREC-8), vo- lume Special Publication 500-246, pages 307–316. Department of Commerce and National Institute of Standards and Technology, October 1999.
[HBH+98] Eric Horvitz, Jack Breese, David Heckerman, David Hovel, and Koos Rommelse. The Lu- mière project: Bayesian user modeling for inferring the goals and needs of software users. InProceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, pages 256–265, Madison, WI, 1998. AAAI Press.
[HBS94] Kristian J. Hammond, Robin Burke, and Kathryn Schmitt. A case-based approach to know- ledge navigation. In Usama M. Fayyad and Ramasamy Uthurusamy, editors,Proceedings of the Workshop on Knowledge Discovery in Databases, KDD-94, pages 383–394. AAAI Press, July 1994.
[HC01] David Hawking and Nick Craswell. Overview of the TREC-2001 Web track. In Ellen M. Voorhees and Donna K. Harman, editors,Tenth Text REtrieval Conference TREC-2001, vo- lume Special Publication 500-250, pages 61–67, Gaithersburg, MD, USA, 2001. National Institute of Standards and Technology (NIST).
[Hei73] M. H. Heine. Distance between sets as an objective measure of retrieval effectiveness. Infor- mation Storage and Retrieval, 9(3):181–198, 1973.
[HGKI02] Taher H. Haveliwala, Aristides Gionis, Dan Klein, and Piotr Indyk. Evaluating strategies for similarity search on the web. InWWW ’02: Proceedings of the 11th international conference on World Wide Web, pages 432–442, New York, NY, USA, 2002. ACM.
[HO01] William Hersh and Paul Over, editors.Interactivity at the Text Retrieval Conference (TREC), volume 37 ofInformation Processing & Management. Elsevier, May 2001. Special Issue. [Hol75] John H. Holland. Adaptation in natural and artificial systems. Bradford Series in Complex
Adaptive Systems. The University of Michigan Press, Ann Arbor, MI, USA, 1975.
[HP06] Daqing He and Yefei Peng. Comparing two blind relevance feedback techniques. In Susan T. Dumais, Efthimis N. Efthimiadis, David Hawking, and Kalervo Järvelin, editors,Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR ’06, pages 649–650, New York, NY, USA, 2006. ACM. Poster Session.
[HV97] Donna Harman and Ellen M. Voorhees, editors.The Sixth Text REtrieval Conference (TREC- 6), number 500-240 in NIST Special Publication. NIST-DARPA, Government Printing Office, 1997.
[HVV+06] Angelos Hliaoutakis, Giannis Varelas, Epimeneidis Voutsakis, Euripides G. M. Petrakis, and Evangelos Milios. Information retrieval by semantic similarity. International Journal on Semantic Web and Information Systems, 2(3):55–73, 2006.
[Jac01] Paul Jaccard. Étude comparative de la distribution florale dans une portion des alpes et des jura.Bulletin del la Société Vaudoise des Sciences Naturelles, 37:547–579, 1901.