The role of the query agent in the COPEMSA model

The role of the query agent in the COPEMSA model can be summarized by figure 9.1 on page 142. In figure 9.1, the query agent receives an augmented user query from the user agent, an- notated in XML, as discussed in the previous chapter. The query agent submits the contextual information contained inside the query to the community agent, in an attempt to discover any specialized search services on the WWW that the community might be aware of for the contexts indicated in the query structure. If any specialized search services are discovered by the com-

munity agent, it downloads the engine’s location as well as its interface definition, and integrates the new found resource into its list of usable search engines.

The query agent then uses the information encapsulated in the query (i.e. keywords, non- keywords, contextual information, query type etc.) to construct one or more restructured queries for submission to various general and/or specialized search services on the WWW. Each search engine has an associated interface definition that defines how queries should be restructured for the specific engine. The restructured queries are then submitted to the search services known to the agent.

After the results are received from the various search engines, the query agent collates the dif- ferent results into a single ranked list for presentation and further processing. At this point the community agent is accessed once more for possible community ratings of web pages that are used in the ranking algorithm. When the collation and ranking operation is complete, the agent stores all links to the potential results in a topics-links database. Finally, the agent returns the most highly rated results to the user agent for initial selection by the user.

From the discussion above, it is clear that the query agent of this section will need to deal with a number of issues internally in order for it to meta-search the WWW effectively. As noted by Chen, Fan, Chow and Zeng any meta-search system has to typically deal with the following four issues [86]:

• Search engine selection, query and output format. The system must be able to choose which set of search engines to send queries to. Hundreds of general and specialized search engines exist on the WWW and it would be highly impractical to send every query to every known search engine [87]. The system must also then be able to understand the query syntax of each search engine selected in order to restructure queries for the specific engine. Lastly, the output formats supported by each of the the selected underlying search engines must also be taken into account.

• Results fusion and quality control. After results have been received from the various search services, the meta-search service should be able to collate the results according to some sort of combination policy (also called a fusion policy [83]). Simply fusing the results is unfortunately only half the solution. A quality meta-search service should also eliminate duplicate pages as well as poor and outdated pages as well.

Collaborative meta-search unit Query agent User agent Topics- links database Yahoo Google Northern light Lycos Specialized search engine World wide web Re s truct ured quer y Web search engines Res tru c tured query R estructured qu ery

Res tructured query

. . . 1: The query agent receives

augmented XML query from the user agent.

5: The Query agent returns the results obtained from the search engines for the query submitted in step 1 for initial selection by the user.

2b: The query agent uses the Information in the augmented XML query to construct restructured queries which are specific to a search engine.

2c: The query agent then submits the restructured queries to various general and specialized search engines on the WWW.

3: Results from the search engines are returned to the query agent for processing. 2a: The query agent submits

the contextual information inside the augmented XML query to the community agent in an attempt to discover search engines that are specialized for the contexts indicated in the query.

2a1: The community agent returns any specialized engines indicated relevant to a specific context submitted in 2b by the community, as well as the engine’s interface definition, R es_tru ctu_{r e} d q ue_{r y}

4: The query agent stores all

the URLs of websites deemed possible results by the query agent.

3b: Query agent checks for community ratings of received web pages through the community agent. If found, they are used in the results fusion and ranking process.

Community agent

Figure 9.1: Role of the query agent in the COPEMSA model.

• Post-processing and re-ranking of search results. Based on the returned results, the meta- search service should be able to process and re-rank the results according to its own ranking criteria.

• Uniform result presentation. The meta-search service should present the user with an unified user interface and display the results in an integrated and intuitive manner.

The issues listed above and how they can possibly be dealt with inside the query agent is dis- cussed in the following subsections.