The following information describes functions belonging to the FEED group.
[prefix_]preProcessFeed
Use this function to perform any operations at the start of processing a feed. If paging is performed using the functions getFeedUrlParameters and getNextUrlParameters, this function is only called for the first page. If paging parameters are set in the URL by the getFeedUrls function and updated using the getNextUrls or getNextUrlXPaths, then this function will be called for each page.
Inputs: xml (XmlDocument)
Outputs: [NONE]
See also: [prefix_]getFeedUrlParameters, on the previous page
[prefix_]getNextUrlParameters, on page 107 getFeedUrls, on page 102
[prefix_]getNextUrls, on page 108 [prefix_]getNextUrlXPaths, on page 108
Example
function myPrefix_preProcessFeed(xml) if (get_feed_param("LATEST_ID") == nil)
set_feed_param("LATEST_ID") = xml:XPathValue("/atom:feed/atom:entry/atom:id") end
end
[prefix_]postProcessFeed
Use this function to perform any operations at the end of processing the feed. If paging is performed using the functions getFeedUrlParameters and getNextUrlParameters, this function is only called after the last page. If paging parameters are set in the URL by the getFeedUrls function and updated using the getNextUrls or getNextUrlXPaths, then this function will be called after each page.
Inputs: xml (XmlDocument)
Outputs: [NONE]
See also: [prefix_]getFeedUrlParameters, on page 103 [prefix_]getNextUrlParameters, on page 107 getFeedUrls, on page 102
[prefix_]getNextUrls, on page 108 [prefix_]getNextUrlXPaths, on page 108
Example
function myPrefix_postProcessFeed(xml)
local latestid = get_feed_param("LATEST_ID") set_datastore_feed_param("LATEST_ID", latestid) end
[prefix_]getFeedModifiedDate
Use this function to return the modified date of the current feed. If the function is defined, the returned date will be stored in the synchronize datastore, and will be used to check whether the feed has been updated and whether to process any entries.
NOTE:
If the date is updated insufficiently frequently or too frequently this function should not be defined. For example, too frequently would be that the feed modified date is generated by the
server to always be the current time. An example of insufficiently frequently would be that the feed modified date is not updated when minor changes are made to an entry in the feed that are required to be indexed. Ideally, the modified date should be updated precisely when any data that is required to be indexed is changed.
Inputs: xml (XmlDocument) Outputs: modifiedDate (string)
See also:
Example
function myPrefix_getFeedModifiedDate(xml) return "2011-11-11T11:11:11"
end
[prefix_]getFeedModifiedDateXPath
Use this function to return an XPath that specifies the modified date of the feed. If the function is defined and getFeedModifiedDate is not defined or did not return a date, the returned date will be stored in the synchronize datastore, and will be used to check whether the feed has been updated and whether to process any entries. If the date is updated insufficiently frequently or too frequently this function should not be defined.
NOTE:
If the date is updated insufficiently frequently or too frequently this function should not be defined. For example, too frequently would be that the feed modified date is generated by the server to always be the current time. An example of insufficiently frequently would be that the feed modified date is not updated when minor changes are made to an entry in the feed that are required to be indexed. Ideally, the modified date should be updated precisely when any data that is required to be indexed is changed.
Inputs: xml (XmlDocument) Outputs: xpath (string)
See also: [prefix_]getFeedModifiedDate, on the previous page
Example
function myPrefix_getFeedModifiedDateXPath(xml) cache_result()
return "/atom:feed/atom:updated"
end
[prefix_]getEntryXPaths
Use this function to return XPaths that represent the root of the entries in the feed with optional prefix.
All subsequent function calls relating to a particular feed entry would require the specified function prefix.
Inputs: xml (XmlDocument)
Outputs: xpaths (list:{string} or table:{string->string} or table:{string->list:{string}})
See also:
Example
function myPrefix_getEntryXPaths(xml) cache_result()
return { "/atom:feed/atom:entry", prefix1=xpath1, prefix2={xpath2, xpath3} } end
[prefix_]getNextUrlParameters
Use this function to return a list of additional parameters to be appended to the feed URL. These parameters are only used to perform the download, and are not used to identify the feed when using built in functions set_feed_param, get_feed_param, set_datastore_feed_param or get_
datastore_feed_param. This is the normal way that paging parameters should be updated.
Inputs: xml (XmlDocument)
Outputs: parameters (table:{string->string}) See also: set_feed_param, on page 98
get_feed_param, on page 98
set_datastore_feed_param, on page 99 get_datastore_feed_param, on page 100
Example
function myPrefix_getNextUrlParameters(xml) local nextpage = get_feed_param("PAGE", 1) + 1 set_feed_param("PAGE", nextpage)
return { return page=nextpage } end
[prefix_]getNextUrls
Use this function to return additional feed URLs to be processed with optional function prefixes. All subsequent function calls relating to a particular feed URL require the specified function prefix. This can be used to handle paging or to process different but related feeds.
Inputs: xml (XmlDocument)
Outputs: urls (list:{string} or table:{string->string} or table:{string->list:{string}})
See also:
Example
function myPrefix_getNextUrls(xml)
return { "http://...", prefix1="http://...", prefix2={"http:// ...",
"http://..."} } end
[prefix_]getNextUrlXPaths
Use this function to return XPaths referencing additional feed URLs to be processed with optional function prefixes. All subsequent function calls relating to a particular feed URL require the specified function prefix. This can be used to handle paging or to process different but related feeds.
Inputs: xml (XmlDocument)
Outputs: xpaths (list:{string} or table:{string->string} or table:{string->list:{string}})
See also:
Example
function myPrefix_getNextUrlXPaths(xml) cache_result()
return { "/atom:feed/atom:link[@rel='next']/@href", prefix1=xpath1, prefix2=
{xpath2, xpath3} } end
[prefix_]getSubEntryXPaths
Use this function to return XPaths that represent the root of the sub-entries in the feed entry with optional prefix. All subsequent function calls relating to a particular sub-entry would require the specified function prefix.
Inputs: xml (XmlDocument)
Outputs: xpaths (list:{string} or table:{string->string} or table:{string->list:{string}}) See also:
Example
function myPrefix_getEntrySubXPaths(xml) cache_result()
return { "~/atom:subentry", prefix1=xpath1, prefix2={xpath2, xpath3} } end