Part IV - Appendixes
Chapter 15 External Data Access
Public Access Reports
Directly access FileCensus reports from an HTTP address without using the FileCensus Portal.
XML-RPC Data Access
How to use access FileCensus image information using XML Remote Procedure calls.
Excel Data Access
How to configure and use Excel to access FileCensus image information.
Contents
FileCensus has the capability to allow access to all of its image information from HTTP addresses, XML-RPC, and Excel.
This chapter details the configuration of each of these access methods and gives examples of their use.
Public Access Reports
Public Access reporting is a process of directly accessing FileCensus reports from an HTTP address without the need of logging in to the FileCensus primary server. By entering an address with several keyword commands, any combination of FileCensus reports, users, scopes, and search conditions can be accessed from a single browser address.
FileCensus Public Access reporting is a powerful feature with many benefits:
• Total independence of the FileCensus Portal
• Report access can be restricted to a set of FileCensus reports
• Users do not need an account to access the information
• The report is fully customizable with all the features of role and scope selection, and basic and advanced report conditions
• Reports can be locked into the selected role, scope, and report conditions.
The following sections discuss some recommended methods for using Public Access reporting.
HTTP Address Format
The HTTP address must include several commands before a report will run. A command is made up of a keyword, an equals sign, and a value. The server address and port number, must be followed by the keyword report, a question mark, and then the list of commands. Each command must be separated by an ampersand “&”. The table below lists the keywords which are used to run a FileCensus Public Access report.
The address of the FileCensus server must be supplied (including the port number), then the word “report”, followed by a question mark, then the keyword commands separated by ampersands.
As detailed in the table above, only the first four keywords (user, role, report, and scope) are required to run a FileCensus report from an HTTP address.
An example of a HTTP address is as follows:
http://127.0.0.1:3030/report?user=Public&role=Public:Access&
report=File Analysis:Browse&scope=DATA:All Servers
User
The name keyword is the name of the user from the FileCensus Users section of the Administration folder (see Chapter 6, Page 90 - Users) which has the Public Access checkbox enabled (see Chapter 6, Page 91 - Public Access). The user keyword must be supplied for the HTTP address command to work successfully.
The user name is case sensitive and must be entered in the correct case for the user command to work. The user keyword is also case sensitive and must be entered in lower case.
Role
The role keyword is the name of the role from the FileCensus Roles section of the Administration folder (see Chapter 6, Page 87 - Roles).
The role keyword must be supplied for the HTTP address command to work successfully.
The role value must be in the format of a fully qualified role, including the role name and the role folder hierarchy, with each of the entries separated by a colon. For example, if a role was several folder levels deep the role value may look similar to the following:
Public:SubSubFolder:SubFolder:RootFolder
The role name must be first, followed by each of the role folders starting with the immediate parent of the role name, then each parent folder, up to the root folder.
The role name and folders are case sensitive and must be entered in the correct case for the role command to work. The role keyword is also case sensitive and must be entered in lower case.
By entering information into the Filter section of the role the report can be forced to use a filter. If the user accessing the report adds their own filter conditions (see below) they will be added to the existing role filter with an “and”.
Keyword Value Example
user User name user=Public
Keyword Value Example
role Role role=Public:PublicAccess
Report
The report keyword is the name of the report from the FileCensus Reports folder (see Chapter 2, Page 18 - Reports). The report keyword must be supplied for the HTTP address command to work successfully.
The report value must in the format of the report section, e.g. File Analysis, a colon, and then the report name.
The report section and name are case sensitive and must be entered in the correct case for the report command to work. The keyword must be entered in lowercase.
Scope
The scope keyword is the name of the scope from the FileCensus Scopes section of the Administration folder (see Chapter 3, Page 28 -Scopes). The scope keyword must be supplied for the HTTP address command to work successfully.
The scope value must be in the format of a fully qualified scope, including the scope name and the scope folder hierarchy, with each of the entries separated by a colon. For example, if a scope was several folder levels deep the scope value may look similar to the following:
DATA_USERS:SubSubFolder:SubFolder:RootFolder
The scope name must be first, followed by each of the scope folders starting with the immediate parent of the scope name, then each parent folder, up to the root folder.
The scope name and folders are case sensitive and must be entered in the correct case for the scope command to work. The scope keyword is also case sensitive and must be entered in lower case.
Filter
The filter keyword is a command statement which is used to apply a filter to the report and is based on the advanced conditions section when running a FileCensus report with the conditions activated (see
Chapter 3, Page 42 - Advanced Options). The filter keyword does not need to be supplied in the HTTP address command for the report to run successfully.
The filter conditions are not case sensitive but the scope keyword is case sensitive and must be entered in lower case.
Keyword Value Example
When using an equals sign in a filter condition you must use a “%3D”
value instead, otherwise the filter command will not work. See the Filter section for a full description of using the “%3D”.
When using an equals sign in a filter condition, you must enter “%3D”
instead of the normal “=” value, because the syntax of the HTTP address will look for the next “=” sign as its identification of another keyword. The “%” indicates that the next value is a hex value, and the
“3D” is the hex value for “=”.
Heading
The heading keyword is used to place a text description in the Page Heading bar of the report (see Chapter 2, Page 20 - Page Heading Bar).
The heading keyword does not need to be supplied in the HTTP address command for the report to run successfully.
Keyword Value Example
filter conditions filter=name %3D ‘*.exe’
Keyword Value Example
heading any text heading=Largest EXE Files
XML-RPC Data Access
XML (eXtensible Markup Language) is fast becoming the defacto industry standard for information exchange. XML-RPC is a specification and a set of implementations that allows software running on disparate operating systems, to make remote procedure calls (RPC) over the Internet.
The RPC implementation of XML uses HTTP as the transport protocol and XML as the encoder of the information being transported. XML-RPC combines the benefits of a markup language in allowing complex data structures to be stored in a structured format with the advantages of remote procedure calling to transmit, process and return the data structures.
The XML-RPC client can be written in any language and on any required platform. You can write your own XML parser for processing the XML data structures returned from the FileCensus XML-RPC server, but there are many high quality XML parsers available on the market today, and many of those are free.
Because of the number of languages and platforms that can be used to access and process the data returned from FileCensus XML-RPC server, we have supplied a separate document, the “XML-RPC Reference Guide”, which discusses how to access the FileCensus XML-RPC server from some of the more popular languages (such as Java, C++, Perl, and Python) with examples for each language.
The XML-RPC Reference Guide is available in PDF format from the Intermine website:
www.intermine.com
Excel Data Access
Microsoft Excel has the ability to get external data from a web source.
In Excel, by selecting the “Data” menu, “Import External Data” sub-menu, and then selecting “Import Data”, you can select a query file (files with an “iqy” extension) to run the file against the FileCensus server.
The ability to access the FileCensus image data from Excel query files allows you to load the FileCensus report data into an Excel spreadsheet. The web query file format is discussed in the following sections.
Formatting an IQY File
An Excel web query file is simply a text file with a file extension of
“iqy”. The correct formatting of this file is critical to accessing data on the FileCensus server.
The file must contain three lines:
• the first line contains the word “WEB”
• the second line contains the number “1”
• and the third line contains the command to access the FileCensus server
As you can see from the list above, the third line of the IQY file is where the actual query is written. Except for a few small exceptions, the format of the Excel query is almost exactly the same as the formatting of the Public Access report command (see Chapter 15, Page 170 - Public Access Reports). The Excel query has an advantage over the Public Access report by being able to prompt the user for the parameters of the query, which is discussed in more detail in the following sections.
The available Excel query parameters are listed in the following table:
The address of the FileCensus server must be supplied (including the port number), then the word “query”, followed by a question mark, then the keyword commands separated by ampersands. Only the first four keywords (username, role, report, and scope) are required to run a FileCensus report from an Excel query file.
Keywords Required
An example of an Excel query file is as follows:
The username keyword is the name of the user from the FileCensus Users section of the Administration folder (see Chapter 6, Page 90 -Users). The username keyword must be supplied for the Excel query to work successfully.
The user name value is case sensitive and must be entered correctly for the user command to work. The “username” keyword is also case sensitive and must be entered in lower case.
Password
The Password keyword is the password of the user from the FileCensus Users section of the Administration folder (see Chapter 6, Page 90 - Users). The password keyword must be supplied for the Excel query to work successfully if the user does not have the Public Access checkbox enabled (see Chapter 6, Page 91 - Public Access).
If you do not want to display the password of the user in the IQY text file you can check the Public Access checkbox. Enabling Public Access removes the requirement for a password when accessing FileCensus via Excel queries.
Because the access for the user is now public, it becomes important to ensure the reports, scopes, and filters accessible by the user are configured correctly. For example, the FileCensus Administrator may wish to limit the user to only running the Duplicates report for a scope with a single server that the user is responsible for.
The password value is case sensitive and must be entered in the correct case for the user command to work. The password keyword is also case sensitive and must be entered in lower case.
Keyword Value Example
username User name username=Public
Keyword Value Example
password Password password=Hello123
Role
The role keyword is the name of the role from the FileCensus Roles section of the Administration folder (see Chapter 6, Page 87 - Roles).
The role keyword must be supplied for the Excel query to work successfully.
The role must have valid entries in the Access section (see Chapter 6, Page 89 - Access) for the role to be able to run the required FileCensus report. The Access section is a useful device for restricting the number and types of FileCensus reports that an external Excel query can run.
The role value must be in the format of a fully qualified role, including the role name and the role folder hierarchy, with each of the entries separated by a colon. For example, if a role was several folder levels deep the role value may look similar to the following:
Public:SubSubFolder:SubFolder:RootFolder
The role name must be first, followed by each of the role folders starting with the immediate parent of the role name, then each parent folder, up to the root folder.
The role name and folders are case sensitive and must be entered in the correct case for the role command to work. The role keyword is also case sensitive and must be entered in lower case.
By entering information into the Filter section of the role the report can be forced to use a filter. If the user accessing the report adds their own filter conditions (see below) they will be added to the existing role filter with an “and”.
Report
The report keyword is the name of the report from the FileCensus Reports folder (see Chapter 2, Page 18 - Reports). The report keyword must be supplied for the Excel query to work successfully.
The report value must in the format a single name of the FileCensus report. The available report are detailed in the following table:
Keyword Value Example
role Role role=Public:PublicAccess
Report Name Description
file.treetotalfiles Lists a summary report of the number and size of files in the a directory, including all of its sub-directories.
quota.hard Equivalent to the NetWare Quotas report (see Chapter 12, Page 154 - Quotas).
File Reports
System Reports
Report Name Description
file.ages Equivalent to the Space By Age (see Chapter 10, Page 140 - By Age)
file.bydate Similar to Space By Year (see Chapter 10, Page 138 - By Date) but also allows summary information by any date type, such as month, day, and minute file.bysize Equivalent to the Space By Size (see
Chapter 10, Page 137 - By Size) file.byuser Equivalent to the Space By User (see
Chapter 10, Page 139 - By User) file.compare Equivalent to the Change Files (see
Chapter 11, Page 145 - Files)
file.duplicates Produces a summary report of duplicated files based on the information from File -Dupicates (see Chapter 8, Page 124 - Duplicated)
file.duplicates.details Equivalent to the File Duplicates (see Chapter 8, Page 124 - Duplicated) file.largest Equivalent to the File Largest report (see
Chapter 8, Page 123 - Largest) file.list Lists the files in a directory
file.search Equivalent to the File Search report (see Chapter 8, Page 120 - Search)
file.totals Lists the totals the number of files and their size for a specific path
file.types Equivalent to the Space By Type report (see Chapter 10, Page 136 - By Type) file.types.selected Equivalent to the Path Profile report (see
Chapter 9, Page 132 - Profile)
Report Name Description
scopes Lists the scopes available to a specified role (see Chapter 6, Page 80 - Scopes) roles Lists the roles available to the user (see
Chapter 6, Page 87 - Roles)
Live Reports
Image Reports
The report section and name are case sensitive and must be entered in the correct case for the report command to work. The keyword must be entered in lowercase.
Scope
The scope keyword is the name of the scope from the FileCensus Scopes section of the Administration folder (see Chapter 6, Page 80 -Scopes).
The scope keyword must be supplied for the Excel query to work successfully. The scope value must be in the format of a fully qualified scope, including the scope name and the scope folder hierarchy, with each of the entries separated by a colon. For example, if a scope was several folder levels deep the scope value may look similar to the following:
DATA_USERS:SubSubFolder:SubFolder:RootFolder
The scope name must be first, followed by each of the scope folders starting with the immediate parent of the scope name, then each parent folder, up to the root folder. The scope name and folders are case sensitive and must be entered in the correct case for the scope command to work.
Report Name Description
live.latest This report is based on the information contained in the 'heartbeat' between the agents and the FileCensus server (see Chapter 7, Page 113 - Agent Reporting) and reports on a server level
live.volume This report is based on the information contained in the 'heartbeat' between the agents and the FileCensus server (see Chapter 7, Page 113 - Agent Reporting) and reports on a per volume level
Report Name Description
image.schema Lists all the tables in the specified image.
The scope must map to a single image, branches are not allowed.
image.table
Keyword Value Example
report report name report=file.largest
The scope keyword is also case sensitive and must be entered in lower case.
Filter
The filter keyword is a command statement which is used to apply a filter to the report and is based on the advanced conditions section when running a FileCensus report with the conditions activated (see
Chapter 3, Page 42 - Advanced Options). The filter keyword does not need to be supplied in the Excel query for the report to run successfully.
The filter conditions are not case sensitive but the scope keyword is case sensitive and must be entered in lower case.When using an equals sign in a filter condition, you must enter “%3D” instead of the normal
“=” value, because the syntax of the HTTP address will look for the next “=” sign as its identification of another keyword. The “%”
indicates that the next value is a hex value, and the “3D” is the hex value for “=”.
Columns
The columns keyword is a command statement which is used to restrict the columns returned from an Excel query to the list of columns supplied. The columns keyword does not need to be supplied in the Excel query for the report to run successfully.
The column values are case sensitive and must be entered in the correct case for the columns command to work. The columns keyword is also case sensitive and must be entered in lower case. If more than one column is to be displayed, each column value must be separated by a space.
Formatting Prompts
The previous section detailed how to format an Excel query in a IQY file using the standard FileCensus keyword commands. But by manually entering the keyword commands into the IQY file there is no way to allow the end user to modify the Excel query if an alternate set of results is required.
Keyword Value Example
scope scope name scope=DATA:AllServers
Keyword Value Example
filter conditions filter=name %3D ‘*.exe’
filter conditions filter=name %3D ‘*.exe’