Improving Analytical Tools The Future of Interrogating Data London, October

(1)

Improving Analytical Tools – The Future of Interrogating Data London, October 26 2010

(2)

OrbisIP Technology Services

_Consultancy

_{Technology scouting and evaluation}

_{Product Distribution}

_{Technology Readiness Level enhancements}

_{Secure Software Development}

_{Horizon‐scanning & Technology watching}

_{Open innovation services}

(3)

Security Technology Clusters

Digital Forensics Biometrics Video Analytics Data Mining & Visualisation

Cryptography Network Security Internet Security Secure Software

Security Management & Architecture Data & Database Security Hardware, Embedded & Device Security Homeland Security Over 150 items of novel security technology exist in the OrbisIP Technology Tracker, available online

(4)

The OrbisIP Model

Sources of IP

Consumers of IP

IP Requirements IP Requirements Tech. Transfer IP Licensing Payment / Royalties Revenue Share University Research Labs National Research Labs SMEs Security Companies Primes Governments Advisory Board Technology Advisers Operations

(5)

SITC conference challenge

_{".... During the investigations into the London} bombings in 2005, 90,000 hard drives and video tapes from CCTV systems were seized, together with 100 computers, 500,000 pages of photocopying, 4000 exhibits, 70 telephones and 10,000 statements” _{“… a responsibility to capture, consolidate and} interrogate massive volumes of structured and unstructured data. How do you make sense out of all the data coming in and then use it to make a difference?” _{" ... technologies which can assist users in making} informed, strategic decisions."

(6)

The challenges of large scale data analysis

_{Data tracking, capture, management and}

isolation

_{Retrieval and analysis of large data sets}

_{Analytical impartiality}

_{Data interrogation}

_{Decision support}

_{Actionable intelligence}

_{Reportage and distribution}

(7)

Helping to address the challenges

_{Intercept Modernisation Programme ‐ IP data}

capture and interrogation –

Network Traffic

Surveillance System

_{High speed data retrieval and management –}

Clusterpoint XML data base

_{Decision Support and tracking technology –}

SheBa

_{Structured and Unstructured Data Analysis ‐}

Leximancer

(8)

Intercept Modernisation Programme

_{“Every email, phone call and website visit is to be recorded and} stored after the Coalition Government revived controversial Big Brother snooping plans.” The Telegraph 20 Oct 2010 _{"We will introduce a programme to preserve the ability of the} security, intelligence and law enforcement agencies to obtain communication data and to intercept communications within the appropriate legal framework … Communications data provides evidence in court to secure convictions of those engaged in activities that cause serious harm. It has played a role in every major Security Service counter‐terrorism operation and in 95 per cent of all serious organised crime investigations.” UK Government Strategic Defence and Security Review October 2010

(9)

Intercept Modernisation Programme

Technical challenges:

Mass capture and storage of data by ISPs

Gathering intelligence from data sets

_{Interrogate to level of all packets generated at IP}

address

_{Produce auditable interactions with the data set}

that conform to existing legislation and can be

submitted in court to support prosecution

(10)

Intercept Modernisation Programme

Technical challenges:

Mass capture and storage of data by ISPs

Gathering intelligence from data sets

_{Interrogate to level of all packets generated at IP}

address

_{Produce auditable interactions with the data set}

that conform to existing legislation and can be

submitted in court to support prosecution

(11)

NTSS collects data on ALL user network activities

Entirely searchable & scalable NTSS database

NTSS

1. All TCP & UDP traffic IP packets

between customer and Internet get forwarded to NTSS

2. IP packets are reengineered back to application level information

units (web pages viewed, e-mails sent, documents transferred).

3. All reengineered and analysed information is fully indexed and stored

in Clusterpoint Server database.

4. Easy to use WEB interface provides

necessary tools, to:

• get a quick situation overviews, • search through the collected data, • receive alerts on user defined criteria, • follow up on network user activities, • preview the reconstructed information.

Decoder Internet Traffic

(12)

High speed data retrieval and management

The issues: _{Size of data base and scalability} _{Retrieval or interrogation time} The solution – Clusterpoint DBMS – TRL 9 _{create a fully scalable and fast response time XML} based tagged database _{easily and lineary scalable ‐ no additional development} required to scale the storage and necessary processing power _{can improve data retrieval times in such unstructured} data storages 100‐fold and more with response times of sub 5 seconds in multi‐terabyte databases

(13)

SECURITY APPLICATION SERVICES Existing databases AUTHORIZED USERS Entirely searchable & scalable NTSS database SECURITY AUDITING AND MONITORING DATA

CLUSTERPOINT NTS ( MULTI-SERVER CLUSTER ) XML Clusterpoint API

Clusterpoint DBMS Architecture

(14)

Objective & Subjective Decision support

The Challenges: Capture, manage and analyse information on various topics of relevance, including multiple items of information supplied at different times, and from different sources; Consider factors that degrade the certainty of the information, such as the length of time that has passed between its collection and its use; Represent the reliability or credibility of information and its source, including its provenance, and the source’s objectivity, access, and specificity. Combine multiple sources of information with varying levels of reliability, and whose reliability may change over time;

(15)

Objective & Subjective Decision support

The Solution: Sheba Estimative Intelligence Tool – TRL 5 _{An application for performing predictive analysis under} conditions of uncertainty _{a framework for users to structure and analyse} estimative intelligence problems _{uses advanced probability theory to manage both} likelihoods and certainty for problems in estimative intelligence transparent analytical structure necessary to allow the analyst to defend, and the consumer to understand, the judgments reached

(16)

Structured & Unstructured Data Analysis

The challenge:

_{How do you extract fully‐automated meaningful} intelligence from vast scalable data sets of structured and unstructured data? _{Generating concept maps and an Automatic Sentiment} Lens _{Avoiding onerous set‐up overhead, user‐manipulation} or prejudices or priori rule‐sets? _{Providing multilingual analysis and integration into} other applications through an API Offering full flexibility for analysis of outcomes and exportability into reporting platforms

(17)

Structured & Unstructured Data Analysis

The solution:

Leximancer – TRL 9

Some applications:

_{Intelligence Profiling: rapid information gathering,} correlation, validation and analysis Email Analysis & Security: validating & predicting security classification of email _{i‐Library Indexing & Search: indexing, search and} retrieval within large databases/libraries _{Tendering: matching tender responses to questions ‐} coverage analysis _{Web searching and analysis: via Hypermancer}

(18)

Leximancer ‐ Conversation Analysis

As long as speaker labels are formatted correctly, Leximancer will automatically extract the speaker identifiers as variables, and associate these labels with their utterances. This allows content from selected speakers to be filtered in or out, and allows comparative analysis between speakers, normally using the discovered concepts as independent variables. Conversation analysis can be extended to incorporate: _{Blogging/forums etc. on the internet – brand/product} analysis etc. _{Email dialogues – litigation eDiscovery} _{Voice‐to‐text translations ‐ call centre dialogues,} meetings, scenario training/simulations

(19)

OrbisIP ‐ Contact Details

Peter Jaco [email protected] OrbisIP Limited 9‐10 St. Andrew Square Edinburgh, EH2 2AF United Kingdom Tel: +44 (0) 131 718 6023 Fax: +44 (0) 131 718 6100 Cell: +44 (0) 7855 308 290