Dissecting Big Data and The Cloud in
Capital Markets
December 2014
Agenda
• Some Background and Drivers
• Big Data Baby Steps
• The Real Capital Markets Cloud
• Recommendations
Agenda
• Some Background and Drivers
• Big Data Baby Steps
• The Real Capital Markets Cloud
• Recommendations
FS Technology No Longer A Leader
Navigational DBs: • Mainframe • COBOL • GML • DBs - IBM IMS 1970 1980 1990 2000 2010 Relational DBs: • Mainframe RDBMS • DBs - Oracle, IBM SQL/DS • AutEx • SWIFT SQL: • SQL ANSI Standard • DBs - dBASE, IBM DB2, Sybase • Desktop PC • DOC, XLS • SGML • ISDA • Bloomberg Terminal The Server: • Unix, Windows • DBs - Microsoft SQL Server, MySQL • Web Servers - Apache, Microsoft IIS • HTML, PDF, XML • FIX Protocol • Google NoSQL: • DBs – Apache Cassandra, MarkLogic, MongoDB, Redis • Apache Hadoop • AWS • Microsoft SharePoint • SmartLogic • FIXML • FpML • G-20 Agreement • QuickFIX • RDF • RegNMS • SalesForce.com • SwapsWire Semantic Web: • Apache Hadoop HDFS • Dodd-Frank Act • FIBO • Knowledge Graph • HTML5 • SPARQL • Thomson Reuters Eikon • TriplestoresData Everywhere
API, .DOC, FIX, FIXML, FpML, HTML, HTML5, JDBC, JMS, ODBC, .PDF, SOAP, Text, XBRL, .XLS, .XML …
Semantic Search
Trading
AML FATCA KYC
COB Risk Management Compliance Surveillance Regulatory Compliance Investment Management Research Management Reference Voice Recordings Chat/ IM Logs Compliance Earnings/ .DOCs HTML RDBMS NoSQL Share Point ERP HR Text Web Servers Enterprise E-Mail Platforms Financial Analysis eDiscovery Queries & Search Data Management
Global Search for Alpha
Not too or not at all challenging 23% Somewhat challenging 37% Very challenging 33% Extremely challenging 7%
Q. In current market conditions, how challenging is alpha generation? (n=30)
Electronic Trading
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Adoption of Electronic Trading, 2001 to e2014
Equities Futures Options FX
Algorithmic Trading
0%
10%
20%
30%
40%
50%
60%
70%
2004
2005
2006
2007
2008
2009
2010
2011
2012
e2013
e2014
Projected Adoption of Algorithmic Trading by Asset Class, 2004 to e1014
Equities
Futures
FX
Options
Algorithmic Trading
25% 28% 33% 38% 45% 52% 60% 63% 64% 65% 65% 3% 7% 12% 18% 22% 25% 28% 32% 36% 39% 42% 1% 1% 2% 4% 9% 13% 16% 20% 24% 28% 31% 0% 10% 20% 30% 40% 50% 60% 70% 2004 2005 2006 2007 2008 2009 2010 2011 2012 e2013 e2014Projected Global Adoption of Algorithmic Trading by Region, 2004 to e2014
Agenda
• Some Background and Drivers
• Big Data Baby Steps
• The Real Capital Markets Cloud
• Recommendations
Big Data Components
Accuracy of data/data quality Scalability Business sponsorship of concept High data availability Centralization Expertise of data scientist Technology Timeliness of response for queriesBudget
Governance framework
(Big) Baby Steps
• Cool Tools – Commoditized and Open-Source
Software
• “Big Data” tools for not so big FS problems
• Mostly tactical
implementations in
specific areas
• Majority of firms
lack enterprise-wide
big data strategy… and
those that do are often
stuck in governance.
Big Data Survey Firm Types
Bank 32% Broker-dealer 18% Asset or wealth manager 27% Third-party administrator 14% Hedge fund 9%Respondent Firm Type (N=22)
Data Management Priority
High priority 55% Medium priority 36% Low priority 9%Internal Priority Level for Data Management and Analytics (N=22)
The Current Data IT Stack
32% 45% 45% 45% 50% 55% 59% 59% 77% NoSQL databases Hadoop Open-source databases Time series database CEP technology Data quality monitoring tools Standard data integration tools Business intelligence tools Standard relational databasesQ. What kinds of data management technology does your organization have in place?
Who’s invested in Big Data?
Yes 50% No 41% Are actively considering 9%Q. Does your organization currently have a big data initiative or strategy in place?
Big Data is in the house… why?
10 9 9 8 7 7 7 7 3 2 1 2 3 3 4 4 4 3 6 7 1 2 1 2 1 2 2 3 3 3 1 1 1 1 Faster data analyticsBetter insight System scalability Granular drilldowns Proactive compliance Better predictive ability Faster time to market Metadata support Documentation tagging E-discovery
Big Data Benefits of Importance (n=13)
Big Data doubters… why not?
1 1 2 2 3 5Don't know enough about it Budget allocated elsewhere Don't understand the benefits No need to use it Perceived to be too expensive
Lack of business
support/executive commitment
Q. Why is your organization not considering a big data technology approach?
Who owns Big Data?
Data management 37% IT Operations 18% Trading 9% Research and development 9%Respondent Job Function (N=22)
Who knows Big Data?
Data Scientists
Statistical skills Programming skills Knowledge of financial marketsWho’s hiring (Big) Data Scientists?
No, no plans to employ 45% Yes, currently employ 41%No, but plan to employ in the next
24 months 9%
What is a data scientist?
5%
Q. Do you employ a data scientist? (N=22)
14% 23% 32% 32% 45% 45% 50% 55% 59% 64% 73% Other Voice and messaging records Legal entity data Records retention Instrument data Performance data Corporate actions data Positional or transactional data Regulatory compliance data Risk data Market data
Q. Which data sets do you believe are best suited to a big data approach at your organization?
(N=22)
Big Data Usage
1 3 4 4 5 5 5 5 6 6 6 7 8 Log data Reference or market dataPositional or transactional …
Strategy development and …
Marketing Regulatory reporting support Market surveillance or fraud Cost reduction Risk management or modeling Client management Revenue optimization Quantitative research Analytics for trading
Q. In which areas is your firm using or planning to use big data technology?
Big Data Challenges
2 2
4
6
9
Technical problems Other Inadequate
business capabilities of
technology
Data privacy issues Inadequate
technical knowledge Challenges Faced During Big Data Projects
What’s next for Big Data?
2 4 4 5 7 7 7 8 8 8 9 9 9 Other Development and testing Revenue optimization Marketing Market surveillance Reference or market data Cost reduction Analytics for trading Risk management or modeling Quantitative research Positional or transactional data Client management Regulatory reporting supportQ. In which of the following areas do you think your organization would consider using big data technology?
Agenda
• Some Background and Drivers
• Big Data Baby Steps
• The Real Capital Markets Cloud
“The Cloud”
Generally Smarter
“The Cloud”
No-Brainer???
Capital Markets Cloud Drivers
• Commoditization facilitating going off-premises.
• Exchanges are exiting infrastructure provision.
(NYSE Technologies valued at $0.)
• Global Search for Alpha
• Colocation reqs. making raw connectivity alone
less relevant.
• 3
rd
-Party Datacenter Proliferation
Capital Markets Cloud Drivers
Exchange Infrastructure Business Model
Old
Transitional
New
Datacenters
Own/Build
Lease/Build
Lease Rackspace
in 3
rdParty
Datacenters
Connectivity
Own Network
3
rdParty Networks 3
rdParty Networks
and POPs
Expenses
High
Medium
Low
Revenue Goals
High
Low
Low to None
IT Offerings
Many
Medium
Medium to None
Market Data Fees High
Medium
Medium
Cloud Survey Firm Types
Broker-dealer/investment bank Asset manager 10% Exchange/ATS 5% Breakdown of Respondents (n=21)Who’s invested in The Cloud?
Yes 52% No
48%
Q. Does your firm currently have cloud initiatives in place? (n=21)
Where’s The (Capital Markets) Cloud?
3
rd
-Party Datacenters
Public cloud 37% Private cloud Combination of both 36%Type of Cloud Services In Use (n=11)
What’s The Capital Markets (3
rd
-party
datacenter hosted) Private Cloud?
• Single-tenant architecture + Encryption + Shared
infrastructure hosted in independent
market/connectivity-agnostic 3rd-party datacenters
• Datacenter Best Practices
• Reasonably Economical Cross-Connects
• POPs
•
Connectivity Providers
•
Exchanges
•
The Cloud (AWS, Azure, Google)
•
Hosted & ASP Software
House in The Cloud… why?
2 2 4 3 4 3 3 5 4 4 3 7 5 3 2 2 2 Reducing TCO System scalability Disaster-recovery and … Faster time to market High availability Moving from capex to opexQ. Rank the importance of benefits that you may get from the cloud (n=11)
House in The Cloud… why?
10% 10% 10% 19% 19% 24% 24% 29% 29% 29% 48% Other Need for a more structured approach to …Strategic consolidation post-merger Coping with a greater variety of … Risk management function support
Regulatory initiatives Migration from legacy systems Client data focused initiatives Strategic data quality related programs Desire to rationalize data feeds/vendor …
Cost reduction
Q. Which factors have or are likely to influence your use of cloud technology for data?
Not so fast… why not?
50% 40% 30% 30% 10% Lack of control Performance issues Reliability and availability Regulatory restrictions ondata storage Other
Concerns about Cloud Initiatives (n=10)