20
CAP4773/CIS6930
Projects in Data Science, Fall 2014
[Review] Overview of Data Science
Dr. Daisy Zhe Wang CISE Department University of Florida
August 25th 2014
Review
Overview of Data Science
• Why Data Science?
• What is Data Science?
• What are some prominent examples of Data Science?
• How to become a Data Scientist?
• Who are hiring Data Scientists Now?
The Dawn of Big Data
• Google, Yahoo today
– Core Business: Web Search and Computational advertising – Google: 35,000 searches/sec
– Yahoo! scale: 600 million users per month, 4 billion clicks per day, 25 terabytes of data collected every day
• Netflix 2007
– Movie recommendations
– 100 million ratings, 500,000 users, 18,000 movies – netflix prize
• Amazon 2003
– Product recommendations, reviews
– 29 million customers, millions of products
• Word Economic Forum 2011 at Davos
– Personal data – digital data created by and about people – represents a new economic “asset class”, touching all aspects of society.
22
How Big is Your Data?
• Kilobyte (1000 bytes)
• Megabyte (1 000 000 bytes)
• Gigabyte (1 000 000 000 bytes)
• Terabyte (1 000 000 000 000 bytes)
• Petabyte (1 000 000 000 000 000 bytes)
• Exabyte (1 000 000 000 000 000 000 bytes)
• Zettabyte (1 000 000 000 000 000 000 000 bytes)
• Yottabyte (1 000 000 000 000 000 000 000 000 bytes)
It is not only about volumes!
5 Vs of Big Data
• Raw Data: Volume
• Change over time: Velocity
• Data types: Variety
• Data Quality: Veracity
• Information for Decision Making: Value
24
Multimodal Data Sources on
World Cup Common
~5M tweets with 244,000
associating images
News & Blogs 776 blog posts and associated
pictures
Youtube 460 minutes of
Flickr
913 images and associated
captions Facebook
250 posts and
associated images Google Images
1,920 images and associated
captions
Current Trends
• Applications has bigger data and need more advanced analysis
– Example: Web, Corporate documents and Emails
• Natural Language Processing
– Example: Social Media
• Network/Graph Analysis
• IT Infrastructure moving to Cloud Computing (2006)
– Parallel Computing, SaaS
– Elastic Computing – pay-as-you-go, scale up/down
• Data Science arise given this application pull and technology push (2011)
26
What is Data Science?
28
Data Science – A Definition
Data Science is the science which uses computer science, statistics and machine learning,
visualization and human-computer interactions to collect, clean, integrate, analyze, visualize, interact with data to create data products.
Goal of Data Science
Turn data into data products.
30
Data to Data Products
• Web Text Data Knowledge Bases with Facts & Relationships
• Twitter Text/Images Real-time Events and News
• Knowledge bases first-order inference rules and constraints
• Text Data, Social Media Data Product Review and Consumer Satisfaction
• Software Log Data Automatic Trouble Shooting
• Genotype and Phenotype Data common patterns New treatment for Cancer
Other Data Products
• Financial products for investment or retirement funds
• Legal profession uses e-discovery tool for retrieval and review of legal documents
• Political campaign management
• Sports (e.g., Oakland baseball team)
• Remote Sensing for Environment Monitoring
32
What are some prominent examples of Data Science?
Data Products – Google
• Web Search
• Google Ads
• News Recommendation Engine
• Google Maps
Currently one of the best if not the best IT
company to work for. (Google event on Jan 21/22)
34
Data Products – Netflix
• Personalized Movie Ratings
• Movie Recommendations
• Similar Movies
• Movie Categories (e.g., 80’s movie with a strong female lead, Kung Fu movies)
BlockBuster is out of the business …
Data Products – LinkedIn/Facebook
• People you may know
• Applications you may like
• Jobs/Events you might be interested
• Classifier for bad users and bad content
• With high accuracy, Facebook can guess whether you are single or married
Who does not have LinkedIn/Facebook Account?
36
Data Products – Twitter
• Text Analysis – Spam Filter/Similarity Search
• User Sentiment/Satisfaction/Feedback
• News Breakout
• Trend and Topics
200 million users as of 2011, generating over
200 million tweets and handling over 1.6 billion search queries per day
Data Products – Splunk
• Degradation, Failure Detection
• Identify Security Breach
• Event Monitoring
• Troubleshoot Tools
• Cross-platform Event Correlation Founded 2004, IPO in 2012
38
How to become a Data Scientist?
The Life of Data (state-of-the-art)
Collect Clean Integrate Visualization Interface
Users
Data Sources
Analysis
40
Challenges in Data Science
• Find Data (Data Collection, Quality)
• Prepare Data (Noisy, Incomplete, Diverse, Streaming …)
• Analyze Data (Scalable, Accurate, Real-time, Advanced Methods, Probabilities and
Uncertainties ...)
• Represent Analysis Results (i.e. data product) (Story-telling, Interactive, explainable…)
Skill Set of a Data Scientist
• Data Management
– Data collection, storage, cleaning, filtering, integration
…
• Large-scale Parallel Data Processing
– Parallel computing
• Statistics and Machine Learning
– Data modeling, inference, prediction, pattern recognition …
• Interface and Data Visualization
– HCI design, visualization, story-telling …
42
Who are hiring Data Scientists Now?
“Sexy Job” in the next 10 years
“The sexy job in the next ten years will be … The ability to take data—to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it—that’s going to be a
hugely important skill.”
-- Hal Varian, Google Chief Economist, 2009
44
Who’s hiring Data Scientist?
• IT companies: Google, Twitter, Lexis/Nexis, Facebook, Pivotal/EMC…
• Media and Financial sectors Fox, CNN, NYT, Bloomburg,…
• Research: Biology, Medicine, Physics, Psychology,…
• Information office in government and corporations…
• Law firms: e-discovery tools…
Books on Data Science
46
Additional Reading Pointers
• Data Science Summit (Strata)
(http://www.datascientistsummit.com/ )
• Kaggle Competitions
(http://www.kaggle.com/)
• Data Science course at Berkeley
(http://datascienc.es/ ) & on Coursera
(https://www.coursera.org/course/datasci/ )
Summary
• Why now: Dawn of Big Data <<- Need for Advanced Analytics and Cloud Computing
• What is it: Data ->> Data Product, many examples incl. Google, Netflix, Splunk, LinkedIn
• How to become: Data management, parallel
computing and data processing, statistical machine learning, and visualization skills
– Life of Data
• Who are hiring: Data Scientists are in great
demands, from industry to government to science.
48
Announcements
• Course Website:
http://www.cise.ufl.edu/class/cis6930fa14pro
• New registration slots will be available @ 4pm TODAY
– For first year MS students with no data mining
background: please first take Introduction to Data Science
Next Few Lectures
• Graph Algorithms and Graph Databases
– Offline queries: page-rank over Web link structure
– Online queries: SPARQL (sub-graph matching) over RDF stores
• Shortest paths (social network transactions)
• Text and Image Extraction and Retrieval
– Features Extraction
– Bag of Words model + Solr indexing
– More advanced: Entity and event extractions/classifications
• Knowledge Base Construction, Querying, Integration and Mining
– General, Health, Ecology, Medical, … – Rule/Constraint Learning
– More advanced: Uncertainty Management and reasoning
50
