PRESENTED BY:
www.evolvediscovery.com
Practical Uses of Analytics in E-Discovery - A PRIMER
Jenny Le, Esq.
–
Vice President of Discovery Services
jle@evolvediscovery.com
E-Discovery & Ethics
Structured, Conceptual, and Predictive Analytics
www.evolvediscovery.com
Updates made to ABA Model Rules of Professional
Responsibility by ABA Commission on Ethics 20/20
recommendations and adopted by the ABA House of
Delegates
August 2012
Updates “reflect the realities of the digital age”
“technology has irrevocably changed and continues
to alter the practice of law in fundamental ways”
Technology
DOING THE RIGHT THING
www.evolvediscovery.com
Technology is also having a related impact on how
lawyers conduct investigations, engage in legal
research, advise their clients, and conduct
discovery
.
These tasks now require lawyers to have a firm grasp
on how electronic information is created, stored, and
retrieved. Lawyers need to know how to make and
respond to electronic discovery requests and to advise
their clients regarding electronic discovery obligations
… These developments highlight the importance of
keeping abreast of changes in
relevant technology
in
order to ensure that clients receive competent and
efficient legal services.
E-Discovery
Keeping abreast of the benefits and risks associated
with relevant technology is especially challenging for
e-discovery practitioners
Relevant technology is everywhere!
www.evolvediscovery.com
Lawyers representing clients in matters related to
e-discovery have a duty to remain current on technologies
relevant to litigation and technologies that enable
relevant data to be preserved, collected, processed,
reviewed, and produced accurately, efficiently and in a
manner consistent with all of the client’s legal
obligations.
Ethical Duty
Maintain a baseline understanding of technology
to leverage greater knowledge and expertise of
your experts
www.evolvediscovery.com
Reasonably
Necessary
Legal Knowledge
Skill
Thoroughness
Preparation
Reasonable
Assurance
Managerial Authority
Supervisory Authority
Monitoring
Reasonable Efforts
Prevent:
Inadvertent/
Unauthorized disclosure
or
unauthorized access
Client Information
CompetenceRule 1.1
Responsibilities Regarding Nonlawyer Assistance
Rule 5.3
Protecting Client Confidence
Rule 1.6
E-DISCOVER Y & ETHICS
www.evolvediscovery.com
Not Every Case Involves E-Discovery, but
Almost Every Case Potentially Does
Can you devise an e-discovery plan in 2 hours?
STATE ETHICS RULES
States and local bar associations
have issued
individual ethics opinions relating to attorney
competence in the areas of technology, cloud
computing, security & Ediscovery competence.
15
E-DISCOVERY & ETHICS
State Adoption
California New York Arkansas Arizona Alabama Delaware North Carolina Pennsylvaniawww.evolvediscovery.com
Standing Order M10-468
Joint Electronic Discovery Submission & Order
Competence
Counsel certifies that they are sufficiently
knowledgeable in matters relating to their clients’
technological systems to discuss competently
issues relating to electronic discovery, or have
involved someone competent to address these
issues on their behalf.
E-DISCOVERY & ETHICS
www.evolvediscovery.com
E-DISCOVERY & ETHICS
California
Consult with
Expert
Mental, Emotional,
Physical Ability
Learning and Skill
Diligence
Case-by-Case Analysis
Knoweldge will depend on the Ediscovery issues and the nature of ESI involved.
E-Discovery Knowledge Options
Acquire sufficient learning and skill before performance
Associate or consult technical consultants or competent counsel Decline the client representation.
Proposed Ethics Opinion No. 11-0004
Rules and procedures, when placed in conjunction with
ever-changing technology, produce professional challenges that attorneys must meet in order to remain competent.
The duty exists in connection with e-discovery.
Competence
Depending on the factual circumstances, a lack of technological knowledge in handling e-discovery may
render an attorney ethically incompetent to handle certain litigation matters involving e-discovery.
www.evolvediscovery.com
Monetary Sanctions
No. 09-15173, 2010 WL 1418861 (Bankr. SDNY, Apr 7, 2010)
Plaintiffs GFI Acquisition LLC
was imposed monetary sanctions because
counsel was “uninformed on the detailed workings of GFI’s computer system
and email retention policies” and “ did not understand the technical depths to
which electronic discovery can sometimes go.”
Court awarded sanctions despite the fact that no spoliation or bad faith
conduct were found.
E-DISCOVERY & ETHICS
In re: A&M Florida Properties II, LLC
www.evolvediscovery.com
E-DISCOVERY & ETHICS
J-M Mfg. Co. v. McDermott Will & Emery
EDiscovery Malpractice Claim
No.BC462832 (Cal. Super. Ct., LA Cnty., June 2, 2011)
Plaintiff
J-M Manufacturing
, alleges that its former law firm breached its
professional duty of care by failing to supervise its EDiscovery vendors and
contract lawyers and, as a result, inadvertently producing thousands of
non-responsive, privileged documents to the government – not just once, but
twice.
“monitoring” is to remain aware of how nonlawyer services are
being performed
Rule 5.3
Discussion between Client and outside
counsel
Who is Monitoring?
Understand the technology being used
www.evolvediscovery.com
2009
2014
32M
189M
THE BASICS
Big Data
the last 5 years. 32 million documents in the largest case in 2009 to 189 million documents in the largest case in 2014.
Big Data
Relativity, a leading E-Discovery hosting platform estimated that the largest cases have grown 6X over
www.evolvediscovery.com
THE BASICS
E-Discovery Analytics
Structured Analytics
Email Threading, Near Duplicate Detection, Language Identification, Repeated Content Detection
Conceptual Analytics
Clustering, Categorization, Conceptual Searching, Keyword Expansion
Predictive Analytics
Predictive Coding, Assisted Review
www.evolvediscovery.com
Email Threading
Near Duplicate
Detection
Language
Identification
Repeated Content
Detection
E-DISCOVERY & ETHICS
Structured Analytics Suite
www.evolvediscovery.com
“Reduce Review”
How-To:
• Sort in Email Thread Order
• Email Thread suppressed review
• Negotiate last-in-time productions protocol
• Propagate coding up-thread, if necessary
E-DISCOVERY ANALYTICS
www.evolvediscovery.com
Understand custodian
communication:
Visually group together
emails in a way that’s easy
to understand.
Review only what you
need to:
Identify the inclusive emails in
a thread, such as the last
e m a i l i n a l e n g t h y
conversation, and avoid the
r e p e t i t i v e r e p l i e s a n d
forwards.
Keep email organized
for reviewers:
B a t c h o u t i n c l u s i v e
documents to your reviewers
and keep email threads intact.
Improve Speed & Cost
E-DISCOVERY ANALYTICS
www.evolvediscovery.com
Decision
Consistency
How-To:
• Verify consistency of coding decisions
• Identify Privilege (Rule 1.6)
• Sort review batches by Near Duplicates
• Cull documents
• Compare opposing production to your own collection
E-DISCOVERY ANALYTICS
Near Duplicate Detection
Use Similarity to Your Advantage
Near Duplicate detection will provide information about:
the number of near-duplicate groups in your data set, the average number of documents per group,
the average percent of similarity between the documents in each group.
www.evolvediscovery.com
Understand dataset based on language prevalence
Gain Insight
Organize review based on language
Improve Workflow
173 languages detected
Flexibility
Language identification analyzes sentence structure
and punctuation. .
How does it work?
E-DISCOVERY ANALYTICS
Language
Identification
Reports
Language Identification Summaries and
Reporting can help in understanding the
Repeated Content Filter
When using analytics, long, repetitive blocks
of text—such as email footers—can cloud
your index and produce inconsistent results.
www.evolvediscovery.com
E-DISCOVERY ANALYTICS
Latent Semantic Index
that have undergone LSI will return results that are conceptually similar in meaning to the search criteria even if the results don’t share a specific word or words with the search criteria.
What is it?
Correlates semantically related terms that are latent in a collection of text – finding concepts. Concept searches, against a set of documents
LATENT
Not Obvious
Capable of Emerging
SEMANTIC
Relating to Meaning
Concepts
INDEX
Comparing
and
Organizing
Information
www.evolvediscovery.com
E-DISCOVERY & ANALYTICS
Analytics Index
Suitable Dataset
File types, sources/other?
Update
Include new data
Optimize Index
Remove non-text based documents, documents will with too much or too little text. Identify and remove repeated content.
Customize Settings
Set the minimum
occurrences, number of words, and other
customization options based on the unique needs of each data set.
www.evolvediscovery.com
Clustering groups or sorts documents based on common content. You can
cluster all documents or smaller groups.
After clusters are created batches can be created based on clusters..
Clustering
Groups issue coded docs with new documents based
on conceptual content.
Categorization
Keyword expansion allows you to see terms or keywords you might
not have originally expected.
Keyword Expansion
Applies a block of text against the database to find documents of similar
conceptual content.
This can help prioritize or help find important documents.
QC for consistency across conceptually similar documents.
Concept Search
E-DISCOVERY ANALYTICS
www.evolvediscovery.com
Improve Efficiency, Speed & Cost
Clustering
Internal Investigation
Witness Preparation
Issue Collection
Received Productions
Keyword Expansion
Internal Investigation
Concept Search
Internal Investigation
Categorization
Internal Investigation
Hot/Key Documents
Issue Collection
Reuse previous work product
E-DISCOVERY ANALYTICS
Predictive Analytics
Allowing a computer to predict properties of
documents based on samples you feed the
computer.
Review QC
Sample
Report /
Verify
Categorize
Review
Training
Sample
Complete
www.evolvediscovery.com
E-DISCOVERY ANALYTICS
Predictive Analytics Success
Harmonic Mean of Precision and Recall
How many Responsive documents did the computer predict, of all the
Responsive documents in the dataset?
Statistical sampling verfies and validates the project
How can you improve your project?
How can predictive coding enhance your litigation strategy?
Statistical Sampling
Passive or Active Learning
Strategy
Proportion of documents predicted Responsive, that are in fact, Responsive
Precision
Recall
www.evolvediscovery.com
“True Positive”
E-DISCOVERY ANALYTICS
Precision & Recall
false
negative
false
positive
true negative
true
positive
www.evolvediscovery.com
E-DISCOVERY ANALYTICS
Precision & Recall Considerations
Recall
160/195 = 82.1%
Precision
www.evolvediscovery.com
E-DISCOVERY ANALYTICS
TAR Disclosure & Cooperation
When Is the Right Time to Disclose Using TAR?
• develop a workable search and review
methodology early rather than late in the discovery process.
• Confine Rule 26(f) discussions to addressing
scope of discovery rather than specific processes.
How Much Disclosure about the TAR Process Is Required?
• Parties are not required, under the FRCP, to
share predictive-coding training documents, or non-responsive sets with opposing counsel.
• training sets and nonresponsive documents are
comparable to information contained in a document-retention policy and therefore also discoverable under the Federal Rules.
• training sets not only go to whether a document is
responsive and discoverable but may also give
insight to how counsel prioritized documents for review and potentially also represent documents that counsel deemed central to the litigation, all of which encroach on work product.
• ESI not responsive to the matter at hand may be
relevant in other matters in the future. The panel warned that it is not unusual for counsel to take discovery in one case in hopes of learning something about another case, a concern that could be especially relevant in unfair competition cases.
Things to consider about using TAR
• Are you working with professionals who can
explain its rationale and process?
• Do you understand the limitations and necessary
customizations?
• Do you understand how much disclosure you are
really giving?
• Do you know how to use TAR effectively?
• How can you calculate cost savings?
Cases:
• In re Biomet M2a Magnum Hip Implant Prods.
Liab. Litig., No. 3:12-md-2391, 2013 WL 6405156 (N.D. Ind. Aug. 21, 2013)
• United States v. ExxonMobil Pipeline Co., No.
4:13-CV-00355, 2014 WL 2593781 (E.D. Ark. June 10, 2014)
• Rio Tinto Plc v. Vale S.A., 2015 WL 872294
