April 30, 2015
New Jersey Law Journal
Attorney: Susan M. Usatine
As a frequent presenter on the topic of e-discovery, I am often approached by a member of the audience at the conclusion of the seminar. Typically, the fellow attorney listened intently throughout the session and generally seemed to "get it"—in other words, he or she recognizes that 99 percent of complex litigations involve e-discovery. Throughout the presentation, the lawyer nodded in agreement during my remarks concerning e-discovery's ethical mandates and our sophisticated clients' increasing demand for litigation efficiencies. Yet, now, in the emptying conference room, my colleague's demeanor shifts from agreeable to slightly agitated. Within moments after introducing himself, the lawyer shakes his head and bluntly states, "I believe e-discovery is ruining litigation." My response, a simple "Why?" most often provokes one of three responses: 1) "There is too much data and too little time"; 2) "Discovery costs are crippling and the number of discovery gigabytes contribute as much to the decision to litigate as do the merits"; or 3) "Nowadays, only large firms can handle complex litigation because they have the manpower."
There is no arguing that the Information Age provides challenges for litigators. Our clients are creating and storing information at an unfathomable pace. Simply locating relevant documents can be a challenge, as the information now resides in multiple places in a multitude of forms and there is too much of it. The volume of electronically stored information (ESI) and liberal discovery standards cause stress for most lawyers who do not consider themselves e-savvy and yet must answer to clients' more-for-less challenge and demands for fixed document discovery budgets.
True, lawyers can counsel their clients on prelitigation data minimization to reduce information hoarding and lower information expenses, however, until information governance protocols are implemented and enforced, massive discovery datasets will continue to be the norm. In sum, the volume, variety and velocity of discovery data cause enormous tension between the lawyers' obligations to discover and disclose relevant information and the clients' understandable fear of unbridled cost and inefficiencies. Something must change.
This article proposes that New Jersey litigators embrace technology-assisted review (TAR), also referred to as computer-assisted review (CAR). TAR is a powerful technology that reduces clients' discovery expenses without sacrificing results. The goal of this article is to dispel three myths concerning TAR and to encourage counsel to consider the benefits of using this technology in complex litigation discovery reviews. What is the difference between TAR and predictive coding?
TAR describes the integration of technology in the process of human document review and classification. A lawyer who has used a keyword search within a discovery dataset has utilized a basic form of TAR. Predictive coding is a type of TAR that uses artificial intelligence in the review of large volumes of ESI. Predictive coding can be utilized when the discovery dataset contains more than 50,000 records and significantly reduces the time and expense associated with traditional discovery reviews without sacrificing quality results. For instance, in a dataset containing 100,000 records, it may only be necessary to perform an "eyes-on" (also known as manual, human or linear) review of less than 10 percent of those records. With the benefit of human "training" and interaction with the predictive coding software, predictive coding can classify 90,000 records (or more) based on a predetermined confidence rate. Myth #1: Human review is the gold standard
Complex litigation typically involves tens of thousands of discovery documents and historically this required a large team of human reviewers. Historically, budgets dictated the number of attorneys that would be utilized to timely complete the review. Budget constraints, deadlines and/or law firm resources would often require that teams employ less expensive contract/nonattorney reviewers.
Unfortunately, and despite the review teams' best efforts, the Luddite approach to document review is enormously expensive and does not positively impact the quality of the review. To the contrary, it is well-established that eyes-on reviews take longer, cost more and typically present a high level of reviewer inconsistencies that are a product of human error. See Herbert L. Roitblat, Anne Kershaw & Patrick Oot, "Document Categorization in Legal Electronic Discovery: Computer Classification vs. Manual Review," 61(1) J. of the Am. Soc'y for Info. Sci. & Tech. 70 (2010); Maura R. Grossman & Gordon V. Cormack "Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review,"XVII Rich. J.L. & Tech. 11 (2011); Maura R. Grossman & Gordon V. Cormack, "Inconsistent Assessment of Responsiveness in E-Discovery: Difference of Opinion or Human Error, 32 Pace L. Rev. 267 (2012).
Inconsistency rates among reviewers is typically as high as 70 percent, which means that different reviewers looking at the same documents would only agree as to the relevance of those documents an average of 30 percent of the time. Some studies show an agreement rate as low as 16 percent. See Ralph C. Losey, "Predictive Coing and the Proportionality Doctrine: A Marriage Made in Big Data," 26 Regent U. L. Rev. 7 (2013-2014). Results and efficiency demand that we consider modern approaches to handling information volume. Myth #2: Computers will take the place of lawyers
The predictable response to the power of predictive coding is suspicion. Does predictive coding take the place of human review? The answer is simple—it does not. The importance of human judgment in a TAR cannot be overstated. In fact, the success of a predictive coding review depends on subject matter experts (SMEs) who collaborate with and train the predictive coding software. It is critical that the SME possess a complete understanding of the dispute, including the causes of action, defenses and the scope of the requests before interacting with the software to define the "targeted" documents. The SME codes a small subset of the total dataset and the software extrapolates the results and orders, compares and ranks the remaining documents from highly likely to be relevant to less likely to be relevant. Initially, the software's predictions are often wrong. The SMEs correct the software's predictions and rerun the machine coding. The process continues in rounds as the human trainers review subsets of the data until the computer predictions satisfy the proportional demands of the case. This author agrees with Ralph Losey, Esq., a thought leader on e-discovery and more specifically predictive coding technologies, that early cooperation between adverse counsel and true SMEs are necessary prerequisites to a quality predictive coding review. See Ralph Losey, "Predictive Coding," Electronic Discovery Best Practices, www.edbp.com/search-review/predictive-coding/ (last visited Apr. 2, 2015). Myth #3: Judges will not approve a party's request to utilize TAR
It has been three years since Magistrate Judge Andrew J. Peck, stated:
This opinion appears to be the first in which a court has approved of the use of computer-assisted review …. What the bar should take away from this opinion is that computer-assisted review is an available tool and should be seriously considered for use in large-data-volume cases where it may save the producing party (or both parties) significant amounts of legal fees in document review. Counsel no longer have to worry about being the "first" or "guinea pig" for judicial acceptance of computer-assisted review. As with keywords or any other technological solution to e-discovery, counsel must design an appropriate process, including use of available technology, with appropriate quality control testing, to review and produce relevant ESI while adhering to Rule 1 and Rule 26(b)(2)(C) proportionality. Computer assisted review now can be considered judicially-approved for use in appropriate cases.
DaSilva Moore v. Publicis Groupe & MSL Grp., 287 F.R.D. 182, 193 (S.D.N.Y. 2012).
Several months ago, Judge Peck authored another opinion regarding predictive coding and stated that in the three years since the DaSilva Moore decision,"the case law has developed to the point that it is now black letter law that where the producing party wants to utilize TAR for document review, courts will permit it." Rio Tinto v. Vale, No. 14 Civ. 3042, 2015 WL 872294, at *1 (S.D.N.Y. Mar. 2, 2015). Judge Peck cited numerous examples of courts throughout the United States approving the producing party's use of TAR, including: Dynamo Holdings v. Comm'r of Internal Revenue, No. 2685-11, 2014 WL 4636526, at *3 (T.C. Sept. 17, 2014); Green v. Am. Modern Home Ins. Co., No. 14-CV-04074, 2014 WL 6668422, at *1 (W.D. Ark. Nov. 24, 2014); Aurora Coop. Elevator Co. v. Aventine Renewable Energy, No. 4:12-CV-3200, 2014 WL 895669 (D. Neb. Mar. 6, 2014); Edwards v. Nat'l Milk Producers Fed'n, No. 11 Civ. 4766, Dkt. No. 154: Joint Stip. & Order (N.D. Cal. Apr. 16, 2013); Bridgestone Ams. v. IBM, No. 13-1196, 2014 WL 4923014 (M.D. Tenn. July 22, 2014); Fed. Hous. Fin. Agency v. HSBC N.A. Holdings, 11 Civ. 6189, 2014 WL 584300, at *3 (S.D.N.Y. Feb. 14, 2014); EORHB v. HOA Holdings, No. 7409, 2013 WL 1960621 (Del. Ch. May 6, 2013) (Order); In re Actos (Pioglitazone) Prods. Liab. Litig., No. 6:11-MD-2299, 2012 WL 7861249 (W.D. La. July 27, 2012) (Stip. & Case Mgmt. Order); Global Aerospace v. Landow Aviation, No. CL 61040, 2012 WL 1431215 (Va. Cir. Ct. Apr. 23, 2012).
It can no longer be credibly argued that courts do not accept predictive coding. From technologically naïve to e-savvy
Lawyers are typically loath to consider, much less adopt, new technologies. We defend our processes and billing models because to do otherwise requires change. It is time to increase our technological competency. The tools exist for us to tame the data and ensure that clients involved in complex disputes are making decisions based on the merits of the dispute and not on the total number of gigabytes. Given that 99 percent of complex litigations involve e-discovery, remaining technologically naïve is a decision with consequences. Our decision to learn and leverage technology is what will ensure that clients continue to regard litigation as an effective method of resolving their complex disputes.
Reprinted with permission from the April 30, 2015 issue of the New Jersey Law Journal. © 2015 ALM Media Properties, LLC. Further duplication without permission is prohibited. All rights reserved.
Originally published in the New Jersey Law Journal.