The Growing Popularity of Technology-Assisted Review

May 2017

                        Published in the Sussman Shank LLP Spring 2017 Newsletter


A company confronted with an impending internal investigation, litigation, or subpoena response typically faces the unenviable task of processing and reviewing documents for potential relevance. The traditional model of human processing and review is typically time-consuming and expensive. The traditional model may also lead to inconclusive or incorrect results, as well as improper disclosures of documents that should have been withheld on the basis of privilege or confidentiality.

The challenges of data collection and review are further complicated by the rampant growth in the generation of electronically stored information (ESI). Now, more than ever, company employees can create and exchange electronic documents such as e-mail with great ease, especially with the constant use of cell phones and personal electronic devices in the workplace. The use of social media across companies has also increased, thus leading to another source of ESI generation. As a result, the volume of ESI stored on a company's servers, computers, and devices is usually enormous.

To better assist clients facing issues related to ESI volume, lawyers have looked towards various computer software tools, more commonly known as technology-assisted review (TAR), to reduce the burdens of electronic discovery ("e-discovery") in litigation. These tools include deduplication, predictive coding, and conceptual searching. TAR has gradually grown in acceptance, to the extent that courts have approved its use in managing e-discovery. While not yet standard practice, TAR can potentially be a resource for companies in the right circumstances.

The Traditional Review Model vs. Technology-Assisted Review

Lawyers who go through the arduous task of ESI collection are then faced with the challenge of sorting through and reviewing their collected ESI to pick out documents that are relevant to the specific investigation or litigation at issue. The situation is quite difficult, especially in cases involving hundreds of thousands or millions of documents.

Under the traditional model of document review, long and often convoluted Boolean logic search strings are applied to arrive at a broad subset of ESI. Human reviewers then manually review these search results, one by one, and designate whether a document is potentially relevant and/or responsive, privileged, or contains confidential client information. Review typically comprises multiple stages: a first-pass review to narrow down the initial search results, and a second pass review to identify key or important documents, protect privileged or confidential information, and ensure quality control.

While the traditional document review model is standard practice and designed to be comprehensive, it is often time-consuming and expensive. These costs increase as the volume of underlying ESI increases. For example, documents often exist in duplicate form across different employees, and e-mail chains are actually comprised of separate documents that must be manually reviewed on an individual basis.

Another complication is the possibility for human error, as individual reviewers may categorize the same document in a different manner. As a result, inadvertent disclosures and insufficient productions can occur, thus exposing companies to business or litigation risk.

In response to these challenges, e-discovery professionals and lawyers have developed computer-based programs to ease the burdens and risks associated with the traditional review model. While offering different functions from each other, these tools all fall under the umbrella of TAR. These tools include:

Deduplication: Documents are compared against each other and those that show a high degree of similarity are grouped together. A reviewer can then review and categorize one example of that group in a process called "bulk coding," thus eliminating the need to review each duplicate individually.

Email Threading: Individual e-mails that comprise a greater discussion are grouped together, even across multiple employees, so that a reviewer can code these e-mails more effectively and consistently.

Clustering: Documents are quickly sorted into groups based on their content to give practitioners a visual snapshot of the underlying ESI. For example, news articles are grouped together, while e-mails are placed into their own group.

Conceptual Search: Reviewers, using a document of interest or a sample set, can search for other documents that share similar concepts or themes rather than limiting their searches to keywords.

Predictive Coding: A more comprehensive review tool that studies previously reviewed and coded documents and applies its understanding to the larger, unreviewed collection of ESI, helping reviewers predict the relevance of those unreviewed documents.

The Benefits and Costs of TAR

When used effectively, TAR carries both practical and substantive benefits. The most significant practical consideration is the potential reduction in time and money spent on discovery by slashing the amount of documents required for human review. These savings would be most notable in the early stages of review, where significant time can be spent reviewing duplicate or irrelevant documents. Predictive coding, in particular, can have great value in circumstances where documents of interest have already been identified and quick, accurate analysis is needed to gauge a client's early level of risk. Lastly, attorneys can also use TAR as a quality control measure by helping to reduce the inconsistencies that plague the traditional model. There are concerns regarding the effectiveness and limitations of TAR. As a practical concern, TAR costs money and vendor pricing can vary. Because TAR tools are relatively new, traditional methods of document review remain the industry standard. The effectiveness of TAR still relies on attorney experience and oversight, as well as the quality of vendors that provide those services; therefore, the use of TAR is still prone to human error.

Courts' Gradual Acceptance of TAR

Courts have exhibited a growing acceptance of the use of TAR in discovery. In one recent federal case, the court stated that "it is now black letter law that where the producing party wants to utilize TAR for document review, courts will permit it." Rio Tinto PLC v. Vale S.A., 306 F.R.D. 125, 127 (S.D.N.Y. 2015). The court even clarified that TAR should not be held to a higher standard than the traditional model because that would discourage parties from using TAR and realizing its cost savings for fear of having TAR challenged in court. Id. at 129. While courts have welcomed parties' reliance on TAR, they have stopped short of forcing parties to use TAR in lieu of traditional methods. In another recent federal court opinion, the court indicated that the parties could use predictive coding to satisfy its discovery obligations, but would not force a party to use predictive coding over its preferred method of traditional term searching and refined sampling. In re Viagra (Sildena_ l Citrate) Prods. Liab. Litig.,
No. 16-md-02691-RS (SK), 2016 U.S. Dist. LEXIS 144925, at *51–53 (N.D. Cal. Oct. 14, 2016).


Courts' growing acceptance of TAR is a sure sign that TAR will only grow in popularity in the near future. However, the appropriateness of TAR for your situation will depend on the volume of underlying ESI and the costs of TAR versus the traditional model. If your ESI volume is sufficiently high, and using TAR may result in sufficient cost savings, here are a few issues to consider:

Effective Data Preservation: Proper preservation is a prerequisite to any effective document collection and review. To that end, implement routine records retention policies with the help of your IT professionals, and distribute prompt legal holds when litigation is anticipated.

Assisting the Traditional Model: Courts are warming to the presence of TAR, but it will not be replacing the traditional model any time soon. TAR however can be an invaluable resource in speeding up the traditional model, ensuring quality control, and protecting against inadvertent disclosures.

Proper Human Supervision: While TAR can reduce the time and costs associated with traditional review, proper attorney supervision is still essential to making sure that TAR is implemented smoothly and effectively.

Choosing the Right Vendor: TAR is typically available through the services of a third party vendor. Because no universal standard yet exists for TAR, these vendors have developed their own proprietary bundles of software that can vary in quality. Thus, companies and practitioners must maintain a discerning eye when choosing a proper vendor and include requests for thorough demonstrations of a vendor's TAR offerings.

If you have any questions about TAR generally, or if TAR may be appropriate for your anticipated or ongoing investigation or litigation, please contact Robert Kim at or 503-227-1111.


Return to Articles