eDiscovery Daily Blog

• August 24, 2016

Our webinar panel discussion conducted by ACEDS a couple of weeks ago has generated some interesting discussion and debate over the past week or so about the content of the webinar, what it should have covered and what it was intended to cover.

The webinar panel discussion, titled How Automation is Revolutionizing eDiscovery was sponsored by CloudNine and presented on August 10 (here is a link to last week’s blog post with an embedded video of the webinar).  Our panel discussion provided an overview of eDiscovery automation technologies and we took a hard look at the technology and definition of TAR and potential limitations associated with both.  Mary Mack, Executive Director of ACEDS moderated the discussion and I was one of the panelists, along with Bill Dimm, CEO of Hot Neuron and Bill Speros, Evidence Consulting Attorney with Speros & Associates, LLC.

On the next day, ACEDS published A Considered Response from Gordon Cormack, which was a letter from Gordon Cormack, Professor with the School of Computer Science at the University of Waterloo in Ontario, Canada who is an expert in the area of technology-assisted review in litigation and has co-authored several influential works with Maura Grossman, a fellow researcher at the University of Waterloo (and, before that, an attorney with Wachtell, Lipton, Rosen & Katz).  Cormack and Grossman authored the 2011 Richmond Journal of Law and Technology (JOLT) study (that I inaccurately referred to in the webinar as a “white paper” – sorry about that) titled Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review, which was discussed to some extent during the webinar that both Cormack and Grossman attended.

In his letter, Dr. Cormack expressed several concerns about the content of the webinar, including their belief “that the webinar presented the false impression that we, and the courts, are resting on our laurels and that no legitimate empirical work has been done with respect to TAR.”  He provided links to several other works that have been authored by Cormack and Grossman that were not discussed during the webinar and also noted his opinion that the webinar lacked “any constructive suggestion as to how to proceed” (among other concerns he noted).

Then, last week, ACEDS also published responses from two of the presenters, Bill Speros (Reconsidering Dr. Cormack’s Considered Response) and Bill Dimm (ACEDS Commentary: Bill Dimm Responds to Gordon Cormack) where both “Bills” (Speros and Dimm) provided responses to the concerns that Dr. Cormack raised in his letter the previous week.

I feel that both Speros and Dimm made several good points in both clarifying the intended scope of the webinar and also in what we feel the webinar accomplished.  Dimm noted that “our goal was to deliver a large amount of information that is useful to a broad e-discovery audience within the confines of a 60-minute webinar” (of which about 40 minutes were devoted to TAR) and that we covered the JOLT report as extensively as we did (instead of other Cormack/Grossman works) “because it is the study that judicial opinions rely upon, and we’re not aware of any subsequent study comparing the quality (not merely the cost) of TAR results to those of human review.”

Speros identified several constructive suggestions that we felt the webinar provided, including “Clarifying the (general lack of) judicial acceptance of TAR”, “Differentiating alternative TAR techniques and technologies” and “Developing independent and valid TAR assessments”.  Speros also noted that “rather than attacking Dr. Cormack and his work, the webinar’s content spoke to the quality of the court’s interpretations [of the JOLT report] in a manner entirely consistent with thoughtful and professional analysis” – a position with which I agree wholeheartedly.

So, what is my response to Dr. Cormack’s letter?

As the other presenter in the webinar, I don’t have much to add to the responses provided by Speros and Dimm, except that they essentially reflect my own thoughts about the intent and accomplishments of the webinar.  Our goal was to challenge several industry-accepted assumptions about TAR and to take a look at the current state of acceptance of TAR, both judicially and within organizations contemplating the use of TAR.  And, I feel we accomplished that.

Nonetheless, I have tremendous respect, not just for my co-presenters, but also for Gordon Cormack and Maura Grossman and the numerous contributions that they have made to the industry through their research and various works (including the 2011 JOLT report).  I consider this to be a healthy discussion and debate among industry thought leaders and look forward to hopefully seeing that healthy discussion and debate continue.  I encourage you to view the webinar and read the commentaries by Cormack, Speros and Dimm and draw your own conclusions.

So, what do you think?  Do you have an opinion on the webinar or on the topic of TAR in general?  Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by CloudNine. eDiscovery Daily is made available by CloudNine solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Daily should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.