I occasionally put up examples where courts or regulators have given their blessing to the use of technology-assisted review. Two more have come to my attention. Neither involves litigation.

The first lies in an order of 17 October of the Canadian Competition Tribunal on an application in The Commissioner of Competition v Live Nation Entertainment, Inc et al. The Commissioner had made several complaints about the respondents’ search for documents. The respondents’ position is recorded thus:

[8] The Respondents have explained away the various deficiencies on the basis that they conducted searches in a more modern manner using computer assisted technology aided by a litigation support company – the technology assisted review (“TAR”). The result was the identification of 2.5 million documents which were then vetted through the TAR and lawyers trained in the TAR system and who trained the TAR system, and ultimately approximately 55,000 relevant documents were identified. All of this was accomplished in a relatively short period of time.

The tribunal said of this:

[13] The Commissioner’s request in this regard is premature. Two senior officials whose documents have yet to be produced but whom the Respondents agree will be produced may shed further light on what is no more than suspicion that the search was inadequate – but it is not an unreasonable suspicion given the way in which the Respondents produced their AODs.

[14] However, there has been no attack on the Respondents’ use of TAR, and other computer technology to assist in the identification and collection of documents. At this point the major problem is the attribution of documents to each of the Respondents.

[15] The Tribunal encourages the use of modern tools to assist in these document-heavy cases where they are as or more effective and efficient than the usual method of document collection and review.

[16] The issue of further and better searches should await the delivery of further and better AODs in form and content complying with the Rules.

Whatever is the final outcome of the disputed documents, it is the statements of principle which matters here – not just about the use of “modern tools”, but about making premature applications to whine about other people’s discovery.

My thanks to Rick Barker of Accuracy for pointing me to this.

__________

The second instance involves the release of emails, 183,000 in all, from the administration of Tim Kaine as governor of Virginia.

The Library of Virginia used Continuous Active Learning technology developed by Maura Grossman and Gordon Cormack of the University of Waterloo to find relevant information from the emails. The University of Waterloo article about this gives a description of Continuous Active Learning which is worth reciting:

CAL™ initially presents the user with the documents most likely to be of interest, followed by those that are somewhat less likely to be of interest or relevance, until no more can be located. Unlike a typical web-search engine, which focuses on identifying a few highly relevant documents, CAL™ uses machine learning to produce a high-recall result — that is, to identify substantially all relevant documents, by refining its understanding about which of the remaining documents are most likely to be of interest, based on the user’s feedback on documents already retrieved. CAL™ learns from the user’s feedback and continues to retrieve documents until no more relevant documents can be found.

My thanks to Maura Grossman for sending this in my direction.