If I come back now to a panel I moderated on technology-assisted review last November, it is partly because I think we are about to see a new focus on the use of TAR to achieve proportionate eDiscovery beyond the US. It is perhaps time for a jurisdiction-neutral survey of the points which arise when the use of TAR is under discussion – or to help kick-start such a discussion if it is not already happening.

The panel was one of three which I moderated at Technology in Practice in Toronto last November. Run by Commonwealth Legal and Ricoh, this event was one of the best I took part in last year. This was partly because they gave me a lot to do, and partly because the organiser, Jennifer Johnson of Commonwealth Legal, assembled an agenda which was both diverse and threaded through with some consistent themes. One of those themes was the human element in technology – thus the title I gave to my event review Law, technology and the human element at Technology and Practice in Toronto. Indeed, one of my three panels was devoted to that human element. Winning acceptance of technology-assisted review depends very much on understanding how people work; it is not just law and technology.

For my first panel, Jennifer Johnson allowed me not only to pick my subject but my panel. I chose technology-assisted review, and asked for Maura Grossman, formerly a red-hot New York litigator and now an research professor at the University of Waterloo and for US Magistrate Judge Andrew Peck (now retired from the bench and Senior Counsel at DLA Piper). Constantine Pappas of Relativity joined us to bring the provider view.

The panel itself was ably summarised by Carolyn Gruske of the Lawyer’s Daily in an article called Experts serve up primer on technology assisted review and with the sub-title If you’re not already using technology assisted review (TAR), now is the time to start learning about it. (This article was originally published on 23 November 2017 by the Lawyer’s Daily (www.thelawyersdaily.ca), part of Lexis Nexis Canada Inc.). I will come back to the article shortly because it and the discussion it reports, cover the broad introduction I mentioned in my opening paragraph.

__________

The subject is relevant to the pending new disclosure rule in England and Wales where one of the issues (at least so far as I am concerned) is the broad definition of TAR, summarised in Carolyn Gruske’s article like this:

According to Grossman, programs only fall under the TAR banner if they either categorize or prioritize large sets of documents. Technology that only performs threading or clustering doesn’t qualify under her definition.

To my eye, the present draft of the proposed England and Wales rule does not draw this distinction and it should.

One of the things which prompted my return to this subject is a webinar run this week by Exterro. Maura Grossman was one of the panel members and, judging by the tweets about the webinar (I was not able to listen), one of her subjects was the reluctance of lawyers to embrace technology-assisted review.

One of the quotations on Twitter was this one:

Where else have we heard a proponent of legal technology described by lawyers as “dangerous”? That is what they said about Professor Richard Susskind when he first suggested that lawyers would communicate with each other and with their clients by email. Similar mockery greeted his suggestion that lawyers would make legal material available for free on public-facing websites.

It is one thing to say that such predictions are fanciful, but quite another to label them “dangerous”. A cynical view suggests that lawyers consider discovery technology to be “dangerous” because it threatens the very profitable business of document review. A less cynical view points to a genuine fear by many lawyers that the technology will not find every document which ought to be found, thus not only potentially impairing the client’s case but also putting the lawyer in breach of obligations to opponents and the court.

Another of the Exterro quotations is relevant here:

No-one will argue with the idea that more scientific measurement is a good thing. Few lawyers, however, understand or want to understand the technology – most of us have an arts background and chose law perhaps because we weren’t very good at science and mathematics (let alone statistics). Maura Grossman is an exception here, as you can tell by reading her papers. Most lawyers really just want to know that they are doing a good job.

__________

Maura Grossman is quoted at the end of the Lawyer’s Daily article as saying that you can’t just say “I don’t understand.” If we want to get lawyers to understand the benefits of technology, we have to focus on satisfying them that they are doing a job which is at least good enough by the standards of the court or regulator who will judge that conduct and (which was Maura’s point) in a manner consistent with their professional obligations.

What is that standard? Well for one thing, it is not perfection. I gave two examples in my recent review of Michael Wheater and Charles Raffin’s new book Electronic Disclosure Law and Practice, saying:

Of all the quotations they give on this subject, perhaps the most useful is one by Fullam J in the Irish case Irish Bank Resolution v Quinn. He said:

“If one were to assume that TAR will only be equally as effective, but no more effective, than a manual review, the fact remains that using TAR will still allow for a more expeditious and economical discovery process”.

One might couple that with US Magistrate Judge Andrew Peck’s often-reiterated point that it is wrong to hold TAR to a higher standard than manual review.

Against that, perhaps, is the judgment of Coulson J in the recent UK Triumph Controls case which (as I put it in the same review) “suggests that at least one senior judge still thinks that manual review is some sort of gold standard.”

Assuming you reject such nonsense (I am not sure Coulson J actually intended this implication), how do you persuade lawyers to consider the use of technology like TAR?

Various factors come into play here. One is that the lawyers can hire people who do understand the technology, either by actually employing them or by outsourcing that aspect to one of the many providers of discovery / disclosure services. Another is to focus on the tools and processes inherent in all TAR tools designed to check the outputs – I often suggest both to providers and to would-be users that they start demonstrations by looking at the checking mechanisms, so that the user does not spend the whole demo wondering about this. Yet a further point is that some of the time and money saved by using the TAR tools and processes can be used to check the results, by sampling and applying further tools like clustering to the results.

__________

Let’s go back to the article in The Lawyer’s Daily about our panel in Toronto and pull out the main points not covered above:

There is more than one kind of TAR, not merely because different developers have their own ideas but because of a broad division between what are called TAR 1.0 and TAR 2.0. TAR 1.0 requires a training phase whereas TAR 2.0 (often called Active Learning or Continuous Active Learning) starts its training from being used – “You keep reviewing and training. From day one the system is learning and it never stops”, as Maura Grossman is quoted as saying.

It is important for judges, as well as the lawyers, to understand the broad differences between systems. It makes little sense for lawyers to wait until the biggest-ever case comes along before starting to familiarise themselves with the available tools. Similarly, judges ought to have some idea of the differences, and the implications of those differences, in advance of hearing argument about them and not just in the possibly contentious context of a hearing.

In practice, it will fall to the lawyers to explain not just the mechanics but the benefits of the chosen system. The proposed new disclosure rule in England and Wales suggests that 30 minutes only will be provided for discussions about disclosure, which emphasises the need for informed and articulate proposals from the lawyers.

You do not need to use all of the tool’s functionality at once. One of the benefits of technology-assisted review tools is that they enable prioritisation, that is, identifying those documents which are most likely to be relevant and offering them first for by-eye review. This not only delivers the (potentially) most important documents first, but also gives an early opportunity to make changes to the algorithm if those early documents reveal important omissions or over-inclusiveness.

TAR is not a substitute for manual review of those documents which are disclosed – this is a common misconception which needs to be addressed. Its purpose is to filter out those which it would (probably) be disproportionate to review (subject to sampling and other checking) so that review time is spent on those which are likely to be handed over.

While TAR brings most value to very large document volumes, it can also be valuable on much smaller exercises, not least when you are first starting to learn how to use it.
Maura Grossman is quoted as saying “You spend some time playing with the tools so you’re not seeing it for the first time in a moment of crisis.”

Summary

In many ways, it was more interesting doing this panel in Canada than in, say, the US or England and Wales. To some extent, the subject has been done to death in the US (which is not the same thing as saying that the general level of understanding is good). In England and Wales, the market is divided between the increasing number just getting on with using TAR and those who will wait until a client, an opponent or (worse still) a judge enquires why a TAR option is not being proposed.

Canada stands to benefit from the fairly painful process of education and acceptance in these jurisdictions; the team of Peck and Grossman is perfect in that both of them well know how to tailor the content to the audience, and my idea of a good time (in work terms at least) is to moderate panels with people of this calibre. Great applause and thanks to Commonwealth Legal and Ricoh for providing an environment in which we could have this discussion.

I believe Maura Grossman will be at a conference in Dublin later this year – more about this when I know more. There is talk of my doing an event in Sydney with retired Judge Peck. Who is up for organising a trip taking in any or all of Australia, New Zealand, Singapore and Hong Kong where we might do our bit to spread the word about the use of analytics in general and TAR in particular in the way Commonwealth Legal allowed us to do in Toronto?

Home