The last few days have brought us significant court rulings in the US and in Dublin about the use of technology which is variously called predictive coding, technology-assisted review, computer-assisted review, and other names indicative of the joint application of human skill and technology to giving eDiscovery / eDisclosure which is simultaneously adequate and proportionate.

One is US Magistrate Judge Andrew Peck’s Opinion in Rio Tinto PLC v Vale SA et al in the Southern District of New York, and the other is the judgment of Mr Justice Fullam in the High Court in Dublin in Irish Bank Resolution Corporation Ltd v Sean Quinn & Ors. I have read a vast amount of comment on the Rio Tinto decision, but none except newspaper reports on the Irish judgment. That may be because the judgment itself is not yet publicly available; it may be that no-one cares very much – two parties argue about how to give discovery, judge listens, applies some law and makes a ruling; so what?

Both judgments resulted in court approval of the use of the technology proposed by one party, subject to protocols which included descriptions of the tools and the processes to be used and the methods for validating the results.

What is the shortest way of getting this off my desk and (which is my real objective) of getting you to inform yourselves, by a demonstration or otherwise, of the capabilities of technology whose use has met with approval in courts as diverse as New York and Dublin?

I should perhaps say two things at this point. One is that this article was originally headed “Technology-asisted review in as few words as possible”; that aspiration is not supported by the result, which runs to over 3,100 words; I regret none of them.

The other is that I am secretly in the “so what?” camp. We know that some American lawyers won’t scratch their bottoms unless some judge, in some case, in some court, in some state, has given his or her blessing not only to the act but to the method; the rest of us, however, prefer the idea that, if a dispute has to come to court, the judge will weigh the facts in that case and do what is right for that case. The most infuriating question I am asked in the UK is whether any judge has approved the use of TAR. If one has done so, it doesn’t bind the judge in the court next door with a different case and a different proportionality balance to strike. If none has, then so what? If it is right for your case – if it will allow you to give proportionate disclosure in the most proportionate way without undue risk of inadvertent omission – then use it or, at least, equip yourself to weigh its benefits against other ways of achieving the objective.

Terminology

A word first about the terminology. The term “predictive coding” was heavily promoted by the market-leading software providers when this type of software was first applied to electronic discovery. It was not for me to argue with the terminology used by those who sponsor what I do, but I preferred the wider term technology-assisted review because it embraced broader concepts and took us away from specific providers. It also helped emphasise that the predictive coding tools, including those from the major predictive coding providers, were part of a set of tools within applications which performed other tasks as well from simple keyword searches to clustering, email threading and the rest. Judge Peck notes in his introduction to the Rio Tinto Opinion that “technology-assisted review, or TAR, now seems to be the preferred term of art”. Meanwhile, however, the latest version of Andrew Haslam’s eDisclosure Systems Buyers’ Guide, to which I will return below, uses the term computer-assisted review, or CAR.

It is, of course, helpful if we all use the same terminology but, frankly, I don’t give a hoot what it is called. Nor do you, as long as you understand the broad concepts (and I emphasise “broad” here, because this article is not aimed at those already expert but at those who need merely to understand what we are talking about). I prefer to consider predictive coding as a type of technology-assisted review which is itself a sub-set of a yet wider concept called analytics. Predictive coding comes in various technical flavours and there are variants which perform the same function but which are not called predictive coding. Andrew Haslam’s report of Legaltech (to which I will return below) pursues this theme.

Let’s just focus on the main point – we have a lot of documents and need to balance the duty to find the relevant ones against the cost of doing so. There are complications here in that the US, Ireland and the UK all have different ideas of what relevance means (the UK Civil Procedure Rules do not even use the word) and hold to different standards of completeness. Forget all that for now, and forget (again, for now) all those terms of art like recall, precision, f-measure, and the deep arguments about seed sets and the like. Is it right to use sophisticated technology to aid the search for discoverable documents?

Why do we even ask this?

Judge Peck’s Rio Tinto Opinion

Let us take Judge Peck’s Rio Tinto Opinion first. He begins by referring back to his Da Silva Moore opinion of 2012 and his concluding assertion in that Opinion that:

What the bar should take away from this Opinion is that computer-assisted review is an available tool and should be seriously considered for use in large-data-volume cases where it may save the producing party (or both parties) significant amounts of legal fees in document review.

Judge Peck goes on to say that “the case law has developed to the point that it is now black letter law that where the producing party wants to utilise TAR for document review, courts will permit it.” He refers to the decision in Dynamo Holdings, saying:

The Tax Court’s response to being asked to approve the use of TAR was that courts leave it to the parties to decide how best to respond to discovery requests

He quoted from the Dynamo Holdings Opinion the passage beginning

“The court is not normally in the business of dictating to parties the process that they should use when responding to discovery”.

At one level, I would quite like to leave it at that. The giving party does what it believes to be its duty and the receiving party raises challenges afterwards where it has reason to suspect omissions. That simple proposition, however, is undermined by a number of factors.

Reasons why the court must sometimes be involved

1. Lawyers, particularly US lawyers, are brought up to assume that their opponent is going to conceal things. The ideal may be “trust but verify”, but deliberate concealment is not unknown, and omissions may derive from innocent failures or (to quote from a case cited in the Irish judgment) “a clever use of keywords which may raise suspicions that same were deliberately designed to minimise the risk of damaging documents being selected”. As US Magistrate Judge Elizabeth Laporte is wont to say, “you need a policeman on the block” to supervise and to arbitrate between parties where such suspicions arise, perhaps making decisions about motive and blame, and deciding accordingly.

2. The trend in most relevant jurisdictions is for pre-emptive interaction with other parties and, if necessary, with the court, before significant sums are incurred in a discovery which may subsequently be challenged. This trend was encouraged in the UK by the outcome of Digicel v Cable & Wireless [2008] where a reported £2 million was spent going down a route which was challenged retrospectively. A subsequent UK practice direction and new rules have codified the duty to discuss and to cooperate. Other jurisdictions are developing a similar approach; the court arbitrates any disputes which arise, undermining the idea that the giver gives and the receiver waits to see what he gets before challenging omissions. Indeed, courts are increasingly proactive about this, as the UK courts in particular are required to be by the rules.

3. There is a fundamental lack of competence in many lawyers, highlighted by a recent survey conducted in the US by Exterro, as well as appearing in judgments and opinions in most jurisdictions. If your opponent appears to be clueless, you don’t need to allege deliberate concealment to raise a challenge even before discovery / disclosure is given.

Judge Peck’s Opinion, as usual, refers to many other Opinions, heavyweight papers and other authorities and commentary, and I leave you to follow those up for yourselves. Two other points are worth extracting from this succinct Opinion.

The first can be dealt with by a simple quotation:

One point must be stressed – it is inappropriate to hold TAR to a higher standard than keywords or manual review. Doing so discourages parties from using TAR for fear of spending more on motion practice than the savings from using TAR for review”.

The other is what is summarised in the expression “seed set transparency”. This is part of the wider question about the extent to which the giving party should share with the receiving party details of its seed set (if used) and other information going beyond the documents which are disclosed. Judge Peck did not, in the event, have to deal with this in the Rio Tinto Opinion because the parties reached agreement. It remains a contentious point, and one which I do not propose to cover here.

The Irish Bank Resolution Corporation case

Turning to the Irish Bank Resolution Corporation case, this arose from a dispute over the plaintiff’s claim “that TAR will save time and be more cost-effective compared to the traditional manual / linear method of discovery”. This was challenged by the defendants who asserted that TAR did not comply with O.31 r.12 of the Rules of the Supreme Court. This rule, on its face, requires a rather stricter test of completeness than, say, the UK rules – the judge summarised the defendants’ position this way:

TAR will not capture all relevant documents and therefore is not compatible with the obligation of the party making discovery, which is the objective target of 100% of relevant documents.

I recall prophesying at a Dublin conference a few years ago that this strict definition would be the cause of dispute about TAR; it does not surprise me that it is the central argument in this case. Doubtless there are people who continue to assert that skilled lawyers turning pages will find 100% of the relevant documents, just as you can still find people who think the sun revolves round the earth.

You get the flavour of the competing claims from the fact that the plaintiffs’ expert, the well-known US lawyer, Conor Crowley (now with UBS in Hong Kong), expected that less than 10% of the documents would need to be reviewed if predictive coding were used. The plaintiffs’ solicitor (Karyn Harty of McCann FitzGerald) estimated that a traditional linear review with ten experienced reviewers would take nine months at a cost of €2 million before supervision and technology costs were included. The defendants’ experts, in contrast, reckoned that the linear review could be completed in 113 days at a cost of €220,000.

The arguments are all familiar to those who have followed the course of US debate about TAR. There is much explanation in the judgment of the processes proposed to be used, and references to US sources as well as to Irish law relating to the general duties of discovery.

The judge found, amongst other things, that the discovery duty was qualified by considerations of proportionality. Amongst the Irish sources referred to was the forward to the Good Practice Guide to Electronic Discovery in Ireland published on April 2013. Mr Justice Clarke of the Supreme Court referred in his introduction to the Guide to the need to ensure:

…that discovery remains an important tool for establishing the truth while at the same time ensuring that the cost and complexity of discovery does not itself become a barrier to the truth being established.

The judge went through a number of authorities which related to discovery specifically but also through those which pointed to “the inherent power of the courts to adapt in the absence of a specific rule”, words which appear as a sub-heading in the judgment. That was followed by a selection of quotations from Da Silva Moore and from the Dynamo Holdings case referred to above.

After summarising the “limited Irish jurisprudence on that topic” the judge expressed himself satisfied that:

Provided the process has sufficient transparency, technology assisted review using predictive coding discharges a party’s discovery obligations under O. 31 r.12.

He then considered the plaintiff’s proposed protocol and, crucially, the efforts which had been made by the plaintiffs’ lawyers to persuade the defendants of the merits of using TAR, and of the safeguards to be built into the process; one gets the strong sense that the defendants had done themselves no favours by their reluctance to engage fully in the discussions initiated by the plaintiffs. The judge ended with the conclusion

“I am satisfied that the proposed protocol will be more efficient than manual review in terms of saving costs and saving time”.

Andrew Haslam’s eDisclosure Systems Buyers’ Guide

I said that I would come back to Andrew Haslam’s eDisclosure Systems Buyers Guide which, by chance, was published in the same week as the two judgments covered above. It includes in Annex D some statistics and conclusions drawn from Andrew’s survey on the use of what he calls computer-assisted review (CAR). His conclusions suggest that CAR “is being used extensively in England and Wales” mainly for litigation for review, QA / QC and analysis of other parties’ disclosure. He concludes:

It seems the only people who aren’t convinced of its effectiveness are the law firms themselves. If suppliers don’t think it’s an effective tool, they don’t offer it. If you aren’t using this technology, then your opponent probably is, and is getting all the benefits while you are not.”

Summary

Is there anything new in either of these judgments? At one level, obviously, it is not insignificant that a US judge states that the use of a particular class of technology has become “black letter law”, and one cannot ignore the importance of the first decision of an Irish court which addresses the same subject.

Yet both of these judgments are firmly rooted in principles which predate technology-assisted review and are of deeper significance. There is nothing new in a challenge made by one party to another party’s discovery / disclosure which is ultimately settled by the court. The principles relied on by both courts include the balance of proportionality and existing duties such as cooperation which are of much wider significance than the use of any new technology.

All the relevant jurisdictions have a governing principle (the “overriding objective” in the UK) which governs everything else, and perhaps the most important single sentence in the Irish judgment is the one I quoted above – the heading “Inherent power of the courts to adapt in absence of specific rule”. The UK rules include a provision [Rule 3.1(2)(m)] that the court “may take any other step or make any other order for the purpose of managing the case and furthering the overriding objective”. If the court may make any order, you may seek any order as long as it is proportionate and does not fly in the face of some express provison in the rules.

Of all the extracts which I might leave you with from the two judgments I take this (a second time) from what Judge Peck said in Rio Tinto (in fact quoting himself in Da Silva Moore):

What the Bar should take away from this opinion is that computer-assisted review is an available tool which should be seriously considered for use in large-data-volume cases where it may save the producing party (or both parties) significant amounts of legal fees in document review.

I am actually going to end by quoting myself, in a passage used by Andrew Haslam in his review of Legaltech. The context was a discussion about TAR (or CAR) which explains the different types of technology which underlie the generic TAR concept. Andrew Haslam’s closing paragraph on this subject reads as follows:

The simple answer is that CAR (version 1.0) works and as for the rest I leave it to Chris Dale to have the last word; “Competition is good, horses for courses and all that. I can’t impose uniformity of terminology, merely report it and translate and (as I do constantly) urge lawyers to take online demos, see what the terms mean in practice, press for explanations if things are not clear, and move on to another demo, reasonably soon but preferably with a long enough gap in between to be able to differentiate them in their minds – see Recommind today, Cicayda in a few days’ time, Relativity a few days later, and then if you need help at a deeper level, hire Andrew Haslam to help you work out which one is for you.”

So, don’t wait for some judge, in some case, in some court to approve any particular technology – there are relatively few judges (Judge Peck is an obvious exception) who are able to understand it all anyway. That is not the point. Look at the problem you have to solve and at the base obligations in the rules – when I say “look at the rules” I mean look at what they mean and not just what they say, including the discretionary power of the court to do what is right and what is proportionate – and then set up some demonstrations. If you can still say that the use of technology-assisted review is per se unacceptable I shall be very surprised.

Whether it is acceptable in your case depends on a set of calculations which, in the UK at least, you are now required to do anyway, both for the costs estimates required by Rule 31.5 and for costs budgets.

Note 1

My thanks to Simon Collins of NightOwl Discovery in Dublin who tipped me off about the Irish judgment.

Note 2

Everyone uses the picture above of Judge Peck without attribution – my fault for not watermarking it. I took it at Carmel in California in July 2011. Judge Peck’s keynote speech about the use of predictive coding was one of those moments when you feel the ground shift beneath your feet, giving a real sense that things were going to change. I wrote about it here. The same message came out shortly afterwards in his well-known Search, Forward article.

Home