An interview with Ron Friedmann of Integreon; the legal paradigm shift, predictive coding, document categorization, and more

This interview is part of our new series “Data! Data! Data!” — Cures for a General Counsel’s ESI Nightmares”.  For our introduction to the series click here.

Ron Friedmann is Senior Vice President of Marketing for Integreon.  He is a leading authority on practice support for lawyers.   Ron managed practice support at then Wilmer Cutler (now WilmerHale), was CIO at Mintz, worked for two legal software companies, and ran Prism Legal Consulting.  He is a Trustee of the College of Law Practice Management and on the Board of Governors of the Organization of Legal Professionals.  Ron has a JD from NYU and BA from Oberlin College.  

Integreon is well known for its e-discovery, managed document review, legal process outsourcing (LPO), research and knowledge support, and middle office business services for law firms, corporations, and financial institutions.  Beyond marketing, Ron helps law firms and law departments improve efficiency and effectiveness using Integreon services. 

But it is through his blog, Strategic Legal Technology, that Ron is best known.  Ron has blogged since 2003; in 2009, the ABA Journal selected his blog for its Blawg 100, the ABA’s annual list of the best of the blawgosphere.  The choice was a good one: Ron is as insightful as he is prolific.  His blog is widely read because Ron writes with two decades of experience at the intersection of law practice, law business, outsourcing, e-discovery, knowledge management, and technology.  

We caught up with Ron at The Masters Conference and then at his D.C. area office.  

TPL:   Ron, you have written extensively on the consequences of the paradigm shift in the legal industry.  In a recent blog post you said, “what is bad news for law firms could be good news for legal technology managers.”  Can you elaborate? 

RF:     Law departments must act to reduce legal spend.  If GCs don’t, we may see CFOs and CEOs step in.   So the new normal for law firms won’t be the same as the old; the time for minor adjustments has passed.  The legal market will likely stay flat and see downward price pressure.  Consequently, firms have lost pricing power and face a battle for market share.  Winning that battle will require that firms offer clients more value.  To do so, firms will get serious about process improvement, project management, outsourcing, and alternative fees.  This in turn means that firms must deploy new technology and use old technology more effectively.  This will open in a new chapter — maybe even new volume — in legal technology. Getting there will require more business and technology professionals.  

TPL:   A recent Hildebrandt article discussed the fact that we need to be wary about how we measure all of this stuff.  For instance, the “demand” for legal services — as currently measured in the legal industry — is usually discussed in terms of either lawyer hours, or legal fees, both of which can be relatively easily measured and captured.  But clients don’t “demand” lawyer hours — they demand solutions to legal problems (just as consumers don’t “demand” auto worker hours — they buy a car).  

RF:     Exactly.  The market finally recognizes that “inputs” – hours billed – don’t equate to “outputs”, that is results.  Good legal outcomes depend on smarts and repeatable processes more than on sheer number of hours.  Where bulk work is necessary, clients want more cost-effective ways to do it. As a result, legal process outsourcers (both domestic and offshore) now handle growing volumes of routine work formerly done in-firm by lawyers and legal staff.  Firms themselves are reengineering work processes to reduce inefficiency and cost.  This trend will accelerate as fixed fees and other risk sharing alternative fee arrangements spread.  

TPL:   So you see this pretty much as a golden opportunity for vendors? 

RF:     Yes.  And it especially creates opportunity for Integreon’s legal process outsourcing (LPO) service.  A recent article in The Economist noted that LPO is booming.  Law firms can now parcel out more of their basic work so they can focus on their core strength of legal advice and strategy.  

TPL:   We are going to post a lengthy piece on legal process outsourcing during LegalTech and we’ll cover Integreon’s services then, so we’ll hold off on those questions right now.  This is an area that most affects our contract attorney membership. 

RF:     Ok, but let me just make a short comment now.  Legal process outsourcing may very well see a watershed year in 2010.  Since the Rio Tinto law department announced its LPO initiative in June 2009, the U.K. legal press has seen a flurry of LPO announcements.  There is less PR in the US, but many private conversations are taking place. 

TPL:   Fair enough. I do want to discuss LPO with you in more detail at LegalTech.  But one other relatively new development has an even more direct affect on the contract attorney market and we have discussed it many times: the rapid move toward predictive coding technology and machine review.  In 2009 saw two “first pass” document reviews that actually skipped human review and were done by machine.  What are your thoughts on predictive coding? 

RF:     I have been passionate about improving the document review process for two decades.  That’s a big reason I joined Integreon – it shares that passion.  Automation is the key to improvement, so I’ve thought about the role of computers a lot and my thinking has evolved.  

We will see computers play a bigger and bigger role in first-pass document review, at least for responsiveness.   This role can range from culling documents, to prioritizing them, to automatically (or predictively) coding them.  What’s driving this? 

As a society, we simply cannot afford to pay people to look at every document.  As a profession, we must recognize that human review as the “gold standard” makes no sense.  Both anecdotes and studies suggest that human review is not nearly as consistent or reliable as lawyers typically assume it is.  

Computers turn out to be more reliable and consistent than people – no surprise.  But it is not so much the particular software that drives this conclusion as it is the combination of technology, process, and training. 

My blog posts on e-discovery and litigation support show the evolution in my thinking.  In July 2003, I started with Thoughts on Full Text Retrieval, where I questioned the value of concept search versus Boolean search but concluded that the choice is an empirical, not theoretical question.  As volumes grew, technology improved, and we gained experience, my views shifted.  By November 2009, I concluded that the Choice of Concept Search Tool in e-Discovery May Matter Less Than You Think (November 2009).  In that post, based on dialog with two leading EDD experts, Tom O’Connor and Herb Roitblatt, I posited that the semantic engine is less important than the overall process, which includes training and how the tool is used. 

In sum, three trends – (1) pressure to spend less, (2) improvements in process and technology, and (3) growing recognition of the limits of human accuracy – will converge and lead to more “predictive coding”. 

TPL:   OK, I see you’ve really thought about this.  So where do you come out on early case assessment (ECA) — winnowing relevant data down to reduce the number of documents to review — versus predictive coding?  Was ECA the mantra in 2009 and predictive coding the “new new thing” for 2010? 

RF:     I don’t draw that big a distinction between ECA and predictive coding.  The goal of both is to reduce the volume of documents that humans must review.  

ECA uses computers to winnow the number of documents lawyers must review.   It uses several techniques, for example, selecting custodians, narrowing data ranges, applying keywords, and categorizing and prioritizing documents.  ECA allows for strategic decisions, such as settle or litigate, before incurring the significant cost of substantial discovery.   

The goal of predictive coding is to use computers to substitute software judgment for human judgment.  That sounds quite different from ECA but I think both are points on a continuum.  Most predictive coding systems require human reviewers to train and tune them.  And most lawyers will still want to vet predictive coding with humans, at least on a sampling basis.  So I focus on the number of documents ‘humans must touch’ rather than ECA versus predictive coding. 

The two share other similarities.  Both can be used to prioritize documents for review.  Both require maintaining “defensibility” – can you show in court that you have taken all the appropriate and reasonable and steps and documented them carefully.  

TPL:   And what’s your view of the computer assisted review study by Patrick Oot, Herb Roitblat, and Anne Kershaw — “Document Categorization in Legal Electronic Discovery: Computer Classification vs. Manual Review “?  Do you think the technology is accurate enough to take the place of contract reviewers and — more importantly — do you think the technology is fully defensible and do you think judges have demonstrated that they consider automated review acceptable? 

RF:     I read the study and blog posts about it and am still mulling the implications.  At minimum, it advances the discussion about humans versus computer review and shows the value of a controlled, statistical approach.  

It should alert lawyers to the notion that humans are not all that accurate and computers are typically more consistent.  My hope is that it fosters a dialog on the issue and perhaps even helps shift the burden of proof.  Imagine going into court and having to explain why you used an army of lawyers instead of software! 

But I don’t think the study sets up a replay of John Henry versus the steam engine.  That is, it’s not man versus machine; rather, it’s how do we integrate the two in a cost-effective process that holds up both to judicial and statistical scrutiny.  And by the way, I don’t think lawyers or judges give statistics sufficient consideration.  

So, picking up on the ECA and predictive coding themes, I don’t think technology will eliminate review lawyers.  I do think that over time, as processes improve and as courts gain experience, computers will substitute for increasing amounts of contract lawyer review time.  I see a future where armies of contract lawyers as the norm will be replaced by much smaller teams of specialized review attorneys.  Whether that specialized role is contract or full-time is hard to say. 

Though I think automation is defensible, we are at risk for bad rulings, either because a party automates poorly or because, in spite of doing it well, a judge just does not understand the process. 

TPL:   So we are headed down the path to where machines can be statistically proven to be as accurate as human reviewer?  Is the technology getting to the point where we can also winnow out the eyeballs — contract attorney reviewers?  No room for a human element and perspective? 

RF:     I’ll repeat what I said above: it’s man and machine, not man versus machine.  I expect that computers do assume a larger and larger role in doc review.  Society and litigants will suffer if that’s not true. 

For the foreseeable future, however, it’s hard to imagine eliminating all human review.  As technology and process improve, I do think it is likely that the ratio of reviewers to gigabytes will decline.  That is, over time, the human effort will be less focused on what I call “bulk review.”  Instead, humans will likely focus on upfront work (ECA and “system training”), on vetting and tuning computer processes, and on reviewing results to refine systems. That’s why we’ll likely see the rise of specialized review attorneys, lawyers who understand law, technology, process, and statistics. 

TPL:   We recently attended an all-day conference in Washington, DC sponsored by Apple and saw some of the work being done by Google on auto-categorization and auto-coding.  Developers told us that Microsoft is also in the race.  Do you think it is just a matter of time before goliaths like Google and Microsoft — with multi-billion dollar budgets — jump into auto-categorization and auto-coding and wipe out the majority of vendors? 

RF:   As I mentioned above, I think the choice of tool matters less than the overall process.   Unless Google or Microsoft come up with a break-through in computational linguistics (the math behind conceptual search), it’s hard for me to imagine a new search tool vastly superior to what’s available today.  I’m not aware of any algorithmic breakthroughs in the last two decades so I don’t expect any soon.  The breakthrough Google made was using web links as “voting engine” – an approach that does not work for corporate document collections, which are not linked. 

Of course, either company could likely spend its way to a big EDD market share, even with “me too” technology.  That does not appear to be their strategy and EDD is small beans compared to web searching or software, so it seems unlikely to me. 

TPL:   And what do you think is at the forefront of the discovery process, the most important thing, the biggest challenge? 

RF:     I think the biggest challenge and opportunity is creating a consistent, reproducible, documented, and defensible approach that integrates the best of technology and standard processes.  The challenge is not who has the best algorithm, the best software, or the best reviewers.  Rather, the challenge is putting algorithm, software, and reviewer together into an economically affordable, statistically sound, and judicially defensible process.  And that must be done in a consistent, repeatable, industrially controlled process.  Think workflows, documentation, training, metrics, formal quality control, feedback loops, sampling, etc. 

Oh, and let’s not forget that the end-game is not doc review and defensibility.  The end game is telling the best story in litigation, or avoiding needless litigation when a settlement might make more sense.  So the challenge for the legal market is to re-focus from discovery to fact-finding and story telling.  With the right process, lawyers will be able to stop worrying about doc review and possible sanctions and instead focus on figuring out who said what and when and interpreting the facts as favorably as possible.    

Discovery used to be a side show and moved into the main ring in the last decade.  I don’t think that will last. 

TPL:   Ron, we greatly appreciate your time.  We’ll chat more at LegalTech. 

RF:     I appreciate the opportunity to share my views.  And more importantly, I think it’s great that the Posse List has become a key voice in the discussion around e-discovery, bringing together interviews, news summaries, and resources that advance the field and help contract lawyers find work. 

Note to readers and attendees of LegalTech in New York:  On February 1, 2, and 3, in New York, as part of the Legal Tech show, Integreon is exhibiting at booth 324 (first level of exhibits).  Also, many senior Integreon EDD and middle office outsourcing professionals will be at Legal Tech.  If you would like to connect with Ron or anyone else from Integreon, during LegalTech, contact Ron at marketing@integreon.com.