News You Can Use

Edition 4 · 14th - 31st Oct 2024

News You Can Use

Deep Dives

Three stories worth sitting with

Clifford Chance Copilot Journey

What
Clifford Chance shared details about their full roll out of Copilot. They have not been working with other vendors for the time being, focusing on what they could get out of Microsoft. CC are in a strong position through previous development work and close relationships with Microsoft, with their data science team working on AI for a number of years.
So what
CC have rolled out Copilot with a strategy to address the risks, spending time on setting ground rules for the wider IT and Innovation team, led by the CIO and Chief Risk Officer. It still shows that not all use cases can be done with Copilot, and they are working on a case by case basis, with some difficult conversations with clients around data and use of AI. Key use cases for CC seem to be summarisation and search, so working within O365 for internal support rather than client facing work. It's a good case study to look at, as we have similar challenges, but we have looked to other solutions and focused on internal development, rather than Copilot.

PWC Annual Report on Law Firms

What
PwC's annual report highlights financials, tech, and strategy trends, with a new focus on GenAI adoption. Key findings show a divide: the Top 10 law firms are optimistic about GenAI, with 83% seeing it as a way to increase client work, while firms ranked 11-100 are more cautious, fearing it may lower fees or reduce demand. Notably, 90% of the Top 100 firms have trialled or implemented GenAI (up from 55% in 2023), though only 19% report measurable gains in productivity or revenue.
So what
These insights can guide client conversations, showing GenAI's uncertain impact across the sector. Only 1/3rd of firms expect cost savings from GenAI, with many concerned it could shrink work volumes. Internally, this highlights areas of resistance and external trends we'll need to address with our team and clients.

Agentic AI

What
Two updates on AI Agents from the market, Microsoft launched Agents at their AI Tour recently, with some additions to Copilot Studio and some case studies with Clifford Chance and McKinsey. Claude also launched the ability to run Claude on your computer, meaning you can give it control and assign it a task to achieve. This is much closer to the actual vision of an AI Agent.
So what
Agents are getting closer and closer, with solutions in the market allowing you to build out more complex workflows that are AI driven. Claude's release is the closest so far to a fully autonomous agent that can achieve specific objectives, but the potential in other solutions is that we can build out a workflow that deals with triage or instruction management without human input. The Copilot Studio case study from McKinsey is similar to what we want to do with a future Legal Front Door, so work is underway to properly test this.

AI Benchmarking

What
There are currently a number of benchmarking initiatives across the market trying to score different AI Vendors. We are part of the ongoing LITIG one, but there was recently an announcement from Vals.ai that it is working on a vendor led benchmark with The Legal Tech Hub.
So what
Our work in The RAG Report has garnered interest in relation to benchmarking, with people asking whether we would like to support other initiatives. It is important to figure out how we grade AI tools and when we decide if they are good or not, at the moment this is subjective, so work to develop a standard will be well worth it. We will still be running our own tests, but will look to be involved in the market where relevant.

Worth Reading

Everything else worth a click

Clifford Chance Copilot journey

Inside view of Clifford Chance's Copilot rollout - traffic-light data classification, mandatory training, and the pragmatic payback maths of 15 minutes per lawyer per week.

PWC Law Firms Survey 2024 (PDF)

[Internal AG resource] PwC's 2024 Law Firms Survey - benchmarking data on firm performance, profitability, and technology investment trends across the UK legal market.

When you give a Claude a mouse

Ethan Mollick puts Claude's new computer-use capability through its paces, including a Great Gatsby lesson plan and a paperclip-clicker game - a practical look at where agents are strong and fragile.

MS Agents release

Microsoft's Copilot Studio announcement of autonomous agents with triggers, dynamic planning, and an activity log - with Clifford Chance name-checked as an early adopter.

Vals AI VLAIR benchmark

Vals Legal AI Report (VLAIR) - the first independent head-to-head of legal AI tools from Thomson Reuters, vLex, Harvey, and Vecflow across seven tasks, benchmarked against a lawyer baseline.

Law Society of Scotland AI Guide (PDF)

[Internal AG resource] The Law Society of Scotland's AI Guide - practical guidance for Scottish solicitors, drafted in collaboration with Wordsmith AI.

Australian Gov trial of Copilot (PDF)

[Internal AG resource] Evaluation of the Australian Government's Copilot trial - one of the largest public sector Copilot pilots published to date, with useful adoption and time-saving data.

Google Gemini Prompt Guide (PDF)

[Internal AG resource] Google's official Gemini prompting guide for creatives and strategists - structured prompt patterns and worked examples from the Gemini team.

Dr Megan Ma on Stanford work

Dr Megan Ma from Stanford CodeX on M&A negotiation simulators, AI personas of senior partners for associate training, and bringing GenAI into the law school classroom.

Stanford ML - Building LLMs

Stanford CS229 lecture on building large language models - the technical foundations (pre-training, data, scaling, evaluation) for anyone who wants the engineering view.

Legal Tech Trends Newsletter #34

Peter Duffy's scan covers Clifford Chance's Copilot story, the compounding error problem in agentic workflows, our RAG research, and KPMG's ContractPodAi tie-up.