News You Can Use

OpenAI ran a day for developers and shared what they have been working on. This includes real time voice APIs, prompt caching (which means that elements of prompts used before can be used again), model distillation, to allow fine tuning of smaller models.

Better voice, prompt caching and small model fine tuning all adds up to give us better tools for building out complex agents. Prompt caching and the ability to distill models means you could set up specific agents using this framework and fine tune them to a task - like negotiating a document as per a set of rules with fine-tuned knowledge beyond that which you include in the prompt - you could then dial your agent into a Teams call and run it in real time!

NotebookLM is a tool from Google that allows you to upload documents to a notebook style LLM powered window and generate AI notes, briefings or text. You can now also create podcasts!

This is a good way of storing notes together and getting decent summaries, the RAG Report briefing and podcast came from NotebookLM. The podcast feature could be a good way of condensing a lot of information into a relatively easy to digest 10 minute clip. It shows progress being made and could be used to create training snippets from detailed training information, or provide real time voice Q&A as it develops. It is similar to OpenAI 'canvas' which lets you work alongside a document, mainly for writing code.

A report from the Association of Corporate Counsel and Everlaw based on a survey of 475 in-house lawyers in the US. They showed that there is an appetite for GenAI, ongoing adoption and in-house seem to be leading the charge, however that most of them expect legal services to get cheaper. Whilst there was a lot of excitement about the potential on careers and work, a large group thought that their legal team is unprepared to actually address changes from GenAI.

There are some good findings within this, even though it's US only a lot of the messages we hear are the same. It gives us some validation on pushing out our workshop offerings as there is a mix of excitement with a lack of confidence and understanding which some more focused training would help with. Some of the stories and feedback indicates that in-house professionals are really keen to skill up on AI, with a belief that this will support them in their role and help to bring more tasks in-house rather than outsourcing. This seems contrary to the private practice culture around GenAI at the moment.

Research undertaken by Apple shows that LLMs do not perform as well as expected on reasoning problems. This may explain why Apple have not been as gung-ho around GenAI, but it also focuses mainly on logic and reasoning in the pure sense rather than applying to specific tasks. This research was done with open source models and some smaller models, so may not be representative of the more common / larger models.

An interesting perspective for the market, one of the big tech companies slightly putting the brakes on, whilst also going all in on things like Apple Intelligence in their consumer devices. However, they saw some similar results to us, around complexity increases in input leading to decreases in quality of output, and maths not being a strong point. But where it comes to hiding information / incorrect answers within documents, this was something testing in the Deal Clarity research and GPT-4 performed well.