What on earth is this work that they're doing that's so resilient to the fallible nature of LLMs? Is it just document search with a RAG?

Everything. Drafting correspondence, pleadings discovery, discovery responses. Reviewing all of the same. Reviewing depositions, drafting deposition outlines.

Everything that is “word processing,” and that’s a lot.

Well that's terrifying. Good luck to them.

To be honest, much of contract law is formal boilerplate. I can understand why they'd want to move their role to 'review' instead of 'generate'

So, instead of fixing the issue (legal documents becoming a barely manageable mess) they’re investing money into making it… even worse?

This world is so messed up.

Arguably the same problem is occurs in programming: Anything so formulaic and common that an LLM can regurgitate it with a decent level of reliability... is something that ought to have been folded into method/library already.

Or it already exists in some howto documentation, but nobody wanted to skim the documentation.

They have no lever with which to fix the issue.

Why not just move over to forms with structured input?

As a customer of legal work for 20 years, it is also way (way way) faster and cheaper to draft a contract with Claude (total work ~1 hour, even with complex back-and-forth ; you don't want to try to one-shot it in a single prompt) and then pay a law firm their top dollar-per-hour consulting to review/amend the contract (you can get to the final version in a day).

Versus the old way of asking them to write the contract, where they'll blatantly re-use some boilerplate (sometimes the name of a previous client's company will still be in there) and then take 2 weeks to get back to you with Draft #1, charging 10x as much.

Good law firms won’t charge you for using their boilerplates, only the time to customize it for your use case.

I anlways ask our lawyer whether or not they have a boilerplate when I need a contract written up. They usually do.

That's interesting. I've never had a law firm be straightforward about the (obvious) fact they'll be using a boilerplate.

I've even found that when lawyers send a document for one of my companies, and I give them a list of things to fix, including e.g. typos, the same typos will be in there if we need a similar document a year later for another company (because, well, nobody updated the boilerplate)

Do you ask about the boilerplate before or after you ask for a quote?

I typically don’t ask for a quote upfront since they are very fair with their business and billing practices.

I could definitely see a large law firm (Orrick, Venable, Cooley, Fenwick) doing what you describe. I’ve worked with 2 firms just listed, and their billing practices were ridiculous.

I’ve had a lot more success (quality and price) working with boutique law firms, where your point of contact is always a partner instead of your account permanently being pawned off to an associate.

Email is in profile if you want an intro to the law firm I use. Great boutique firm based in Bay Area and extremely good price/quality/value.

Yeah the industries LLMs will disrupt the most are the ones who gatekeep busywork. SWE falls into this to some degree but other professions are more guilty than us. They dont replace intelligence they just surface jobs which never really required much intelligence to begin with.

I bet they still charge for all the hours though.