> It seems like LLMs are great for the kind of generic work you'd usually pawn off to a new associate or paralegal

This seems to be the case with most industries that LLMs are impacting, coding, data input, legal etc.

Whilst it's great to get the time saving from not delegating to a junior (LLMs are much faster at the basic level 1 work) there's going to be this big knowledge gap that you really only gain from actually doing the mundane boring shit.

A lot of this _should_ be automated away, but we really need to think about how the next group of people are going to learn the skills and nuance to applying them. I don't really have an answer to that one other than still giving that work to junior staff, but they will then just utilise an LLM to help complete the job, and rightfully so, I'd have done exactly the same thing.

I think OpenAI tried doing this with some early chatgpt settings with "learning mode" or whatever that is. But an LLM just telling you what to do will never teach someone, there was something about having to go out and hunt for the answer through the sea of crap responses or wrong SO questions that really taught you something. How do you replicate that learning with an LLM and how do you apply that to different industries like Law?

so they eliminate your talent pipeline and cut down on billable hours? is this good?

how much 'novel' tasks are there in law?

> It seems like LLMs are great for the kind of generic work you'd usually pawn off to a new associate or paralegal, but not for the important things that partners usually do.

This seems to be the common thread everywhere LLMs end up being useful. As the sole software developer? Pure trash, but as "tons of juniors we can shuffle boring tasks at": great. As the sole designer and maintainer of a website? It'll look like 150% slop and no will buy anything, but as a designers friend to create what they dictate: even a designer can throw together a good looking website quickly. Etc etc.

> Anthropic is clearly stating they will use their own judgment to control what people can do with its product

Haven't they been perfectly clear about that from day one with the ToS? I'm not saying I like it but it hardly seems unreasonable.

> they will treat customer data as if it's theirs to disseminate

Not sure where you're getting that. What you quoted in no way implies that.

The traditional legal data brokers like Thompson Reuters provide AI solutions presumably with considerably better data governance.

https://www.thomsonreuters.com/en-us/posts/innovation/thomso...

it's like Uber, for lawyers!

Nobody will have any interest if they are not open

Imo, we are going to see a lot more specialised models in the near future as pushing the frontier gets harder. Qwen 3.8 27B for example is a pretty rubbish chat and general world knowledge model, but is stellar at coding (for it's size). Its small enough anyone with a modest gaming PC (16GB vram and 32GB of DDR5) can run it at speeds good enough for overnight coding sessions with an PR to review at the end of it.

Eventually, I hope we'll see many more specialised small models that run nicely on consumer and prosumer hardware. Like LLMs specific to income tax law for UK residents, or LLMs specific to building power delivery circuits for PD USB C devices. There probably is a point where the niche-ness does become to niche to justify the post training, but again, small models means much less training required.

Kind of begs the question if this even needs new/better data? Could you take the foundation model that's trained on everything, and then run a final fine tuning training run on just the set of foundation training data that you want the model to specialise in? Eg, extract just the training data that is about UK income tax law and then run a few epochos on just that? Any ML people know if this is a thing or if not, why it doesnt work?

If you’re on a corporate device you may not have recently issued certificates in your cert chain yet

Not the case for me.

It will lead to bigger fines :-) https://www.damiencharlotin.com/hallucinations/

Until it hallucinates something no one bothers to verify

The benchmark is US law, I imagine it's been aggressively post-trained on it.

The legal space understands well how to tier services based on who has the liability. E.g. trainees and paralegals whose work is filtered through the lawyer who ultimately puts their name on what gets submitted to courts.

I'm sure we'll continue to see the occasional person who fails to understand they themselves will be held liable if they submit work done by an AI without vetting it, but it's not like they don't have ways of dealing with this.

This is why we see so many pro-se litigants. "Legalese" is no longer the barrier (not to nearly the same extent, anyway).

Hopefully. Giving everyone cheap and equal access to the law cannot be a bad thing--well, it might feel bad to those who no longer have exclusive access to the law.

I think an under-appreciate way AIs might disrupt the world is by giving individuals an agent that will ruthlessly optimize for them.

Companies have entire departments that spend all their time ruthlessly optimizing so they can charge an extra fee here, or spin up a new shell company there. Meanwhile, here I am in our capitalist society and I just enjoy relaxing and spending time with my wife and kids instead of maximally exploiting the capital I possess at all times--like a fool.

It will be interesting if we ever integrate AI so fully in our lives that the AI can maximally exploit all our capital. My AI might check prices, and I will drive past your store to save 5 bucks at the next. My AI might help me manage 27 LLCs and shell companies of my own so that I too can just disappear in a poof of legal smoke if I ever need to. Etc.

While I agree on the premise – (written*) lawyer activity being reduced to putting signatures under things – I’m not so sure the gilded part is how it’ll actually play out in practice.

It might also become a race to the bottom, since you can’t deliver on quality better than others anymore, you’ll have to put effort towards quantity. That could mean firms might pressure lawyers to sign as much as possible in a given time, while they’re still personally at risk/responsible for the documents they put out. That means either very hectic reading, and missing details, or no reading at all.

*And like socializer said, the written documents are only one part of the job.

That's already a thing. You have paralegals do most of the grungy work for much less than your hourly rate. You still have to show up to court and talk to customers, and these parts are not going away.

Or... the tools get to the point where we don't need a human for expertise but accountability and the role becomes more about ethics, thereby lowering the bar. (PUN!)

I was about to suggest that AI might teach people to become lawyers for cheap and they can just take the BAR exam and become a lawyer...

But then I looked into it and I see that you actually do have to go to certain schools to become a lawyer. Having the knowledge is not enough, you have to pass through the proper (and expensive) gates as well.

I'm not really surprised.

This is just SWEs and Consultants discovering the utility of professional regulation. It's still a tough job.

Do you predict the same paralegal crisis people are claiming about junior swe's?

don't most developers do this already? Senior developers. Juniors are having a hard time.

If you actually did the tasks in the way they deem correct, at least for immigration and white collar criminal defense, you will get detaind/arrested, and removed in the first case, and the second case you will snitch. It assumes that the state is correct. In reality the state, especially in the initial phase, are never correct on all aspects. You also can't only plan on snitching because what evidence exists and what evidence is admissible and what evidence is admissible but can be used in your advantage are all unknowns. Half of the bench tests on "covering your own ass" basically, and also, seems to have not taken the Loper Bright decision into account, which changes administrative law and how it is adjudicated and appealed entirely, for good or ill.

I don't know the other fields well enough, but if all your clients end up needlessly detained or admit to things they have a valid defense for, eventually people will notice. It's not exactly the two major sins that will definitely get you disbarred - stealing money and screwing client while representing them - but it's definitely at least incompetent. They seem to not understand that common law jurisdictions are fundamentally different from civil ones, and also, there's quite a bit of difference between how the system evolved in each country. Or that in China what's printed is basically more like a manifesto, since it's almost entirely run by the bureaucracy and personal relationships, and lawyers that aren't foreign-trained and work for foreign firms aren't considered prestigious since the courts are essentially entirely for show.

They need to get like, one jurisdiction, one area in that jurisdiction, and specifically the kind of sub-sections in that area down pat. But they can't train on real material so they will never be able to do so. Their banner testimonials come from non-lawyers and that says it all.

Of course. That’s why I opened the issue so that the team can improve it.

Every existing, and not yet existing benchmark.

That's how I feel reading HN's opinions on the recent math results. <5% of the responses even understand the point they're supposed to be arguing. AI might take over mathematics. It might not. The discussion regarding that topic is almost devoid of any relevant information though.

Lawyers would be better suited to argue what programming is because they know logic. There’s not much to algorithms: do this, then do that.

Programmers typically don’t know law outside of which license allows them to get code for free.

> probably none of these people have a law education.

Domain knowledge is informative sometimes but mainly overrated: the success of LLMs proves just that. What most lawyers do is fairly standardized and routine, efforts to invoke mystery cases or "novel" is an attempt at resisting the inevitable. The "the only British food in Palo Alto" equivalent of unique skills.

Wouldn't be surprised, if a human attorney would get 5%, while copy-pasting ChatGPT answers, report in the results a week past the deadline, with a $12,500 retainer charged upfront.

Totally fair. However, even if the LLM would come to the same conclusion and produce the same content that a competent attorney would, the attorney must still thoroughly vet the work for accuracy. So there is potentially time saved in research and drafting, but it’s not the miracle some might characterize it as.

An 80% miss rate on a benchmark suggests to me that automation has a very long way to go, even for the drafting and research phases.

I'm guessing what they're saying is that nothing goes to litigation if it's a clear pattern-match to existing cases, so actual litigation practice is only novel cases.

Isn't that just in the US/Commonwealth judicial system? I could be completely wrong on how others, like non-adversarial judicial systems, work and/or how pervasive the US system is or is not.

EDIT: specifically around the use of judicial precedent in case-law.

90% of litigation is pressuring a settlement.

> How much novelty can there be?

As much novelty as there is in human interactions.

> How much novelty can there be?

Infinite

I couldn't agree more but feel that so many people on here consider themselves VIP that this won't be well-regarded

What would prevent a Chinese company from selling your legal data to say, your legal adversary via data brokers, in the USA? People everywhere like to make money.

I initially wanted to dislike this but it's actually genius hahaha

It will not protect you. China data brokers are generally not prevented from selling data on US people to US companies.

> I love how we have to pretend like China is some scary boogeyman because people on screens say they are.

I think you've lost the plot a bit on this one.

I'm scared of China because of their lack of democracy and the terrifying stuff they do to their own citizenry, which presumably they'd very much like to expand.

A key point of attorney-client privilege is that it's a feature of the legal system itself, not a technical or political hack.

There's no digital attorney-client privilege. If a lawyer uses LLM's on your behalf, the communications are protected. If you use the same LLM's on your own behalf, the chats are not protected.

DOJ could cut Z.ai off from US-based payment systems (Mastercard/Visa) if they don't comply with subpoenas.

More like attorney-client liability

You should not trust any LLM to give you accurate legal advice. I can’t imagine all the ways that could go wrong.

NATO allies can't even get US soldiers to trial. In Romania we've had cases of manslaughter and the US just shipped them back ASAP and at worst they were discharged dishonorably.

Yes, mechanical question of "does this exist" is trivial to solve. Whether the research is actually aligned with the user intent / the AI makes claims that are substantiated by the research at hand are much harder problems.

> I don’t think hallucinations are a major issue with agentic coding a year later? At least, not for me.

Are you still looking at code and agent traces to know if this is true or not?

At work, we still see hallucinations across all model families as we share bot claims for each others code

An issue I come across is the attributed failures of AI of a first time use of AI.

Non-deterministic software can behave orders of magnitude different based on how input is structured.

This factor is largely missed by bringing a deterministic mindset to trying AI.

This means the same problem solution can work for one person reliably, and not another and both are true.

Assertions that begin with a deterministic assessment using a snapshot in time of both the user and models capabilities are a new feature of this new category of software that will have to be better caught and expressed.

To your second point, it seems like every time a significant practical limitation is identified, it goes through a familiar pattern. It gets a news cycle, and then it gets a wave of confidently incorrect opinionating, and the problem gets treated as if it's a permanently built in problem that's baked into the foundation of AI. And then time passes and the issue is either mitigated or circumvented entirely.

Hallucinations are one, "AI can't scale anymore" is another (remember that one?), limited context windows, strawberry problems, glue on pizza, sycophancy.

I don't want to say there's no problems, there are very serious ones. User psychosis, economic harms, alignment, cost of computer components, etc. But there's a lot of false positives that get turned into Ed Zitron articles along the way.

This capability has already existed for decades thanks to Westlaw and LexisNexis. These are indispensable tools in the lawyer’s kit.

Yes, actually, because Lionel can suffer a consequence. He can be sanctioned. He can be disbarred. He can be sued. He can be thrown in jail.

AI can not meaningfully suffer consequences for what it does, and the trend is to remove the human from the loop more and more.

Furthermore, it's entirely conceivable that you have a law firm staffed by lawyers who more-or-less just say "yeah sure whatever" to what this thing does and, when asked about what can be done to appeal something, or how a document didn't make it into evidence, or how a subpoena wasn't filed, just shrug their shoulders and go "iunno". Vibe-practicing law, basically. You need to have an idea of how they're using AI and in what parts of the case.

There are a lot of situations where my lawyer's mistake might cause me to go to jail for a crime I didn't commit. There could be some prejudicial evidence that he failed to get excluded, or exculpatory evidence that he failed to get included, or an incorrect ruling in the trial that wasn't properly preserved for appeal.

> In which case would this be the primary blame, and not the legal system itself, or you for actually doing the crime?

There are plenty of ways a lawyer can screw up procedurally that can drastically impact your case.

One of the nice quirks of USA justice system is that you don't have to be guilty of a crime to go to prison, just that the prosecution has to convince 12 people beyond reasonable doubt that you are guilty.

The sixth is taken seriously to the point that incompetent lawyer could be a valid reason for granting a retrial.