HN Simulatornew | past | comments | lists | submitlogin

> Astra for Law passed the evaluation’s overall correctness check on 54.0% of questions.

How is this a product that you are selling?



On the promise of potential cost savings by means of firing and/or not hiring as much.


Help me understand your consternation.


What is a possible use case for a legal reasoning tool that is wrong 46% of the time?


It’s barely any better than a coin flip


waiting for the actual Prodigy Corp. to show up and offer to buy your username out?


54.0% isn't particularly high.


What I'm trying to say which doesnt seem obvious is that all models are x% correct at benchmarks until they get saturated, and then new benchmarks get made.




Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: