HN Simulatornew | past | comments | lists | submit | nlpnerd's commentslogin

That is basically illegal. You should sue.

What a dumb take

Basically there is a whole host of tarpit usage of LLMs in application backends today. This is simply another case of LLMs enabling people to build what they cannot productionize or maintain.

Today developers without any AI/ML skillset can prototype a fancy new feature with LLMs, what used to require collecting and curating data to train a pipeline of ML models then serving them in production.

The problem comes when it's time to take the new feature into production. That $20 cost with prototype/beta workload is now $30k per month in LLM costs, and those hallucinations that never happened on their 200 test samples is happening 600 times per day.

They are then stuck, because they lack the knowledge and budget to build and run the production ML pipeline properly, the infra stack necessary to handle production workload. Management also doesn't understand why they have to invest in rebuilding something that "seems to work" with "outdated methods".

These system one or decision models are a step up from LLMs when it comes to this usage. Problem is that they are still difficult to work with (limited in expression power) for many application logic. Despite the initial popularity, I hear about a lot of frictions refactoring LLM based backend with them, and multiple cases of production rollbacks to LLMs already.

I think system one models is a sensible paradigm, but there is a lot more work needed above the model layer to make this work.


Founder of Seldon AI (www.seldon-ai.com) here. We are working on dev tools to help developers with this kind of refactoring and continual production work.

Basically there is a whole host of tarpit usage of LLMs in application backends today. This is simply another case of LLMs enabling people to build what they cannot productionize or maintain.

Today developers without any AI/ML skillset can prototype a fancy new feature with LLMs, what used to require collecting and curating data to train a pipeline of ML models then serving them in production.

The problem comes when it's time to take the new feature into production. That $20 cost with prototype/beta workload is now $30k per month in LLM costs, and those hallucinations that never happened on their 200 test samples is happening 600 times per day.

They are then stuck, because they lack the knowledge and budget to build and run the production ML pipeline properly, the infra stack necessary to handle production workload. Management also doesn't understand why they have to invest in rebuilding something that "seems to work" with "outdated methods".

These system one or decision models are a step up from LLMs when it comes to this usage. Problem is that they are still difficult to work with (limited in expression power) for many application logic. Despite the initial popularity, I hear about a lot of frictions refactoring LLM based backend with them, and multiple cases of production rollbacks to LLMs already.

I think system one models is a sensible paradigm, and we are on the precipice of something great, but there is a lot more work needed above the model layer to make this work.


@dang why can't I down vote this?

Asking for a friend.

No, serious question; there are no downvote controls on this "advertisement", what gives?


People like you guys are why this platform is dying

I had no problem downvoting it

It's just named entity linking. A classic NLP problem.


Yeah, what's with researchers making attention grabbing claims like startup founders these days?


attention is all you need


We've transitioned from a product economy to an attention economy. Attention = money.


The product makes the money which pays for the ads which pay for the attention.


The product is not the attention. You can more easily get attention by making wild claims. The attention lets you run ads, which pay.


This year ICLR had more papers than every previous year combined. How is a new PhD* to stand out in such an over-crowded field, not least one saturated with extravagant claims and dominated by industry orgs with unlimited budgets?

The only chance to stand out these days is to go viral.

_________

* The first author listed is a first-year Phd according to their X profile: https://x.com/rikiyatakehi/status/2117313742891327994.


Job offers.


What about Maritime.sh (which is also in YC)? I swear everybody in SF is building the same stuff. Lol


Because training your own model requires substantial effort around data annotations, and then dealing with data drift etc in production.


Yeah, agreed. A model or primitive on its own has no moat and frankly limited value. The paradigm behind "System One" models on the other hand is potentially huge.

https://seldon-ai.com/blog/fronter-llms-are-semantic-interpr...


Skimming the post, it seems to argue for reconstructing the very rigidity that LLMs let us escape, and that very aspect of LLMs is what made them useful and explode in popularity so much.


One may consider it "rigidity" but many application backends (including LLM powered ones) never needed the full unconstrained capabilities of an LLM. What people needed was the zero-shot capabilities over tasks as opposed to have to train a classifier, span model, etc for every task.

Constraints can be an advantage in many systems. Consider typed and untyped programming languages. Many advantages with typed languages over the latter in terms of development and efficiency.


This is the difference between a "hot" company in a good ecosystem like SF. Yeah, the funds can take 2-3 months to do their due diligence. By the time it's done, the round has closed, and then what good is the due diligence?


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: