This is a cycle. As of now the companies with server farms are buying up everything that can be manufactured, but this is not a forever model. The companies that are in the business of making things will always want to make more in the future and they will need to come back to general consumers to soak up that supply. The only question is how long it will take. I was hoping the GPU situation to normalise after the blockchain hype started to die out, but AI picked up that demand and shot it up to the stratosphere
When all this is done, will general consumers even be interested in general purpose computers, vs say walled garden devices that can run their office apps and slack client? Will the hobbyist market of people building little linux home servers to run their containers or whatever be even large enough to justify investing in that sort of business at all? It is already a market where many people participating in it are buying hardware second hand, bevvied by the fact that enterprise has at least up till recently bought their own compute and replaced it every three years, creating plenty of constant inventory suplly of anything from capable enough clusterable i5 usff pcs to xeon powered rack servers.
But that was then, this is now. I've been realizing lately how perilous general purpose compute has been, and how much of its continued access depends on things remaining exactly as they've been for the last 25 years.
As an analyst who vibe codes a lot I think it is perfect, along with Product Manager. There is a lot more to these roles, but I think they're the closest.
Product rather than project makes much more sense for vibe coding.
It's a weird one, because recently I've been working on a JS based system when I don't know JS and it's definitely very very different from when it's a language you're familiar with.
These are essentially zero-shot classifiers; they don't need to be trained for a specific classification task. You could include some natural language context on the rules for classification and it should get good enough accuracy.
At least a year ago they were not that accurate when compared to a good many-shot classifier that (1) has 10k training examples and (2) has the learning capacity to learn from 10k training examples (most fine tuned BERTs don't seem to.)
I'm skeptical of the quality of probability calibration for models if you aren't giving them training data. The issue is that there is a prior distribution that's unique to your specific data and the calibration is really sensitive to that.
1) you can parallelize the request
2) since a structured output is still just text then every previous property of the structured output effects subsequent ones
This isn't really anything new it just seems like a new API but you could do the exact same thing with just a little bit of prompt engineering all the way back when GPT-3 was first released. Am I missing something?
No amount of prompt engineering will give you the true probabilities for the model producing a certain response; this is something you can only get by inspecting the internal state at inference time.
Does this give you the true probabilities for a certain response either? How does it work exactly? The probability of an overall positive answer isn't just the probability that the next token is "Yes"
For most use cases is that actually needed though? Just having it choose between predefined responses seems like enough but I'm curious about specific use cases because I do feel like I'm missing something
This is useful for classification problems; any time you need to write software that looks at some fuzzy data and needs to make a probabilistic decision. It's far more cost-efficient and performant to use this type of model instead of an LLM.
Before now you had to train a model on your specific classification problem, now these new models don't require any specific training at all to do pretty well on novel problems.
I understand but you can already do that with an LLM, it just comes down to your prompting. It is not exactly the same, but I was asking for a specific use case where that really matters
Becuase it is multiple orders of magnitude slower and more expensive to do it with an LLM. Look at Typesafe's Doom demo where they run complex queries tens of times per second for around $7 an hour.
There are classes of problem where this shifts the economics from "paying a human to do this is cheaper than AI" to "the software is now cheaper than the human"
I've asked twice now about what I'm missing and for a specific use case where you can't just do this with a regular LLM call and nobody has replied that so if you have the answer that would be great. Looking for something specific instead of just it's faster or cheaper which is definitely nice but I'm just not seeing what this opens up that was not previously possible
I don't have as deep an understanding as most people here, but I have a mental model of it, so I may be able to help more, ironically.
Yes, you could ask an LLM to choose between 4 options, and sure have it include a confidence score. But at the end of the day it's still LLM-ing and outputting tokens based on the total probability of the chain of tokens that precedes it to figure out what's most likely to come next.
This taps into a lower level of the model, skipping that tedious token generation step, and exposes you to more raw output from the model, e.g. the actual probabilities involved in answering the question.
It still needs to "think like an LLM" to interpret the question, so it's in the same family, but the thought process gets short-circuited once the question is understood and the raw weights are then extracted as pure, weighted probabilities.
Why would you want to use it? What does it enable beyond a traditional LLM?
(a) It's much more efficient (read: cheaper and faster), and not by a %, but an order of magnitude or two. This makes it practical to be used when an LLM isn't, due to its speed and accuracy, which opens up new use cases.
(b) By getting the raw options and weights, you're not getting the LLM's "spin" on the output, nor hallucination risk — instead, you're using an MRI to pull out exactly what it's thinking before the bullshit verbalization engine kicks in.
Does that help? Please correct me if I'm wrong or went too far in simplifying it, but that's my high-level understanding. I hope this helps.
You’re getting answers, you just don’t like them. It’s faster and cheaper. And if you were to ask an LLM to give you a confidence score it would just be a fabrication.
I was curious about specific use cases because at the end of the day in either way you are relying on an LLM to interpret the query so whether you use a probability threshold or prompting techniques it would be a similar result, so yes this new method is much, much nicer but doesn't open up brand new use cases from what I see
It is an absolute requirement if you want to build something that (1) uses a classifier as part of a larger system, (2) can be let off the leash with no human supervision and (3) is not an AI slop demo that will be [dead] and [flagged] in seconds after being posted to HN.
For instance if you have a predictive model for market prices that is not calibrated that's... nice. If you have a calibrated model you can add a Kelly better and you have a trading strategy that makes money. Similarly if you are classifying articles or images or other contents to make a feed you might believe that 70% or 95% or some other level of precision is "good enough" and you can set the knob and turn on the cruise control.
OMG what a blast from the past. We were taught FoxPro in school in 2002 for some reason. Not even Visual FoxPro, the old school DOS version. It taught me a lot about scripting and I remember helping out a medical rep who had built a FoxPro app to manage his business while in college.
Yes you can! Chart can have links assigned, either automatically or dynamically. Those links can either take you to drill down pages with the data listed out, or it can dynamically change filter/variable values and change the data displayed on the whole dashboard. See https://docs.dbtcharts.com/boards/linking/?h=links
The actual reason was when they pushed Google+ and tried to make that the centre of Google. Everything else that had a social element had to be killed. Anything that breathed the same oxygen as plus got the boot.
Including the inanity of the + operator being a "this word unchanged must appear in results" modifier. So when they hot patched search, it broke, as +aliens looked for the plus name aliens only.
Then some bonkers spokesperson for google said "oh just use quotes", which at the time did nothing even remotely the same. I loath this person for eternity, her lies, and her waving off of reporters concerns.
Thus verbatim was introduced, validating quotes weren't the same, yet which wouldn't work with date ranges, and is lame.
So goes the "professionalisn" of Google, or "screw around like uncoordinated idiots".
To say Plus was inane and destructive to Google is vastly understating. And everyone involved in Plus were buffoons.
You know, I wish could speak freely on this topic, but as a public forum, I have held back some vitriol.
reply