Deno does not have its own type-checker. The `deno check` command is a wrapper for `tsc`. It will use the Go version if given a flag. The code is not first-party and is not in Rust.
Yes, the video has a transcript available. You'll find a "show transcript" button in the description.
That said, you'd be missing a bunch of hand-made illustrations and animations from one of the best math communicators in the world. If that's not appealing to you, I'm not sure the transcript will do much for you either.
Admittedly, I don't think visualizations do much for me, but I wanted to see the essential pitch without spending 40 minutes watching a video. Surely any discussion of this topic (as opposed to mathematical concepts themselves) doesn't require visualization.
As it turns out, the only bits relevant to the title are the last five minutes of the video, and his contribution comes down to basically ignoring peoples' concerns and asserting that more technology means more joy. He conflates interest in math with a career in math and doesn't seem to address why actual mathematicians go into the field or what they get from it. In fact, the channel seems aimed at people who don't ever intend on going into the field.
If he doesn't grapple with the angst of turning an entire field of discourse into watching a chatbot beam-search a proof space, why make the video?
This isn't really anything new it just seems like a new API but you could do the exact same thing with just a little bit of prompt engineering all the way back when GPT-3 was first released. Am I missing something?
No amount of prompt engineering will give you the true probabilities for the model producing a certain response; this is something you can only get by inspecting the internal state at inference time.
Does this give you the true probabilities for a certain response either? How does it work exactly? The probability of an overall positive answer isn't just the probability that the next token is "Yes"
For most use cases is that actually needed though? Just having it choose between predefined responses seems like enough but I'm curious about specific use cases because I do feel like I'm missing something
This is useful for classification problems; any time you need to write software that looks at some fuzzy data and needs to make a probabilistic decision. It's far more cost-efficient and performant to use this type of model instead of an LLM.
Before now you had to train a model on your specific classification problem, now these new models don't require any specific training at all to do pretty well on novel problems.
I understand but you can already do that with an LLM, it just comes down to your prompting. It is not exactly the same, but I was asking for a specific use case where that really matters
Becuase it is multiple orders of magnitude slower and more expensive to do it with an LLM. Look at Typesafe's Doom demo where they run complex queries tens of times per second for around $7 an hour.
There are classes of problem where this shifts the economics from "paying a human to do this is cheaper than AI" to "the software is now cheaper than the human"
I've asked twice now about what I'm missing and for a specific use case where you can't just do this with a regular LLM call and nobody has replied that so if you have the answer that would be great. Looking for something specific instead of just it's faster or cheaper which is definitely nice but I'm just not seeing what this opens up that was not previously possible
I don't have as deep an understanding as most people here, but I have a mental model of it, so I may be able to help more, ironically.
Yes, you could ask an LLM to choose between 4 options, and sure have it include a confidence score. But at the end of the day it's still LLM-ing and outputting tokens based on the total probability of the chain of tokens that precedes it to figure out what's most likely to come next.
This taps into a lower level of the model, skipping that tedious token generation step, and exposes you to more raw output from the model, e.g. the actual probabilities involved in answering the question.
It still needs to "think like an LLM" to interpret the question, so it's in the same family, but the thought process gets short-circuited once the question is understood and the raw weights are then extracted as pure, weighted probabilities.
Why would you want to use it? What does it enable beyond a traditional LLM?
(a) It's much more efficient (read: cheaper and faster), and not by a %, but an order of magnitude or two. This makes it practical to be used when an LLM isn't, due to its speed and accuracy, which opens up new use cases.
(b) By getting the raw options and weights, you're not getting the LLM's "spin" on the output, nor hallucination risk — instead, you're using an MRI to pull out exactly what it's thinking before the bullshit verbalization engine kicks in.
Does that help? Please correct me if I'm wrong or went too far in simplifying it, but that's my high-level understanding. I hope this helps.
You’re getting answers, you just don’t like them. It’s faster and cheaper. And if you were to ask an LLM to give you a confidence score it would just be a fabrication.
I was curious about specific use cases because at the end of the day in either way you are relying on an LLM to interpret the query so whether you use a probability threshold or prompting techniques it would be a similar result, so yes this new method is much, much nicer but doesn't open up brand new use cases from what I see
It is an absolute requirement if you want to build something that (1) uses a classifier as part of a larger system, (2) can be let off the leash with no human supervision and (3) is not an AI slop demo that will be [dead] and [flagged] in seconds after being posted to HN.
For instance if you have a predictive model for market prices that is not calibrated that's... nice. If you have a calibrated model you can add a Kelly better and you have a trading strategy that makes money. Similarly if you are classifying articles or images or other contents to make a feed you might believe that 70% or 95% or some other level of precision is "good enough" and you can set the knob and turn on the cruise control.
Sidebar: if you're also mitxela on youtube, I just had the strange experience of reading this comment while listening to your voice from a different window!
On Macintosh keyboards it's ⌥⇧-. On the iOS soft keyboard, it's a long-press on the hyphen key. People often point out that various software will turn a double hyphen -- into one by default. We're overflowing with different ways to type an em dash, for those with any curiosity to learn.
I have to be honest. While this is obviously a smart and useful idea, it misses one of the core features of Jev: its confidence scores. Partial confidence could easily be mapped to fractional spaces, using unicode characters like U+2009: THIN SPACE. As it stands, this package is not harnessing the full power of Jev.
important to note that the "confidence" score is... maybe not what people think it is - kind of useless, and just a convenience step from the probabilities.
from the docs: "confidence is a statistic computed from the probability distribution the answer already gives you." [0] I actually encourage people to visit the docs because it has a specific page on this with a little applet to really make this clear.
What else do people think it is? If Typesafe had found a way to measure arbitrary AI results against objective reality (past, future and present) they'd either be making a killing on the stock market or working for the NRO, not publishing that as a confidence value on their API
Well I think there's an expectation that it's similar to the probability score, something that's outputted by the model itself, and so there's some level of "intelligence" (e.g. being able to recognize if the subject matter is relevant to the data it's been trained on, or as an accumulation of errors e.g. it couldn't figure out the question).
Yeah I keep getting this weird sense that Jev is kinda poorly reinventing ML. I guess the graphs don't lie and theoretically I can replace luna with it, but I don't really use luna anyway.
What is the use case for a classifier that works 90% of the time...? I feel like if I'm classifying something, I probably care enough that 90% ain't gonna cut it...
I guess the answer is just agential stuff that effectively gets double checked by the LLM in the driver seat, anyway? That tracks, though it means that jev is mostly just for the people making harnesses. Which is all of us but still!
I think the argument would be that the classifiers of classic ML can be very useful and that Jav is a geenral purpose classifier you can just use that doesn't need to be trained per-task.
I mean when you get your bloodwork done to check for an illness, the test you get will give the right result 90% of the time - and depending on the result, you doc might order more tests, which could be more expensive but no mrpe reliable than the first - but they are going to be statistically independent, and after 2 more, he can be 99.9% sure.
Which begs the question, can Jev retest until it gets the right result? Can it tell how corellated two of its results are? 90% correct makes for a wonderful iterator, but a poor oracle.
Yup, you articulated what I was trying to say but much more clearly (thanks!). I suppose I haven't tried just setting n=3 or something, but presumably the Jev docs would mention that if it were enough to get it from 1 figure to 2 (i.e. 90% -> 99%). I agree that there are places where 90% certainty can help, but it really needs to be an agential system; when a doctor runs more tests, they are experimentally engaging with empirical reality in a context-appropriate way.
I guess, in the end: I think it'll end up being fantastically useful for artificial engineers with their vastly superior ability to keep track of fine details and rapidly context switch, but fairly niche for any of us organic engineers that are left.
All that doesn't apply to low stakes stuff like games, though -- can't wait for the first truly open world game, NGL. A silver lining to the cobalt cloud?
That's an excellent question for the maintainers of the MacPorts project! It can supplement the very clear and polite answer you've already received from a maintainer of Homebrew.
> Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
> Eschew flamebait. Avoid generic tangents. Omit internet tropes.
https://news.ycombinator.com/newsguidelines.html
reply