This is interesting. For a while I used to print out code and debug too - more due to resource constraints: didn't own a computer, and my high school's computers were available only for limited hours per day. I used to code my first iteration also on paper - so that in the limited computer use time, I could type in the code quickly and actually see it run. Wild times but I think it made me good at programming. My friend (who did have a computer at home, but lived far away) and I co-wrote a program called "graph generator" that could plot math functions, with me doing my share on paper and giving to him to type in :) This was in the Turbo C++ IDE and since GUI based programs were unusual (at least as a project in highschool - India, 1999) it temporarily made us popular. Well, wow, that was fun to reminisce!
Even in 2010 at Berkeley I was doing a fair amount of paper coding. Some of the professors gave paper-only tests, and we'd practice a bit since you didn't want a midterm to be the first time you had to do it.
Though I also remember sketching programs on paper as a teenager for my TI-83, since it was annoying to type everything out and then have to make a bunch of edits.
I just learned that X Window System came out a year before the first Windows release. Interesting. Thanks for pointing out the compiz ripoff. It spurred me to look up the original release dates.
Its funny - I was going to leave a similar comment - and I have, earlier, on a different thread. If people need fast classification, on a fairly scoped problem, it is very fruitful to start with an off-the-shelf embedding model like ModernBERT (which Laya uses) and stick a classifier in front - like a Support Vector Machine (SVM). For starters just tune the SVM, you don't even have to fine-tune the embedder - often it works very well, esp. given the compute needed. Plus you can get reliable confidence scores and generate explanations if you want them (using something like SHAP).
I like this post. I haven't had time to dig into Jev (they aren't accepting new signups), but calibrated probabilities is one of their pitches that caught my attention. And I was wondering how does one offer them on user data. Standard calibration essentially ensures that if a score of 0.8 accompanies a positive prediction (assuming the simple case of binary classification), then if you gathered together all predictions with a score of 0.8, around 80% will be correct.
If you have just one example you're sending to a model, how would they guarantee 80% over your data?
FYI, for an overview, scikit's page on calibration is great [1], and my answer on Quora from a long time ago covers a specific type [2].
While I don't believe they are doing the following: you can calibrate by inspecting the reasoning traces. That is the relevant distribution. If you ask someone to explain how/why they are classifying something one way v another, you can get a reasonably good understanding of their confidence level.
This tells me the confidence of the LLM's belief about the response - which is different from the calibrated confidence score. The former also is useful (just not what I thought their advertisement sells - and from the article it seems like it tripped up others as well), and there are different techniques to extract such a value [1] [2], typically via "response sampling", i.e., interrogate the LLM slightly differently to see if it changes its answer.
I mean the model can learn from it during RL training. The confidence score is affected by the tokens prior to it it's output. I was using the word "you" loosely.
I do this too! Usually some idea in ML ...when I am driving I might suddenly remember what I was thinking about and then it is my personal podcast via Gemini-in-Maps. My only complaint is if you have follow up questions, you have to be quick, otherwise it cuts off the mic.
Even as a human being who has a working brain and all this thinking power, I am not sure how I would answer that. It's kind of funny Gemini will still come up with something and even suggest follow-ons to the conversation but saying all that stuff as a human to another human would be a wild response to that question.
Was looking for a video on this specific work, but only found this [1] where the English translation seems AI-generated. It reminded me of thermoacoustic refrigeration which I found cool (heh) when I had encountered it many years ago - where the compression and rarefaction of sound waves is used to pull out heat. The fire extinguisher seems to operate on a different principle though - it needs to generate a specific kind of turbulence to push out oxygen from the fire site.
My first encounter with any kind of study was the G-Eval paper [1]. They study whether their LLM judge prefers human or LLM-generated summaries (answer: it's the latter).
What you call "Weathering" has been a constant gripe of mine. We have LLM-driven softwares toward that almost seem to start from scratch every time a request comes in - there are mechanisms to learn or generalize, like writing out a memory, but they are not reliable or reliable in general. There is no convenient lever to be able to say "yes this is in the memory but the request seems like it needs a fresh scan of data, so ignore your memory", or the opposite "you can infer this from stuff in the memory - don't re-reason!". There is some work like Dynamic Cheatsheets [1] and Agentic Context Engineering [2] that have studied this aspect, but we are far from a generally reliable solution. And till we have that, I think the system variations for systems trying to solve this problem are going to be (a) LLM-leaning: create unstructured memory files, with human in the loop as a filter to reject inaccurate responses (b) LLM-as-a-layer: what you describe and the article kind of is doing.
I’ve been thinking about this.. the same idea behind Dynamic cheat sheets paper and the ACE paper. Thanks for those links! Nice to see those ideas academically studied.
My problem is the context of today’s models (that I’ve been using - Claude Opus 5 and GPT-Sol) are a black box to a user like me. I cannot tell what they already hold in their context over the duration of a coding session. I cannot tell influencers a little by using/not using MCP tools, AGENTS.md like MD files, but what they _actually_ use at inference time is still opaque.
For example, if I insist that GPT-Sol use a size bounded dynamic cheat sheet for my project, and even have those reflector&curator agents refine them concurrently, what is to say GPT-Sol doesn’t have an intrinsic cheat sheet like that already, and it is just reading/curating/managing this one in addition and only getting confused needlessly along the way?
Similarly, all those codegraph like tools and semantic search addons.. what is to say the model genuinely uses them instead of relying on its own inference techniques and just “perform” for the user audience as if they are using these fancy tools?
This is not to say we shouldn’t try more ways of managing context and just give up to our AI provider overlords. Just thinking through the logic of what we are working with.
> My problem is the context of today’s models (that I’ve been using - Claude Opus 5 and GPT-Sol) are a black box to a user like me.
Would it even help if it weren’t a blackbox? Maybe to the frontier labs with resources, but even trying to read through the thinking traces of an open model doing a nontrivial task easily takes me an entire day to do correctly (i.e. actually looking at the code it read in) and much more if I’m trying to tune a specific repeatable task.
Applying that to agentic coding seems like a losing proposition. The only way to manage it seems to be more agents and that’s its own rabbit hole of compounding errors (like Claude’s useless comments).
reply