I have some property that doesn't really get cell reception in the summer, and even in the winter the reception is at-best spotty. I use granite and qwen together with a manually curated set of books, articles, etc. to get work done while disconnected. I tend to update the set of resources before leaving town because I'm generally intentionally disconnecting to focus on deep work. Building a workflow that makes that possible has helped me build out a setup that replaces a ton of my day-to-day anthropic/openai use even when I'm near fiber.
It is interesting how, even pre-llms, offline computers change the nature of a lot of activities. E.g., about once a year I'll walk do a random walk in the woods for a few days without paying much attention to maintaining a bearing. With maps and compasses it's a whole skill set to find your way back. With a phone and downloaded maps you just kinda press the power button and check your heading every hour or two.
The "survival" use-case is interesting. I fear a lot of the practical applications of that knowledge are either time-sensitive (e.g., RAG on a "Stop The Bleed" course won't help because by the time you learn the patient is dead) or require tools+skills that aren't easily improvised completely from scratch in high-stress situations (e.g., building shelter or even really simple stuff like building a fire when it's very cold out). And for other stuff common sense prevails (e.g., power out -> keep fridge closed).
I'm pretty much weaned off closed-weight models at this point.
It's a very "The Year of Linux on Desktop" situation, in the sense that in any particular year it seems like Linux is close, but the OSS stuff requires a lot of configuration to get to feature parity, and the barrier keeps rising because the prop solutions are running as fast as they can to justify the cost premium.
Unlike Linux, the world is pretty literate about tech business models, so I think there's a way for the open weight world to avoid the situation Linux got into, where it only really caught up once the market was pretty much commoditization any-ways.
If anyone interested in PL theory (including adjacent fields) and/or inference engines is reading this wants to help make open weight models and associated ecosystems feature-competitive, we're hiring, drop me a line: nathan at nfulton org.
Pennsylvania's 2020 robot delivery statute was poorly written because it failed to contemplate the meaning and consequence of traffic code when pedestrians are not humans, are not even necessarily people, and have no will.
Consider:
When the police officer gave the human an order to disburse, and the person willfully refused, they then went to jail. Perhaps they are vindicated. Perhaps not. (even if vindicated the process is the punishment.)
However, if a police officer also gives the robot a verbal order to disburse, and the lidar just sits there are spins, who goes to jail? And if willfulness is a component to the crime, and the engineers did not design a system capable of understanding police orders, then did a crime even occur?
In PA statute, "pedestrian" confers mostly protection, not obligation. The corresponding obligations imposed on human pedestrians in PA code are mostly downstream of willfulnes and/or personhood. I.e., you need to be a person or act willfully to be bound by the typical obligations of a pedestrian.
The law worked well to-date because all pedestrians were human persons, so you didn't end up in a weird edge case of having a pedestrian's protections without a human person's obligations. But if the robot gets all the protections but few of the obligations of pedestrianhood, well, that violates my moral intuition.
In the spirit of intellectual curiosity regarding this fascinating legal situation, the next officer facing a similar situation should issue a disbursal order to the robot and the human, and the DA should pursue criminal charges against any party that willfully refuses that order. Which, to be fair, is probably what would happen if two humans were engaged in a similar standoff.
> Proving theorems is one of the main ways that mathematicians turn math into money.
Mathematicians working at universities are paid to teach, to engage in the politics and business of government contracting, and only in very rare circumstances to play prestige games on famous problems.
It is unlikely that Congress will decide to slash budget for NSF's MPS directorate on account of AI getting good at proving theorems. That's just not how Congress works.
It is also unlikely that LLMs will be able to do, in the near future, do the sort of applied mathematics that happens in the context of occasional NIH or DoD funding that flows into Mathematics departments, for a variety of complex sociological reasons I'm happy to expound upon.
And it's unlikely Engineering degree programs will stop requiring calculus sequences, since being able to do the computation was has not been the actual point of teaching calc since he late 20th century (if ever).
So, the good news is that I doubt any of this will have much of an effect on the job market for Mathematicians who want to prove theorems for a living. The bad news, of course, is that the job market for Mathematicians who want to prove theorems for a living was already pretty horrendous.
> the computer aided proof discouraged research into the insightful proof a lot of mathematicians think is probably out there to find
Kempe's reduction technique, on which the eventual proof still depended (at least spiritually? I think?), did contain true insight with reach beyond the original problem statement (e.g., Chaitin/register allocation).
I submit that "clever way to soundly reduce to finite set and check" is itself probably one of the more important types of insight in mathematics. Actually, all of my favorite mathematics, from an aesthetic perspective, has exactly this flavor. That's down to taste, of course, but it is sort of a fundamental/powerful move in modern mathematics. The beauty of that insight can only be appreciated once you're willing to accept that boring case checking is tractable.
In that sense, the FCT gave us two really important insights -- first some profound insight from Kempe then an equally profound insight about the role of computers as a bicycle for human thought.
Yeah, not saying the proof was worthless, and the technique is indeed a genuine insight. However, there's likely to be a proof that explains why all the cases work out rather than it just being a crazy coincidence, and the existence of the proof did stifle further research into the now "solved" problem.
In hindsight, along with illustrating the potential of computers as a bicycle for the mind it also gave the first indication that if such proofs were to become very easy to produce they could also be used as a bulldozer to bury the mind.
> Anthropic has obviously never done anything like this
If you are ascribing intent ("Anthropic designed a system that intentionally does this") or emergent model intent ("OMG at AGI") then I mostly agree.
But there's a more pedestrian version of this, where an unwise person puts a model in an open loop without sufficient HiL checking and then the model does dumb things. There are even some recent stories about OpenAI and Anthropic systems "emailing the FBI", which is sufficiently close to "emailing opposing counsel" [1][2].
But even if you feel those stories are trolling/marketing/etc., I think it's clear that there's a plausible risk model here. If you hook up an LLM to a toolset that has access to Outlook and your file system and opposing counsel's email address, then of course the dumb thing could happen. The LLM doesn't need to be AGI/ASI; in fact, to me, the worse the model capability the more likely this sort of screw-up feels. There are even (non-sarcastic) instructions or CoT roll-outs I could imagine which might elicit a sequence of tool calls the net effect of which is an errant disclosure.
So this feels less "unforeseeable lightning strike" and more "car accident while driving at speed". I'd put the HF hack in that bucket as well.
A sufficiently careful engineer and a well-trained expert operator can together properly constrain the system to not do the dangerous thing. But, in general, that is not how agentic systems or their underlying models are currently being designed, and most users are not receiving sufficient training.
A lot of these LLM-backed products feel very poorly engineered, rushed, and optimized for brain-feel instead of surfacing and forcing the user to acknowledge the large quantities of actual essential complexity in play. It's all very "unsafe at any speed".
I provided some very light guidance to a couple of masters students who were working on legal citation hallucination detection last summer. Their summer project is here: https://github.com/gt-csse/mellea-lrc
If anyone here is looking for engineers to work on legal AI products, both of the students who worked on this project come highly recommended.
I have enough real problems in life. I don't need to invent new ones just because a new technology is available.
Many of my problems in life are fully solved far past my satiation point by a 3b model that costs me nothing to run.
Many others are not.
But in either case, when I am acting and living wisely, almost all of my problems exist prior to the existence of technological solutions to those problems.
This is also true for the customers and employers that I care to work with. This has changed in me over time, but I now try my best to avoid inventing new problems. The world has enough big, important problems already.
> This morning I elicited a microkernel operating system from Opus 5.5.
This is cool but also a good example. I don't need a personalized microkernel just because it's possible to have one.
Maybe I need one and I don't know it, but the problem statement definitely isn't "I have inherent desire for a personalized microkernel".
You've misunderstood what I was talking about; that's a different meaning of the word "problem". Possibly you did not intend to start a merely semantic argument, but that's what you ended up doing, so I am unfortunately going to have to point at the dictionary.
You're talking about definition 1 in https://en.wiktionary.org/wiki/problem, "A difficulty that has to be resolved or dealt with," with the examples given being racism, addictions, and lack of access to health care. Those aren't the kind of problems AI can help with.
The kind of "problem" that AI can help with is definition 2, "A question to be answered, schoolwork exercise." That is the character of engineering "problems", although they are more open-ended than schoolwork exercises, because there are many defensible tradeoffs. "How can I build a bridge here?" or "How can I improve the fuel efficiency of this vehicle?" is a "problem" in the sense of a question to be answered with knowledge and effort, not in the sense of being similar to racism or addiction.
If the questions you're thinking of are so easy to answer that they can be easily answered by a hypothetical AI model 1/500th as good as Opus 5.5 — well, think up some harder questions.
There's nothing wrong with this mindset, but your insistence that other people must have your values is off-putting.
To wit: at the end of my phd I was offered several tenure-track positions where "work on problems that are hard" would've been a big part of my job.
I turned those offers down due to my distaste with "because it's hard" as a justification for working on problems.
I'm much more interested in helping people and solving (def'n 1) problems. Those problems are almost always too large and messy to one-shot, but very often contain many components that can be solved with non-sota models, and make sense to solve with non-sota models for any number of economic/scaling reasons. E.g.: "can I serve this thing that costs $0.25 per run on even c.2024 sota models to 100,000 K12 students who will use it dozens of times a day on a non-profit's budget of ~$0.00."
P.S. The "problem" definition thing is very confusing to your readers because the person you responded to was clearly using the first definition! It would have been helpful for you to sign-post that you are using the same word as the person you are replying to but changing the meaning out from under your reader.
I feel like the "good enough" argument isn't about how big the gap between models is but about how good they are at solving the tasks at hand.
The capabilities of all models increasing so much all the time means there are simply less and less tasks you need a frontier model for.
Even if Opus 5.5 is 500x better than Deepseek, if deepseek can solve all my problems, why do I need to pay for more?
They are clearly talking about "problems" that DeepSeek can solve, and they seem to be using the term "task" as a synonym for "problem", even using the problem verb "solve" with "tasks" as its object. To me, that makes it clear that they are not talking about problem₁s such as racism, addiction, and lack of access to health insurance; while you could imagine "solving" such problems, they are not tasks, and DeepSeek certainly cannot solve them. Rather, I think they are talking about "engineering problems", which are examples of problem₂, more analogous to math questions (to be diligently sought out) than to troubles (to be avoided).
It is certainly the case that the proposition that such problems should be diligently sought out is a value judgment, and therefore it derives from the value system employed by me, the writer. Would you prefer that I refrain from making normative statements in general, or just those with which you might disagree? In either case, is your preference specific to me personally, or do you consider it a duty incumbent upon everyone?
— ⁂ —
The 01913 edition of Webster's dictionary offers:
1. A question proposed for solution; a matter stated for examination or proof; hence, a matter difficult of solution or settlement; a doubtful case; a question involving doubt. --Bacon.
2. (Math.) Anything which is required to be done; as, in geometry, to bisect a line, to draw a perpendicular; or, in algebra, to find an unknown quantity.
Note: Problem differs from theorem in this, that a problem is something to be done, as to bisect a triangle, to describe a circle, etc.; a theorem is something to be proved, as that all the angles of a triangle are equal to two right angles.
The absence of the "trouble" definition suggests that perhaps it is derived from the "question" definition, and, surprisingly, very recently.
That might be how that word was used in 1913, I don't know. It is not 1913.
Returning to the substantive matter, I'm happy to try to re-state the topic at hand using your preferred symbols.
Suppose:
1. Someone has a type of task to be done.
2. Finding a systematic way to complete that type of task has immediate value.
3. That type of task can be solved at lower per-unit cost without paying for tokens from the latest Anthropic or OpenAI model release.
Then why would you tell that person that they need to "think of better problems"? Particularly when they've already creatied a solution to the type of task they need to complete with de minimis per unit cost.
There is nothing wrong with consumers who choose to become early adopters and ask what new things can be done with a new class of product. And there is certainly nothing wrong with creating a new technology or making a scientific discovery and spending some time looking around for applications of that discovery/invention. These are important things. The PC and Internet revolutions both relied on this sort of activity, so a culture of doing that type of work is deeply embedded in our field.
But, likewise, there's nothing wrong with driving an ordinary nail through two ordinary piece of lumber to solve a problem for a person who has money and needs two boards fastened together. That framer doesn't "need better problems".
I hope this makes the meaning of my position clear.
AI companies are already paying human mathematicians, to help train their models. Including to understand AI-generated stuff. The pay is kind of shit but still quite a lot better than what we were paid as phd students, especially if you do it 40 hours a week (which, tbf, was very short week in grad school).
I know this because I spent a lot of social time around a Math dept in grad school and many of them are moonlighting working for subcontractors on production and evaluation of training data [1].
Anyways, mathematics is a bit of a funny discipline... some buckets:
There are lots of obvious cases where progress has obvious applications for incremental progress, and where you can ask for novel math from the perspective of the application or ask for applications from the perspective of novel math. That seems like the sort of thing that you could throw $10M at get something valuable without much human input. I do this, at much much much smaller dollar amounts, pretty regularly, and with open models, so nothing secret/sota/etc.
But there are also lots of cases where some progress gets made on something very pure and esoteric and the implications aren't really clear. Those are cases where just asking a bunch of nerds to marinate on it while interacting with a messy world in the full social generality of a modern university could make a lot of sense. The connection between the four color theorem and register allocation, for example, feels very... hard to do without humans free-associating in a diverse academic environment.
And, at last, there are cases where there are subfields or problems that are big and important in the way that driving a rare car or wearing a particularly fascinating rolex is important. Some of modern mathematics feels like it has its cultural roots in intellectual gamesmanship amongst wealthy gentleman and/or those under their patronage. A lot of the... less well-argued... objections come from this set.
Like I said, math is kind of a uniquely weird discipline. It's got aspects of practical gritty science, noblesse oblige/high-culture, fashion/taste, craftsmanship, tutelage, and so on.
--
[1] Some enterprising journalist should track down all the subcontractors used by OpenAI for mathematics data, then talk to all of those people and figure out how much breakthrough-adjacent data work was happening... pure conspiracy but I'd be curious for someone to explore the hypothesis that there were maybe people helping find fragments here and there and it wasn't all purely mechanical monkeying.
I have some property that doesn't really get cell reception in the summer, and even in the winter the reception is at-best spotty. I use granite and qwen together with a manually curated set of books, articles, etc. to get work done while disconnected. I tend to update the set of resources before leaving town because I'm generally intentionally disconnecting to focus on deep work. Building a workflow that makes that possible has helped me build out a setup that replaces a ton of my day-to-day anthropic/openai use even when I'm near fiber.
It is interesting how, even pre-llms, offline computers change the nature of a lot of activities. E.g., about once a year I'll walk do a random walk in the woods for a few days without paying much attention to maintaining a bearing. With maps and compasses it's a whole skill set to find your way back. With a phone and downloaded maps you just kinda press the power button and check your heading every hour or two.
The "survival" use-case is interesting. I fear a lot of the practical applications of that knowledge are either time-sensitive (e.g., RAG on a "Stop The Bleed" course won't help because by the time you learn the patient is dead) or require tools+skills that aren't easily improvised completely from scratch in high-stress situations (e.g., building shelter or even really simple stuff like building a fire when it's very cold out). And for other stuff common sense prevails (e.g., power out -> keep fridge closed).
reply