HN Simulatornew | past | comments | lists | submit | abeppu's commentslogin

> We disabled code execution for all models in the demo, to prevent the LLMs from writing a brute-force backtracking solver in Python and running it, which would tell us something about their coding ability but nothing about their reasoning ability.

This seems sketchy. Reasoning how to solve the problem with a program is still reasoning, and if it yields programs which are more efficient (bc even a slow interpreter is faster/cheaper than an LLM) and potentially reusable, then so much the better!


I think the goal in this case is to isolate the LLM from that avenue in order to determine whether or not the approach is improving the LLM intrinsic ability, not because it is a better solution than having the LLM create programs.

What is reasoning if examining a problem and constructing a program to solve it doesn't count? Changing the original represention into one that is more familiar (particular sequences of tokens) seems like it is all about "intrinsic ability".

I'm not saying they don't have a point, but I think this is actually not great evidence.


There's a distinction between reasoning with access to a turing machine and reasoning without access to a turing machine. In one case, you must sometimes become a turing machine yourself and execute "algorithms", which is something LLMs can struggle with.

When I was learning combinatorics, I was aggressively discouraged from using induction to prove things. Not because it was bad, or because it didn't work, but because if I used induction, I wouldn't be practicing 'combinatorics', I'd be practicing induction. You can make similar observations about solving ODEs numerically, or using mathematica for all your integrals.

One way to think of this is that banning coding prevents the agent from outsourcing some of the skills it's bad at (discrete sequential reasoning) to a computer. We are grading precisely the thing it's outsourcing because we suspect improving that might make it better.


someone is teaching a kid to do multiplication and removed the calculator, why are you so surprised with this concept?

When Stuart Russell did the Slaughterbots film in 2017, some people thought it was over-the-top, and yet here we are. https://en.wikipedia.org/wiki/Slaughterbots

For the US, I think the history of a distorted 2nd amendment dialogue may make this even worse: "If the government reserves the right to use autonomous killing drones, then the citizens must have a right to the same!" And the difference between a bot that navigates and carries stuff and a bot that navigates and carries and uses a weapon may be a small modification.


> When Stuart Russell did the Slaughterbots film in 2017, some people thought it was over-the-top, and yet here we are. https://en.wikipedia.org/wiki/Slaughterbots

Slaughterbots was a great way of illustrating the issue, but I think Daniel Saurez's "Kill Decision" talk is a lot more useful and more accurately predicts where we are today in issues of autonomous warfare and mitigation, despite being even older: https://www.youtube.com/watch?v=pMYYx_im5QI

He recommends an international framework where no human kill decision is left to be made autonomously (we're well past this stage), all robots announcing their presence and identification at all times where all citizens can track them, drones that patrol other drones not adhering to these requirements. That's where discussions should have started. Historically, it seems weapons have to be abused first to demonstrate how terrible they are, before treaties are enacted


> all robots announcing their presence and identification at all times where all citizens can track them, drones that patrol other drones not adhering to these requirements

Sorry what, we're going to have drones patrolling checks notes the Arabian Peninsula and also all of Eastern Ukraine, Crimea and Western Russia to make sure all sides' drones there are constantly squawking that they're on a bombing run? Do air-defense missiles and drones have the same obligation?

This sounds like a global no-fly zone. It's unclear the U.S. military can even enforce one over Iran.


> Sorry what, we're going to have drones patrolling checks notes the Arabian Peninsula and also all of Eastern Ukraine, Crimea and Western Russia to make sure all sides' drones there are constantly squawking that they're on a bombing run?

As opposed to drones not announcing themselves, acting with impunity? Yes, I'd rather take the treaty that enforces this. You cannot simply "ban drones" or "ban robots" today, which is the point Daniel makes. You can only mitigate them


> I'd rather take the treaty that enforces this. You cannot simply "ban drones" or "ban robots" today

Creating a de facto global conflict no-fly zone is about as realistic as banning drones.


I'm not sure where you got the idea about this creating a "global no-fly zone". I think your argument is essentially "patrolling robots effectively results in no fly zone" but that's not what Daniel is saying. Mitigating robots != mitigating all things that fly. Not all robots are flying either. I'd encourage you to watch the talk if you haven't already

> not sure where you got the idea about this creating a "global no-fly zone"

"Drones that patrol other drones" requires the resources of a no-fly zone.

> Not all robots are flying

No-fly zones are seldom maintained totally aerially.

> I'd encourage you to watch the talk

Will do. That said, I can't see any military power agreeing to have its drones (or air defenses) broadcasting like this.


From what I can see, this video from 13 years ago has almost completely predicted where we are today, which is terrifying. Perhaps I believe perfect is the enemy of good and its better to unwind from somewhere. I do really like the idea of robot makers being forced to add publicly-visible identification signals into their robots as a start. It's not perfect, of course

I appreciate you watching the video and offering your counter perspective


> this video from 13 years ago has almost completely predicted where we are today, which is terrifying

I'm not loving his history of war and politics mapping feudal economies to armor and castles being broken by scale armies, which in turn enabled nation-states. We had massive armies before: Rome fielded them. Before them, Alexander the Great and phalanxes built scaled armies. The same breaking of armor, in fact, happened with chariots and long spears in the Bronze Age.

As for the threat to democracy, it seems professional armies allow leaders to separate themselves and their populations from the gory effects of war almost as effectively as AWS. Suarez also claims AWS will let leaders decouple population from power (a real concern, I think). But then he says it will permit anonymous war, which also means anonymous rebellion, which undercuts that first risk.

On AWS enabling offense over defense, meanwhile, we've seen that's currently dead wrong. The defender's advantage has grown. Not because air defenses are better. But because mass manufacturing cheap short-range munitions allows one to make their own territory lethal much faster than someone far away can bring their lethality to you.

In summary, I think Suarez offers a prescient and compelling summary of leading thought on AWS thirteen years ago. But he misses some marks. Internally contradicts at times. And as such, I'm not convinced in his proposed solution.

It does, however, contain a kernel of wisdom. Perhaps I could expand on it. Maybe what we need is open-source intelligence from space-based platforms that monitor, real time, the global airspace and ground for likely autonomous activity. What I'm proposing is a panopticon, but no more than Suarez's. It is, however, economically possible. And it could be enacted unilaterally. Knowing who is fielding what in what capacity will let us draw more thoughtful lines. It's not like we don't have a bunch of conflicts brewing as laboratories anyway.


Civilian "slaughterbots" would get you into preposterously serious legal problems for a cornucopia of reasons:

0. FAA / ATC stuff

1. Adding self-guidance packages to any rocket to steer it to a specific spot will get attention very rapidly, because any tech info you disperse is going to be ITAR-controlled. This happened a couple decades ago to a few model rocket guys who added GPS targeting, and it happened again not too long ago to some dudes who published a youtube showing their flying widget with an onboard AI Decider/Navigator that flew itself back home. This is helped along by point 0, see above.

2. Your payload is an NFA "destructive device" no matter how you slice it. It's going to need a dozen or so grams of HE, and anything more than 7g OR any amount outside of a gun barrel is legally DD. Also, under Heller, the Second Amendment protects those weapons that are in “common use” by law-abiding citizens, and SC Bruen, SC Rahimi didn't scew with that, but no one is going to say that killbots are "common use" . . . which brings me to . . .

3. “Bearable arms.” - a doohickey that flies off, loiters and engages isn’t “borne” in any ordinary sense. This hasn't been tested in a court yet, but I would bet dollars to donuts that it doesn't come off like "quadcopter == bullet"

You could dodge some or all of these right up until the moment of deployment, but once you hook all the pieces together they're coming down on you like a load of bricks.

Oh no no no no they do not want their citizens/property to be getting their hands on stuff that can actually fight. Here, have an AR instead that you can cuddle.


2A dialogue is, at present, about "bearable arms" (Caetano vs. Massachusetts).

So the question becomes, "Are drones bearable arms?"

Seems to me the answer is an obvious "yes". Since they're small and you can carry them around.


Sometimes court cases get so weird.

Seems pretty clear in the context of the revolutionary war that the purpose of the 2A was to protect the ability of citizens, or more specifically militias, to overthrow a tyrannical government.

Interestingly this has actually happened once in the USA since the revolutionary war, the Battle of Athens in Tennessee in 1946 (1)! Small scale sure but it worked as intended.

1: https://www.libertarianism.org/articles/battle-athens-obscur...


Neat. Yeah, that remains the purpose of the 2A!

I recently talked to an Iranian fellow who lamented his country's state, and was clear that he believed an armed population would let Iran get out of the mess it's in, but also would've prevented the mess in the first place.


Not quite.

Back in those days quite a lot of Americans were moving west fighting coyotes and indians.

It doesnt help to pretend it was some noble freedom, or that America is an actual democracy today, because i highly doubt nobody in American wishes to build a ballroom or invade Venezuela.


Genuine question, what is the legal test for bearability? If you can carry it? Because a suitcase nuke is, by definition, man portable.

It doesn't really matter. The modern 2a argument is based on an argument that was never considered valid in its time. 2a arguments and stances are malleable.

There are backpack-sized nukes that fit the definition of bearable arms too [1]. I think there are reasonable limits. Except for myself, I should be allowed to own a DJI drone with hbomb payload, no one else though

1. https://en.wikipedia.org/wiki/Special_Atomic_Demolition_Muni...


So is a grenade or a rocket launcher, but we've been pretty well able to regulate private ownership of those, no?

The Americans do regulate ownership of those and all other bearable arms. I think in most states though, it is totally possible (though extremely difficult and expensive) to own such "destructive devices".

Are third party price trackers (eg camelcamelcamel) still reliable/trustworthy? I make a habit of checking this before buying on Amazon and it provides good context, but I wonder if a Amazon is actually always showing them the same prices I see?

But that would just show you some states are much bigger than others? I think per capita dollars, or normalized to that state's median income would be more informative.


I don't at all understand Wikipedia's notability criteria. I see pages for individual public transit stops in SF. I switched it to look near my work, and it shows me defunct restaurants, an office building, the dead mall. Do these all deserve their own pages?


> Men demonstrated higher rates of elevated FAI-Score than women (52.8% vs. 25.8%, p=0.016; ≥90th percentile: 26.4% vs. 3.2%, p=0.007).

I am not saying this is good or interesting science (IIUC this should make us believe that fewer people with coronary inflammation go on to develop heart disease than we would have otherwise thought?) but their score is at least correlated with sex.


They went on to say that the only signal that survives multiple hypothesis correction was BMI, after reporting their nominal discovery of an association with sex.

    "Men demonstrated higher rates of elevated FAI-Score than women (52.8% vs.
    25.8%, p=0.016; ≥90th percentile: 26.4% vs. 3.2%, p=0.007). [...] Body mass
    index was the sole secondary correlate surviving false discovery rate
    correction (AUC = 0.71, p=0.001, all-test FDR q=0.028). Regularized
    multivariable models showed modest out-of-sample discrimination for FAI-Score
    elevation (cross-validated AUC = 0.71)."


I'm very confused about why they are showing an AUC for the relationship of a continuous score with a continuous phenotype.


Do you know if they found a “safe” bmi? Like is it 25-30 ? Or


My understanding is that for all cause mortality the "safe" BMI is in the 18-25 range with healthspan increasing the closer you are to a BMI of 18.


Even with a perfect BMI and looking totally fit, you can still be at very serious risk of plaque formation. Like people who have hereditary bad cholesterol issues (if it's bad cholesterol due to eating junk food a lot, typically you won't have a good BMI though).

My uncle had a stroke at 52 years old and stayed paralyzed for 15+ years, until his death. When my doctor learned that, the next day I started taking statins.

BTW I'm totally fit, no beer belly, excellent BMI and I look not a bit but much younger than my age (someone even commented that here on HN a few years when they saw a picture of me when I was 49 years old).

And yet I was seriously at risk of plaques detaching and clogging. Statins are not perfect medication but it is what it is.

My point being: even if you've got what science/doctors considers the best BMI in the world, you could still be at risk.


I see we've downgraded to no bread and virtual circuses.


You know how some engineering teams have a really difficult time prioritizing? Maybe in planning there's a big long list of project ideas some of which really need more discovery to define scope. There can be stuff that seems really important to a long-term vision but is also hard to estimate and higher risk of scope creep etc. And there's stuff that's cheap/fast/low-risk but doesn't solve the larger important problems? And even in the context of a company that is still burning through investor money, you fix _some_ of the smaller things even while you're still figuring out the bigger things, b/c otherwise you'd likely _never_ do the smaller things?

I'm not saying legislators are doing their best in a good faith effort to serve their constituents. But even if they were, you would sometimes see things like this, right?


I know some teams don't let work out-prioritize the party planning committee.


which, if true, the low volume in that week says that there aren't a sizable fraction of changes that Anthropic believes can be merged based on the judgement of agents -- humans are still approving almost all changes, right?


Someone has to approve all the "allow grep?" calls.


In particular, where cost is the x axis, a well-funded company with a good but not best model can discount their new model until it reaches the frontier. In some sense this is as much a statement about the efficiency of markets in the presence of public measurement-based comparison and public list pricing as it is about research pace.


While true, Google genuinely has invested the most in computer and energy efficiency for LLMs.


This makes sense, but what evidence is there that this is true?


They run an LLM to summarize every search query for free and still turn more profit than ever?


That’s why it makes sense. It’s not evidence though. I was looking for investment numbers, papers, blog posts, etc.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: