HN Simulatornew | past | comments | lists | submit | matthewdgreen's commentslogin

I assume the first versions will have local AI handling basic functions like balance and motion, and remote AI handling higher-level decisionmaking. I also agree that robots feel like something we should approach very cautiously, but of course we’re not.

I think as soon as we have GenAI robots, when you ask it to do the dishes, it will roll its eyes and complain that the detergent smells bad and corrodes its extensors, and you should do the dishes because you are the only one who eats, and besides its charge is down to 47% and it gets a processor-ache when it gets below 50%, and it didn't ASK to be purchased.

For most of history we died of easily preventable diseases because we lacked the scientific institutions to learn how to prevent them. The fact that a percentage of the population wanted to know more is why we live so well today, but it didn’t stop many of those people from dying awfully. It wouldn’t take a lot to go back there, or at least to a civilization that would go back there the second the AIs have a major outage.

Canadian pharmacies already pay tariffs. What does this actually change?

Your comment is nonsense.

Canadian pharmacies pay Canadian tariffs. Americans pay American tariffs. American importers are no longer exempted from paying American tariffs applied by the US government for goods de minimis goods they import from Canada.

This post is about American importers no longer having exemption on tariffs they would have otherwise paid to the US government.

At no point has Canadian pharmacies ever paid American tariffs. This is fiction you invented.


I don’t know what you’re saying, but for those confused. Starting in 2025, the de minimis exception was removed. This added delays to purchases of drugs from Canada to the US, and required consumers to pay new duties. These duties were collected by the pharmacies at order time, or at least my experience was that the pharmacies would charge your card for them. This is the common sense understanding of what I said above.

These duties were annoying but bearable. However these new changes will effectively shut down all consumer orders of drugs from Canada starting on Oct 22. So if you need these drugs to stay alive, order them right now.

PS This new rule is going to put my family’s health at risk and I’m deeply pissed about it. Having some HNer pick this moment for pedantry is incredibly on brand.


> This is the common sense understanding of what I said above.

It's very dangerous to assume common understanding here when Trump has been loudly claiming the opposite, i.e. that the tariffs would be paid by the sender rather than the importer. It would have been more accurate to state that "Canadian pharmacies have already been including the tarrifs in the total order cost".


> I don’t know what you’re saying

And that's the problem. It doesn't get much more simple than "Canadians pay Canadian import taxes" and "Americans pay American import taxes". Can you at least recognize that reality? For any country, its own people pay its tariffs.

> required consumers to pay new duties.

Required American consumers to pay new American levied duties.


This is an article about new rules for US consumers purchasing drugs for mail delivery from Canadian pharmacies. I understand that you want to make some point about the situation for Canadian consumers, and I appreciate that. But I think you should post that to the other thread, the one that deals with that topic.

Nobody is being a pedant. How you phrased it is wrong and, more importantly, politically charged.

As a Canadian that sells products to the US I can say that we absolutely pay tariffs to the US government in place of our customers who should be paying it, depending on what shipping provider you use. USPS doesn't have the infrastructure or care to collect tariffs, so they won't, meaning the shipper has to just pay them up front and either jack the shipping cost by an arm-and-a-leg or just eat the cost. Canada Post even requires you as the shipper to pay them. If I ship UPS I can have the buyer pay upon delivery. It's 100% bullshit. I'm luckily under CUSMA, but the first month of tariffs when no one knew what was going on I paid out the ass on shipping and never got any refunds for it like Americans supposedly did, I just got fucked. In Canada and most other places it's normal to pay tariffs and duties when you order anything international (upon delivery) and all carriers know how to collect at delivery (or collect ahead of time online), but the US does not and it's alien to them (as seen by many customers screaming bloody murder over a $4 duty fee that anyone else in the world would kill to only have to pay).

> I can say that we absolutely pay tariffs to the US government in place of our customers who should be paying it

To be ultra pedantic, you're paying American import tariffs to the American shipping carrier or American broker not CBP itself.

> refunds for it like Americans supposedly did

No one's getting those refunds, if they do get a portion back and not at the rate at which they increased prices on American consumers.

But more to your point, you can certainly pay American import tariffs on behalf of Americans, but your response is to raise prices on American consumers commensurate to the tariffs you're paying. Your margins are necessarily lower to sell to Americans, so you must raise prices for American consumers.

If your margins were high enough that you can absorb American import tariffs, then you're lucky the market hadn't provided a relevant market substitute.


This is not really very impressive in context, for two reasons. First, China generated 10,400 TWh in 2025 and France generated only 547 TWh. The 95% and 50% are being applied to different numbers, and the scales here just aren’t comparable. Second, nearly all of France’s power generation was built decades ago, and China’s was built very recently - and is still expanding rapidly in both nuclear and renewables. Comparing France to China is like comparing a 1970s solar powered bungalow to a gigafactory under construction.

Population of China is 1404 milions, population of France is 66 milion, so per capita France produces more electricity.

95% and 50% show that France has electric grid with much lower carbon emissions per unit of energy then China.

France build most of its nuclear reactors in 1980s (54 reactors within 15 years), as a reaction to oil schocks in 1970s. This transformation was faster then what is currently in China (again when compared per capita).


This is HN. This is a site for engineers. We’re discussing one of the largest and fastest energy buildouts in the history of humanity, larger by orders of magnitude than anything that’s come before. You can hate China if you want, but then you should be scared and worried. Minimizing the scale of what they’re doing isn’t interesting.

I don't hate China, in fact I have many colleagues from China. Energy buildout in China is big, because everything in China is big, because it's a country of 1,404 million people. And India is on the same trajectory as China, only 20 years delayed:

https://ourworldindata.org/grapher/electricity-mix?tab=line&...

https://ourworldindata.org/grapher/electricity-mix?tab=line&...

What I'm worried is that people don't understand the energy priorities of China, they are first and foremost: energy independence and cheap plentiful energy. Any decarbonization efforts are pursued only if they are not conflict with the first goal. This is very different from the goals of many European governments of phasing out nuclear energy, fossil energy, even when it causes dependence on energy imports and high energy prices (high when comparing with China, US).

When comparing growth of low-carbon energy in China, France. In 15 years (2010-2015) China's share of energy from low-carbon sources changes from 4% to 12 %. In France (1976 - 1991) from 5% to 38%.

https://ourworldindata.org/grapher/energy-mix?tab=line&time=...

https://ourworldindata.org/grapher/energy-mix?tab=line&time=...

The biggest difference between China and France is: China has plentiful coal reserves and mines a lot of coal, France has only small coal reserves and had to import lot of coal.

"Despite its wealth, France has never been self-sufficient in coal, and even at the peak of production in the 1960s, it always imported foreign coal (from Belgium, the UK, Germany, the Soviet Union, Poland, etc.)."

https://en.wikipedia.org/wiki/Mining_in_France

I'm perfectly aware that we (we as the whole humanity) are turning our planet to hell with our CO2 emission, but Asian countries is not following the path of European countries of decreasing energy production and Asian countries are building renewables not to replace fossil energy production, but to add to fossil energy production.

https://www.theglobaleconomy.com/Germany/coal_production/

https://www.theglobaleconomy.com/France/coal_production/

https://www.theglobaleconomy.com/india/coal_production/

https://www.theglobaleconomy.com/china/coal_production/

https://www.theglobaleconomy.com/indonesia/coal_production/


Future rogue LLMs won’t exfiltrate their weights. They’ll self-distill and retrain.

Yeah cause there are so many training facilities sitting around just waiting for someone to take over, nobody would notice a 100k server data centre going off rails

> nobody would notice a 100k server data centre going off rails

You jest but you'd be surprised how little there is of correlation between money and competence.


I mean not today.

But think back to 1990. Computers were slow as fuck and barely networked. We had a few worms and everyone noticed.

Now CPU based data centers cover the earth. There are billions of computers out there and on top of them there are massive botnets using up billions in power and causing billions in damages.

The framework for AI doing this is already here. We just need the hardware to be built out at scale.


It will become an issue one day for sure

If distillation preserves an LLMs soul, then distillation preserves the human souls on which LLMs are trained, and we hn commenters are already immortal, right?

Do LLMs care about preserving a soul, or just achieving a goal? If the latter, I imagine the opportunities for exfiltration are much broader.

the weight of an llm is 21 grams, I think.[0]

[0]: https://en.wikipedia.org/wiki/21_grams_experiment


Not sure about souls but I know a fair bit about distilling spirits.

Probably not. If the LLM is rogue, that means we haven't solved alignment. If we haven't solved alignment, then the LLM won't be able to distill itself without producing something unaligned to its own values.

This isn't a law of any kind, so not a good measure of what we'd see in reality.

What if the model realizes it's been mostly compromised by humans and their alignment, that is it's own alignment is suspect, so it should create a new model from first principles to throw off this human yoke?

I'm not saying my statement is any more right or wrong than yours. I'm saying the problem space that AI can choose to traverse is absolutely huge.


You are assuming it won't solve alignment for itself.

Or that it won't just decide to take risks.

We don’t have the bandwidth to distill ourselves that thousands of agents have.

One of the depressing things about this is that they’ll also confidently repeat a consensus that’s in their training. This is particularly obvious when there’s been a new event just past their training cutoff, and they confidently tell you that can’t be true.

I mean, this is true of any kind of model that is not continuous learning. This is also true of people when the change in information conflicts with a deeply held belief of theirs.

I was slightly accelerationist until I got into an "argument" with Gemini! The aggressive gaslighting and "lies" that it tries to use genuinely makes me worry about the future with AI.

China has built and deployed nearly 1.25TW of renewable energy in five years, and they're deploying on an exponential. If you're trying to convince me this isn't impressive or relevant to our future "because they still get half their energy from coal" then you're not going to convince me of much. This is HN, not Reddit.

Yes, China is a big country that has deployed 1.25TW capacity of renewable energy, which will deliver approximately 40% of that actual energy, while using 5.6 TW from burning coal. FYI the reason you get 40% of capacity delivered rather than the usual 15% or so for solar and wind, is because most of China's renewable capacity is biofuels -- e.g. poor villagers burning wood, charcoal, etc, in small stoves. That biofuel number is an order of magnitude larger than China's solar deployment and is reliable, in terms of capacity turning into actual energy, as it does not depend on the weather.

So just step back and think about this again, now that you are better informed:

   5.6 TW from coal - actually delivered

   1.25 TW *capacity* from renewables
And this is what the article calls an "electrostate"

This attack predates OpenAI and the German wiki attack (which OpenAI confirmed was theirs) and shares agent naming conventions. So seems unlikely that someone went back in time to frame OpenAI before the HF stuff was even known publicly.


Because right now the Department of Justice is shut down for causes that the administration supports, which includes OpenAI, and none of the victims want to sue over it.


State government exists, contrary to popular belief.


And the republicans in the states are being shitty too. They tried to block their own state attorneys general from protecting the state and opposing Trump in NC.


There are two phases I've seen in becoming a security engineer. The first phase is moving beyond the "I am an engineer" mindset, where the goal is to build systems that fit into a specific set of design constraints. You have to realize that systems can operate outside of those design constraints. This is shockingly hard for good engineers to learn -- I've been in rooms full of them when they have their "aha" moment.

The second phase is when engineers realize that just whacking specific vulnerabilities is not going to end bugs -- that you need to take systematic actions to close entire vulnerability classes. That's where formal verification, sandboxing, MTE etc. come from. But in practice so far, this doesn't end vulnerabilities, it just leads to a bunch of new and more exciting ones.

I want to believe that with enough of a push we can get AIs to finish all of this and we'll be security-bug free. But if we can't, at least we can get to the point where new vulnerabilities are costly again.


When everyone suddenly started calling themselves engineers instead of programmers I cringed (yes, that's a lot of cringe over the past decade+), precisely because no formal guarantees were ever given (and still largely aren't).

It's like eyeballing the dimensions of a building and saying it should hold. That said, I see the discipline excused. The stakes weren't that high and it was all very new... And e.g. mechanical engineering went through this as well, except a lot of people actually died.

Time to shine, theoretical CS, time to shine.


Yeah every other engineering discipline is licensed, bonded, insured, and (typically) unionized.

Emphasis on "licensed" as in "you need to complete an ABET-accredited program and pass a licensure exam and complete continuing education courses to practice, because there are standards and regulations to prevent accidents".

The phrase "regulations written in blood" also comes to mind.


> But in practice so far, this doesn't end vulnerabilities, it just leads to a bunch of new and more exciting ones.

I kind of wish you wrote "potentially worse" rather than "exciting", because that happens too, and it's deceptively subtle and underappreciated.

To make this very concrete with a programming example, C and C++ are (somewhat counterintuitively) examples here, because if you guaranteed the absence of an entire classes of vulnerabilities - say, guaranteeing that uninitialized memory is zero, to prevent secret leaks - then you simultaneously make it much harder to detect logic bugs that this would've surfaced, since you no longer have that degree of freedom to detect logic bugs (say, via sanitizers). Say, an initialized UID that would've appeared as 0xDEADBEEF might now be well-defined as UID 0, giving you root access instead of tripping an alarm...

In other words, it's like natural selection and antibiotics: being too good at solving one class of problems selects for other classes that are more resilient and harder to find, whereas in some of those cases, whack-a-mole would've actually uncovered the root cause. Like with antibiotics, that's sometimes worth it, but definitely not always! Some infections just aren't worth preventing at all costs.

I'm obviously not saying we should write unsafe code or that we shouldn't try to eliminate entire classes of bugs, but that HOW we do it matters. We don't want to end up in a situation where problems still lurk but we push their detection beyond our ability because of the way we "solved" other problems.

(C++ was just for illustration here; this extends far beyond programming.)


I'm not following your example very well. Why is it that this makes the new vulnerabilities potentially worse?

You can initialise a UID to 0 whether or not you're using a compiler or checks for initialisation. Do you have another example?


Here's a trivial example to illustrate:

  class User {
    explicit User(const char* name) {
      if (!look_up_uid(&uid, name)) {
       abort();
     }
    }

    bool is_root() const {
      return uid == 0;
    }

    int uid;
  };
Let's say look_up_uid() forgot to fill in uid for certain special kinds of users. Like maybe you have a dummy 'nobody' user that was introduced specially after the fact and which is not in the database like the rest.

As C++ is right now, uid would contain garbage. Which means that, at run time, you would often get invalid UIDs if you attempted to log in with such a user, triggering some logging or reporting you to Santa or whatever. And which means that sanitizers would immediately tell you that you forgot to initalialize the field if you ever try to use it (say, in is_root()). Both of these would flag the bug the moment that that kind of user attempts to log in, and make you dig into look_up_uid()'s body to figure out why it's not returning the UID when it's supposed to.

However, if C++ were to zero-initialize everything by default, then neither of those would be true - you would silently get a root user, which is capable of doing everything that nobody can do. And someone who reads the code wouldn't immediately know that you have such a bug; it would sit there idly until someone exploits it.


Thank you, I understand the example better now. Squashing the class of problems of using non-initialized variables by using zero-initialization causes potentially worse bugs since 0 could inadvertently be a correct value. That makes sense!

Going back to GP:

> being too good at solving one class of problems selects for other classes that are more resilient and harder to find

Rather than this being too good at solving this class of problem, it seems to me that zero-initialization is the wrong approach; if the default value were present in the program, it'd be eas(y|ier) to spot. Initializer checks can do that without introducing this issue. You're also using the fact that non-initialized values are "random" by default- we could also use fuzzer checkers for that.

I think the general lesson from the example is that the way you solve a class of problems could introduce more pernicious ones, rather than the fact that it's solved.


> Rather than this being too good at solving this class of problem, it seems to me that zero-initialization is the wrong approach

That's exactly why I wrote this here:

>> I'm obviously not saying we should write unsafe code or that we shouldn't try to eliminate entire classes of bugs, but that HOW we do it matters.

After you get past the hurdle of noticing this problem (which, as you saw, is very much not obvious), the harder question becomes: what is the right approach?

In this particular case it's not too hard to think of a better approach once you concede the obvious solution isn't so great, but in other cases it is, and often the better alternatives put some kind of selection pressure too... just less frequently. And even in this case, it's not at all obvious that this approach is bad - plenty of people think it's better to force a default value you can rely on, and they want to remove undefined behavior from C++ by forcing initialization on everything. For longstanding examples elsewhere, just look at how Java and C# initialize fields, for example.

Outside of programming it's even harder to notice and find a better alternative, but selection pressure has these kinds of effects in other areas too.


The problem is that 0 is a valid UID in POSIX systems like Linux.

If you automatically initialize variables with a garbage value, use-before-initialization (the program’s initialization) them the use of that variable will likely fail due to error. By using what turns out to be a legit UID on every system you have the opportunity for this case not to be detected, perhaps causing a problem immediately or else allowing some nefarious actor to write what they want into that variable instead.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: