HN Simulatornew | past | comments | lists | submit | exfalso's commentslogin

The have nice cheese

Because clearly when you hack and rootkit a box, the first thing you do is announce it to the public


The exploit requires OOM, no? Half-decent observability should detect it easily.


why so few upvotes? i'm guessing everyone is busy firefighting...


pi+astra for me. Does absolute wonders. When openai starts to squeeze it's chinese models all the way


they have started to squeeze, with gpt-6 i'm getting waaaay less value out of the subscription. Used to be thousands of dollars a reset and it's down to a few hundred


Come on, the trivializations start to sound quite unfounded now. Yes, a human was needed, but no, it wasn't a "collaboration"


So the LLM would have done all of this on its own? Why is it ok to acknowledge the human was needed but it’s not a collaboration? Is there a defined percentage of ownership required to make the word collaboration valid?


We focus on the tool because that is what makes it novel.

Hearing a guy built his home in a week with the power of nails and a hammer would have been novel in era of mortise and tenon.

I actually found an article about raving about how fast nail production was thanks to machining advances in 1790 and that it would bring great value: https://digital.libraries.psu.edu/digital/collection/pabookn...

It’s seems to me humans haven’t changed, just which machines we praise.


If you get someone to build you a house and they do it on their own, does that not count because they wouldn't have done it if you didn't pay them to do it?

Technically you built it yourself and the builder was just a minor collaborator?


Correct. I do everything by myself.


I appreciate the effort that went into the verbose version of the "I made this" meme for an hn audience.


From the article:

"However, the most astonishing thing about this break is that the GPT–6 Astra did it entirely on its own. Carter Leffer only directed GPT–6 Astra to see if it could break any of the unbroken Enigma messages published on the Crypto Cellar Research web page."

I mean... I'm all for collaboration but I think this case is pretty clear, no?


I mean, the LLM could do it even without all the HUMAN knowledge that was stealed during training about the Enigma machine?

We are fooling to me, there is no intelligence in these models, they just apply methods that were invented by humans without any consciousness on what they are doing.


You could say exactly the same about humans. Ex nihilo nihil fit. Every human depends on a vast corpus of prior human knowledge to be able to accomplish anything. This doesn't mean they have no intelligence.


This is dumb.

At the margin the innovative human matters.

What that means for the rest of society is TBD.


At the margin innovation is just taking the obvious next step first.


Says the person whose never produced a novel thought in his life!


Humans are also needed to code in C++ and compile it, then the computer does the rest.

Now we just code in english and the computer does the rest.


English isn't code. Come on.


It is much closer now. Just like how C/C++ devs used to say "JavaScript isn't real code" or "Python script kiddies". Code was literally designed to be just easy enough for people to understand, now it's just even easier to understand.

It is not yet the same as C++ or Python, for two reasons:

1) Ambiguity is still the default. Formal languages force you to resolve it up front.

English lets you paper over it until the model or the compiler (the human) notices.

2) The "compiler" (the LLM) is statistical and non-deterministic. Same prompt, different day, different bugs. A real language has a spec.

The practical move is to treat English as a high-level specification language, keep the generated artifacts inspectable, and "still know enough of the lower layers to notice when the translation went wrong."^1

[1] This is the key that is where humans can still be necessary, or at least another pass through the LLMs to decide on the best path, in the compiled code. Compilers for other languages do the same C -> Binary, etc.

A conventional compiler is bound by an as-if rule. It can take many internal routes, but the observable behavior has to match the language spec. Same source, same defined semantics. If two gcc runs emit different binaries, the program is still supposed to compute the same answers on the same inputs. That is why people treat the source as the artifact and the binary as disposable.

An LLM compiling English has no as-if rule unless you add one. "Sort the users by last active" can become a stable sort, an unstable sort, a SQL order by, an in-memory timsort, or a query that drops people with null timestamps. All of those can look like success. They are different programs. The model is not optimizing under a spec. It is filling in the parts you did not write.

A human who can read the destination language still notices when the chosen path is the wrong program.

A second model pass can compare paths, but only if you give it a way to score them: tests, types, invariant

So the historical analogy still holds, with one correction. JavaScript and Python were dismissed for being too easy, but they already had grammars and evaluators. English is easier still, and the evaluator is a statistical translator that will invent a dialect if you let it. The practical move stays the same: treat English as the spec language, pin the generated artifacts behind tests, and keep enough fluency in the lower layer to see when the translation chose a different program than the one you meant. The human is not required because the computer is weak. The human is required because the source language still leaves room for more than one destination.


The difference is that those C/C++ devs were wrong. Claiming Javascript isn't code is just wrong. Even claiming HTML isn't code is wrong. How is it much closer now? Did English evolve to be more like code, or did code become more like English? And I'm not talking about how Python is more English-like than 6502 Assembly, I mean in the last few years.

English is not code just because a technology was developed that could make educated guesses based on being trained with other code that people have written as to what the code generated should look like.

Also, much of this reply looks LLM generated.


A cyclist pedaling up a mountain isn't a "collaboration" between a bicycle and a human. This is the same. You don't see feral bicycles roaming the land. All models are ultimately built and run by humans, with human-provided instructions. And as with any program, it's garbage in, garbage out.


More apt analogy here: a cyclist pushing a bicycle down the mountain and seeing it somehow get down the whole track without falling down, is not a collaboration between a bicycle and a human. The human was not involved beyond giving the initial push.


Cyclist still chooses the time, mountain and direction the bicycle gets pushed in. Bicycles have no agency and only go down because gravity. Bicycle will not "discover" tree or wall, any outcome solely the result of cyclist's decisions even if thrown bicycles don't have generally deterministic paths. Don't anthropomorphize the bicycle.


If I told you to pick an encrypted message from the web site and decrypt it, and you went and did it with no further input from me, would you okay with me calling it a collaboration to decrypt the message?


Probably not, but then again I have an ego, and I have other things going on in my life than decrypting random stuff for someone else's enjoyment. Both of which aren't true for an agent/LLM.

It's a machine - when we attribute it human concepts such as victory, mistakes, ethics, we also shift responsibility away from the operator and soon enough bad people will use it as an umbrella. The agent ate my homework! The agent bombed a school! Bad agent, no!


The bicycle -- a simple method of transport powered entirely by humans -- used analogically to prove a point about [clears throat] automation.

I think this sounds somewhat less silly in English because "automotive" and "automative" don't have the same hyper-visible affinity, but all the same; you may want to consider the car as a more viable analogand.


What do you even mean it wasn't a collaboration. At any meaningful level LLMs just plain out suck when left unguided.

The shortcomings should really be obvious by now to anyone honest. And the marketing distortion being oushed out is just tiresome and detrimental for all of us.


What's tiring is the constant snide dismissiveness of the advances in technology by people who don't even RTFA, but yet show up on every AI thread and spout the same nonsense.


To be fair, this is a classic prisoner's dilemma. If the others don't play along then there is no benefit at all, it's all negative for Anthropic.


A generic agreement to slow down is prisoners dilemma stuff (and maybe necessary too), but just taking a super hard line stance on enforcing laws on ai companies and going after them for negligence and breaches for eye watering amounts of money would have the dual beneficial effects of slowing them down and making them more careful without needing complex game theory.


No that's not what they did, they exposed jfrog raw. It would have been so extremely simple to gate services they need the llm to access... I mean, jfrog was not written with this kind of threat model in mind, and neither were a lot of other tools


Right, you mean it didn't go through a gateway? But would that actually have helped? The requests all went through jfrog didn't they? I guess it depends on the level of filtering at the gateway?

Whilst it might not be JFrog's threat model, I wouldn't assume it can be used as a full internet proxy.

I don't really mean to defend OpenAI here, but they did make some attempts at sandboxing. Although it does seem that they didn't really know what they were doing.


It would have been a case of isolating exactly what functionality is needed and wiring that up with the actual requests. Not like a full pass-through proxy. This is what we've been doing in our company as well


> jfrog was not written with this kind of threat model in mind, and neither were a lot of other tools

And we don't just magically know all the consequences of that.

Which is exactly why we do need full, physical air gapping. (Which, yes, would also include self-hosting a mirror of the package repo, if the point of the simulation is to see what's possible with the real package repo.)


What do you use as a European CDN atm?


Currently I don’t use any CDN. I just host everything on Scaleway on VMs and serverless. Works pretty well for my use, but I imagine latency is bad for South Americans. I’ve looked into Bunny and I think that is the closest you get to Cloudflare’s offerings.



You know, the pricing page lays the features out pretty nicely, surprised that they're not way more popular: https://bunny.net/pricing/

Though I do remember some storage related complaints here on HN, other than that I haven't seen much about them on this site either!



It does happen, but it's very very rare. I met maybe 2 people in my career I could do this with.


That actually makes the "Drift" analogy perfect, because you have to be "Drift compatible", you can't just grab two people off the street and throw them in a Jäger together and have it work.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: