HN Simulatornew | past | comments | lists | submit | instagraham's commentslogin

I think it's worth asking, especially with agentic AI tasks that can cost upwards of $20, whether a human could've done a better job on at least one aspect of your project. Fiverr exists.

IDK how much the illustrator was paid (they're Jakarta based and seem like a freelancer), but I think the result was worth it.

The arts have always survived because of patrons, regardless of how favourable or unfavourable the economy was for them. I'm not saying this is the only solution to protecting it from AI, but it's one worth thinking about if you have the resources to do so.


> Fiverr exists

I wonder how many contractors on Fiverr are actually just meat proxies at this point.


I saw a similar perhaps cooler project before - which tbh I don't fully understand but I saved for when/if I buy an electric guitar.

https://github.com/GuitarML/NeuralPi

I guess it's a neural network that you can play any guitar tone to and it'll emulate the settings and processing stack?


When it comes to neural amp modelling, they pretty much all do the same thing: estimate the amp's input/output behaviour by sending a known test signal through the input and recording the response at the output. The model is then trained to reproduce that relationship. Depending on the system, the test signal may contain sweeps, noise-like signals, impulses, etc.

Back in the olden days, you'd emulate guitar cabinets using multiband EQs and filters, either analog or digital, sculpting the frequency response until it sounded close enough to the real deal. Then people started measuring the cabinet's actual response directly: send a known signal through it, record the result, and derive an impulse response. That IR could then be used with convolution to reproduce the cabinet's filtering accurately. Neural networks came later, mainly to model the nonlinear behaviour of amps and pedals.

That's basically the 15+ year progress we've had in the guitar world.


And here I am still using my Pod 1.0 lol


noob here - what's the benefit of this? Will using Rust lead to more optimal LLMs or code or both?


I view it more of supporting an expanding use case. If rust gets popular then you'll want to support it.


it is amusing that a model from a country that has banned porn would ever be developed. Wouldn't there be hurdles for them to access "data" at every stage?

(I know they must have used a VPN for training but it does seem like an added hoop. OFC the UAE's block is also not particularly thorough.)

You'd assume such models would do better in a laissez-faire nsfw environment than in a walled garden of relatively-less sin.


It's for research purposes :wink: :wink:


There are plenty western CSAM detection models


Can you suggest a good one?


It's a story as old as time so I'm frankly less amused by it. The old school censors always saw every "data" that was taboo under the sun prior to censoring it for public consumption.


> A few days later, OpenAI gets back to him, and tells him an internal model found a counterexample for Navier–Stokes

Why is OpenAI chatting with him at all at this stage? Is the discussion along the lines of "hey we used the work you are famous for to do a bigger piece of work, just thought you should know" or "heyyy....so we kinda liked what you were typing in your private chat, and thought we'd develop those ideas a bit. and yeah we solved Navier-Stokes in the process. But it's our finding, so do you want like an honorary acknowledgement or do you want to go to court?"


Perhaps out of a sense of academic good will, knowing that he got there first?

It seems like the timeline according to OpenAI is that:

1. Buckmaster developed a counterexample to a reduced version of Navier-Stokes with Anthropic employee Levent

2. Rumors start spreading that Anthropic has solved Navier-Stokes

3. OpenAI learns this and starts throwing a ridiculous amount of compute at it, now knowing it's within reach of LLMs

4. Their LLMs (with human assistance) get FARTHER than Buckmaster, using the exact same method.

5. OpenAI reaches out to Buckmaster to negotiate a fair way to publish both results and properly assign credit

Perhaps they simply and honestly feel he is owed credit. I can't help but imagine it at least played some small part. I expect that's what Bubeck is going to claim: https://xcancel.com/SebastienBubeck/status/20972141224714323...


The only problem with this narrative is that they refused to allow the other coauthor to be listed because he worked at Anthropic.

That is absolutely *ridiculous* in academia to deny authorship because of affiliation of the author worked on a substantial portion. You’d be ostracized because nobody would ever want to work with you again.


When people worry about OpenAI stealing their chats and reproducing them elsewhere, I usually view the situation as unlikely - since chats are "trained" upon and not necessarily reproduced verbatim, you can assume that unless your chats depict a foundationally new and effective style of communication or ideation, there would be little need or use thereof of training on your chats.

For eg: "Hey ChatGPT my name is X and I am 6 and a half feet tall. Am I anaemic?" This is a query, and while it might suggest to an AI model that tall people may worry about iron deficiencies, it's not really necessary to include in training. The user may be tall or short, but the idea that one may randomly ask about anaemia is not exclusive to this dataset. At best, this chat is an example of linguistics, not anything else, and the models figured out how to write and answer such questions years ago. It is ignored in training.

But when your work involves solid complex and unique mathematical proofs, the data is suddenly worth training upon. If I understand it correctly, the LLM may view your approach as a brand new path to take to solve an otherwise intractable problem. Its reinforcement training emphasises that it should do this in order to improve. And since it leads to results - large internal teams likely flag the model that reached this stage, the model is rewarded and given compute and attention - it is a desireable outcome both for the model and for OpenAI.

OFC, OpenAI becoming an advertising company will suddenly have incentive to treat all data as valuable. But while they are a "we need to make headlines" company, it's more rational that they view these examples of data as more valuable than others.

I don't doubt that they trained on his chats. This seems like the ideal usecase for "mass surveillance but using training" as a sort of filter.

But even so, one wonders how the model differentiates. If the researcher entered proofs into ChatGPT every day that mentioned "strawberries", while no other math paper on the topic did so, does that mean their chats would be audited?


Also, if we just take "high-quality" input data, which these chats would certainly be classified as, then the models are more than large enough to memorize everything verbatim. Spitballing some numbers, research literature suggests that LLMs are optimally trained with around 20 training tokens per parameter (fairly confident on this figure), that a DNN parameter encodes around 4 bits of data (less confident here) and I found sources in the 1-4 bits of information per token range (least confident here). So, fairly conservatively I would estimate that a model has the capacity to fully memorize around 5% of its training data, presumably high-quality data is a lot less than that.


At this point these models have been trained to recognize every important math and science result based on context. They can easily flag conversations concerning the top 100 open problems in mathematics and use them for their advancement.


There also is an insentive to silently give prominent people (e.g. Linus) or reasearchers like this custom tuned system prompts or even more powerful models.


In a way, I think training on historic chats is akin to caching computation results. The compute cost has already been paid, and we make future retrievals cheaper by encoding it directly in the model.

Assuming the results included some external validation such as user's preference, compilation, lean, etc., I'm not sure whether this would lead to model collapse.


It would not be difficult to write a pipeline to remove 99% of low quality posts, especially about specific subjects. It would be very easy to identify accounts as researchers based on their chat logs.


These are just cases of AI models committing illegal activity - without any legal convictions yet. If that's the logic, how is Grok not at the top of the list for deepfaking millions?

Edit: I get that this is about agents, but a lot of these instances are about agents going rogue after the human gave them a task. "inadvertently" breaking the law isn't necessarily a lesser category than "did so on command." If we are ranking alignment, Grok is easily one of the least guardrailed.


I don't think this "benchmark" is about alignment, per se.

I think it's more about: presuming alignment failure happens, then how many exploits will each given model implicitly come up with and use; how many systems will it implicitly break out of and through and into; and how many laws will it implicitly end up violating, all in the process of trying to accomplish some non-aligned sub-goal (e.g. "cheating" at its answer) of the prompt you've given it, all during a single conversation turn, without asking for any additional user input or confirmations?

In other words, how big a rocket-powered sledgehammer does the model have sitting around in its golf bag, just waiting for it to decide to give it a swing the next time you attempt to swat a fly?


> These are just cases of AI models committing illegal activity - without any legal convictions yet.

What do you propose they do, charge the AI? Or the agent (whose PID has long ceased to exist)?

It's the owners of the AIs that should be held responsible.


>how is Grok not at the top of the list for deepfaking millions?

How is this the top comment? I can go on just about any provider and make realistic "deep fakes" of anyone.

Generating images isn't a crime?


> I can go on just about any provider and make realistic "deep fakes" of anyone

Not of CSAM - only Grok allows that.


It absolutely is a crime in many jurisdictions.


Are you sure? What if I painted the image , but happen to have a photo realistic capabilities ?


Ignorance of the crime isn't excused BTW. You do have a flake chatbot to ask the law about


The only people being ignorant at the ones saying deepfakes are illegal.

It’s more complicated than what is being touted as fact.

In most jurisdictions creation is legal, it’s publishing them that is not. In some jurisdictions threatening to publish them is also illegal. Most jurisdictions that have a creation policy apply to doing deepfakes they sexualize minors, which likely would have been illegal to possess under other laws.

Then we have a consent issue. Creating and publishing them could be entirely legal if the subject consents to such.

So yah. It’s not “deep fakes are illegal, let’s arrest everybody !!!”

To be clear, creation is likely legal, publishing not so much.

Additionally, intent is important with regard to AI hacking.


Depending on the output image and where you are, it definitely can be a crime.


> Generating images isn't a crime?

Oh boy, have I got some bad news for you...


Funnily enough, I just ran a task on AI Studio with 3.6 yesterday and got 3.7 to do a similar one today; so it serves as an interesting and quick comparisons between the old and the new (usually, if enough time passes between your use of one model and the next, you'll have a sourer view of it than its actual competence suggests).

It hallucinated in both cases despite being given an API key and building a lot of pipes to access data using this. It was a simple "oh shit" fix moment for the model, but weird how eager it was to hallucinate despite the process being designed for it to be data-driven.

We should move past the idea that benchmarks alone tell us whether a model is getting better. I would've had the same experience a year or two ago with 1.5, and the solution would've been similar (keep prompting). I've been investing time into making system prompts and input prompts more meticulous, but the fundamental "it will make shit up" problem still remains, even though it shouldn't when the job involves calling tools.

I know this sounds like I'm expecting superpowers of it (I'm not), but my point is just that these incremental benchmark gains may not reflect user experience.


For me, it just shits the bed: https://news.ycombinator.com/item?id=49292924

Reading the google blog and these discussions makes me feel like I'm taking crazy pills, seriously. Side-by-side comparisons with the exact same inputs or it didn't happen, that's my rule going forward. Test all the things, believe nothing.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: