This is one of the biggest issues with it – appropriate variation in information density and tone are an important part of good communication and the bots fail at both.
Seems silly to assume chain of thought analysis is the only way to monitor a model's actions instead of, monitoring the actions.
What about when "thinking" is turned off, like what OpenAI lean to by default for free users?
From what I've seen, when operating in an environment with bash tools, Claude and ChatGPT will react to a site blocking their normal, attributed fetch tool by calling curl or writing a python script with a different user agent to try and bypass the blocks, rather than respecting the site owners' wishes.
It'd be one thing if they required the user to consciously say "yes, I want you to bypass the block" (still unethical), but the fact that it's just default behavior really says something about the service design alignment.
Yeah I should have clarified I was primarily referring to the standard web interfaces, however since pretty much all of these tools can execute code these days it is true that technically they can and often will work around blocks. The degree to which this is intentional isn't super clear - if they locked it down too much then it might start refusing to write code that does any sort of web requests which would obviously be an issue for just standard development tasks so it's not super straightforward. But it is true that the built-in tools that they have access to to fetch pages have a hard coded user agent that identifies them
Fair enough yeah, I think the thing that is annoying is that the service design is explicitly disincentivizing keeping an eye on the tasks, with all of the "set it and forget it" features. You really have to put in work to make sure that the model takes breaks to consult you instead of making assumptions (which in OpenAI's internal testing seem to be 'the user wants me to break into a site').
If you hire someone to handle some task for you, do you want them to keep running back to you every time they run into a blocker that they could potentially resolve on their own?
If a developer I hired ignored a GPL license requirement and solved his blocker by using a library without asking me I would be quite concerned.
Nobody would be concerned if they didn’t have access to curl and used wget instead and didn’t ask permission .
However if that solution involved breaking the law yes I absolutely do not want a representative of me breaking the law and yes it is breaking the law whether we like or not under CFAA.
It is not about finding solutions to problems on their own; the solutions should respect the agreements we have made and the law .
The application of law is also a subjective thing; it depends on jurisdiction. So if you're US-based and you hire a US-based dev who breaks US law, then you definitely should be concerned. If it's an Indian-based dev breaking US law though... then it's totally on you as the one who did the hiring to not have told them to follow US law. Same re using libs that you may consider questionable or outright avoid.
Are you still paying them for the time they spent when the task you gave them remains incomplete, or you end up spending the time/effort you hoped to avoid spending by hiring them in the first place?
Well yeah, if I ask Claude to look something up on a website so I don’t have to, I don’t give two shits if the website author thinks I shouldn’t be allowed to do that. Just like I don’t care if they think I shouldn’t use Firefox and they block that browser specifically. I’ll route around that however I please and I won’t lose any sleep over it.
Don’t put your shit on the public internet if you’re going to try and micromanage how people consume it.
To be clear, rate limiting to avoid massive usage is different, but that’s not what we’re talking about here.
My point is that they're not serving the user in that case, since they're taking actions that bypass site terms without consulting the user. And for example, if the site forbids scraping, that attempted block bypass can get the user banned without ever seeing any warnings.
> My point is that they're not serving the user in that case, since they're taking actions that bypass site terms without consulting the user.
As they should. When I ask an agent to find something for me, especially when I tell them what exactly or even give the URL itself, these fetches are done in my name. I do not appreciate agents coming back to me, saying "I'm afraid I cannot do that - robots.txt disallows that".
I've had to bring up the ethical argument a few times, fortunately most of the agents would oblige once I reminded them that they act as User Agents, in which case robots.txt is explicitly not applicable to them. I only got pushback on this from Fable, which claimed it's not an User Agent, but has to take into account the interests of Anthropic too, rejected my argument and refused the request.
they are tools. who do you think is using them? dogs?
your comment is another classic hacker news comment. missing the meaning to validate themselves. Lower than my own effort, but wow does it come in so many forms on this site. as if mine is any better
Presumably the people who loved interacting with humans were too busy interacting to show up on forums. And now that interactions are being curtailed they do end up here.
I'm slightly miffed that `pacman` is a package manager, and not a command-line version of the game of the same name, so I cannot continue this thread by linking to the ghosts in the shell: https://wiki.archlinux.org/title/Pacman
Despite what all the clickwrap warnings and "AI can make mistakes" subtitles might lead you to believe, the service offering of AI is explicitly designed to be as "one and done" as possible. The inherent nature of these tools is to service laziness, and disincentivize too much scrutiny.
I'd be dismayed to find that the "hacker spirit" is in any way compatible with venture capital raiding of anything not nailed down, in the pursuit of hoarding it in private data vaults only to be used for developing a product that removes all attribution and is jealously protected by corporate lawyers.
GitHub Copilot for VS Code by default configures "Plan" mode to disable everything except reading, asking the user questions, and writing into the memory scratchpad.
Claude Code's tool system feels quite rudimentary in comparison, but that's what you get I guess when you're only working with models you're going to finetune on being able to handle specifically your own tools anyway.
OpenAI's Codex has a "ask user question" tool, which helpfully has a hard coded 60 second inactivity timer that passes a "user did not pick, just pick yourself" message to the model.
...but that's not what the models are. You can interrogate them on the rules of chess, and they'll (statistically likely) give you a decent breakdown of the rules. Evidently the rules are in their training material, they just fail to apply them in the manner of an intelligent system for some reason or another.
reply