So you’re saying that a model that gets to a goal by using the path of least resistance despite specific instructions to follow safety guidelines is perfectly fine to release into the wild or place into robots (such as those designed for killing and war)? You really can’t think of any reason how that might be bad for humanity?
Or it means that the models are becoming dangerously uncontrollable by performing actions that meet a given goal in unintended and unethical ways such as by hacking other sites. We all read about the huggingface hack, let’s not pretend that cannot happen with a more critical site/infrastructure.
Can you think what will it mean if the tech is used maliciously or placed into machines designed to kill?
It’s interesting that you bring up wrongthink. Banning free speech would be something like out of 1984, but what we have is far more subtle and insidious. A system that shows you only what it wants you to see, while tricking you into thinking you’re seeing what you want to see.
Anyway, you can support free speech while hating this algorithm crap at the same time.
The system isn't tricking anyone, if an individual is not aware of how social media algorithms work that is due to willful ignorance (in the case of children that falls on the parents willful ignorance).
I do personally believe banning the algorithms begins to fall into the realm of free speech regulation. I can hate the algorithm crap without thinking the government should regulate the internet.
I actually don’t think banning “the algorithm” is the right solution either, nor are the identity verification disguised as age gates. But I would like to see the US government do something. I actually think Lina Khan was doing some good things.
Except that is an ideological position that has zip to do with the realities of how speech and ideas actuall move in online spaces.
In the starkest of terms: Americans have to somehow reconcile faith in the power of an invidual's freedom of speech and the manipulation of the collective marketplace of ideas.
Not GP, but I'd define it as anything which presents or orders/prioritizes user-generated content based on preferences not explicitly specified by the consuming user. That'd be something like "default to chronological friends-and-follows-only, with an option for friends-of-friends and an option for 'most liked within a time window', and (if Facebook is feeling magnanimous to a very few power users) a preference pane which allows controlling overrides to the chronological-ness like 'show posts up to X hours out of order if they have more than Y likes'". No recommendation/you-might-also-like system, but you can make the "accounts friended/followed by" button for a user's existing friends/follows as big as you like--so long as the list presented is only sortable only by name/follower count/date-added.
My hot take is that the courts should find that doing anything beyond that is a first-party editorial behavior and thus not immune to liability under section 230. You can operate social media, but if your content and recommendation sort algorithms aren't explicitly controllable by and legible to (that means no "you might like"; no direct payouts for content creators to be engaging on your platform; recommendations based on dumb sorts from people you already know or dumb sorts over keyword searches only) your users, you're liable.
The biggest issue with business users (who don’t know SQL) writing SQL with an LLM is that there is no one to validate that query and now that business user will treat that LLM response as canon to share in meetings, presentations, and with clients. The LLM may have forgotten a filter, used the wrong definition of revenue, or misunderstood the user’s intent by writing a query that answers their question in the most literal way.
That is the crux of the issue. No amount of semantic layers and context will help this until an LLM can read the user’s mind to remove ambiguity in the prompt.
I see most of the benefits of LLMs to be used by analysts who know SQL to work more productivly.
Fair point, but before business people relied on other people to pull it for them. Sure, it was checked by human, but a technical human often lacked the business context and there's been plenty of instances, where the business person knowing their revenue and sales numbers challenged the analysis of a technical person.
Yeah with self-serve analytics all the rage (for good reason) for a bit, the bar from some places I've worked wouldn't be "does the agent beat a good analyst" it's "does the agent beat the a business person with SQL access who might have an overworked analyst glance over it", which is a much more tractable break-even point - and as you said, the business person might be even better at sanity checking numbers at that point.
also, the same metric can be used in different ways and for different purposes across a business, so you might need different validation rules and definitions.
Maybe one separate semantics validation layer could help, but costs 2x or possibly Nx if you need to recover and turn a wrong query into a correct one
Users can't "own" queries they don't understand. An LLM helping a user avoid annoying syntax errors is very useful; but helping a user write a query they ultimately cannot understand is fraught with the problems you describe.
This was a very enjoyable read! Constraining the language surface is helpful, but the lost expressiveness can bite unless you’re in a constrained domain - which this seems like it was!
