This is unrelated to the above comment, but I figured I'd ask while we're here. You state at the end of your blog post on this topic that you'd now probably recommend the NIST curves over Curve25519. Why is that?
It's possible and I dare say even likely that you are missing nuance and observing bias from your bubble. It's possible to avoid being an AI "opponent" while still being a sceptic. It's possible to use AI reluctantly and with tact instead of going all in.
The whole thing is far from over when we haven't experienced the worst of the technology yet: deskilling, concentration of power, extreme imbalance of wealth, military and surveillance uses, etc.
Of course, sorry if that came across as more combative than necessary. I was trying to point out two things I guess:
1. "Your experience is not universal from my perspective, and I offer my experience as evidence"
2. The possibility that this adaptation you are observing is neither a stable equilibrium given future developments, nor is necessarily an indication that the people using AI have bought into it fully. I use AI heavily while remaining a sceptic.
Hey, no problem, and thanks for the level-headed response. We need more of that in this world.
And yes, I agree that we still don’t know the long-term consequences of AI. But current LLMs are nowhere near good enough at judgment or new ideas to replace most jobs.
I expect they’ll mainly automate parts of jobs rather than replace them entirely. That’s exactly what I’m seeing as a software developer. I use an LLM to write my code, but it only works well when I apply good judgment and shape the context and prompts to remove ambiguity and then carefully review the results.
I think this matters very little. These were adaptations to initial advances of small scope which we are rapidly leaving behind. If there is truly no near and practical ceiling to the abilities of AI in mathematics, not even in terms of efficiency, then I think it is likely there is similarly no ceiling in other cognitive endeavours.
In that case, everything stands to be disrupted and those who wield AI will have massive amounts of power in the short term. But this is not a stable situation, and therefore those who wield AI will have to cede control of it or otherwise enter into a great war with those that do not.
I hope for all our sakes that there is a practical ceiling.
I would advocate for a shift of power by creating greater individual rights over their data (as their irrevocable property) , and greater oversight and consequences for those in power (ab)using the data.
It's one thing for our data to be out there such that a NSA employee could stalk their girlfriend, it's another thing for said person to be allowed to gracefully "retire" rather than be put in prison.
This is not advocating, this is wishing. It is as meaningful as "advocating" that you were rich and that Elon was poor, or "advocating" that you should be six inches taller.
If you consider the data on which an LLM was trained on to be points on a very highly multidimensional object, the claim is that the LLM can interpolate a convex hull spanned by those points, therefore recovering a subset of consequences attainable from those points. Obviously this hull includes completely novel points that were not present in the initial data set, so the output of the LLM goes beyond its initial training. And yet, there are clearly points outside a convex hull spanned by any finite number of points, such that we can imagine not all possible outputs are attainable using this method.
The claim is furthermore that truly original thinking, the infamous leaps in understanding and creativity, happen by attaining points outside such a convex hull.
It's hard to rigorously verify or disprove this claim. Hopefully this helps build an intuition of why the claim is not as shallow and obviously wrong as it may seem initially.
reply