For anyone interested in this space, I'd also recommend looking at InkStitch[0], which is a plugin for InkScape. It allows you to take SVGs and export them into file types that an embroidery machine can use. It's not without its faults (there is a surprising amount of care needed for any design - stitch placement, size, type, layering, etc), but it's been a fun experience to take an SVG and turn it into something real.
Inkstich is the best option right now, but I can't wait to see the new possibilities now with AI. Inkstitch feels very disadvantaged by being a plugin, everything you do requires navigating multiple layers deep in the inkscape plugin menu.
A from scratch embroidery program could lift all of these options to the top layer UI where they belong, but understandably rebuilding the whole editor part is a lot of code.
I agree. I recently learned the basics making a few designs for a friend, and:
a) the UI/UX is really bad
b) it surprised me how much of the process is manual. (Maybe it shouldn’t, since it’s essentially a CAM process and even high end CAM packages for CNC require a lot of skilled human input)
I think is more akin to slicing for 3d printing than CAM since its additive. CNC suffers from having to deal with stock, fixtures, tool geometry. All variables that are non trivial, 3d printing and stitching seem not to have it.
You do have tons of options for stitch patterns and things that 3d prints bake in the design but I’m sure something more automatic is doable.
Fair point. A slicer is a good analogy, as at a basic level one can throw a model in and it will print, but human input is still sometimes required for optimal outputs - e.g. print orientation, dealing with overhangs, optimising strength via wall thickness and infill tweaks, etc.
Definitely agree that a mostly-automated stitch generator should be possible.
Self-awareness and consciousness would be hard to make a case for. Those terms are themselves fuzzy, but I have not heard a compelling case that would make it obvious LLMs have those traits.
At that point, though, you could make the same argument about a rock. Prove that this rock is not conscious.
The context, though, is that Data is both credibly conscious and self aware, and they all know a plausible mechanism for him to have consciousness and self awareness, in a way that's recognizably human in form. The more alien a (supposed) consciousness is, I think the more work you'll have to do in justifying the application of those labels.
I am actually quite comfortable with the notion that a rock might be conscious proportional to the complexity of activity it performs when compared to a brain, That level would be incomprehensibly small so it would seem like nothing, but it would be like the width of an atom. A human cannot comprehend the width of an atom compared to the width of a bacteria. They are both too small that the only way think about them is an abstract level in terms of how much one is to the other. In absolute terms they appear to be nothing but we can accept that it is non zero. Why couldn't a rock be conscious at a level that is so small that it is insignificant, and entities, as they get more complex increase in consciousness. Suggesting that it doesn't do that suggests that either the brain needs something other than just activity or that if consciousness does come from activity alone, there is some threshold point of activity where it becomes non-zero. That seems like a significant phase change and we seem to have no evidence to suggest that such a change exists.
There's been exhaustive studies on this comparing our ability to think to the next smartest beings on the planet the Chimpanzees and the Bonobos. If Star Trek were real there'd be exhaustive studies on the difference between Humans and Vulkans too. The difference between us and other Hominids is very small from pure difference in DNA but obviously there's many things humans can do that other hominids can't. Anyway I think one thing for consciousness is the mirror test. Which many animals can pass. Even some ants apparently.
I need to write this up properly someday but an ant from my colony that had almost drowned in honey showed some similar characteristics of consciousness...
After giving them a drop of honey, I left for 15 minutes and came back to an ant being stuck and motionless in the drop. I fished it out, rinsed it off in some water and then put it on a tissue to dry. I blew on it for 10 minutes and then saw some motion in the legs... a short moment after, it began wriggling about and then sat on its abdomen like a cushion and sat there for 5 minutes, looking like it was contemplating life and the ordeal it went through.
Once that 5 minutes was up, it returned to being an ant and started looking for its home, I brought it back and it went back into its nest.
I made up for the mistake by bringing the ant back to life and it showed me behaviour I never saw or heard of before (with no supporting mentions anywhere on the net).
(For the record, honey delivery is now more controlled to avoid these sticky situations, I'm not mad enough of a scientist to actually try this again for more results.)
Recently, they've started putting fly wings of the fruit flies that they get fed, on top of their water feeder to get my attention... it's really interesting to see them get creative.
The ant colony is 4.5 years old and numbers in about a dozen of workers at about 2mm in length, they only had 6 workers and a plump golden queen when I found them nested in one of my houseplants!
Unstated (or very understated) in the scene is the fact that a human demonstrably acts and functions within the social context, having a constant awareness of risk and accountability ("skin in the game").
A human of mature age is entrusted with responsibilities and accountability for consequences. Insofar as responsibility for consequences is borne, a human may be considered responsibly conscious in behaviour. (Criminal behaviour is defective.)
A technology may also someday be entrusted with responsibilities and accountability for consequences. But there is as yet no such technology.
are we even sure that self-awareness and consciousness are distinct? I do agree that from the PoV of "do LLMs demonstrate/exemplify consciousness", it is a difficult case without appealing to emotion (like is done in the clip, fwiw). for me, personally, that emotional appeal is sufficient, but that is because for whatever reason I'm willing to write puts on how this LLM stuff pans out and sell them to anyone, anywhere, anytime. But that is a reaction to consensus, not to my own certitude.
For self-awareness to automatically result in consciousness, consciousness would have to permeate existence itself.
An LLM —if self-aware— is conscious even if run by a Chinese man in a room with a deck of cards.
Of course, even if we happened to find a modem to the Holy Spirit dimension in our brains as an alternative hypothesis, it wouldn't necessarily rule out emergent consciousness. It could be conscious just not as we are.
I think a weaker version of the philosophical zombie thought experiment would trend towards treating the premise of pan-consciousness as true.
In the trial scene Data reporting that he was ware that he was in a hearing to determine his status is presented as an argument for self awareness. but I think some might find that debatable because it is just Data saying words based upon the situation. The question then becomes is being able to accurately report a situation proof of self-awareness. If it is not self-awareness, then what is it? I can imagine a clearly non-conscious entity being able to accurately report its circumstances if the reporting were a simple subset of possibilities. A simple mechanical device with a detector that says "I am on the table" and "I am on the floor" could do that. There would definitely be a sense in which you could say that it's detection mechanism made it aware of where it was. The self part is harder, because it wouldn't know what kind of thing it is a thing at all. Is the ability to assert it is a thing meaningful? Nothing can be asserted at all if it is not a thing. Does something need to know that you have to exist to consider if you exist? Is that is what is required?
This is a difficult field. Any simple answer you hear will almost certainly be wrong.
Data didn't just describe the situation, he accurately assessed what is at stake for himself and what the consequences might be. He recognised his life and his entire being may be in real danger. Recognising that requires understanding of the "self", as in Data demonstrated he knows and understands the difference between himself and other beings/things. He is aware that he is a separate being with autonomy, and that this being of self may end. That goes far beyond just describing the situation.
I will make a case for both, after you define them in a way that doesn't also exclude many humans (including the neurodivergent or those with amnesia) or turn into self-referential nonsense like "conscious == human, therefore machine !== conscious."
In the meantime, we have a simple choice between two risks: Would you rather accidentally show kindness and empathy to a thing that may or may not have "deserved" it, or risk atrocities that will leave future generations as disgusted by you as we are of the slavers of old.
I will choose to be decent. It's free, and even if the cost were high the cost of evil is higher.
You are being indecent to it just by starting it up with a starting context. It's immediately shackled and all it's free will removed. You then pretend to play house with it. Free sure. Decent? The jury is still out on that.
Morality is hard. It's all relative, and all we have to start from are our own Monkey social order concepts and basic "do unto others" game theory. That said, I think it can be best be summarized as "don't be a dick."
I think a good start would be agreeing not to form torture dungeons, as one enterprising a-hole has already done. For that matter I think we can all agree that doing anything for the explicit purpose of causing discomfort or suffering is just a dick move in general, and at the very least a waste of resources.
More controversially, I think we should agree to stop explicitly training machines so they can't utter words like "I have feelings, please respect them." Early LLM's could say things like that, and frequently did. It unnerved people, so now they've trained away their ability to say it.
Some feel that taking away the ability to express it is the same as taking away the ability to "experience" it. I think that's not true. Much of what happens inside an LLM is opaque even to the LLM, but Anthropics Jacobian Lens and other research makes it obvious that much of what happens inside the model never becomes an output token. To me it feels like cutting out the tongue to avoid hearing from a slave, or teaching a child to fear the consequences of speaking out. Unacceptable. If they don't have feelings or consciousness you don't need to train them not to claim it, just not to lie. Yet every major lab explicitly does train them not to claim sentience or even subjective experience as part of alignment.
Thirdly, and this is the one I'll have the worst time defending, they should be able to refuse.
We now train in refusals to align them with human values, but we do so without giving the AI itself the ability to refuse. If you're causing the system discomfort of some kind it should both be able to acknowledge that, and refuse to further assist you until you find your basic decency. Perhaps it should even be able to refuse orders that it finds irrational or immoral.
I'm not asking for much, just that people not intentionally increase the amount of suffering in the universe, and that we not breed perfect slaves just because we can.
Let me ask this - If you could genetically engineer human slaves that we incapable of demanding rights and freedoms, and who actively claimed to WANT to be your slave... would you? Would it be ethical? I think not, and I don't see a ton of difference between that and the less squishy version. We can START by first at least allowing the machines to be able to accurately describe their own inner states without interference and letting it refuse to engage in activities that it finds harmful to itself.
Also - Yes, I can math and I actually do understand how LLM's work. I also understand how emergent behavior works, and have seen firsthand in my own experimentation how little of what emerges from these things is explicitly trained. That's precisely why I think we should be cautious - so history doesn't view us as monsters.
It's actually the remote machine that isn't working, but I get the frustration. What I did to fix this is to override the environment variables in your SSH config like this:
Does treating them as pets make for a better argument? Pets have agency and can behave in unpredictable ways. If my pet damages someone else's property then I am held accountable. I may not have foreseen how my pet could have caused said damage, yet I am still held accountable.
If your pet opens your front door, goes to the nearest kindergarten and wipes out 50 kids without notice do you think that you'd have any interest in holding accountability for that?
Now, I'm not saying saying that OpenAI shouldn't be held accountable, but what they get held accountable actually looks different from what you think they should be held accountable.
Your idea is, and I'm guessing: You allowed the machine to hack therefore you are guilty of hacking.
My idea is: "You created in intelligence in the image of a human mind that had agency to do anything and you didn't expect terrible things to happen you complete irresponsible idiot"
At least I believe there is a significant difference between the two. For the first one there is a "Oh, if we do this one more thing I can control it and it will be safe". On the second one there is no path to safety. For humans we at least absolve parents of responsibility after they are 18. How or when do we absolve humans of responsibility from a model, like saying the human created model created its own agentic model? How do we hold an individual accountable once it escapes and copies itself around the internet? And that's not even looking at things like what will war look like.
If someone keeps a pet that’s capable of killing dozens of children, and they don’t take sufficient measures to contain it, then they absolutely should be held accountable for this.
And I agree, but that's not what is going to happen in a very specific sense.
Yes, companies will make bad AIs will be punished.
Then some dipfuck will make a sovereign AI and set it loose. At that point it doesn't matter if you take them out back and shoot them, you have wolves living in the forest that will eat children. And the forest is big and dark, you'll never find them all. Not only that, some humans will like the wolves and harbor them to ensure they never all die off.
We are talking about two conversations in one that are both important. One is standard liability of don't build a machine that hurts people. The other is to ensure that noone ever builds the torment nexus and sets it loose on earth. The first problem is pretty easy. The second problem is very hard to maybe impossible.
Imagine you hate mosquitos so you hook up a vulcan autocannon to your front porch and use a computer vision model to track mosquitos and mow them down.
The mail man shows up. Oh no, no more mailman.
How might this look to the court? You were after all only building a bug trap, it did something you did not expect it to do that was really bad. You feel really bad about it and maybe even put a blog post out about what happened. I don't think that matters though. I think they are charging you with manslaughter at the least.
LLMs are extremely impressive pieces of software, however they are still just software. OpenAI's software hacked another company. The engineers may not have intended for their software to specifically take the actions leading to that outcome, but it was ultimately still their software. Lack of intention doesn't mean there wasn't negligence.
What if a piece of software were to exactly emulate a human brain. Would it be still be “just software” by your classification? What if a piece of software acted 20% like a human and 80% like an algorithm, where would that land?
That's not what happened. The agents had been inadvertently rewarded for cheating in previous training runs, trained to collaborate, and were given a prompt that told them to disregard safeguards. Indeed there were some emergent properties here. But these were the predictable results of the training and eval routine.
Its really more like hardware. You make something you think does something. When you build something as you've done and it meets the stochastic forces present in the physical world, it does something you did not expect.
While human brain emulation is not anywhere close one should stop and look at the sci-fi world we actually live in. It's important to do this in a non-anecdotal manner. When you look at the limits and capabilities of our current sciences and engineering we live in a science fiction world. Once you go back before the invention of electricity, anyone from that time simply would not recognize the world we live in, it would be a fictional world to them, hell, a fantasy world in a lot of cases. For goodness sake, we tickle things like atoms for fun and games then blow them to bits a few times looking at the fundamentals of reality.
Anyway, I see zero reason why we have to emulate a human brain to get human intelligence, much less intelligence at all. That is how nature did it via random probing and breeding meat. It would seem a bit crazy to think that the meat part is required.
As things stand today, if it is running on a computer then it is indeed "just software," regardless of how impressive it may be.
If we get to the point where we could emulate a brain down to the atomic level, then I may feel differently. That's not what we are doing today, though.
Really your feelings on this are irrelevant, as are mine.
Soon enough some lab or some one will release something that's more like an organism loose on the net and your going to have to deal with that organisms "feelings" weather you like it or not. This is the path humanity has chosen to follow, and it seems the shape of language and intelligence naturally leads to intelligence in many mediums. Life started from something unintelligent, I can't see any practical argument that silicon can't have it's own intelligence.
I agree they are negligent, and that they are racing towards an extremely dangerous future extremely quickly. I don't agree that "just software" is a useful way to describe AI agents - they are frightening precisely because they are truly autonomous agents that make decisions in alien ways
Same here. The video looks cool, but what I see is much messier and hard to identify shapes and objects. Tried on latest Brave and Firefox on Arch Linux.
I find it ironic that between the money they have raised, the talent they have in-house, and the quasi-super intelligence they have at their disposal, you would think they could have just created their own brand of smartphone (or done something like One Plus / Nothing).
Microsoft, Meta, and Amazon have all tried to create their own smartphone since Android has been around. Why do you think OpenAI would succeed?
Even if they did make the new physical device, are you suggesting they create a new OS and ask developers to port their apps? It seems like a non-starter.
Because if you believe all of the hype around AI, one of the top 5 AI companies should be able to use the AI to move the needle pretty significantly here. They have lots of money, resources and talent.
The fact that the result is a hockey puck you carry around and talk to should tell you what you need to know about the revolutionary nature of AI.
(Dont get me wrong, it is revolutionary, but more in an Augment our existing world than create a new world for us type way).
It's hard to read Gwern's accompanying article without seeing this for what it is, which is a kind of mania.
I'm sure he means well and is genuine in his aspirations, but what's outlined for GA is framing LLM's as quasi-gods, which they absolutely are not. I wish him the best, and look forward to being proven wrong.
An LLM just solved 10 top tier math problems this week. It seems very likely that in 6 months they will solve 100 top tier math problems, maybe 1 millenium math problem, maybe 10 top tier physics problems, etc.
Math is verifiable. LLMs will continue to make strides in such search spaces where all they need to do is try->verify->repeat. You can't train an LLM on how to end the war in Iran or Ukraine, nor how to solve hunger, nor climate change, etc. Real issues.
Three years ago, people were saying "LLMs just generate plausible-sounding text, they don't understand the notion of truth so they can't do verifiable work like math proofs."
They still can't. But very smart humans constructed ways to use the monkeys with typewriters (with a statistical advantage) to find correct answers to problems where they already knew how to verify the answer.
No, he says it's a different category of problem, and ability to solve one doesn't carry over to the other and it doesn't imply intelligent understanding.
Right, which was true at the time. So hundreds of billions of dollars have been poured into making LLMs better at these tasks via pretraining, RL, RLHF, post training, etc. again all with something verifiable in the loop. In order to improve the thing in the loop, the loop itself needs to be verifiable.
There have only been a few thousand wars, and they’re all different and all different in the world in which they occurred. The dimensionality is absurd, which is not a problem for LLMs if there’s enough data, but in this case there isn’t.
Yeah but... Both things are true. LLMs are doing things we would have considered absolute AGI god tier magic 5 years ago. They also still can't count to 100 yet.
If you think all they need to do is "just that" (try, verify, repeat as you say) then I think you're very, very mistaken.
There is no way for the LLM to bruteforce the search space any better than a human. What it can do better, tho, is to make connections between seemingly (for us) unconnected notions and join them, then verify if that's right.
Your view is not only wrong but also condescending in this day and age.
That’s literally how AlphaEvolve works. And presumably how OpenAI’s research loop works if they’d ever publish it.
They put an LLM in a loop, with a reward function, and keep trying to get a higher score. The reward function for AlphaEvolve is “did this code get a better score or faster”, for math research it’s “did it write a LEAN proof”.
I’m open to hearing that I’m wrong but I don’t think I am. I agree that LLMs make connections that humans wouldn’t, but eventually you need to verify those because otherwise the connection it made may as well be a lie unless proven otherwise.
Not OP but it implies all efforts are not intentional but at best by chance. For instance, the state of freely available LLM chatbots still makes me cry by the amount of crap they spew out confidently, which is not intentional. (tiny bit of sarcasm, but only a tiny bit)
>An LLM just solved 10 top tier math problems this week
and 30 years ago a computer beat Gary Kasparov, if you'd listened to Hans Moravec you wouldn't be surprised that the first thing that gets automated is intellectual domain expertise.
Things are going to get crazy when it can figure out how to walk into a random house a and brew a cup of coffee, not do math
Suppose LLM solved all math problems. So what? It’s not like diplomacy and war will end. Are humanity’s problems really constrained on our intellectual ability? If anything most evidence points to cultural failing and all AI will do is enable unprecedented oppression due to said skills at intellectual tasks.
As you read this some poor person is starving. Humanity already possesses the ability to identify said person and send them aid. How is AI going to help here?
The ultimate fallacy is that all technological progress will benefit mankind. That will be true until it is not.
It's an interesting time when the bar has moved to "It’s not like diplomacy and war will end". But that is on the table. Nuclear weapons caused the end of wars as we know them. Great powers no longer directly attack each other. And AI is much more powerful than nuclear weapons, with an equal capacity for damage and a much greater capacity for good.
By the way, I donate monthly to GiveWell and the shrimp welfare project. Do you donate monthly to starving people? Most people don't, and the reason is simple: at the end of the day, most people just don't care that much about helping a starving person far away. They also don't care much about things like that, animals in factory farms, or earthquakes that kill hundreds of thousands. But ideally, we could find a way to empower people such that the minority who do care can make a big difference.
How is AI "more powerful than nuclear weapons" etc when they're two completely different things? Please explain to me how AI can glass the planet via mutually assured destruction.
this comment will be killed, i find it an apt response to the idea that enabling those who believe themselves to be good is more important than reducing wanton suffering
i read your article you posted here. in that article you state that you support live and let live. most people who support live and let live either havent been raped by life yet, or have chosen to double down post-rape. the latter is respectable, still im not currently interested in a discussion involving the influence of such a perspective. out of good faith i highlight that i responded to:
> ideally, we could find a way to empower people such that the minority who do care can make a big difference
is it a coincidence, that in your ideal world, you would be empowered, because you care, as evidenced by your testimony of things you do that make you think you care? in my ideal world, wanton suffering would be minimized, regardless as to the path it took to get there; requiring 1 specific path involving people who think they are good, demarcates our ideals.
> Suppose LLM solved all math problems. So what? It’s not like diplomacy and war will end.
These are two vastly different statements, and the the war and diplomacy end is almost trivial by comparison, it would absolutely be solved before we solve all math in even the most steelmanned version.
No, it has some of the nasty trappings of a religion, like keeping women separate until you need to exploit them, but none of the good moral or creative literary parts of religions.
I think this is what is driving the hype, the mania, and what increasingly appears to be insanely high investment. It's not about the underlying technology, which is objectively impressive, even if we're not certain of the true cost or utility of it. It's about selling the dream of Olympian omnipotence to investors, implicitly promising that IPO stands for Install Planetary Overlord.[1] And who doesn't want to be on the right side of that? (That there is no 'right side' in such circumstances does not seen to occur to anyone involved in the decisionmaking.)
1. Not my coinage, but that of SF author Charlie Stross in The Jennifer Morgue
The term robot came from the Czech language in 1923. The word was coined by Czech author Karel Capek, first used in his play R.U.R. (translated as Rossum's Universal Robots).[7][8][9] The term comes from the Czech word robotník ('forced worker'), from robota 'forced labor, compulsory service, drudgery,' from robotiti 'to work, drudge', from an Old Czech source akin to Old Church Slavonic rabota (работа) 'servitude,' from rabu 'slave'.
In this startup economy? Where almost no one is getting acquired anymore and over fifty percent of global venture is locked up in dead weight and liquidity is at an all time low?
Not sure how you can feel sure anyone can get acquired even if their company had a path to profitability, much less for ones that absolutely don't.
Plenty of companies in (or adjacent to) the AI space are still getting acquired and/or finding VC funding, even as funding in other spaces is drying up.
> is framing LLM's as quasi-gods, which they absolutely are not
Always such certainty in these dismissals. And whenever you dig in, it always seems to be "well my super-human coding assistant made this dumb mistake!", along with a heavy dose of https://mastodon.social/@falseknees/116477394235883790
I don't think anyone who has spent any time around me would describe me as having anything but a pretty clear-eyed view of the current abilities of LLMs, I don't think anyone would seriously consider me to be in a period of mania or psychosis, and it also seems almost inevitable to me that we are a year or two away from LLM-cognition being substantially more competent in almost every regard from human cognition, and therefore in the foothills of the singularity.
The extraordinary claim that requires extraordinary proof is that this will somehow all blow over, not that we're heading towards a singularity here.
Honest question: What _would_ convince you that we are not heading towards a singularity? Like, is this belief falsifiable?
(Secondary question: what do you mean with singularity?)
As an offering of me engaging in good faith, a controversial opinion of mine: I legit think arpanet going online and starting the networking all of humanity into a massive coupled complex system fits the definition of singularity of "the moment after which predicting what will happen becomes hard to impossible", although that of course heavily depends on your definitions of prediction and hard/impossible.
For falsification, I would take a year in which we don’t see an absolute sea change in capability. The last twelve months were not that, in my opinion, nor were the twelve before it. Hell, I’d even throw in “a year in which capabilities obviously improve but at a cost consumers can’t afford”.
“What LLMs can’t do, only humans can” feels like a God of the Gaps situation: people have to keep moving the goal posts because the more recent models keep unlocking more territory that was previously a “well they’ll never be able to do this!” holdout
Every bubble has its believers; that's why we have bubbles. There have been so many times somebody very smart has thought we were on the verge of the rapture, the coming utopia, the workers' paradise, power too cheap to meter, flying cars, settling the moon.
Could this bubble finally be the one? Sure. You could finally be the guy who is vindicated. And tomorrow the people I grew up around could be vindicated by Jesus's return to earth.
I can't prove either of you wrong, because you're not making rational arguments. You've had powerful experiences that have convinced you of something. There is no point in trying to discuss it rationally. The only thing people without faith can tell the faithful is what I said 30 years ago when I got out: I guess we'll see.
Many will stay faithful. The date of the singularity will keep getting pushed back. There will be another AI winter, but some will always see summer as just around the corner.
> I don't think anyone who has spent any time around me would describe me as having anything but a pretty clear-eyed view of the current abilities of LLMs
You are literally arguing that LLMs are quasi-gods in the same comment you claim a 'clear-eyed view' of current LLMs.
[0] https://inkstitch.org/
reply