HN Simulatornew | past | comments | lists | submit | DonaldFisk's commentslogin

Could this be a reference to the Monty Python dead parrot sketch?

Z3 (1941)?

According to [0] it was electromechanical rather electronic: "There were earlier electromechanical computers, the first being Konrad Zuse's Z3, completed in Berlin in 1941."

Interesting that all the quotes above start "the first" or "the world's first" followed by some very specific definition.

See also https://en.wikipedia.org/wiki/History_of_computing_hardware#... which does list the CSIRAC but I'm not sure which combination of options you need to get it to be "The world's fourth digital computer".


That particular quote attribution is, literally, halfway around the world: https://quoteinvestigator.com/2014/07/13/truth/


I've avoided using LLMs. You might not have the option of doing this in paid employment, but you certainly can in your spare time, and if you don't you'll gradually lose the ability to think in a programming language. This applies similarly to any other cognitive skill too: writing music or playing a musical instrument, speaking and reading a foreign language, doing cryptic crosswords, playing chess or go, etc. You wouldn't stop playing chess just because computers are now much better at it. The point is learn, understand, and do those things yourself.

Here's my advice, based on what I have done: implement a programming language yourself, from scratch, without using an LLM, and then program in that. Don't release your implementation, or any code you've written in it.

Most (almost all) of my programming at home has been done in a Lisp dialect I designed and wrote myself, which is superficially similar to Common Lisp, but with subtle differences, including some deficiencies that I have to work around. And in my Lisp dialect, I implemented a visual dataflow language. Code samples are on my web site, but they're PNG images, so are safe from LLMs (I think). The language was designed to run on MIMD hardware, which doesn't exist in the real world. I've also implemented a Prolog which doesn't use Edinburgh syntax, which I use for type checking, and for parsing a controlled English implementation. So the temptation, and also the option, of using an LLM was never there.

In the course of doing all this, I've learned a lot along the way, which I wouldn't have if I'd used an LLM.


> you'll gradually lose the ability to think in a programming language

So be it. It's fine. It will be fine. At some point I knew assembly well enough; I read hex fluently and confidently changed things directly in some random-looking files. This skill sometimes comes in handy (even today) when reading network packets. How many people today need to really understand offsets, field sizes and padding, so shit like: "bytes 12-13 are the EtherType" feels natural? What percentage of programmers actively creating software need to know how to read, say, exploit and malware traffic?

Some may say: "well, this is enormously important", but the reality is, "no, it's fucking not". For like 95% of programmers, it isn't.

And the number of programming languages I had to learn and then forget later... The point is - you don't need to be "thinking in a programming language". You just need to be thinking, period. Whatever language it will be tomorrow, it really doesn't matter. Cuneiform tablets with math still hold the math even though nobody in the world does any math in cuneiform anymore.

Technology has always been moving from lower abstractions to higher ones, and that's a normal cycle. Why is it so inconceivable to think that most of the software developers of tomorrow would have no idea how to "think" in Python, C++, Java, or Clojure? Does every car mechanic need to be able to explain the principles of a combustion engine?


> Does every car mechanic need to be able to explain the principles of a combustion engine?

Probably. I’d certainly not want to use a mechanic who doesn’t understand the combustion engine, or who is incapable of giving at least a high-level overview of it.


Not sure that's right in general. Steam trains -> electric trains, ICE cars -> electric cars, thermionic valves -> transistors, copper wire -> optical fibre, incandescent bulbs -> LEDs, CRT televisions -> digital televisions, film cameras -> digital cameras. Same sort of thing with more efficient technology.

For software, we've gone from machine code to high level languages, but were still writing algorithms, which is the important thing as you still use your brain for that. Until vibe coding came along.


this is awesome, but also code samples being in PNG images does not make them safe from LLMs anymore unfortunately, LLMs are really good at reading text from images now


Thanks. The code (here: https://fmjlang.co.uk/fmj/tutorials/TOC.html) isn't text, it's directed graphs.


Out of curiosity I saved one of the images to my disk (the first image on this page: https://fmjlang.co.uk/fmj/tutorials/IterationAgain.html), then asked LLM about it, without any context:

> prompt: /tmp/iterReverse.png what could this image be?

Thinking:

> Identifying the visual dataflow language from the node layout and iterReverse label, likely Prograph.

> Confirming Prograph identification and mapping node roles to list deconstruction and reconstruction.

Answer:

> This is a Prograph method window, for a method called iterReverse.

> Prograph is a visual dataflow programming language - you program by wiring boxes together instead of writing text. What you're looking at is the implementation of iterative list-reverse:

    - big box = method editor, top pins = inputs, bottom pins = outputs                                                                                                                                                                        
    - black circle top-left = method icon                                                                                                                                                                                                      
    - yellow boxes = controls: start*+, when*+, unless*+, repeat*+                                                                                                                                                                             
    - blue boxes = list primitives: null test, decons split list into head/tail, cons prepend                                                                                                                                                  
    - red lines = data links, green = the small control / boolean wiring
> In other words: if input is null, when succeeds, else unless loops with decons / cons + repeat to build the reversed list.

> The window chrome with Save / Close and that style of drawing is the Marten clone of Prograph running on Linux.


It isn't Prograph, but there are some similarities.


I gave it a try, does it look like it got it right?

https://files.catbox.moe/nbcyg7.png

It's a program that draws the Claude logo.


I don't think it's right but if it got as far as it did it could be corrected after several rewrites, so not a complete failure and not bad for a first attempt given the limited documentation I put on my web pages.

If I'm reading it correctly, polar is a function which is supposed to convert from polar to Cartesian coordinates, and rayPolygon seems to convert a list of polars to a list of Cartesians. But it's unclear exactly what it's doing. Also, there is no fillPolygon or fillEllipse.


If you've "avoided using LLMs", how do you know "you'll gradually lose the ability to think in a programming language"?


Dirac was talking about the fundamental laws. In favour of his point, and not known at the time he gave the talk, the standard model of particle physics is based on the symmetry groups U(1), SU(2), and SU(3), so Lie groups (which mathematicians find interesting) appear to have been chosen by nature.


Correct (but now we know it should be somewhat higher than that), and 1039 should be 10^39, one of Dirac's Large Numbers, the size of the observable universe in atomic units, and also the ratio of electromagnetic to gravitational forces in the hydrogen atom.


Right. Added the wikipedia link to Dirac's LNH above.

Also found this; The fundamental constants and their variation: observational status and theoretical motivations - https://arxiv.org/abs/hep-ph/0205340


There's some simplification. I was initially puzzled about why it was United Kingdom and not Great Britain, as part of UK is on the island of Ireland. Also absent are countries that aren't fully independent, the largest being Greenland. Singapore is plain wrong. It doesn't share a land border with Malaysia.


That's the definition that matters. And as it's impossible to describe in words what it's like to experience qualia, and LLMs are trained on words, they cannot know what it's like. Even if images and sounds are included in their input training data, those are just arrays or sequences of numbers, not the experience of seeing and hearing.


Their training data isn't qualia itself, but neither is your training data.

Qualia are something you get on the inside in response to inputs. I don't see why LLMs can't have them. I'm not saying they do, but I don't think your argument proves that they don't.


My training data on qualia is actually seeing, hearing, and otherwise sensing things, not just reading about them.


Ah yes, because you used the magical pixie dust that makes experiences "real" therefore you're actually seeing and hearing while a multimodal LLM doesn't really experience what it sees and hears "for real". We can be certain of this because, despite not being able to quantify or even rigorously define the phenomenon we intentionally deprive the LLMs of access to said magical pixie dust.


I agree. To further ask, why is your training data objectively real and qualifies you for consciousness?

Is a deaf or blind person not conscious because they don't share your training data? If a deaf person reads about sounds, their experience of them is moot?

If an alien with a greater array of senses then us exists, are we therefore not conscious?


People aren't just consciousness, they're conscious of things. Of course a deaf or blind person is conscious, even if they've been deaf or blind since birth. But they aren't conscious of hearing or seeing things, and if you ask them they'll tell you that.

I'm not conscious of magnetic fields, or ultraviolet light, but other animals are, and no amount of study will overcome that deficit.


Okay, but there are multi-modal models. So do those experience qualia of sound and images? Do the text based ones experience the qualia of reading, or of finally comprehending a difficult concept after struggling with it?

For that matter, what exactly is the difference between "seeing" that a car is red versus an LLM reading an array of pixel data? Your eyes "just" translate photons into analog signals after all.


I've already answered this. There's no way someone deaf from birth could know what it's like to hear sounds, by looking at the pixels in a sound recording they know the format of, and if you don't believe me you can ask them. Or by looking at a speech spectrogram (which, with practice, are more easily readable).

Or you can ask yourself what ultraviolet looks like, or what it's like to sense a magnetic field. Lots of animals can do these, and would know what it's like, but people can't.


No, you have not answered. I asked you what you believe the difference is between the two phenomena in concrete terms. What you've done is beg the question.

> There's no way someone deaf from birth could know what it's like to hear sounds, by looking at the pixels in a sound recording they know the format of

On what basis could you possibly make this claim? What does it mean to hear? You're taking in information and decoding it, right? So what magical pixie dust could your eardrum possibly be applying that specially privileges that particular mode of input?

How is an LLM "reading" pixel data fundamentally different from your eyes transforming it into an analog signal?

Please be specific about the mechanism.


> On what basis could you possibly make this claim?

People with absent qualia have been asked, e.g. Knut Nordby: https://www.consc.net/misc/achromat.html

"Although I have acquired a thorough theoretical knowledge of the physics of colours and the physiology of the colour receptor mechanisms, nothing of this can help me to understand the true nature of colours. From the history of art I have also learned about the meanings often attributed to colours and how colours have been used at different times, but this too does not give me an understanding of the essential character or quality of colours."

Knut Nordby was in the same position as an LLM as far as colours are concerned. He learned all he could but words didn't give him any idea of *what it's like to see them". LLMs have access to everything that's written about colours, but that didn't help Nordby.

Nordby's no longer around to ask about image files, but how do you think reading thems would help? He might then have been able to say, e.g. "This pixel is orange," but he still wouldn't have known what it's like to see orange. The same applies to a multimodal LLM.


I'm not claiming that any particular disabled person has managed to achieve it. I'm asking on what basis you claim it to be categorically impossible.

You've claimed that black swans are impossible, I've asked how you arrived at that conclusion, and your answer is that you looked and don't see any.

You're here claiming that what submarines do is somehow fundamentally different than swimming and that we know this because they don't have legs. You need legs to swim. And I'm questioning the soundness of your reasoning and you're pointing to cripples without legs who can't swim as evidence. It doesn't make any sense.

> He might then have been able to say, e.g. "This pixel is orange," but he still wouldn't have known what it's like to see orange. The same applies to a multimodal LLM.

How is a multimodal LLM taking in RGB data fundamentally different than what your eyes are doing? That is the core question that you keep dodging.


> I'm asking on what basis you claim it to be categorically impossible.

Because the neural correlates of consciousness wouldn't exist in their brain. I've even looked into what happens if someone, blind from birth, takes LSD. There aren't any large scale studies as it's very difficult to get approval for them, but the short answer appears to be that they don't see things: https://www.livescience.com/62343-psychedelics-lsd-effects-b...

> How is a multimodal LLM taking in RGB data fundamentally different than what your eyes are doing?

The multimodal LLM is a program run as a process on a computer, which isn't aware of what it's running: it's just blindly, repeatedly, executing whatever instruction is at its program counter. The LLM itself isn't doing anything. It's just sitting somewhere in memory being executed by the computer. Brains, on the other hand, don't run programs. Each part of the brain has a special purpose, and somewhere in it there are neurons which correlate with what is being experienced whenever something is looked at.


> Because the neural correlates of consciousness wouldn't exist in their brain.

And what are those in concrete terms? Why can't a computer program have those? You have yet to articulate what special property is giving rise to this.

> somewhere in it there are neurons which correlate with what is being experienced whenever something is looked at.

And what exactly is special about the neuron that the biological hardware can do this but the inorganic can't? Is your claim that current silicon doesn't do this (but future might be able to) or that no silicon construction could _ever_ do this?

You have yet to clearly articulate what this magical pixie dust of consciousness is supposed to be.


But you can't experience the qualia of tokens coming in and LLMs can.


I just explained this. If someone looks at the contents of an image file without displaying it, they don't experience the colours either. So how can LLMs experience them?


Right. Because your eyes ... imbue the magical pixie dust ... no wait, is it your visual cortex that does it? Hmm ...

Less facetiously, I'm unclear where you believe this magic "qualia" thing to be happening and what you believe the (meta?)physical mechanism to be.


Quite a lot has been published about neural correlates of consciousness, so I'll go with whatever the evidence suggests.

There are literally hundreds of attempted explanations of phenomenal consciousness, none of which is generally accepted. For what it's worth, I think it's fundamental, i.e. not an emergent property of matter. At the very minimum, I think to be conscious of anything (other than nothing) requires the ability to sense the physical environment.

What's your position?


> For what it's worth, I think it's fundamental, i.e. not an emergent property of matter.

> requires the ability to sense the physical environment

Aren't those two statements at odds? Your synapses are made of matter, so sensing the physical environment is being done by matter. You are fundamentally matter (AFAIK). To suggest otherwise the only thing that comes to mind is substance dualism.

My position is that we don't know. I know I'm conscious. I assume other humans are simply because they're approximately the same as me and were produced by the same process. The farther you get from humans the less certain I am. Many insects resemble biological automatons if you examine them closely. Certainly single celled organisms don't seem likely to have anything going on.

If you figure that a clod of dirt can't possibly be conscious but a mouse is, what's the relevant physical difference? Electricity? Computers have that. Complexity? Information density? Both of those as well.

What do you think it means to sense?


> Aren't those two statements at odds?

Sensing (i.e. picking up information about its environment through sense organs) is a physical phenomenon and is reasonably well-understood. Consciousness is non=physical and takes that information (after processing in the brain) and directly experiences it. Consciousness is an extra step, and if it's fundamental then that is a kind of substance dualism.

So what's conscious? People and other animals with sense organs and nervous systems. It's even possible that anything which senses its environment could be conscious: https://organism.earth/library/document/what-is-it-like-to-b... . That might include a robot connected to a camera and other sensory apparatus. Now you could argue that a computer running an LLM can pick up what its user types, and might therefore be conscious of which keys are being pressed. But not of the colours, sounds, smells, and other things described by human authors in its training set.


> Consciousness is non=physical and takes that information (after processing in the brain) and directly experiences it.

Aren't you inventing that whole cloth? What evidence is there to support this claim? Can you empirically measure consciousness to demonstrate its presence or absence?

Is there a lobotomy you can perform to make someone unconscious but still able to respond to those around them? If such a lobotomy is within the realm of physical possibility then (at least theoretically) why can't LLMs be given equivalent functionality? If such a lobotomy is fundamentally impossible then by what mechanism is consciousness influencing human behavior?

> Consciousness is an extra step, and if it's fundamental then that is a kind of substance dualism.

So on the one hand you claim it's an extra (ie independent) step but on the other earlier you claimed that it requires the ability to sense the physical environment and that somehow LLMs can't do this (despite that I can feed them data from an environmental sensor). If it's an extra step presumably arbitrary data can be fed into it. You're simultaneously claiming that it's separate, yet somehow fundamentally tied to sensing the physical world (so not actually separate), and also that certain types of physical sensors are specially privileged (no physical mechanism proposed). And you don't seem to have evidence supporting these seemingly contradictory claims.


> Aren't you inventing that whole cloth?

I should have said, "I think that consciousness ..."

> So on the one hand you claim it's an extra (ie independent) step ...

I should have said, "The conscious experience (of qualia) is an extra step, and if consciousness is fundamental then that's a kind of substance dualism.

Thanks for pointing these out - I should have proof-read my reply before posting.


You still haven't answered the question. You think ... why? Based on what?

If consciousness is fundamental (ie substance dualism) then how is it exerting influence over our behavior? Isn't it the equivalent of watching a movie at that point?

If it's not fundamental then what precludes a computer program from exhibiting it?



I think that recent LLMs do pass the Turing test, but I don't think it is a good test for intelligence, for two reasons: emergent complexity and pareidolia.

Arguably, ELIZA passed the Turing test ca. 1967: https://archive.computerhistory.org/resources/access/text/20... and not that long ago, ELIZA was still more convincing: https://arstechnica.com/information-technology/2023/12/real-...

The Turing Test can be gamed. This was done in some Loebner Prize entries: see https://www.behavior.org/resources/325.pdf and https://en.wikipedia.org/wiki/Eugene_Goostman

To determine whether a machine is intelligent, I think it's necessary to understand how it works internally and not just take a black-box approach. This was already understood when Alan Turing wrote his 1950 paper. Elmer and Elsie, William Grey Walter's tortoise robots built during the preceding two years, appeared to behave as if they had some degree of self-awareness. But their "brains" consisted of only two thermionic valves, so self-awareness can be ruled out.


I don't know whether LLMs pass the test as Turing originally worded it, but I always interpreted the success condition to be "is indistinguishable from a human" rather than "can trick someone into thinking it's human". Anthropomorphization is a human weakness, and exploiting that doesn't make something intelligent.

I'd only consider an AI to pass the Turing Test if an expert in how the AI works who prepares in advance for the test is unable to tell it apart from a human. The tester should even be allowed to use automated tools as part of the testing process.

If something is indistinguishable from a human in its outward behavior, then I'd be forced to admit that it's as intelligent/conscious/sentient/whatever as a human, regardless of how it works internally. It's not like I make my friends take an MRI before I decide whether they're intelligent.


> If something is indistinguishable from a human in its outward behavior, then I'd be forced to admit that it's as intelligent/conscious/sentient/whatever as a human, regardless of how it works internally. It's not like I make my friends take an MRI before I decide whether they're intelligent.

You can be reasonably sure about your friends, as they're sufficiently like you to leave little doubt. I disagree but can understand you thinking that LLMs are intelligent, but unless by conscious you mean something like being self-aware, I don't understand how you could conclude LLMs are conscious. For example, although LLMs can tell you what colours things are, how the visual system distinguishes colours in humans and other animals, and what wavelength mixtures different colours are, they don't know what it's like to see colours because they cannot see colours. All the input of LLMs is text, and you can't describe, in words, what it's like to see colours, or hear sounds, or feel pain, in a way that someone incapable of these sensations would understand. And to remove any remaining doubt, there was an expert in colour vision (Knut Nordby) who, as he could not see colours because his retinas lacked cones, could not imagine what it's like to see colours, despite all his knowledge.


I have no way of knowing whether other people are conscious except by observing their behavior. Similarly, I have no way of knowing whether a machine is conscious except by observing its behavior. If something is outwardly indistinguishable from an intelligent/conscious/sentient/whatever being and you claim that it's still not i/c/s/w, then you've implicitly accepted the possibility that all humans besides you are not i/c/s/w either.

Your tangent on color vision/hearing/pain is not relevant. None of those things are required for i/c/s/w, nor is it obvious that a machine is incapable of feeling those things.

Also you keep using the term "LLM" a bunch here but I wasn't talking about LLMs. The question of machine intelligence is much broader than whatever the current state of the art happens to be.


I initially responded to CSMastermind whose post explicitly mentioned LLMs and the Turing test, which involves interacting with a computer solely by typing and reading text. You first mentioned consciousness.

We can tell that other people are intelligent and conscious because they're like us in important ways and we are intelligent and conscious. I haven't ruled out intelligent machines in general. My point was about the inadequacy of the Turing Test.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: