My dad is extremely irritated with Android TVs as a concept. He just wants to watch something and complains about it taking a few minutes on startup to even show the screen where can can select YouTube.
He does not understand that it is just like a phone. Unless you keep it on standby (which he does not want to do: wastes power), it will boot, slowly, till all the subsystems are up and running. The low powered processors, the low speed ssds, and minimal RAM makes everything worse.
Your dad is right to not understand that. There is zero reason for any appliance to take longer than a second to turn on from zero power. Even complex systems can use fast boot techniques (i.e. hybernation) to power on quickly without any power draw while idle. All it takes is developers who care and/or consumers who demand better.
Which is an implementation detail with practically no power use on a modern SoC and significantly better than wearing out flash memory in a couple of years.
To his credit, there's no technical reason that a processor running tens of billions of instructions per second couldn't immediately respond to menu inputs on the next frame.
Sure, the network might add a perceptible delay, but but the interface could load right away and populate the videos titles and thumbnails as they load.
Modern TVs use a max of 5 watts on standby. Does he think it's higher? Maybe a watt-meter plug is a simple solution (or it may make him obsess about the power usage when turned on!)
The pricing is opaque for stupid reasons. All the information is actually available for you to calculate your usage exactly.
- You pay $10/mo for Go and get $10 worth of usage per month
- Now, each model they make available has a multiplier from 1.5-8x
- So, the expensive ones like Kimi K3 have a 1.5x multiplier, cheaper ones like DeepSeek 4.1 Flash have 8x
- This gives you usage of $15 and $80 per month respectively
- You can mix and match
- So, you can use 25% of your quota with K3 and 75% with DS41Flash ... usage of $3.75 and $60 respectively
- You divide the two figures by their respective multiplier ... $3.75 / 1.5 and $60 / 8 ... and you end up with $2.5 + $7.5 = $10 which is what you paid
- You should ignore the requests table and go directly to the price/M tokens table where all the actual rates are given
- They control usage at 5h/1w/1mo intervals
- You can use 25% of your monthly quota in a 5h block, 50% in a week, and 100% in a month.
- If you were only using DS41Flash, that is $20m, $40 and $80 worth of usage respectively which is $2.5, $5 and $10 of your subscription figure after accounting for the multipliers.
It would be so much simpler if they credited 1M microdollars per $1 of subscription and quoted the per M token rates for each model in this currency.
But then Claude/Codex/StepFun and all the other subscription services are far more opaque.
> Using a GPU has, once, caused a driver reset that took the desktop with it.
Only once? This has happened to me so many times with LLM-assisted development. I was writing an OCR engine and pathological shapes would cause OOM that would take the display out. On other occasions, the LLM would try to load 24GB of bf16 weights onto a 16GB GPU without understanding the consequences.
Have to encode this in logic: keep some space reserved for the display, and check before allocation. But some errors are bound to slip through.
---
Inspired by DwarfStar4 and Strata, I am trying to build my own gemma4-specific inference engine designed to run on gfx1201.
It is a lot of work. You cannot simply vibecode one in two days.
It will be when done. I have a lot of stuff in the works and switch between them based upon my current interest. I like plotting and writing stories and Gemma is one of the nicer writers.
The goal is to get the 31B working on a 16GB card. Which means weights have to be < 4 bits. The 9070 XT has wmma support for int4. Which doubles compute performance on pp compared to i8/fp8. tg is limited by bandwidth.
Have you tried OpenCode Go? $10/mo is what I am willing to spend because most of the stuff I do is hobbyist. And OpenCode Go gives you $60 worth of MS Contributor 1.3 usage for that $10. Starting today, they actually provide $80 worth of DeepSeek 4.1 Flash usage if you want to use that.
Unless you are running a swarm of agents, it is extremely difficult to exhaust these plans in a month.
Yes, I'm on OpenCode Go too... and it's great they raised the usage levels on DS4.1 Flash. But I was really thinking of cancelling it to get either a 2nd Muse Code sub or to go up to the $50 tier.
You have to realize that $15 Muse Code sub with Muse Spark 1.3 gets you around $1500 worth of usage. Neither Opencode Go or really anything else is even in the same ballpark of value. https://x.com/PawelHuryn/status/2106724136932327458
I was hoping this level of subsidization would last a long time but today's announcement make we worried it'll end pretty quickly. There's a lot of AI influencers noting just how fast Muse Wonder uses up their quotas so hopefully fixes are made.
Subscriptions like these make money by relying on the fact that most people do not try to (or have the time to) extract every last dime out of their plan. The eighty people who do not use up their quota cover for the twenty that do.
That and deals with providers to use spare capacity I guess.
There is a lot of terminological confusion in the field. The fundamental difference is about how the "AI" learns. And who, if at all, supervises the learning.
Some systems are fully formed AI. Often based on heuristics. Humans have encoded the rules into the system. Others do limited learning based on pattern recognition. Still others learn based on tagged data. And LLMs don't need even that. In a stream of N tokens, the trainer has access to K+1 when it is training a model that knows 1...K and can check if the prediction was correct.
AI literally refers to any artificial thing that feels intelligent to us: does some thing a simple bash script will not. Your rules based chess engine is AI. The bot you compete against in single player Age of Empires 2 is AI. DeepBlue was AI.
ML is where things become interesting because we teach machines to learn.
NN is where we borrow terminology from neuroscience and create "neurons" that learn.
DL is where we stack neurons deeply in layers.
I am trying to make sense of all this myself.[1] So I am building a curriculum for it as I learn. So much of the teaching material in the wild revolves around torch. Makes it useless if you want to peek inside the actual machinery.
I feel like this statement is too anthropomorphizing. We don't teach a computer anything. We write normal computer programs that can process data(training), turning it into what is essentially a configuration file(weights/parameters) for another normal computer program(inference engine). Upon receiving a request, the inference engine is invoked and passed the input, then it processes that input and its configuration(weights), producing an output. "Thinking" is just feeding the output back in as an input, allowing the model to revise it and refine it before it's returned to the user.
There is no teaching, nor learning. There is just data processing. There is nothing happening between requests to the inference engine. It's a relatively simple system altogether, but when the training data set is large enough it produces unbelievably good outputs.
With this in mind I really don't understand how people think these programs can be conscious. They're not doing anything fundamentally different than any other computer program. It's just crunching numbers. You might as well argue that a computer game or 7zip or a chess bot or whatever is conscious.
I know you specifically didn't claim they're conscious, I just thought it was worth mentioning because I think it's ridiculous how many people will seriously argue this as if there is any difference between "ai" and other programs. There isn't. It's just a set of instructions for doing computation, as computers have always done.
3D rendering is actually quite similar. There is a bunch of files (input/configuration) describing 3D meshes, textures, lighting etc which are fed into a rendering engine which processes all that data and turns it into a 2D image. Similarly to AI, once the input files are sophisticated enough, this can produce mindblowingly realistic and beautiful images, and you can run it in a loop to make video or make it interactive, moving the camera around in real time etc. But at the core of it all is just a bunch of files containing a bunch of numbers, and a program that reads those files and does computation with them.
The current leading materialistic model of consciousness is essentially that human consciousness is a computation in itself, and that the brain is basically a very very complex computer. So it's not at all illogical for someone who holds this view and who fully understands that AI is just a regular computer program to also believe that it could be conscious.
Of course, I'm not saying that you have to believe any of this - even defining what consciousness is is famously elusive. And there are a lot of people who think consciousness is not materialistic to begin with, or that we have not yet discovered the scientific nature of it.
I definitely agree that consciousness is mysterious. But I think the assumption that AI is conscious begs the question "then why isn't 7zip conscious?".
Is a simple calculator conscious? How about a mechanical calculator? Is an abacus conscious? Are they only conscious when executing a sufficiently sophisticated program? What if I hypothetically had infinite time and patience, and performed the inference on paper with a pencil. Would it then be conscious?
Well, the exact same question applies to biology. If humans are conscious, is all life conscious? If it's just the brain that's conscious, is every part of the brain conscious? Are individual neurons conscious? How about other cells in your body, or tissues or organs? Our stomach has more cells and more neurons than many full animals - is our stomach conscious?
The answer in the "consciousness is a computation" model is that consciousness can arise in any computational medium. But the model doesn't say that any computation is conscious - it says that consciousness is any of a set of specific computations, any computation with properties X, Y, Z whatever. So it's perfectly possible (according to this model) that a GPU running game graphics is not conscious, while a GPU running some future even more advamced LLM is conscious. Or a network of GPUs + storage + whatever.
Good examples. We don't know what human intelligence, understanding or consciousness is but we do know what all of those examples are. And we do know what an LLM is. Like you said, it's a data processing machine and we know what algorithms it uses. It's more productive to discuss it in those engineering terms because we've done plenty of data processing, statistics, control systems, curve-fitting etc. before. There's no need to make this jump to consciousness/magic except for marketing or fantasy purposes.
The computational theory of mind is the leading theory for pure cognition. There is no leading theory of consciousness, although some people also favor the computational theory of mind for that as well but this is far less popular.
I think that among those who believe in materialism, the only two common positions are either that consciousness is an emergent property of the computational mind / an illusion (I view these as somewhat similar, though you can disagree), or that consciousness is some kind of physical interaction that we have basically no understanding of; and of course there are plenty of "agnostics" on this topic. Among these, I think it's fair to say that the first group is the larger one among materialists.
If someone thinks that a human mind operates like a computer, they might think that humans are interchangeable with computers, and then decide that we are better off without humans.
This is a bit like saying that if someone thinks God doesn't exist, they might think it's OK to disobey God's laws and start killing people.
Morals are separate from metaphysics, and a moral person would never advocate for eugenics or genocide regardless of their metaphysical beliefs. I personally subscribe to the belief that human minds are computers, and yet feel no urge to replace humans with computers.
As a complete side note, I don't think replacing humans with computers would be a form of eugenics. Eugenics refers to the practice of deciding which humans have a right and/or duty to reproduce for the (supposed) betterment of the species - such as through forced sterilization of "inferior" individuals, incentives for or even compulsory breeding for "superior" individuals etc. Simply killing all humans to allow for machines to rule is more like a form of genocide.
We humans have the habit of falling into the trap of anthropocentric views and explanations of the universe and everything within it. The history of AI is the history of moving goalposts. I don't particularly care if they are conscious or not. Humans have a history of treating even actual, conscious, intelligent beings as chattel or food. So I don't really see why our attitudes would change if we somehow realized that LLMs are conscious.
I have been thinking about this for a while in the context of the quest for AGI. If all we want is something to take over some parts of our intellectual labor, then LLMs are already there. They can communicate with you in a language you understand and do stuff that you command them to. Often, on their own.
AGI will have its own telos. An LLM does not (at least I refuse to believe that it does).
So, yeah, you could think of LLMs as programs processing data. But there is definitely some emergent behavior you notice that emerges when the LLM runs. How you interpret all this depends on your view on consciousness and intelligence.
I also don't really care whether they're conscious, but I do think it's worth challenging the claim that they are.
LLMs are programs processing data. Lots of programs exhibit emergent behavior. A simple example is Conway's game of life. It can be programmed in less than an hour, exhibits lots of interesting behavior. In fact you could run LLMs inside of it if you wanted to.
I'm not denying that LLMs are unbelievably cool and useful. It still blows my mind when I send a half-assed complex screenshot and a vague description and it just gets it and fixes the issue in seconds. It's like magic. But it isn't magic, it's just some relatively simple algorithms processing a crapload of data, and I think it's useful to understand and remember that.
Yes but what part of a human being isn't a program processing data? As far as we know all known physics is a turing computer? If that's thr objection against llms, then you have an objection to everything thr universe including human beings.
> I feel like this statement is too anthropomorphizing.
That's number one mistake preventing people from getting a good grasp of how LLMs work and how to use them effectively.
That is, not anthropomorphizing is the mistake. It gives people bad mental models.
LLMs are not people in general, but behaviorally and in terms of system design, they may as well be. They have same failure modes and cognitive characteristics humans have, and are very much unlike regular, non-ML software tools.
Not really sure what you mean, I have the view I presented and I'm able to use LLMs effectively. The understanding that they're not human allows me to treat them as such - I don't say please or sorry or thanks or other pleasantries, I just instruct them in the simplest most efficient way I can. Humans would find me rude, be annoyed if I give them lots of work for little benefit, and say the same things over and over etc.
> "There is nothing happening between requests to the inference engine."
This is true for vLLM/SGLang/etc but LayerScale is making use of idle GPUs. A new novel inference engine for agentic workloads. Our advanced scheduler enables double the workload on the same number of GPUs (e.g. prioritization of on-demand queries and background processing of batch workloads simultaneously). Let me know if you want to try it out!
Software has rarely been the moat. Or file formats. You have always been able to reverse engineer them. The problem, always, has been network effects.
I can build an entire, fairly useful, spreadsheet app over a weekend. But can I send my "expenses.cells" files to my accountant? Will it work with the Excel/Google docs he uses?
AI can build or reverse engineer anything as long as you are motivated enough to do it.
One example is Affinity 3 released for free. But it's a huge pain in the ass because all the guides for how to do things are for Photoshop or Affinity 2.
Your vibecoded app won't have years of reddit posts showing how to do things. This also seems to be where LLMs are the weakest at giving advice, they hallucinate 80% of the time I ask them how to do something in Affinity, giving buttons and menus that simply don't exist.
This is a knowledge problem. You can fix it by pointing the model to documentation (if it exists). Otherwise the model will give you the next best guess
Huh? Both Anthropic and OpenAI have computer use tools. I frequently just tell the models to test their own work. I'm a little paranoid so I only grant access to one window and make that window VMware Workstation.
You combine it with /goal. I usually set a goal like "Complete the application defined in goal.md as written. Then test it end to end autonomously using Compter Use. Record all issues discovered during testing in a to-do. Then fix the issues in the to-do. Repeat testing until no more issues are discovered."
> The people complaining most loudly about RAM costs are the people who want to run local models
This is a tiny percentage of the population.
Samsung is cutting phone production because of RAM prices.[1] The consumer market is badly affected: budget phones, laptops, general electronics.
The budget segment of sub $100 devices in India has been almost wiped out. Manufacturers cannot afford to spend 50% BOM on RAM+storage. Unless employees are getting a 15-20% wage rise this year, I expect a similar situation in most places.
Between the engineered conflict in the ME triggering O&G price rises, and stratospheric RAM pricing, the situation is pretty bad.
I do it for the LE certificate. But "cheap" is subjective. Domain registration and management should not be as expensive as it is. Nothing the registrar or registry does could be worth $8/10/200 a year with guaranteed 5-10% price increases every year. It is a grift.
I have been using Python for the past two years for various things. But it is criminally slow. I joke that using Python on your modern 2020s CPU upgrades it to a Pentium 4 from 2004 running code compiled with C. And the 2004 version might still be faster.
Yes, some of the syntax is nice. But the rest of it, the scoping rules, the obstinacy around the lambda syntax, the venv/pip nonsense, path resolution etc is a hot mess. Were it not for astral tooling like uv, I would have abandoned it in a few weeks.
I hate thinking about memory outside of very specific workflows, so I am willing to accept a 2-3x penalty over C for cleaner and shorter code that follows the happy path. But anything beyond that and you are wasting energy and people's time.
For Python to work as something other than a glue language where we write all critical code in C/Rust and call it from inside Python, something like PyPy is almost mandatory. But the design of the language, the exposed innards, and the need to maintain backward compatibility makes optimization a chore.
I did some benchmarking of Python against C, Go, Rust, Node, LuaJIT etc in preparation for my runtime.[1] My observations based on some stats:
- You can blindly replace C with Rust for most tested workflows. There is very little performance difference outside of compiler speed. (C23 is a nice language though.)
- Go is 2-3x slower than C
- Node and LuaJIT are 3-8x slower on average compared to C
- Python is 60-70x slower. It can get far, far slower in certain cases.
You can run your own benchmarks. I think you might end up in the same ballpark.
[1] I have been interested in reverse engineering, hobbyist compiler/VM development etc for a couple of decades and wondered what is the point of ranting for two years about it without doing anything concrete. So I decided to build a runtime and statically typed language borrowing stuff from Python and Erlang/BEAM. But, even with LLMs implementing my ideas faithfully, the last 20-30% is a never-ending process which means I get bored and move on to something I can finish in 3-4 days.
It also depends on what your use case is for the programs you write in those languages. If the use case relies heavily on CPU performance, then yeah, Python is awful in that regard. But if you're like me who mostly deals with ETL processes and report generation from several different systems over the internet, then Python's 60-70x slowdown means NOTHING in the face of network latency and the need to work with pagination limitations these external systems impose on REST endpoint users (and no, gRPC or its binary-serialization ilk are not available options).
Outside of that, though, what really grinds my gears about Python aside from its broad standard library compared to other languages is that it is home to what I think is the best property testing library in the world, Hypothesis. I know that the folks at Antithesis are working on getting Hegel [1] up and running, but they don't even have plans to support the language that my workplace prefers: C# (and I have been leading the charge in my workplace to keep up with the latest versions of .NET). Given how agents are now creating and modifying whole codebases, property tests live in that nice middle ground between unit tests and formal verification when it comes to validating (as best as we can) that the code works as intended.
Yes, the workflow matters. And it is one reason why Python is popular in the ETL and other processes. You are essentially offloading computation to efficient code while Python simply orchestrates. You probably spend one percent of the time inside Python itself, so it is pretty convenient.
I normally have one "tooling" language that I write all kinds of stuff in. And it goes beyond glue. So performance matters to me.
re: testing, I am pretty old-fashioned. I don't test everything. Only stuff that is critical or complex enough where I suspect breakage in the future. Also, I think borrowing ideas from Ada/Haskell/Clean etc for a new language could solve a lot of these pain points, now that LLMs are writing most of the code.
Pretty charitable to call Salesforce a bastion of "efficient code" hahaha
You're lucky to be able to control as much of the stack as you want with your one "tooling" language to rule them all so that performance matters again. Although in all fairness, given the regulatory and workplace culture as well as constrained budgets, sometimes you do have to let go of chasing performance in the name of making things easier for the maintainers at work. As I mentioned in a different post, it also helps to be working in a captive market, which helps relieve the pressure a bit.
Also, my Python ETL scripts do calculations for certain things, but attempting to speed that part up with a low level language like C or Rust doesn't really move the needle for total processing time when you have latencies measured in the hundreds of milliseconds per page of data. Scale that up to multiple pages and the whole runtime is dominated by how long it takes for packets to move from my Windows Server to Salesforce and back. It's basically a whole lot of effort for very little gain.
Work is JVM. So not much space there. But I did manage to incorporate a tiny LISP-like language into my principal application to make some configuration tasks easier. Everything outside it, I can control.
I get your usage. If you never spend enough time in slow mode, there is no point moving the code elsewhere as the savings do not justify the labor.
My issue is that I do not want to use or maintain code in 10 different languages. A single language, even one that is 3-5x slower than C would work. My only asks are that it should:
- be performant
- have basic infrastructure (cryptography, database etc)
I looked at it briefly, but did not use it. I actually made an effort to use PyPy, but the issues I mentioned probably ensure that it will always lag behind CPython as far as language support is concerned.
The situation is tricky and the friction is considerable. It is not like JS where v8 made the existing language blazing fast with no changes to the language itself.
Would rather use something like Nim with Python-like syntax.
>I joke that using Python on your modern 2020s CPU upgrades it to a Pentium 4 from 2004 running code compiled with C. And the 2004 version might still be faster.
It really doesn't for a lot of the things that matter, and people should seriously ignore your opinions if you're making these kind of fatuous jokes. Any serious person should understand what CPU bound tasks are vs what IO bound ones are. Python is used for the latter, with some C/C++/Rust bindings for the former. You are ignorant and tedious. When should I actually give a fuck? is the question that you types that build nothing never manage to answer. If you can improve some significant metric by 50x by moving away from Python, you should be fired for having been stupid enough to choose the wrong tool for the job.
> Any serious person should understand what CPU bound tasks are vs what IO bound ones are.
This is a decision fork only when you are stuck using a slow language. I don't have to do this --- at all --- when using C/Java and so many other languages.
Using Python as glue and writing extensions in C/Rust is a choice. But I'd rather write everything in C23 by that point. Or move to a faster language if syntax/aesthetics is an issue.
Further, you often do not have a choice on what platform you get to work with. Someone might have built an expensive-to-replace website using PHP. And then you have to build HipHop to speed it up.
He does not understand that it is just like a phone. Unless you keep it on standby (which he does not want to do: wastes power), it will boot, slowly, till all the subsystems are up and running. The low powered processors, the low speed ssds, and minimal RAM makes everything worse.
reply