But you’ve arrived at the ER and the attending staff are oversubscribed, so you have interns and residents primarily trying to keep you alive. Sure, if you are a VIP you’ll get extra special care, but most of us aren’t.
Most ER work is urgent but familiar even to junior staff, and a lot of it is not life-threatening either. If they need to 'keep you alive' then they are following established protocols, not asking an LLM for advice.
I'm implementing a similar process right now (calling it cleanroom for now):
Given a specification as a source of truth:
1. the lib agent can write lib code, but cannot look at test code.
2. the test agent can write test code, but cannot look at lib code.
3. the QA auditor can run tests and can look at lib and test code, but cannot modify either. It determines blame and then delivers feedback to lib or test agent in a way that doesn't reveal details about the code they are not allowed to look at.
You spend more tokens using this process (since multiple agents have to read the spec and similar files), BUT you get better results. The project is here:
Sweden is not grid connected to Texas, well, nor is also any other grid in the USA actually. Smelting was placed near cheap hydro resources (and why Boeing built planes in Seattle) for good reasons. Just replace cheap watts with clean watts.
Because Sweden's clean energy capacity is fixed. Which means a "clean" Watt spent on inference is taking away a "clean" Watt that would have been spent on heating or steel. Which means Sweden needs to buy energy or energy intensive goods on the global market, which raises prices, which means Texas is being paid.
This is a weird assumption to make. Why would you assume they wouldn't just build out more capacity as demand increases?
They certainly aren't buying electricity from Texas, this feels like a tortured argument you're just making to retroactively support whatever opinions you have on the pros or cons of clean energy.
There's only so much hydro power to build out, and nuclear takes forever. Solar only works in summer that far north, and without insane batteries wind power just becomes greenwashed natgas.
No, electricity is categorically not. Exhibit A for this is persistent regional variation in unlevered pre-tax prices. Even in well-integrated power markets, only a fraction of electricity is listed, and that's before we get to off-grid power.
You can’t ship and store electricity without inefficiency and capital costs. They can only smelt so much aluminum and they are exporting that also, so there are costs associated with that. In comparison, an AI result just needs a network connection to send.
> you can’t ship and store electricity without inefficiency and capital costs
This per se doesn't make something non-traceable. Plenty of perishables operate on a global market.
Power being hyper-perishable, hyper-regulated and opaque (lots of off-grid and unlisted-on-exchanges power) sort of destroys the international power market. (Regionally, power markets are also pretty efficient in most parts of the world, at least in the short term. That makes profiting with a ship of capacitor carriers hard if not impossible.)
Maybe to encourage longer stays? The cleaning fee is paid once per stay, not once per night. I get why it is a separate charge, since they aren't cleaning the place daily (like a hotel), only at the end of your stay.
That MoE model is fast. 30B A3B gets 140 tokens/second on my M3 Max, and…the A3B part means a reasonable KV cache size and reasonable prefill times. 3.8 27B is a dense model that is mostly unusable, 40 tokens/second using MTP sure, but any cache miss is a few minutes of prefill. You are better off going with a lobotomized 3.8 flash MoE for the same amount of memory.
The report is still bizarre, but that model is still very relevant to local users.
AR glasses plus an LLM would be great for DIY maintenance, renovation tasks. Plumbing, electrical work, kitchen renovation, etc… often times it’s just knowing what to look for or check.
I have a cousin who has a certain fearlessness when it comes to home DIY projects. Unfortunately he could probably use a little more restraint, but on the whole it's an interesting quality that I noticed LLMs/coding agents help with quite a bit. They give me, an impostor syndrome type/overthinking type a certain confidence to move forward that really helps break into projects. I think this could probably be very effective in other domains like home improvement. And in case of my cousin, might also help avoid some missteps.
Yeah, this would be a much better version of the well-meaning but poorly produced youtube videos I typically use to fix things up around the house. Probably a lot of value there.
I wouldn’t know since I haven’t found a model to test it on, but in my experience with smaller MoE models (35b A3b), they are the only choice for my M3 Max because prefill times are reasonable (I don’t even need to quant my KV cache, which would mess up MTP anyways). The only reason dense models (3.8 27b) are unusable for me are the high prefill times (mitigated if prefix caching is working correctly).
Is it really possible to get determinism while maintaining good GPU utilization? Parallelism alone introduces enough non determinism that I only see this really working on maybe a CPU?
reply