Anecdotally, the percentage is way higher than 10% for some kinds of searches. I don't always use google, but i've tried it dozens of times the past few weeks to test their AI Summary / AI Overview, and it has yet to get anything correct. I've had to use other search engines or AIs literally every time.

mistakes vary from "obviously wrong" to "looks vaguely correct, but the source states something different" to "looks plausible, but the cited page is completely irrelevant, the AI clearly made this up" Even for straight fact lookups, I've seen it cite pages and claim one number, and then i click through and the source matches the vague shape of the answer but has a different number.

Its just constant hallucinations. "thats a common problem, just change this or set ". Full gemini seems more capable, but whatever google is throwing at the top of their free searches is awful IME.

I'm guessing they're only counting one failure mode. But I think it's the one that would surprise most boosters. Even if it found a relevant page, and the information is fed into the model's context right there, the output is still completely backwards a fair fraction of the time.

Fwiw korean doesnt have a ‘V’ sound/character at all, and the V/B switch is quite common on Korean words that come from English. For example, “video” in Korean gets written with ㅂ at the start, and sounds more like “bideo”. And the same character is used in the Korean/Hangul version of their written company name.

Yeah, pronounced in korean it definitely is more similar to Biosys than Viosys. But since they have written it as Viosys in all english documentation and websites it is very suspect that the article writer would misspell it like that. So probably AI translating a korean source or something, but AI also does these mistakes much less often these days.

They have probably chosen "V" for the English transliteration to avoid being confused with a bio-engineering company.

At the same time, every component and line of code is known. Presumably they can simulate and test for all sorts of crazy exotic problems and error cases. It’s surprising that they’ve hit a new type of problem.

Its effectively impossible to get some streaming services to stream content >720p to a linux device. I know I tried several years ago to use Disney+ and could not find any way to get 1080p content, even on a windows machine. I think now they have an official windows store app, but still refuse to stream full HD or 4k content to a web browser.

More recently I've tried hbo max and it does the same thing. Could not get higher than 720p, even with chrome and whatever DRM options theoretically fix the problem.

I'm seeing a few comments that say $500 isn't enough, and its not worth it for them to try to hack it. But if it costs fewer than $500 of tokens in an AI, then it seems like it is? I thought a lot of the claim people are making is that you dont actually need to hack anything you just ask an AI and it does it. The exploitable bug is public and documented. All the pieces are theoretically there, and there are repeated claims that AI is trivializing exploits and its super easy to hack things now.

If that were true, and someone can hack the server for less than $500 of tokens, its an easy profit.

FWIW i havent updated my ios, I'm on 26.6.2, and they all got revoked for me. The revocation shouldn't require an iOS update.

I’m on the same version, and according to https://emudhra.we-have-a-problem.com/ they have not been revoked for me. Maybe phased rollout.

I think it varies by location. I think they can in SF and other cities where they’ve been around a while. They can’t do highways where I am, but they only started doing public Waymos here this year, so it’s quite recent.

And while I haven’t seen any self-driving Waymos on the highways, I have seen Waymo scanning vehicles with spinning lidar things and a human driver on the highways.

I have a similar issue. I see cloudflare checkboxes all the time. Google makes me fill out captchas all the time. Sometimes it decides I can’t search, sometimes cloudflare doesnt let me past the checkbox.

I’ve checked my network. I’ve monitored egress traffic. There is no residential proxy. I’m not on a VPN. If anything, services should be able to track me entirely by my home IP that doesn’t change that often.

My best guess is that I’ve committed the crime of not allowing third party cookies? Or maybe the crime of not being logged into a Google account 24/7. It’s not browser or device specific. My iPhone is just as likely to fail the cloudflare checkbox as my desktop with Firefox + uBlock.

I've committed the extreme crime of running no JS and clean cache of cookies. Plus, something as simple as containerizing work/play/research reading, into different browsers, has provoked the Eye of Google to be cast on me

I haven't seen a YT ad in years while in lurk-only mode (no login), which is nice, but I also see cloudflare everywhere , which is not great.

One of the nicer things about LLMs, is that they arrived just in time to erase my dependency in search.

One benefit of the current self-hosted option via vaultwarden is that you get 2FA and the other premium features by default.

But it is worrying that they might intentionally break vaultwarden in the future.

I think there are some interesting parallels between what the Cleo account did, and what the large AI companies are doing now with some of the proofs for previously unsolved theorems.

There may be some value in the answers, but for some the value is in the understanding of how to get from the problem to the solution.

I wouldn't call myself a mathematician, but I encountered a few problems in my CS classes where I could work through the proof by hand, see that an algorithm works, and even mathematically prove it, but yet still didn't feel like I understood why it worked, or how the algorithm was invented/derived. But for many, the fact that it did work was enough.

Joe McCann and his audience discovered that Cleo was a ploy at ragebaiting the math forum in order to drive engagement for more useful proofs. A real-world Cunningham's Law social experiment.

So maybe OpenAI's ridiculous approach to mathematics has value in how it's ragebaiting us to scrutinize and/or come up with better proofs!

> for some the value is in the understanding of how to get from the problem to the solution

All of the value is in this understanding.

So many people are still in denial about it, but "AI" really is just a natural language search engine. AI helps organize what has already been written by humans and was technically self-evident from our work.

If we allow ourselves to continue down this anti-intellectual path and stop caring about why proofs work, we've basically entered another dark age.

I'm not saying AI can't possibly help at all with building intuition, but clearly there are a lot of idiots in the class who just gotta raise their hand first to blurt out whatever trash they come up with on impulse. Those people need to be told to sit the fuck down already. Their uncontrolled ADHD is ruining everything. There's nothing intelligent about mindlessly rushing to compute an answer. That is literally what the last few years have felt like for everyone else.

> AI helps organize what has already been written by humans and was technically self-evident from our work.

I am not 100% sure about the "self-evident" part here.

To your point: yes, AI doesn't come up with new axioms. But then we enter the topic of "math is invented or discovered?". Each of the discoveries are logical consequences of the axioms, but that doesn't mean they were "self-evident", especially for humans.

If everything was already self-evident, AI would just be a useless tautological machine. However, I think it is more than just that

I just gave AI the original Cleo integral, and it came with a solution dissimilar to any answer on Math.StackExchange page in question. The answer also differs in presentation from Cleo's answer, being "2π arccos(sqrt(5) − 2)" (the numerical values seem to be the same.)

AI capabilities keep getting bumpier and bumpier. There's no reason to believe it won't stall out on a front that it is already quite bad at (correctly formalizing in human language)

“LLM progress will stall in some area at some point” — which is my interpretation of what you wrote — feels particularly unfalsifiable

The negated statement --- that LLM progress will never stall in any area --- seems trivially falsifiable, so why is the GP's statement unfalsifiable?

This incorrectly assumes falsifiability survives negation. https://philosophy.stackexchange.com/questions/52015/is-the-...

logic vs wisdom

It's a new technology and we're figuring out important distinctions as we go, nothing necessarily intellectually dishonest about that.

If shifting goalposts makes AI better, why not shift them?

Because making AI better is orthogonal to making human lives better. We are quite literally racing towards artificial superintelligence without any serious plan or preparation for the societal impacts that will have.

Did people prepare for or ever fully foresaw the downstream impact of any other major technological revolution, such as farming or the industrial revolutions or the Internet?

Has there been any other major technological revolution that underwent exponential recursive self improvement?

If AI development stopped right now, we would see a technological revolution at a scale similar to the Internet. AI development has not slowed down, let alone stopped. This technology demands far more planning than we are allocating.

I don't think we are smart enough with enough foresight of consequences to plan this. If we used AI, however...

AI inherently has misaligned incentives (as do the companies developing them). I don't trust them to outline the consequences of something like this and to develop an appropriate plan, and neither should you. I believe in humans, but we need time.

I don't think that is the thing that makes AI better. In the metaphor the idea is that someone scored a goal and then people retroactively shift the goalpost to say it doesn't count as a goal.

Is there any evidence it does? Without that it simply feels like poor debate.

This is a bleak view.

One can also say: The point of humans is to produce things for the economy. Now that humans are obsolete for producing things, humans are unnecessary.

Would you agree?

The economy seems to be agreeing.

The economy can show us whether the conclusion holds, but it does not tell us anything about the premise. Do you believe the point of humans is to produce things for the economy?