We can't rule out different architecures, but absent some architecure or hardware or quantisization breakthrough, frontier models will simply not be fitting on phones.
Right? Occasionally, I would think about how the concept of the singularity or recursive self improvement was a bit incongruous today. It was made when people expected generally intelligent machines to be these logic automatons that could 'alter their own code'. It seemed to me that this path would be closed off with the advent of Giant Neural Networks that take weeks/months to train. That GPT-X would 'improve itself' by designing a better/more efficient NN architecture. I didn't write it off completely but that just seemed a LOT more likely than altering weights directly for much the same reasons you describe. Maybe not lol.
This is pretty massive, and a result I (and I think most?) would have never expected. Holy Shit. How?? Did OpenAI figure out a way to train this? Are they even aware?
It's probably emergent from training on other ML tasks and general ML/mathematical related reasoning rather than specifically trained for to be honest. Otherwise it would probably have been in an official marketing communication or something.
For A, a common misconception is that models write out lean proofs from the start. They write/develop the Natural Language proofs, and then translate (sometimes not even the same model) to lean afterwards.
Today with Opus, Sol, etc there's basically no reason to visit the site, but when chatGPT launched it was powered by GPT 3.5, which would realistically only be able to answer a (minor) chunk of all possible StackOverflow questions.
There are many potential explanations, including the policy changes in 2018. LLMs are definitely a concurrence to the service, but since the decline started years before they were really useful, other reasons might be relevant. I don't know to what degree. I just wondered why some of the forums had significant peaks after the GPT line.
>the problem is that the Python community at large rather prefers their C bindings or nowadays Rust, sprinkle a bit of Python FFI on top, and call them "Python" libraries.
They are Python libraries. Don't make the mistake of screwing up the cause and effect here. C, Rust etc - those are the replaceable parts of the chain. There is an official C++ pytorch library and virtually nobody uses it for ML Research. People want to code in python and build those libraries so they can.
I know what bindings are. A front-end to C++ for pytorch was only released from 1.0, literally 2 years later. The relationship is very clear here. There was no native C++ library. It was an afterthought. Even in instances where there was, like Tensorflow, functionally it was still an afterthought.
They are python libraries because people reached for those languages so they could continue work in python. They are there to serve python and nothing else. It's the same with Rust. It's by no means a C++ thing.
EDIT: Not saying this is always the case. Qt>Pyside is an example where it's both technically and functionally bindings, but the vast majority of the time they are serving python. Some don't even have a native library to speak of.
Are you arguing that JIT Python or GraalPy will be as performant in a reasonable time as Python on top of those C++/Rust libraries? If not then I don't think it matters since unless you are claiming that all the existing C++/Rust libraries can be re-written in reasonable time and with the same performance. Keep in mind that many of these libraries are using low level tricks to optimize performance. Python is the UI and while the UI is important in the end it's just the UI.
Results come from an internal model that has not been publicly released. And there are lesser problems and conjectures ordinary folks have solved with Astra/Sol.
Yeah. People have very strange ideas about competition and choice and moat. If all you needed to unseat the entrenched was a competitor then Bing would be going toe to toe with Google, and Edge with Chrome. They have a billion+ devices they can stick those options as defaults and still nothing. There are many examples.
People are naturally sticky. If they're satisfied with a product, they'll stick around unless something comes along that is so much better it gets people talking. Open AI's 1B+ customers won't give a shit about Chinese models. Enterprise customers won't move to Chinese models, especially just to save money. They are a non issue, especially if the M.O is just going to be playing catch-up.
You're wrong. Are you sticky about your search engine? Now try having your search engine charge you $100 a month and tell me you'd stick with google search. Open AI's 1B+ customers will give quite the shit about cost if chatgpt stops being free.
Because I might look elsewhere if they starting charging several orders of magnitudes more then I'm not sticky. Does that remotely make any sense to you ?
They're sticky, but not for everything. There are common goods where the market has commoditized greatly because customers are conscious of quality and not provenance. Something like apples.
Most people buy their apples from the same grocery store every week. They're not shopping around every month to find who has the cheapest apples or 'best' apples. They go where they're accustomed to going, and as long as price and quality are satisfactory, they have little reason to change. Walmart, price-sensitive as they are, have relationships with fruit suppliers they've maintained for well on a decade. Even in commoditized markets, people are sticky.
Google, the titan mega-corp with more money than they know what to do with and a billion+ android devices to ship gemini on and it's made no dent for ChatGPT's consumer growth and adoption. Yet some plucky open source chinese models will?. It takes more than that.
Yes it is. Tiktok's content and content recommendation algorithms are much better than Instagram. Users spend more time on Tiktok than they do Instagram and it's not close. Tiktok's engagement rates are several times better than Instagram. Reels did nothing to dent it (yet another example in the book). Again, nobody will care about Chinese models if they're just more of the same. They can take a piece of the pie, but this idea that they'll spell doom for the frontier labs is nonsense.
reply