> Rather than storing all attributes in a single JSON column, we automatically extract the most frequently accessed attributes into separate, strongly typed columns.
I'm glad they made the discovery but this is DB 101
I assume the author is talking about `fold`, as in `[A] -> B -> ((B,A) -> B) -> B`, and not what I often think of as reduce as `[A] -> ((A,A) -> A) -> A`.
`fold` is awesome and super useful. It's the easiest and most convenient way to turn a collection into a single value. Put me anecdotally in the opposite bucket.
I think part of the issue is that a lot of programming languages don't make a strong distinction between the two, and only provide the (more powerful) fold, but in a way that makes reduce operations harder to reason about (like OP said, with 0 types).
Associativity also makes fold hard. It's not super trivial to know when you might need e.g. left fold vs right fold
These are pretty close to each other, to the point where I wouldn't bother strongly distinguishing them.
Suppose we have foldr as in [A] -> B -> ((A, B) -> B) -> B, foldl as in [A] -> B -> ((B, A) -> B) -> B, and reduce as in [A] -> ((A, A) -> A) -> A.
Then we have foldr list value operator = reduce [\b -> operator a b | a <- list] (.), foldl list value operator = foldr (reverse list) value (flip operator), and in the case of a finite non-empty list and associative operator, we have reduce list operator = foldr (tail list) (head list) operator = foldl (init list) (last list) operator.
So these are all basically slight re-parametrizations of each other.
There are way more places where a simple typo will ruin you in a for loop than a reduce or fold or map. Using briefer abstractions in place of nested loops is almost always preferable.
Nah. It involves multiple passes and setting the answer to the wrong value before (hopefully) setting it to the right value.
Plus it forces you out of whatever lazy/streaming paradigm you had going on. If your foldr produces a list, downstream can start consuming it in constant memory as long as you let it do its thing.
fold kinda does too, for setting the first combined value that you are assembling, and thus on an empty list you end up with that wrong value, same as the for loop
If your data should not be successfully folded if it's empty, you should've already parsed it as an Optional nonEmptyList instead of letting illegal states fly around for a while in your application.
Yeah that's good application design, but those concerns aren't so relevant to the person writing the standard library for a language. You can certainly include a specialized version of reduce for nonEmptyLists which just returns A, but that doesn't change the fact that you have to return an Optional for a normal possibly-empty iterator if you want your reduce function to be non-partial.
For loops are not easier or more convenient than fold.
fold sum 0 collection
versus
acc = 0
for x in collection:
acc = acc + x
or the even worse
int acc = 0;
for(int x = 0; x < collection.length; ++x) {
acc += collection[x];
}
You can read one line and know exactly what's happening in the fold example. In the Python and C++ examples, you have to scan more lines and there's way more opportunity for typos.
A for loop gives you better memory management and speed, but the tradeoff only makes sense to me if you're doing embedded work or something. Otherwise, eat the .000000000001% speed loss to reduce the risk of logic errors, typos, etc. and to improve developer ergonomics.
Only real difference is that `fold` is denser. Both require prior knowledge to understand in their respective paradigms.
Adding numbers like this is not common in real world code. Now let's say instead of adding x, you have too look up X in a cache with an additional "type" param and update a metric of cache hits (or misses). You have to define a free function to keep your fold readable and understandable. In for loop it's much easier to understand.
If you mean the tables in the https://eddie.codes/posts/pandas-should-go-extinct/#i-have-b... section, it looks like those came straight from the Amazon paper, in which case __eddie__ shouldn't fix them, on general "quote accurately even if the quoted document has mistakes in it" principles. (Though a [sic] might help).
> Kunreisiki is also better for serious Japanese learners as it greatly simplifies the description of Japanese grammar: cf. matu/matte/matimasu (all with the same verb stem mat-) vs. Hepburn matsu/matte/machimasu (with 3 different verb stems)."
This is a silly argument because anyone learning Japanese is going to need to learn how to pronounce it anyway. Also "serious Japanese learners" would likely be using kana.
Absolutely agree. Romaji was absolutely pointless for me and probably a detriment overall to learning a language. Japanese uses consonant + vowel pairs as a rule, and the faster you train your intuition for this, the better.
Remembering both Hiragana and Katakana took me no more than a day each. Although I have to admit that I already tried learning them around 5 years ago and gave up back then out of lack of motivation.
The pair is something of a simplification, and romaji makes certain things in the language more obvious. Like the good old mat-u -> mat-anai, mat-imasu, mat-eru. Added the hyphen to make it clearer, but this hyphen splits a kana neatly in half. The stem is “mat” but you can’t write the stem using kana (in some sense of the word ‘stem’, maybe that’s not the correct word and there’s a different word for it, I’m not a linguist).
I’m not trying to say you should avoid kana, just that there is some amount of intuition you can develop with romaji that does not work in kana.
The traditional way to develop a similar intuition with a kana-only approach is to think of it as "moving across the row" in the kana table, which works fine. I think the fact that with romaji we can represent the stem as mat- is interesting and valuable in a theoretical sense but not all that useful to a learner.
I have trouble comprehending the idea that this is only interesting in a theoretical sense but not all that useful to a learner. Could you give me something additional here? I’m not trying to be a dick here but my first reaction is “that’s just nuts” and I want to know what I’m missing.
The reason I say it's not useful (as opposed to merely interesting) to a learner is that I think if you're ever explicitly forming a structure like "mat-" in your head while thinking about how to say something or how to interpret something in Japanese, you are doing something quite unnatural, since that form cannot actually be expressed in the language. A fluent or native speaker knows that, eg. to make "matsu" a polite order, the tsu (tu) becomes chi (ti) and then you add nasai; they are not thinking about the isolated 't' consonant (which in this example is not actually even the same consonant - and native speakers know that, considering they distinguish ティ in loanwords!).
As for why it's interesting and useful theoretically - well, Japanese is of course older than the kana chart, and if you're doing diachronic linguistics I think taking mat- to be the stem is probably a useful frame to think about the origins of the modern conjugation paradigm.
It is equally unnatural to form kana in your head when thinking about how to say something! Or, it’s unnatural to think about whether a verb is godan or ichidan when speaking, it’s unnatural to think about what particles are, etc.
You get an explanation for a concept and you internalize it. The explanation is a framework for you to internalize the concept, and once the concept is internalized, you no longer need it. The explanations are still useful, like romaji is still useful. None of the explanations are necessary, some are useful.
Maybe you get a few different partially wrong explanations of what the “wa” particle means before you internalize it. You’re not getting damaged by those explanations, they’re Wittgenstein’s ladder. (Well, maybe you got some bad explanations, but even the good explanations are wrong.)
You throw a bunch of different explanations at different students, combine it with practice and lots of examples, and eventually it sticks.
> which in this example is not actually even the same consonant
By that standard, the “t” in “butter” is not the same consonant as the “t” in “tap” (try it!)
People are really good at understanding and learning phonology and can really internalize it and understand it on an intuitive level, even if they can’t explain the rules for it. If you’re a native English speaker, you accept that there’s a “t” in “butter” and a “t” in “tap” even though they are not the same sounds (phones). That’s because they are the same phonemes.
Romaji is an effective way of writing down Japanese phonemes.
You don’t need to visually split the consonant apart as a separate character to recognize the relationship between か/き/く/け/こ - after all, the Japanese manage to do it without Latin letters. It adds very little to do so except as a crutch to let a learner continue to think using their native orthography, which only impedes learning. And given what a small hurdle it is, especially compared to the multitude of enormous hurdles the language has in store for you, you’re better off just jumping over it and moving on to those more interesting ones.
- “Need to” is a very different criterion from “is useful”, and “visually split” is, well, kind of unexpected to me!
- The comparison to native Japanese speakers seems especially irrelevant here,
- You’re not using your native orthography here, you’re using romaji, I don’t think it’s anyone’s native orthography.
- You probably want to use the Romaji heavily anyway, because that’s how most people will use computers.
When you’re learning a language you naturally multiple approaches to understanding a single topic. You’ll see a word written, you’ll hear it. You’ll hear a sentence and then someone will give you an explanation of the grammar, and you’ll compare that with your notes. It’s multimodal and all of these different modes are useful.
Like how romaji is useful for understanding conjugations. Obviously not necessary, just useful.
Have you considered that different things are difficult for different people? Learning hiragana and katakana has taken me about a month, and I found it very difficult. There is no need to make it prerequisite for starting to learn grammar, forming sentences, or pronunciation.
Notice how I didn't say that I "learned" both in a day, I "remembered" them. Of course actual learning will take some time, especially with less used characters.
But remembering shapes and differentiating between them using mnemonics should not be too hard for most people.
> Also "serious Japanese learners" would likely be using kana.
I think the exception is people using the JSL textbooks :) there is a reasonable defense to using JSL. But you’re right. I think it’s indefensible to teach Hepburn to people interested in learning Japanese.
This is speculation right now. The idea would be that if someone used the product and granted training rights, which is the default for many subscription levels, then some knowledge would have been imparted into the general weights of the new model.
oAI has made clear they did not specifically pull in any user data to context for this run.
Tristan Buckmaster’s post cited extensive use of LLMs in the process of his collaboration with Levent:
“We used several LLMs throughout: Anthropic’s Claude, OpenAI’s Codex, especially with GPT-5.6 Sol and, more recently, Astra.
The latter was only used for writeups and auditing our arguments. For most of the past year progress was slow. We worked through the literature and upgraded various preliminary results, up to obtaining finite time blow up for the Incompressible Porous Media equation (with smooth forcing).
This was until about a month ago, when we had real progress: on August 15th, we obtained the blow up results, with smooth forcing, for both Boussinesq and Euler. I can say the first LLM generated proof Levent sent me was the most horrendous I have ever read; we verified it on Lean on August 22nd. Since this point, we have been working around the clock to understand this proof and turn it into something readable.”
The OpenAI research post states they began training GPT-6 internally on August 28th, and that user chats are used to train models.
“We (the researchers and the agents) did not see any of their work through any means until they released it publicly — in particular, no specific user data was accessed in order to solve this problem. While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models .”
For an incredibly niche topic like this, I believe it’s extremely likely that Buckmaster/Levant’s work would influence the direction of OpenAI’s agents’ work even as a de-identified drop in the overall bucket of training data.
Yep, it's possible. But, we have literally no idea how much a set of prompts would impact training as far as general usefulness. I don't think we even know if Tristan's said he allowed training on his prompting or not. This is about money, ego, primacy, all the usual mathematician priority disputes.
Shenanigans is an inaccurate word; it implies underhanded behavior that's hidden / concealed. I think "to act so aggressively" is more balanced.
Here's my answer: If you think we're getting to AGI in the next 9 months, then you believe, with all your heart, that these problems will fall soon. However, there's an ocean to boil in terms of what you could point your limited clusters at. In the meantime, the market is desperate for any sign your company might be first to AGI. Therefore, news of tractability with current models might focus an organization intensely - internally they have a huge leg up on the public, and therefore it's minimal compute to check - and if they are successful, they get approximately $50 million of free PR, likely adding 10-20% to their valuation.
Likewise someone like Tristan is fighting for his (metaphorical) life right now, hoping to preserve his claims of primacy and have a shot at some of that prize money, despite being only partway to a full solution for N-S.
I don't think we see any behavior at all that isn't simple to understand and well described by the setup here, but tell me what you see differently.
OpenAI most certainly did not as you put it, "to act so aggressively". They have simply indulged in plagiarism/malpractice/fraud all for the sake of pumping up their evaluation in light of their forthcoming IPO and to one-up their arch-rival Anthropic and try to get themselves to AGI certification.
In the process they have shafted real-world hardworking mathematicians, which is to say the least, despicable. Note that stories are now coming out from other mathematicians who have also been shafted in a similar manner. Also there are cases where they have had mathematicians accept their "deal" (like the one they offered Buckmaster that he refused) and have OpenAI name linked to their work.
Regarding "their proof", their claim is only solving Navier-Stokes partially for when a smooth force is applied and not a fully general solution (which is maybe impossible). The proof is still being verified and we don't know whether it is just an approximation/hallucination or not.
Their most blatant lie is that they "gave only the problem statement" to their system which then went ahead and solved it. This is almost an impossibility. Problems like these need to identify a specific lead/approach and some work to be done on that path before you can even know whether that approach is promising and worth pursuing. This problem has resisted all attempts at solution for over two centuries. This is where the Buckmaster/Levent's work's importance comes in. They identified a promising approach based on other mathematicians work and have been using both OpenAI and Anthropic's models to make progress and had reached a promising milestone which they published. But they inadvertently gave away their approach to the model's training data set which OpenAI capitalized on by throwing a large amount of compute at the problem to get to the finish first.
This is straightforward stealing of other people's work and building upon it to claim it as your own which can and should be sued. All Scientists/Mathematicians/Researchers who feel OpenAI has done them dirty should band together and file suit.
This whole thing could easily have been avoided if they had worked with the researchers so everybody's concerns/needs are met.
By engaging in this sort of backstabbing, OpenAI has effectively killed the "Goose that laid the Golden Eggs". viz. Researchers were giving away their hard-earned highly specialized knowledge freely to the models in the hope that it will help them get quicker to the result. But now everybody is going to lockdown their research findings and will stop sharing it with the models to the overall detriment of advancement of Science.
“We can say categorically that it is impossible for Dr. Buckmaster’s Codex prompts over the last two months to have influenced the system in any way, including training.”
“After investigating, we can say with full confidence that no user inputs past July 3rd could have influenced this system in any way.”
Buckmaster reached the key result on August 15, far after the training cutoff.
It wouldn't surprise me that even if you opt-out they still train on 'de-identified' chat data. After all, facts cannot be copyrighted and one cannot be sued for unethical behaviour.
I'm glad they made the discovery but this is DB 101
reply