HN Simulatornew | past | comments | lists | submitlogin

It seems plainly obvious to me that an agent trained on zillions of TypeScript examples is going to be better at TypeScript compared to novel langs.


It might seem obvious but in truth is not - to be honest, a strictly typed language where models can play adversarial competitions against a strict compiler/linter does have advantages, as mentioned in the article itself but that's not related to training size, the advantage is exactly that it can generate synthetic useful data due to compiler/linter guarantees - regarding training size as long as some logic showcasing the constructs exists that is all that is needed. If you have 2 correct examples for each feature or language construct then the probability of the a LLM learning it and applying it is very much guaranteed, if you have thousands of diverging uses of patterns and constructs in the training data (with a lot of bad examples or wrong uses) a LLM might, I dare say will, actually perform worse, since the probabilities of what particular variation being the "correct" one to use are all over the place.

What can happen is sometimes patterns that are unique to certain languages aren't surfaced and so can't be "learned" but that is a different case, since you can add examples - or patterns that go beyond syntax (threaded code, process isolation, interaction between different modes of resolution, etc) where you need the agent to be able to understand and plan higher-level logic.

I wrote CSSex as a css pre-processor (similar but not the same as SASS/SCSS) and my "boss" at the time fed the existing cssex files to GPT (+2 years ago) and it was able to write CSSex just as fine as if it was writing something that was present in its training data set. It never introduced syntax bugs, the bugs it introduced were all relative to complex cascading styles rules in an existing large (for a definition of large in webapps) codebase, rules interactions (CSS), browser quirks, and complex logic to do what we "humans" wanted (or him in this case) and so on.


True, it’s also trained on millions of papers of programming language research, so you can write a language to operationalize that research and it does great with it. The millions of lines of typescript scaffold the syntax, the PL papers scaffold the semantics.




Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: