> the models generalize well only on tasks within a small neighborhood of the specific tasks they've been trained on
Unless frontier labs have surprisingly trained their models in the exact tasks my team works on, this is patently false. We are getting very good results on automation and I'm bullish we will be able to mostly remove humans in the loop for most of our infra tasks by the end of the year.
I have no opinion on the other theses, but given that OP doesn't back up these claims in any way, I have my doubts about the conclusions of this article.
Funnily enough this very morning I asked an LLM to implement an algorithm for an NP-hard problem (a variation of the knapsack problem). I gave it 2 directives:
* Do not implement an np solution trying to get the perfect score. Implement a fast solution that gets within x% of optimal
* If a solution seems impossible or it takes too long, return the closes solution you can find, and a warning about the solution being suboptimal
I got the code in a few minutes. On a sample of random inputs, the algorithm produces a solution within 1% of optimal in ~99.9% of the cases. p95 execution time is well below 2ms in my laptop.
That's it, that's everything you need for a production system. "close enough" very fast is sufficient, and the impossible cases very rarely happen. Even when they do, you can simply work around them.
My last job search 2 years ago took me 9 months of grinding while working full time at Big Tech and +15 years of experience. After loops with ~30 companies I ended up with about 6 offers.
The offers were all over the shop. Some places downleveled me to mid level while others upleveled me. The comp range was about 3x between the lowest and the highest.
Also I couldn't find much of a pattern on the processes and outcomes. No name companies rejected my application without even a phone screen. Lower tier companies offered me lower levels while famously hard to crack ones offered me higher levels. I also didn't see much correlation on how hard interviews were.
I'm totally convinced the job market is a numbers game and there is a high degree of randomness and luck involved. Discipline and hard work help, but outcomes are far from guaranteed. Last time I almost didn't leave my job since most offers were worse than my then current one, until I got 2 high quality offers at the end.
I have only skimmed through the article but I'm impressed OP got a job as a Research Engineer with no formal background, given that I got doors slammed even after PhD + years of relevant experience. I don't want to diminish the accomplishment, but I would be very wary of anecdata and formulas for success. OP quit his job and focused 100% on the search, and it paid off. I know many more similar stories that didn't pan out, and people had to rush to get lower quality jobs or, in the worst cases, got into long term unemployment.
> the models generalize well only on tasks within a small neighborhood of the specific tasks they've been trained on
Unless frontier labs have surprisingly trained their models in the exact tasks my team works on, this is patently false. We are getting very good results on automation and I'm bullish we will be able to mostly remove humans in the loop for most of our infra tasks by the end of the year.
I have no opinion on the other theses, but given that OP doesn't back up these claims in any way, I have my doubts about the conclusions of this article.