Google published a lot. I still wonder this original transformer paper, and the attention, etc. Why would they allow it to let go in the open? Perhaps because was intended for translation first before someone decided to loop it over itself? Or was so obscure to fellow researchers what do they actually publish?

It took 6 years to get from Attention Is All You Need to a consumer product. Up until ChatGPT’s release, LLMs were obscure and really were just fancier autocomplete. Deepmind was largely a speculative research division until Google decided to play catch up, and even that took a bit of time as they figured out how to proceed in a way that wouldn’t cannabalize search. So in short they didn’t realize the potential. Thats my somewhat naive take.

Google itself was shipping transformer-based features to consumers within 2 years, notably BERT for websearch.

and Google Translate: https://arxiv.org/abs/1804.09849

Groundbreaking includes the “explain to the world why it matters” part

databricks is a top ai startup?. i thought they did spark hosting or something.

what makes them a top ai startup

$160 billion valuation and market capture among some top (non-AI, traditional enterprise) companies, I think.

Yeah, but that capture is usually just hosted spark usage.

The AI usage has mostly been getting it to explain error messages

they also have a pretty handy button right next to the error to ftfy

They just raised their series Z or something lol.

weird, for so long I'm damn sure that researchers are obligatory to release research papers to fulfill their quotas. otherwise they will lost their titles.

so unless those companies (openai, anthropic) gave the researchers some fortune big enough to make them give up their research careers, they will keep publish research papers. aka it is not the good will of those companies.