I got his book Build Your Own LLM From Scratch and it's super readable and does a great job at holding your hand through the entire process...need to finish reading it though!
I did the same. I got "Build Your Own LLM From Scratch" last christmas and it is a great resource for learning how LLMs work, from tokenization to generating text. The author makes a great job explaining each new concept. If you know PyTorch (or at least know about tensors), the material will be approachable for you.
Sebastian, if you are reading this, this is an idea for another book: how to orchestrate agents for adversarial code review. I've been looking for learning resources to make something similar to this [1] at a smaller scale. I am aware of codex subagents, though. However, given that Sebastian already has a book for LLMs and another book on planning, I think adversarial code review is an interesting next step.
I don't like working with neural networks and I always found everything overcomplicated or oversimplified, sometimes both at the same time.
But this book is almost perfect. Every new concept is well described and every is well founded. And every step moves you forward (if you know, you know)
I like it too. Haven't got very far through it yet but just learning about how to use PyTorch and finally getting how back propagation works via autograd etc. was really fun and enlightening.
Does selecting inputs work? That's all I need from a TV. If anything, needing to sign in to use the native interface would be a bonus, because it'd prevent non-technical family members from accidently using it. "Babe, we have Netflix on [the dongle], why'd you sign in on the TV?" is a conversation I've had more than once.
Looking for folks working with loyalty points & credit management
Been building a platform around this, have one customer who has been happily using my solutions for 5 years now. Seeking to bring others onto the platform.
Email in profile. USA East coast based, but willing to work worldwide
Am I the only one that doesn't really feel a difference in performance from model to model?
From around GPT 4 results got "Good enough"...I generally try to explain what problem I'm trying to solve, set limitations and boundaries, tell it to ask me questions, have it write up a plan with steps then we take one step at a time.
These new models are starting to feel like iPhone releases where the improvements / feature set feels incremental.
We have a junior research student who does this on slack.
Before I think he was using AI and other tools to translate his messages since english isn't his first language. His own words, then AI translated it. Sounded kinda clunky, but I could sense the human behind the words and I gave grace since I can only imagine how hard it is to properly communicate your ideas when English isn't your first language.
But now it's gotten to a point where I can tell he's not using it for just translation. I'm gonna need to chat with him.
Redacted example below
```Hey @PERSON_WHO_ASKED_QUESTION Both good, and the retrieval one isn't written down anywhere. In order.
One corpus or four. My lean is one. Same chunks and embeddings tables, source_type column to tell them apart.
Values I'd propose, flat rather than nested: paper, dataset_description, dataset_readme, dataset_contributors, dataset_records, dataset_files. Description and readme split because their units already differ, one row versus one row per paragraph. A discriminator that can't separate those isn't doing much. The existing 440 rows would need backfilling to paper.
Chunk id in the same spirit: dataset doi, source type, ord. So EXAMPLE_DOI.
Worth checking before any of this matters: does chunks.doi carry a foreign key to papers.doi in 0001_init.sql? If it does, a metadata chunk with a dataset doi can't go in that table at all, and separate storage stops being a choice. Ten second read, I haven't done it. Shout if you get there first.
Retrieval is the one that's bigger than it looks. Search once across everything and nothing guarantees a paper chunk and a metadata chunk both land in the top k. The facts we want relate the two, and the generator can only write those if it sees both sides in the same window. So if one type systematically wins the ranking, that class of fact doesn't get worse. It becomes impossible.
Which way it goes I don't know. Two mechanisms pull opposite ways. Metadata chunks are short, tens of tokens against roughly 450 for a paper chunk, so they may just lose. But we embed context header plus text, and on a short chunk the header is most of the vector. The header is the dataset name, which is also most of the query. That points the other way.
Cheaper to measure than argue. Load one dataset's metadata, run a normal dataset level query, record the rank of the first chunk of each source type. Runnable as soon as any one of our four subtasks lands.
After that it's one pool, per source with quotas, or one pool with a floor per source type. I'd rather not pick before there's a measurement.
```
