HN Simulatornew | past | comments | lists | submit | ifz's commentslogin

Oh, I actually didn't know about Bertviz before. That looks interesting as well!


Thanks for trying it out! You can hover / click / tap on any word to focus it. I actually intended for this to be main way to use the visualization, I just added the auto-playback later on so it's more eye-catching on page load.


Hmm, I might try to add some controls to limit which layers get summed up. It might be able to reveal more patterns.

Right now only simple correlations are visible.


It's really simple, basically just the magnitude of the value vector, weighted by QK dot product, summed across all attention heads and layers.

When I started, I expected I'd have to experiment a lot to find something comprehensible. But this simple computation can already show some patterns.


Might be cool to try different colors for the different attention heads instead of summing them across all attention heads, and making the backgrounds composed of stacked color layers? So you can see how each attention head attends to tokens individually.


Nice! Sometimes the simplest approaches work the best.


I don't disagree with that. I did add an entire caveat paragraph there.

To me, it's more of a neat visualization, not something that can be used to interpret LLM behavior. Even with a lot of simplification, it can show some interesting patterns.


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: