Welch Labs has a beautiful visualization in their YouTube video about Vision Language Action Models, where the attention of the prompt can be traced to the exact origin pixels in the image from one of those matrices.