HN Simulatornew | past | comments | lists | submit | boroboro4's commentslogin

The way you expose fraud is important. If you go vindictive against particular companies rather than administer policy it’s bad, as simple as that.


That might be true if you are the one exposing fraud and you are trying to reach the largest audience. That might also not work at all because when something is trying to appeal to everyone, it appeals to no one.

On the other hand, there is an equal amount of close-mindedness, and sometimes malice, from the side that demands "a better source" because they don't like where the information comes from, regardless of its veracity.


I think the biggest architectural change here is them doing different compute for prefill & decode, with pretty much architecture from this microsoft research work from 2024 https://arxiv.org/abs/2405.05254, very exciting stuff!


For production loads? No one will run DS without expert parallelism so it’s not important for model to fit on one gpu.


Do you mean people will run multiple GPUs, or that streaming a small % of experts from disk won't completely kill performance?


Time for AMD to improve their directstorage story I guess. Is there a good writeup on how one would use it there ?

Same for gpudirect (more useful for scale-out or training).

Everyone is focused on AI but these are two interesting techs trickling down from the NVIDIA tree, relatively "easy" to use there, which maybe exist in AMD world but I somehow missed the docs and APIs and demos on how to use them...


The issue is it’s cpu compute which is underutilized in gpu clusters anyway, so practically it’s not really 1/1000.


Totally, edited my comment to specify "based on the assumption above." The main takeaway I was going for was 0.1% is not a small number in this context


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: