80 t/s w/ 3090s and 3.05bpw exllamav3

id imagine if you have the location of a site that the largest pipes visible from satellite photography would be consistently scaled in some manner with the tank size. this is guess

Yea man the federal government is on top of protecting civilian groundwater supplies and communicating discoveries of contamination

I know you jest, but the US govt funds a lot of water science. Whether that's put to use in the public interest is another question ofc and one we should be asking

importantly, does your die have a side that corresponds to a multi-continental cataclysmic natural disaster of magnitude only known unknown to pre-recorded-history

Will it then be possible to query TJunc hotspot temps on linux?

pardon but aren't disks usually the bottleneck? im all for CUDA acceleration and CUDA accelerating culture

Parent post was talking about an 8GB dataset.

8GB isn't even CPU RAM these days. That's GPU super-mega-awesome ram. Ordinary Server CPUs are regularly pushing 2TB capacities.

GPUs are in the 8GB to 32GB typically, at least for smaller and more regular GPUs. This GPU RAM is also well known to be at least 10x the bandwidth of CPU RAM.

Yeah, I have a GPU from almost six years ago in my desktop that has twice that much VRAM. Less than a year ago my wife got a 5070 Ti with the same for around $750 without needing to wait for it to be in stock or anything. I'm inclined to think that for a server that needs a GPU, even 32 GB would probably be considered small.

Which is why a good query plan is so important, so that as much disk I/O can be avoided as possible (predicate push down, index elimination, join ordering, partition/scan pruning). Like the old CTE optimization fence problem.

I remember projects like PG-Strom back in the day, very cool stuff

not necessarily, no. With SSDs you get much better IOPS for cold data, and many datasets fit in RAM. So a lot of (OLTP/HTAP) workloads can become CPU-bound due to sorting/hashing - bread and butter of joins.

Though many use network block storage (e.g. EBS), which is significantly slower than an SSD.

Something like fpcalc for soundbytes? Neat

Is anyone else thinking of cracking open the back cover and desoldering any wireless chips found? I have had a firewall rule to deny my LG TV and related domains, but that doesn't look to be enough

IIRC, most wifi enabled televisions have internal antennae. Remove that (or otherwise disable it) and you significantly improve your security posture by ensuring the radio can't hold a connection beyond the TV bezels.

I'm going to see how this shakes out on my machine with a few 3090s. I see you all are leveraging some custom cuda kernels, so it may not work out of the box on Ampere (30xx) architecture yeah?

Yep, might need some changes.

I love larping as a vacant scrum master