People don't always have full control of their devices - I can't install arbitrary software on my work machine, for example, so a VPN back to my house isn't possible without a tool like this. This applies for appliances (eg, media streamer) where you can't install a VPN but want to access content from your home network.
There's other reasons too, such as not having to configure arbitrary wifi connections on your devices, as well as making sure there's zero traffic outside the VPN (eg, during the initial few seconds before an on-device VPN connects).
It would, but installing random vpn software on IT controlled hardware is a very much against policy
I also have to remember to turn the VPN on before I connect to the wifi, but only some wifis
Sure they are minor inconveniences, but configuring a device once and being confident I haven’t skipped a step or made a mistake and my network connection is secure - that’s a big deal for me.
And not just "against policy but unenforced and whatever" - for most large orgs I've been with, it's a serious, "conditions of employment" issue. There ARE innocent usages of personal VPN on work computer, but the risk and impact of the nefarious ones massively outweighs them.
I suppose if you've got a number of devices (phone, laptop, tablet), it's easier to enter Wi-Fi details and agree to any captive portal stuff once rather than three times.
No, to have negative income tax you need to set a baseline relative to which income tax is negative. E.g. your income tax is what you earn above 30k, positive or negative. Then if you earn 0 then b>0
> It also helps combat the "LLMs ArE nOt DeTeRmInIsTiC" complaints.
Reality says otherwise, especially from those who run LLMs in production at a large scale. Hacks such as baking in the weights with tiny experiments doesn't disprove anything.
The same LLM boosters also continue to repeat one of the biggest misconceptions: "sEt TeMperaTurE tO zERo tO aChiEvE fUlL dEterMinIsm."
The accurate statement is that LLMs are not inherently non-deterministic. From a mathematical perspective, they’re obviously deterministic. But choices in the way we deploy them introduce non-determinism.
The question should really be the other way around - LLMs are just computer programs, and in general, computer programs are theoretically deterministic. Given the same inputs and the same initial state, they produce the same outputs. So the question is, what would make an LLM non-deterministic?
For any computer program, there are two main ways determinism can be lost:
1. Randomization is introduced deliberately, e.g. via random number generators.
2. The program's behavior depends on factors beyond its explicit inputs, e.g. its execution environment, timing, concurrency, or interactions with other systems.
Actual deployed LLMs typically lose determinism because of both of the above factors:
1. On each request, the core model outputs a list of likely tokens, ranked in probability order. To avoid repetitiveness or getting "stuck", particularly in natural language output, it's common to introduce randomization when picking a token from this list. This is what's called "temperature". This is deliberate non-determinism that's applied to the output of the core neural network model. Setting temperature to zero can eliminate this kind of non-determinism.
2. In practice, consumer-facing deployments run in complex hardware environments - distributed processing, GPUs, hardware shared with other processes, etc. This leads to non-determinism due to things like concurrency and timing issues. Calculations can execute in different orders, numerical errors can accumulate in different ways, etc.
For companies like OpenAI or Anthropic, which serve millions of concurrent sessions, eliminating non-determinism from #2 would be difficult, and not worth the effort.
But if you're running an LLM on e.g. a single machine, it's certainly possible to eliminate non-determinism. People do this for various kinds of models. Machine learning frameworks have explicit options to help with this. E.g. with Pytorch, you can use `use_deterministic_algorithms(True)` to force it to use deterministic algorithms where possible.
One point that sometimes confuses people here is that even if you run a perfectly deterministic LLM, the output is unpredictable in general. That's because the output of the model depends on a large number of trained weights, and the only practical way to "predict" what output a model will produce is to run input through the model. This is not non-determinism, though.
In diagram terms, at a high level an LLM looks like this: input --> neural network --> token selection --> output
Token selection can be randomized, but doesn't have to be. When it's not, the above is perfectly deterministic in theory, and can also be run that way in practice.
Now take that diagram and run it across a data center full of machines, and achieving determinism gets more difficult. It could be done if we wanted to, but in general it's not important enough to try in that scenario.
thank you for the detailed answer, this is all very interesting to me as I'm just starting a Masters in AI. I always had the wrong idea that LLMs couldnt be made deterministic. Now that I know they could be this opens up more possibilities for me.
Glad it was helpful. If you have time for some extracurricular study, I'd highly recommend implementing an LLM. Implementing a toy version is really not that difficult, and can be very illuminating.
There are quite a few resources on the web that can help with this. One good one is Andrej Karpathy's "Let's build GPT: from scratch, in code, spelled out.": https://www.youtube.com/watch?v=kCc8FmEb1nY
Two separate things. Sampling randomness goes away with greedy decoding: always take the highest-probability token instead of sampling, which is what temperature 0 approximates.
The remaining nondeterminism comes from floating point. Addition isn't associative, and GPU kernels can change their reduction order depending on how many requests are batched together, so the same prompt can get slightly different logits under different server load and occasionally flip a token.
Density is insanely highly correlated with the perception of premium quality. No one’s increasing the volume of premium devices beyond what’s needed to hold components.
They're still around, making good shit elsewhere. ChatGPT often helps me track down some of the people behind my favorite games and check out their later work.
reply