I visited a maritime museum which showed a film clip from WW2 about a radio broadcast set up so that merchant sailors could get messages from their family.
I am unable to find an online resource with more info, though I know from general reading and from visiting WW2-era museum ships that sailors were listening to radio shows while at sea. I assume it was all shortwave, back in the time when radio dials included city names.
I was reading Moby Dick earlier this year and was surprised that the mail service seemed to be "I am a ship returning to port, I have a bag of letters meant for sailors SOMEWHERE in the ocean, I will give it to this ONE ship and maybe they will cross paths with these sailors". Curious how often the messages ended up getting to where they needed to go.
"For the long absent ship, the outward-bounder, perhaps, has letters on board; at any rate, she will be sure to let her have some papers of a date a year or two later than the last one on her blurred and thumb-worn files. And in return for that courtesy, the outward-bound ship would receive the latest whaling intelligence from the cruising-ground to which she may be destined, a thing of the utmost importance to her. And in degree, all this will hold true concerning whaling vessels crossing each other’s track on the cruising-ground itself, even though they are equally long absent from home. For one of them may have received a transfer of letters from some third, and now far remote vessel; and some of those letters may be for the people of the ship she now meets. Besides, they would exchange the whaling news, and have an agreeable chat. For not only would they meet with all the sympathies of sailors, but likewise with all the peculiar congenialities arising from a common pursuit and mutually shared privations and perils." - https://www.gutenberg.org/cache/epub/2701/pg2701-images.html
Does anyone know of a current college level literature class analyzing Moby Dick? I know it was a popular book to study in the past, but I think I heard professors were moving away from classic literature because there was so much analysis for LLMs to have trained on. Now that they are so capable on anything, I hope classes are returning to the great works.
For anyone interested (Melvillians), I found some at Princeton, Rutgers, SUNY Maritime College, Harvard continuing Ed, and this Dartmouth course on Coursera:
The article mentions that some messages, like a death notice, are sent multiple times, and gives an example of how a listener can pass on a message to people to others.
I have a copy of "Our Dumb Century: The Onion Presents 100 Years of Headlines from America's Finest News Source".
Quoting its Wikipedia page: "The book, spun off from The Onion weekly-newspaper format of dryly satirizing current events, features mocked-up newspaper front pages from the entire 20th century, presented as though The Onion had been continuously in print since before 1900." https://en.wikipedia.org/wiki/Our_Dumb_Century
If we regard large nuclear power stations as emblematic of centralized power generation, rather than just the nuclear aspect of it, then another target is the step-up transformer and the rest of the connection to the transmission grid.
I don't think those are designed to withstand an aircraft dropping in, nor would their destruction count as the use of nuclear weapons.
I agree. The only real argument against large nuclear reactors is the negative security aspect. All other arguments are nonsense as far as I am concerned. Even this aspect can be reduced by using smaller reactors. The existence of a negative argument doesn't mean it should influence our decisions in the long run. In general, the argument is real but isn't significant enough to be taken seriously.
I think it's because the new_uninit_slice call Rust will trigger a panic? Or abort? With little-to-no chance for recovery? (I know little about Rust.)
If so, I can see why someone that someone coming from Rust might consider exit() to be the appropriate solution for C, even for library code which should never be in charge of deciding how a program should exit.
I think the essay could be improved by highlighting the different worldviews.
I'm also old enough that
// Allocate enough room for the string and its null terminator.
char* reversed = malloc(len + 1);
makes me nervous. Even for char -- I've never been on a system where sizeof(char) != 1 -- I want to see the sizeof included in the calculation, like:
so I don't have to think about sizeof(char) being special.
As long as I'm here, I'm a bit confused about the purpose of the "char* error_message" in the proposed Result. Why a char* vs a const char * or even better, an int with an error code? Who sees the message? Do we expect they know English, or will they be localized? Will the error message text be frozen forever, or might it change in the future?
It’s not any of that. It’s because of overcommit being the default for basically every Linux system. With that, malloc will never fail, and it’s the later access of that memory that will. In practice, you’ll virtually never see malloc actually return a failure, and so most software, no matter the language, is generally not robust to this condition.
How is it then that my stock KUbuntu system won't let me allocate 50,000,000,000 bytes?
$ uname -a
Linux boxcar 7.0.0-31-generic #31~24.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Aug 10 09:38:02 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
$ cat tmp.c
#include
#include
int main() {
char *s = malloc(50000000000ULL);
if (s == NULL) {
printf("Boo, hoo!\n");
} else {
printf("Look at all that memory!\n");
}
return 0;
}
$ cc tmp.c
$ ./a.out
Boo, hoo!
As to the lack of robustness of most software, that's a fact. But, for example, Daniel Stenberg of curl fame is a developer of robust software who does not appreciate how Rust handles out-of-memory errors makes it impossible to implement libcurl in the way he expects a library to work. See https://www.youtube.com/watch?v=HFH2vZRTKrA&t=2080s from 4.5 years ago as an example.
To get back to the essay, I can easily understand why someone who has Rust as their first-and-only systems language, and therefore expects abort-when-out-of-memory, will not immediately consider how C allows a different approach to how to handle that condition, but instead will try to replicate Rust's behavior in C.
> How is it then that my stock KUbuntu system won't let me allocate 50,000,000,000 bytes?
Assuming that it has the default overcommit setting, it's based on a heuristic. Currently, that heuristic is something like "is this single allocation larger than all of physical memory + swap", if I remember correctly.
However, most real software does not try to allocate like this. It allocates reasonably small chunks of memory regularly. So again, in realistic conditions, you generally won't see a failure.
> Daniel Stenberg of curl fame is a developer of robust software who does not appreciate how Rust handles out-of-memory errors makes it impossible to implement libcurl in the way he expects a library to work
I am not re-watching this video right now, and of course have a lot of respect for Stenberg, but either he or you misrepresents the situation here: allocation is purely a library concern in Rust, not a language one, and so you can absolutely write your own code that allocates memory however you'd like. This is a concern about Rust's ecosystem, not about the language itself. It is surely not impossible to do this, embedded folks write Rust that doesn't even have an allocator present regularly, let alone infallibly allocate.
> 4.5 years ago
As I mentioned elsewhere in this thread, the standard library types are gaining methods that return Result instead of panic, and have had them for a long time, though they've been nightly only, since it's a pretty niche case.
My software does work with large blocks of contiguous data, where malloc can fail in exactly this normal way.
For one example, my library uses memory-mapped access, which is great for files which push the memory bounds. However, using memory-mapped files, especially over a networked file system, are slower than simply using RAM, so I have an option to read the entire contents into memory. I know the file size, I allocate space, and read in the data. If malloc fails, I report the failure, and let the caller figure out how to resolve things.
I don't think my software is all that special. A well-written library is not supposed to cause the entire program to panic. At the very least, the library doesn't know how to clean up all of the resources opened by the rest of the system. (Especially in a polyglot program where Python with a finally: cleanup calls C++ with a RAII object, then calls Rust.)
Concerning "I am not re-watching this video right now", I linked to the specific timestamp where he talked about it, so you wouldn't need to re-watch the entire video. I did that because the only text version I found was from 2020, at https://github.com/hyperium/hyper/issues/2265#issuecomment-6... , and I wanted something more recent than that.
> and so you can absolutely write your own code that allocates memory however you'd like
The linked-to essay uses new_uninit_slice in Rust, compares it to malloc in C, points out the issues should malloc fail, and how it needs something like an exit() to handle malloc failures.
My observation is that this would be an appropriate time to point out a philosophical difference between the Rust approach, where the default is for Rust to handle OOM errors, and the C approach where you are in charge of checking and handling malloc failures, and how some C library developers prefer different behavior than what new_uninit_slice provides.
Your suggestion that you can absolutely write your own Rust code this purpose also holds for the C code. You can write your own malloc wrapper function which does the check-and-exit() for you, if that fits your needs, then only call "malloc_or_die()".
I think that regularly requesting a single chunk of memory that is likely to be larger than total RAM + total swap in the desktop context is unusual, but it's true this is an opinion.
> The linked-to essay uses new_uninit_slice in Rust
If you wanted to check the allocation, you'd use try_new_uninit_slice, which is the same thing but returning a Result instead of panic-ing.
> where the default is for Rust to handle OOM errors,
Again, it's worth splitting out the language from the library and or culture here: Rust itself never allocates, and so never OOMs.
> also holds for the C code.
I don't mean to imply Rust is better than C in this regard, only that "you can't check for OOM in Rust and therefore it's impossible to build the library how I want" is untrue.
My belief is that "C for Rust Programmers" should not be "how to write Rust in C" but "How to write C, for people who are Rust Programmers". The expectations for a production quality C library appear to be different than for a production quality Rust library, and I think one of those differences is memory allocation.
While I am certainly ignorant of Rust, my overall issue is that this statement:
"In order to avoid unhelpful segfaults, the program should check for a null pointer and gracefully exit if one is found:"
The following example shows that "gracefully exit" means a perror() and exit() call, which is not appropriate for a C library.
I presumed this is because the author is someone with Rust experience, and who wants to replicate the expected Rust behavior in C. I think the better alternative is to explain that a well-written C library can have different expectations.
I'll add that the essay has a footnote saying "The corresponding Rust function is not idiomatic, and can only correctly handle ASCII text. If I were writing a production version of this function, it would be a single line of code: forward.chars().rev().collect::()."
I see that production version doesn't handle allocation failures either.
My follow-on presumption is that most Rust libraries use this sort of idiomatic production code, where allocation failures simply panic.
I could well be wrong, but this essay certainly doesn't clearly show that I am wrong in my belief.
That's fine. My aim is to clarify Rust's design and choices. I don't have strong opinions about what should be in the post.
> I'll add that the essay has a footnote saying
Honestly I have an entirely different beef with this footnote: `chars` is for codepoints, not really "characters", because what a "character" is in Unicode is very complicated.
> My follow-on presumption is that most Rust libraries use this sort of idiomatic production code, where allocation failures simply panic.
Yes, I wouldn't argue that most Rust code uses these try_ variants, because of the whole overcommit thing. But in contexts where it does matter, people do. The expectations are different based on context. Lots of Rust code expects (and enforces) no allocations at all, but that doesn't mean I'd claim that all Rust code does.
I also think it's fine to want to hold libraries, C or Rust, to a higher standard with regards to allocation failure, but in practice, most people (in both languages) do not find the juice worth the squeeze, and so don't really do it.
> How is it then that my stock KUbuntu system won't let me allocate 50,000,000,000 bytes?
I can't speak to the specific implementations, however servicing an allocation request has multiple fallible steps: reserve address space, then confirm/acquire/defer the physical memory backing the allocation.
My understanding is that overcommit allows deferring assigning physical memory to the allocation, however it could still fail to find a chunk of address space to service the allocation request.
For context, steveklabnik suggested that malloc will virtually never fail on basically every Linux system, because of overcommit being the default.
I showed a trivial example of malloc failing, when trying to allocate more space than on my machine.
Another example is when using resource limits. I've modified my code to malloc only 500M bytes, the set a virtual limit of 500000KiB, which works, then 40000KiB, which doesn't
$ grep malloc tmp.c
char *s = malloc(500000000ULL);
$ cc tmp.c
$ ulimit -S -v 500000
$ ./a.out
Look at all that memory!
$ ulimit -S -v 400000
$ ./a.out
Boo, hoo!
I clearly disagree with steveklabnik's because it's easy to demonstrate cases where malloc fails on Linux-based machines.
I learned programming from a tradition without overcommit; even though one failure mode is reported synchronously, I can't trust a success to mean I won't fail later.
"It's hard to handle correctly for beginners" is a reason for beginners to ignore a return value. It's not a reason for the system to (by default) not do the bookkeeping to know this will (likely) fail.
Sparse mapping & reserving address space then deliberately faulting in pages is a good & advanced technique, but it shouldn't share interface with the beginner/default API.
recently noticed that iced (rust ui framework) crashes on oom - from the logs it seems to be aware and crash explicitly.
I wish it would just reduce the fps or hang a bit instead of crashing, perhaps use exponential backoff
> I've never been on a system where sizeof(char) != 1
And you never will, since sizeof(char) is guaranteed to always be 1.
I'm guessing you were thinking of CHAR_BIT != 8, but even then I'm not sure it would make a difference since malloc takes its argument size in bytes and a char more or less is a byte in C.
(Consider that char*s are also how you access the byte-level representation of objects in C. If chars were not the minimum addressable unit then that use wouldn't work)
So I don't believe your statement "sizeof(char) is guaranteed to always be 1" is correct.
> chars are also how you access the byte-level representation of objects in C
Where does the spec say that a char can be used to address any point in an object?
There's all sorts of oddities like tagged architectures which the C spec handles which I know essentially nothing about, but which break common expectations about how C works. I believe this is one of them.
I believe the following is undefined behavior in C, even though your compiler may let you do it, at least sometimes, and on modern desktop hardware:
int i = 12345;
char *s = ((char *)&i) + 1;
char c = *s;
I believe the following is the correct (or less incorrect) way to do it:
char tmp[sizeof(int)];
memcpy(tmp, &i, sizeof(int));
char c = tmp[1];
Yes, but in C standardese a "byte" is not necessarily the 8 bits that it's normally thought to be these days. From C89 Section 2.2.4.2 Numerical Limits [-1]:
> maximum number of bits for smallest object that is not a bit-field (byte) CHAR_BIT 8
i.e., CHAR_BIT is the number of bits in a byte. C23 has a similar definition, and further defines CHAR_WIDTH that is defined to expand to the same value as CHAR_BIT.
> So I don't believe your statement "sizeof(char) is guaranteed to always be 1" is correct.
From C89 section 3.3.3.4 The sizeof operator [0]:
> When applied to an operand that has type char, unsigned char, or signed char, (or a qualified version thereof) the result is 1.
This wording remains basically identical through C23 [1].
> Where does the spec say that a char* can be used to address any point in an object?
From C89 section 3.3 Expressions:
> An object shall have its stored value accessed only by an lvalue that has one of the following types:
>
> * a character type.
This also remains the case up through C23.
I think you're thinking of the strict aliasing rule with your example, but character types are one of the exceptions to said rule so I think your example is actually fully defined. It'd be UB if you casted to an incompatible type like a float, I believe.
I .... okay, my mind is blown. It absolutely says that, and https://en.cppreference.com/cpp/language/types adds "this allows the extreme case in which bytes are sized 64 bits, all types (including char) are 64 bits wide, and sizeof returns 1 for every type."
So that scaling factor is placed into CHAR_BIT, and would be 64 on that DSP compiler.
> An object shall have its stored value accessed only by an lvalue that has one of the following types:
Yes, my confusion comes down to my confusion of what "byte" means in the C spec.
Yeah, it's one of those things that is not exactly intuitive if you didn't already know about it. The menagerie of other byte-ish-sized types added in recent versions of C and C++ probably doesn't help either.
Bit of a (not so?) fun fact: char* being a universal alias can lead to some potentially unexpected slowdowns [0], especially if the char* bit is behind a typedef.
(And CHAR_BIT is required to be 8 by POSIX, so even though the language and some exotic hardware will have it at non-8, it’s exceedingly rare at this point.)
Not sure I agree with this pattern for strings, but if you're gonna multiply for an allocation like that and are concerned about consistent idioms, use calloc or [non-standard] mallocarray, which will check the multiplication for overflow.
After being bitten by mistakenly using int for the result of a sizeof, then overflowing the int multiplication with the count, my personal idiom is only use malloc if it's a count multiplied by a sizeof.
I just grep'ed my code base and found one counter-example, which I did on Aug 29, 2011.
The original design used two projector screens, one for each wall, alternating with left- and right- eye views, with the user wearing shutter glasses keeping the vision in sync, while wearing a head tracker. They did use 7 SGIs (6 VGX workstations and a Personal Iris) for the system.
Traditional UNIX was entirely proprietary, with source you're intended to tinker with.
AT&T was forbidden by its consent decree from productizing Unix. Licensees received the source code, on an as-is basis, with no official support from AT&T.
Sure, but that just means UNIX vendors had access to the source code.
It doesn't mean end users had access to the source code.
In fact, vendors would have been prohibited from distributing AT&T source code to customers who weren't themselves AT&T licensees, just as CSRG was prohibited from distributing BSD to customers who weren't licensees before 4.4BSD-Lite.
The same goes for most other proprietary source-available products. For example, game developers can't open source Unreal Engine projects even though Unreal Engine source licenses are available to end users for free.
As jeremyjh wrote at https://news.ycombinator.com/item?id=50032733 , "A system that the hardware owner cannot modify is not really on the same wavelength as traditional UNIX.". Since traditional Unix was source-available, with the expectation that people would hack on the source, I fully agree with jeremyjh's characterization.
I think you are talking about a later era in Unix history? The early Unix licensees were educational institutions and commercial sites, not vendors or resellers.
Every one of those academic and commercial end users got the source code.
> "In 1973, AT&T released Version 5 Unix and licensed it to educational institutions, and licensed 1975's Version 6 to companies for the first time, ... Anyone could purchase a license, but the terms were very restrictive; licensees only received the source code, on an as-is basis. ... Unix was entirely unsupported by AT&T"
From the same link, "Perkin-Elmer began distributing Wollongong Unix for Interdata as Perkin-Elmer Edition 7, the first version supported by a computer company". That was in 1980.
So while yes, "Traditional UNIX was entirely proprietary" and could not be redistributed to non-licensees, it was also provided with "full user control and manipulation of the OS", and not merely "connected with well-defined text-based interfaces", quoting jdiff at https://news.ycombinator.com/item?id=50033080.
You would absolutely normally get source to the vast majority of the unix components with a unix install.
At a bare minimum, you would almost have to have source to the vast majority of the kernel because there was no kernel modules. Sysgenning for a particular system involved compiling the kernel for that set of hardware.
I'm literally working on an early sun system emulator right now, heavily assisted by the standard sec tape that came with the standard software distribution of sunos, which has plenty of AT&T code.
Sure, if by “you” you mean universities, Sun, DEC, IBM, etc.
As a non corporation end user with a sublicense from one of the above licensees (which is the apt analogy for an individual user in my opinion) you most certainly did not receive the source or have the rights to it.
I regard "traditional Unix" as the 1970s era. There were no sublicensees. You bought the source code from AT&T. That was the only option.
You then likely hacked the source code to make it work on your machine.
I can understand that others might include the commercialization of the 1980s and the Unix wars as part of "traditional Unix", but even then, my IRIX system ca. 1990 supported end-user kernel extensions, to handle new types of devices. Our imaging hardware was not "connected with well-defined text-based interfaces".
I am unable to find an online resource with more info, though I know from general reading and from visiting WW2-era museum ships that sailors were listening to radio shows while at sea. I assume it was all shortwave, back in the time when radio dials included city names.
reply