Until a few years back most of my systems used Linux From Scratch. It's amazing for learning your system and how all its libraries integrate. I created my own boot scripts, my own pacman package manager repository, etc. One of the main reasons I stopped using LFS was the amount of work required for updates.
Firefox is bad to compile yourself, because it's slow and often requires a library version you don't have, such as a new ICU version. Updating ICU also requires you to build packages that depend on it: harfbuzz, ffmpeg, qt5, poppler and the list goes on.
Chromium takes it to a whole new level. While Linux packages typically have an automake flow (configure, make, make install) Chromium requires ninja, not just ninja but a specific version depending on the Chromium version. Then you need a ton of libraries with specific versions (or custom patches) or use the built-in versions provided by chromium, which may or may not compile at all. Then there is clang, while most software is compiler agnostic, Chromium does not compile with gcc. Of course, it needs a specific clang release, sometimes even a pre-release, or again a load of patches to compile. It seems the chromium build process is mostly built by engineers desperate to reinvent the wheel. While everything can work fine with sane Linux defaults (GCC, automake, system libraries) they convinced themselves their software is so complex and unique it requires a special treatment.
Chromium looks like a build system for a complex integrated project where the engineers have one priority: make the application work as best it can. You will quickly find that when you focus on that then making the build system convenient, especially convenient for distribution packagers with their myriad of own rules and modifications that they want to make on top of your system, is quite a big blocker to the primary goal. You can see similar things with applications like OBS, where they bundle modified libraries to fix problems and then distributions unbundle them and release broken packages, causing no end of grief for users and the developers.
The Chromium build process is just a consequence of the monorepo mindset. For Google, depending on specific versions of tools and dependencies is not a problem because all their engineers will automatically get those versions. For the Linux distribution model it's a pain.
> Firefox is bad to compile yourself, because it's slow and often requires a library version you don't have, such as a new ICU version. Updating ICU also requires you to build packages that depend on it
> Chromium does not compile with gcc. Of course, it needs a specific clang release, sometimes even a pre-release, or again a load of patches to compile
And that is what is wrong with today software: it is an uncompilable mess.
It is "This program requires Windows 98 and Internet Explorer 4" brought to a new level.
There is not some arbitrary notion of wheel reinvention just for browsers. Calling browsers 'just browsers' is doing their feature set a huge disservice: they are bona fide operating systems or virtual machines, which have to load and run untrusted, unverified data and code within critical system libraries (many of which with kernel-mode access, such as GPU drivers, sound drivers, USB subsystems, and more), and do a huge amount of sandboxing to guard against escalations.
Within Google, a Chromium build uses remote build execution by default, upstream Clang by default (for the latest features and fixes, and Chromium developers regularly contribute to LLVM), and needs to cross-compile easily for at least a dozen different OS-architecture triples (Linux x86-64, Linux ARM, macOS, Windows x86-64, Windows on ARM, iOS, Android x86-64, Android ARM, Android TV, some BSDs, etc).
Everything you've listed—from ninja, to Clang, to libraries with custom patches, to Clang releases, including requiring pre-releases sometimes—is optimised for Google's scale, rather than the lone warrior developer. What you call 'sane Linux defaults' (especially the examples you have listed: GCC, automake, so-called 'system libraries') are highly antagonistic to all of the above.
They may make life easy for the so-called 'default case' of desktop Linux (arguable; Chromium was first released on Windows, and secondly since desktop Linux's most common C library, glibc has an unstable ABI, making back-portability impossible), but make it incredibly hard to get out a project of such immense complexity. All of Chromium's tooling and 'complexity' was developed in response to the 'sane' defaults being not so sane after all.
To get a GCC cross-compile for a given set of architecture triples, one needs a complete graph of GCC toolchains targeting every pair of host and target. On the other hand Clang is natively a cross-compiler; pass in `--sysroot` and the host compiler will redirect for the target instantly. Automake (and the autotools in general) is slow as molasses compared to ninja. It is based on computing and compilation from the 1970s, and their defaults probe for long-since-dead string comparison functions that are expected to work by default nowadays on most hosted systems. Not to mention it has an extraordinarily complicated configure step[1], and an equally appalling macro-based configure language.
Ninja is purely declarative, and is not even meant to be human-readable, as it is a back-end for a variety of build generator tools (CMake, XMake, build2, Bazel (Blaze internally at Google), Meson, probably more). Google have in fact rewritten Ninja to be a remote-build-capable tool from the outset[2], and they call it Siso. Siso has more than halved my Android build times even compared to Ninja because of its incredible use of RBE.
Frankly I think it should be evident why they have opted for 'rewriting the world' including vendoring and modifying system libraries, as well as including efforts like Skia, ANGLE, and more: directly relying on 'system libraries' is painful. I mentioned GPU drivers. Imagine rewriting the same code over and over to have precisely the same behaviour on D3D, Vulkan, OpenGL, Metal, and every single vendor GPU driver on every single system (and there are a lot more on mobile than on desktop: ARM Mali, Qualcomm Adreno, Samsung Xclipse, Imagination PowerVR, Huawei, MediaTek; this is not even close to exhaustive, just look at the Khronos vendor list).
When comments like yours are made I wonder if there is ever a consideration beyond 'it makes the developer's life hard'. As developers, our ultimate target audience are the users of our software. We should be willing to do anything come hell or high water, even making the dev-ex extraordinarily convoluted, if it means the user experience will be the smoothest and fastest possible. Sure, there is a balance, but I believe it should be strongly skewed towards the user story first.
>glibc has an unstable ABI, making back-portability impossible
Of all the libraries to call out for this, glibc is not the one. It's basically the only library on linux that actually has a strong commitment to providing a stable ABI, with its own symbol versioning system to allow it. It is a bit of a PITA to compile for an older version of that ABI, but it's very possible to be compatible with glibc across a wide range of versions and distributions (I don't know why there isn't a flag to do it: the most widely supported option is compiling on a system with the oldest glibc you want to support, and there's some header hacks which claim to be able to avoid that).
It's everything else necessary to interact with the system (especially anything desktop/multimedia) that's wildly unstable.
I singled it out because I've been badly burnt by it in the past, and I dislike how it works compared to Win32. Back-targeting old glibc from newer (a very common case because developers tend to have newer distros than their users), requires either changes in source code (not build code, which is where it ought to be) to explicitly pin versions, or the ridiculousness of firing up an entire Docker container just for old glibc. It has regularly caused issues for things like Steam on Linux.
If you build on random developer's machines rather than a unified build environment you've already failed at so many things that the glibc problem is certainly not your biggest concern.
Regardless it's still a PITA to maintain an ancient distro as a build environment and the alternatives are hacky. It might be the best of the linux libraries but it's still not good.
No need if you are targeting a current linux distro. You only need to do that if you are targeting the ancient distro, in which case you need it around to actually test your software on it as well.
All I can say is that Gentoo specifically forbids downgrading glibc. I think it's the only package where they did this. [1] The .ebuild is full of patches and workarounds.
Yeah, that's forwards-compatibility, which is different. Something built targeting a newer version of glibc is unlikely to work with an older one. (and gentoo will by default build against the latest you have installed, so yeah, downgrading has a good chance of breaking near everything).
> All I can say is that Gentoo specifically forbids downgrading glibc.
Nope. You're protected from accidentally doing it, not forbidden. Look:
# I_ALLOW_TO_BREAK_MY_SYSTEM=yes emerge -1v =sys-libs/glibc-2.43-r4
These are the packages that would be merged, in order:
[ebuild UD ] sys-libs/glibc-2.43-r4:2.2::gentoo [2.44-r3:2.2::gentoo]
Total: 1 package (1 downgrade), Size of downloads: 0 KiB
>>> Verifying ebuild manifests
>>> Running pre-merge checks for sys-libs/glibc-2.43-r4
>>> Emerging (1 of 1) sys-libs/glibc-2.43-r4::gentoo
* glibc-2.43.tar.xz BLAKE2B SHA512 size ;-) ... [ ok ]
* glibc-2.43-patches-5.tar.xz BLAKE2B SHA512 size ;-) ... [ ok ]
* Checking whether python3_14 is suitable ...
* dev-lang/python:3.14 ... [ ok ]
* Using python3.14 to build (via PYTHON_COMPAT iteration)
>>> Unpacking source...
* Checking general environment sanity.
* Sanity check to keep you from breaking your system:
* Downgrading glibc is not supported and a sure way to destruction.
* Performing simple compile test for ABI=amd64 ...
If you don't set that environment variable, the downgrade blows up at the "Sanity check to keep you from breaking your system:" step with a message that includes a call stack that pretty much tells you how to override the safety check:
* Call stack:
* ebuild.sh, line 143: Called src_unpack
* environment, line 3570: Called sanity_prechecks
* environment, line 3199: Called die
* The specific snippet of code:
* [[ ${I_ALLOW_TO_BREAK_MY_SYSTEM} = yes ]] || die "Aborting to save your system.";
Fair enough for poor Google, if only they did not strong-armed Chrome on everyone's computer by leveraging their default search engine/home page position, their ads and pop-under "Try Chrome" that were plastered everywhere 15 years ago or so.
It is: builds being painful enough that it's near impossible for smaller distro's to maintain a Chromium package, is community-hostile in itself.
A true community project is flexible w/ respect to dependencies, builds in a way that's doable by anyone who puts in some effort, and sends bugfixes in dependencies upstream so that -normally speaking- project can build against common (versions of) system libraries.
Doing the exact opposite screams vendor lock-in disguised as an open source effort. Of course no surprise this coming from Google (also notice similarities with the Android project).
It's relevant because Chrome market dominance still has a very strong influence on websites working (tested?) only with Chrome. Add to that Google's NIH syndrome and its "move fast" mantra at the cost of interoperability with the wider Linux community and then both things are relevant.
> which have to load and run untrusted, unverified data and code within critical system libraries (many of which with kernel-mode access, such as GPU drivers, sound drivers, USB subsystems, and more),
No, they don't have to. The fact that someone decided that a browser shall run remote code (JS) is sad.
gentoo's emerge is awesome. I keep a gentoo vm just to compile firefox, because it is impossible any other way. even though i don't use gentoo overall.
the comments trying to defens chrome (and old OBS) are completely corporate shills or missing the point. yeah obs at first did that because they had to, since it was such a small and niche project the devs had no pull upstream. after distros did all the unbundling job and helped them move patches upstream now every software benefit from those libraries fixes.
Google took chromium et al from open source and actively moved in the wrong direction, while having all the pull and influence like no open source project will ever have! its malice and/or incompetence. if you don't see that you're either shillings or lying to yourself. or have no knowledge of what open source is about.
> Linux defaults (GCC, automake, system libraries)
Last time I tried Gentoo desktop I ended up with 8 separate autotools versions, and gcc & glibc was not that far behind in count. And just like in https://xkcd.com/927/ every C developer tries to improve the situation by creating another build system.
> Last time I tried Gentoo desktop I ended up with 8 separate autotools versions, and gcc & glibc was not that far behind in count.
When was this?
AFAIK, you simply can't end up with more than one glibc version as that package hasn't had multiple Slots. [0] GCC is Slotted but I can't remember a time when there were more than three or four Slots defined for it. IME, it's rare for a package to declare an upper bound on GCC version, but it does happen. It's definitely true that there are many Slots for the various autotools tools.
[0] A "Slot" defines a particular subset of package versions that can be installed alongside other "Slots" of the same package. For example, every sys-kernel/gentoo-sources version has its own Slot, every minor version of dev-lang/python [1] has its own Slot, etc, etc, etc.
This is also why I stopped using Gentoo. I used it regularly from 2004-2005 (which I admit was a very long time ago now) on my graphite G4, because support was very good for PowerPC. But then one day emerge suggested I update glibc. I don’t know if I botched the update or if it really was suggested in error, but I was just starting my studies in CS so I did not know what that particular “library” did and said yes. The update must have included an ABI change. Not surprisingly, the machine did not come back up after a reboot.
Gentoo was fun while it lasted (fluxbox user here!). I decided that MacOS X wasn’t so bad after all after that.
Remember when distros used complicated partition setups, one for /root, one for /var one for /home, a swap partition etc. Was always a bad choice because one of them would be at 99% while others would linger below 10%
For swap, the best advice is to disable swap on your desktop, unless <8GB RAM. Really, I've never needed it and you probably won't either.
On the Desktop fine-grained partitioning is a bit questionable but on a server it's still quite practical. E.g. preventing the database or excessive logs from filling the disk space. The latter is also an attack vector actually.
Also generally swap is recommended on all systems. During normal use it should rarely be needed but if it's needed it prevents spurious crashes. E.g. make -j16 shouldn't result in a hard to comprehend crash because I miss 1 GB of RAM.
I tried that a few years ago. The result was worse than swapping. The kernel really tries its best not to kill anything, and you end up with "thrashing" again, where code pages (shared objects, executables, mmaped files) are evicted and need to be loaded again from disk. In practice the slowdown was worse than swapping to disk, and much more sudden. I'd recommend at least zram to soften the OOM landing.
> For swap, the best advice is to disable swap on your desktop, unless <8GB RAM.
That maybe reasonable advice if you never intend to hibernate your desktop. I like to shut my desktop down when I'm not using it, and it's nice to be able to quickly boot it back to the same state it was in when I went to bed.
I'm not sure if that's a bad choice... BTW, I think STIG still requires it. So, if you want to be compliant with some (US) government requirements, you'd still have to put different bits of your system on different partitions / devices.
Why I'm not sure this is a bad idea: a pathological process that quickly captures the entire allocated disk space will be contained by this layout and will likely not affect the system as a whole. Consider, for example, a process that excessively logs into /var/log: once the filesystem mounted at /var fills up, it might fail / hang, but you will still be able to use stuff from /bin or /sbin. But, if your entire root filesystem fills up, you probably won't be able to run any applications at all.
Was really happy with my FP3 until it died: "sudden death". No power, nothing. According to their own forum, it's a very common problem after a while. Of course there is no warranty and replacing the mainboard is not possible...
You always have the right to a minimum 2-year guarantee from the moment you received the goods. However, national rules in your country may give you extra protection.[1]
Given two files, a compressed file which must be uncompressed to play, and an uncompressed one which does not require extra processing - why would the game ever bother uncompressing te compressed file when the same data is already available uncompressed?
With hard disks, I could see reading the compressed version from disk and decompressing giving you the bytes faster than reading the uncompressed version. With SSDs that advantage likely went away.
Yep. If you had the CPU overhead available, you could speed up your entire computer by enabling full-disk compression and then defragmenting. It's difficult to accurately describe just how slow it was to move data to and from a spinning platter over a PATA interface.
It's not about size or file count or disk space consumed. The question is about why would you require the extra overhead of playing the compressed version which involves decompressing it on the fly in some way, versus just playing the already decompressed version.
In this case it was stated that both the compressed and decompressed versions live side by side. So there is no install size saved.
"The game Titanfall on PC has the audio in it twice, compressed and decompressed."
'has it twice' could just be on the install media, and compressed/ uncompressed chosen as a trade off between install size and performance.
Anyway searching around the OP seems to be misrembering. Titanfall seems to be well-known for 35gb out of the 48gb install being uncompressed audio. I couldn't find a reference to it also having compressed audio also.
There are many good reasons to use NoSQL instead of a "full SQL database". OTOH I can relate to your experience, I remember colleagues switching to MongoDB because they couldn't get good performance on the (MS) SQL database. They didn't know enough about proper (multi-column) indexes, use proper isolation levels, etc.
Firefox is bad to compile yourself, because it's slow and often requires a library version you don't have, such as a new ICU version. Updating ICU also requires you to build packages that depend on it: harfbuzz, ffmpeg, qt5, poppler and the list goes on.
Chromium takes it to a whole new level. While Linux packages typically have an automake flow (configure, make, make install) Chromium requires ninja, not just ninja but a specific version depending on the Chromium version. Then you need a ton of libraries with specific versions (or custom patches) or use the built-in versions provided by chromium, which may or may not compile at all. Then there is clang, while most software is compiler agnostic, Chromium does not compile with gcc. Of course, it needs a specific clang release, sometimes even a pre-release, or again a load of patches to compile. It seems the chromium build process is mostly built by engineers desperate to reinvent the wheel. While everything can work fine with sane Linux defaults (GCC, automake, system libraries) they convinced themselves their software is so complex and unique it requires a special treatment.
reply