One potential weakness with using groff for professional publishing is that, according to its current mission statement (2014), its line-layout algorithm is limited and well behind the times:
Groff currently uses a greedy algorithm to format paragraphs one line at a time, but the Knuth-Plass algorithm, implemented in TeX, Heirloom troff and elsewhere, optimizes linebreaks throughout whole paragraphs and achieves a more uniform typographic grey. One of the most exciting challenges facing groff will be implementing an improved formatting algorithm, the most likely candidate being Knuth-Plass. It’s a big job, but with groff’s clean codebase and helpful community of experts, it is hoped additional developers will step forward and contribute their programming skills.
Equally important will be instituting native support for TrueType, Open Type, and other non-Type1 PostScript fonts, and improving Unicode support.
That's right. From the colophon of 1989's Nutshell Handbook on Using C on the UNIX System:
> The text of this book is set in Times Roman; headings are Helvetica; examples are Courier. Text was prepared using the troff text formatter and the devps PostScript filter. Figures are produced with a Macintosh. Printing is done on an Apple LaserWriter.
One thing I have noticed about S-tier writers is that most of them have an established routine and stick to it. Why risk breaking something that works? With an established routine, you settle into it, and muscle memory puts you into writing mode. Start mucking with the routine, and that is how you get writer’s block.
I think it’s fascinating to see all the different ways that techfolk who write fiction set up their writing pipelines. Some people go with a bare-bones coding editor like Vim; others use “pro” tools Scrivener. And then there’s the typesetting pipeline. Using troff is pretty out there.
When I started writing fiction, here’s what I came up with. I write in Google Docs because it’s lightweight, works everywhere, and can easily export to Markdown format. From there, I manage the Markdown source in Git and Emacs. For typesetting—I cannot abide crappy word-processor output—I used to use Pandoc to generate LaTeX markup and run LuaLaTeX to produce PDFs. But that took several seconds. Now, I use Typst to go straight from Markdown to PDF files. It’s pretty much instant.
If you want to see what the output looks like, here’s the first chapter of my current project:
(Notice: This is a typesetting sample, not a writing sample. It’s my W40k-adjacent fanfic, so you have been warned.)
It’s set in 12pt Erewhon, which is an enhanced version of Andrey V. Panov’s Heuristica font family, which extends Robert Slimbach’s Utopia font family. Slimbach is a sublime type designer—he created Minion—and Utopia is one of his early works that is especially spirited.
Indeed. One irresistible temptation for coders who go into writing is to bikeshed their pipeline. I know I have spent many hours tweaking my writing pipeline when I should have been writing. I mean, why just write when you can create a Containerfile to build a hermetically sealed image that houses your typesetting system and a dozen carefully chosen fonts, and then you can write scripts managed by a Makefile to launch containers from the image and inject into them your source text and extract from them the typeset PDFs? I mean, surely all of that is essential work on the critical path to getting your writing done. Right?
I remember a quote by Pandoc creator slash professor of philosophy John MacFarlane: "I would like to think that the time I’ve spent developing tools to make writing more efficient has been repaid by the time I’ve saved using these tools, but I’m pretty sure it hasn’t."
One writer recently wrote his own program for the job, but at least he eventually finished some books: https://hughhowey.com/neo/
Me, I just write in Emacs, set up a makefile, and export to PDF with Pandoc. But I admire the work to make a comfortable yet undistracting writing environment.
Agreed, it seems writers that also code have all kinds of different workflows. I end up using (x)html and doing print stylesheets and rendering pdf for print using weasyprint (works really well, and they have sample stylesheets on their website that are a good starting point).
troff is no longer fashionable, and it's not what you want for top-quality typesetting, but I don't think of it as "out there". My non-technical mother was using an HDOS program called TEXT.ABS, which is similar to a limited nroff, to format the articles she wrote in the 01980s. It wasn't as easy to use as something like MacWrite, but it was a lot easier than a typewriter. The text editor she used was called PIE.ABS, and from my point of view, the usability distance from a typewriter to PIE was bigger than from PIE to MacWrite.
She didn't have much choice of fonts on the pre-Graftrax Epson MX-80 she used. I think it was Pica or Elite.
Typesetting is certainly very satisfying, but it's not essential to writing.
Huh. I wrote into vim and made a bunch of .md files, and then just had Claude assemble them into an epub to make my book for Kindle. How did you decide on that size of page for the PDF?
That PDF is for proofreading and review, so I just used U.S. Letter size to make it easy to print on my home printer. For actual publication, I would use the page size for, say, a trade paperback.
For epubs, right now I'm using Pandoc: https://pandoc.org/epub.html. But, again, that's just now for the writing and proofing stages. For final output, I'll consider my options. (I may still go with Pandoc, but right now I'm using it because I was already using it for typesetting and so it was a low-friction choice.)
I still use a custom markup language I made for myself [0] because I don't find Markdown flexible enough, Latex's syntax is disgusting, and neither is particularly consistent.
Tired of low-effort YouTube clickbait slop? Just click on the Subscriptions heading to go to https://www.youtube.com/feed/subscriptions. You'll see only videos from the creators you have subscribed to. This one-click trick boosts the signal-to-noise ratio dramatically.
I'm curious: With your approach, how do you make sure your code is secure? For example, how do you show that it isn't riddled with XSS or other injection vulnerabilities?
Tl;dr: The trouble with ntile seems to be that, when the author of the post imagined what it did based solely on its name and she imagined wrong, she ran with her imaginary version way too long before checking the documentation, which was very clear about its behavior:
> Unlike other ranking functions, ntile() ignores ties: it will create evenly sized buckets even if the same value of x ends up in different buckets.
I’m pretty sure NTILE is just a generally cursed function. SQL NTILE also requires everything to be loaded in memory because of the odd rule that larger buckets precede smaller buckets, so it’s unusable on anything decently sized.
Not being able to specify how ties are handled (and choosing, as far as I can tell, a fairly useless definition) fits the bill.
If it is "cursed" for a function to do what it is clearly documented to do, instead of what someone mistakenly imagines it to do, then what function isn't cursed?
> Not being able to specify how ties are handled (and choosing, as far as I can tell, a fairly useless definition) fits the bill.
Actually, you can completely specify how ties are handled—and should, in any study designed to be repeatable. The docs for dplyr::ntile tell us that:
> To rank by multiple columns at once, supply a data frame.
So, repeatable, completely specified tiebreaking is as easy as adding a tiebreaker column to the dataset, using whatever strategy makes sense for your study. For example, if we wanted random tiebreaking using R's built-in `runif`, all it takes is one extra line of code:
The explanation of "What's that extra 1 for?" in the column representation of 3-d coordinates (x y z 1) could benefit from mentioning that translation—moving things—is not a linear transformation (the origin is not mapped to itself) but an affine transformation. Therefore, you cannot represent translation in 3-d space with a 3x3 matrix. What you can do, though, is embed that 3-d space within a 4-d space fixed at some coordinate on its 4th dimension, typically w=1. Then, a translation in the original 3-d space can be represented as a linear transformation in the 4-d space and thus can also be represented by a 4x4 matrix multiplication. So the extra 1 is actually what allows all common 3-d operations, including translation, to be done via linear algebra and thereby harness the brutal power of matrix multiplication on modern computing devices.
You can do rotations and translations "via linear algebra" without homogeneous coordinates (the 4-element tuple representing a 3d point or vector), since adding two vectors is linear algebra.
What this lets you do is composing multiple transforms into one matrix multiplication instead of a sequence of multiplications and additions; that's what dramatically increases performance, on modern computing devices but most especially on ancient ones, where we were fighting for every MUL.
You don't need to use homogeneous coordinates to represent a series of linear transformations as a single matrix. That's just basic linear algebra: any series of linear transformations is itself a linear transformation, and any linear transformation can be represented as an equivalent transformation matrix.
But you do need homogenous coordinates if you want to include translations (fixed distance shifts, such as moving the camera) in the set of operations you can represent as linear transformations and thereby gain all the benefits of linear algebra, including the benefit of being able to include them in a series of operations that you can represent with a single transformation matrix.
Rick Szeliaki’s “Computer vision algorithms and applications” has a lot of great motivation for homogeneous coordinates in the first chapter and appendix. For example, in 2D, the cross product between two points in homogeneous coordinates is the line joining them, and the cross product between two lines in homogeneous coordinates is their point of intersection.
nice intuition there, this comment prompted me to consider a simpler example, 2d embedded within 3d. does the 2d plane (embedded in 3d) go through the 3d origin (where 0 maps to 0) and is thus a linear transformation in 3d but a 2d affine transform in 2d? it feels like this is the case?
No, the 2d x-y plane in your example cannot pass through the 3d space’s origin because that would imply that you fixed the z coordinate at zero. The plane must be fixed at some nonzero z because you need to be able move x and y values by some scaled version of z to make translation happen. If z is zero, that scheme does not work.
Consider a transformation f where we wish to move x-y coordinates s units to the right. In 2d, we could express it as:
f(x, y) = (x + s, y)
But that transformation is affine not linear. There is no way to generate the value s as a linear combination of the inputs x and y. So, our workaround is to embed the x-y plane into 3d space at z=1. Then we can move (x,y,1) points in that plane s units to the right using this transformation:
f(x, y, z) = (x + s*z, y, z)
This new transformation is linear: it maps (0,0,0) to itself. But it maps our embedded 2d plane's origin (0,0,1) to (s,0,1), shifting it right by s units, as we want.
The matrix form of that transformation is:
[[1 0 s]
[0 1 0]
[0 0 1]]
The same scheme would work if we had embedded the plane at any fixed z=r for nonzero r. We would only have to rescale the s in the matrix to s/r. Again, however, if r=0, this scheme will not work, as 1/r has gone to infinity.
ive been thinking -- if i have a 3x3 matrix [1 0 q; 0 1 r; 0 0 1], when dotted with x, those rows are planes in 3d with normals n1=(1 0 q), n2=(0 1 r) and n3=(0 0 1). plane 3 is parallel to the x-y plane and 1 unit up. plane 1 is tilted by q and parallel to the y-axis, plane 2 is tilted by r and parallel to the x-axis. the intersection of those planes (i.e. the solution x) when calculating Ax=b gives an output vector b sitting in 3d space at b=(x+qz, y+rz, 1*z). since we always specify z=1 we have b=(x+q, y+r, 1). in 3d this is a linear shear because we are translating proportionally by z but because z always equals 1 in this case we effectively get a translation in 2d.
so as the other 2 helpful commenters also just said: 3d shears using linear algebra degenerate to 2d affine transformations when z=1 (or w in 4d)
re-read this a day later and ofc it still makes sense to me but really most of it is barking up the wrong tree, focusing on solutions (which we already know) rather than outputs. if i could delete it i would but oh well.
Is that why quaternions are interesting for computer graphics? I studied mathematics and computer science but never understood (or even looked so hard into) the connection.
Kind of! The scalar allows vectors and scalars to be expressed together as one object, which has a lot of computational and mathematical niceties.
The killer feature is that you can put the two together for a rotation axis (~ vector) and angle (~ scalar).
With just three gimbals (rotating circles), if gimbals A and B are aligned, you only have two degrees of freedom (rotating A is the same as rotating B). Because of this, interpolating angles is unwieldy in vector space. 'Gimbal lock' confounds animation (in hilarious but unrealistic ways) but also aerospace (four hours before 'one small step for man', just after landing, Collins joked he would like a fourth gimbal for Christmas).
Another is Kalmia latifolia, the mountain laurel, the state flower of Pennsylvania. According to records at the Carnegie Museum of Natural History [1], "Benjamin Smith Barton (an American botanist in the late 1700s) wrote that in the autumn and winter of the year 1790, many people died in Pennsylvania from the effects of wild honey, collected from Kalmia plants."
Groff currently uses a greedy algorithm to format paragraphs one line at a time, but the Knuth-Plass algorithm, implemented in TeX, Heirloom troff and elsewhere, optimizes linebreaks throughout whole paragraphs and achieves a more uniform typographic grey. One of the most exciting challenges facing groff will be implementing an improved formatting algorithm, the most likely candidate being Knuth-Plass. It’s a big job, but with groff’s clean codebase and helpful community of experts, it is hoped additional developers will step forward and contribute their programming skills.
Equally important will be instituting native support for TrueType, Open Type, and other non-Type1 PostScript fonts, and improving Unicode support.
Source: https://www.gnu.org/software/groff/groff-mission-statement.h...
reply