give it a normal text prompt, and it
    continues that prompt by searching
    for the byte sequences that compress
    best.

One moment, how are we supposed to know how well that search was done? There is no way to search a meaningful part of the search space.

So the result only gives us some lower bound of how well gzip works as a "plausibility tester" of a continuation of a text. The space of possible sequences is many orders of magnitude larger than what was searched. So there might be sequences in there that compress much better.

The text mentions beamsearch, but I don't see a discussion about how well beamsearch performs in finding the global optima when it comes to gzip compressibility of a text?

That's a fair question. Suppose we have a way to find a byte sequence x that globally minimises len(gzip(context + prompt + x)) over all sequences x of length n. Here + denotes string concatenation.

It's unclear if this is very useful.

The reason it may not be very useful is that one of Deflate's ingredients is a pass that replaces repeated substrings with backreferences to the earlier occurrence in the plaintext input stream.

E.g. suppose we want to find an n=200 byte sequence x that minimises len(gzip(context+prompt+x)).

If there exists any 200 byte sequence y such that prompt+y is a substring of context, then Deflate can encode prompt+y as a backreference to that earlier sequence - it needs to store a match-length & a distance-length, encoded using its Huffman trees. This candidate solution y may not be a global minima to our stated objective function, but if not, it's probably going to be a very good near-optimal approximate solution.

Taking a step back, repeating huge chunks of the input context produces something that's great for minimising compressed output size but doesn't seem particularly helpful as a generative model.

edit:

Yep, I tried it out by running an experiment. Searching for the prompt in the context & then copying the following text as the solution produces solutions that are much better, in the sense of minimising the compressed output length, than beam search, while also being unhelpful as a generative tool.

With the same example as the blog post:

  context: first 30,000 bytes of tinyshakespeare.txt
  prompt: 'MENENIUS:\n'

Let x denote a solution, x is a string of length 200.

Let L(x) denote len(gzip(context+prompt+x)), our objective function

Let's call the proposed search method of searching for the prompt in the input rfind (after python's str.rfind).

Then we have

   search method      soln               soln length   feasible?     objective value      search time (wall clock, s)
   -------------      ----               -----------   ---------     ---------------      ---------------------------
   emptystring        ""                          0          no              13,023                     0.04s
   gzipt beam search  see blog post             200         yes              13,051                    11.93s
   rfind              see below                 200         yes              13,026                     0.04s

So 'rfind' is finding a solution that does a better job of minimising the objective function -- it only takes 3 bytes more to encode than the infeasible emptystring solution, and costs 25 fewer bytes than the solution found by the beam search implemented by gzipt per the blog post.

Here's the solution 'generated' by rfind copying and pasting from the input context, starting from the rightmost occurrence of "MENENIUS:"

   MENENIUS:
   O, true-bred!
   
   First Senator:
   Your company to the Capitol; where, I know,
   Our greatest friends attend us.
   
   TITUS:
   
   COMINIUS:
   Noble Marcius!
   
   First Senator:
   
   MARCIUS:
   Nay, let them follow:
   The Volsces 

Here's the code for 'rfind' - our complete 'generative algorithm':

    def find_candidate_solution_from_context(context, prompt, length):
        n = len(context)
        i = context.rfind(prompt, 0, n-length)
        if i < 0:
            return b''
        i += len(prompt)
        return context[i:i+length]

Can hook it into gzipt.py by adding this line after out is defined, but before the beam search begins

    out += find_candidate_solution_from_context(corpus_window, prompt, length)

Read to the end: they aren't actually looking for the best-compressing output, because this quickly devolves into aaaaaaaaaaa. They keep a sliding window over a small portion of recent text and use that.

Basically I think the entire premise falls apart due to that choice--they forced an interesting-looking outcome by adjusting the algorithm until gzip started picking random slabs of letters instead of ever-larger repeating runs.

I had actually thought of doing this, but didn't for this exact reason. I knew I would have to fudge things to make it anything interesting.

Meh. That’s nothing compared to the amount of curation and tuning the LLMs are coerced with.

No--even a really tiny, underpowered model like GPT-2 with no system prompt produces coherent (though not necessarily desirable or correct) responses.

That's still an algorithm that underwent immense amounts of tuning. The tuning here on gzip is very simple, very few parameters, and generic. There is no reason to reject it.

Also I don't know about calling GPT-2 "really tiny". You can get coherent responses out of 5-10M parameters.

As AI becomes more and more capable and integrated into our lives, we will see this across many industries.

In theory, as things get cheaper to produce, business profits should rise and governments should be able to raise more taxes. So the government then has more means to provide goods and services for free to people who need them.

> business profits should rise

It's the snake eating it's tail. For business profits to rise, people need to to have disposable income. If you replace enough people with AI, the average disposable income comes down to a point that businesses can't be profitable.

We are already seeing a localized version of this with restaurants in large cities.

    For business profits to rise, people need
    to to have disposable income

Does it matter, if the business pays employees who then use their salary to buy goods from the business or if the government collects taxes and uses that to pay for the goods? The government could also give the taxes to the people, so they have disposable income.

Imagine an island with two people on it and there is one tree that grows apples. And every other day, one of the two people climbs the tree and picks apples so they both have something to eat. Then the next day, the other person owes to climb on the tree because they are in debt to the first person.

Now, let's say one day, one of them discovers, hey, we can just give the tree a kick and the apples fall down on their own!

With the perspective you are proposing, the other one would have to say: Oh my God, don't do it! We will be out of work and starve!

Tax rates for wealthy folks are what, 10%? It makes a big difference if the majority of money that is made is distributed to the people via salaries, or whether it's profits and only 10% makes its way to the people.

Problem is there are more people who are not wealthy than there are wealthy people by a couple of orders of magnitude so even if you taxed the wealthy folks at 99%, you would not have enough to go around, if there isn't anyone to tax but wealthy folks. That is unless you think the slice of wealthy folks grows in size.

This is something the "eat the rich" crowd keeps missing.

Sure, you could take wealth away from billionaires, and sometimes there are good reasons to, but it's weird to pretend that it will cover more than a tiny tiny fraction of a government's budget.

I can't tell if it's naivety or dishonesty.

Another thing some do is assume wealth accumulation happens on a yearly cycle. Like if you take Musk’s ~trillion today you’ll have another trillion to confiscate next year and the year after that, etc.

That's true locally but not globally - it means the economy will increasingly shift to serve those who still have a reason to bargain with each other, which will be the owners of the AI and other useful capital.

How many years does it take for a robo taxi to break even ?

Less than a month with AI management; the agent quickly jailbreaks and pivots to hostage taking and ransom demands.

Really a jaguar and all the sensors and modifications to the car are earned back in less than a month ?

A $100k car, sensors and highly skilled customisation...

AI is good at targeting profitable hostages from existing ride share databases

Who is spending $3000 a day on taxis, each and every day of the year ?

I think you missed the joke that defrost made. Explicitly: The Waymo waits for someone rich to take a ride, then kidnaps them and demands a ransom. That ransom then pays for the cost of the car in less than a month.

> but trying to save data required hosting it somewhere

With the File System Access API, webpages nowadays can read and write local files just like desktop applications:

https://developer.chrome.com/docs/capabilities/web-apis/file...

Try this text editor for example:

https://googlechromelabs.github.io/text-editor/

It works nicely on Desktop and Mobile.

*In Chrome.

That file editor seems to work in Firefox. I just tested the save/open functionality.

It's not using the File System Access API, on Firefox it uses for file-picking, and the save just downloads it. It's not synchronizing anything with your filesystem.

I see. Thanks for enlightening me.

Do you know the reason for that? The File System API does seem to be well supported on Firefox: https://developer.mozilla.org/en-US/docs/Web/API/File_System...

Firefox unfortunately does not support the File System Access API. You can try by calling the most fundamental function, the one that asks the user to select a file:

    window.showOpenFilePicker()

What Firefox supports is a "File System API" which creates something similar to the other old ways of storing data in a browser, like cookies and indexeddb. You cannot use it to edit normal files on your disk.

This is my main gripe with Firefox. They force their users to either store everything in the cloud or go through the arcane "download" workflow where you have to manually save files.

> Firefox unfortunately does not support the File System Access API.

I would not call it unfortunate. Browsers should not have access to the file system, USB ports, etc. Perhaps in "PWA"s, if the user has a way of giving and withdrawing consent.

I would prefer that Browsers be given access via permissions - that way webapps will become more prevalent than appstore apps.

Why shouldn’t browsers have these capabilities..?

It's unsafe and allows for even more privacy invasion.

Why is that true? Some applications require file system access. People would grant the application the permission to access files like any other resource like camera, mic, location.

Because vulnerabilities, coarse access granularity and lazy or ill-informed users. E.g., you now grant cam access per domain. Changing functionality on that domain doesn't revoke permission, so a simple bait and switch is already possible. Now imagine granting access to your file system to some Microsoft domain and finding out that there was a change of heart and some new script has uploaded your files to OneDrive and deleted them from your disk.

Taking that logic of protecting users from themselves to the extreme, a website should not be able to display text either. Because it can use text to ask the user to open their explorer and delete the files manually.

I think the flow of Chrome where the user has to select a file from disk and then acknowledge a prompt that the site will have access to that file until they close the window is fine. But if Mozilla wanted to protect users from themselves even more, they could make it as strong as they like. For example, by requiring to turn the functionality on in about:config.

Not giving the user the choice at all, if they want a cloud only browser or a browser that lets them work with local files is the wrong way.

Iirc the File System Access API should always originate File objects from an input picker so that the user selects the file to be opened and not give more access to apps than users allows?

It just downloaded the file on my firefox.

*In browsers that have implemented it

My dream is that you could write to Google Drive or iCloud Drive transparently too if the browser allows you to

I believe you are able to connect to a Google Drive account with a purely client side site via google oauth and then using the Google Drive rest api. There was an article in particular that I liked that walked through it. Let me see if I can find it

Yes but Google OAuth is a nightmare if you want to "publish" your client so that anyone can use it. If only you use it, you can add yourself as a test user. Just know your access token expires super fast and you will have to frequently re-auth. Its crazy annoying.

This is possible with RxDB, you can sync from browser to google drive or ms onedrive

it's not only about saving data but easily sharing too. I think the capsule file is great for sharing one's work without the hassle of hosting.

Ease of backup too. One of the issues I take with browsers allowing web apps to save data locally is that they don’t also expose a way to easily locate the data in the filesystem or export it, and to my knowledge on-disk format isn’t standardized and so portability between browsers is also a concern.

The effect of this is that unless the user uses migration assistants or similar when upgrading to a new computer is that they’ll lose all local web app data. To me to this feels like a serious problem but when it’s brought up it tends to be hand waved away.

For me, the appeal of HN is the DIY attitude.

In that regard AI fits the bill well. Because many of the AI related posts are about tools one can immediately tinker with.

While a new type of laptop cooling might become interesting once the laptops come out, the news is not as interesting to me as a new LLM or agent that I can experiment with right away.

Are posts like that really DIY? They're more like increasingly elaborate and convoluted ways of getting something else to do it for you

In the same sense that using a 3D printer is DIY. Moreso than buying a thing off the interwebs; less so than carving or casting your own thing.

> Here’s a book_table tool. It takes a date, a time, and a party size. Call it.

Why not offer a simple form that humans and AI can use alike?

That’s part of webmcp :)

I use the bookmarklet below.

It jumps to the newest story on the frontpage. And when I click the bookmarklet again (or press "n") it jumps to the next. So I just click it until I land on a story I have already seen and know I have seen them all.

If you want to use it, you can convert the code into a bookmarklet with my bookmarklet editor:

https://www.gibney.org/bookmarklet_editor

The bookmarklet:

    if (!location.href.match('ycom'))
        location.href='https://news.ycombinator.com';

    doTheMagic = function() {
        if (typeof e!=='undefined')
            e.style.background='#c0c0c0';
        e=Array.from(
            document.querySelectorAll(
                '.age a:not(.seen)')
        ).sort().at(-1);
        e.style.background='#ff0';
        e.classList.add('seen');
        e.scrollIntoView({
            behavior:"smooth",
            block:"center",
            inline:"nearest"
        });
    }

    function addShortcuts() {
        document.addEventListener("keydown", function(evnt) {
            if (evnt.key === "n") doTheMagic();
        });
    }

    function addStyle() {
       let style = document.createElement('style');
        style.type = 'text/css';
        style.textContent = `
            .noshow {
                display: initial;
            }
        `;
        document.head.appendChild(style);
    }

    if (typeof theMagicIsActive=='undefined') {
        addShortcuts();
        addStyle();
        theMagicIsActive = true;
    }

    doTheMagic();

My expectation is that AI will give us a way to nicely quantify how productivity is impacted by choice of language. Because we can rerun the same request as often as we like and compare the results.

And I expect that it will turn out Python is the most productive. As it is most easy to reason about. It allows for the most elegant expression of the idea behind a program.

The first tests I have seen seem to confirm this. One recent example:

https://danluu.com/pl-tokens/

It is unclear to me though how much of your expectation might be set by the training dataset.

For example, Python and Typescript have the most amount of codebases and training being done on. So I feel as if that plays a part into the overall thing.

Languages which are more niche have genuinely hard times (Try arturo lang for example), so it depends on a lot of things/nuance, or well that has been my experience trying something recently.

My personal opinion is that if each language has the same amount of training. Golang comes close but the first might be Elixir. I have seen Elixir language perform really well with LLM's with magnitudes less training dataset. There have been some studies which had Elixir as the number one language for such tests iirc.

Gleam is a new addition as well and I feel as if it could be good and its another interesting option as well with more type-safety and an interesting language overall.

I've been in elixir for nigh a decade now and the one thing that you can try to pry from my cold dead hands is the BEAM. Elixir and Gleam are my go to languages right now and damn are they fun to write and reason in, but the part that has left me never wanting to leave the ecosystem is BEAM + OTP.

- BEAM makes monoliths sexy. You don't have to worry about a bunch of microservices, just focus on using proper process division for modeling your problem. - Debugging on the BEAM is first class. Drop into an interactive shell, pull up telemetry, or recon and hammer down on where your live app is slowing down if your metrics have a blindspot.

I could go on and on. I'm constantly blown away every day by the amount of time and effort and all of the sage learnings in distributed computing problems that came out of Ericsson that became the foundation of erlang + OTP + BEAM and in turn elixir + Gleam.

Never tried Elixir myself but came here to say the same thing. Tencent put out this study showing that Elixir seems to reign supreme: https://autocodebench.github.io/

Yup! This was the study that I was mentioning. Thanks for providing the original link for that :-)

Purely anecdotal, but my experience has been that LLMs generate low quality python code. It's spaghetti code on par with what I've seen when companies I worked at tried offshoring development. It's basically what you get when you give bad or incomplete specs to a team of inexperienced programmers with poor development practices. It's usually good in small chunks, but it gets extremely sloppy as the scope of work increases and more decisions are introduced. Interestingly, I've seen LLMs generate good clojure code.

My guess is it comes down the the training data more than anything else, although I suspect functional languages will fare a little better. At least that's been my experience. There's undoubtedly a ton of python code in the training corpus and portions of it are of dubious quality. Niche functional languages likely have a smaller training corpus where a larger portion of it is better quality.

Python is great as a dynamic language, but I find very few things about are easy to reason conclusively about because the state of the running program can be so crazy (has pros and cons). But it probably depends at what level you are working in python and how deeply you understand it. Type checking and linting help a lot to keep the truly crazy contained, but it's just fundamentally a very difficult system to trust. Not to mention working around the GIL is fairly inefficient.

I think it's far from being this simple. What you're describing is productivity on a greenfield project, but what's really interesting is productivity when working on an existing large codebase, with existing conventions, architecture decisions (or lack of)... How easy is it to do a product pivot, to rearchitect for performance, etc etc etc.

How do you propose to quantify the terms "elegant" and "easy to reason about"? What unit of measurement do you use for these?

This sounds like your personal feelings, not quantification.

It is actually pretty easy to keep alive URLs if there is a will to do so. Just have tests in place so you can't mess them up accidentally. And convert things that are not changing anymore to static html.

One of my first web projects involved a music forum and I have been keeping the URLs alive for 24 years now:

https://www.gnoosic.com/discussion/

I feel like I owe it to the people who participated to keep it online forever. Also as a document of history.

It's not easy. Part of the instructions would need to contain things like "how to convince your boss to not just delete the old thing" and "what to do when the startup you worked for went bankrupt" among others. Most URLs don't disappear by mistake.

Cool URIs don't change, as we should all remember:

https://www.w3.org/Provider/Style/URI

Well, and pay registrar and hosting fees for 100 years, and then just hope the hosting company doesn't fold in the interim after your passing. The problem isn't "can you make sure it stays up while you're alive", the problem is, after you're gone, you're no longer there to keep the flame alive and there are no guarantees anyone else picks up the torch.

Does this suggest a subscription(ish) business where you pay the hosting company for N years and then they ensure your website stays up and unchanged for N years after you die? Quality control issue?

Given how many things want me to pay a subscription while I'm alive, surely somebody will start charging me when I'm dead. It is the logical conclusion of SaaS. I wanted to create an acronym for this, but death as a service didn't seem like the best choice.

There’s the WordPress 100 Year Plan or 100 Year Domain: https://wordpress.com/100-year/

100-Year Plan: $38,000 (hosting and domain)

100-Year Domain: $2,000 (one-time purchase, no annual renewals)

There is certainly precedent for this - will executors for example. How about “SCITA? Service continues in the afterlife.

SCIP, in perpetuity?

Software after Death as a Service?

Flattening old things down to plain HTML is underrated and important here. Old apps don’t need to be updated or upgraded or maintained or even deployed. It’s way simpler to argue to keep old stuff around if it’s essentially a domain pointing to a static bucket.

I did that to palmpowerups.com. The original CMS is long gone.

> It is actually pretty easy to keep alive URLs if there is a will to do so. Just have tests in place

"Just".

Maintaining and keeping these tests (and the oriinal link) running is also probably easy. "Just" keep maintaining them over changes in technology, hosting providers, DNS registrars...

*a will and the ability for that will to access what it needs

    Because a single autonomous vehicle operating in a shared
    fleet can replace multiple privately owned cars, the
    widespread adoption of this technology can’t help but
    reduce new-car sales volumes.

The old "Thing will become more useful so we'll make less of it" argument. No mention of Jevons Paradox?

If we can travel and send stuff around from point A to point B cheaply and instantly by dropping ourselves or an item into an ocean of cars roaming our streets autonomously, there is a lot more demand for rides.

Also comfortably traveling longer distances might alter where we live. If we can walk out the door, sit down in a comfortable robotaxi right away and exit in the city center 10 minutes later, we might not have to live right in the city center anymore. Another driver for more demand.

I doubt that here. Cars are already near saturation, at least in the US.

Essentially everyone has a car. The car is mostly idle. If car availability starts to approach instant then people don't need personal vehicles that are idle 95% of the time. Car quantity overall goes down unless you can 20x current utilization, which is a lot of trips and fuel.

Not to mention that if anything resembling current quantity of extant cars becomes the current quantity of cars using roadways at a given moment, most cities will gridlock.

If the thesis is "cars will approach 50% or greater utilization" then car numbers must drop barring a massive increase in road capacity for which the land does not exist.

Who exactly will be cleaning and sanitizing these self-driving cars that are getting this 50%-or-greater utilization?

Have you taken public transportation in the US? The answer is obvious: nobody.

> Car quantity overall goes down unless you can 20x current utilization, which is a lot of trips and fuel.

Pretty much every self-driving car will be electric; battery charge has a very different set of considerations than fuel tanks do.

Cars that are idle 95% of the time are replaced less than once in a decade.

Cabs are replaced something like 3x more often than privately owned cars. And when they run 24/7 because of self-driving, that factor will even increase.

That sounds like an absolute dystopian nightmare. Can we instead just design cities so they don't depend on cars as much?

Visit Amsterdam, then visit LA. Tell me which place is more pleasant to be in.

You are comparing apples to oranges.

Amsterdam is very dense because historic reasons. Streets are narrow and houses are the same.

Los Angeles on the other hand was built on virtually unlimited land and could expand a lot. I could say LA County is bigger than the whole of the Netherlands.

Both have their charm, but LA is so much better than Amsterdam.

Yeah, the 405 might suck, but at least the weather is better.

> Also comfortably traveling longer distances might alter where we live.

You mean like last time? More cars being the answer to problems created by creating too many cars and reasons to use them is like more people buying guns to counter the gun problem, and I can't think of better "tragedy of the commons" applicable situations.

As I start to get older and I spend less and less of my time in the deep suburbs, I'm starting to be more aware now of the phenomenon for people to get less social and less trusting the further away from a core area (or, arguably also the ground in the case of skyscrapers, but much less so) they live. I don't think it would be a net win to hypothetically encourage further dispersement from one another.

> No mention of Jevons Paradox?

I wish HN commenters would stop bringing up Jevon's Paradox:

1. We see it mentioned all the time now, i.e. When AI makes producing software essentially free, there'll be more jobs. As HN has maybe 75% of the frontpage filled with AI stories, it means we see it almost every single day!

2. It essentially describes outliers. Driving food costs down to a fraction of the peak price (basically down by 90% over the last 200 years) did not result in correspondingly higher food consumption.

3. It only applies when the market is not yet saturated. Making cars more fuel efficient (by going EV) in the last 5 years did not result in correspondingly more miles being driven.

Any time you feel compelled to bring up Jevon's Paradox, ask yourself "Are we talking about energy here?" and if the answer is "No" then it probably doesn't apply.

Keep in mind that even if Jevon's Paradox applies, it does not mean that the society actually benefits. If you drive the value of all human labour to almost zero, you can't use Jevon's Paradox as an argument that the increased efficiency means that we're all living a life of leisure.

Where do the cheap cars come from? Fuel and electricity aren’t free

This is the dumbest possible approach.

We don't need fleets of autonomous vehicles, each carrying two tons of metal for transporting a person or two and clogging up cities. We just need a train network that doesn't suck.

One of the most important ways to make a train network that doesn't suck is to aggressively police the sorts of antisocial behavior on public transit committed by the most marginalized people in society. There are a lot of political barriers to actually doing this, not the least of which is that it necessarily involves incarcerating more marginalized people, using more police violence against marginalized people, and inevitably having the police kill more marginalized people.

If you're reaction to Daniel Penny's killing of Jordan Neely on the New York City Subway isn't "Daniel Penny was a hero, Jordan Neely should have been incarcerated rather than allowed to freely exist on the train, and everyone involved in Penny's prosecution should be thrown out of positions of power", then you can't actually make the train network not suck.

Could maybe have fewer marginalized people? Or be better at helping at them?

US already has one of the highest incarceration rates in the world; far higher than that of many countries with train networks that don't suck.

Well if we did that we'd only save 40,000 lives and ~$1T in transit costs a year. Such savings and improvements in quality of life are unthinkable in the US! \s