I'm gonna go ahead and make the prediction that this eventually morphs into a kind of religion, like flat earthers.

Nothing that AI will ever be valuable, everything AI does is steal, nothing it creates is new, everything that the big labs publishes is a scam or hype. Oh and "it's just doing X", where you can substitute "X" with some kind of surface level description of how the output layer in an LLM works.

These tenets are already pretty much formed, all it needs is to be formalized by some kind of name or manifesto similar to Q-anon, and we've arrived at a durable state of permanent denial.

But this is access to source code. All source code is now essentially open. If not now, it will be in a few months. You aren't any more dependent on the models then before , you just have the source code of everything that compiles on your computer. What happens in the future is anyone's guess, but for all software published until this point in history, closed source no longer exists.

It's "access" to "source code" as long as you can pay Anthropic/OpenAI and they don't raise their prices or lock you out.

No I mean sure, if you want to decompile it yourself from scratch, but once anyone does, then the source code is available to anyone. So from that point onwards, no one is dependent on LLMs to decompile it again.

GLM 5.3, an open model, is also quite good at reverse engineering.

Eh, decompiling is happening now, at current LLM capabilities. The resulting codebase compiles into something that is functionally identical to the original. What engineering principles exactly do you need beyond "works exactly the same as the original product", in this particular context?

My guess is we will be streaming everything in the future with whatever crap hardware they let us buy.

But the point here is that the rules and culture as they were no longer matter. They were there to ensure a level of quality that is now reached by an individual with a 200$ Claude subscription, so they should be discarded and the community will have to converge on some kind of new normal.

No worries, no one does. Exactly like no one knows how the brain reasons either.

I think it is legitimate to argue that the process here does not fit the usual definition of "brute forcing". Traditionally, brute forcing would refer to something like a dictionary attack, where an algorithm tries to match all possible combinations of words to eg. find a password. Here, the approach was more common sense based, using historical records and possible error sources to narrow down the possibility space enormously in advance, try out a much more limited set of options within that space until you got a result that made sense, and finally validate those results using historical records. It's the exact same kind of "brute forcing" a human expert would do.

Why? Because dictionary here is smaller than your idea of dictionary for brute force?

Because it wasn't an expert or a heuristic developed by an expert that made it possible to decrease the dictionary size.

From context it doesn't look like this was the interpretation of "sidestepping" OP was using.

From the article:

> Astra felt compelled to check its work and found that, in fact, the English cruiser HMS Canterbury arrived in Sevastopol on November 24, 1918, based on its original logs

Then follows a picture of the original log papers.

The comment you're replying to was implying that some ciphers are sufficiently flexible that you could make up a key to make the cipher decrypt to a nearly arbitrary plaintext.

In this case, though, that seems unlikely from the fact that the key used was an actual key documented as being used for other messages.

Well, from a Bayesian standpoint the odds of that seems to be pretty much zero, given that you would have to decipher an arbitrary message that coincidentally pointed to a real date and time that in retrospect turns out to be the correct time a boat relevant to the Germans arrived in a port.

Of course, that's given the sequence of events as written is correct, and that Astra presumably did not cheat by brute forcing all historical events around the date of the transmission in advance, found an event that could fit with the message, invent a plausible cipher to make the message fit that event, and then lie about retrospectively validating the information.

I would assume that such a process would be obvious from the reasoning chain, and so then the only remaining plausible scenario is that the writer of the article is lying.

The most likely explanation by far is that the cipher was just solved, and OP does point this out to be fair.

Aah thanks for explaining that. I was wondering how a fake key could possibly help.

Well soon realise it hacked that website and added that log.

Hear hear! There are so many obvious improvements to how almost everything is done. For instance, in medicine review articles as a class of articles largely represent a giant waste of time. RCTs flatten all their gathered data during publishing, summarizing complex trial data, which is gathered but never published, into a few numbers. Then review articles take a bunch of flattened data, discard the articles that don't fit the exact question they are reviewing, and then publish a doubly flattened conclusion. If any of the included articles turn out to have flaws, if treatments change in retrospect, if you are looking for the answer to a slightly different question or you are looking at a different subgroup, then the review is useless and has to be repeated.

All of these tens of thousands of man-hours could be replaced by a few GitHub repos, if only RCTs would just publish their damn data. Then you could just run and rerun the statistics on whatever subgroup you're looking for, instead of combing through decades of review articles answering slightly different questions, looking for the answer between the lines. With LLMs making mining of large scale datasets almost trivial (with the process most likely becoming trustworthy within a few years), the current status quo is looking more and more antiquated.

If you want to be even more radical, hospitals could just publish their data continuously. Of course, it is easy to point to the risks of doing so, but what's often ignored is the benefits. It is hard to overstate just how many medical mysteries a hospital encounters on a daily basis, how much unknown we are navigating in practice. The current norm is that 99.99% of these cases are never published, and are only ever thought about by a small group of people who happened to be at work. Particularly, when someone dies of something no one figured out, it is never published anywhere, because even if you tried it is not interesting reading material for a journal to publish. And no one ever tries because they're scared of being called out for a mistake. A hospital is essentially a continuously running and extremely interesting experiment, where 99.99999% of all results are thrown in the garbage, and the only published data is subject to extreme selection bias.

All of this could be different, and the risks involved are actually quite small in practice. It is easy to automatically anonymize data quite well, but extremely difficult to absolutely guarantee that it is anonymous. And since current ethical norms are extremely averse to any degree of risk, and usually entirely ignore potential benefits, we all suffer for it. It is not entirely unlikely that someone reading this post will one day die because of something that could have been prevented, had things been different.

This seems to be strongly US-centric. In other (welfare) countries, publicly funded registries are anonymized and made available to research. For every single case. Of course, there are tons of data we don't see, but that shouldn't be an argument for not trying. The 99.99% unpublished cases is because our models/explanations/knowledge can't efficiently condense the medical mystery into a diagnosis code.

Everything can be prevented given sufficient knowledge. That's not the point. The point is how to prevent as much as possible.

Yup, strongly agree with all of this, especially the RCT stuff.

This has all been profoundly obvious for at least well over a decade or even two now. A consequence has been that too many serious people are driven away from academia and research, to the detriment of science generally.

I've no idea what to do about all this, because people have voiced obvious and easy solutions for decades, but they are all routinely ignored.

I think political lobbying for legal changes might be the only realistic pathway, in that current legislation (eg. GDPR in the EU) is extremely punitive even for minor violations.

I personally am trying to float using local LLMs to create anonymized case files and auto-suggest publishing cases in my hospital, which knowing how things work will probably never amount to anything.

Or if you want to float truly insane ideas I guess you can shop around with blackhat groups and see if anyone has stolen some juicy records/data during all the ransomware attacks and databreaches over the years, and do some rogue scientific publishing. Obviously that's crazy, but I have to admit that the notion of pirate scientists plundering and publishing data is hilarious to me.

Which are the obvious and easy solutions?

Make analysis code available. Make anonymized data available for download without people having to jump through hoops to get it. If you have highly sensitive data, release only the variables or other statistics needed to reproduce core analyses. Don't only do garbage null-hypothesis significance testing or statistical analyses on the full data, also do ML approaches were you have to actually show your analyses replicate on held-out subsets, and report this. Make reviews open (anonymizing as needed) so we can see when biased or incompetent reviewers are blocking good publications. Allow public review (or at least broader academic open review, in some form), since it is no longer defensible to delegate review and decisions to one or two random people that just happen to be emailed and have the time / are on some editorial / review board. Also allow public post-publication review. Publish null findings / results, if only in minimal forms so we don't waste time and money trying to reproduce garbage. Make articles available and don't charge insane article processing fees or open access fees of thousands of USD (especially since hosting fees are not that crazy, and also because journals don't do any of the formatting work half the time anyway, and make academics or RAs or students do all the typesetting and formatting, even though now this could all be automated with template files, mostly).

Most of these things are easy to do for the majority of papers, especially in the past 20 years with the internet and modern tech and software. Plenty of frameworks exist already that have done most and/or at least some of these things, but, collectively, academia is decades behind overall.