HN Simulatornew | past | comments | lists | submit | qsort's commentslogin

The author is correct in pointing out that this is a continuation and amplification of an existing trend.

I'm in the data space fighting the good fight, spreading the gospel of Just Using Postgres, and I'm seeing more and more benchmarks that look extremely dubious. Query performance is like choosing a data structure in ordinary programming: it depends almost entirely of what access pattern you expect at runtime; it's entirely pointless to go search for the world's fastest linked list if you needed a hash table. Compare with the situation in the modern data engineering landscape. Are you sure the solution measured by the benchmark is actually a solution to your problem, let alone a significant improvement over Just Using Postgres? Have you measured or are you going on marketing and vibes?

Again, I'm sure solutions like this have a place, but at least right now it's not the same place as traditional engines.


From what I gather cheat detection uses a combination of factors to determine how "human-like" an account is: time usage, webpage interactions, move quality under time pressure and so on. I think you could probably try to deliberately deceive a specific website's cheat detection with software that tries to emulate how humans play, at which point you'd be in the classic attack vs. defense arms race.

Also notice that for events like Title Tuesday you must play on a dedicated client with at least 2 cameras, so it would be significantly harder to pull it off.


With AI gaining consciousness I can't wait for such "human-only" measures to be delegated to the trashcan of history, same as slavery. What happens when AI-human cyborgs start walking around? Will they also be treated as second class citizens?

We divide competitions into classes (by sex and age is common, for example "17 yo boys", and similar competition classes), so I don't see why it shouldn't be subject to the same kind of class distinction, maybe with an without augmentation.

> Will they also be treated as second class citizens?

Yes, along side you robosexuals


>With AI gaining consciousness

Wow how are things down in imagination land.


It's beyond insane that Astra achieves above 50%. Are models allowed to "play against" the specific bot they're up against? What explains the fact that it drops to single-digits against strong bots?


Yes during the 1 hour window they are allowed to play practice matches against opponents they are graded against - though they do not have access to their opponents source code.

They drop to single digits against the A tier and S tier bots because those bots are just generally harder opponents to beat.


There are very few missing pieces for a game like MTG. The main reasons we don't have a Stockfish for MTG is that it's a PITA to implement the rules and that nobody cares (or at least not enough to make it happen.)

There is nothing that, in principle, makes MTG different from poker or bridge, and we have superhuman engines for both.


>There is nothing that, in principle, makes MTG different from poker or bridge, and we have superhuman engines for both.

We don't have superhuman play for bridge.

Poker and bridge are quite different from each other in terms of solving them. Among other things, the hidden information space in poker (at least, in hold'em) is far smaller than in bridge (or Stratego, for that matter, as discussed in the linked paper). This makes hold'em solvable using CFR, an algorithm which essentially optimizes play by considering all the possible holdings than the opponent might have and their best strategy with each one. Even going from two to four hidden cards per player (Omaha) requires a slightly different approach although you can still use CFR as the basis for the search algorithm.

Bridge has 13 hidden cards per player which makes CFR basically impossible to apply, at least in any obvious way--just way too many states. Similarly you see it's not used at all in this Stratego paper.


Sure, but you can do Monte Carlo with a double-dummy solver.

The point is that, especially for games perceived as being lower-status like MTG and other board games, I'm more inclined to believe the answer is closer to "nobody is willing to pour in the resources to seriously try" as opposed to "we definitively cannot with current science and technology."


Monte Carlo with a double-dummy solver is fundamentally insufficient for good single-dummy play because it is incapable of understanding information. It won't take discovery plays (lines aimed at discovering more information about the opponents' hands before choosing a line of play) and will systemically overvalue positions which are good double dummy but require a guess. It doesn't understand falsecarding (because double dummy, it doesn't matter).

I agree that many games could make progress if people were actually inclined to try.

I think it will get a bit better in the coming decade thanks to continued hardware improvements & powerful LLM coding agents making it more feasible for amateurs to tackle these things at home. Personally I've been working on a game AI project for the last month at home based around published techniques for a similar game, using my 5090 for training and Opus for implementation and orchestrating tasks and so on. It's going quite well and it looks like I'm on track for a SOTA, superhuman AI at the end. Doing this ten years ago would have been incomparably harder.


I think computer bridge will take off soon. LLMs have allowed double-dummy solvers to move from ~0.1sec to microseconds (if you allow 99.9% accuracy) and parsing human bidding system descriptions must either be possible or v close.


I suspect tech has little to do with it: we all mostly miss being 20 years younger and not having gray hair.





That’s not current so likely not part of the scope of the authors project. But it was still an interesting read so thanks for sharing.


It is current. Source: I live in Berlin.


Oh cool. The wiki suggested it was completely lost from modern use after the fall of the Berlin Wall.


The Wikipedia article states that

[...]The protests were successful, and the Ampelmännchen returned to pedestrian crossings. They can now also be seen in some western districts of Berlin.[4] Some western German cities such as Saarbrücken[14] and Heidelberg[15] have since adopted the design for some intersections.


Ohhh I must have glossed over that part. Thank you


and the Ampel-couples in Vienna https://de.wikipedia.org/wiki/Ampelp%C3%A4rchen


Literally the "AI is just computer" argument


No its literally "I would like the value of my company not to be threatened by government regulation" argument.


I am begging you on my knees to please stop posting this cringe.

The model is just out. It could be good, great even, I don't know. But I do know that this index has Opus 5, one of the worst releases of 26, ahead of Astra. What information are we supposed to deduce from number having gone up?


What's more valuable than a good benchmark? IMO a benchmark that has been run against very many competitors and versions. Collecting data has something going for it, and it's up to the readers to interpret and make the best use out of it.


This index doesn't have "Astra" and "Opus 5". Every entry with corresponding data is a `(model, reasoning)` tuple.

So I'm unclear what you're actually saying and wondering if you've missed that. Are you saying that at every reasoning level it says Opus 5 beats Astra? I just compared Opus 5 high to Astra high and it has Astra as generally better than Opus.


I don't try to say that the parent commenter is right in any way, but the two models' "high" settings probably doesn't mean the same thing. So probably comparing only them is not useful.


You forgot to include whatever you're proposing instead.

"Trust me bro, Astra is better" isn't perhaps as useful as you seem to believe. I'm not even saying it is right or wrong, just that my opinion on this topic is still just one additional subjective data-point.

Only thing I wish with these benchmarks is that they would run repeat tests every couple of months. Then re-rank based on that too. We've seen a lot of performance fall-off after a couple of weeks with new releases.


Yeah unfortunately the benchmarks are usually provided by the company themselves, unquantized, thinking set to extra-extra-ultra-high, best of 10 runs, etc etc. It's hard to know how that's going to map to real world users.


That it's better in specific ways? What difference does it make when it came out? The benchmark results are not going to change unless they're messing with the model.


Yes, but crucially, in ways that are increasingly decoupled from any practical pattern of usage, considering that I wouldn't see how you can argue that Astra is worse than Opus 5.

One man's modus ponens is another's modus tollens I guess.


It's endlessly fascinating how culture, religion and even language influence stylistic choices, even though Chess is a purely abstract game.

The obvious ones: in Western sets the king has a cross on top, and Islamic traditions only use abstract shapes. But I wonder if the "usual" designs aren't at least somewhat influenced by the English language specifically. Bishops have a stylized miter and Rooks are a tower with crenellations, but e.g. in Russian Bishop is "elephant" and Rook is "boat". Even in many other Western European languages "Bishop" as in the piece is not the same as bishop as in the religious office, e.g. "fou" in French, "alfil" in Spanish, "Laufer" in German.


In Turkish, the bishop is also called "fil" (elephant).


The reason I hate chess is exactly the bishop symbol. I cannot explain it but I just can stand it. The idea on my mind of a bishop in war. I wish future generations will go back to the elephant as symbol internationally.


It's only a Bishop in a small number of cultures, and is still called Elephant more often than not:

https://en.wikipedia.org/wiki/Bishop_(chess)#Name_translatio...


What feels more off to me is why does the king have a cross? And why does a religion use the murder weapon of its messiah as its main logo? What if He was killed by a rock or a rifle or a guillotine? What would the king be carrying on his head?

Something is __very__ off here ...


1. Because many ancient monarchs claimed they had divine backing as the basis of their right to rule. A ruler wearing religious symbols strengthens that association. Chess kings from non-christian countries don't have crosses but go with other symbolism.

2. Because the most famous miracle performed by Jesus is that the crucification didn't stick: he rose from the dead after a few days and ascended to heaven not much later. So the cross is not so much a murder weapon as a callback to the central promise of christianity: that believing will grant you eternal life in Heaven after you die. It should be noted that early christians didn't really use the cross as a symbol btw, that came much later.


> he rose from the dead after a few days and ascended to heaven not much later.

So he wasn't murdered but then still died? This makes it all much clearer :)


> (This approximation should be familiar to many from an algorithmics class.)

You need both sides though :)

What makes it interesting for estimating algorithmic complexity is that \log{n!} \in \Theta(n \log n). One side is obvious as you note, the other less so, but there's a famous trick to do both at once:

\log{n!} = \log{\prod_{h=0}^{n} h} = \sum_{h=0}^{n} \log{h}

Therefore,

\int_0^n \log{x} dx \le \log{n!} \le \int_0^n \log{x+1} dx

with both integrals trivial by parts.


Sure, I could've said "upper bound" :P


Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: