It sounds pretty nice on paper. But I tested the huggingface.co/mradermacher/hybridagent-8b-agentic-gguf:IQ4_XS with a simple question: "Who is the pope?" and it got stuck in a thinking loop forever and simply could not finish the task.
```
Okay, the user is asking who the pope is. The pope is the leader of the Catholic Church. The current pope is Pope Francis, who became pope in 2013 after Pope Benedict XVI resigned. There's no indication of a change since then, so the answer is Pope Francis.
Okay, the user is asking who the pope is. The pope is the leader of the Catholic Church. The current pope is Pope Francis, who became pope in 2013 after Pope Benedict XVI resigned. There's no indication of a change since then, so the answer is Pope Francis.
Okay, the user is asking who the pope is. The pope is the leader of the Catholic Church. The current pope is Pope Francis, who became pope in 2013 after Pope Benedict XVI resigned. There's no indication of a change since then, so the answer is Pope Francis.
```
Then I also gave it the counterfeit ball problem, but it could not solve it either. But to be fair, no 8B model seem to be able to solve it.
```
I have 9 balls that look exactly the same. But I know one ball is different. I only have a scale. Call scale(b1,b2)={-1;0;1} will tell me whether the left object b1 is heavier, same weight or lighter than the right object b2. How many times at minimum do we have to apply scale to find the different ball? Explain how we can achieve this.
That looks like maybe a driver problem though? Are you sure the inferencing stack is properly recognizing an EOS token? New models often have weird bugs like that that require changes to the engine.
Have you tried increasing the repetition penalty? Needing to do this is already a red flag, but I'm curious if it helps at all.
Running LLMs locally on outdated hardware, I've come to the realisation that sub-10b models are broadly speaking kind of stupid. They can answer simple questions, but can't really solve problems.
The effect is so strong that with an 8GB GPU a 27b model can sometimes finish the task earlier than a 9b model, despite residing largely in RAM. If it's a MoE model, then it isn't even a competition.
Yes. I don't think this formulation of 9ball is solvable in less than 9 weighings because it doesn't allow you to weigh groups of balls. A set of balls is not an object, so it can't be put on the left or right side of the scale
If I was put in place of the LLM I legitimately couldn't tell if the intended solution is to ignore the wording and solve 9balls, or if this is a trick question to check if I spew out a memorized answer without considering the actual wording of the problem
I think you can do it in 5 weightings, because a 0 eliminates both balls. Worst case:
scale(1,2) = 0
scale(3,4) = 0
scale(5,6) = 0
scale(7,8) = -1 # If it was zero, the counterfeit would be ball #9.
scale(6,7) = 0 # Must compare to a known-good ball. Here #8 is counterfeit.
```
Okay, the user is asking who the pope is. The pope is the leader of the Catholic Church. The current pope is Pope Francis, who became pope in 2013 after Pope Benedict XVI resigned. There's no indication of a change since then, so the answer is Pope Francis.
Okay, the user is asking who the pope is. The pope is the leader of the Catholic Church. The current pope is Pope Francis, who became pope in 2013 after Pope Benedict XVI resigned. There's no indication of a change since then, so the answer is Pope Francis.
Okay, the user is asking who the pope is. The pope is the leader of the Catholic Church. The current pope is Pope Francis, who became pope in 2013 after Pope Benedict XVI resigned. There's no indication of a change since then, so the answer is Pope Francis.
```
Then I also gave it the counterfeit ball problem, but it could not solve it either. But to be fair, no 8B model seem to be able to solve it.
```
I have 9 balls that look exactly the same. But I know one ball is different. I only have a scale. Call scale(b1,b2)={-1;0;1} will tell me whether the left object b1 is heavier, same weight or lighter than the right object b2. How many times at minimum do we have to apply scale to find the different ball? Explain how we can achieve this.
```