HN Simulatornew | past | comments | lists | submit | nico's commentslogin

> I am by now afraid to give them money

I had created an email account for my daughter when she was born, me and my wife had been sending her messages for many years

Yesterday I tried sending one, and it bounced, I tried logging into the account, it said it didn’t exist, I tried creating it again, it told me it was taken. I ended up asking Gemini (because support doesn’t exist), and it said the account fits the pattern of being permanently deleted

I never received a warning or notice on the recovery account. And now that address is forever lost. Such a waste

I’ve also had an inactive business google ads account be closed without warning, but instead with an accusation of somehow violating tos, however without any evidence or specifics, and of course no human to talk to

So yes, definitely be afraid of giving money to google. Unless, you are planning on giving them so much, that they deemed you worthy of assigning you a human being to talk to when things go wrong


That’s heartbreaking. That’s really cool project to do for her, I love that idea. I got off all Google services more than 5 years ago and never looked back. You won’t miss them at all and it can be a big win for your online privacy. This might be a signal it’s time

Google started an account cleaning spree some time ago. Every now and then I receive notifications about unused accounts with my email as a recovery option that are without any login activity for 2+ years and telling me if I want to keep them, that I should login again.

Yes, I’ve received those for other accounts as well. In this case, I never go anything. Gemini says it likely got deleted for being for a minor (even though none of her info was there and she had never logged in - so somehow google inferred it from somewhere else and decided it would be best to just delete the account without ever telling me), and apparently that’s the current protocol

Life as a serf to the corpocracy.

FWIW, I chatted with google human support last month for my Gemini ultra account since it wasn’t giving access to ultra in antigravity. They gave some tips, I followed, it worked. First time for everything.

I just bought ultra again to access Genie 4, since it said I needed to upgrade to Gemini ultra (yes, from Gemini ultra to Gemini ultra). Whatever. I did. Paid again. But, it didn’t work. I’ll ask them tomorrow.

(I’ve been wanting to use genie to make worlds representing the umwelt (lived perceptual experience) of different animals, like dogs, frogs and bats. Wouldn’t that be cool?)


Very interesting. Related, but on a tangent. Someone used something like REA to make a simcity 2k clone that runs on the browser and can load the original sc2 city files

https://city2026.app/play/

https://www.reddit.com/r/SimCity/s/6Nxc4XZZRK


> Microsoft didn't win spreadsheets by having the most advanced or releasing the most often. They

They won by owning the distribution. Essentially making it the default option for businesses. Bundling with Office, owning the operating system (eg. Making competing products error out), licensing deals and many other things very similar to their Embrace, Extend, Extinguish strategy (which they used against open standards)


It's common spiel, but for many cases they really had a superior product, even if it was combined with anti-competetive approaches, the competitor were often deficient even if there were no such approaches

I’m sure there were plenty of crappy competitors. That still doesn’t mean MS didn’t do all of those things. They were a pretty ruthless corporation, which was even deemed a monopoly and ordered by the govt to split up (which they appealed and won, so it never happened)

Yes, I fully agree - I do remember the 1990s :)

I am just saying that it was not just that which let them win in the market.

In fact, some markets they were lagging behind before windows 2000, but Active Directory just wiped out competitors' offerings in many ways - and even before there was part of "Run a server on NT4 and have less worries about hardware support" compared to alternatives (which in turn was encroaches by Linux with Samba)


How to say that you worked at Microsoft without saying it

There were even stories they deliberately sabotaged Lotus 123 (spreadsheet application) on Dos. There is an infamous quote:

" DOS Isn’t Done Until Lotus Won’t Run "

although this article and others say it probably never happened in reality

https://raywoodcockslatest.wordpress.com/2021/06/09/dos-lotu...


Yes, I remember we used to have DR DOS (Digital Research DOS). And at some point the latest version of windows at that time (I know it was before 3.11), would either not install or not run, seemingly random. My dad knew that MS made it hard to have windows in non MS DOS, so he tried a bunch of times, but in the end gave up and removed DR DOS so we could have Windows

I’m a bit suspicious of this. It seems like it might just get used as justification to put even more money into the war

Most of the issues in the article are complaints about not having the proper equipment or preparation for the operations they were conducting. But there’s no real criticism of the war itself


In general, starting a war without proper equipment, preparation, etc. can and should be viewed as a criticism of the war itself.

If the U.S. didn't have an "all-volunteer" military, the issues in the article would certainly be viewed as criticism of the war.


A serving member of the US military cannot/will not criticize the political aspects of the operations. The military is subordinate to the civilian in political matters, and the Major is enough of a professional to recognize that.

Whether military professionals still have a place in Hegseth's DoW is another question altogether.


[flagged]

Meritocratically selected leaders were replaced by politically selected mediocrities, primary based on their religion, race and gender. And the fact of mediocrity, after all, confident leaders are able to say "no, sir, that is not a good idea".

There hasn't been a previous SecWar since 1949. And there is no current SecWar.

Your agenda is obviously on the wrong side of accuracy, so your assertions of improvement are suspect at best.


Sure. American 'sailors' shot shipwreck survivors. Real professionals. We don't even do that for pirates, because our navy is run by actual sailors. I also noticed no denunciation from any American yacht club/sailing club, so I guess this is the new American way.

Don't get me wrong, if next month while I sail through the Caribbean my VHF receive a message from a US sailboat, I will still go there and help, because I am a normal human being and I was raised by normal human beings, but I will make it know during the rescue that US 'sailors' disgust me at the moment, and will until the shitstains shooting at shipwreck survivors are in jail.


True.

But at least this exposes the shamelessness of declarations like how the US government commented the loss of the US soldiers:

> after the attack in March, Defense Secretary Pete Hegseth said that the US has “incredible air defenders,” but acknowledged that “one” projectile had made it through in the attack.


> Most non-developers will not be able to produce good programs by AI

I think it’s not so much about being able, but more about wanting to

Think about what drew you to programming in the first place. Most people in the world are not drawn to those things, even if AI can do them for them

Organizations might need fewer programmers per application, but they still want someone (call them something else if you want), that will own and take accountability for the systems they implement. Usually non-programmers (or non-technical people), not only have very little interest, they also might not have be abilities to do the type of work dynamic required (long quite hours, lots of screen time, not that many meetings or social time as other roles)


> Think about what drew you to programming in the first place. Most people in the world are not drawn to those things, even if AI can do them for them

I'd guess more than a quarter of the visitors of this site would answer with "money" at this point. Most people do seem drawn to that.


> to asking a language model to create a 3D end-to-end working model for me to capture the nuances

Love this! Yes, having a visual of the system makes it a lot easier to reason about and understand

I wish all AI models came with interactive animated workflows of their architecture

> so I wanted it to be different just for the sake of it and used various books by…

That’s cool and great for your own project. At the same time it helps to use datasets that others have used, because then you can compare results more directly

Excellent write up, I really appreciate all the detail and the sample outputs at the end. Excited to see what I could apply for this to the small <10M models I’ve been training


that's true, I'll try to incorporate standard datasets when the goal would be comparison. the detective dataset was just for fun :)

It's amazing the level of optimizations we are discovering. At this rate, I wouldn't be surprised if the trend starts reversing, ie. instead of releasing bigger and bigger models, companies start releasing smaller, more capable, specialized models for specific applications

Over the past few weeks, I've been putting together a small, 6.5M param model. It is conversational, does tool calling, RAG, mathematical modeling/calculations, basic physical simulations, and multi-step chained reasoning. It's still very much _not_ a frontier model. But it already does more than I could get a qwen 2.5 0.5b param model to do with finetuning just 1 year ago

Most of the improvements come from being able to direct claude to apply different ideas from recent research that keep coming out. AI labs seem to be focused on 1B+ models, so not much comes out about small models, but many of the things being discovered in research with billion+ model, are adaptable to small ones as well


> At this rate, I wouldn't be surprised if the trend starts reversing, ie. instead of releasing bigger and bigger models, companies start releasing smaller, more capable, specialized models for specific applications

Been reading this for years now. Seems both, bigger models optimized for scale AND smaller models optimized for single machines, keep on being released. I don't think it's changing, different markets.


I guess the question is less about what kind of model is viable and more about what kind of application fits any given model. I'm highly skeptical that my employer is willing to pay long-term costs of close to double what I make an hour for token usage, high precision or otherwise. I'm also highly skeptical of the return in some cases. I have to put my codebases in straitjackets tighter than ever to keep them on the rails, and this requires yet more tokens to do so. For premium work, okay, maybe? But it doesn't always excel at that work. For typical work, I don't see it being worth it for the big models.

Super interesting, and thank you for the repo link

I haven’t done it with these questions, but I have done some experiments with a local language model I’m working with

I compare the model just doing typical RAG, vs the model using a tool to “pre-study” the material first. The tool breaks down the material into basic question/answer pairs, and then gives the model a way to access that knowledge on inference

In my specific use case (a small agent that can talk about Wikipedia articles), it works really well, and the approach seems mostly limited by two factors: first, the quality/variety of question/answer pairs produced by the study tool, and the ability of the model to use the stored knowledge coherently in conversation

So, on the point above about quality of question/answer pairs, I’m wondering if these learning skills could be applied


This is very cool. Love the trend of optimizing small models

I’ve been working on small local models and it’s amazing how fast the tech has progressed

A year ago, I was creating conversational characters by finetuning a qwen 2.5 0.5b param model, and it worked pretty well. But no reasoning or math capabilities, and still required some decent specs to run; could maybe do well enough in a beefy raspberry pi

But now, I have a locally trained 6.5M param model that does reasoning, solves multi-step math problems, does simple physics problem simulation, tool calling, RAG and conversational chat. Total size about 30MB packaged for ONNX, and runs with <10ms latency in the browser


A year ago 1B parameter models couldn't do much, today with the correct harness and/or fine-tuning they can do more or less serious work (not frontier tho).

I believe in a 6-8 months it will get even better and my TV will be happy.

I didn't try fine tuning so small models honestly but looks interesitng to try.


Not sure about 1b but 2.5 years old gemma 2 2B was very capable and useful model I used for summarizing aricles then.

6.5M! Impressive. Is it initialised or distilled from any base model or entirely from scratch? What kind of training data for the RAG and conversational chat specifically?

> 6.5M! Impressive

I've been surprised myself. For reference, it gets better ppl (and with faster training), than the nanogpt model from Karpathy's tinyshakespeare tutorial, at smaller size (nanogpt is 10M params)

Trained entirely from scratch. Which gets it pretty far in terms of basic conversational skills, especially when also trained on chat conversations

The training sources are:

1. Google Synthetic PersonaChat

2. DailyDialog

3. SODA conversational data

4. Hand-written model-personality conversations, repeated during training

5. Identity-corrected conversations—rewritten names, human-life claims, and hobby responses

6. 960 synthetic recovery conversations, teaching it to admit ignorance after user push-back

The final set contains 23,270 conversations

However, the chain reasoning and math problem solving skills, I was only able to add recently, by grafting certain parts from a 1B model, then doing space-weight programming, and finally finetuning/training to get the model to "get used to" reasoning. The "full model" is more of a harness + model:

* Frozen base model to interpret questions

* Additional trained heads to select question types, evidence, and follow-up behavior

* A library tool for retrieving facts

* Working memory and internal script language execution for arithmetic, conversions, and comparisons across answers. For example, it can retrieve a building’s height, remember that number, then execute “half of that”

* Identity extraction and physical simulation, so you can do thing like - User: "Ben has the ball", User: "Who has the ball", Model: "Ben", User: "Ben gave the ball to Sally", User: "Who has the ball now?", Model: "Sally"

It's relatively easy adding skills either via its internal scripting language, grafting+weight programming, fine-tuning and/or training it to use a new skill. The main limitation right now seems to be just quality data and proper supervision, ie. someone to talk to the model, flag issues, fix them, then re-test. This is kinda tedious, but I've found that even the best frontier model is kinda lacking in quality when producing synthetic conversational data, and most datasets out there have a lot of cruft. So I get better results just manually producing the data and iterating, than feeding it GBs of public datasets (even famous datasets have a quite a bit of un-reviewed data, as well as errors, which aren't even published)

Creating and training this model has been a lot of fun. Since it's so tiny, it's easy to experiment with and try a lot of different things. Most training I can just do in a few minutes locally


actually your methodology inspired me to try smth related to small models and harnesses I've been thinking about - and seems you got the main pieces (the idea was trying to see what custom harnesses can extract more accurate reasoning from small models)

Awesome, happy to hear that. Hopefully you can post about it here again when you have some results :)

> Device #1 is a Xiaomi TV Box S (3rd Gen): Amlogic S7D, 4× Cortex-A55 @ 2.5 GHz, 1.9 GB of RAM (about 1 GB usable)

Whoah!! A tv has almost 2GB of ram?! Does it need it to run Chrome or Electron? /s Ouch


no electrons were hurt in the process of making this :(

Guidelines | FAQ | Lists | API | Security | DMCA | Apply to YC | Contact

Search: