> [I give a small introduction to myself and my own work, that I’m omitting here, as you’re on my website]
kkukshtel: You should keep that info, with links, even if its links to your site. Dont burden the reader.
Edit: I read the first half of the article then skimmed the rest (I dont use LLMs outside of occasional searches).
I've been a gamedev for five years. I have a hunch LLMs will not take over game development because of the necessity to control many, many small details to shape a great experience for the player, and because some parts of development cannot be described with words.
Small details: My hunch is when it comes time to polish and fine tune all the moving parts, this process will remain much faster to do by hand. It will be faster to update a specific field or line of code and test/iterate than to prompt your way through that process (the last 20% of the work takes 80% of the time)
Describing without words: An artist with a vision can only bring that vision to life with body movements. It is not possible to communicate all the details that go into a great piece of art with words. The colors, lines, shapes, positioning, specific emotions, history... is beyond spoken language.
You are talking from a perspective of games as a medium for an artist with a certain bias and an audience that is interested in this specific bias.
You say "vision, colors, positioning, etc", yet you have Candy Crush being the most played game on mobile for 13 years which simply employs direct signals and ques from cognition and cognitive neuroscience to achieve this.
The games that you describe are a small subset of what the general public is playing or is interested in.
This can further be pushed into a monoculture by shaping the players expectations about what a game should be. Maybe Skyrim has too much walking and dead-time, it's better to optimize for fast combat and action, like in Fortnite, and mold the playerbase in a way which makes them susceptible to simpler games more tuned to neurophysiology.
Even Candy Crush, a huge part of what made it successful, was the tiny details. The bright colors, the art, the subtle and not-so-subtle animations... An LLM can technically do all that, if you keep reminding it, and have strong rules, and constantly keep it in check, but there's no way it would do it on its own and keep it consistent and coherent.
Eg, if the tall green pill candy and the red round one both scale by exactly 3px when you drag them, it might still "look" wrong, and get into uncanney valley territory. And that's even assuming the LLM picked the same value to scale when you prompted it to "add a red candy", or it picked a different one arbitrarily.
I've tried making games a few times, or even CSS animations, and I can tell that they don't "feel" right, even if I have no idea why, or what to change. I have mad respect for artists and animators that can manage that, and I have no idea how you tell an LLM "It doesn't feel right when I drag the candies", and have it be able to figure out that a tall vertical candy needs to scale differently on the X and Y axis than a round one.
The OP is absolutely not talking about making Candy Crush. They ARE talking about making art. Both can exist in the same world and be successful. The existence of and success of reality TV doesn’t negate the art of filmmaking and the enjoyment of watching a movie. Same with games.
OP is invalidating the blog with the reasoning of detail, art, taste, etc is necessary for a good game.
I am saying that a small subset of people care about this given what people choose to play in practice. It's currently much, much easier to not care about these things and focus on the cognitive aspects to capture player attention, and the human cognitive incentives are not stacked in favor of OPs view.
> I am saying that a small subset of people care about this given what people choose to play in practice.
1. Candy Crush is one of the highest grossing games of all time, but if you look at the other top 20 highest grossing games, the vast majority are nothing like it.
2. Janky crush wouldn’t have been the success it was without massive attention to detail and polish.
Also there is a huge effort to optimise the game for retention, LTV and UA arbitrage.
These are not the fun details but they are details none the less and whilst LLMs will probably be more useful here the insights that drive their actions are the taste humans provide.
Candy crush is not the first of its genre, yet it is the most successful. What made it stand out from its bejeweled predecessors? Vision. Colors. Positioning. Animations. Smooth movement. Feedback. Tighter gameplay loops. Sense of accomplishment. All of the above
What are choices of vision, colors, positioning, feedback animation, sounds, etc, if not the exact things you call "employing direct signals and ques from cognition and cognitive neuroscience"? The former is the execution of the theory of the latter.
>which simply employs direct signals and ques from cognition and cognitive neuroscience to achieve this.
Does anyone actually do this or is this all post hoc rationalizations on why certain things work? My experience and understanding is that it's all just a bunch of "let's try lots of things and then do more of what did well". Ie it's the result of a crazy amount of iteration rather than "employing cognitive neuroscience".
Candy crush has a team of hundreds of people, it's far from a simple game. I know for a fact they had individual product managers for certain features inside the game.
> My hunch is when it comes time to polish and fine tune all the moving parts, this process will remain much faster to do by hand. It will be faster to update a specific field or line of code and test/iterate than to prompt your way through that process (the last 20% of the work takes 80% of the time)
I think you're mostly right here -- however, I think you've framed prompting as an all-or-nothing (or maybe the OP did, as I only skimmed the article too...). LLMs are great for some things (scaffolding, review, refactoring, etc) but way less efficient at others (tweaking and letting the artist eye shine).
I think the crux is this: you shouldn't be asking an LLM to make these tweaks for you; you should be asking an LLM to build the tools that let you make these tweaks by hand faster or more effectively.
For example, we have probably a hundred tools [0] for quickly making all kinds of changes and tweaks in one of our Steam games, that are of course all done by hand. The tools were built by LLMs based on what would speed up our existing by-hand workflows, and once they're built they're just tools that let our people shine.
1. Do a thing by hand
2. Have LLMs build a tool that makes it easier to do that thing by hand
3. ????
4. An artist with a vision can bring that vision to life faster
I've found it to be useful for prototyping. You can throw a game mechanic up, have it build some AI players, tell you any balance issues it runs into, play against the AI yourself to see if the mechanic feels fun. Then iterate.
A few times I've made a tweak to a mechanic and the LLM balance tests immediately found a major issue. Sometimes the AI players it creates just suck at the game. You still have to think, but just having balance tests and then having them cheap to update when I experiment, it extremely valuable. I tested this on a game I had mocked up with cardboard and forced my friends to play. It found all sorts of stuff I missed in those gameplay tests. It didn't replace those sorts of tests with humans, but it let find issues faster so I got more out of human interaction tests.
When I'm using an LLM to program these days, I increasingly feel like I'm directing one of those renaissance-era painting schools that were sort of like painting-factories, where some Great Master would direct the output of a dozen or more painting-technicians to construct unreasonably-large and detailed paintings on unreasonably-brief time-scales.
They would still come in at the end of the day to add their brush strokes to pieces of the canvas that they thought needed extra attention. We still refer to those paintings as if they were made by the Great Master, and justly so because their artistic imprint is felt in the final work.
In the same way where if you and I were asked to write the same game with an LLM, they would come out looking pretty distinct from each other. And if we wrote them by hand, they would be distinct from each other and the LLM-assisted ones again. All of them are still art and games, though, it'd be silly to call them anything else.
I agree with you that polish and fine-tuning is faster to do by hand when you are well-appraised of the foundations. I think a lot of the challenge of developing Real Software with LLMs today is maintaining the appropriate proximity to the work, when we are no longer holding the paint brush ourselves, so to speak.
I'd encourage you to actually run this exercise! I think you'd be surprised.
I've proctored some interviews where we give people an LLM, and a fairly open problem (similar to "Build a game where you do X").
The first couple of iterations are a similar, but it diverges pretty quickly depending on where the human is focusing their attention. The logical bugs of different subsystems are a bit predictable, but the final results pretty reliably contain the "fist" [1] of the human.
Of course they're similar in the ways that all software are similar. But every human is knowledgeable and/or cares about specific things more than others, and that absolutely shows up in each iteration final product.
"My hunch is when it comes time to polish and fine tune all the moving parts, this process will remain much faster to do by hand"
Depends if what you want to do is new, or has been done a thousand times before
"Describing without words: An artist with a vision can only bring that vision to life with body movements. It is not possible to communicate all the details that go into a great piece of art with words. The colors, lines, shapes, positioning, specific emotions, history... is beyond spoken language."
And you absolutely can use sketches, pictures, drawings as input to the AI.
So in general I agree, at least right now it needs a lot of fine tuning. About the future I make no prophecy, I did not see AI coming this way, so why would I be right about AI in the future now.
I am making a novel game right now with heavy help from Fable and it is awesome. Because even though my game is novel - parts of it are not, but standard and I reinvented the wheel way too often before and am glad don't have to do that anymore.
LLMs will absolutely take over game development. That does not mean the result will be a fun game that has value. Just like a group of expert humans can put together a technically and artistically impressive game but it has little appeal.
Taste remains king. Despite that. LLMs will absolutely be able to create elaborate games by this time next year.
> I dont use LLMs outside of occasional searches
> I have a hunch LLMs will not take over game development
Not easy to make statements about things you don't use.
If you're developing software, LLMs will be ubiquitous eventually.
They aren't going to tune the turn radius or how long to hold a freeze frame but they will bang out a UI to let you do that quickly with quick turn around times.
So many simple little things that are helpful but not really worth doing will now be worth doing because the implementation cost is a fraction of what it once was.
I honestly didn't except this to get traction on HN so I'm updating the blog post now. I just read some recent post too that was like "mistakes technical blogs make" that I think brought up this same point yet here I am! Thanks for the tip.
For those of you who grew up with the city builder classics, I just announced Jerry Martin (SimCity, The Sims) and Paul Zimmermann (Space Haven, Wildfrost) are the two composers I am working with.
That's an incredible get. As someone who grew up with SimCity (DOS), I thought I'd been hurt too many times (CS:2, SC2013, [..]) to get my hopes up again, but here we are. Wishlisted.
I've heard a quick way to determine if the UV index is dangerous is if your shadow is shorter than you. Though location matters too.
I moved from Vermont to Texas and had to purchase an umbrella with UV protection to walk around during the day. The highest the UV index hits in the Vermont's summer is 6. In Texas, it's 11+ !!! 15 minutes and you're toast
From conclusion of paper, I read the opposite of what you state. "A better strategy would be to expose a larger fraction of the body for a shorter time period, preferably when the sun is lower in the sky to allow a greater margin of error."
A sun lower in the sky means your shadow is longer. Maybe I'm confused?
Good catch, it is kind of confusing as they seem to contradict themselves, and also depends on the season.
Figure 6 shows that for full body exposures, there is a wide window between the time for sufficient UV and the time for too much UV. As the fraction of body that is exposed decreases, the window of optimum UV exposure times also decreases. If only hands and face are exposed, there is generally only a small window between receiving insufficient UV for vitamin D production, and too much UV for skin damage (erythema).
As the UVI becomes smaller, it becomes more difficult to produce sufficient vitamin D without inducing erythema. For low sun conditions typical of midday in the mid-latitude winter (SZA ∼65°), where R∼1, the skin area exposed must be more than 12% (see Eq. 5) This means that for these low sun conditions it is improbable that sufficient vitamin D can be produced from exposing the hands and face alone, since that area is less than 12% of the full body.
If you are able to expose a lot of your body at a time then yes later in the day is workable (in the summer). In the winter, (depending on your latitude) you still need mid day sun to make any meaningful Vitamin D.
In some parts of South America the UV index reaches 15-20 due to near-equatorial latitude and extremely high elevation. It is strong enough that you need protection from indirect UV radiation.
I'm openly (very) curious to hear the reasoning behind that statement, not having lived in SF.
From what I've read, George Lucas funded 100% of the project, and the Presidio trust rejected it. Then he tried building the museum in Chicago, but he was sued and blocked for a couple years and eventually gave up. Finally he found a spot in LA. IIRC the project took 12 years. The same source mentioned SF hasn't started any major museum projects since 2016.
America is falling behind on public works. We used to have the largest aquarium in the world, and now we're sitting at #5 or #6.
So I am genuinely curious why you and others are opponents of public works projects? I honestly do not understand.
The presidio is huge though and has a ton of old military warehouses? Perhaps it could have taken one of the major commercial buildings already there. Is sports basement warehouse located entirely in the presidio or any of the other major commercial enterprises there really a better fit than a cool museum.
Perhaps it's getting more difficult to move the 'goal posts'?
- Train LLMs on texts -> OMG AI can write!
- Train LLMs on code -> OMG AI can code!
- Train LLMs on images -> OMG AI can generate images!
- Train LLMs on video -> OMG AI can generate videos!
- Train LLMs on math -> give free access to 100,000 researchers/scientists/mathematicians -> invest resources into Lean -> OMG AI can solve math problems!
I say 'goal posts' because that's a common argument, but the perspective that seems to not be considered is that the LLMs are doing the same thing over and over again in different domains while be fine-tuned by labeling and human review.
The emergent behavior argument is a separate consideration, but I can imagine an AI being given a list of words (a harness) that guides how it operates--not the other way around--and the difficult problem is narrowing down and filtering all the associations with the words/harness to properly align it.
And perhaps the recursive looping being discussed creates incredibly difficult (impossible) to predict behavior? If the AI could modify it's own harness, is that AGI? If we fully understood what happens in high dimensional space, and could predict the output of any combination of input tokens, then is it not AGI?
Or perhaps I just tell myself this to stop worrying about a future involving ASI (:
I doubt AI will be good enough to _scale_ many things because human language isn't sufficient to explain it (especially when not understanding what is being built). Humans invented more concise languages (math, C++) to pair with their intellectual acuity. [1]
In one extreme, we have art, which cannot be accurately reproduced or explained in any language (unless a concise language was used to create it, like math or CS art).
At the other end: 1 + 1 = 2 and PI = 3.14...
[1] It seems any language that has a binary outcome (correct/wrong) helps the AI tremendously, however when e.g. creating software, that software is not functioning in isolation. The software is an attempt to codify a fuzzy human system or need, and that information only flows in one direction: human to machine.
When I was in high school, I took a physics class. I think I scraped by but one thing the teacher said that has stuck with me ever since was "when you want to test a hypothesis, set the variables to extreme numbers". Very useful framework IMO!
If we pursued the "Most Qualified" in its truest form, then ~95%[1] of the jobs in this country would be handed over to foreigners. In isolation, this would be great for US companies. In reality, 95% of the local population would riot.
If we pursued "only hire Americans", then our growth as a country would be stunted, and we'd lose our global competitive edge.
Somewhere between is a balanced medium.
Side observation: I've noticed tech leaders/CEOs/PG have been choosing (or succumbing) to what the algorithm wants them to say or how phrase. I stopped following all of them because I can feel my emotions being manipulated whenever I read what they have to say.
[1] The USA is less than 5% of the worlds population.
The reality is you couldn't hand over jobs to the most qualified person. In most cases the most qualified person already works for you (or is you!), and just needs an assistant who by definition is less qualified.
Suppose you found someone at your competitor who is doing your exact same job (already unlikely - their have a different structure in some way that is important) who is better than you. Would they be willing to move to where you company is? Would your company be willing to pay enough to make them join? Even if the above are true, all those structures your company has that are different mean there will be some time before they are better than you.
Most jobs don't need the best person. Is my doctor the best in the world - probably not - but she is more than good enough for my needs.
Though I have to object to your 95% number. Most people in the world do not have the education that the US provides. Most people are not fluent in English which is almost always a critical part of working for a company in the US. Once you add those requirements, the something closer to 50% of the most qualified people for your job live in the US (now if New Zealand was your country 95% of the most qualified don't live there - and most probably wouldn't want to move even though it is apparently a nice country)
Which is to say even if there was unlimited immigration allowed, your best hire in the US will be an American in most cases.
Though it doesn't matter who is most qualified you need to a plan to mentor and train up the younger generation. (who then grow old and train the next generation)
Your hypothesis straight up assumes that a) Americans have the best possible education required for the work (we don't; functional illiteracy has never been higher in the last few decades). And b) Americans are the most fluent in English (again see a.).
Even if the above assumptions are true the set of Americans who are qualified will be completely outnumbered by, say, even 2% of qualified people from a huge population like China/India/etc.
Your first assumption is completely and utterly wrong. I do not require anyone to have the best education. America does have a great education system. Do others do better? Well, that's very, very arguable, but let's accept it for discussion. The American education system is still very good. For most purposes, the American education system is more than good enough for the job at hand, and so it doesn't matter that an arbitrary education system is better since better doesn't matter once you've crossed the line of knowing the required material.
As for English, most Americans have English as a first language. Now there are certainly accent differences, but that actually plays in the favor of Americans when you're in the U.S. I can understand someone from Australia. However, they do speak a little different and I have to work a little bit at it. That makes them worse at any communication they would have to do with me. The majority of the world does not speak English as a first language. Some people are better than others, but the vast, vast majority of people I've heard who don't live in America have poor English skills. Sure, in countries like England or New Zealand, they will have great English skills as well, but when you get to Germany, some have great, some can just barely get by, and some, despite years of studying, can't actually speak it. Germans are one of the more likely people to have good English skills. If we were to go to China or India, English gets really really hard to understand. Their grammar is typically very very wrong.
Two billion people between India and China with rough numbers, two percent of that is about the population of the US. You're not looking at overwhelming the number of people in the US with only 2% of your population, at most half, and that is before we get into it, there are a lot of people in India and China who are older and not well educated. By contrast, your older people in the US are still likely to have had a fair amount of school and so be better.
> If we pursued the "Most Qualified" in its truest form, then ~95%[1] of the jobs in this country would be handed over to foreigners. [1] The USA is less than 5% of the worlds population.
This would be true if you're talking about randomly picking people from around the world to live in the US for their formative years and go through the US education system (at which point they barely count as "foreigners"). This isn't reality, though, and if you're talking about hiring a worker today, a large swath of that 95% won't speak English, or won't be literate, or will have never received a formal education, all of which are disqualifying for pretty much every job in the US.
I don't follow that "Most Qualified" would mean 95% of jobs going to foreigners. I get that the U.S. is 4-5% of the world's population, but if I live in San Antonio, Texas, the pool of plumbers for my house must also live in San Antonio, Texas, meaning that 99.9+% of the world is not remotely qualified for the job.
I mean, this debate is about immigration, right? So current location doesn't super matter, assuming you can amortize the cost of relocating a plumber across several years' worth of plumbing work.
In practice, you hire the most qualified within the pool of people that can reasonably work for you: right to work in the right country, compatible hours, is able to connect socially with the rest of your team, etc.
It wouldn't be 95%. The US is like 5% of the world, but it disproportionately contains the most qualified workers in many fields. Also, in many cases, even if a worker was somehow qualified, they would be lacking some basic skill like English proficiency or basic cultural compatibility, that would make them actually unqualified on net. But your overall sentiment is right. There are hundreds of millions of people who want to come to the US and make less than you.
It is also a fact that "most qualified" is fuzzy. It comes down to something more like, performance per dollar, or "culture fit", or some other thing. Unless you are trying to absolutely have the most advanced shit in the world at any price, "qualifications" or "talent" are not the ultimate factors. Having been in this industry for a while, most foreign workers obviously aren't "the best" like that. They're not even more talented or hard-working than the average American worker. But they usually work for less and the job gets done (roughly speaking), and their mere presence drives down everyone's wages.
You clearly have worked in a bubble. I find more qualified people in the usa than out of it, this is more of a payment issue than a lack of skill issue.
Just throwing out an idea: I recently stumbled upon an old-school website where someone is uploading a photo of their day (almost) every single day since 1998.
Came across screen captures of the original Everquest (SO COOL) and other games, photos of anime/cosplay events, city shots, family, friends, etc. Really fun for them, and fun for people who like me who still enjoy surfing the web.
When my mom was still alive and relatively new to a smartphone (at age 80) I decided to send her some sort of relevant picture every day via Whatsapp, to get her used to communicating that way. Sometimes it was a real stretch to come up with something. But yes, this forces you to evaluate your life from that viewpoint "surely something was interesting today? Let's keep a picture of it". Even if it's just what I'm having for lunch that day.
I'm not sure if you are in this demographic, but a lot of people I know born in 1999-2003 or so in the US used snapchat as a primary method of communication for many years, and a large part of it was keeping up "snap streaks" of sharing one photo a day with many different friends.
This is one thing for all of the pairs of genuine friends doing this, but I also have two friends who I introduced to each other exactly once, and who have never spoken since, who have a 6 year snap streak together, and both of whom have streaks like that with dozens of other people, many of whom are virtually strangers.
That is interesting. For a streak to count, do you have to both send out your own, and open incoming images? I wonder how much you could tell about someone if you had a 6 year snap streak with them. Would they be more like a friend, or an awkward relative with whom you seldom meet and have little in common?
Both of you have to send them to the other person as a specific recipient (so not just putting it on your story). I'm not sure about opening them.
For the two friends I'm thinking of, my impression is that they don't know each other any better through snapchat than they do from me knowing both of them and mentioning them to each other (they live 1000 miles apart and only met once). I guess the best I can say is that I would be comfortable seating them at the same table at my hypothetical wedding. Same vibe as my cousin I have on Facebook but have only really met twice in the last decade.
Well I knew someone who did this ritual and they would send black photos of their leg so close to the camera. So I'd imagine nothing and its classic number go up social media.
Everquest looked sooooo good back then. Man! No more Meridian 59!
Why did you have to destroy my memories? :)
It's like looking at the first DivX encoded movie that was soooooo good vs. VideoCD/LaserDisc. And then you look at it today and it's so damn blocky and also in that tiny resolution. Ugh!
My first job was working at Wendy's (c. 2006/2007). I recall the kitchen being busy enough that you had to constantly navigate around your coworkers. I worked with a bunch of Brazilians (all super nice people) and a handful of kids around my age.
IIRC lunch rush had (my memory is fuzzy..):
2 drive through positions, 2-3 people on cash registers, dining room cleaner, fry guy (sometimes 2?), 2-3 burger assemblers, one person on drinks, griller, dish washer, and the assistant manager and manager helping out everywhere. Prep team might still be working too.
I remember really enjoying the food a kid (in moderation). And everyone having a mini tantrum when someone ordered fries with no salt :D
Was also shocked to see one of my Brazilian coworkers working at the crack of dawn on my way to school, cause he typically had closing shifts.
Statistical anomaly detected: this morning I woke up from an earth quake in my dream. It was powerful enough to really sway the buildings and room I was in. Im in Texas though, but >= 7.0 earthquakes only happen a handful of times a year.
when was this? USGS shows no earthquake in Texas. Are you trying to say that Texas as >= 7.0 earthquakes a handful of times a year? According to wiki[0], Texas has never had anything close to a 7.0
kkukshtel: You should keep that info, with links, even if its links to your site. Dont burden the reader.
Edit: I read the first half of the article then skimmed the rest (I dont use LLMs outside of occasional searches).
I've been a gamedev for five years. I have a hunch LLMs will not take over game development because of the necessity to control many, many small details to shape a great experience for the player, and because some parts of development cannot be described with words.
Small details: My hunch is when it comes time to polish and fine tune all the moving parts, this process will remain much faster to do by hand. It will be faster to update a specific field or line of code and test/iterate than to prompt your way through that process (the last 20% of the work takes 80% of the time)
Describing without words: An artist with a vision can only bring that vision to life with body movements. It is not possible to communicate all the details that go into a great piece of art with words. The colors, lines, shapes, positioning, specific emotions, history... is beyond spoken language.