Large Language Models in Video Games?

I think using LLMs to provide the dialog for NPCs in a RPG is a use case that's just begging to happen. Ie townsfolk that don't just give the same few replies every time, and who react to things you've done in the past beyond just whatever prewritten options the developer thought of.

We won't see large language models. We will likely see a stripped down version like a small language model (or Domain Specific models if you want the fancy marketing wank term) because a NPC in a fantasy game doesn't need to know about 13th century Europe or 19th century Asia.

Not anytime soon. Nvidia tried, and nobody liked it. LLMs still suck at creative writing and need a ton of RAM/VRAM just to work. They also often get confused or trail off in any discussion/roleplay.

The only game that sort of made it work was Suck Up!, where you're a vampire that has to convince an AI to let you in their house so you can suck their blood. It's a fun concept but even that game gets repetitive quick and the LLM is very stupid and random.

Do I think it’ll happen? Yes, even if it’s not good, because AAA companies are cheap and have no taste. They thrive on just spewing out more content than a smaller studio could make, quality be damned.

That said, whether or not it COULD be useful in the future I think depends on the context and how well you could tune the models.

I think it has absolutely no place in a narrative game where intentional authorship is what people come for. Even if it’s passable, I want to know that what I’m hearing or reading was something SOMEONE wanted to say.

But I think it could be interesting in more open ended, replayable sim games where you want to be able to try a wide variety of approaches and have different experiences each time, but it would be impractical for a dev to implement all those possibilities to the point where players would feel like the game adequately responds to their actions. However, I don’t think you could just drop a copy of chat gpt in there and call it a day. You want different NPCs to be different and you want some consistent reality that they all exist in and respond to. So you’d probably need to put in some constraints based on some hidden file describing a particular world gen’s state. A basic example would be the NPC knowing that the town you asked about is to their north or perhaps an existing relationship between 2 characters.

Idk how technically feasible this would be, but it’d be a cool tool in the right context if done right. I think the key here is it can be good when it enhances what you want to do and you put in the effort to make it work vs just using it as a lazy shortcut.

A good fit would be random background NPCs. For example, pedestrians in a GTA like game. Can potentially increase the variety in the things they can say, and maybe even talk about things the player has just done.

Don't know if it qualifies, but some time ago I saw a VTuber play "Suck Up!": you play as a vampire and you have to actually talk to try and convince NPCs to let you in so you can suck their blood. The "thinking" isn't done on your PC though, you push-to-talk your line and the answer is elaborated by their servers.

I think the PC vs. console divide is relevant here. I'm not sure how advanced text entry on consoles is these days, but I imagine PCs have the advantage with keyboards. Maybe if they use voice recognition on the consoles? But AAA games usually target both, and if interacting with the model is clunky for a big chunk of your market then the big developers might not use the technology.
Of course, indie devs that only target PC can go wild.

There are a couple of mystery games that are available in early access/beta that already use LLM AI to generate dialogue.

I've messed with 1 and it's alright. The problem with it is mostly in how it doesn't necessarily regurgitate the info it knows into something coherent and logical to actually solve the mystery. That or they never ever tell you the truth. Since the only way to solve the mystery is by talking to NPCs to get enough info to piece the puzzle out, it often leads to unsolveable mysteries.

Now, if that problem with LLMs could be fixed, so it isn't conflagrating multiple pieces of data into something new and novel, I think it would be awesome for making game worlds that are more alive and natural. Like instead of walking up to an NPC and clicking "rumors" you can actually just talk to them like a real person and they would respond in kind, as if the NPC was actually a person living in that world. It's really the only thing I think would actually work well with generative AI chat bots, but the generative AI still has too many problems to make it truly viable.

It actually works better with trying to play D&D, because even IRL, players often misinterpret rules and argue. Which you genuinely have when trying to play with 3 AI characters.

The voice actors for The Finals are completely made with AI, there is no real actor behind the scenes they started from.

That's not really what you had in mind, you're probably imagining a live-response model, but it is something that's happening now.

Are you asking if company is willing to risk all their reputation to add software that generates random text based on user input, that can be exploited or just faked as hate, racists or whatever you could think of evil is ?

I don't think so. Maybe it could be used to generate some large dialogue loops but putting random text generator to the actual game if it's not your first game, you have nothing to lose or you want to feel the AI hype ? I don't think so.

I think so. New GPUs will be able to handle AI models running locally before too long. I think this will be used for NPC behavior as a replacement for procedural quest / dialogue generation. I have seen a lot of mobile games leveraging this but they don't seem very good yet. Models need to be trained more specifically for each game I think