This is not a general overview of world partition, nor is it an explanation of how to use world partition or terms used in the user-facing parts of it. See Epic's own docs for that - this is meant to cover the "undocumented" innards of world partition.

Top-Level Concepts

World partition at runtime is "just" normal level streaming, but with a ton of additional features built on top to make managing that streaming much easier. Being built on normal level streaming, actors are not loaded individually (that'd be slow) - instead actors are grouped based on the runtime grid and data layers they are placed into at edit-time and also based on what references they have to other actors.

Streaming

Runtime grids (defined in world settings) specify how actors get placed into grid cells, and each cell results in one or more streaming levels being created during cook (and PIE!) for use at runtime. Streaming levels are also generated for combinations of runtime data layers that overlap within a single cell, meaning multiple streaming levels may generate for one physical location if there are actors in different runtime grids or data layers. As such, having lots of "overlapping" data layers and grids is harmful to streaming performance. On the flip side, a space with no actors in a grid or data layer will not generate a streaming cell and has no overhead - grids and data layers for the most part only have streaming costs in locations that contain actors.

Actor clusters are groups of actors that have hard references to each other and thus must always be loaded within the same streaming level in order to guarantee those references stay intact.

Actors and actor clusters larger than a cell or too far across multiple cell boundaries may be "promoted" into a higher grid level, which is a cell that encompasses a larger area. As such, even within a single runtime grid you can have multiple streaming levels overlapping each other due to (among other reasons) grid promotion.

Runtime grids are defined as part of the world partition runtime hash, which is a class that decides the layout of the runtime grid through the use of a spatial hash. Spatial hashes are selected as part of world partition settings - the current default as of the time of writing is the LH Grid, which attempts to place actors into a regular grid but will expand/shrink the boundaries of cells if the cell would be slightly over- or under-sized.

The spatial hash is effectively a function that takes any location in the world and then outputs the name of the streaming cell that corresponds to that location. At cook time this is used to generate streaming levels, at runtime it is used to decide which levels to stream for a particular location.

The way that the engine decides what locations to feed into the streaming system at runtime is controlled by two main APIs: streaming sources and data layers. Streaming sources allow objects like the player controller to register themselves as a location that the world should stream from, including options such as the radius and target runtime grid(s) to have some control over what exactly gets streamed in.

Runtime data layers are a complimentary method which allow controlling groups of actors to load and unload as defined in the editor. Actors on a data layer still obey spatial loading settings and will only load if non-spatially-loaded or if a streaming source tells them to. Editor data layers only exist within the editor and are primarily an organizational tool, though it is possible to build other editor tools on top of them (such as actor filters,).

External data layers (EDLs) are data layers generally registered by game feature plugins. Content placed into an EDL is stored inside the owning plugin's content folder instead of in the owning level's content folder. This allows completely unregistering the content inside the EDL from the engine - you can simply disable/not cook the owning plugin and the EDL's content won't be packaged. This is particularly useful for DLCs or for live-service content like Fortnite uses (for example, a seasonal event that won't be going out in the same release as the map it takes place on, or one that will need to "inject" content for a limited period of time between client updates).

Content bundles are effectively an older implementation of EDLs with less features and a worse API and are considered legacy.

Level Instances

Level instances are levels that have been placed into a world like a normal actor - sort of like a sublevel, but you can have multiple copies of the same level instance scattered across the world. At runtime, only standalone level instances exist - embedded level instances are broken apart at cook time (and thus have no runtime representation or overhead, aside from the actors broken out of them), while standalone instances stream as a unit even at runtime. This can have some benefits, like being able to manually control the streaming of an entire level instance.

Sub-world partitions are standalone level instances that point to a world partition level that has streaming enabled. This lets you define entirely separate streaming settings for that level instance and have it stream just like it would if you were loaded into it directly. Note that this can be more expensive, and that if the level instance itself is not streamed then nothing within it will be either. That includes non-spatially-loaded actors - they will only load if the outer level instance is loaded!

In the Editor

The editor does not operate on the same concepts as runtime does as the runtime representation of the world requires having cooked all actors into streaming levels. Instead, the editor needs to deal with each actor individually and has an entire data model for working with actors without having to actually load them.

The entire system is built on top of the asset registry, where metadata about each actor is serialized as part of asset registry tags. Asset registry tags do not require explicit loading and thus provide a fast way for the editor to learn important information about an actor such as its label, position, bounds, class, etc - everything you need to be able to display the actor in the outliner and decide whether to load it on the world partition minimap.

Note that this reliance on the asset registry means that sometimes world partition data in the editor can become stale - this usually results in "ghost" actors appearing in the outliner that can't actually be loaded. This isn't a big deal and can be ignored, but clearing your asset registry cache can sometimes help resolve it.

Descriptors

Actor descriptors (FWorldPartitionActorDesc) contain all data about an actor that is needed by the editor when that actor hasn't yet been loaded. These are serialized and then transformed into a base64-encoded string that is saved into an asset registry tag on the uasset that contains the actor it represents. The default descriptor type includes the actor label, runtime and editor bounds, data layer assignments, content bundles, and other data required by the editor.

It is possible to create your own custom actor descriptor subclass to allow storing out whatever editor-only data you desire to ingest in your own tools. This involves creating a subclass of FWorldPartitionActorDesc and then implementing CreateClassActorDesc on the actor to return your new actor descriptor.

Actor Descriptor Containers (UActorDescContainer) contain a list of actor descriptors that generally are fed to streaming generation as a unit - for example, all of the actors in a basic world partition level will be placed into a single top-level container that represents that level. Every external data layer in that level will then create an additional container to store all of the actors within that EDL, since those actors are stored and processed from a separate location.

Containers are not "saved" anywhere and cannot store any persistent data - they are created on the fly by the editor from each directory that world partition actors are stored in. You can generally think of a container as an editor representation of a single level's directory in the __ExternalActors__ folder of the project or a plugin's content directory.

It is possible to make a custom container subclass but this is unusual - currently the only additional implementation of a container in the engine is for property overrides for level instances (a feature that Epic has sadly abandoned because of SceneGraph... which is something that does not yet exist for anyone outside of Fortnite and is thus not particularly useful).

Actor descriptors represent an actor saved to disk, but they do not represent the actual placement of that actor into a level - for that we have actor instance descriptors (FWorldPartitionActorDescInstance). These are not saved to disk but instead are basically a handle to an actor descriptor along with information about the placement of that actor and what level owns it. This is important because embedded level instances can result in the same on-disk actor being instanced multiple times into the same level. Actor instance descriptors cannot be subclassed and do not carry any "custom" data.

Actor descriptor container instances (UActorDescContainerInstancecontain a list of actor instance descriptors, a reference to the container that the original descriptors are in, and a list of sub-container instances.

Container instances are used for a variety of reasons - a basic level may only have a single container instance and a single container, but every level instance will result in a new container instance. Furthermore, actor filters on a level instance may change which actor descriptor instances get created for a container instance.

Container instances are hierarchical, so one container instance can contain another container instance. This is how the hierarchy of nested level instances works - you have the outermost level instance parented to the level's root container instance, and then every level instance inside it is a sub-container, creating a hierarchy that mimics the placement of the level instances themselves.

Custom container instance subclasses are also possible by subclassing UActorDescContainerInstance, though this is again somewhat unusual. Container instances aside from the root are created through actor descriptors (see IsChildContainerInstance and related functions) - thus any actor descriptor can in theory provide an additional container instance to the level.

Streaming Generation

Streaming generation is the process in which the editor takes the above "unloaded" data model for the world and generates a list of streaming levels (and what goes in them) for use at runtime. This process is used not only during cooking for saving out the streaming level packages, but also to generate the world on the fly in PIE.

Streaming generation is an incredibly complex topic in and of itself, but it boils down to the following steps:

1. Collect the list of container instances that will generate streaming levels
2. Filter actors within each container (remove editor-only actors, for example)
3. Cluster actors based on how big they are, what runtime grid they are in, what they hard-reference, and what data layers they belong to
4. Feed the sets of actor clusters to the configured spatial hashes to decide on the list of streaming levels to generate and which actors fit in each.

This process can be tested in-editor with the command wp.Editor.DumpStreamingGenerationLog which will run streaming generation on the active level and output a log of the results to Saved/Logs/WorldPartition. Additionally, doing anything that requires streaming generation to run (such as entering PIE or cooking a world partition level) will also create a streaming generation log in that folder to aid in debugging.

Actors with Is Spatially Loaded disabled and which have no runtime data layers assigned will be saved into the outer level directly, just like an actor in a legacy sublevel-based world. This guarantees that these actors are loaded at all times and simplifies their loading process.

Non-spatially-loaded actors on runtime data layers will still get extracted to separate streaming levels and will no longer provide any guarantees about loading behavior since they stream in like everything else.

Runtime Cell Transformers are used to apply additional transformations to the generation of streaming levels. These are classes with simple "(Pre/Post)Transform" methods that can be added to world settings - every streaming level generated is fed into the transform methods and the transformer class can opt to modify the level or run any other operation you want. A "simple" example is provided in the engine in the form of an ISM transformer that finds all actors using the same mesh in the same streaming cell, and then replaces them with a single ISM component. The FastGeometryStreaming plugin also uses a transformer in order to replace actors with an optimized representation.

More about External Data Layers

Actors can be placed into a single EDL at a time but may be placed into other non-external-data-layers.

EDLs must be registered through the use of a game feature action (or custom code calling into the EDL subsystem) before they can be used. Actors placed into an EDL will be stored into the content directory of the plugin that registered the EDL as noted previously, and thus the generates streaming cells for the EDL will also be stored as part of that plugin to be injected at runtime when the EDL is registered.

EDLs work by creating a new container for each EDL on a map, with the container's path pointed at the content directory of the plugin that registered the EDL. The result is that streaming generation places any generated levels inside the plugin that owns the EDL instead of whatever owns the outer world partition level.

External Streaming Objects

The output of streaming generation for an EDL, aside from the streaming level packages, is also an external streaming object which stores metadata about each runtime streaming cell and the streaming levels that were created for them. At runtime, external streaming objects are injected into a world partition and are what allow the EDL's content to be loaded by the outer world partition.

Level Instance Limitations

As mentioned earlier, level instances are not compatible with EDLs. They can't be placed inside EDLs, and they can't contain EDLs (well they can, but those EDLs won't work). Also, content bundles have the same limitation and won't work either. This is a pretty massive problem if you need to ship any kind of live service/scheduled content on a common level over time. I'm not sure how Epic avoided implementing these features - my understanding is that they have pretty heavy uses of both level instances and EDLs/content bundles, though I think I've heard that the Fortnite branch of the engine has a heavily customized variant of the content bundle system that maybe addresses the issue.

Anyway, I've managed to fix both of these issues though the changes are fairly complex and deep in world partition. I can't post the full set of changes, but I can briefly describe how it works. This is NOT for the faint of heart - it involves a large number of changes to low level parts of world partition, and this is meant to be more of an explanation of how it works in theory rather than instructional on how to do it yourself.

First, a primer on how EDLs work. When you add an EDL to a level it first spawns a new AWorldDataLayers actor. When you make a world partition level you always get one of these actors by default and it is used to store and configure your list of data layers. This extra EDL actor is used to configure the EDL itself along with any sub-layers, and is stored within the EDL's external content path.

When an EDL is injected into a world, the data layer manager (UDataLayerManager) searches the EDL's content path for this actor (via its actor descriptor) and then attaches a container instance pointing to the rest of the EDL's content for that level. This results in the EDL's contained actors being made known to the level's world partition, and also segments the content so it can be cooked to the EDL's parent plugin instead of the level it was placed in.

(Container Instance) Root
|- AWorldDataLayers (default instance)
|- AWorldDataLayers: EDL_MyLayer
|-- (Container Instance) EDL_MyLayer

The first problem with level instances - not being able to place them inside EDLs - is I believe just a small oversight. When the data layer manager creates a container instance for an EDL, it does not ask world partition to create the full hierarchy of containers below that instance (done by setting UActorDescContainerInstance::FInitializeParams::bCreateContainerInstanceHierarchy to true when creating the container instance). I believe there are some other small checks/ensures that need to be removed or fixed up as a result of this change, but otherwise this is the fix.

The second problem is much more complicated to fix. It stems from two rules around how the hierarchy of container instances works:

1. All container instances aside from the root must have a "parent" actor descriptor (this descriptor doesn't necessarily have to create the container instance, but the container instance must belong to the actor).
2. A single actor can have at most one child container instance.

So a level instance actor descriptor creates a single child container instance that contains the referenced level's actor instance descriptors. But that level instance actor could not, for example, spawn a second level by itself since it would not be able to create another container instance.

Let's illustrate a small hierarchy of containers:

(Container Instance) Root
|- ALevelInstanceActor: MyLevel
|-- (Container Instance) MyLevel

Here's a simple hierarchy showing the root container, a level instance actor, and the container instance it created. This is all allowed. Now, if we wanted to have an EDL inside the level instance, we'd need to add a container to the level instance actor to store any actors inside that level that are in the EDL.

(Container Instance) Root
|- ALevelInstanceActor: MyLevel
|-- (Container Instance) MyLevel
|-- (Container Instance) EDL_MyLayer <--

Except this isn't allowed - now there are two container instances parented to a single actor!

The solution is to try to mimic what the external data layer manager already does - use the EDL's AWorldDataLayers as the parent of the container. Something like this:

(Container Instance) Root
|- ALevelInstanceActor: MyLevel
|-- (Container Instance) MyLevel
|--- AWorldDataLayers: EDL_MyLayer <--
|---- (Container Instance) EDL_MyLayer

Doing so is a bit complicated - the data layer manager needs to be updated to listen for any container instance being registered (via UWorldPartition::OnActorDescContainerInstanceRegistered), then iterate through injected EDLs and run asset registry searches to find an actor descriptor for the EDL's AWorldDataLayers for that container's level. If one doesn't exist, then ignore it. If one does, add the actor descriptor to the incoming container instance and then add a container instance pointing at the EDL's content to that AWorldDataLayers actor descriptor.

That's the gist, but there are also a whole list of other problems to fix that are all complex in their own right too:

1. EDLs won't act properly in the data layer outliner if under a level instance. This is due to a level instance's external data layer manager not injecting data layers for that level - the fix is to let the manager inject EDLs for the level instance itself. Note that the EDL manager for level instances should not listen for further container instances being registered to inject EDL container instances into - only the outermost manager should handle that process to avoid multiple trying to register at once.
2. Sequencer bindings will break if bindings reference actors in EDLs inside level instances. This can be fixed by adding a new container id to FWorldPartitionResolveData and filling it out in FWorldPartitionLevelHelper::LoadActorsInternal - this can then be used from within FActorLocatorFragment::Resolve (via FActorLocatorFragmentResolveParameter which also needs a new field to store the EDL container) to calculate the proper path to the actor. This also requires plumbing through the parent container id as part of FWorldPartitionRuntimeCellObjectMapping to set the resolve data properly in the first place.
3. Soft references into/out of EDLs will end up mangled and not resolve properly. Similar to the sequencer issue but for any soft references rather than just sequencer bindings. FWorldPartitionRuntimeContainerResolver::BuildContainerIDToEditorPathMap must be updated to calculate the proper editor path if the given container instance is an instanced EDL. This requires plumbing through whether a container is an instanced EDL as part of FWorldPartitionRuntimeContainerInstance, which can have its value set in FWorldPartitionStreamingGenerator::CreateContainerResolver

If somehow you manage to do all of the above, you'll have EDLs working inside of level instances and level instances working inside of EDLs.

The End

Bye!

That sounds like the lead-up to a post about depression (something I certainly have struggled with in the past), but ironically what has brought this on is the opposite. My current work is fulfilling in all of the ways that my side projects had been previously. The result is not necessarily that I'm spending more time working (though I certainly do at times compared to my previous position) but that I simply don't feel the need to do that kind of work in my own time anymore.

That said, this is certainly a cycle. Nothing lasts forever, and I expect I'll be back on the side-project-train one day. I'll probably work on little things here and there even before then - I'm certainly active enough in various gamedev communities to pick up ideas for little things to go after - but it's unlikely I'll be attempting anything too big (or finishing up my current projects) in the near term.

CW for a jumpscare/loud noise at the very end of the video

The original idea just started with the monster killing you when you looked at it, which seemed to me like an interesting design for a horror game since it means you have to listen carefully for where the monster is in order to avoid looking in its direction. To that end, I've been messing with steam audio/phonon a bit for accurate spatialization but I still have a lot of work to do on that.

It also means the AI can have a somewhat unusual design, where its goal isn't to find the player but instead to stand in places that the player is likely to look towards. Right now the AI is fairly simple - it chooses to walk to a spot either in front of the player, behind the player, or near an active objective (the buttons in the video and certain "screens" that display important information).

After implementing the basic enemy AI I realized that there could be some interesting uses for visibility checks in puzzles. This resulted in the many "displays" that change depending on whether you're looking at them, and the glass material that blocks visibility checks. I've got a few (currently unimplemented) ideas of how to use these for more puzzle designs beyond the simple "button code" example above so stay tuned :)

On the technical side, knowing whether the player is looking at a particular actor is done with the use of custom depth/stencil in order to give near-pixel-perfect accuracy for whether the player can see something. I've created a subsystem that is assigned a certain range of stencil values, and the subsystem assigns out those values every frame to actors that want to know whether the player is looking at them.

When there are too many actors to assign each one a unique value, the subsystem instead cycles through the list of actors changing which ones are being checked for visibility every frame. Additionally, the list of actors to check is culled by a number of functions including some basic broad visibility checks (don't check actors that are behind the player, don't check actors that are far away).

Here's what it looks like when I severely limit the number of values available for checking visibility and slow down the process (no jumpscares this time, just showing what visibility checks look like):

It's practically impossible to tell by eye, but each of the "squares" on the render target have a slightly different value, which allows the system to know exactly which ones are on screen.

All that said, the visibility system is somewhat inefficient at the moment. It depends on a scene capture with post processing material that writes custom depth to a render target, and then reading that render target back on the CPU to dispatch visibility information to the relevant actors. This means an extra render of the scene which is definitely not necessary - ideally I'd be running a shader during the regular render pass and output a small buffer with visibility information. But that's a project for a later time - I'm just happy this works as well as it does.

Anyway, that's all for now. Not sure where I'll go with this (or when...) but I think it's neat :)

For some games it might just involve this wonderful option in the right-click menu of the Unreal Editor:

For many games, the one I'm working on in my spare time included, this isn't quite enough though. In a game where a single level can have multiple states of progression, you need a way to set those states easily.

This is a panel I built with a list of "preset saves" which are really simple blueprints that setup a save data object however I want. The highlighted section at the top is the currently loaded level in the editor - I can use this to load into any part of the game but generally if I'm actively working on a level then that's the one I want to load into. In addition to this panel, I also have a console command that will let me do the same:

Clicking on any of the options here will run the specified blueprint on the active save file and then reload the save.

Most blueprints simply set the level path, player start name, and a "chapter progression flag" (basically an enum that says how far the player has progressed within a set of levels). Some may set other options like level progression flags (for progression within a single level) or attributes on the player themselves (whether they can use abilities, their health, etc).

Here's an example of what a blueprint looks like:

This one sets the current level and player start (both in the panel on the right), and then the chapter's progression flag and a level progression flag (the node graph on the left).

I cannot overstate how incredibly useful this functionality is. To jump to the middle of a level it's just a click of a button. There are certain areas of levels that would otherwise be impossible to jump to without playing up to that point. A major example is when there are sublevels that are streamed in only after certain events are triggered - I would have to play up until the point those events happen if it wasn't for these preset saves.

Why not json?

I previously posted a snippet showing how to import and export saves as json. This seems like a good way to store a bunch of preset save files that are easily editable right? Well, yes, but there are some benefits to sticking with blueprints here - it's easier to update references to assets that have moved and there's extra programmability if a bunch of saves start from a common base. When the game is actively under development and the save format is changing constantly, it's much easier to keep blueprints up to date instead of json files.

I do still use json-based saves whenever I need to debug how saves are created or to make a quick edit so there's still value for me. It just isn't quite as useful as something blueprint-based.

More information about this video can be found in this twitter thread:

I've put in about a year of spotty work - it's all done in my spare time and very inconsistently. Most of the work has been setting up systems for later use - my engineering background coming into play. Things like a dialogue system, combat design, level design tools (camera triggers, "object fade" triggers, encounter managers, etc). There are a number of little editor tools that I've built to facilitate things as well.

Almost anything "artistic" aside from a handful of materials, effects, and models was pulled from the Unreal marketplace or other free sources. It's both impressive and disappointing what you can build with purely an engineering background. I have a budget that one day I'll be using to commission bespoke work - in the meantime, generic assets is what I'm working with. Being a passion project rather than my career means I have all the time in the world so there's no rush.

The video below is two short levels - I actually have a few others that are in various states of blockout, but these are the most "complete". I'm using that term very lightly - these two levels are still absolutely in the blockout stage, but they are more than literal grey boxes. There's some level and dialogue scripting in place, some placeholder cinematic sequences, and a whole lot more that I'm likely forgetting to mention. It's hard to show off something in this state, but I'm proud of what I've done so far.

Additional Credits:

• This work is built on Unreal Engine 5
• Music (c) Hayden Folker under the CC BY license - I needed some placeholder tracks to test music systems and there's a lot of good stuff here.
• My good friend @NelHex who has been helping me out with some concept art, and drew the only placeholder dialogue portrait that isn't just a white square.

I've spent way too long getting this right. It seems so simple to just have a letter appear every fraction of a second in a text block... and it is. At least until you start caring about details like line wrapping not breaking mid-effect and rich text support.

I've posted the relevant code to get this all working on github. The code will not work out of the box - it relies on some classes that I have not included (related to the dialogue system it was pulled from - they're very easy to swap out though) and needs some setup in UMG. It also isn't super optimized and there are some limitations, but it's something I'll improve on in the future.

Check out the readme for a more extensive explanation of the problems that this solves, and feel free to file issues on it if you find any problems with it.

It's a pretty common trope, but at least as far as games go it's also somewhat required for the player to have any chance against a large group of enemies.

That's a short gif from a prototype I'm working on. There are eight AI enemies attempting to attack at once - there are slight pauses here and there (each AI is set to take a short break after attacking) but when they decide to attack there's just too much going on.

DOOM (2016) uses an interesting method to restrict how chaotic having large numbers of enemies on-screen can be - each AI that wants to attack must request a "token" before it can do so. In this way, the designers can limit how many enemies can be attacking at once by limiting the number of tokens available. To make things even more dynamic, AI can also steal tokens from each other if they believe they have a better chance at attacking.

I've seen this technique simply referred to as using "combat tokens" but I've also seen it called having a "kung-fu circle" (technically a different method, but it is a similar idea).

I'm not entirely sure how widespread this technique is overall, but you can see evidence of at least similar techniques in other games. The Batman Arkham series from Rocksteady and WB Montreal very obviously does something to prevent every enemy from attacking at once - usually there are just two or three at most that can attack at the same time. I'm sure there are plenty of other similarly styled fighting games that use such a system as well (not to mention other genres).

I've been taking these ideas and slowly building a similar system for my prototype. It's fairly simple, basically just a way to force the AI to take turns attacking. But it results in much better behavior:

Now there are at most three enemies attacking any one time, and usually it's only one or two. So how does this work?

On every character that can be targeted by an AI, there's a "combat token source" component. This component manages handing out tokens to AI that want to attack.

When an AI asks to attack a character, it tells the token source a value and is given a spot in line in return. This value is set by a designer to represent how "big" an attack is - if it's something that should be the only attack happening then it should have a high value, if it's something that's fine to mix with lots of other attacks then it should have a low value.

The token source has a maximum value after which it stops letting AI attack. Once that threshold is hit, AI sit in a queue and are notified once enough other AI have finished with their tokens.

To make sure AIs aren't just attacking instantly one after the other, there's also a cooldown associated with token values. Whatever value was taken up by a token is made available over a short period after the token is released.

Here's what that looks like:

If the field names aren't clear enough, here's a more thorough description of what's going on:

Max Token Value: The maximum value the token source allows to be used at a time. Each token has a value, active token values added up cannot exceed the max token value.

Token Cooldown Multiplier: This is how fast value is made available.

Active Queue Size: Token activation happens on a first-come first-serve basis - only the token at the front of the queue will be considered for activation. The problem with this is if it has a particularly large value as the queue may get backed up. To combat this, a designer could tell the token source to instead look further ahead in the queue if the first AI has requested too much value. The active queue size is the number of tokens to look ahead.

Active Token Value: This is the total value of all active tokens.

Cooldown Token Value: This is the value that has been released from tokens, but hasn't been made available yet. This counts down at a constant rate (see Token Cooldown Multiplier) until it hits zero.

Conclusion

This system is in its early stages and still requires a lot of tuning, just like most of the AI I am prototyping. How well it will work (or how necessary it will end up being) for the style of game I'm creating still has yet to be seen. It was interesting to build, however, and should give some interesting knobs to turn in order to adjust difficulty down the road.

Here's a very short bit of dialogue just to show the basics:

This short exchange is driven by a basic blueprint (extending a C++ base class) with actions for character and level interaction (in this case, making the characters face each other) and for displaying dialogue.

This is very simple, and works very well. It even supports branching dialogue trees:

There's just one small problem... Take this short exchange:

This is just 10 lines of dialogue, with a few extra actions to control timing. If written into a word document it'd take up practically no space on screen, but in blueprint...

Even on a 27" monitor it is impossible to read anything while keeping this short set of dialogue on screen. And that isn't even the entire scene - only about a third of it!

Even in dedicated branching dialogue software like Chat Mapper or articy:draft I've always felt like viewing large sets of dialogue is clunky, especially when actually writing said dialogue since you are constantly context-switching to add and move nodes around.

My solution to this isn't particularly novel and took only a few days of real work. I had looked at Yarn Spinner previously but unfortunately it only has a runtime for Unity. I decided to write my own scripting system inspired by it, but designed with the explicit goal of integrating into my existing "TalkMoment" system and blueprint.

Player = /Game/Flame/Characters/Player/Sp_Player
Uncle = /Game/Flame/Characters/Tower/Sp_Tower_Uncle

Uncle [Left, Default]: Is something on your mind? {Player}?
Player [Right, Default]: Huh? Oh, did you say something?
Uncle: You've got your mind stuck on something. What's going on?
Player: Just a weird dream. Some people talking and then a loud noise that scared me.
Uncle: ...
Uncle: Are you making mischief even in your dreams now?
Player: Hey!
Player: I mean...
Player: No...
Player: I can't even really remember what was being said. And I don't think I could see anything.
> ClearLine

This is the script powering the same short cutscene as before, but now it takes up a tiny portion of the screen. It allows me to set portraits in [brackets] and let's me call into both C++ and Blueprint with > commands. I can reference /assets/on/disk which let me share properties between scripts (such as the name of a character and the portraits they use). I can also use standard Unreal {text formatting} to intersperse variables with handwritten text. Since the original system is called "Talk", I call this "Supertalk".

It supports branching dialogue:

Person1: Foo or bar?
  * Foo
    Person1: You chose foo!
  * Bar
    Person1: You chose bar!
Person2: Cake or pie?
  * Cake
    -> ChooseCake
  * Pie
    -> ChoosePie
    
# ChooseCake

Person2: You chose... poorly.

# ChoosePie

Person2: You chose pie!
Person2: This was the right choice.

And it can even be used in a mixed blueprint/supertalk environment:

This scripting language is very simple and I want to keep it that way - commands aren't really handled by the scripting runtime, but instead defer to Unreal to parse arguments and call functions. The "compiler" just takes the text script and serializes the syntax tree as a normal Unreal asset that can then be loaded when the game is running. There's no such thing as an if statement or any other flow control beyond jumping between sections of dialogue after a choice. All of that is better left to blueprint (or C++) which can handle more complex interactions, while leaving the actual text very simple.

Conclusion

I'm slowly going back through any prototype cutscenes I've written and finding any large masses of dialogue that can be replaced with Supertalk. It works pretty well so far - I'm sure I'll keep improving it but the simplicity is nice. The only thing I'd really like to add at this point is some custom syntax highlighting for an editor like VS Code. Maybe some day once I've polished things up a bit I'll release it.

The caption said, "The navigation system is a little funky and needs a rewrite"... Well, now I've done a large refactor of the navigation system and it works much better.

Before the Refactor

First, a bit on how things used to work. The pathfinding algorithm used is a very simple implementation of A*. When an agent needs to query the navigation system, the world is broken down into a grid with each grid cell being a size based on the agent's size. From there, the navigation system can query the world as necessary to see if a space is blocked or not.

Here's the same scene as before with the grid visible:

There are four types of cells here:

• Green cells are part of the final path
• Red cells are blocked by something
• Yellow cells were candidates to be chosen for the path (not blocked and in the closed list)
• White cells weren't looked at (in the open list)

There are some fairly obvious issues with the above pathfinding system - the first is that agent movement is fairly jerky - this is because the path following algorithm is very naive (we'll come back to this later as it isn't directly related to the navigation system).

The bigger issue is towards the end of the gif when the two agents are trying to path to each other. In order to facilitate generating paths to moving objects (and to account for dynamic obstacles) the "navigate to" action used by the cinematic system (along with other systems) re-queries the navigation system for a new path every so often. That time is set fairly high by default - 2 seconds. This is because any navigation query happens in full immediately and if there are enough of them in a single frame (or the query is sufficiently large) it can tank performance.

Before anyone asks - no, the answer was not to make things multi-threaded (navigation relies on the physics engine which is not thread-safe).

The New Navigation System

First, here's how things look with the new navigation system:

No more going up and left over and over again - the agents walk in (mostly) straight lines (again, we'll come back to this)! No more weird dancing around when they path to each other!

Here's the same scene again with debugging on:

Probably the most important change is that navigation is now updating almost every frame. And, while it might not be obvious here, there's little risk for performance problems. Why? Because pathfinding doesn't need to complete on the same frame the navigation system is queried!

Here's the new flow for how navigation works:

1. Agent requests a "token" from the navigation system
2. Agent updates token with pathfinding parameters (agent size, start/end, etc)
3. (every frame) navigation system loops through the list of active tokens and processes pathfinding for each until it a certain "navigation budget" is reached
4. (every frame) agent checks for a new path from the token

As long as an agent hasn't cancelled its navigation token the system will continue to update it with new pathfinding results as often as possible. This lets agents react to changes in the environment in real-time without having to worry about overburdening the navigation system each frame.

As for the "navigation budget", this can theoretically be anything - I think ideally there'd be a set time constraint where the navigation system can do its thing but right now this is just a hard limit of how many physics queries can happen in a single frame - if a pathfinding result is incomplete when the budget is reached the navigation system pauses and then picks up where it left off next frame.

There are still some improvements to be made - primarily in the pathfinding algorithm itself. There are a bunch of limitations with the current implementation:

• If an agent is in an open (infinite) space but trying to path somewhere that isn't possible to reach, it will infinitely expand the pathfinding grid. Currently this is mitigated by limiting how far the pathfinder can search but a better solution would be to path from both sides at the same time - if either side fails to find a path then we know that side is in an enclosed space and a path isn't possible.
• Incomplete paths aren't possible - ideally we'd be able to at least have paths that get close to a target even if they can't reach the target.

These are problems to tackle in the future - for now I think the system works pretty well.

Better Path Following

The other big piece of this was rewriting how path following works. Originally path following was very naive - it would internally keep track of the next point on the path to head towards and would try to hit that point exactly, even if it ended up overshooting the point at first. Another side-effect was that diagonal movement resulted in near constant horizontal-vertical direction changes.

My solution was twofold - the first was to swap over to a path following steering behavior which uses a "predicted" position to know where to head towards next and also to allow a bit of leeway rather than strictly adhering to the path.

The second part of the solution focused on making movement a bit more natural - rather than allowing instant turns I now have agents interpolate between their current movement direction and their desired direction. This is somewhat of a halfway between the instantaneous movement that is used for the player and a more classical steering agent that uses forces to control movement. Here's what that looks like:

The orange line is the "desired" movement direction as calculated by the path-following behavior. The green line is the "real" movement direction - you'll notice that the orange line is constantly changing while the green line changes slowly. The algorithm still needs some tweaking but the result it gives is to my eyes pretty good.

That's it for now, I'll leave off with a gif of a bunch of agents all trying to navigate to the player at the same time to attack them:

The custom engine is built on top of MonoGame (C#) and includes a fairly comprehensive Lua scripting system (via MoonSharp). One of the cool features of Lua is support for coroutines - a form of cooperative multitasking. You can start a coroutine and that coroutine can pause itself to be resumed later.

Building Cinematics

One area that's particularly suited for using coroutines is scripted cinematics. A cinematic script can be a coroutine that yields whenever some action needs to take multiple frames or whenever the script needs to wait for something to complete. Using a coroutine means I can write these scripts completely linearly. Here's a short example of something that would actually work in my engine:

-- These are just aliases of existing functions to make things a bit faster to
-- write.
local yield = coroutine.yield
local CA = CinemaActions

return Cinematic("MyFirstCinematic", function (data)
	local scene = data.Parameters["Scene"]
    
    -- Get our "actors" for the cinematic
    local person1 = scene:FindEntityByName("Person1")
    local person2 = scene:FindEntityByName("Person2")
    
    -- We need to register the fact that we're using these entities as actors
    -- in our cinematic. Other systems (like player input) can disable or
    -- cancel themselves if they see that an entity is taking part in a
    -- cinematic.
    yield (CA.ActorRegistration{person1, person2})
    
    -- Have person1 walk to person2
    yield (CA.NavigateToEntity(person2, person1))
    
    -- Wait 2 seconds
    yield (2)
    
    -- Have person2 walk to (100, 200)
    yield (CA.NavigateToPosition({100, 200}, person2)
    
    -- Have both people walk to each other in parallel
    yield (CA.Parallel(
    	CA.NavigateToEntity(person2, person1)
        CA.NavigateToEntity(person1, person2)
    ))
    
    -- All done!
    -- Actors automatically get released at the end of the cinematic
end)

Of course, there's a lot going on behind the scenes here. There's a navigation system behind NavigateToEntity and NavigateToPosition, and there's some scene management going on at the top of the script.

But the focus here is the use of yield() - every time we call that we return a value to the cinematic system. In some cases that value is an action - like NavigateToEntity - in which case the action will be executed to completion before our coroutine continues. In other cases the action is a number in which case the cinematic waits that long (in seconds) before continuing our coroutine.

Cinematic Actions

What's really cool about this system is that not only can you write cinematics linearly in Lua, but it cleanly integrates with C# as well. C# has support for defining iterators as functions with similar syntax to Lua's yield:

public IEnumerator<int> MyIterator()
{
    yield return 1;
    yield return 2;
    yield return 5;
}

foreach (int value in MyIterator())
{
	Console.WriteLine(value); // Prints 1, then 2, then 5
}

Sort of similar to Lua coroutines, right? Which means we can implement anything from individual actions in a cinematic to entire scripts in a similar way on the C# side. For example, here's a trivial implementation of an action that just waits a few seconds before letting further actions continue:

public class WaitAction
{
    private float timeLeft;
    
    private WaitAction(float timeToWait)
    {
    	this.timeLeft = timeToWait;
    }
    
    // The private constructor and static function here are just to keep
    // the API as clean as possible - only one way to create this action from
    // both C# and Lua. The [ScriptFunction] attribute is how we register this
    // into the CinemaActions module for Lua.
    [ScriptFunction("Wait", Module = CinemaActionsModule.Name)]
    public static CinemaAction Create(float timeToWait)
    {
    	return new WaitAction(timeToWait).Run;
    }
    
    private IEnumerator<object> Run(ICinematicData data)
    {
    	while (this.timeLeft > 0)
        {
            this.timeLeft -= (float)data.GameTime.ElapsedGameTime.TotalSeconds;
            
            // yielding null ends the current frame but continues
            // execution next frame.
            yield return null;
        }
    }
}

A bit more boilerplate - mostly because this action should be exposed to both Lua and C# - but still pretty simple. Here's another simple example showing how actions can be nested:

private IEnumerator<object> Run(ICinematicData data)
{
    // Wait 10 seconds
    yield return WaitAction.Create(10);
    
    // do something, idk
    
    // Wait 5 seconds
    yield return WaitAction.Create(5);
}

Of course, nesting actions can be done over and over again, though usually you don't need to nest things too much.

Backend Implementation

The implementation of the cinematic system is done on the C# side. The basic idea is that the root cinematic object maintains a "stack" - the active action is always on the top of the stack. Actions can be from C# as IEnumerator<object> or Lua coroutines. At the start of a cinematic the initial action (usually the Lua script itself) is added as the only item on the stack.

Every frame, the cinematic gets the next result from the active action, running whatever logic is inside it. When the action returns, the return value is examined:

• If the action didn't return any more values (i.e. we reached the end of the enumerator) then we remove the action from the stack and immediately continue the new topmost action (or end the cinematic if nothing is left)
• If it's null (or nil, if you're using Lua) the return value is ignored and the game proceeds to the next frame.
• If the value implements IEnumerator<object> it is placed at the top of the stack to be executed immediately
• If the value is a CinemaAction delegate (delegate IEnumerator<object> CinemaAction(ICinematicData data)) then it is immediately executed and the resulting IEnumerator<object> is added to the stack and executed.
• If the value is a Lua function then the function is added to the stack and immediately executed as a coroutine.
• If the value is a number then a WaitAction is added to the stack and immediately executed. Other "shortcuts" work similarly.

There are some other minor complications thrown in as well - parallel actions are a bit harder to deal with and don't work with the stack. Currently the cinematic spins up "sub-cinematics" for each action running in parallel - that way each parallel action gets its own stack.

Closing Words

I'll leave off with one of the more interesting bits of cinematic scripting I've done. Recently I've been writing scripts for the combat tutorial - and part of the flexible nature of this cinematic system is that I can have the player perform actions while the cinematic continues running in the background, waiting for them to complete a task. So to end this here's a section of that script that starts the tutorial combat encounter:

local encounter = scene:FindEntityByName("Enc_Dummies"):GetComponent(Components.Encounter)

local lowHealth = false

-- This will be called every time the player gets hurt during
-- this part of the tutorial. If the player is about to die,
-- we set a flag and reset their health back to max.
local modifyHit = function (_, e)
    local h = player:GetComponent(Components.Health)
    if e.Damage >= h.Health then
        lowHealth = true
        e.Damage = 0
        h.Health = h.MaxHealth
    end
end

player:GetComponent(Components.DamageReceiver).ModifyDamageEvent.add(modifyHit)

-- Remove the player from the cinematic so player input starts working again
data.Cinematic:DeregisterActor(player)

-- Enables AI for the encounter
encounter:BeginEncounter()
while encounter.Running do
    if lowHealth then
        -- Let the player know they ran out of health (though we reset it in
        -- modifyHit) and then continue the encounter.
        encounter:PauseEncounter()
        yield (CA.ActorRegistration{player})
        yield (CA.Dialogue(dialogue, "swordtut_fail"))
        
        -- Continue the encounter - we're not resetting anything but the player
        -- health so it should get a bit easier if they've gotten any hits on
        -- enemies.
        data.Cinematic:DeregisterActor(player)
        encounter:BeginEncounter()
        lowHealth = false
    else
        -- do nothing until the encounter is over or the player has low health
        yield (nil)
    end
end

-- Remove our modify damage function
player:GetComponent(Components.DamageReceiver).ModifyDamageEvent.remove(modifyHit)

I've been working on a multiplayer top-down shooter in Unreal 4 recently as a way to learn more about Unreal's multiplayer systems. I recorded a really quick video to show progress after about 1.5 weeks.