Overview
LexBot connects Unreal Engine 4.27 and 5.8 to the Amazon Lex V2 runtime. Design the conversation in the Lex console (intents, slots, prompts, Lambda fulfillment), then let players talk to it with typed text or the microphone. Requests run on background worker threads with automatic retries; every result arrives on the game thread.
- Recognize Text: text in — messages, intent with slots, interpretations with NLU confidence, sentiment, session state and dialog action out.
- Recognize Utterance: 16-bit PCM speech in (any rate, converted to 16 kHz mono), what Lex heard plus an optional spoken reply as a playable sound wave.
- Put / Get / Delete Session: start at an intent, pre-fill slots, set attributes, read or end a session.
- LexBot Conversation component: session id per NPC, turn queue, microphone with voice-activity detection (Win64, Android), automatic reply playback, events for messages, intent changes, filled slots and the end of a conversation.
- Session-state builders, response helpers (card buttons, message text, SSML stripping), locale mapping from the game culture, WAV and PCM helpers.
Request / response onlyLexBot uses RecognizeText, RecognizeUtterance and the session operations. They work over HTTP/1.1 on every platform, including Linux and Android where the engine's libcurl has no HTTP/2. The streaming StartConversation operation is not used.
Requirements
| Requirement | Details |
|---|
| Unreal Engine | 4.27 and 5.8 |
| Platforms | Win64, Linux and Android. Microphone capture: Win64 and Android (Linux: send audio with Send Audio / Recognize Utterance). |
| Modules | LexBot (Runtime), LexBotSdk (prebuilt AWS SDK for C++ 1.11.916 in Source/ThirdParty) |
| Dependencies | The engine's AndroidPermission plugin (enabled by default) for the Android microphone permission. On Linux and Android the SDK uses the engine's libcurl, OpenSSL and certificate store. |
| AWS | An AWS account and an Amazon Lex V2 bot with a built locale and an alias. Lex and Cognito usage is billed by AWS. |
Installation
- Install LexBot from Fab into your engine, or copy the
LexBot folder into your project's Plugins folder.
- Enable LexBot in Edit > Plugins and restart the editor.
- Open Project Settings > Plugins > LexBot and set Region, Identity Pool Id, Default Bot Id, Default Bot Alias Id and Default Locale Id.
- C++ only: add
PublicDependencyModuleNames.Add("LexBot"); to your Build.cs.
Several AWS plugins in one projectEach Alpha XP cloud plugin carries its own copy of the SDK under a plugin-specific name, so they install side by side without conflicts.
Setting up AWS
- In the Amazon Lex V2 console create a bot, add your intents and slots, and Build the locale. Note the Bot ID (10 characters) and the region.
- Create an alias for the version you ship (Bots > Aliases). For testing, the built-in draft alias
TSTALIASID works.
- Optional: enable sentiment analysis on the alias to fill Interpretation.Sentiment, and set a Polly voice on the locale for spoken replies.
- In Amazon Cognito create an identity pool (guest access if players talk to the bot before signing in) and give its role permission to talk to your bot alias only:
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": [
"lex:RecognizeText",
"lex:RecognizeUtterance",
"lex:PutSession",
"lex:GetSession",
"lex:DeleteSession"
],
"Resource": "arn:aws:lex:us-east-1:123456789012:bot-alias/ABCDEFGHIJ/TSTALIASID"
}]
}
List one ARN per alias (bot-alias/<bot id>/<alias id>). Drop lex:PutSession if players must not change the session state themselves.
Never ship access keysStatic keys in the settings end up in DefaultGame.ini and in packaged games. Use them only for development and dedicated servers; players should always use the identity pool. Shipping builds refuse settings keys unless Allow Static Keys In Shipping is enabled.
Quick start
Typed chat with an NPC
- Add the LexBot Conversation component to the NPC actor (leave Bot Id empty to use the settings).
- When the player submits text, call Send Text on the component.
- Bind On Bot Message: show Get Message Text in the dialogue widget. For cards (Is Response Card), make buttons from Get Card Buttons and send the button's Value with Send Text.
- Bind On Conversation Ended: when Reason is Fulfilled, read the slots of Intent (Get Slot Value) and run the game action — open a shop, give a quest, spawn a vehicle.
Voice commands
- Android: call Request Microphone Permission once.
- Set the component's Response Audio to PCM 16 kHz (sound wave) if the NPC should answer out loud.
- On a key press call Start Microphone. The recording is sent when the player stops speaking; On Transcript shows what Lex heard and the reply plays automatically.
Without the component
Use Generate Session Id once per conversation and call Recognize Text with it on every turn. Response > Session State > Intent holds the slots; Is Conversation Ended tells you when the bot closed the intent.
Conversation component
ULexBotConversationComponent keeps one conversation: the session id (a new GUID per component, Reset Conversation starts over), the current intent and its slots, session attributes (sent with the next turn after you change them) and request attributes (sent with every turn). Turns sent while the bot is answering are queued.
| Event | Parameters | When |
|---|
| On Bot Message Event | Message, Index | Once per message of a turn: plain text, SSML, custom payload or image response card. |
| On Intent Changed Event | Intent, Previous Intent Name | The bot recognized another intent. |
| On Slot Filled Event | Slot Name, Slot | A slot of the current intent got a new value. |
| On Conversation Ended Event | Reason (Fulfilled / Failed / Closed), Intent | The dialog action is Close. |
| On Response Event | Response | After the other events of every turn. |
| On Transcript Event | Text | What Lex heard (speech) or the text sent. |
| On Error Event | Error | AWS errors, NoSpeech, MicrophonePermission, Unsupported, CaptureFailed. |
| On Speech Started / Ended Event | — | Voice-activity detection of the microphone. |
| On Response Audio Finished Event | — | A reply played by the component finished. |
Microphone settings on the component: Silence Threshold (RMS), Silence Timeout Seconds, Max Utterance Seconds, No Speech Timeout Seconds, Microphone Gain, Auto Send On Silence and Stop Audio When Listening (barge-in). Audio is converted to 16 kHz mono (or 8 kHz with the Microphone Sample Rate setting); half a second before speech starts is kept so the first syllable is not cut.
Node reference (90 nodes)
Latent nodes run in the background, can be stopped with Cancel on their return value and fire exactly one final pin on the game thread. Empty Bot Id / Bot Alias Id / Locale Id inputs use the project settings.
Conversation (latent)
| Node | Inputs | Outputs | Purpose |
|---|
| Recognize Text Latent | Input Text, Session Id, Bot Id, Bot Alias Id, Locale Id, Send State, State, Request Attributes | On Success / On Failure (Response, Error) | One text turn (RecognizeText). |
| Recognize Utterance Latent | Pcm Audio, Sample Rate, Session Id, Response Audio, Num Channels, Bot ids, Send State, State, Request Attributes | On Success / On Failure (Response, Error) | One spoken turn (RecognizeUtterance); Response.Sound is the spoken reply with Response Audio = PCM. |
| Recognize Text With Voice Reply Latent | Input Text, Session Id, Response Audio, Bot ids, Send State, State, Request Attributes | On Success / On Failure | Text in, spoken reply out (RecognizeUtterance with text input). |
| Recognize WAV File Latent | Wav File Path, Session Id, Response Audio, Bot ids | On Success / On Failure | Sends a 16-bit PCM WAV file as speech. |
Session (latent and helpers)
| Node | Inputs | Outputs | Purpose |
|---|
| Put Session Latent | Session Id, State, Bot Messages, Response Audio, Bot ids, Request Attributes | On Success / On Failure (Response, Error) | Sets intent, slots, attributes and dialog action; optionally makes the bot say Bot Messages. |
| Get Session Latent | Session Id, Bot ids | On Success / On Failure | Reads the session: intent, slots, attributes, last messages and interpretations. |
| Delete Session Latent | Session Id, Bot ids | On Success / On Failure | Ends the session on the bot. |
| Make Session State Pure | Intent Name, Slots (name -> value), Session Attributes, Dialog Action, Slot To Elicit | Session State | Quick builder for Put Session or the State inputs. |
| Make Slot Value Pure | Value, Original Value, Resolved Values | Slot | A filled scalar slot. |
| Make List Slot Value Pure | Values | Slot | A filled List slot. |
| Make Intent Pure | Name, Slots, State, Confirmation State | Intent | An intent with typed slots. |
| Set Intent Slot Pure | Intent, Slot Name, Slot | Intent | Copy of an intent with one slot set. |
| Make Dialog Action Pure | Type, Slot To Elicit, Elicitation Style | Dialog Action | What Lex does next. |
| Make Active Context Pure | Name, Attributes, Time To Live Seconds, Turns To Live | Active Context | Enables follow-up intents. |
| Make Runtime Hint Pure | Intent Name, Slot Name, Phrases | Runtime Hint | Phrases that improve recognition of one slot (item or character names). |
| Session State To JSON Pure | State | Return Value | Lex V2 JSON of a session state (save, log). |
| Session State From JSON Pure | Json | State, Return Value | Reads a session state from Lex V2 JSON. |
| Generate Session Id Callable | — | Return Value | New random session id (lower-case GUID). |
| Decode Lex Header Pure | Encoded | Return Value | Base64 + gzip decoding of utterance headers. |
| Encode Lex Header Pure | Text | Return Value | Gzip + base64 encoding Lex expects in utterance headers. |
Response helpers
| Node | Inputs | Outputs | Purpose |
|---|
| Get Message Text Pure | Message | Return Value | Plain text, SSML without tags, card title + subtitle or the custom payload. |
| Join Messages Pure | Messages, Separator | Return Value | Readable text of all messages. |
| Is Response Card Pure | Message | Return Value | True for image response cards. |
| Get Card Buttons Pure | Message | Buttons | Buttons of a card (Text + Value). |
| Get Messages Of Type Pure | Messages, Type | Messages | Filters messages, for example Custom Payload only. |
| Intent State To Text Pure | State | Return Value | "InProgress", "Fulfilled"... |
| Dialog Action Type To Text Pure | Type | Return Value | "ElicitSlot", "Close"... |
| Get Slot Value Pure | Intent, Slot Name | Value, Return Value | Interpreted value of a slot. |
| Get Filled Slot Names Pure | Intent | Return Value | Names of the slots with a value. |
| Get Top Interpretation Pure | Response | Interpretation, Return Value | The interpretation Lex chose, with NLU confidence and sentiment. |
| Find Interpretation Pure | Response, Intent Name | Interpretation, Return Value | The interpretation of one intent. |
| Is Conversation Ended Pure | Response | Reason, Return Value | True when the dialog action is Close; Reason Fulfilled / Failed / Closed. |
| Strip SSML Pure | Ssml | Return Value | Removes tags, decodes entities. |
Locale
| Node | Inputs | Outputs | Purpose |
|---|
| Locale From Culture Pure | Culture (empty = current) | Return Value | en-US -> en_US, de -> de_DE, es-MX -> es_419, zh-Hant -> zh_HK. |
| Get Supported Locales Pure | — | Return Value | Lex V2 locale ids. |
| Is Supported Locale Pure | Locale | Return Value | True for a listed locale id. |
Conversation component
| Node | Inputs | Outputs | Purpose |
|---|
| Send Text Callable | Text | Return Value | Queues a text turn. |
| Send Audio Callable | Pcm, Sample Rate, Num Channels | Return Value | Queues a spoken turn. |
| Is Waiting For Response Pure | — | Return Value | A turn is on its way or queued. |
| Reset Conversation Callable | Delete On Server | — | New session id, no intent; attributes are re-sent. |
| Sync Session From Server Callable | — | (On Response / On Error) | Get Session into the component's state. |
| Get Session Id Pure | — | Return Value | The current session id. |
| Set Session Id Callable | New Session Id | — | Continues another session. |
| Set Bot Callable | New Bot Id, New Bot Alias Id, New Locale Id | — | Switches the bot. |
| Get Current Intent Pure | — | Return Value | Intent with slots. |
| Get Slot Value Pure | Slot Name | Value, Return Value | Slot of the current intent. |
| Get All Slots Pure | — | Return Value | All slots of the current intent. |
| Get Dialog State Pure | — | Slot To Elicit, Return Value | Idle, Waiting For Bot, Listening, Eliciting Intent / Slot, Confirming, Processing, Ended. |
| Get Last Transcript Pure | — | Return Value | What Lex heard last. |
| Get Last Response Pure | — | Return Value | The full last response. |
| Get Conversation History Pure | — | Return Value | Player and bot lines (up to Max History). |
| Clear History Callable | — | — | Forgets the history. |
| Get Session Attributes Pure | — | Return Value | Attributes shared with the bot. |
| Set Session Attribute Callable | Key, Value | — | Sent with the next turn. |
| Remove Session Attribute Callable | Key | — | Removes one attribute. |
| Clear Session Attributes Callable | — | — | Removes all attributes. |
| Set Request Attribute Callable | Key, Value | — | Sent with every turn, not stored by Lex. |
| Clear Request Attributes Callable | — | — | Stops sending request attributes. |
| Start Microphone Callable | Device Index | Return Value | Records the player with voice-activity detection. |
| Stop Microphone Callable | — | Return Value | Stops and sends the utterance. |
| Cancel Microphone Callable | — | — | Stops and discards. |
| Is Microphone Active Pure | — | Return Value | True while recording. |
| Get Input Level Pure | — | Rms, Peak | Smoothed microphone level for meters. |
| List Capture Devices Callable | — | Return Value | Audio input devices. |
| Play Response Audio Callable | Sound (None = last reply) | Return Value (Audio Component) | Plays a reply 2D or at the owner. |
| Stop Response Audio Callable | — | — | Stops the reply. |
| Is Playing Response Pure | — | Return Value | True while a reply plays. |
Audio and microphone
| Node | Inputs | Outputs | Purpose |
|---|
| Create Sound Wave From Pcm Callable | Pcm, Sample Rate, Num Channels | Return Value | Playable sound wave from 16-bit PCM. |
| Load WAV As PCM Callable | File Path, Target Sample Rate | Pcm, Problem, Return Value | 16-bit WAV to mono PCM at the target rate. |
| Save PCM As WAV Callable | Pcm, Sample Rate, Num Channels, File Path | Return Value | Writes a WAV file. |
| Resample PCM16 Pure | Pcm, In Sample Rate, In Num Channels, Out Sample Rate | Return Value | Mono conversion with linear resampling. |
| Is Microphone Supported Pure | — | Return Value | Win64 and Android. |
| Has Microphone Permission Pure | — | Return Value | Android RECORD_AUDIO granted. |
| Request Microphone Permission Latent | — | On Granted / On Denied | Android permission dialog (Win64: granted). |
| Get Pcm Data Pure | (LexBot Sound Wave) | Return Value | The reply's 16-bit PCM. |
| Get Spoken Text Pure | (LexBot Sound Wave) | Pcm Rate, Return Value | Text the reply speaks, and its sample rate. |
Config, identity, JSON
| Node | Inputs | Outputs | Purpose |
|---|
| Set Region Callable | Region | — | Switches the AWS region. |
| Set Credential Mode Callable | Mode | — | Overrides the credential mode at runtime. |
| Set Login Token Callable | Provider, Token | — | Signs the player in to the identity pool. |
| Clear Login Tokens Callable | — | — | The player becomes a guest again. |
| Set Static Credentials Callable | Access Key Id, Secret Access Key, Session Token | — | Static Keys mode from now on. |
| Get Identity Id Pure | — | Return Value | Cognito identity id. |
| Make User Pool Provider Name Pure | Region, User Pool Id | Return Value | cognito-idp.<region>.amazonaws.com/<pool id>. |
| Is Configured Pure | — | Problem, Return Value | Checks credentials and the default bot. |
| Resolve Identity Latent | Force Refresh | On Success / On Failure (Identity Id, Error) | Fetches credentials ahead of the first turn. |
| Struct To JSON Callable | Struct | Json, Return Value | Any struct as JSON (for a session attribute). |
| JSON To Struct Callable | Json | Out Struct, Return Value | Fills a struct (for a custom payload). |
| Get JSON Field Pure | Json, Path | Value, Return Value | One value by path, e.g. reward.items[0].id. |
| Cancel Callable | (latent node) | — | None of the node's pins fire any more. |
| Is Finished Pure | (latent node) | Return Value | True after the node finished or was cancelled. |
Credentials
| Mode | Use for | How |
|---|
| Cognito Identity Pool | Game clients | Guests get unauthenticated credentials; after Set Login Token the player gets the authenticated role. Credentials refresh automatically before they expire. |
| Static Keys | Development, dedicated servers, keys from your backend | Settings keys or Set Static Credentials (with an optional session token). |
| Default Credential Chain | Servers and tools | Environment variables, the ~/.aws profile, ECS and EC2 instance roles. |
C++ API
// Build.cs: PublicDependencyModuleNames.Add("LexBot");
#include "LexBotLibrary.h"
const FString Session = ULexBotLibrary::GenerateSessionId();
ULexBotLibrary::SendTextNative(TEXT("I want a coffee"), Session,
FLexBotNativeDelegate::CreateLambda([](const FLexBotResponse& Response, const FLexBotError& Error)
{
// runs on the game thread
UE_LOG(LogTemp, Log, TEXT("%s / %s"), *ULexBotLibrary::JoinMessages(Response.Messages, TEXT(" ")), *Error.Message);
}));
The component (ULexBotConversationComponent) and the latent node classes (ULexBotRecognizeTextAction and the others) can also be used from C++: create the action through its static factory, bind the delegates, then call Activate().
Types
| Type | Kind | Members |
|---|
| FLexBotResponse | Struct | Session Id, Request Id, Messages, Interpretations, Session State, Request Attributes, Input Transcript, Input Mode, Recognized Bot Name, Sound, Audio Bytes, Audio Content Type, Duration |
| FLexBotMessage | Struct | Content, Content Type, Card (Title, Subtitle, Image Url, Buttons) |
| FLexBotIntent / FLexBotSlot | Struct | Name, State, Confirmation State, Slots — Has Value, Original / Interpreted / Resolved Values, Shape, List Values, Sub Slots |
| FLexBotInterpretation | Struct | Intent, Nlu Confidence (-1 = none), Has Sentiment, Sentiment (type + 4 scores), Source |
| FLexBotSessionState | Struct | Dialog Action, Intent, Active Contexts, Session Attributes, Runtime Hints, Originating Request Id |
| FLexBotTurn / FLexBotCaptureDevice / FLexBotError | Struct | History line; input device; Code, Message, Http Status, Retryable |
| ULexBotSoundWave | Class | Runtime sound wave from 16-bit PCM (spoken replies) |
| Enums | Enum | Intent State, Confirmation State, Dialog Action Type, Message Type, Sentiment, Slot Shape, Conversation End, Audio Format, Dialog State, Credential Mode |
Settings
Project Settings > Plugins > LexBot (saved to DefaultGame.ini). Changes apply to the next request.
| Setting | Default | Description |
|---|
| Region | us-east-1 | Region of your bot and identity pool. |
| Credential Mode | Cognito Identity Pool | Cognito Identity Pool, Static Keys or Default Credential Chain. |
| Identity Pool Id | (empty) | For example us-east-1:01234567-89ab-cdef-0123-456789abcdef. |
| Allow Guest Access | On | Unauthenticated identities before Set Login Token. |
| Access Key Id / Secret Access Key | (empty) | Static Keys mode - development and servers only. |
| Allow Static Keys In Shipping | Off | Settings keys are refused in Shipping builds unless enabled. |
| Endpoint Override | (empty) | Custom endpoint. |
| Request Timeout Ms / Connect Timeout Ms | 30000 / 5000 | Time limits per request. |
| Max Retries / Worker Threads | 3 / 4 | Automatic retries; background threads. |
| Verify SSL / Log Sdk Warnings | On / On | TLS verification; SDK warnings in LogLexBot. |
| Default Bot Id | (empty) | Bot used when a node or component leaves Bot Id empty. |
| Default Bot Alias Id | TSTALIASID | Alias used when left empty (TSTALIASID = draft test alias). |
| Default Locale Id | en_US | Locale used when left empty. |
| Default Response Audio | None | Initial Response Audio of new conversation components. |
| Microphone Sample Rate | 16000 | 8000 or 16000. |
| Silence Threshold / Silence Timeout Seconds | 0.02 / 1.2 | Voice-activity detection. |
| Max Utterance Seconds / No Speech Timeout Seconds | 15 / 6 | Recording limits. |
Troubleshooting
| Error code | Cause and fix |
|---|
NotConfigured | No identity pool, or no bot: set Default Bot Id / Alias Id in the settings or pass them to the node. |
AccessDeniedException | The identity pool role lacks lex:RecognizeText / lex:RecognizeUtterance on that bot alias ARN. |
ResourceNotFoundException | Wrong bot id, alias id, locale or region, or the locale was not built. |
ValidationException | Invalid session id characters, a slot that does not exist on the intent, or an unsupported audio format. |
DependencyFailedException | The bot's Lambda function failed or timed out. Check its CloudWatch logs. |
NoSpeech | The microphone heard nothing above Silence Threshold within No Speech Timeout Seconds. Lower the threshold or raise Microphone Gain. |
MicrophonePermission / Unsupported | Android permission not granted (Request Microphone Permission), or a platform without capture (Linux). |
Unavailable | The SDK library could not be loaded or the engine is shutting down. Check the Output Log for LogLexBot errors. |
Credits
© 2026 Alpha XP — alphaxp.net · Support: support@alphaxp.net.
Includes the AWS SDK for C++ and the AWS Common Runtime (Apache License 2.0). Built for Unreal Engine 4.27 and 5.8.