Empowering crowd simulations with generative AI

We’re thrilled to share groundbreaking research combining crowd simulation with generative AI to control individual agents, pushing the boundaries of replicating human behavior in complex scenarios.

This research opens new avenues for urban planning, security management, disaster response, public health, and training. We are excited about the potential of LLMs to revolutionize crowd simulation and deepen our understanding of human behavior in complex scenarios.

Our architectural framework integrates 6 modules, i.e., advanced persona, perception, memory, planning, action, and reflection, to emulate human-like behavior in dynamic environments. The design uses the dual-process theory of cognition, which distinguishes between two modes thinking: Fast and instinctive versus Cautious and analytical.

The framework allows agents to process, store, and retrieve information efficiently, enabling adaptive and realistic decision-making. By incorporating higher-level reflections and adaptive planning, the system enhances the agents' ability to interact coherently and effectively within complex simulations.

Persona module: This module ensures diverse agent behavior by combining three profiling methods: manual creation, automatic generation via prompts, and dataset alignment. This module allows for detailed personality traits, relationships, and social group dynamics, enabling more realistic simulations of individual and crowd behaviors. By blending these approaches, we achieve a nuanced representation of human actions and interactions, crucial for accurately simulating crisis scenarios.

Perception module: This module enables agents to understand and interact with their environment by processing textual inputs and spatial information in real-time. It gathers and updates information within an agent’s perceptual radius, classifying events by importance and storing relevant data in memory, ensuring realistic and adaptive behavior. This module supports the generation of accurate and contextually relevant responses, crucial for maintaining the integrity and effectiveness of the simulation.

Memory module: This module allows agents to reason about past and current experiences to improve future decision-making. It selectively stores and retrieves memories based on recency, importance, and relevance, ensuring agents access the most pertinent information in real-time. This process enhances agents' adaptability and realism by emulating human-like memory functions, essential for effective and consistent behavior in simulations.

Planning module: This module empowers agents with human-like planning and decision-making capabilities, incorporating long-term planning, task decomposition, and immediate planning to navigate both immediate circumstances and long-term objectives. Agents create daily schedules based on internal factors (e.g., goals, emotions) and external influences (e.g., environment, interactions), adjusting their actions dynamically in response to real-time feedback. This module ensures that agents act coherently and believably within the simulation, storing plans in memory for future reflection and adaptation.

Action module: This module translates agents' decisions into specific outcomes by connecting to the game engine’s goal-setting functions, facilitating tasks such as communication, environment navigation, and task completion without physical manipulation. It generates actions based on both memory recollection and pre-generated plans, allowing flexibility in execution and mirroring human behavior. The module’s impacts include updating memory streams, altering internal states, and triggering new actions, which dynamically influence agents' future behaviors and interactions within the environment.

Reflection module: This module enables agents to process vast amounts of observational data by generating higher-level inferences and insights, filtering out noise, and facilitating well-informed decision-making. It periodically prompts agents to reflect on and summarize their experiences, identifying patterns and forming abstract representations of the data, which are then stored in associative memory. This reflective process allows agents to adapt dynamically, making better decisions by considering past observations, insights, and emerging patterns.

Next steps

While our engine can handle 0.5M agents in real-time on a modern PC or 1M+ in the cloud, the video showed a capability demonstrator in SimCrowds / Unity with at most 500 agents. We intend to scale this up to a city with 1M+ real-time agents based on realistic personas, and scale in quality.

Call to action

Are you working on a digital twin application that needs a realistic pattern-of-life (PoL) simulation, a synthetic single world that needs intelligent crowds, a research project that can use our contributions, or powerful visualizations of corresponding crowds, please don’t hesitate to contact us.

Let’s collaborate and bring your vision to life!

#Crowd #Simulation #GenerativeAI #PatternsOfLife #LLM #SimCrowds #Digitaltwins

Read, like and share this post on our LinkedIn!