Disney Research

Recent advancements in generative motion models have achieved remarkable results, enabling the synthesis of lifelike human motions from textual descriptions. These kinematic approaches, while visually appealing, often produce motions that fail to adhere to physical constraints, resulting in artifacts that impede real-world deployment. To address this issue, we introduce a novel method that integrates kinematic generative models with physics based character control. Our approach begins by training a reward surrogate to predict the performance of the downstream non-differentiable control task, offering an efficient and differentiable loss function. This reward model is then employed to fine-tune a baseline generative model, ensuring that the generated motions are not only diverse but also physically plausible for real-world scenarios. The outcome of our processing is the Robot Motion Diffusion Model (RobotMDM), a text-conditioned kinematic diffusion model that interfaces with a reinforcement learning-based tracking controller. We demonstrate the  effectiveness of this method on a challenging humanoid robot, confirming its practical utility and robustness in dynamic environments.

This paper presents an experiment with three artificial agents adopting different strategies when being interrupted by human conversational partners. The agent either ignored the interruption (the most common behavior in conversational engines to date), yielded the turn to the human conversational partner right away, or acknowledged the interruption, finished its thought and then responded to the content of the interruption. Our results show that this change in the agent's conversational behavior had a significant impact on which personality traits people assigned to the agent, as well as how much they enjoyed interacting with it. Moreover, the data also indicates that human interlocutors adapted their own conversational behavior. Our findings suggest that the interactive behavior of an artificial agent should be carefully designed to match its desired personality and the intended conversational dynamics.

We present ongoing work to develop a robust and natural turn-taking behavior for a social agent to engage a dynamically changing group in a conversation.