Published:
A team of PhD students from the National Robotarium at Heriot-Watt University have reached the final of Amazon’s Alexa Prize SimBot Challenge.
The competition, launched in October 2021, challenges robotic researchers from around the world to develop an advanced conversational artificial intelligence, capable of interacting with Amazon Alexa customers.
The Edinburgh-based Team Emma is the only non-US team selected among the five finalists with the winner set to be crowned in early May.
I’m proud of the hard work and effort it has taken Team Emma to reach the final.
Advancing artificial intelligence
Each team was selected based on, among other things, customer feedback and scientific merit of the technical papers they produced.
Dr Alessandro Suglia, a tenure-track research fellow in Robotics and Autonomous Systems, leads Team Emma. He said: “I’m proud of the hard work and effort it has taken Team Emma to reach the final. It is, of course, hugely exciting to showcase our work to industry leaders and it's testament to the exceptional talent we have at the National Robotarium and Heriot-Watt University to be the only team from outside the US to reach this stage.”
The SimBot Challenge is focused on helping advance the science of Embodied AI agents that can engage effectively with humans to understand, learn and collaborate to achieve their given missions.
“We have developed an Alexa skill which provides extra functionality,” continues Dr Suglia.
“In the case of the Simbot challenge, the idea is that users can use their Echo Show to play the Robot Arena, which is a video game involving a robot butler that supports the user in completing tasks in a futuristic lab. In this lab, you have very sophisticated computers and machines that the user can use to solve tasks such as “repair the bowl using the time machine and put milk and cereal in it”.
“Users talk with our robot using natural language, so we had to develop sophisticated Natural Language Understanding skills into our robot to make sure it was able to understand the actions to execute in the environment to accomplish the tasks requested by the users.”
Competing in the finals
During the finals phase, university teams are competing to develop a bot that best responds to commands and multimodal sensor inputs from within a virtual world. Similar to previous Alexa Prize challenges, Alexa customers participate in this phase as well.
Alexa customers can interact with the finalists’ SimBots by saying "Alexa, play with robot" on Echo Show or Fire TV devices to solve progressively harder tasks within the virtual environment. After the interaction, they may provide feedback and ratings for the university bots. That feedback is shared with university teams to help advance their research to win the top awards, totalling $650,000 in cash prizes.
“Next generation autonomous AI assistants will need to be robust and versatile, capable of learning and solving challenging tasks that require multimodal interactions with humans, other agents and the environment,” said Reza Ghanadan, a senior principal scientist in Alexa AI and head of Alexa AI Prize. “It’s wonderful to see the tremendous talent, creativity, passion and contributions of all Alexa Prize SimBot teams during this inaugural challenge towards accelerating the science of robust human AI interactions and conversational embodied AI.”
SimBot Challenge team members
The members of Team Emma are Amit Parekh, a second-year PhD student focused on multi-agent negotiation, and three students from the EPSRC Centre for Doctoral Training in Robotics and Autonomous Systems. These are Hemanthage Supun Bhathiya, a second year PhD student working on multimodal conversational AI; Maria-Vasiliki (Malvina) Nikandrou, a second year PhD student working on continual learning techniques for vision and language models, and Georgios Michail Pantazopoulos, a first year PhD student developing novel techniques to improve the generalisation skills of multimodal machine learning models.
Team Emma will be competing alongside fellow finalists from the University of California (Santa Barbara), Carnegie Mellon University, University of Michigan and the University of California (Santa Cruz).