Conversational agent MIRIAM

MIRIAM programme

The ORCA Hub is the largest academic centre of its kind in the world. A collaborative project led by Heriot-Watt and Edinburgh universities, its focus is on investigating how teams of robots and autonomous intelligent systems (AIS) can enable cheaper, safer and more efficient working practices for the offshore energy industry. 

One critical area of investigation is communication between operators and robots deployed in hazardous environments offshore. Robots and AIS are now able to adapt their behaviour to what they experience and make certain decisions by themselves, so it’s essential that their operators understand what they are doing and why. A lack of transparency can lead to reduced situation awareness, undermine confidence and hinder the human-robotic team relationship, resulting in unnecessary mission aborts or laborious manual manipulation of the robot.

The ORCA Hub is developing techniques that address this issue through better ‘explain-ability’ and collaborative feedback between operators and robots. It has created a conversational agent, MIRIAM – Multimodal Intelligent inteRactIon for Autonomous systeMs – that uses natural language, making it much easier for operators to ask robots questions about their behaviour and to receive easily understandable explanations in return. 

MIRIAM allows users to speak or text their queries, so they can check where a robot is going, what it plans to do next and why, and query data it has gathered during its missions. This invaluable transparency makes for much more trustworthy interactions, helping to build the confidence of users in the robots, and breaking down barriers to using them for complex tasks. 

Currently, testing of MIRIAM is underway with a tracked maintenance robot deployed at global energy company Total’s Shetland Gas Plant. MIRIAM is being modified to the specifics of Total’s robotics programme. Partners involved with ORCA for this project include Phusion IM, which is creating a Master Data Model that will enable the framework for a digital twin of Total’s facilities. As the robot undertakes maintenance checks and feeds back the results in a consistent manner to the digital twin, a single source of information for the operating asset will be created. Merkle Aquila is the industrial data science, data engineering and agile delivery collaborative partner. It is working with the team to develop solutions using AI to extract maximum value from robotic image data, and demonstrate the automation of manual human inspection using computer vision algorithms.

Contact the National Robotarium about research collaboration and business partnership opportunities at NationalRobotarium@hw.ac.uk