Human-Robot Interaction for Cashier Robot

Final Year Project
BSc Computer Science (Industrial)

Student: Rafael Papallas
Supervisor: Dr. Mehmet Dogar


The Project Is On GitHub

Demonstration videos of myself. Let the magic begin... 🎩

Here is a video of myself serving a customer 🤗

Another one if you are a bit geeky 🕵

Hi! I am Baxter... Nice to meet you 🤝

Some more information about me


What I do 💪

  • I am detecting new customers entering my area.
  • I accept sweet orders.
  • I will ask you for money (not real ones; in my shop we only have Baxter Banknotes).
  • I will accept your payment by recognising your banknotes.
  • I will return change if necessary.

Banknotes 💸

In my shop only cool banknotes can be used, like the ones below.

They are called "Baxter Banknotes" 💰🤑

Money Recognition

I need to recognise banknotes, and my programmer tried few ideas (three in specific!) and he decided to use AR recognition.


Once I receive the customer's banknotes, I use my forehead camera to do the recognition as you can see from the pictures below.

Don't try to trick me in any way, I am well programmed to avoid your tricks 😇

Lets see the world through my eyes 👀

I don't have eyes 😞 but I do have cameras, I have three in total so I have more "eyes" than you do 😝


I use a depth sensor (Kinect-like sensor) to detect the hand position of the customer.

Here are some pictures of the interaction


On the left is the customer ready to pay 🤑, and on the right I am returning change to the user 😭.

Institution



This website does not belong or own by the University of Leeds. The website was created by the student as part of his final year project and after his own initiative.


Copyright © 2016-2017 The University of Leeds and Rafael Papallas