The tablet accesses the web server with the Online Bar via WiFi.
The web server is hosted on the same Raspberry Pi to which the webcam is connected.
After pressing the start button on the web page, the web server locally calls a Python script, which sends the message to start the robot to the IP-to-RS232 gateway via WiFi.
The gateway forwards the message via RS232 to the KRC1 robot controler.
The KRC1 starts the robot to open the refrigerator door and then confirms to the Python script that the robot is standing in front of the open refrigerator.
The Python script starts the object detection based on pattern matching with OpenCV, decides which bottle to grip, saves the analysed image and the calculated position in two files.
After finishing the Python script, the web page uses AJAX technology to display the analyzed image and starts a second Python script that sends a message to the robot to pick up the bottle from the read position and open it.