Emerging Practices 5: Evaluation research/ planning
Over the past week I’ve looked at a number of papers surrounding practical evaluation of the effectiveness of autonomous mobile robots in order to understand how to evaluate aspects of the robocourier. Of two main papers I am using to model my evaluation the first is a study that constructed three full scale arenas to test the ability of a robot to perceive their environment and complete an objective by navigating through the maze [1]. This system developed in this study judges robots that participate objectively based on their completion of objects along with time taken. The second used a mathematical/simulation approach in which the researchers attempted to model every aspect of a robotic courier’s operation in order to compare them with human couriers in a hospital environment [2]. From these I am hoping to be able to create a role-played test that shows the difference between a human and robot performing the same courier role within a limited environment.
The initial design for my evaluation testing is to designate 8 points within aut and take “goods” between each while rolling a dice to determine which point to go to next and recording travel time and units of goods moved. I would firstly do the test for half an hour roleplay as a human meaning I can use stairs, unload myself but only carry half of the robot’s maximum load at any one time. After the human half hour I would repeat the test roleplaying as a robot meaning I have to have someone else load/unload my cargo, limited to elevators or accessible areas at all times but I’ll be able to carry twice as many “units” as compared to the human test.
The test is designed to evaluate the efficiency of each party by comparing the number of units of goods moved vs time taken overall and per route to help determine which one is more efficient as well as which was more efficient per route. My expectation is that in terms of time taken per delivery the human will be better but overall will not be able to compete with the robot’s ability to carry twice as many goods at a time which will lead to overall more deliveries.
I will be refining and then putting this test into action over the next week and will post a finished detailed plan as part of my next evaluation blog post.
Sources:
[1] Jacoff, A., Messina, E., & Evans, J. (2002). Performance evaluation of autonomous mobile robots. Industrial Robot: An International Journal, 29(3), 259-267.
http://www.emeraldinsight.com/doi/pdfplus/10.1108/01439910210425568
(Direct Link to .pdf)
[2]Rossetti, M. D., & Selandari, F. (2001). Multi-objective analysis of hospital delivery systems. Computers & Industrial Engineering, 41(3), 309-333.
http://www.sciencedirect.com/science/article/pii/S0360835201000584
(Link to page with .pdf download)