Performance Evaluation of Convolutional Networks on Heterogeneous Architectures for Applications in Autonomous Robotics
Abstract
Humanoid robots find application in human-robot interaction tasks. However, despite their capabilities, their sequential computing system limits the execution of computationally expensive algorithms such as convolutional neural networks, which have demonstrated good performance in recognition tasks. As an alternative to sequential computing units, Field-Programmable Gate Arrays and Graphics Processing Units have a high degree of parallelism and low power consumption. This study aims to improve the visual perception of a humanoid robot called NAO using these embedded systems running a convolutional neural network. The methodology adopted here is based on image acquisition and transmission using simulation software: Webots and Choreographe. In each embedded system, an object recognition stage is performed using commercial convolutional neural network acceleration frameworks. Xilinx® Ultra96™, Intel® Cyclone® V-SoC and NVIDIA® Jetson™ TX2 cards were used, and Tinier-YOLO, AlexNet, Inception-V1 and Inception V3 transfer-learning networks were executed. Real-time metrics were obtained when Inception V1, Inception V3 transfer-learning and AlexNet were run on the Ultra96 and Jetson TX2 cards, with frame rates between 28 and 30 frames per second. The results demonstrated that the use of these embedded systems and convolutional neural networks can provide humanoid robots such as NAO with greater visual recognition in tasks that require high accuracy and autonomy.
References
S. R. Fanello; C. Ciliberto; N. Noceti, G. Metta; F. Odone, “Visual recognition for humanoid robots”, Rob. Auton Syst., vol. 91, pp. 151–168, May 2017. https://doi.org/10.1016/j.robot.2016.10.001
E. Cha; M. Mataric; T. Fong, “Nonverbal signaling for non-humanoid robots during human-robot collaboration”, in 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2016, pp. 601–602. https://doi.org/10.1109/HRI.2016.7451876
S. Shamsuddin et al., “Initial response of autistic children in human-robot interaction therapy with humanoid robot NAO”, in 2012 IEEE 8th International Colloquium on Signal Processing and its Applications, 2012, pp. 188–193. https://doi.org/10.1109/CSPA.2012.6194716
J. G. Hoyos-Gutiérrez; C. A. Peña-Solórzano; C. L. Garzón-Castro; F. A. Prieto-Ortiz; J. G. Ayala-Garzón, “Hacia el manejo de una herramienta por un robot NAO usando programación por demostración”, TecnoLógicas, vol. 17, no. 33, pp. 65-76, Aug. 2014. https://doi.org/10.22430/22565337.555
P. Vadakkepat; N. B. Sin; D. Goswami; R. X. Zhang; L. Y. Tan, “Soccer playing humanoid robots: Processing architecture, gait generation and vision system”, Rob. Auton. Syst., vol. 57, no. 8, pp. 776–785, Jul. 2009. https://doi.org/10.1016/j.robot.2009.03.012
A. Härtl; U. Visser; T. Röfer, “Robust and Efficient Object Recognition for a Humanoid Soccer Robot”, Springer, Berlin, Heidelberg, 2014, pp. 396–407. https://doi.org/10.1007/978-3-662-44468-9_35
D. Budden; S. Fenn; J. Walker; A. Mendes, “A Novel Approach to Ball Detection for Humanoid Robot Soccer”, Springer, Berlín, Heidelberg, 2012, pp. 827–838. https://doi.org/10.1007/978-3-642-35101-3_70
P. Sermanet; D. Eigen; X. Zhang; M. Mathieu; R. Fergus; Y. Le Cun, “Integrated recognition, localization and detection using convolutional networks”, 2013. https://arxiv.org/abs/1312.6229
A. Krizhevsk; I. Sutskever; G. E. Hinton, “ImageNet classification with deep convolutional neural networks”, Commun. ACM, vol. 60, no. 6, pp. 84–90, Jun 2017. https://doi.org/10.1145/3065386
K. Simonyan; A. Zisserman, “Very deep convolutional networks for large-scale image recognition”, arXiv preprint, Sep.2015. https://arxiv.org/abs/1409.1556
K. He; X. Zhang; S. Ren; J. Sun, “Deep Residual Learning for Image Recognition”, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778. https://doi.org/10.1109/CVPR.2016.90
H. V. Nguyen; H. T. Ho; V. M. Patel; R. Chellappa, “DASH-N: Joint Hierarchical Domain Adaptation and Feature Learning,” IEEE Trans. Image Process., vol. 24, no. 12, pp. 5479–5491, Dec. 2015. https://doi.org/10.1109/TIP.2015.2479405
M. Podpora; A. Gardecki, “Extending vision understanding capabilities of NAO robot by connecting it to a remote computational resource,” in 2016 Progress in Applied Electrical Engineering (PAEE), 2016, pp. 1–5. https://doi.org/10.1109/PAEE.2016.7605119
M. Puheim; M. Bundzel; L. Madarasz, “Forward control of robotic arm using the information from stereo-vision tracking system”, in 2013 IEEE 14th International Symposium on Computational Intelligence and Informatics (CINTI), 2013, pp. 57–62. https://doi.org/10.1109/CINTI.2013.6705259
K. Noda; H. Arie; Y. Suga; T. Ogata, “Multimodal integration learning of robot behavior using deep neural networks”, Rob. Auton. Syst., vol. 62, no. 6, pp. 721–736, Jun. 2014. https://doi.org/10.1016/j.robot.2014.03.003
A. Biddulph; T. Houliston; A. Mendes; S. K. Chalup, “Comparing Computing Platforms for Deep Learning on a Humanoid Robot”, Springer, Cham, 2018. https://doi.org/10.1007/978-3-030-04239-4_11
A. Dundar; J. Jin; B. Martini; E. Culurciello, “Embedded Streaming Deep Neural Networks Accelerator With Applications,” IEEE Trans. Neural Networks Learn. Syst., vol. 28, no. 7, pp. 1572–1583, Jul. 2017. https://doi.org/10.1109/TNNLS.2016.2545298
H. Park et al., “Optimizing DCNN FPGA accelerator design for handwritten hangul character recognition”, in Proceedings of the 2017 International Conference on Compilers, Architectures and Synthesis for Embedded Systems Companion, 2017, pp. 1–2. https://doi.org/10.1145/3125501.3125522
C. Zhang; P. Li; G. Sun; Y. Guan; B. Xiao; J. Cong, “Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks”, in Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, 2015, pp. 161–170. https://doi.org/10.1145/2684746.2689060
Q. Xiao; Y. Liang; L. Lu; S. Yan; Y.-W. Tai, “Exploring Heterogeneous Algorithms for Accelerating Deep Convolutional Neural Networks on FPGAs”, in Proceedings of the 54th Annual Design Automation Conference 2017, 2017, pp. 1–6. https://doi.org/10.1145/3061639.3062244
E. Del Sozzo; A. Solazzo; A. Miele; M. D. Santambrogio, “On the Automation of High Level Synthesis of Convolutional Neural Networks”, in 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2016, pp. 217–224. https://doi.org/10.1109/IPDPSW.2016.153
C. Zhang; G. Sun; Z. Fang; P. Zhou; P. Pan; J. Cong, “Caffeine: Toward Uniformed Representation and Acceleration for Deep Convolutional Neural Networks”, in IEEE Trans. Comput. Des. Integr. Circuits Syst., vol. 38, no. 11, pp. 2072–2085, Nov. 2019. https://doi.org/10.1109/TCAD.2017.2785257
R. Andri; L. Cavigelli; D. Rossi; L. Benini, “YodaNN: An Ultra-Low Power Convolutional Neural Network Accelerator Based on Binary Weights”, in 2016 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), 2016, pp. 236–241. https://doi.org/10.1109/ISVLSI.2016.111
L. Ni; Z. Liu; H. Yu; R. V. Joshi, “An Energy-Efficient Digital ReRAM-Crossbar-Based CNN With Bitwise Parallelism”, IEEE J. Explor. Solid-State Comput. Devices Circuits, vol. 3, pp. 37–46, Dec. 2017. https://doi.org/10.1109/JXCDC.2017.2697910
A. Kulkarni; T. Abtahi; C. Shea; A. Kulkarni; T. Mohsenin, “PACENet: Energy efficient acceleration for convolutional network on embedded platform”, in 2017 IEEE International Symposium on Circuits and Systems (ISCAS), 2017, pp. 1–4. https://doi.org/10.1109/ISCAS.2017.8050342
T. Gong; T. Fan; J. Guo; Z. Cai, “GPU-based parallel optimization of immune convolutional neural network and embedded system”, Eng. Appl. Artif. Intell., vol. 62, pp. 384–395, Jun. 2017. https://doi.org/10.1016/j.engappai.2016.08.019
D. Strigl; K. Kofler; S. Podlipnig, “Performance and Scalability of GPU-Based Convolutional Neural Networks”, in 2010 18th Euromicro Conference on Parallel, Distributed and Network-based Processing, 2010, pp. 317–324. https://doi.org/10.1109/PDP.2010.43
O. Michel, “Cyberbotics Ltd. Webots TM: Professional Mobile Robot Simulation”, Int. J. Adv. Robot. Syst., vol. 1, no. 1, p. 39-42, Mar. 2004. https://doi.org/10.5772/5618
E. Pot; J. Monceaux; R. Gelin; B. Maisonnier, “Choregraphe: a graphical tool for humanoid robot programming”, in RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication, 2009, pp. 46–51. https://doi.org/10.1109/ROMAN.2009.5326209
M. Blott et al., “FINN- R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networks”, ACM Trans. Reconfigurable Technol. Syst., vol. 11, no. 3, pp. 1–23, Sep. 2018. https://doi.org/10.1145/3242897
S. Wang; S. Jiang, “INSTRE: A New Benchmark for Instance-Level Object Retrieval and Recognition”, ACM Trans. Multimed. Comput. Commun. Appl., vol. 11, no. 3, pp. 1–21, Feb. 2015. https://doi.org/10.1145/2700292
M. Mattamala; G. Olave; C. González; N. Hasbún; J. Ruiz-del-Solar, “The NAO Backpack: An Open-Hardware Add-on for Fast Software Development with the NAO Robot”, 2018, pp. 302–311. https://doi.org/10.1007/978-3-030-00308-1_25
D. Wang; K. Xu; D. Jiang, “PipeCNN: An OpenCL-based open-source FPGA accelerator for convolution neural networks”, in 2017 International Conference on Field Programmable Technology (ICFPT), 2017, pp. 279–282. https://doi.org/10.1109/FPT.2017.8280160
S. Xu; A. Savvaris; S. He; H. Shin; A. Tsourdos, “Real-time Implementation of YOLO+JPDA for Small Scale UAV Multiple Object Tracking”, in 2018 International Conference on Unmanned Aircraft Systems (ICUAS), 2018, pp. 1336–1341. https://doi.org/10.1109/ICUAS.2018.8453398
J. Ma; L. Chen; Z. Gao, “Hardware Implementation and Optimization of Tiny-YOLO Network”, Springer, Singapur, 2018, pp. 224–234, https://doi.org/10.1007/978-981-10-8108-8_21
Downloads
