Geodesic
Vision-Based Robotic Comanipulation
Russian journal of nonlinear dynamics, Tome 18 (2022) no. 5, pp. 843-858.

Voir la notice de l'article provenant de la source Math-Net.Ru

Although deformable linear objects (DLOs), such as cables, are widely used in the majority of life fields and activities, the robotic manipulation of these objects is considerably more complex compared to the rigid-body manipulation and still an open challenge. In this paper, we introduce a new framework using two robotic arms cooperatively manipulating a DLO from an initial shape to a desired one. Based on visual servoing and computer vision techniques, a perception approach is proposed to detect and sample the DLO as a set of virtual feature points. Then a manipulation planning approach is introduced to map between the motion of the manipulators end effectors and the DLO points by a Jacobian matrix. To avoid excessive stretching of the DLO, the planning approach generates a path for each DLO point forming profiles between the initial and desired shapes. It is guaranteed that all these intershape profiles are reachable and maintain the cable length constraint. The framework and the aforementioned approaches are validated in real-life experiments.
Keywords: robotic comanipulation, deformable linear objects, shape control, visual servoing. 
@article{ND_2022_18_5_a7,
     author = {K. Almaghout and A. Klimchik},
     title = {Vision-Based {Robotic} {Comanipulation}},
     journal = {Russian journal of nonlinear dynamics},
     pages = {843--858},
     publisher = {mathdoc},
     volume = {18},
     number = {5},
     year = {2022},
     language = {en},
     url = {http://geodesic.mathdoc.fr/item/ND_2022_18_5_a7/}
}
TY  - JOUR
AU  - K. Almaghout
AU  - A. Klimchik
TI  - Vision-Based Robotic Comanipulation
JO  - Russian journal of nonlinear dynamics
PY  - 2022
SP  - 843
EP  - 858
VL  - 18
IS  - 5
PB  - mathdoc
UR  - http://geodesic.mathdoc.fr/item/ND_2022_18_5_a7/
LA  - en
ID  - ND_2022_18_5_a7
ER  - 
%0 Journal Article
%A K. Almaghout
%A A. Klimchik
%T Vision-Based Robotic Comanipulation
%J Russian journal of nonlinear dynamics
%D 2022
%P 843-858
%V 18
%N 5
%I mathdoc
%U http://geodesic.mathdoc.fr/item/ND_2022_18_5_a7/
%G en
%F ND_2022_18_5_a7
K. Almaghout; A. Klimchik. Vision-Based Robotic Comanipulation. Russian journal of nonlinear dynamics, Tome 18 (2022) no. 5, pp. 843-858. http://geodesic.mathdoc.fr/item/ND_2022_18_5_a7/

[1] Zhu, J., Cherubini, A., Dune, C., Navarro-Alarcon, D., Alambeigi, F., Berenson, D., Ficuciello, F., Harada, K., Kober, J., Li, X., Pan, J., Yuan, W., and Gienger, M., “Challenges and Outlook in Robotic Manipulation of Deformable Objects”, IEEE Robot. Autom. Lett., 29:3 (2022), 67–77

[2] Cirillo, P., Laudante, G., and Pirozzi, S., “Vision-Based Robotic Solution for Wire Insertion with an Assigned Label Orientation”, IEEE Access, 9 (2021), 102278–102289

[3] Chang, P. and Padír, T., “Model-Based Manipulation of Linear Flexible Objects: Task Automation in Simulation and Real World”, Machines, 8:3 (2020), 46, 22 pp.

[4] Zhu, J., Navarro, B., Passama, R., Fraisse, Ph., Crosnier, A., and Cherubini, A., “Robotic Manipulation Planning for Shaping Deformable Linear Objects with Environmental Contacts”, IEEE Robot. Autom. Lett., 5:1 (2019), 16–23

[5] White, J. R., Satterlee, P. E., Jr., Walker, K. L., and Harvey, H. W., Remotely Controlled and/or Powered Mobile Robot with Cable Management Arrangement, Patent US No. 4 736 826, 12 Apr 1988

[6] Yin, H., Varava, A., and Kragic, D., “Modeling, Learning, Perception, and Control Methods for Deformable Object Manipulation”, Sci. Robot., 6:54 (2021), eabd8803, 16 pp.

[7] Sanchez, J., Mohy El Dine, K., Corrales, J. A., Bouzgarrou, B.-Ch., and Mezouar, Y., “Blind Manipulation of Deformable Objects Based on Force Sensing and Finite Element Modeling”, Front. Robot. AI, 7 (2020), 73, 10 pp.

[8] Delgado, A., Corrales, J. A., Mezouar, Y., Lequievre, L., Jara, C., and Torres, F., “Tactile Control Based on Gaussian Images and Its Application in Bi-Manual Manipulation of Deformable Objects”, Robot. Auton. Syst., 94 (2017), 148–161

[9] Almaghout, K., Boby, R. A., Othman, M., Shaarawy, A., and Klimchik, A., “Robotic Pick and Assembly Using Deep Learning and Hybrid Vision/Force Control”, Proc. of the 2021 Internat. Conf. “Nonlinearity, Information and Robotics”\ (NIR, Innopolis, Russia, Aug 2021), 6 pp.

[10] Sanchez, J., Corrales, J. A., Bouzgarrou, B. Ch., and Mezouar, Y., “Robotic Manipulation and Sensing of Deformable Objects in Domestic and Industrial Applications: A Survey”, Int. J. Rob. Res., 37:7 (2018), 688–716

[11] Tang, T., Wang, Ch., and Tomizuka, M., “A Framework for Manipulating Deformable Linear Objects by Coherent Point Drift”, IEEE Robot. Autom. Lett., 3:4 (2018), 3426–3433

[12] Li, X., Wang, Z., and Liu, Y.-H., “Sequential Robotic Manipulation for Active Shape Control of Deformable Linear Objects”, Proc. of the 2019 IEEE International Conference on Real-Time Computing and Robotics (RCAR, Irkutsk, Russia, Aug 2019), 840–845

[13] Yang, Y., Stork, J. A., and Stoyanov, T., “Learning to Propagate Interaction Effects for Modeling Deformable Linear Objects Dynamics”, Proc. of the 2021 IEEE Internat. Conf. on Robotics and Automation (ICRA, Xi'an, China, Oct 2021), 1950–1957

[14] Zhu, J., Navarro, B., Fraisse, Ph., Crosnier, A., and Cherubini, A., “Dual-Arm Robotic Manipulation of Flexible Cables”, Proc. of the 2018 IEEE/RSJ Internat. Conf. on Intelligent Robots and Systems (IROS, Madrid, Spain, Oct 2018), 479–484

[15] Zhu, J., Navarro-Alarcon, D., Passama, R., and Cherubini, A., “Vision-Based Manipulation of Deformable and Rigid Objects Using Subspace Projections of 2D Contours”, Robot. Auton. Syst., 142 (2021), 103798, 35 pp.

[16] Heisler, P., Steinmetz, P., Yoo, I. S., and Franke, J., “Automatization of the Cable-Routing-Process within the Automated Production of Wiring Systems”, Appl. Mech. Mater., 871 (2017), 186–192

[17] Herguedas, R., López-Nicolás, G., Aragüés, R., and Sagüés, C., “Survey on Multi-Robot Manipulation of Deformable Objects”, Proc. of the 24th IEEE Internat. Conf. on Emerging Technologies and Factory Automation (ETFA, Zaragoza, Spain, Sep 2019), 977–984

[18] Lv, N., Liu, J., and Jia, Y., “Coordinated Control of Flexible Cables with Human-Like Dual Manipulators”, J. Dyn. Syst. Meas. Control Trans. ASME, 143:8 (2021), 081006, 11 pp.

[19] Jin, Sh., Wang, Ch., and Tomizuka, M., “Robust Deformation Model Approximation for Robotic Cable Manipulation”, Proc. of the 2019 IEEE/RSJ Internat. Conf. on Intelligent Robots and Systems (IROS, Macau, China, Nov 2019), 6586–6593

[20] Lagneau, R., Krupa, A., and Marchal, M., “Automatic Shape Control of Deformable Wires Based on Model-Free Visual Servoing”, IEEE Robot. Autom. Lett., 5:4 (2020), 5252–5259

[21] Guo, Z. and Hall, R. W., “Fast Fully Parallel Thinning Algorithms”, CVGIP: Image Understanding, 55:3 (1992), 317–328

[22] Berenson, D., “Manipulation of Deformable Objects without Modeling and Simulating Deformation”, Proc. of the 2013 IEEE/RSJ Internat. Conf. on Intelligent Robots and Systems (Tokyo, Japan, Nov 2013), 4525–4532

[23] Bradski, G., “The OpenCV Library”, Dr. Dobb's Journal of Software Tools, 25 (2000), 122–125

[24] Zhang, Zh., “A Flexible New Technique for Camera Calibration”, IEEE Trans. Pattern Anal. Mach. Intell., 22:11 (2000), 1330–1334

[25] Heyden, A. and Pollefeys, M., “Multiple View Geometry”, Emerging Topics in Computer Vision, eds. G. Medioni, S. B. Kang, Prentice Hall, Upper Saddle River, N. J., 2004, 45–108

[26] Wang, Y. M., Li, Y., and Zheng, J. B., “A Camera Calibration Technique Based on OpenCV”, Proc. of the 3rd Internat. Conf. on Information Sciences and Interaction Sciences (ICIS, Chengdu, China, Jun 2010), 403–406

[27] Robotic Operating System (ROS Melodic Morenia), , 2018 https://www.ros.org

[28] Hennersperger, Ch., Fuerst, B., Virga, S., Zettinig, O., Frisch, B., Neff, Th., and Navab, N., “Towards MRI-Based Autonomous Robotic US Acquisitions: A First Feasibility Study”, IEEE Trans. Med. Imaging, 36:2 (2017), 538–548 \if0

[29] Lv, N., Liu, J., Xia, H., and Jia, Y., “Dynamic Modeling and Control of Flexible Cables for Shape Forming”, Proc. of the ASME 2019 Dynamic Systems and Control Conference (Park City, Utah, Oct 2019), v. 1, DSCC2019-9049, V001T03A006, 8 pp.

[30] Jin, Sh., Romeres, D., Ragunathan, A., Jha, D. K., and Tomizuka, M., “Trajectory Optimization for Manipulation of Deformable Objects: Assembly of Belt Drive Units”, Proc. of the 2021 IEEE Internat. Conf. on Robotics and Automation (ICRA, Xi'an, China, Oct 2021), 10002–10008

[31] Leizea, I., Mendizabal, A., Alvarez, H., Aguinaga, I., Borro, D., and Sanchez, E., “Real-Time Visual Tracking of Deformable Objects in Robot-Assisted Surgery”, IEEE Comput. Graph. Appl., 37:01 (2017), 56–68 \fi