The Moonshot Research and Development Program
“Realization of Artificial Intelligence (AI) robots that autonomously learn, adapt to their environment, evolve itself in intelligence, and act alongside human beings, by 2050.”
System Integration Division Conference of Society of Instrument and Control Engineers (SICE) was held in Morioka from December 18th to 20th, 2024. At the conference, we held an organized session focused on this project and presented 16 technical papers.
A team from Seikei University conducted a public demonstration experiment of an automated crushed stone transport system at SK Material's Agano Field.
Osaka University's separable and combinable robot that can be reconfigured for unspecified environments was exhibited at the IEEE ICRA 2024 (International Conference on Robotics and Automation).
Osaka University's i-CentiPot-Ammonite, a separable and combinable robot that can be reconfigured for unspecified environments, and BRAINS, a general-purpose transport container for the robot, were proposed.
The results of the visit will be used in the demonstration experiments of the development equipment.
A demonstration of the force control of the next-generation work equipment Cafe developed in this project was given at a remote construction demonstration by the MLIT.
Tohoku University conducted a demonstration of ground strength estimation and moving using bucket excavation. The photo shows a mini excavator moving with avoiding the soft ground by estimating the ground strength.
We carried out a transportation and installation test of a topographical measurement sensor pod in order to survey the topography of river channel blockage.
Tokyo University, Seikei University, Public works research institute, Osaka University, and Kyushu University jointly conducted an experimental demonstrations of an automatic sediment transport system at the outdoor field of Kyushu University.
We performed the experimental demonstrations on lunar construction works using five different types of small robotic platforms, deployed at JAXA’s test field.
An outdoor experiment on rough terrain was conducted using a flexible mobile robot developed by Osaka University. The photo shows i-Centipot Amphibian Jr. flexibly moving from water to land.
We performed experimental tests on robotic mobility and construction works using the small robotic platforms at JAXA’s test field.
An outdoor experiment was conducted using a flexible mobile robot developed by Osaka University. The photo shows the ''d-FlexCraw'' running outdoor experiment field.
We have conducted a demonstration of sensing equipment，a communication test, and aerial drone photography at the Futase Dam (Chichibu City).
We conducted compaction experiments using a vibratory roller to verify the ground stiffness estimation method by Kyushu University in the experimental pit.
We developed a stiffness evaluation method for compacted ground using distributed sensor pods.
We developed a method for estimating the position of construction vehicles using 3D point clouds obtained from sensor pods.
We experimented on the remote and autonomous driving control for a wheel loader with our developed retrofitted device.
PWRI has developed and released “OPERA” on GitHub.
We have conducted preliminary tests of the small robotic platforms at the sand field at JAXA
Organized Session on Disaster Response Robot System was held at the conference SI2024,
The University of Tokyo and Seikei University jointly conducted of an automatic transport system at the field of SK Material Corporation.
Osaka University’s Disaster Response Robot System was exhibited at the international conference ICRA2024.
A press demonstration of a disaster response robot system (i-CentiPot-Ammonite, BRAINS) based on “open design” was held at Osaka University.
This project has transitioned to a new system.
We, the CAFE project, are working on the research and development of “Collaborative AI robots for adaptation of diverse environments and innovation of infrastructure construction” in the group of Moonshot Goal #3 “AI robots that learn and act by themselves and coexist with people.” The CAFE project is derived from the Collaborative AI Field robot Everywhere. Starting in 2024, we will transition to a new organizational structure, aiming for system integration of the technologies we have developed so far.
Disaster response at the site of
a natural disaster in 2050
In Japan, where we live, various natural disasters frequently occur across the country. To protect the lives of the people living here and minimize the damage caused by these disasters, further technological development for emergency restoration is desired. Therefore, in this project, We will conduct research and development of “collaborative AI robots” that can respond flexibly to different situations and perform tasks in difficult environments such as disaster sites. This technology will also be useful in the construction and maintenance of infrastructure on the ground.
The illustration on the left depicts the concept of emergency restoration at a natural disaster site in 2050. Multiple field robots assess the disaster situation accurately based on sensing data, and then engage in collaborative work to handle unexpected situations while carrying out emergency restoration. As a milestone for this project by 2025, we aim to develop a prototype system of multiple collaborative AI robots capable of flexibly responding to unforeseen situations, specifically to mitigate the impact of natural disasters, with a focus on river blockage disasters.
The overarching goal of this research and development project is “to achieve infrastructure construction adapted to various environments through a collaborative AI robot system.” To realize this goal, we have established three research and development themes: 1. AI Robot System Revolutionizing Earthwork (hardware), 2. Dynamic Collaboration System for Multiple Robots (AI controlling multiple robots), and 3. Sensor Pod System Overseeing the Site (sensing technology and AI evaluating the environment). We have been advancing research and development on these themes in parallel.
From 2024 onwards, as shown in the figure on the right, we will focus on advancing robotic systems capable of responding to natural disasters, particularly river blockage disasters. Alongside technological innovations, we aim to integrate these technologies into a cohesive system.
The project named “Collaborative AI robots for adaptation of diverse environments and innovation of infrastructure construction,” aims to develop collaborative field robots that can be useful in “natural disaster response.” To achieve this goal, robots must be able to flexibly respond to various situations. In this project, we aim to achieve this goal through a variety of approaches.
The introduction to this project is provided in the section “Overview of CAFE Project,” and I, the project manager, would like to share three things that are important to me in this project.
The first is the basic philosophy of this project. The basic philosophy of this project is “to develop technology that is useful to people.” It is the same basic philosophy that I have always cherished in my robotics research. I do not want to excuse myself from saying, “I completed my research, and it will be practical use if somebody complete other required parts.” in this policy.
The second aspect is the horizontal connection between the researchers and engineers. This project requires interdisciplinary cooperation, and it will not be possible without “civil engineering,” “mechanical engineering,” and “AI.” Therefore, we would like to promote the research and development, where researchers from each field respect each other's field and closely exchange information to strengthen the horizontal collaboration.
The third is a field-oriented approach. In this project, we do not want to artificially create and demonstrate a field where robots can efficiently operate. Instead, we want to develop robots that can operate in actual fields.
The members of this project are very powerful researchers and engineers who are currently active in their respective research fields. I am happy to work with them. Please look forward to our future research.
Keiji Nagatani, Project Manager
He received his master’s degree from Osaka University and Ph.D. (Engineering) from the Graduate School of Engineering Science in 1984. He is a professor at the Department of Mechanical Engineering, Graduate School of Engineering, Osaka University, Director of Creative Design Studio on Technology, and Director of the Komatsu MIRAI Construction Equipment Cooperative Research Center, where he researches on robotics and control engineering.
Koichi Osuka
Professor in Graduate School of Engineering, Osaka University
He received his master’s degree from the Graduate School of Tsukuba University in 1993 and Ph.D. (Engineering) from Bielefeld University in 2010. He is currently engaged in the research and development of field robotics for agriculture, civil engineering and construction, as well as marine applications at Yanmar Holdings.
Hisashi Sugiura
Group Leader of Robotics Group, System Research Center, R&D Center, Innovation and Technology Division, Yanmar Holdings Co.
Part-time lecturer at Ritsumeikan and Meijo University
He received his Ph.D. from Yokohama National University, the Doctoral Program in 1990. After working for Toshiba Corporation, Micromachine Center, and Okayama University, he was appointed to his current position in 2014. He is the president of s-muscle Co. Ltd., H-MUSCLE Co. Ltd., and the representative of the new academic field “Science of Soft Robots.”
Koichi Suzumori
Professor of Institute of Science Tokyo
He received his Ph.D. from the Doctoral Program of Engineering, Kyushu University, in 1999. He researches on the applications of geospatial information and disaster prevention based on rock engineering and geo-environmental engineering.
Yasuhiro Mitani
Professor of Disaster Risk Reduction Research Center, Graduate School of Engineering, Kyusyu University
He received his Ph.D. from Wayne State University in 2010. He worked as a research associate at Yonsei University, Ehime University, and is currently a project associate professor at the University of Tokyo. His research interest is in civil engineering, especially in the application of ICT technology to the maintenance and management of bridges and tunnels.
Pang-jo Chun
Project Associate Professor, The University of Tokyo
He received his B. Sc., M.Sc., and Ph.D. degrees from the University of Tokyo in 1994, 1996, and 1999, respectively. He is currently a Professor in Computer Vision at Tohoku University, and serves as a leader of the infrastructure management robotics team at RIKEN Center since 2016. His research interests are in the fields of computer vision and machine learning.
Takayuki Okatani
Team Leader of RIKEN, Japan
He graduated from the Faculty of Engineering, Tokai University, in 1983. He engages in the development and research of construction ICT, automation of construction machinery, and unmanned construction technology at Kumagai Gumi Co., Ltd.
Shigeo Kitahara
General Manager, ICT Promotion Office, Civil Engineering Division, Kumagai Gumi Co., Ltd.
He received a master’s degree from the Graduate School of Agriculture, Iwate University Master Course in 1985. Currently, he engages in research on the planning and design of volcano robots for natural disaster prevention during volcanic eruptions.
Toru Shimada
Manager of Business Planning,
Public Consultant Division Kokusai Kogyo Co., Ltd.,
He received M. S. and Dr. Eng. from the University of Tokyo in 1984 and 1989. He became a professor of Research into Artifacts, Center for Engineering (RACE) of the University of Tokyo, a professor at the School of Engineering since 2009, and the director of RACE since 2019. His main research interests are service robotics, distributed autonomous robotic systems, embodied brain science systems, and cognitive ergonomics.
Hajime Asama
Professor, The University of Tokyo
He received his Ph.D. in Information Science from NAIST in 2007. He is currently a tenure-track associate professor and director of the Robot Learning Lab. His research interests include machine learning and its applications in real-world robots.
Takamitsu Matsubara
Associate Professor, Institute for Research Initiatives, Nara Institute of Science and Technology.
He received his Ph.D. from the Tohoku University Doctoral Program in 1986. His main research field is terramechanics.
Hiroshi Takahashi
Professor of Graduate School of Environmental Studies,
Tohoku University
He received his Ph.D. degree from the Department of Mechanical Engineering Science, Tokyo Institute of Technology in 1998. He is a JSME Fellow, SICE Fellow, and RSJ Fellow. His current research interests include legged robot control, computer vision, multiple mobile robots, space robots, service robots, medical imaging, biometrics, and dementia care.
Ryo Kurazume
Professor at the Graduate School of Information Science and Electrical Engineering, Kyushu University
He received his Ph.D. from the University of Tsukuba, Japan, in 2007. After working at the Ibaraki Prefectural Institute of Industrial Technology, the National Institute of Advanced Industrial Science and Technology, the University of Tsukuba, Seikei University, and Hirosaki University, currently holds the present position. Engaged in research on the social implementation of robotic control technologies and haptic technologies.
Toshinobu Takei
Associate Professor, Seikei University.
Hiromitsu Fujii
Associate Professor in Chiba Inst. of Technology
Robert received his Ph.D. in Mechanical Engineering from the Massachusetts Institute of Technology (MIT) in 2018. He is an Assistant Professor of Robotics at ETH Zurich, where he leads the Soft Robotics Lab. He is also a core faculty member at the ETH AI Center, the ETH Center for Robotics, and the ETH-MPI Center for Learning Systems. His research focuses on musculoskeletal robotics, soft robotics, and biohybrid robotics, aiming to develop lifelike robots that interact safely with humans and the environment.
Robert Katzschmann
Assistant Professor, ETH Zurich
08/18/2023
Journal
Thannarot Kunlamai, Tatsuro Yamane, Masanori Suganuma, Pang-Jo Chun, Takayuki Okatani (2023), “Improving visual question answering for bridge inspection by pre-training with external data of image–text pairs”, Computer-Aided Civil and Infrastructure Engineering., August, 2023. pp. 1-17
07/02/2023
Journal
Yusuke Tamaishi, Kentaro Fukuda, Kazuto Nakashima, Ryuichi Maeda, Kohei Matsumoto, Ryo Kurazume (2023), “Evaluation of ground stiffness using multiple accelerometers on the ground during compaction by vibratory rollers”, Proceedings of the 40th International Symposium on Automation and Robotics in Construction., July, 2023. pp. 262-269
04/10/2023
Journal
Yuki Kadokawa, Lingwei Zhu, Yoshihisa Tsurumine, Takamitsu Matsubara (2022), “Cyclic Policy Distillation: Sample-Efficient Sim-to-Real Reinforcement Learning with Domain Randomization”, Robotics and Autonomous Systems., April, 2023. Vol. 165, pp. 1-30.
03/22/2023
Journal
Hirotaka Tahara, Hikaru Sasaki, Hanbit Oh, Edgar Anarossi, Takamitsu Matsubara (2023), “Disturbance Injection under Partial Automation: Robust Imitation Learning for Long-horizon Tasks”, IEEE Robotics and Automation Letters.,March, 2023. Vol. 8(5), pp. 2724-2731
03/02/2023
Journal
Huyen Tran, Takayuki Okatani (2023), “Bright as the Sun: In-depth Analysis of Imagination-driven Image Captioning”, Asian Conference on Computer Vision., March, 2023. Vol. 13844, pp. 675–691.
02/15/2023
Journal
Hidenori Takamiya, Ryosuke Yajima, Jun Younes Louhi Kasahara, Ren Komatsu, Keiji Nagatani, Atsushi Yamashita, Hajime Asama (2023), “Reinforcement Learning-based Motion Generation for a Tracked Robot to Go Over a Sphere-shaped Non-fixed Obstacle”, 2023 IEEE/SICE International Symposium on System Integration (SII),. February, 2023. pp. 1-6.
01/25/2023
Journal
Riku Tada, Yusuke Tsunoda, Teruyo Wada, Koichi Osuka (2023), “Proposal of efficient shepherding controller with adjusting gains according to the navigation phase”, AROB-ISBC-SWARM2023,. January, 2023. pp. 1439-1444.
01/25/2023
Journal
Runze Xiao, Yusuke Tsunoda, Koichi Osuka (2023), “The Exploitation of “Unfavorable” Environmental Effects in a Centipede-type Swarm Robot System for Unknown Environment Navigation and Exploration”, AROB-ISBC-SWARM2023,. January, 2023. pp. 879-882.
11/01/2022
Journal
Kang-Jun Liu, Masanori Suganuma, Takayuki Okatani (2022), “Bridging the Gap from Asymmetry Tricks to Decorrelation Principles in Non-contrastive Self-supervised Learning”, Thirty-sixth Conference on Neural Information Processing Systems., November, 2022. vol. 35, pp. 1-12.
10/06/2022
Journal
Tatsuro Yamane, Pang-jo Chun, Riki Honda (2022), “Detecting and Localising Damage Based on Image Recognition and Structure from Motion, and Reflecting it in a 3D Bridge Model”, Structure and Infrastructure Engineering., October, 2022. pp. 1-13.
JAPANESE