Working Papers
Sorry, no publications matched your criteria.
Under Review
Sorry, no publications matched your criteria.
Journal Articles
Sorry, no publications matched your criteria.
Conference Papers
1.
Fang Wan, Xiaobo Liu, Ning Guo, Xudong Han, Feng Tian, Chaoyang Song
Visual Learning Towards Soft Robot Force Control using a 3D Metamaterial with Differential Stiffness Conference
Conference on Robot Learning (CoRL2021), London & Virtual, 2021.
Abstract | Links | BibTeX | Tags: Authorship - Corresponding, Conf - CoRL
@conference{Wan2022VisualLearning,
title = {Visual Learning Towards Soft Robot Force Control using a 3D Metamaterial with Differential Stiffness},
author = {Fang Wan and Xiaobo Liu and Ning Guo and Xudong Han and Feng Tian and Chaoyang Song},
url = {https://proceedings.mlr.press/v164/wan22a/wan22a.pdf},
year = {2021},
date = {2021-11-08},
urldate = {2021-11-08},
booktitle = {Conference on Robot Learning (CoRL2021)},
address = {London & Virtual},
abstract = {This paper explores the feasibility of learning robot force control and interaction using soft metamaterial and machine vision. We start by investigating the differential stiffness of a hollow, cone-shaped, 3D metamaterial made from soft rubber, achieving a large stiffness ratio between the axial and radial directions that leads to an adaptive form response in omni-directions during physical interaction. Then, using image data collected from its internal deformation during various interactions, we explored two similar designs but different learning strategies to estimate force control and interactions on the end-effector of a UR10 e-series robot arm. One is to directly learn the force and torque response from raw images of the metamaterial’s internal deformation. The other is to indirectly estimate the 6D force and torque using a neural network by visually tracking the 6D pose of a marker fixed inside the 3D metamaterial. Finally, we integrated the two proposed systems and achieved similar force feedback and control interactions in simple tasks such as circle following and text writing. Our results show that the learning method holds the potential to support the concept of soft robot force control, providing an intuitive interface at a low cost for robotic systems, generating comparable and capable performances against classical force and torque sensors.},
keywords = {Authorship - Corresponding, Conf - CoRL},
pubstate = {published},
tppubtype = {conference}
}
This paper explores the feasibility of learning robot force control and interaction using soft metamaterial and machine vision. We start by investigating the differential stiffness of a hollow, cone-shaped, 3D metamaterial made from soft rubber, achieving a large stiffness ratio between the axial and radial directions that leads to an adaptive form response in omni-directions during physical interaction. Then, using image data collected from its internal deformation during various interactions, we explored two similar designs but different learning strategies to estimate force control and interactions on the end-effector of a UR10 e-series robot arm. One is to directly learn the force and torque response from raw images of the metamaterial’s internal deformation. The other is to indirectly estimate the 6D force and torque using a neural network by visually tracking the 6D pose of a marker fixed inside the 3D metamaterial. Finally, we integrated the two proposed systems and achieved similar force feedback and control interactions in simple tasks such as circle following and text writing. Our results show that the learning method holds the potential to support the concept of soft robot force control, providing an intuitive interface at a low cost for robotic systems, generating comparable and capable performances against classical force and torque sensors.
Conference Workshops & Extended Abstracts
1.
Tianyu Wu, Xudong Han, Haoran Sun, Zishang Zhang, Bangchao Huang, Chaoyang Song, Fang Wan
2025, (Short Paper accepted to CoRL 2025 Data Workshop on Making Sense of Data in Robotics: Composition, Curation, and Interpretability at Scale & Demo Presentation at CoRL 2025).
@workshop{Wu2025MagiClaw,
title = {MagiClaw: A Dual-Use, Vision-Based Soft Gripper for Bridging the Human Demonstration to Robotic Deployment Gap},
author = {Tianyu Wu and Xudong Han and Haoran Sun and Zishang Zhang and Bangchao Huang and Chaoyang Song and Fang Wan},
doi = {10.48550/arXiv.2509.19169},
year = {2025},
date = {2025-09-28},
urldate = {2025-09-28},
abstract = {The transfer of manipulation skills from human demonstration to robotic execution is often hindered by a "domain gap" in sensing and morphology. This paper introduces MagiClaw, a versatile two-finger end-effector designed to bridge this gap. MagiClaw functions interchangeably as both a handheld tool for intuitive data collection and a robotic end-effector for policy deployment, ensuring hardware consistency and reliability. Each finger incorporates a Soft Polyhedral Network (SPN) with an embedded camera, enabling vision-based estimation of 6-DoF forces and contact deformation. This proprioceptive data is fused with exteroceptive environmental sensing from an integrated iPhone, which provides 6D pose, RGB video, and LiDAR-based depth maps. Through a custom iOS application, MagiClaw streams synchronized, multi-modal data for real-time teleoperation, offline policy learning, and immersive control via mixed-reality interfaces. We demonstrate how this unified system architecture lowers the barrier to collecting high-fidelity, contact-rich datasets and accelerates the development of generalizable manipulation policies. Please refer to the iOS app at https://apps.apple.com/cn/app/magiclaw/id6661033548 for further details.},
note = {Short Paper accepted to CoRL 2025 Data Workshop on Making Sense of Data in Robotics: Composition, Curation, and Interpretability at Scale & Demo Presentation at CoRL 2025},
keywords = {},
pubstate = {published},
tppubtype = {workshop}
}
The transfer of manipulation skills from human demonstration to robotic execution is often hindered by a "domain gap" in sensing and morphology. This paper introduces MagiClaw, a versatile two-finger end-effector designed to bridge this gap. MagiClaw functions interchangeably as both a handheld tool for intuitive data collection and a robotic end-effector for policy deployment, ensuring hardware consistency and reliability. Each finger incorporates a Soft Polyhedral Network (SPN) with an embedded camera, enabling vision-based estimation of 6-DoF forces and contact deformation. This proprioceptive data is fused with exteroceptive environmental sensing from an integrated iPhone, which provides 6D pose, RGB video, and LiDAR-based depth maps. Through a custom iOS application, MagiClaw streams synchronized, multi-modal data for real-time teleoperation, offline policy learning, and immersive control via mixed-reality interfaces. We demonstrate how this unified system architecture lowers the barrier to collecting high-fidelity, contact-rich datasets and accelerates the development of generalizable manipulation policies. Please refer to the iOS app at https://apps.apple.com/cn/app/magiclaw/id6661033548 for further details.
Doctoral Thesis
Sorry, no publications matched your criteria.