One Fling to Goal: Environment-aware Dynamics for Goal-conditioned Fabric Flinging. Workshop on the Algorithmic Foundations of Robotics (WAFR2024), Chicago, USA, 2024, (Accepted).
Presented by Dr. Fang Wan @ Chicago, USA, during WAFR2024. 
In industrial settings, swiftly and accurately moving fabrics to target positions for downstream tasks is a common operation. Workers often employ dynamic flinging motions to enhance efficiency. This observation raises our first challenge: Can we train robots to achieve precise fabric positioning in one fling? 
Previous research often involves robots making multiple attempts to learn the delta dynamics between each trial, continuously optimizing the fling trajectory to reach the target position. However, multiple trials reduce efficiency and delta dynamics are difficult to generalize to complex environments, typically only being effective in flat scenarios. 
Furthermore, fabric properties significantly influence the outcome of the fling. For example, With the same trajectory, more elastic fabrics tend to produce more wrinkles, needs a lighter Fling. This diversity leads to our second challenge: How can we adjust the fling trajectory when fabric properties are unknown? 
Beyond industrial applications, fabric manipulation tasks are commonly-used in daily life, where environments are more complex and often involve interactions between the fabric and surrounding objects. This complexity introduces our third challenge: Can we encode the rich interaction between fabric and environment for robotic manipulation? 
To address these challenges, we propose 1Fling2Goal. Our objective is to manipulate the fabric dynamically into the target state with only one fling in complex environments. We utilize a dual-arm robot setup and an RGB-D camera to capture real-time state of the fabric. Our method is tested across various complex scenarios, as shown on the right, where environmental objects are randomly placed within the workspace, and corresponding fabric target states are defined. 
We begin by training a dynamic model of the fabric where the state is represented as a point cloud. We employ a Graph Neural Network (GNN) to predict the fabric’s point-wise velocities at the next time step, capturing the complex dynamics involved in fabric manipulation. 
To enhance the dynamic model’s performance during interactions with the environment, we integrate environmental information into the node features of our GNN. We term this approach Environment-Aware State Representation. For each node vi, we calculate the distance between the fabric xi and the nearest object point qj in the environment. Later we will show our model more accurately predicts motions by involving fabric-environment contact. 
We collected training data from the PyFlex simulation environment. During data collection, we randomized key parameters related to fabric properties, such as weight and size. Which allows our model to have good generalization capabilities across different fabrics. 
With the dynamic model, we implement a two-stage model-based control strategy to optimize the fling trajectory. In stage one, we acquire the fabric’s current state using the RGB-D camera. 
In this work, we simplify potential trajectory to a fling-then-pull movement, which can be parameterized by a single turning point. To get an initial estimation of the fling trajectory, we randomly sample 50 turning points. 
And then utilize the dynamic model to infer the final state for each sampled trajectory. We select the optimal trajectory whose final state has the minimum distance to the target state. 
Despite this initial estimation, unknown fabric properties may prevent accurate task completion. Therefore, we implement real-time refinement of the fling trajectory. At every ten time steps, we sample 9 delta actions to adjust the current action of the initial estimation. Here delta actions include faster, slower flinging movements, as well as no change to the initial estimation. 
We select the best delta actions based on the dynamic model’s inference. For instance, if we detect that the fabric is highly elastic, we adjust by reducing the fling speed appropriately. 
At each control step, we update the fling trajectory and the fabric’s current state, ensuring that adjustments are responsive to real-time observations and predictions. 
Through this iterative process, the fabric is eventually manipulated to reach the target state. 
We evaluate the performance of the proposed GNN dynamic model through a series of comparative experiments. The results show that incorporating environmental awareness into the fabric representation and collecting data across multiple scenarios significantly improve the model’s accuracy and generalization. The video on the right illustrates our model’s ability to accurately predict the fabric’s motion, even during contact with environmental objects. 
For the goal-conditioned fabric manipulation task, we compared our method with state-of-the-art approaches such as FlingBot and IRP. We also conducted ablation studies on key components of our algorithm, namely environmental awareness and Model predictive control. The experimental results indicate that our method exhibits notable stability and generalization, especially in complex environments. 
Here, we present simulation results across different scenarios, including Flat Scenario without environment awareness, and Blue particles represent the fabric’s state, while red arrows indicate velocity of the fabric’s motion. The green rectangles show the fabric’s target state for visualization purpose; We use the fabric’s point cloud to compute the distance between current state and the target state. 
Platform Scenario, 
Hemisphere Scenario, and 
Pole Scenario with Environment Awareness. 
The proposed two-stage strategy transfers seamlessly to real-world robotic experiments. we randomly place the fabric within the workspace and allow the robot to plan an initial trajectory to fling the fabric to the target position and do real-time adjustment according the RGBD observation. Our method proves to be effective across various settings, including flat surfaces. 
platforms, 
hemispheres, 
poles, and 
stools. We also validate our approach on 
unseen fabrics with novel shapes and 
unseen folded target states. It works well since our dynamic model is generalized and target-free. And the control strategy is applicable to any feasible target. 
In summary, we propose a method that encodes environmental information into the dynamic model of fabric, achieving precise fabric manipulation in one fling. Our approach addresses the challenges of unknown fabric properties and complex environmental interactions. 
There are some limitations. Inaccurate perception due to fling speed and partial observation can sometimes cause the fabric to contact the environment too abruptly, leading to task failure. Additionally, the model’s effectiveness can be limited when interacting with unseen fabric configurations that differ drastically from the training data. Future work will focus on improving the robustness to perception accuracy and extending the model to more diverse task configurations. 
Presented by Dr. Fang Wan @ Chicago, USA, during WAFR2024.
Abstract
Fabric manipulation dynamically is commonly seen in manufacturing and domestic settings. While dynamically manipulating a fabric piece to reach a target state is highly efficient, this task presents considerable challenges due to the varying properties of different fabrics, complex dynamics when interacting with environments, and meeting required goal conditions. To address these challenges, we present One Fling to Goal, an algorithm capable of handling fabric pieces with diverse shapes and physical properties across various scenarios. Our method learns a graph-based dynamics model equipped with environmental awareness. With this dynamics model, we devise a real-time controller to enable high-speed fabric manipulation in one attempt, requiring less than 3 seconds to finish the goal-conditioned task. We experimentally validate our method on a goal-conditioned manipulation task in five diverse scenarios. Our method significantly improves this goal-conditioned task, achieving an average error of 13.2mm in complex scenarios. Our method can be seamlessly transferred to real-world robotic systems and generalized to unseen scenarios in a zero-shot manner.
Links
BibTeX (Download)
@conference{Yang2024OneFling,
title = {One Fling to Goal: Environment-aware Dynamics for Goal-conditioned Fabric Flinging},
author = {Linhan Yang and Lei Yang and Haoran Sun and Zeqing Zhang and Haibin He and Fang Wan and Chaoyang Song and Jia Pan},
url = {https://www.algorithmic-robotics.org/},
doi = {10.48550/arXiv.2406.14136},
year = {2024},
date = {2024-10-07},
urldate = {2024-10-07},
booktitle = {Workshop on the Algorithmic Foundations of Robotics (WAFR2024)},
address = {Chicago, USA},
abstract = {Fabric manipulation dynamically is commonly seen in manufacturing and domestic settings. While dynamically manipulating a fabric piece to reach a target state is highly efficient, this task presents considerable challenges due to the varying properties of different fabrics, complex dynamics when interacting with environments, and meeting required goal conditions. To address these challenges, we present One Fling to Goal, an algorithm capable of handling fabric pieces with diverse shapes and physical properties across various scenarios. Our method learns a graph-based dynamics model equipped with environmental awareness. With this dynamics model, we devise a real-time controller to enable high-speed fabric manipulation in one attempt, requiring less than 3 seconds to finish the goal-conditioned task. We experimentally validate our method on a goal-conditioned manipulation task in five diverse scenarios. Our method significantly improves this goal-conditioned task, achieving an average error of 13.2mm in complex scenarios. Our method can be seamlessly transferred to real-world robotic systems and generalized to unseen scenarios in a zero-shot manner.},
note = {Accepted},
keywords = {Corresponding Author, WAFR},
pubstate = {published},
tppubtype = {conference}
}
