RoboLetics 3.0: Benchmarks for Agile Physical Human-Robot Interaction

Venue and Date

Overview

Core Research Questions

Our workshop proposes mixed human-robot sports as a principled design space for benchmarking and catalyzing agile physical human-robot interaction research. Our goal is to identify benchmark tasks, evaluation methodologies, safety constraints, fairness criteria, rules for difficulty level progression that stress-test aspects of agile physical HRI, embodiment design, models, control algorithms and push their translation into real-world applications.

• Mapping sports properties to benchmark design: How to systematically translate sports’ characteristics into benchmark specifications? For instance, tasks with central objects in play (e.g., soccer, basketball, tennis) test agility and collaborative strategy, physical contact collaboration (e.g., cheerleading, acrobatics) balances coordination with safety, etc.
• Stress-testing algorithms and models: How to design benchmarks that effectively challenge models and algorithms for control involving dynamic contact, real-time safety verification, verbal/non-verbal communication, intent estimation, initiative-taking and role switching, human partner modeling, theory-of-mind reasoning?
• Progressive specification: How to formulate sports’ rules that enable systematic progression across difficulty levels so that it provides meaningful incremental challenges? How to design benchmarks that test collaboration and strategic consensus at various timescales—from real-time coordination to multi-step strategic planning?
• Multi-faceted safety constraints: How to formally specify and evaluate diverse safety dimensions such as physical safety during contact or psychological/perceived safety for human partners?
• Fairness and robot morphologies: How to define fair evaluation when robots have different morphologies, identify which have systematic advantages/disadvantages in specific sports, and design handicapping mechanisms that provide insightful comparisons across embodiments?
• Research prioritization: Which research questions should be prioritized in the short and long term to advance the field of agile physical HRI through sport-based benchmarking?

Benchmark Prototype

As a concrete workshop outcome, we will develop a community-informed simulation benchmark prototype for mixed human-robot sports, based on doubles tennis. The prototype, seeded by our prior wheelchair tennis research, will be refined through input collected during the breakout sessions, moderated discussions, and panel discussion. The goal is not to prescribe a single benchmark in advance, but to use the workshop to identify reusable design principles for sport-inspired simulation benchmarks, including task abstractions, difficulty progressions, safety constraints, fairness criteria, and evaluation metrics.

The doubles tennis benchmark will consider the following progressively rich levels. Videos of baseline behaviors are available below. Workshop feedback will be used to define both objective metrics, such as task success and game score, and human-centered metrics, such as perceived safety, workload, and enjoyment.

1. Kinematic Interception: The agent must navigate to place an incoming ball within the racket workspace.
2. Dynamic Returns: The agent must return the ball to the opponent’s side considering actuator, ball-flight, and racket-ball contact dynamics.
3. Collaborative Rally: Human/robot collaborators are introduced in 1+1 or 2+2 configurations and success is measured by safe sustained rallying. This level supports two settings: (1) policies for all agents are learned jointly; (2) policies for collaborators are predefined.
4. Competitive: The agent must compete and defeat the opponent in a 1v1 setting with a standard tennis scoring scheme. Opponent policies of various proficiencies are provided.
5. Mixed Collaboration-Competition: The agent must collaborate with its teammate and defeat the opponent team in a 2v2 configuration. Opponent team policies of various proficiencies are provided. This level supports two settings: (1) policies for the agent and teammate are learned jointly; (2) policies for the collaborating teammate are predefined.

Predefined behaviors for proxy-human agents will be initially realized with scripted, parameterized policies, baseline learning/planning agents, and in the long-term support richer community-developed human behavior models.

Schedule

Structured Breakout Sessions

Participants will break out into groups of 5–7. The breakout session will be organized around different axes of agile physical HRI that arise across multiple sport-inspired domains. Each group will be assigned one axis and will collaboratively develop benchmark specifications and concrete evaluation protocols. Candidate breakout axes include:

• Coordination across timescales: Benchmarks for collaboration across fast reactions, between-play adaptation, and longer-horizon strategy.
• Physical coupling and contact: Benchmarks for shared-object interaction, physical proximity, contact, and recovery from unexpected interaction.
• Communication and intent inference: Benchmarks for verbal, gestural, gaze-based, and implicit communication during fast physical collaboration.
• Role switching and initiative-taking: Benchmarks for dynamic transitions between leader, follower, supporter, defender, and initiator roles.
• Safety, trust, and graceful failure: Benchmarks for physical safety, perceived safety, trust calibration, and safe degradation under failure.
• Morphology and fairness: Benchmarks for comparing heterogeneous robot embodiments and defining fair evaluation rules across platforms.

Groups will collaborate on the assigned axis for 20 minutes and then present their outcomes in brief 2-minute pitches to the full workshop audience.

Confirmed Speakers

Dr. Nico Rojas

Robotics and AI Institute, USA

Talk title: TBD

Dr. Nico Rojas brings expertise in the mechanical design of high-performance robotic systems as a project lead for dynamic manipulation at RAI. His perspective will help the workshop determine how morphology, actuation, inertia, power, and mechanical robustness should be represented in sport-based benchmarks, including how embodiment enables athletic performance and how evaluation can remain meaningful and fair across heterogeneous robot platforms.

Dr. Guanya Shi

Carnegie Mellon University, USA

Talk title: TBD

Prof. Guanya Shi develops learning and control methods for reliable, adaptive, and agile general-purpose robots, with recent work on humanoid whole-body control and loco-manipulation. His expertise is directly relevant to designing benchmarks that couple locomotion, object interaction, contact, and rapid adaptation, and to defining progressive task levels that distinguish motion imitation from robust, physically grounded, whole-body athletic behavior.

Accepted Submissions

• TBD

Call for Papers

TBD

Partnership and Best Paper Award

Organizers

Dr. Matthew Gombolay

Georgia Institute of Technology, USA

Dr. Andrew Morgan

Robotics and AI Institute, USA

Varshith Sreeramdass

Georgia Institute of Technology, USA

Kin Man Lee

Georgia Institute of Technology, USA

Qingyu Xiao

Georgia Institute of Technology, USA

Zixuan Wu

Georgia Institute of Technology, USA

Kiersten Lange

Georgia Institute of Technology, USA

Dylan Barnes

Georgia Institute of Technology, USA

Zulfiqar Zaidi

Georgia Institute of Technology, USA

Contact

Previous Workshops

• RoboLetics 1.0
• RoboLetics 2.0