United States

A long term goal in robotics is lifelong learning, in which a robot learns to perform new tasks without human intervention. For this a promising alternative is reinforcement learning (RL), in which an agent learns how to solve sequential decision-making problems by interacting with its environment. Despite the effectiveness of RL, learning in a trial-and-error way can be infeasible in robotics applications, in which long training periods can significantly deteriorate hardware. For robots that face long sequences of tasks, transfer learning represents an alternative to reduce training time, by transferring knowledge from previously learned tasks. However, in domains such as service robotics, identifying reusable pieces of knowledge and transferring them across tasks can be challenging, due to the mismatch between the state-action spaces (heterogeneous tasks). Thus, in this article we present an inter-task similarity measure and a transfer learning method for heterogeneous tasks. Through experimental evaluations, we show that the proposed measure is able to rank tasks in a way that prioritizes those that cause a larger increment of the learning performance in a target task. Additionally, results show that for some tasks the similarity measure can be computed with few data, enabling significant speedups in the learning process by transferring knowledge at an early stage.

RoboCup 2021

Inter-Task Similarity Measure for Heterogeneous Tasks

inter-task similarity

transfer learning

reinforcement learning

[![](https://assets.underline.io/uploads/markdown_image/1/image/21d13e03491d7e9acf6b8c46c7979c25.png)](https://docs.google.com/spreadsheets/d/1SCDDfkQk7es4J6RULKHVx1RzbB0Mutla--Sle5SVfb0/edit?usp=sharing)

**RoboCup 2021 will be Worldwide!**

We are excited to announce that RoboCup 2021 is a fully remote event with RoboCup competitions and activities taking place all over the world. Everyone is invited to participate in, or observe, the [competition leagues](https://2021.robocup.org/leagues) and the [symposium](https://2021.robocup.org/symposium)!

**RoboCup Symposium**

The 24th RoboCup International Symposium will be held in conjunction with RoboCup 2021 in a purely online setting. Scientific papers will be presented reporting innovative, original research with relevance to robotics and artificial intelligence. Within the described scope of topics we also encouraged submissions of high-quality overview articles, papers describing real-world research, and papers reporting theoretical results. In addition to the main track with regular research papers, there will be a special track focused on systems, data, and benchmarks featuring papers describing novel, open-source hardware and software systems, as well as datasets and benchmarks relevant to the community. In addition to research paper presentations, the RoboCup Symposium 2021 includes keynote talks by Dieter Fox, Jean-Paul Laumond, and Stefanie Tellex, and several interactive sessions.

**RoboCup Introduction Video**
<iframe width="995" height="516" src="https://www.youtube.com/embed/Cnh43cCsDKU" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>


Registration is required for entrance into RoboCup 2021. <br><br>
**Fees:** 
* Major Faculty or Symposium Faculty - US$50
* Major Student, Junior Mentor, Junior Student, or Symposium Student - US$20 

***Members of qualified teams:** Please contact your league for your registration password and proceed to register with your team. *<br>
***All non-team members:** Please register as "Symposium Only".*

You need to be registered to explore this event!

The 24th RoboCup International Symposium will be held in conjunction with RoboCup 2021 in a purely online setting. Scientific papers will be presented reporting innovative, original research with relevance to robotics and artificial intelligence. Within the described scope of topics we also encouraged submissions of high-quality overview articles, papers describing real-world research, and papers reporting theoretical results. In addition to the main track with regular research papers, there will be a special track focused on systems, data, and benchmarks featuring papers describing novel, open-source hardware and software systems, as well as datasets and benchmarks relevant to the community. In addition to research paper presentations, the RoboCup Symposium 2021 includes keynote talks by Dieter Fox, Jean-Paul Laumond, and Stefanie Tellex, and several interactive sessions.

poster

In this paper, we propose a method for predicting the path of the ball on the soccer field for the humanoid robots. A cost-function-based k-nearest neighbor regression method is first proposed to account for the part of the prediction which is based on previously observed data. Next, the autoregression method is utilized in order to carry out the prediction based on the current ball path. Finally, these two methods are combined to form the final prediction model. Moreover, two different schemes are introduced based upon the proposed model: fixed and adaptive schemes. In fixed scheme, the prediction is made once during the
initial steps of the motion and is used throughout the whole ball movement. However, in adaptive scheme, autoregression method coefficients are updated in fixed predened steps during the motion. This is beneficial to robustify the prediction against an externally applied disturbance on the ball path. Our proposed method is tested by simulation and practical implementation and the results demonstrate a high precision rate.

Ball Path Prediction for Humanoid Robots: Combination of k-NN Regression and Autoregression Methods

Soccer Simulation 2D (SS2D) is a simulation of a real soccer game in two dimensions.
In soccer, passing behavior is an essential action for keeping the ball in possession of our team and creating goal opportunities.
Similarly, for SS2D, predicting the passing behaviors of both opponents and our teammates helps manage resources and score more goals.
Therefore, in this research, we have tried to address the modeling of passing behavior of soccer 2D players using Deep Neural Networks (DNN) and Random Forest (RF).
We propose an embedded data extraction module that can record the decision-making of agents in an online format.
Afterward, we apply four data sorting techniques for training data preparation.
After, we evaluate the trained models' performance playing against 6 top teams of RoboCup 2019 that have distinctive playing strategies. Finally, we examine the importance of different feature groups on the prediction of a passing strategy.
All results in each step of this work prove our suggested methodology's effectiveness and improve the performance of the pass prediction in Soccer Simulation 2D games ranging from 5\% (e.g., playing against the same team) to 10\% (e.g., playing against Robocup top teams).

Engineering Features to Improve Pass Prediction in Soccer Simulation 2D Games

Human support robots (mobile robots able to perform useful domestic manipulative tasks) might be better accepted by people if they can communicate in ways they naturally understand: e.g. speech, but also facial expressions, postures, among others. Subtle (unconscious) mirroring of nonverbal cues during conversations promotes rapport building, essential for good communication. We investigate whether, as in human-human communication, the ability of a robot to mirror its user's head movements and facial expressions in real time can improve the user's experience with it. We describe the technical integration of a Toyota Human Support Robot (HSR) with a facially expressive 3D embodied conversational agent (ECA) (named ECA-HSR). The HSR and the ECA are aware of the user's head movements and facial emotions, and can mirror them, in real time. We then discuss a user study we designed in which participants interacted with ECA-HSR in a simple social dialog task with three conditions: mirroring of user's head movements, mirroring of user's facial emotions, and mirroring of both user's head movements and facial emotions. Our results suggest that interacting with an ECA-HSR that mirrors both the user's head movements and the facial expressions is preferred over the other conditions. Among other insights, the study revealed that the accuracy of open source, real-time recognition of facial expressions of emotion needs improvement for the best user's acceptance.

Towards building rapport with a Human Support Robot

Fan input and support is an important component in many individual and team sports, ranging from athletics to basketball. Audience interaction provides a consistent impact on the athletes’ performance. The analysis of the crowd noise can provide a global indication on the ongoing game situation, less conditioned by subjective factors that can influence a single fan. In this work, we exploit the collective intelligence of the audience of a robot soccer match to improve the performance of the robot players. In particular, audio features extracted from the crowd noise are used in a Reinforcement Learning process to possibly modify the game strategy. The effectiveness of the proposed approach is demonstrated by experiments on registered crowd noise samples from several past RoboCup SPL matches.

Learning from the Crowd: Improving the Decision Making Process in Robot Soccer using the Audience Noise

A stable and fast walk for humanoid robots is a challenging problem. In particular for multiple robots with different degrees of wear, it is necessary to tune several control parameters, which consumes a lot of time and wears down the hardware even more. In this paper, we present the latest improvements of our current walk, which enable a significant boost in the walking speed and stability, basically independent of the robot's hardware state. Based on a linear inverted pendulum model and thresholds determined statistically, the feet positions are adapted to follow the center of mass' direction of movement and let the robot walk in the direction of a potential fall if necessary. In addition, we ensure that the movement of the swing foot is parallel to the ground with a closed-loop feedback controller over the measured rotation errors of the support foot, without the direct use of the inertial measurement unit. The approaches presented allow for a up to 40% faster walk on different robots of the same type, without the need for a manual calibration.

Step Adjustment for a Robust Humanoid Walk

Detecting the field boundary is often one of the first steps in the vision pipeline of soccer robots. Conventional methods make use of a (possibly adaptive) green classifier, selection of boundary points and possibly model fitting. We present an approach to predict the coordinates of the field boundary column-wise in the image using a convolutional neural network. This is combined with a method to let the network predict the uncertainty of its output, which allows to fit a line model in which columns are weighted according to the network's confidence. Experiments show that the resulting models are accurate enough in different lighting conditions as well as real-time capable.

Premium content

Inter-Task Similarity Measure for Heterogeneous Tasks

Downloads

Next from RoboCup 2021

Ball Path Prediction for Humanoid Robots: Combination of k-NN Regression and Autoregression Methods

Similar lecture

Improving Sample Efficiency in Behavior Learning by Using Sub-optimal Planners for Robots