Bringing constraint-based robot programming to real-world applications

Workshop at the IEEE/RSJ International Conference on Robots and Systems (IROS)

Aim and scope

In robot programming, there is a big gap between academic experimental frameworks and the state-of-thepractice in industry. Widespread programming paradigms of industrial robot systems are built around a robot domain-specific language (like ABB RAPID, KUKA KRL, FANUC KAREL) that contain a motion stack with a set of primitives (e.g., MoveJ/PTP, MoveL/LIN) to interact with a real-time motion controller. The roots of these systems lie in the era of caged robots performing repetitive tasks with no or limited sensor interactions. Sensor-guided motions are typically added with special plugins/language extensions.

Nowadays, we want to deploy robot applications not only in highly conditioned production lines, but also in more human-like production lines with a large variability and uncertainty in the task, or production lines manipulating or processing natural products. Sometimes, we even want robots to physically interact with humans, jointly performing tasks. Defining and implementing how to react to disturbances (behaviour, skill) is key in these aspects, going beyond specifying a nominal prescribed trajectory execution. At the same time, we see that these applications typically consist of smaller production serie.  Development costs need to be written off against a smaller batch, leading to the need for decreased development costs. The central objective of this workshop is to identify how we can resolve these seemingly conflicting goals: on the one hand, we want to have complex sensor-based applications; on the other hand, we need to decrease development costs.

Constraint-based approaches, where the robot task is specified in terms of constraints, as opposed to traditional, trajectory-based task specification, look very promising to tackle this problem. In these approaches, the task is described (either explicitly or implicitly) as an optimisation problem subject to a number of constraints. These constraints can be in the continuous domain [1] or also the discrete domain [2]; they can involve geometry of robot and task or can be sensor-based [3]. Some of these approaches are kinematic [1], others include a dynamic model [4]. Typically, these task specifications also involve an hierarchical optimization problem, where some constraints are considered more prioritized than others [5].

In this workshop we aim to bring together leading researchers and industrial/real-world early adopters in constraint-based programming theory, software development and applications [6]. Which paradigm shift do we need to obtain the reactivity and modularity needed to resolve this conflict? How do we reuse robot “skills” in new robot applications? What are the key missing steps/hurdles/pitfalls to bring them from a lab to the real world? How should we organize the different stakeholder roles in the development of tomorrow’s robot applications? What can we learn from early adopters?

  1. E. Aertbeliën and J. De Schutter, "eTaSL/eTC: A constraint-based task specification language and robot controller using expression graphs," 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, 2014, pp. 1540-1546.
  2. P. S. Schmitt, F. Wirnshofer, K. M. Wurm, G. v. Wichert and W. Burgard, "Modeling and Planning Manipulation in Dynamic Environments," 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 2019, pp. 176-182.
  3. K. Bouyarmane, K. Chappellet, J. Vaillant and A. Kheddar, "Quadratic Programming for Multirobot and Task-Space Force Control," in IEEE Transactions on Robotics, vol. 35, no. 1, pp. 64-77, Feb. 2019.
  4. L. Sentis and O. Khatib, "A whole-body control framework for humanoids operating in human environments," Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006., Orlando, FL, 2006, pp. 2641-2648.
  5. N. Mansard, O. Stasse, P. Evrard and A. Kheddar, "A versatile Generalized Inverted Kinematics implementation for collaborative working humanoid robots: The Stack Of Tasks," 2009 International Conference on Advanced Robotics, Munich, 2009, pp. 1- 6.
  6. A. Kheddar et al., "Humanoid Robots in Aircraft Manufacturing: The Airbus Use Cases," in IEEE Robotics & Automation Magazine, vol. 26, no. 4, pp. 30-45, Dec. 2019.

Call for participation

Prospective authors are invited to submit a two-page abstract reporting on their current research and ideas that motivate discussion during the workshop. Authors of accepted work will be able to present their research as a lightning presentation and/or poster.

In this workshop, we aim to bring to the attention of the audience the following topics, all related to constraint-based programming:

  • real-world/industrial cases of constraint-based programming
  • domain-specific languages for robot task specification
  • visual/intuitive programming interfaces for building robot applications
  • (re)usability, composability, and verifiability of robot software
  • sensor-rich robot applications
  • physical and/or cognitive human-robot interaction in a constraint-based framework
  • constraint-based robot programming by demonstration
  • multi-objective, under- and over-constrained, robot tasks
  • dealing with uncertainty and redundancy in robot applications

“Constraint-based programming” must be considered in the wide sense of the term. For example, but not limited to:

  • continuous-time control derivation from formal specifications of mathematical relations
  • discrete time coordination based on goal description and system constraints
  • task-based control
  • task-based planning

Submission Please submit your two-page abstract using the IROS template by email to . Each submission will be reviewed by two reviewers.

Important dates

Call for abstracts opens:July 10, 2020
Abstract submission deadline:August 10, 2020
Notification of acceptance:August 31, 2020
Flanders Make will sponsor an award for the best contributions presented during the workshop poster session.

Agenda & Structure

The workshop will consist of 1) invited talks (cf. Invited Speakers section), 2) lightning talks and a poster sessios for contributed abstracts, and 3) an interactive discussion connecting developers and users of software for constraint-based programming. We encourage participants to check the Software section in order to actively participate in the discussion.

Note that we are monitoring COVID-19 measures, and the workshop structure will be adapted if required.

Invited Speakers

Erwin Aertbeliën, KU Leuven/Flanders Make@KU Leuven

Constraint-based robot programming for advanced sensor-based applications and human-robot interaction

Pierre Gergondet or Abderrahmane Kheddar, CNRS

Multi-objectives and multi-sensory task space quadratic programming control

Enrico Mingo Hoffman, IIT

Robot programming at your fingertips with OpenSoT and CartesI/O

Lars Tingelstad, NTNU

Welding automation using constraint-based robot programming

Joris De Schutter, Flexible Robotic Solutions/KU Leuven/Flanders Make@KU Leuven

The first industrial application of eTaSL: ultrasound inspection of fold-glue joints of car parts in a human-robot collaborative setting at Audi Brussels


Erwin Aertbeliën, Gianni Borghesan, Herman Bruyninckx and Wilm Decré, KU Leuven/Flanders Make@KU Leuven

Pierre Gergondet and Abderrahmane Kheddar, CNRS

Darwin Caldwell and Enrico Mingo Hoffman, IIT

Lars Tingelstad, NTNU

Support & Sponsors

The workshop is supported by the IEEE RAS Technical Committee on Whole-Body Control.

The workshop is sponsored by Flanders Make, the manufacturing innovation network and strategic research center for the manufacturing industry in Flanders, Belgium.




CARTESI/O is a novel Cartesian control framework with a focus on online control of multi-chained, hyper-redundant floating-base robots.


eTasl is a task specification language for reactive control of robot systems. It is a language that allows you to describe how your robotic system has to move and interact with sensors. This description is based on a constraint-based methodology. Everything is specified as an optimization problem subject to constraints.

Contact us