Hamidreza Kasaei

I am an Assistant Professor in the Department of Artificial Intelligence at the University of Groningen, the Netherlands. My research group focuses on Lifelong Interactive Robot Learning (IRL-Lab), which we work at the cutting edge of robotics research. My main research interests lie in the area of 3D Object Perception, Grasp Affordance, and Object Manipulation. My goal is to achieve a breakthrough by enabling robots to incrementally learn from past experiences and safely interact with non-expert human users using data-efficient open-ended machine-learning techniques.

/ CV / PhD Thesis / Publications / Google Scholar / ResearchGate / LinkedIn / Github / YouTube /

Latest News


* May. 2021: I gave an invited talk on Lifelong Robot Learning in Human-centric Environments: From Object Perception to Object Manipulation at the University of Lincoln, UK
* May. 2021: Our paper titled Open-Ended Fine-Grained 3D Object Categorization by Combining Shape and Texture Features in Multiple Colorspaces got accepted to IEEE-RAS International Conference on Humanoid Robots (Humanoids2020)! Congrats Nils!
* April. 2021: Our paper titled 3D_DEN: Open-ended 3D Object Recognition Using Dynamically Expandable Networks got accepted to IEEE Transactions on Cognitive and Developmental Systems! Congrats Sudhakaran!
* March. 2021: Our paper titled Self-Imitation Learning by Planning got accepted to ICRA2021! Congrats Sha!
* Feb. 2021: I am serving as associate editor for the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021)
* Feb. 2021: I gave an invited talk at the Bosch Center for Artificial Intelligence (BCAI) on Robots Beyond the Factory: Open-ended Robot Learning in Human-Centric Environments!
* Jan. 2021: Our paper titled Investigating the Importance of Shape Features, Color Constancy, Color Spaces and Similarity Measures in Open-Ended 3D Object Recognition got accepted to Intelligent Service Robotics Journal!
* Dec. 2020: I am beyond excited to announce that I have accepted a tenure track Assistant Professor position in the Department of Artificial Intelligence at the University of Groningen, Netherlands!
* Nov. 2020: Our paper titled OrthographicNet: A Deep Transfer Learning Approach for 3D Object Recognition in Open-Ended Domains got accepted to IEEE Transactions on Mechatronics! (IF 5.71).
* October 2020: I am serving as associate editor for the IEEE/RSJ International Conference on Robotics and Automation (ICRA 2021) .
* June 2020: Our paper titled Learning to Grasp 3D Objects using Deep Residual U-Nets got accepted to IEEE RO-MAN 2020! Congrats Yikun!.
* March 2020: Our paper titled Accelerating Reinforcement Learning for Reaching using Continuous Curriculum Learning got accepted to IJCNN 2020! Congrats Sha!
* October 2019: I am serving as associate editor for the IEEE/RSJ International Conference on Robotics and Automation (ICRA 2020) .
* September 2019: I will be teaching a new course on Cognitive Robotics .
* June 2019: Our paper Local-LDA: Open-Ended Learning of Latent Topics for 3D Object Recognition got accepted at IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI - IF = 17.730).
* June 2019: Our paper Look Further to Recognize Better: Learning Shared Topics and Category-Specific Dictionaries for Open-Ended 3D Object Recognition got accepted at IROS2019.
* May 2019: We will organize a full-day workshop on Task-Informed Grasping (TIG-II): From Perception to Physical Interaction at RSS2019.
* April 2019: We will organize a full-day workshop on Open-Ended Learning for Object Perception and Grasping: Current Successes and Future Challenges at IROS2019.
* April 2019: I am honored to be a member of the 2019 cohort of the RSS Pioneers.
* January 2019: Our paper Interactive Open-Ended Object, Affordance and Grasp Learning for Robotic Manipulation got accepted at ICRA2019.
* November 2018: I got the qualification of University Teaching Skills from the University of Groningen, the Netherlands.
* October 2018: My NVIDIA GPU Grant has been approved. Thank NVIDIA Corporation for supporting our works.
* July 2018: Our paper Coping with Context Change in Open-Ended Object Recognition without Explicit Context Information got accepted at IROS2018.

Research & Publication

My research interests focus on the intersection of robotics, machine learning and machine vision. I am interested in developing algorithms for an adaptive perception system based on interactive environment exploration and open-ended learning, which enables robots to learn from past experiences and interact with human users. I have been investigating on active perception, where robots use their mobility and manipulation capabilities not only to gain the most useful perceptual information to model the world, but also to predict the next best view for improving detection and manipulation performances. I have evaluated my works on different platforms including PR2, robotic arms, and humanoid robots. My up-to-date list of publications and corresponding BibTeX files can be found on my Google scholar account . My research is summarized by the following projects:

OrthographicNet: A Deep Transfer Learning Approach for 3D Object Recognition in Open-Ended Domains

We present OrthographicNet, a deep transfer learning based approach, for 3D object recognition in open-ended domains. In particular, OrthographicNet generates a rotation and scale invariant global feature for a given object, enabling to recognize the same or similar objects seen from different perspectives. Experimental results show that our approach yields significant improvements over the state-of-the-art approaches concerning scalability, memory usage and object recognition performance.

  • IEEE TMECH (2020)
  • Preprint on arXiv (2019)
  • Demo1 --- Demo2
  • Simultaneous Multi-View Object Grasping and Recognition in Open-Ended Domains

    Most state-of-the-art approaches tackle object recognition and grasping as two separate problems while both use visual input. Such approaches are not suitable for task-informed grasping, where the robot should recognize a specific object first and then grasp and manipulate it to accomplish a task. In this work, we propose a multi-view deep learning approach to handle simultaneous object grasping and recognition in open-ended domains. In particular, our approach takes multi-view of the object as input and jointly estimates pixel-wise grasp configuration and a deep scale- and rotation-invariant representation. The obtained representation is then used for open-ended object category learning and recognition. Experimental results on benchmark datasets have shown that our approach outperforms state-of-the-art methods by a large margin in terms of grasping and recognition.

  • Preprint on arXiv (2021)
  • Investigating the Importance of Shape Features, Color Constancy, Color Spaces and Similarity Measures in Open-Ended 3D Object Recognition

    Despite the recent success of state-of-the-art 3D object recognition approaches, service robots are frequently failed to recognize many objects in real human-centric environments. Most of the recent approaches use either the shape information only and ignore the role of color information or vice versa. Furthermore, they mainly utilize the Ln Minkowski family functions to measure the similarity of two object views, while there are various distance measures that are applicable to compare two object views. In this paper, we explore the importance of shape information, color constancy, color spaces, and various similarity measures in open-ended 3D object recognition.

  • Intelligent Service Robotics (open access)
  • Preprint on arXiv (2020)
  • Demo
  • Local-HDP: Interactive Open-Ended 3D Object Categorization

    We introduce a non-parametric hierarchical Bayesian approach for open-ended 3D object categorization, named the Local Hierarchical Dirichlet Process (Local-HDP). This method allows an agent to learn independent topics for each category incrementally and to adapt to the environment in time. Hierarchical Bayesian approaches like Latent Dirichlet Allocation (LDA) can transform low-level features to high-level conceptual topics for 3D object categorization. However, the efficiency and accuracy of LDA-based approaches depend on the number of topics that is chosen manually. Moreover, fixing the number of topics for all categories can lead to overfitting or underfitting of the model. In contrast, the proposed Local-HDP can autonomously determine the number of topics for each category. Furthermore, an inference method is proposed that results in a fast posterior approximation.

  • Preprint on arXiv (2020)
  • Demo
  • The State of Service Robots: Current Bottlenecks in Object Perception and Manipulation

    Nowadays, robots are able to recognize various objects, and quickly plan a collision-free trajectory to grasp a target object. While there are many successes, the robot should be painstakingly coded in advance to perform a set of predefined tasks. Besides, in most of the cases, there is a reliance on large amounts of training data. Therefore, these approaches are still too rigid for real-life applications in unstructured environments, where a significant portion of the environment is unknown and cannot be directly sensed or controlled. In this paper, we review advances in service robots from object perception to complex object manipulation and shed a light on the current challenges and bottlenecks.

  • Preprint on arXiv (2020)
  • Accelerating Reinforcement Learning for Reaching using Continuous Curriculum Learning

    In this study, we focus on accelerating reinforcement learning (RL) training and improving the performance of multi-goal reaching tasks. Specifically, we propose a precision-based continuous curriculum learning (PCCL) method in which the requirements are gradually adjusted during the training process, instead of fixing the parameter in a static schedule. To this end, we explore various continuous curriculum strategies for controlling a training process. This approach is tested using a Universal Robot 5e in both simulation and real-world scenarios.

  • Preprint on arXiv (2020)
  • IJCNN conference
  • Demo1
  • Look Further to Recognize Better: Learning Shared Topics and Category-Specific Dictionaries for Open-Ended 3D Object Recognition

    In human-centric environments, fine-grained object categorization is as essential as basic-level object categorization. In this work, each object is represented using a set of general latent topics and category-specific dictionaries. The general topics encode the common patterns of all categories, while the category-specific dictionary describes the content of each category in details. We discovered both sets of general and specific representations in an unsupervised fashion and updated them incrementally using new object views.

  • IROS2019
  • Preprint on arXiv (2019)
  • Demo1 --- Demo2
  • Interactive Open-Ended Learning Approach for Recognizing 3D Object Category and Grasp Affordance Concurrently

    This paper presents an interactive open-ended learning approach to recognize multiple objects and their grasp affordances concurrently. This is an important contribution in the field of service robots since no matter how extensive the training data used for batch learning, a robot might always be confronted with an unknown object when operating in human-centric environments. Our approach has two main branches. The first branch is related to open-ended 3D object category learning and recognition. The second branch is associated with learning and recognizing the configuration of grasps in a reasonable amount of time.

  • ICRA2019
  • Preprint on arXiv (2019)
  • Demo1 - Demo2 - Demo3 - Demo4
  • Learning to Grasp 3D Objects using Deep Residual U-Nets

    In this study, we present a new deep learning approach to detect object affordances for a given 3D object. The method trains a Convolutional Neural Network (CNN) to learn a set of grasping features from RGB-D images. We named our approach Res-U-Net since the architecture of the network is designed based on U-Net structure and residual network-styled blocks. It devised to be robust and efficient to compute and use. A set of experiments has been performed to assess the performance of the proposed approach regarding grasp success rate on simulated robotic scenarios.

  • RO-MAN2020
  • Preprint on arXiv (2020)
  • Demo1
  • Coping with Context Change in Open-Ended Object Recognition without Explicit Context Information

    One of the main challenges in unconstrained human environments is to cope with the effects of context change. This paper presents two main contributions: (1) an approach for evaluating open-ended object category learning and recognition methods in multi-context scenarios; (2) evaluation of different object category learning and recognition approaches regarding their ability to cope with the effects of context change.

  • IROS2018
  • Demo: multi contexts open-ended scenario
  • Perceiving, Learning, and Recognizing 3D Objects: An Approach to Cognitive Service Robots

    This paper proposes a cognitive architecture designed to create a concurrent 3D object category learning and recognition in an interactive and open-ended manner. In particular, this cognitive architecture provides automatic perception capabilities that will allow robots to detect objects in highly crowded scenes and learn new object categories from the set of accumulated experiences in an incremental and open-ended way. Moreover, it supports constructing the full model of an unknown object in an on-line manner and predicting next best view for improving object detection and manipulation performance.

  • AAAI2018
  • Demo1 --- Demo2 --- Demo3
  • Active Multi-View 6D Object Pose Estimation and Camera Motion Planning in the Crowd

    In this project, we developed a novel unsupervised Next-Best-View (NBV) prediction algorithm to improve object detection and manipulation performance. Particularly, the ability to predict the NBV point is important for mobile robots performing tasks in everyday environments. In active scenarios, whenever the robot fails to detect or manipulate objects from the current view point, it is able to predict the next best view position, goes there and captures a new scene to improve the knowledge of the environment. This may increase the object detection and manipulation performance.

  • ICCV2017-WS
  • Bin-Picking Synthetic Dataset (RGB-D)

  • Hierarchical Object Representation for OpenEnded Object Category Learning and Recognition (Local LDA)

    This paper proposes an open-ended 3D object recognition system which concurrently learns both the object categories and the statistical features for encoding objects. In particular, we propose an extension of Latent Dirichlet Allocation to learn structural semantic features (i.e. topics), from low-level feature co-occurrences, for each category independently. Moreover, topics in each category are discovered in an unsupervised fashion and are updated incrementally using new object views. In this way, the advantage of both the local hand-crafted and the structural semantic features have been considered in an efficient way.

  • NIPS2016 --- TPAMI (To appear)
  • Demo1 --- Demo2

  • GOOD: A Global Orthographic Object Descriptor for 3D Object Recognition and Manipulation

    The Global Orthographic Object Descriptor (GOOD) has been designed to be robust, descriptive and efficient to compute and use. GOOD descriptor has two outstanding characteristics: (1) Providing a good trade-off among: descriptiveness, robustness, computation time, memory usage; (2) Allowing concurrent object recognition and pose estimation for manipulation. The performance of the proposed object descriptor is compared with the main state-of-the-art descriptors. Experimental results show that the overall classification performance obtained with GOOD is comparable to the best performances obtained with the state-of-the-art descriptors. Concerning memory and computation time, GOOD clearly outperforms the other descriptors. The current implementation of GOOD descriptor supports several functionalities for 3D object recognition and object manipulation.

  • Pattern Recognition Letters --- IROS2016
  • Demo1 --- Demo2
  • Source Code (GitHub) --- Part of PCL 1.9
  • Towards Lifelong Assistive Robotics: A Tight Coupling between Object Perception and Manipulation

    In this work, we propose a cognitive architecture designed to create a tight coupling between perception and manipulation for assistive robots. This is necessary for assistive robots, not only to perform manipulation tasks in a reasonable amount of time and in an appropriate manner, but also to robustly adapt to new environments by handling new objects. In particular, this cognitive architecture provides perception capabilities that will allow robots to, incrementally learn object categories from the set of accumulated experiences and reason about how to perform complex tasks.

  • Neurocomputing Journal --- RoboCup2016 --- IROS2015
  • Demo1 --- Demo2 --- Demo3

  • Interactive Open-Ended Learning for 3D Object Recognition: An Approach and Experiments

    This work presents an efficient approach capable of learning and recognizing object categories in an interactive and open-ended manner. In particular, we mainly focus on two state-of-the-art questions: (1) How to automatically detect, conceptualize and recognize objects in 3D scenes in an open-ended manner? (2) How to acquire and use high-level knowledge obtained from the interaction with human users, namely when they provide category labels, in order to improve the system performance?

  • Journal of Intelligent and Robotic Systems-
    -- RAS Journal --- IROS2014
  • Demo1 --- Demo2
  • Restaurant Object Dataset v.1.0 (RGB-D)

  • Learning to Grasp Familiar Objects using Object View Recognition and Template Matching

    In this work, interactive object view learning and recognition capabilities are integrated in the process of learning and recognizing grasps. The object view recognition module uses an interactive incremental learning approach to recognize object view labels. The grasp pose learning approach uses local and global visual features of a demonstrated grasp to learn a grasp template associated with the recognized object view. A grasp distance measure based on Mahalanobis distance is used in a grasp template matching approach to recognize an appropriate grasp pose.

  • IROS2016
  • Demo1 --- Demo2
  • Face Recognition Using Single Normal Reference Image and Statistical Features (Master Thesis)

    Many types of research have been conducted to improve the accuracy of face recognition techniques. The majority of reported techniques make use of databases where a number of images are available for each person. Since collecting face samples is a challenging task, there are some face recognition methods that work based on a single sample per person (SSPP). I studied face recognition using single normal reference image and statistical features. We encoded the face information by making use of a Modular Principal Component Analysis. The nearest neighbour classifier was finally used to assess the dissimilarity between the target face and trained faces.

    Humanoid Robots (RoboCup-HL)

    After obtaining extensive knowledge about real-time intelligent robotic systems in Middle-Size League, I tried to make humanoid robots and formed two new robotic teams namely Persia and BehRobot for participating in RoboCup humanoid leagues. We worked on three different types of humanoid robots including kid-size (height = 59cm, weight = 4kg), teen-size (height = 93cm, weight = 7Kg) and adult-size (height = 155cm, weight = 11:5Kg) robots. We were one of the successful teams in the humanoid leagues and achieved several ranks in national and international competitions.

  • ICARSC2016
  • Demo1
  • Middle Size Soccer Robots (RoboCup-MSL)

    During the second year of my undergraduate program, I got familiar with RoboCup competitions. I formed a team of Middle Size Soccer Robots (RoboCup-MSL) namely ADRO in 2006. We provided five player robots and one goalkeeper robot with similar structure but equipped with some additional accessories and sensors. Through this teamwork, I took an active role in the development of the robots’ software. Furthermore, I worked on the mechanical design of the robot via Autodesk Inventor. We achieved several ranks in national and international RoboCup competitions.

  • Demo1
  • A Novel Morphological Method for Detection and Recognition of Vehicle License Plates

    License plate detection and recognition is an image-processing technique used to identify a vehicle by its license plate. This notable technology has got multiple applications in various traffic and security cases. This study presented a novel method of identifying and recognizing license plates based on the morphology and template matching. The algorithm started with preprocessing and signal conditioning. Next license plate is localized using morphological operators. Then a template matching scheme will be used to recognize the digits and characters within the plate.

  • American Journal of Applied Sciences -
    -- EISIC2011
  • Iranian Car Plate Dataset v.1.0 (RGB)

  • Students

    PhD Students

    • Sha Luo (Oct.2018 ~ )
    • Deep Reinforcement Learning for Flexible Visually Guided Grasping
    • Advisors: Lambert Schomaker, Hamidreza Kasaei

    Master Students

    • Jos van Goor (Feb.2021 ~ )
    • Leveraging Deep Object Recognition Models for Per-Point 6-DOF Grasp Synthesis
    • Krishna Santhakumar (Feb.2021 ~ )
    • Lifelong Object Grasp Synthesis using Dual Memory Recurrent Self-Organization Networks
    • Arjan Jawahier (March.2021 ~ )
    • Descriptive Viewpoint Prediction: Simultaneous Object Recognition and Grasping in Service Robots
    • Hari Vidharth (Feb.2021 ~ )
    • Learning and Generalization of Long-Horizon Sequential Pick and Place Tasks with Deep Reinforcement Learning
    • Georgios Tziafas (Feb.2021 ~ )
    • Sim2Real Transfer of Visiolinguistic Representations for Human-Robot Interaction
    Paper Demo
    • Tommaso Parisotto (July.2020 ~ March 2021)
    • MORE: Simultaneous Multi-View 3D Object Recognition and Grasp Pose Estimation
    Paper Code Thesis
    • Thijs Eker (Nov.2020 ~)
    • Viewpoint-invariant Ship Classification using 3D Reconstruction Models
    • Sudhakaran Jain (March.2020 ~ Dec.2020)
    • Open-Ended 3D Object Recognition using Dynamically Evolving Neural Networks
    Paper Demo Code Thesis
    • Subilal Vattimunda Purayil (April.2020 ~ Dec.2020)
    • Learning Deep Spatio-Temporal Features for Human Activity Classification
    Thesis
    • Diego Cabo Golvano (Feb.2020 ~ Dec.2020)
    • Exploring Novel Hierarchical Reinforcement Learning Approaches to Lifelong Learning
    Thesis
    • Yikun Li (Jan.2019 ~ Aug.2019)
    • Learning to Detect Grasp Affordances of 3D Objects using Deep Convolutional Neural Networks
    Paper Demo Dataset Thesis
    • Mario Rios-Munoz (Jan.2019 ~ Mar. 2020)
    • Learning to Grasp: A Deep Learning Approach to Generalized Robust Grasp Affordance
    Paper Thesis

    Undergraduate Students

    • Jeroen Oude Vrielink (Feb.2021 ~ )
    • Learning grasp affordances of 3D objects using Deep Convolutional Neural Networks
    Proposal Thesis Page
    • Anne-Jan Mein (Feb.2021 ~ )
    • Investigating the influences of different colour spaces in open-ended 3D recognition
    Proposal Thesis Page
    • Junhyung Jo (Feb.2021 ~ )
    • Fine-grained 3D object recognition: an approach and experiments
    Proposal Thesis Page
    • Jim Wu (Sep.2020 ~ Jan.2021 )
    • Lifelong 3D Object Recognition: a comparison of deep features and handcrafted descriptors
    Proposal Thesis Page
    • Andreea Toca (Feb.2020 ~ July 2020)
    • Investigating the Importance of Textures, Color Spaces and Similarity Measures in Open-Ended 3D Object Recognition
    Proposal Thesis Page
    • Vlad Iftime (Feb.2020 ~ July 2020)
    • Autoencoder-based Representation Learning for 3D Object Recognition in Open-Ended Domains
    Proposal Thesis Page
    • Roberto Navarro (Feb.2020 ~ July 2020)
    • Learning to Grasp 3D objects using Deep Convolutional Neural Network
    Proposal Thesis Page
    • Nils Keunecke (Feb.2020 ~ July 2020)
    • Three-Dimensional Object Recognition using OrthographicNet and Color Constancy
    Proposal Thesis Page Demo
    • Sandra Bedrossian (Jan.2019 ~ July 2019)
    • Vehicle License Plate Recognition Using Pixel Information
    Proposal Thesis Page

    Open Positions

    If you are interested in doing your Bachelor/Master/PhD thesis in one of the above areas, or working on a project with me, please send me an e-mail including:

    • Short CV
    • Short motivation letter

    The motivation letter should state (½ - 1 page):

    • Topics that you are interested in
    • Type of project (theoretical/applied)
    • Intended starting date
    • Your relevant experiences

    Contact

    Dr. Hamidreza Kasaei
    Artificial Intelligence Department,
    University of Groningen,
    Bernoulliborg building,
    Nijenborgh 9 9747 AG Groningen,
    The Netherlands.
    Office: 340
    Tel: +31-50-363-33926
    E-mail: hamidreza.kasaei@rug.nl