Call for Interns

OMRON SINIC X (OSX) is looking for research interns throughout the year to work with our members in challenging research projects on a variety of topics related to robotics, machine learning, computer vision, and HCI. Many students have participated in our internship program, and their achievements have been published as academic papers at international conferences such as CVPR, ICML, IJCAI, ICRA, CoRL, or as OSS libraries. For more information about our activities at OSX, please visit Medium and GitHub

Conditions How to apply Explorer Activities

Keyword Search

Select keyword(s) to narrow down available projects

No results

3D-printed one-piece force transmission mechanism design

We will develop a design method for a 3D-printed one-piece force transmission mechanism to be used for lightweight manipulators that are actuated from the root, such as the manipulator we developed in the past (https://omron-sinicx.github.io/twistsnake/).

  • Required skills and experience
    • Experience in robotic mechanism design
    • Publication in robotics (IROS, ICRA, etc.)
  • Preferred skills and experience
    • Robot competition
    • Experience in team-based development
    • Experience of receiving an award from an academic society or a scholarship
    • Experience of coding in ROS, Python, or C++ in the development of a robot
  • Robotics
  • Mechanics design

Learning Manipulation with Segmentation Using Large-Scale Language Models

We will develop a method for learning manipulation with segmentation using Large-Scale Language Models

  • Required skills and experience
    • Publication record in robot learning (ICRA, IROS, CoRL, ICML, NeurIPS, ICLR, etc.)
    • Experience in manipulation learning
  • Preferred skills and experience
    • Management of projects and code using git/GitLab/GitHub
    • Experience in team-based development
  • Machine learning
  • Manipulation

Modeling the Cognitive Mechanisms of Robot Embodiment

This project focuses on studying the cognitive mechanisms that enable individuals to perceive a robotic system as an extension of their own body (embodiment). By combining psychophysical experiments with machine learning techniques, we aim to identify factors contributing to embodiment and construct a mathematical model that explains this phenomenon. The project also includes submitting research findings to top-tier journals (e.g., Science, Nature). This project requires on-site work in Tokyo.

  • Required skills and experience
    • Knowledge and experience in deep learning
    • Understanding of embodiment in psychology (e.g., sense of agency, sense of body ownership)
    • Skills and experience in developing VR systems using Unity
  • Preferred skills and experience
    • Experience presenting research papers in fields such as HCI, VR, CV, or ML
    • Experience designing and conducting user experiments
    • Basic knowledge of statistical analysis
  • Machine learning
  • Robotics
  • Computer vision
  • Interaction
  • Embodied cognition
  • Sense of agency
  • World model

Research on AI-Assisted Remote Communication Support

This project involves the development and evaluation of remote communication support tools utilizing generative AI. Through quantitative and qualitative user evaluations, we aim to investigate the impact and potential risks of AI-driven communication tools. The project also includes submitting research findings to top international conferences and journals in the field of HCI (e.g., CHI, UIST, CSCW). This project requires on-site work in Tokyo.

  • Required skills and experience
    • Research experience in HCI or CSCW
    • Knowledge and experience in developing systems utilizing generative AI
  • Preferred skills and experience
    • Experience in presenting research papers at HCI/CSCW-related conferences (e.g., CHI, UIST, CSCW)
    • Experience in designing and conducting user experiments
    • Basic knowledge of statistical analysis
  • Machine learning
  • Interaction
  • Natural language processing
  • AI-mediated Communication
  • Generative AI

Imitation learning of object manipulation skills generalized to objects and tasks

We will develop a method for imitation learning of object manipulation skills generalized to objects and tasks.

  • Required skills and experience
    • Publication record in robot learning (ICRA, IROS, CoRL, ICML, NeurIPS, ICLR, etc.)
    • Experience in manipulation learning
  • Preferred skills and experience
    • Management of projects and code using git/GitLab/GitHub
    • Experience in team-based development
  • Machine learning
  • Robotics
  • Manipulation
  • Imitation learning

Research on Practical LLMs in the Medical Field

This research explores the practical application of large language models (LLMs) in the medical field, focusing on solving real-world challenges in diagnostic support, treatment planning, and patient communication. Key approaches include designing multimodal models that integrate diverse data for intuitive use, fine-tuning LLMs to enhance medical expertise, and connecting external knowledge bases for up-to-date medical information access. By addressing these aspects, the study aims to reduce the workload of healthcare professionals, improve clinical efficiency, and develop foundational technologies that can be effectively utilized in real medical settings.

  • Required skills and experience
    • Publication record in related fields
    • Knowledge and research/development experience with LLMs
  • Preferred skills and experience
    • Experience related to the medical field
  • Machine learning
  • Computer vision
  • Interaction
  • Natural language processing
  • LLM
  • Medical

Research on Procedural Video Analysis Using Vision & Language Technology

We aim for our research on understanding procedural videos, such as those of cooking and biochemical experiments, to be accepted at leading conferences. This research doesn't just focus on using procedural texts (recipes and protocols); it also explores developing a broad range of technologies using the latest Vision-and-Language Models (VLM).

  • Required skills and experience
    • Experience in writing papers in the field of Natural Language Processing or Computer Vision
    • Experience in implementing code using APIs for LLMs or VLMs
    • Experience in project development using Python, Docker, and GitHub
  • Preferred skills and experience
    • Experience in research and development of video analysis technology using VLMs
    • Experience in having papers accepted at top conferences in the relevant field
  • Machine learning
  • Computer vision
  • Interaction

Study on multi-agent path planning algorithms from the topological viewpoint

We will study approaches to multi-agent path planning from the topological viewpoint. Accepted interns are expected to work in collaboration with the mentors to submit research results to top international conferences in the field of artificial intelligence and machine learning.

  • Required skills and experience
    • Basic knowledge in topological geometry and computational geometry
  • Preferred skills and experience
    • Research and development experiences on path planning, especially multi-agent path planning
    • Publication record in the field of artificial intelligence (e.g., AAAI, IJCAI, AAMAS, ICML, NeurIPS, ICLR)
    • Expert knowledge in topological geometry and computational geometry
  • Algorithm
  • Path planning
  • Multi-agent system

Development of a manipulator driven by all motors at the base link.

We will develop a lightweight manipulator driven by all motors at the base link like our previous manipulator (https://omron-sinicx.github.io/twistsnake/).

  • Required skills and experience
    • Experience in robotic mechanism design
    • Publication in robotics (IROS, ICRA, etc.)
  • Preferred skills and experience
    • Robot competition
    • Experience in team-based development
    • Experience of receiving an award from an academic society or a scholarship
    • Experience of coding in ROS, Python, or C++ in the development of a robot
  • Robotics
  • Mechanics design

Versatile peg insertion skill learning

We will develop a framework for learning versatile peg insertion based on our model based reinforcement learning methods, such as transfer learning (https://kazutoshi-tanaka.github.io/pages/transam.html), model switching (https://kazutoshi-tanaka.github.io/pages/smmrl/), sub-task transition using tactile information (https://omron-sinicx.github.io/saguri-bot-page/).

  • Required skills and experience
    • Publication record in human-computer interactions (ICRA, IROS, CoRL, ICML, NeurIPS, ICLR, etc.)
    • Experience in manipulation learning
    • Experience of coding in ROS, Python, or C++ in research
  • Preferred skills and experience
    • Management of projects and code using git/GitLab/GitHub
    • Experience in team-based development
  • Machine learning
  • Robotics
  • Manipulation
  • Model based reinforcement learning

Multimodal understanding of specialized documents

Understanding technical documents such as papers and patents require understanding data, including structured text and diagrams. It requires efforts that go beyond the framework of conventional natural language processing. In this project, we will conduct research and development on a multimodal understanding of such specialized documents and write papers for international conferences in related fields.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Mathematical knowledge and formulation ability in machine learning and deep learning
  • Computer vision
  • Natural language processing
  • Multimodal understanding

Research on fusion understanding of image/video and natural language

While there are tons of research on machine learning for understanding images and natural language, deep learning has led to the commoditization of modules from each other, and research combining multiple modalities is also increasing. In this project, we will develop research on fusion understanding of image/video and natural language and write papers for relevant top international conferences.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Knowledge and experience in computer vision
  • Machine learning
  • Computer vision
  • Natural language processing
  • Multimodal understanding

Research on application of machine learning to physics simulation method and results

We will conduct research and development on the application of machine learning to the algorithms or the data analysis of physics simulations, e.g. DFT/MD/Tensor Network, and write papers to journals e.g. Nature/Science, Physical Review, or conferences e.g. SC/ICML.

  • Required skills and experience
    • Knowledge and experience in physics simulation e.g. DFT/MD/Tensor Network
  • Preferred skills and experience
    • Knowledge and experience in machine learning
    • Pytorch, Python
    • Github, Docker
  • Machine learning
  • Physics simulation

Research on application of machine learning methods to physics simulation

We will conduct research and development on the application of your machine learning methods to physics simulations and write papers in related fields (Nature/Science, Physical Review, SC/HPCG, etc.).

  • Required skills and experience
    • Knowledge and experience in machine learning
  • Preferred skills and experience
    • Knowledge and experience in physics simulation e.g. DFT/MD/Tensor Network
    • Pytorch, Python
    • Github, Docker
  • Machine learning
  • Physics simulation

Learning soft robotic tool manipulation with tactile sensors

The goal of this project is to enable robots equipped with physically soft bodies and tactile sensing to learn complex tool manipulation tasks, thereby elucidating the synergies among soft robots, tactile sensing, and motor learning.
To this end, developing a tactile-based learning framework, employing data-efficient learning approaches, and exploring strategies for learning from sub-optimal datasets will be essential.

We would like the accepted interns to develop the robot learning algorithm with tactile sensor fusion, implement robot software, perform experiments, and write papers.
We aim to submit top-tier robotics or AI conference or journal papers such as (ICRA, IROS, CoRL, RA-L, T-RO, and NeurIPS). The mentors will meet with the interns at least once a week to discuss the research progress, plans for paper submissions, and distribution of writing tasks to ensure a more reliable submission.

The following themes are relevant but not limited. Themes will be flexibly determined based on the intern's expertise. The project actively seeks interns with experience in machine learning, reinforcement learning development, and a strong interest in robotics applications. The project assumes on-site work in Tokyo.
- Soft robotic manipulation learning
- Tactile-based manipulation using vision-based or distributed tactile sensors
- Offline and online reinforcement learning, and Imitation learning
- Language-guided manipulation
Accepted papers that interns contributed as first author advised by this mentor: IROS2024, ICRA2024 (2 papers), IROS 2023 (2 papers), NeurIPS 2023, IEEE ACCESS, IEEE CASE 2021, and CoRL 2020.
Related projects: learning robotics assembly using soft wrist and tactile sensors. https://omron-sinicx.github.io/saguri-bot-page/, https://omron-sinicx.github.io/symmetry-aware-pomdp/, https://omron-sinicx.github.io/soft-robot-sim-to-real/

  • Required skills and experience
    • Experience in Python or C++

    At least one of the following:

    • Research and development experiences in robot learning, sensing, control theory, or motion planning
    • Research and development experiences using machine learning or reinforcement learning
  • Preferred skills and experience
    • Research and development experience in ROS
    • Experience in submitting papers of the field of robotics and artificial intelligence
    • Experience in participation in robot competitions
  • Machine learning
  • Robotics
  • Signal processing
  • Algorithm
  • Development
  • Soft robotics
  • Tactile sensing

Research automation for machine learning

You will be working on an AI that performs machine learning research autonomously.
To date, we have been working on the automatic improvement of machine learning algorithms in the project AutoRes (https://www.autores.one/) and have obtained budding results. In this project, we aim to further develop this direction and create a system that automatically proposes machine learning algorithms and verifies their improved performance.

  • Required skills and experience
    • Experience in Python development
    • Experience in machine learning development
  • Preferred skills and experience
    • Knowledge of large-scale language models
    • Experience writing papers in machine learning related fields
  • Machine learning
  • Natural language processing
  • Algorithm
  • Development
  • Large language models (LLM)
  • Research Automation

Robust Image Recognition Model

Image recognition models are noted to be vulnerable to domain shifts caused by environmental changes. This project aims to construct robust image recognition models resilient to environmental changes. We plan to select research topics on robust image recognition models from various fields, including but not limited to generalization of image classification models and applications such as Vision-Language models, and aim to submit them to international conferences.

  • Required skills and experience
    • Experience in Python development
    • Experience in developing machine learning model for image recognition
  • Preferred skills and experience
    • Knowledge of image recognition model
    • Knowledge of transfer learning and multi-modal model.
    • Experience writing papers in related fields.
  • Machine learning
  • Computer vision
  • Natural language processing
  • Algorithm
  • Domain Generalization
  • Image recognition
  • Multi-modal model
  • Robust

Research on law discovery from observed data

Research on causal analysis on time-series data and explanatory AI is progressing to make some predictions while clarifying the laws between data. For example, symbolic regression for science discovery is one of the research topic. In this project, we will conduct research and development on methods from a new angle for discovering such laws and write papers aiming for acceptance in top international conferences in machine learning or journals. The internship is mainly aimed at Ph.D. students and is expected to last at least three months.

  • Required skills and experience
    • Python, Github, Docker
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Knowledge and experience in signal processing
  • Machine learning
  • Data mining

Research on human-in-the-loop machine learning

Machine learning research that incorporates humans in machine learning and makes efficient use of feedback from humans is expanding. In this project, we will conduct research and development on human-in-the-loop machine learning research and write papers aiming for publication in top international conferences in machine learning and interaction or journals.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in human-computer interaction
  • Machine learning
  • Interaction
  • HCI
  • HRI
  • HAI

Representation learning for structured data

Pre-training models based on supervised/self-supervised learning using large amounts of data are widely used in machine learning for natural language and images. The concept of foundation models is gaining ground. In this project, we will conduct research and development on representation learning of data with unique structures other than images and natural language and write a paper aiming at a journal such as Nature and Science families.

  • Required skills and experience
    • Python, Github, Docker
  • Preferred skills and experience
    • Knowledge and experience in representation learning with images/natural language
    • Knowledge and experience in machine learning with point clouds/graphs
    • Mathematical knowledge of machine learning and deep learning and ability to formulate equations
  • Machine learning
  • Representation learning
  • Point cloud processing
  • Graph processing

AI for Science

You will work on AI research that accelerates and automates research and development itself. You will participate in partial projects in the realization of AI scientists who can formulate research claims, run experiments, analyze the results, and write papers in an interactive co-evolution with human researchers.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing, computer vision, and data science
    • Mathematical knowledge and formulation ability in machine learning and deep learning
  • Machine learning
  • Interaction

Research on 3D vision including visual SLAM and NeRF

In this research project, we will pursue new models and optimization techniques for image-based 3D sensing technologies such as Visual SLAM and NeRF. We aim to publish papers at top international conferences in the computer vision field, such as CVPR, ICCV, and ECCV.

  • Required skills and experience
    • Research and/or development experience in deep learning using PyTorch, etc
    • Good mathematical understanding about 3D geometries
    • Python
  • Preferred skills and experience
    • Knowledge and experience in 3D deep learning or classical VSLAM
    • Knowledge and experience in munerical optimization
    • Implementation skills of custom forward&backward functions and GPU kernels in PyTorch, etc
    • Programming skills in C++
    • Knowledge and experience with GitHub/GitLab and Docker
  • Machine learning
  • Computer vision
  • Algorithm
  • 3D vision
  • Optimization

Development of an Explainable Language-Based System Control Framework

We are at the dawn of the social utilization of applications based on LLMs, such as LangChain. In this project, we aim to create a framework for "trustworthy" system control through language directives, capturing market share opportunities unique to this period. We plan to develop and release an OSS (Open Source Software) that generates reusable and shareable processes through user interactions. Additionally, we aim to submit our work to the OSS Track of international conferences.

  • Required skills and experience
    • Knowledge of Python sufficient to understand the internals of PyTorch
    • Basic knowledge of git
  • Preferred skills and experience
    • Knowledge of optimization methods like linear programming
    • Experience contributing to open-source software (any level is acceptable)
    • Experience in utilizing LLMs for coding
  • Interaction
  • Algorithm
  • Development

Realizing Deep Graph Neural Networks Effective for Dense Graphs

In general, Graph Neural Networks (GNNs) are known to lose unique information per vertex due to oversmoothing when the edge density is high. The traditional method to address this issue, known as WeaveNet, has problems with high memory consumption and computational redundancy. This project aims to solve these issues to make it applicable to practical-sized problems and to tackle various unresolved issues for general GNNs, with the goal of submitting to the most prestigious international conferences in the field of machine learning.

  • Required skills and experience
    • Knowledge of PyTorch sufficient to implement GNNs
    • Basic knowledge of git
  • Preferred skills and experience
    • Knowledge of important areas such as combinatorial optimization, which are challenging for GNNs
  • Machine learning
  • Algorithm
  • Graph Neural Network

Participation and Management of Competitions Related to Fixed-Viewpoint Video Analysis Technology

We are recruiting interns to participate in and manage various activities leading up to the Competition-based Workshop to be held at the top image processing conference in 2025. This includes both participating in and organizing the competition.

  • Required skills and experience

    (At least one of the following )

    • Participation in Competition-based Workshops
    • Research experience in Vision & Language technology
    • Research experience in Human-Object Interaction detection technology
  • Preferred skills and experience
    • Experience in organizing Competition-based Workshops
    • Research experience in analyzing work videos
  • Machine learning
  • Computer vision
  • Natural language processing
  • Vision&Language
  • Procedure Understanding
  • Human-Object Interaction Detection

Conditions

Term: From 3-month duration (assuming 5 working days a week). Start and end dates can be adjusted. Some projects accept short-term interns from 1-month duration.
Hours: Full-time or part-time (e.g. 3 days a week, etc. negotiable). 45-minute breaks. Holidays and weekends off.
Location: On-site, hybrid, or remote options are available. Hybrid and remote options are only available if you live in Japan; due to legal issues, we cannot pay salaries to remote interns who live outside of Japan. If you join our internship program, you must come to Japan. In such cases, we offer support for travel expenses. Some internship projects may require on-site work. In this case, you will be assigned to one of our offices in Hongo or Shinagawa.
Salary: Full-time monthly salary ranges from 240,000 JPY to 480,000 JPY. Hourly rate is applied for part-time work. Social security and other benefits are provided according to the working conditions. Transportation and housing expenses are fully covered. In addition, other expenses necessary for research activities (PC, laptop, etc.) are fully supported.
Language: Japanese or English (English-only communication is also fine.)
Others: Two or more mentors with extensive research experience will provide in-depth support for each project. Computational resources (workstations, and server clouds with GPUs) and robotic facilities (robotic arms, various sensors, 3D printers, motion capture systems, and other prototyping and experimental equipment) are available.

How to apply

Please fill the application form. We will first screen each application based on those information.
For other inquires, please contact internships@sinicx.com. We will first screen each application based on those information.

Those who pass the above screening will be interviewed remotely. Please prepare slides or other materials to introduce past research and development activities and achievements.

Please contact us at least three months in advance if you need a visa to enter Japan.

Explorer Activities