No results

Machine learning applications to multimodal biological data

The approach for biological data processing is shifting from inference based on the understanding of either time-series data or medical images in isolation to the fusion of multimodal data with different attributes and structures. In this project, we will conduct research and development on the application of machine learning to such multimodal biological data and write a paper for an international conference in the related field.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in XAI
    • Knowledge and experience in computer vision
    • Knowledge and experience in tabular data machine learning
  • Machine learning
  • Computer vision
  • Signal processing
  • Multimodal understanding

Integrated Task and Motion Planning for Agile Robotic Manipulation

This project aims to bridge the gap between high-level task planning and low-level motion planning for agile manipulation tasks. Interns will work on developing algorithms that enable robots to autonomously plan and execute complex manipulation tasks by efficiently integrating task-level objectives with continuous motion planning. The project involves exploring motion primitives, constraint reasoning, and optimization techniques.
We aim to submit top-tier robotic conference or journal papers, e.g., ICRA, IROS, RA-L. We are also actively hiring interns with experience in machine learning or reinforcement learning development with a strong interest in robotic applications.

  • Required skills and experience
    • Research experience in task/motion planning
    • Research or development in ROS
    • Programming with Python or C++
    • Ability to code using tools such as Git, Docker, and VSCode.
  • Preferred skills and experience

    * Experience in submitting papers in the field of robotics

    * Experience in participation in robot competitions

  • Robotics
  • Algorithm
  • Development
  • Task planning
  • Motion planning

Learning contact-rich manipulation skills from few demonstrations

This project focus on enabling robots to achieve dexterous contact-rich manipulation tasks through limited demonstrations. Interns will explore techniques that facilitate robots in learning complex manipulation skills from a sparse set of demonstrations. The emphasis will be on distilling essential information from these demonstrations to enable robots to grasp, handle, and manipulate objects in contact-rich scenarios. The project involves algorithm development, sensor integration, and learning methodologies aimed at enabling robots to generalize from a few demonstrations to versatile manipulation tasks.
We aim to submit top-tier robotic conference or journal papers, e.g., ICRA, IROS, RA-L. We are also actively hiring interns with experience in machine learning or reinforcement learning development with a strong interest in robotic applications.

  • Required skills and experience
    • Research experience in robot learning (RL, LfD, IRL, and such)
    • Research or development in ROS
    • Programming with Python or C++
    • Ability to code using tools such as Git, Docker, and VSCode.
  • Preferred skills and experience

    * Experience in submitting papers in the field of robotics, machine learning or related fields

    * Experience in participation in robot competitions

  • Machine learning
  • Robotics
  • Algorithm
  • Development
  • Task planning
  • Motion planning

Learning-Infused Planning: Interpretable Long-Horizon Task Execution via Learning-Enabled Planning

This project aims to combine task and motion planning with learning-based robotic skills to solve extensive tasks, emphasizing the interpretability of robot actions and plans. Interns will explore the fusion of task-level planning with learned behaviors to enable robots to reliably navigate long-horizon tasks. The focus will be on developing algorithms that allow robots to interpret and explain their planned actions, enhancing transparency and trust in their decision-making processes. The project involves designing interpretable frameworks that merge symbolic task planning with learned control policies, emphasizing explainable and transparent actions.
We aim to submit top-tier robotic conference or journal papers, e.g., ICRA, IROS, RA-L. We are also actively hiring interns with experience in machine learning or reinforcement learning development with a strong interest in robotic applications.

  • Required skills and experience
    • Research experience in robot learning or task and motion planning
    • Research or development in ROS
    • Programming with Python or C++
    • Ability to code using tools such as Git, Docker, and VSCode.
  • Preferred skills and experience

    * Experience in submitting papers in the field of robotics, machine learning or related fields

    * Experience in participation in robot competitions

  • Machine learning
  • Robotics
  • Algorithm
  • Development
  • Task planning
  • Motion planning

Learning soft robotic tool manipulation with tactile sensors

This project aims to make a physical soft robot with tactile sensors perform contact-rich and dextrous tasks using tools. To this end, high-dimensional tactile information processing and data-efficient learning approaches that learn the control policy with a few training data will be required.
We would like the accepted interns to develop the robot learning algorithm with tactile sensor fusion, implement robot software, perform experiments, and write papers.
We aim to submit top-tier robotics or AI conference or journal papers such as (ICRA, IROS, CoRL, RA-L, T-RO, and NeurIPS). The mentors will meet with the interns at least once a week to discuss the research progress, plans for paper submissions, and distribution of writing tasks to ensure a more reliable submission.
The following themes are relevant but not limited. Themes will be flexibly determined based on the intern's expertise. The project actively seeks interns with experience in machine learning, reinforcement learning development, and a strong interest in robotics applications. The project assumes on-site work in Tokyo.

・Soft robotic manipulation learning
・Tactile-based manipulation using vision-based or distributed tactile sensors
・Offline and online reinforcement learning
・Sim-to-real transfer learning

Accepted papers that interns contributed as first author advised by this mentor: IROS 2023 (2 papers), NeurIPS 2023, IEEE ACCESS, IEEE CASE 2021, and CoRL 2020.
Related projects: learning robotics assembly using soft wrist and tactile sensors. https://omron-sinicx.github.io/saguri-bot-page/

  • Required skills and experience
    • Experience in Python or C++

    At least one of the following:

    • Research and development experiences in robot learning, sensing, control theory, or motion planning
    • Research and development experiences using machine learning or reinforcement learning
  • Preferred skills and experience
    • Research and development experience in ROS
    • Experience in submitting papers of the field of robotics and artificial intelligence
    • Experience in participation in robot competitions
  • Machine learning
  • Robotics
  • Reinforcement learnig
  • Soft robotics
  • Tactile sensing

an LLM-based personalized conversational agent

The current LLM excels in conversational and problem-solving abilities as a dialogue agent. However, it lacks capabilities such as learning user habits and long-term memory, making it insufficient as an intelligent companion. This study aims to achieve a personalized conversational agent that can provide users with a pseudo-experience characterized by being "friendly, intelligent, and exhibiting a sense of growth" through the construction of LLM's long-term memory and learning of rules.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in LLM
  • Preferred skills and experience
    • LangChain
    • Vector Database
    • Knowledge Graph
  • Natural language processing
  • LLM

Autoformalization with LLM

While mathmatical proofs can be mechanically checked for validity by rewriting them in formal language, generally it requires a huge amount of manpower. For this reason, rewriting using language models has been proposed, but has not reached a practical stage at present. This study aims to create suitable training data and benchmarks for the practical application of autoformalization.

  • Required skills and experience
    • Familiarity with some proof assistant
    • Basic knowledge in abstract mathematics
  • Preferred skills and experience
    • Knowledge and experience in LLM
  • Machine learning
  • Natural language processing
  • LLM
  • Proof Assitant

Development of shape memory alloys actuators for an agile humanoid robot

We will develop shape memory alloys actutators for an agile humanoid robot

  • Required skills and experience
    • Publication record in human-computer interactions (ICRA, IROS, etc.)
    • Experience in development of fuild control systems
    • Experience in development of actuator design
    • Experience of coding in ROS, Python, or C++ in the development of a robot
  • Preferred skills and experience
    • Management of projects and code using git/GitLab/GitHub
    • Experience in team-based development
  • Robotics

Development of a manipulator driven by all motors at the base link.

We will develop a lightweight manipulator driven by all motors at the base link like our previous manipulator (https://omron-sinicx.github.io/twistsnake/).

  • Required skills and experience
    • Experience in robotic mechanism design
    • Publication in robotics (IROS, ICRA, etc.)
  • Preferred skills and experience
    • Robot competition
    • Experience in team-based development
    • Experience of receiving an award from an academic society or a scholarship
    • Experience of coding in ROS, Python, or C++ in the development of a robot
  • Robotics
  • Mechanics design

Versatile peg insertion skill learning

We will develop a framework for learning versatile peg insertion based on our model based reinforcement learning methods, such as transfer learning (https://kazutoshi-tanaka.github.io/pages/transam.html), model switching (https://kazutoshi-tanaka.github.io/pages/smmrl/), sub-task transition using tactile information (https://omron-sinicx.github.io/saguri-bot-page/).

  • Required skills and experience
    • Publication record in human-computer interactions (ICRA, IROS, CoRL, ICML, NeurIPS, ICLR, etc.)
    • Experience in manipulation learning
    • Experience of coding in ROS, Python, or C++ in research
  • Preferred skills and experience
    • Management of projects and code using git/GitLab/GitHub
    • Experience in team-based development
  • Machine learning
  • Robotics
  • Manipulation
  • Model based reinforcement learning

Development of an acutation system for sustainable robots

We will develop an acutation system for carbon neutral robots.

  • Required skills and experience
    • Robot development experience
  • Preferred skills and experience
    • Publication record in human-computer interactions (ICRA, IROS, etc.)
    • Management of projects and code using git/GitLab/GitHub
    • Experience of coding in ROS, Python, or C++ in the development of a robot
    • Experience in team-based development
  • Robotics

Research on 3D vision including visual SLAM and NeRF

In this research project, we will pursue new models and optimization techniques for image-based 3D sensing technologies such as Visual SLAM and NeRF. We aim to publish papers at top international conferences in the computer vision field, such as CVPR, ICCV, and ECCV.

  • Required skills and experience
    • Research and/or development experience in deep learning using PyTorch, etc.
    • Good mathematical understanding about 3D geometries
    • Python
  • Preferred skills and experience
    • Knowledge and experience in 3D deep learning or classical VSLAM
    • Knowledge and experience in munerical optimization
    • Implementation skills of custom forward&backward functions and GPU kernels in PyTorch, etc.
    • Programming skills in C++
    • Knowledge and experience with GitHub/GitLab and Docker
  • Machine learning
  • Computer vision
  • Algorithm
  • 3D vision
  • Optimization

Research on application of machine learning to physics simulation method and results

We will conduct research and development on the application of machine learning to the algorithms or the data analysis of physics simulations, e.g. DFT/MD/Tensor Network, and write papers to journals e.g. Nature/Science, Physical Review, or conferences e.g. SC/ICML.

  • Required skills and experience
    • Knowledge and experience in physics simulation e.g. DFT/MD/Tensor Network
  • Preferred skills and experience
    • Knowledge and experience in machine learning
    • Pytorch, Python
    • Github, Docker
  • Machine learning
  • Physics simulation

Research on application of machine learning methods to physics simulation

We will conduct research and development on the application of your machine learning methods to physics simulations and write papers in related fields (Nature/Science, Physical Review, SC/HPCG, etc.).

  • Required skills and experience
    • Knowledge and experience in machine learning
  • Preferred skills and experience
    • Knowledge and experience in physics simulation e.g. DFT/MD/Tensor Network
    • Pytorch, Python
    • Github, Docker
  • Machine learning
  • Physics simulation

Study on multi-agent path planning algorithms from the topological viewpoint

We will study approaches to multi-agent path planning from the topological viewpoint. Accepted interns are expected to work in collaboration with the mentors to submit research results to top international conferences in the field of artificial intelligence and machine learning.

  • Required skills and experience
    • Basic knowledge in topological geometry and computational geometry
  • Preferred skills and experience
    • Research and development experiences on path planning, especially multi-agent path planning
    • Publication record in the field of artificial intelligence (e.g., AAAI, IJCAI, AAMAS, ICML, NeurIPS, ICLR)
    • Expert knowledge in topological geometry and computational geometry
  • Algorithm
  • Path planning
  • Multi-agent systems

Research on machine learning for few training data

In modalities and domains where are pre-trained models so-called foundation models, target tasks can be achieved by finetuning on a small number of data. On the other hand, more advanced machine learning techniques are required for tasks not in such modalities or domains. In this project, we will conduct research and development on such small-data machine learning and write a paper aiming at top international conferences in machine learning and computer vision.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in computer vision
  • Machine learning
  • Transfer learning
  • Domain adaptation

Research on law discovery from observed data

Research on causal analysis on time-series data and explanatory AI is progressing to make some predictions while clarifying the laws between data. For example, symbolic regression for science discovery is one of the research topic. In this project, we will conduct research and development on methods from a new angle for discovering such laws and write papers aiming for acceptance in top international conferences in machine learning or journals. The internship is mainly aimed at Ph.D. students and is expected to last at least three months.

  • Required skills and experience
    • Python, Github, Docker
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Knowledge and experience in signal processing
  • Machine learning
  • Data mining

Action generation of a robot to entertain humans in a game

We will develop a method for generating actions of a robot to entertain humans in a game, such as table tennis and air hockey.

  • Required skills and experience

    • Publication record in human-computer interactions (HAI, HRI, RO-MAN, CHI, SIGGRAPH, etc.)
    • Experience of coding in ROS, Python, or C++ in the development of a robot
    • Knowledge related to robotics (e.g., control, optimization, HCI, computer vision, etc.)
  • Preferred skills and experience
    • Management of projects and code using git/GitLab/GitHub
    • Experience in team-based development
  • Robotics
  • Interaction
  • Development
  • HCI
  • HRI
  • HAI

Research on learnable discrete information processing

Research to modify specific computational modules by making them machine-learnable is underway (e.g., differentiable rendering) to include them in a deep learning pipeline. This project will develop research on making discrete information processing learnable and writing papers for international machine learning conferences, such as ICLR, ICML, and NeurIPS.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Mathematical knowledge and ability to formulate methods for machine learning and deep learning
    • Expertise in convex optimization
  • Machine learning
  • Algorithm
  • Optimization

Representation learning for structured data

Pre-training models based on supervised/self-supervised learning using large amounts of data are widely used in machine learning for natural language and images. The concept of foundation models is gaining ground. In this project, we will conduct research and development on representation learning of data with unique structures other than images and natural language and write a paper aiming at a journal such as Nature and Science families.

  • Required skills and experience
    • Python, Github, Docker
  • Preferred skills and experience
    • Knowledge and experience in representation learning with images/natural language
    • Knowledge and experience in machine learning with point clouds/graphs
    • Mathematical knowledge of machine learning and deep learning and ability to formulate equations
  • Machine learning
  • Representation learning
  • Point cloud processing
  • Graph processing

Research on fusion understanding of image/video and natural language

While there are tons of research on machine learning for understanding images and natural language, deep learning has led to the commoditization of modules from each other, and research combining multiple modalities is also increasing. In this project, we will develop research on fusion understanding of image/video and natural language and write papers for relevant top international conferences.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Knowledge and experience in computer vision
  • Machine learning
  • Computer vision
  • Natural language processing
  • Multimodal understanding

Multimodal understanding of specialized documents

Understanding technical documents such as papers and patents require understanding data, including structured text and diagrams. It requires efforts that go beyond the framework of conventional natural language processing. In this project, we will conduct research and development on a multimodal understanding of such specialized documents and write papers for international conferences in related fields.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Mathematical knowledge and formulation ability in machine learning and deep learning
  • Computer vision
  • Natural language processing
  • Multimodal understanding

Research on human-in-the-loop machine learning

Machine learning research that incorporates humans in machine learning and makes efficient use of feedback from humans is expanding. In this project, we will conduct research and development on human-in-the-loop machine learning research and write papers aiming for publication in top international conferences in machine learning and interaction or journals.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in human-computer interaction
  • Machine learning
  • Interaction
  • HCI
  • HRI
  • HAI

AI for Science

You will work on AI research that accelerates and automates research and development itself. You will participate in partial projects in the realization of AI scientists who can formulate research claims, run experiments, analyze the results, and write papers in an interactive co-evolution with human researchers.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing, computer vision, and data science
    • Mathematical knowledge and formulation ability in machine learning and deep learning
  • Machine learning
  • Interaction

Research on high-dimensional black box optimization

Black-box optimization, such as Bayesian optimization, is subject to computational overheads as the number of parameters to be optimized increases. In this project, we will develop research on high-dimensional black-box optimization and write papers for top international conferences in machine learning or journals.

  • Required skills and experience
    • Python, Github, Docker
  • Preferred skills and experience
    • Knowledge and experience with black box optimization such as Bayesian optimization
  • Algorithm
  • Optimization

Vision-based manipulation learning

We will develop a method for learning trustworthy manipulation.

  • Required skills and experience
    • Publication record in human-computer interactions (ICRA, IROS, CoRL, ICML, NeurIPS, ICLR, CHI, HRI, etc.)
    • Experience of coding in ROS, Python, or C++ in research
  • Preferred skills and experience
    • Management of projects and code using git/GitLab/GitHub
    • Experience in team-based development
    • Knowledge related to robotics (e.g., control, optimization, HCI, computer vision, etc.)
  • Machine learning
  • Robotics
  • Interaction
  • Manipulation
  • Trustworthy AI

Conditions

Term: From 3-month duration (assuming 5 working days a week). Start and end dates can be adjusted. Some projects accept short-term interns from 1-month duration.
Hours: Full-time or part-time (e.g. 3 days a week, etc. negotiable). 45-minute breaks. Holidays and weekends off.
Location: On-site, hybrid, or remote options are available. Hybrid and remote options are only available if you live in Japan; due to legal issues, we cannot pay salaries to remote interns who live outside of Japan. If you join our internship program, you must come to Japan. In such cases, we offer support for travel expenses. Some internship projects may require on-site work. In this case, you will be assigned to one of our offices in Hongo or Shinagawa.
Salary: Full-time monthly salary ranges from 240,000 JPY to 480,000 JPY. Hourly rate is applied for part-time work. Social security and other benefits are provided according to the working conditions. Transportation and housing expenses are fully covered. In addition, other expenses necessary for research activities (PC, laptop, etc.) are fully supported.
Language: Japanese or English (English-only communication is also fine.)
Others: Two or more mentors with extensive research experience will provide in-depth support for each project. Computational resources (workstations, and server clouds with GPUs) and robotic facilities (robotic arms, various sensors, 3D printers, motion capture systems, and other prototyping and experimental equipment) are available.

How to apply

Please fill the application form. We will first screen each application based on those information.
For other inquires, please contact internships@sinicx.com. We will first screen each application based on those information.

Those who pass the above screening will be interviewed remotely. Please prepare slides or other materials to introduce past research and development activities and achievements.

Please contact us at least three months in advance if you need a visa to enter Japan.