No results

Study on leveraging machine learning in path planning algorithms

We will study integrated approaches to path planning and deep learning, such as Neural A* (ICML-21) and CTRMs (AAMAS-22). Particular focus will be placed on multi-agent path planning and on path planning for dynamic and uncontrolled environments. Accepted interns are expected to work in collaboration with the mentors to submit research results to top international conferences in the field of artificial intelligence and machine learning.

  • Required skills and experience

    At least one of the following:

    • Research and development experiences using deep learning
    • Research and development experiences on path planning or motion planning
  • Preferred skills and experience
    • Research and development experiences on multi-agent path planning or multi-agent reinforcement learning
    • Publication record in the field of artificial intelligence (e.g., AAAI, IJCAI, AAMAS, ICML, NeurIPS, ICLR)
    • Experience participating in competitions related to machine learning
    • Expert knowledge in related areas (e.g., optimization, control, HCI, graph algorithms, computer vision, computer graphics, and software development)
  • Machine learning
  • Path planning
  • Multi-agent systems

Study on local planning algorithms for multi-agent and dynamic environments

We study local planning algorithms for multi-agent environments that require the cooperative movement of multiple robots, or dynamic environments with pedestrians and moving objects. In particular, we will work on reinforcement learning-based approaches for crowded environments such as L2B (IROS-20), and other studies that actively utilize machine learning. Other tasks include theoretical studies and library development for sim-to-real transfer in the context of mobile robot motion planning. Accepted interns are expected to work in collaboration with the mentors to submit research results to top international conferences in the field of artificial intelligence and robotics.

  • Required skills and experience

    At least one of the following:

    • Research and development experiences using deep learning
    • Research and development experiences on path planning or motion planning
    • Advanced experinece in ROS navigation stack
  • Preferred skills and experience
    • Research and implementation experience in transfer learning, domain adaptation, and meta-learning
    • Competition or publication records in the field related to autonomous mobile robots
    • Expert knowledge in related fields (e.g., optimization, control, HCI, multi-agent systems, graph algorithms, computer vision, computer graphics, software development)
    • Research and implementation experience using simulators such as PyBullet and Unity ML Agents
    • Publication records in the field of artificial intelligence and robotics (ICML, NeurIPS, ICLR, AAAI, IJCAI, AAMAS, ICRA, IROS, CoRL, RSS, CVPR, ICCV, ECCV, etc.)
  • Robotics
  • Multi-agent systems
  • Path planning/motion planning
  • Reinforcement learning
  • Transfer learning/domain adaptation
  • OSS development

Robotic learning using soft robots or force control for contact-rich manipulation

We develop robotic learning systems using soft robots and force control and aim to apply them to industrial assembly or cooking.
In this project, we would like accepted interns to develop robotic learning systems, manipulation with multi-modal sensing, and motion planning. Besides, we have another project to develop a simulator for soft robots. We aim to submit top-tier robotic conference or journal papers, e.g., ICRA, IROS, RA-L. We are also actively hiring interns with experience in machine learning or reinforcement learning developments and a strong interest in robotic applications.

  • Required skills and experience
    • Experience in Python or C++

    At least one of the following:

    • Research and development experiences in robot learning, sensing, control theory, or motion planning
    • Research and development experiences using machine learning or reinforcement learning
  • Preferred skills and experience
    • Research and development experience in ROS
    • Experience in submitting papers of the field of robotics and artificial intelligence
    • Experience in participation in robot competitions
  • Machine learning
  • Robotics
  • Manipulation
  • Reinforcement learning
  • Task & Motion planning

Robot software or hardware engineering

We would like accepted interns to be involved in robot software or hardware engineering, such as programming, designing robot environments to launch robot systems, or hardware maintenance.
We also highly encourage interns to publish system papers or OSS libraries as achievements during this internship.

  • Required skills and experience

    At least one of the following:

    • Research or development experience in ROS
    • Research or development experience in robot hardware design
    • Research or development experience in sesing design
  • Preferred skills and experience
    • Experience in participation in robot competitions
  • Machine learning
  • Robotics
  • Manipulation
  • Robot system integration
  • Hardware design
  • OSS development

A mechanism and system of a high-speed soft robot

We will develop a mechanism or system of a high-speed soft robot. In particular, we will develop a computational design of mechanisms, such as cable-driven systems, linkage systems, and cam mechanisms and additive manufacturing, such as 3D printing. We will also develop a robotic system that quickly responds to an object and moves fast, such as state estimation and prediction of a fast-moving object and motion planning in a short time.

  • Required skills and experience

    Experience in robotic mechanism design or robotic system design

  • Preferred skills and experience
    • Robot competition
    • Experience in team-based development
    • Publication in robotics (IROS, ICRA, etc.)
    • Experience of receiving an award from an academic society or a scholarship
    • Experience of coding in ROS, Python, or C++ in the development of a robot
  • Robotics
  • Mechanics design
  • Robotic system development

Video analysis of a ball game using machine learning

We are developing a technology to observe the behavior of humans and estimate their intention and desire. In this project, we will apply machine learning techniques to video analysis of a ball game, such as table tennis, extract the actions of players, and analyze the mechanism of decision making and interactions of multiple players.

  • Required skills and experience

    Research experience using deep learning

  • Preferred skills and experience
    • Experience of winning a prize of a machine learning competition such as Kaggle
    • Experience of research using Git/Github and Docker
    • Experience of receiving an award from an academic society or a scholarship
    • Publication in artificial intelligence (CVPR, ICCV, ECCV, NeurIPS, ICML, ICLR, TPAMI, etc.)
  • Machine learning
  • Robotics
  • Computer vision & graphics
  • Human sensing
  • Sports robot

Action generation of a robot to entertain humans in a game

We will develop a method for generating actions of a robot to entertain humans in a game, such as table tennis and air hockey.

  • Required skills and experience

    (Any of the following)

    • Research experience in HAI, HRI, or HCI
    • Research experience using deep learning
    • Research experience in robotics
    • Robot development
  • Preferred skills and experience
    • Publication record in human-computer interactions (HAI, HRI, RO-MAN, CHI, SIGGRAPH, etc.)
    • Management of projects and code using git/GitLab/GitHub
    • Experience of coding in ROS, Python, or C++ in the development of a robot
    • Experience in team-based development
    • Knowledge related to robotics (e.g., control, optimization, HCI, computer vision, etc.)
  • Robotics
  • Interaction
  • Robotic system development
  • Sports robot

Learning dynamic nonprehensile manipulation

We will develop a method for learning dynamic nonprehensile manipulation, such as dynamic re-orientation of a grasped object, hitting an object, and catching a flying object. A robot utilizes information from sensors for learning, such as a camera, a force sensor, and a tactile sensor.

  • Required skills and experience

    Research and development experiences using deep learning

  • Preferred skills and experience
    • Knowledge related to robotics (e.g., control, optimization, HCI, computer vision, etc.).
    • Publication record in robotics or artificial intelligence (IROS, ICRA, ICML, NeurIPS, ICLR, etc.)
    • Experience of receiving an award from an academic society or a scholarship
    • Robot competition or a machine learning competition
    • Management of projects and code using git/GitLab/GitHub
  • Machine learning
  • Robotics
  • Manipulation
  • Multi-modal

Enhancing remote communication via physical avator robots

Removing the restriction in the physical workplace may resolve many social problems. However, remote work is still not a widely-accepted working style even under the current social situation. This project tackles this problem through robot-mediated human-to-human communication. Interns are expected to work with the mentors to submit a paper for an international conference in HCI/HRI.

  • Required skills and experience
    • Experience in user studies for dialog analysis
    • Experience in participating any full-remote projects
  • Preferred skills and experience
    • Experience in JavaScript / Python Flask implementation
  • Interaction
  • Human sensing

Multi-modal eyes image prediction

We will develop research on multimodal eye estimation and prediction methods and write papers for EMBC and other international conference in biomedical engineering.

  • Required skills and experience
    • Knowledge and experience in deep learning
    • Knowledge and experience in git, docker and vscode
  • Preferred skills and experience
    • Knowledge and experience in Vision&Language or multimodal analysis
    • Medical Knowledge about eyes and genes
  • Signal processing
  • Multi-modal
  • Human sensing
  • Biological signal

Effect analysis of real-time information visualization on web meeting

For a multi-person web meeting held at Zoom etc., we will design and carry out experiments using tools that visualize real-time information (amount of conversation, facial expressions, etc.) related to communication. In addition, the experiment will verify the effect of facilitating exchanges and changes between conference participants. The research results will be submitted to international conferences in the field of interaction such as CHI.

  • Required skills and experience
    • Interested in human-computer interaction
  • Preferred skills and experience
    • Experience in conducting user-participatory experiments such as dialogue analysis
    • Experience in paper submission to CHI
  • Interaction
  • Dialog analysis

Research on improving machine learning performance by improving image datasets

According to the concept of "data centric AI", improving the quality of datasets, such as label consistency, often contributes significantly to improving the performance of machine learning models. This research aims to verify the effectiveness of such a data-centric approach in the field of computer vision and to publish paper in a related international conference.

  • Required skills and experience
    • Python
    • Knowledge and experience in deep learning and image processing
  • Preferred skills and experience
    • Experience in implementing and experimenting with object detection/segmentation models on large-scale datasets
  • Machine learning
  • Computer vision & graphics
  • Dataset

Research on fusion understanding of image/video and natural language

While there are tons of research on machine learning for understanding images and natural language, deep learning has led to the commoditization of modules from each other, and research combining multiple modalities is also increasing. In this project, we will develop research on fusion understanding of image/video and natural language and write papers for relevant top international conferences.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Knowledge and experience in computer vision
  • Computer vision & graphics
  • Natural language processing
  • Multi-modal
  • Vision and language

Research on learnable discrete information processing

Research to modify specific computational modules by making them machine-learnable is underway (e.g., differentiable rendering) to include them in a deep learning pipeline. This project will develop research on making discrete information processing learnable and writing papers for international machine learning conferences, such as ICLR, ICML, and NeurIPS.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Mathematical knowledge and ability to formulate methods for machine learning and deep learning
  • Machine learning
  • Optimization

Research on high-dimensional black box optimization

Black-box optimization, such as Bayesian optimization, is subject to computational overheads as the number of parameters to be optimized increases. In this project, we will develop research on high-dimensional black-box optimization and write papers for top international conferences in machine learning or journals.

  • Required skills and experience
    • Python, Github, Docker
  • Preferred skills and experience
    • Knowledge and experience with black box optimization such as Bayesian optimization
  • Machine learning
  • Blackbox optimization
  • Bayesian optimization

Representation learning for structured data

Pre-training models based on supervised/self-supervised learning using large amounts of data are widely used in machine learning for natural language and images. The concept of foundation models is gaining ground. In this project, we will conduct research and development on representation learning of data with unique structures other than images and natural language and write a paper aiming at a journal such as Nature and Science families.

  • Required skills and experience
    • Python, Github, Docker
  • Preferred skills and experience
    • Knowledge and experience in representation learning with images/natural language.
    • Knowledge and experience in machine learning with point clouds/graphs.
    • Mathematical knowledge of machine learning and deep learning and ability to formulate equations
  • Machine learning
  • Representation learning
  • Point cloud processing
  • Graph processing

Research on law discovery from observed data

Research on causal analysis on time-series data and explanatory AI is progressing to make some predictions while clarifying the laws between data. For example, symbolic regression for science discovery is one of the research topic. In this project, we will conduct research and development on methods from a new angle for discovering such laws and write papers aiming for acceptance in top international conferences in machine learning or journals. The internship is mainly aimed at Ph.D. students and is expected to last at least three months.

  • Required skills and experience
    • Python, Github, Docker
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Knowledge and experience in signal processing
  • Machine learning
  • Data mining

Research on data2text

Machine translation is a technology that converts one input text into another. There is also growing research on data2text, which generates captions and news headlines when the input is data such as images or economic data. This project will develop research on data2text for table data and a particular form of texts and write papers aiming for acceptance in top international conferences in the machine learning field or journals.

  • Required skills and experience
    • Python, Github, Docker
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Knowledge and experience in signal processing
  • Machine learning
  • Data mining

Research on human-in-the-loop machine learning

Machine learning research that incorporates humans in machine learning and makes efficient use of feedback from humans is expanding. In this project, we will conduct research and development on human-in-the-loop machine learning research and write papers aiming for publication in top international conferences in machine learning and interaction or journals.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in human-computer interaction
  • Machine learning
  • Interaction
  • Human-in-the-loop

Multimodal understanding of specialized documents

Understanding technical documents such as papers and patents require understanding data, including structured text and diagrams. It requires efforts that go beyond the framework of conventional natural language processing. In this project, we will conduct research and development on a multimodal understanding of such specialized documents and write papers for international conferences in related fields.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Mathematical knowledge and formulation ability in machine learning and deep learning
  • Machine learning
  • Computer vision & graphics
  • Natural language processing
  • Multi-modal
  • Document understanding

Research on machine learning for few training data

Target tasks can be achieved in modalities or domains where a pre-trained foundation model exists by finetuning it on a small number of data. On the other hand, more advanced machine learning techniques are required for tasks not in such modalities or domains. In this project, we will conduct research and development on such small-data machine learning and write a paper aiming at top international conferences in machine learning and computer vision.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in computer vision
  • Machine learning
  • Computer vision & graphics
  • Transfer learning/domain adaptation

Machine learning applications to multimodal biological data

The approach for biological data processing is shifting from inference based on the understanding of either time-series data or medical images in isolation to the fusion of multimodal data with different attributes and structures. In this project, we will conduct research and development on the application of machine learning to such multimodal biological data and write a paper for an international conference in the related field. We welcome applications from undergraduate students.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Knowledge and experience in computer vision
    • Knowledge and experience in signal processing
  • Computer vision & graphics
  • Signal processing
  • Multi-modal
  • Human sensing
  • Biological signal

Radiance Field Optimization

Neural Radiance Field (NeRF) is widely used for general volumetric scene representation. In this project, we research the modeling of radiance fields and the optimization method of the model and write papers for publication in top international conferences in the field of computer vision.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in 3D deep learning
    • Knowledge and experience in munerical optimization
  • Computer vision & graphics
  • Numerical optimization

Research on machine learning of physics simulation

We will conduct research and development on the application of machine learning to physics simulations and write papers in related fields (Nature/Science, Physical Review, SC/HPCG, etc.). We will solve various problems of physics simulation, especially multi-scale properties from the quantum and atomic level to the earth and space level using machine learning.

  • Required skills and experience
    • Knowledge and experience in either deep learning or physics simulation
  • Preferred skills and experience
    • Pytorch, Python
    • Github, Docker
  • Machine learning
  • Physics simulation

Automatic game playtest using deep reinforcement learning

You will study and develop a deep reinforcement learning algorithm for automatic game playtest. Concretely, we aim at developing an algorithm that is efficient and generalizes over different games, leveraging existing movies of game bugs and glitches.

  • Required skills and experience
    • Basic knowledge on reinforcement learning
    • Experience of implementing a deep learning algorithm with deep learning libraries such as PyTorch and Tensorflow
  • Preferred skills and experience
    • Interest on games
    • Experience of implementing deep reinforcement learning algorithms
    • Research experience in STEM (any field)
    • Experience of receiving an award from an academic society or a scholarship
  • Machine learning
  • Reinforcement learning
  • Game

Unsupervised manipulation primitive learning

We will develop a method for acquiring general robotic manipulation skills without reward function or demonstration.

  • Required skills and experience
    • Research experience using deep learning
    • Research experience in robot learning
  • Preferred skills and experience
    • Publication record in human-computer interactions (ICRA, IROS, CoRL, ICML, NeurIPS, ICLR, etc.)
    • Management of projects and code using git/GitLab/GitHub
    • Experience of coding in ROS, Python, or C++ in the development of a robot
    • Experience in team-based development
    • Knowledge related to robotics (e.g., control, optimization, HCI, computer vision, etc.)
  • Machine learning
  • Robotics
  • Manipulation

AI for Science

You will work on AI research that accelerates and automates research and development itself. You will participate in partial projects in the realization of AI scientists who can formulate research claims, run experiments, analyze the results, and write papers in an interactive co-evolution with human researchers.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing, computer vision, and data science
    • Mathematical knowledge and formulation ability in machine learning and deep learning
  • Machine learning
  • Interaction

Procedure understanding using cooking videos

Vision and Language technologies are extending their target from still images to movies, and their scope from simple moves to complex operations, such as cooking and manufacturing. You will be engaged in Vision and Language tasks that promote understanding of manuals and operations using videos of humans preparing food based on recipes. This project is a joint research with Cookpad Inc.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing, and computer vision
    • Mathematical knowledge and formulation ability in machine learning and deep learning
  • Natural language processing
  • Multimodal understanding
  • Computer vision

Linguistic instruction concretization for robots

This theme aims to estimate an action sequence for robots to complete a linguistic direction. This is a part of a project that seeks to generate robot motion based on linguistic direction and visual observation. The achievement will be unified with those from some other themes and submitted to robotics conferences/journals.

  • Required skills and experience
    • Knowledge and experience in deep learning
    • Basic knowledge in natural language processing
  • Preferred skills and experience
    • Experience in public dataset design and distribution
    • Experience in study and development of machine translation or captioning task.
  • Natural language processing

Representation learning for state transition

Foundation models, such as CLIP, have been developed and used in various applications. CLIP is, however, trained with the pairs of sentence and still image; thus, it does not seem well optimized for state transition in an environment. In this theme, we consider extensions to CLIP that can represent differences in environmental change. We aim to publish the achievement in a related conference or journal.

  • Required skills and experience
    • Knowledge and experience in deep learning
    • Basic knowledge in either computer vision or natural language processing
  • Preferred skills and experience
    • Experience in study and development with CLIP, GPT or any other foundation models
    • Experience in public dataset design and distribution
    • Basic knowledge in both of computer vision and natural language processing
  • Natural language processing
  • Computer vision

Point Cloud Registration

Development of point cloud deep learning and the availability of sensors and datasets, research on comparing and mapping multiple point clouds using deep learning (rigid and non-rigid registration and flow estimation) is developing. The aim is to apply trainable matching of point clouds to point cloud alignment with good use of geometry and spatial continuity.

  • Required skills and experience
    • Python
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in 3D data processing.
    • Knowledge and experience in irregular data (e.g., point clouds, graph data).
  • Computer vision
  • Point Cloud Matching
  • Point Cloud Deep Learning

GPU-accelerated Machine Learning at Billion Scale

GPU-accelerated machine learning is relatively underexplored despite its usefulness.
This project explores to derive massively parallel machine learning algorithm scalable for billions of data points considering its theory and SIMT (Single Instruction Multiple Threads) programming model.
Accepted interns are expected to work in collaboration with the mentors to submit research results to international conferences in machine learning and open-source achievement as a library.

  • Required skills and experience
    • Programming Experiences using GPUs
    • Programming Experiences with C++/CUDA
    • Fundamentals of Matrix Computations
    • Research and development experiences in machine learning
  • Preferred skills and experience
    • Advanced experiences in Parallel Programming
    • Advanced expertise in CUDA programming
    • Programming experiences in Cython/pybind/nanobind
    • Advanced expertise of performance analysis using GPU profilers (nvprof, nvvp, NSight systems)
    • Publication recors in the field of machine learning (ML) and relevant fields (CV,RO,HPC).
  • Machine learning
  • GPGPU
  • HPC

Operation Interface/Interaction that Embodies Robots

This project aims to research and develop an interface/interaction for the intuitive operation of the home and industrial robots as if they were part of the human body. We will develop simulators for virtual environments and apply the simulation results to real environments and robots. Part of the results of this project is expected to be submitted to a top-tier conference in human-computer interaction, such as CHI and UIST.

  • Required skills and experience
    • Research experience in human-computer interaction (HCI), human-robot interaction (HRI), or virtual reality (VR)
    • Knowledge of embodiment (embodiment, body ownership, and agency) in the field of psychology
    • Skills and experience in developing VR systems with Unity
  • Preferred skills and experience
    • Experience in writing papers and conference presentations in the fields of HCI, HRI, and VR (e.g., CHI, UIST, HRI, IMWUT, TVCG, ICRA, IROS)
    • Robot programming skills and experience
    • Knowledge and experience in designing and conducting user studies
    • Basic knowledge of statistical analysis
    • Experience in robot hardware and software development
    • Electronic construction and fabrication skills
  • Machine learning
  • Robotics
  • Interaction
  • Virtual Reality

Can you feel it? Research on virtual chatbots that acquire human profiles using Good and Bad

The objective function used in current dialogue systems learns to represent statistically plausible responses. Therefore, even if the response is statistically correct, it is logically incorrect. In this study, We research and develop prototypes of virtual chatbots that allow chatbots to understand users by making the chatbot corrects incorrect responses by marking good for correct answers and bad for incorrect answers.

  • Required skills and experience
    • Pytorch
    • Transformer
    • Reinforcement
    • NLP
  • Interaction
  • NLP
  • Reinforcement learning

Development of functionality to allow virtual chatbots to recognize user facial expressions

People determine their own behavior by obtaining as rewards the facial expressions, voices, and words expressed by others.
We will develop and research a function to estimate emotions from the avatar's facial expressions.
The main task is development, not research.
However, if you want, you can research as an extension of this project.

  • Required skills and experience
    • Coding in python and OpenCV
    • Knowledge and experience in deep learning using pytorch
    • Knowledge and experience in git, docker and vscode
  • Image Processing

Development of avatar creation system

We will develop a system that allows you to easily create your own preferred avatar and chat with it, or turn it into a virtual chatbot that is your partner.
The main task is development, not research.

  • Required skills and experience
    • Experience in web system using AngularJS
  • Web Application

Estimation of flow field by reinforcement learning using blood flow measurement and fluid simulation

In this study, We attempt to create a model that estimates the fluid field by learning an inference model using reinforcement learning to reduce the error between the measurement(environment) and the calculation.

  • Required skills and experience
    • Programming skill in Pytorch and Fortran
    • Fluid simulation using Finite volume method
    • Reinforcement learning
  • Fluid simulation
  • Reinformecement learning
  • finite volume method

Conditions

Term: From three months. Start and end dates can be adjusted. Some projects accept short-term interns within a month.
Hours: Full time or part time (3 days a week, etc. negotiable), 45 minute breaks, weekends and holidays off.
Location: At Hongo office or from home (determined flexibly based on social situations and request). Full remote work from home or other remote locations is possible upon request. We will ensure your health and safety when you come to work in our office.
Salary: Full-time monthly salary of 240,000 yen to 480,000 JPY. Hourly rate is applied for part time. Social insurance and other benefits are provided according to working conditions. Transportation and accommodation expenses are fully covered. Lending of a personal computer and full support for other expenses necessary for research activities.
Language: Japanese or English (English-only communication is also fine.)
Others: Two or more mentors with extensive research experience will provide in-depth support for each project. Computational resources (DGX, other GPU workstations, ABCI) and robotic facilities (robot arms, various sensors, 3D printers, motion capture systems, other prototyping and experimental equipment) are available.

How to apply

Please contact internships@sinicx.com with (1) your CV and (2) the topic(s) you would like to work from the list above. We will first screen each application based on those information.

Those who pass the above screening will be interviewed remotely. Please prepare slides or other materials to introduce past research and development activities and achievements.

Please contact us at least three months in advance if you need a visa to enter Japan.