No results

Study on leveraging machine learning in path planning algorithms

We will study integrated approaches to path planning and deep learning, such as Neural A* (ICML-21) and CTRMs (AAMAS-22). Particular focus will be placed on multi-agent path planning and on path planning for dynamic and uncontrolled environments. Accepted interns are expected to work in collaboration with the mentors to submit research results to top international conferences in the field of artificial intelligence and machine learning.

  • Required skills and experience

    At least one of the following:

    • Research and development experiences using deep learning
    • Research and development experiences on path planning or motion planning
  • Preferred skills and experience
    • Research and development experiences on multi-agent path planning or multi-agent reinforcement learning
    • Publication record in the field of artificial intelligence (e.g., AAAI, IJCAI, AAMAS, ICML, NeurIPS, ICLR)
    • Experience participating in competitions related to machine learning
    • Expert knowledge in related areas (e.g., optimization, control, HCI, graph algorithms, computer vision, computer graphics, and software development)
  • Machine learning
  • Path planning
  • Multi-agent systems

Study on local planning algorithms for multi-agent and dynamic environments

We study local planning algorithms for multi-agent environments that require the cooperative movement of multiple robots, or dynamic environments with pedestrians and moving objects. In particular, we will work on reinforcement learning-based approaches for crowded environments such as L2B (IROS-20), transfer learning-based approaches such as MULTIPOLAR (IJCAI-20), and other studies that actively utilize machine learning. Other tasks include theoretical studies and library development for sim-to-real transfer in the context of mobile robot motion planning. Accepted interns are expected to work in collaboration with the mentors to submit research results to top international conferences in the field of artificial intelligence and robotics.

  • Required skills and experience

    At least one of the following:

    • Research and development experiences using deep learning
    • Research and development experiences on path planning or motion planning
  • Preferred skills and experience
    • Experience developing distributed systems using ROS/ROSpy and experience using the ROS Navigation Stack
    • Research and implementation experience in transfer learning, domain adaptation, and meta-learning
    • Competition or publication records in the field related to autonomous mobile robots
    • Expert knowledge in related fields (e.g., optimization, control, HCI, multi-agent systems, graph algorithms, computer vision, computer graphics, software development)
    • Research and implementation experience using simulators such as PyBullet and Unity ML Agents
    • Publication records in the field of artificial intelligence and robotics (ICML, NeurIPS, ICLR, AAAI, IJCAI, AAMAS, ICRA, IROS, CoRL, RSS, CVPR, ICCV, ECCV, etc.)
  • Machine learning
  • Robotics
  • Multi-agent systems
  • Path planning
  • Reinforcement learning
  • Transfer learning/domain adaptation

Robotic learning for soft robots or deformable and fragile object manipulation

We develop robotic learning systems for soft robots or deformable and fragile object manipulation and aim to apply them to industrial assembly or cooking.
In this project, we would like accepted interns to develop robotic learning systems, manipulation with multi-modal sensing, and motion planning. Besides, we have another project to develop a simulator for soft robots. We aim to submit top-tier robotic conference or journal papers, e.g., ICRA, IROS, RA-L. We are also actively hiring interns with experience in machine learning or reinforcement learning developments and a strong interest in robotic applications.

  • Required skills and experience
    • Experience in Python or C++

    At least one of the following:

    • Research and development experiences in robot learning, sensing, control theory, or motion planning
    • Research and development experiences using machine learning or reinforcement learning
  • Preferred skills and experience
    • Research and development experience in ROS
    • Publication records in the field of robotics and artificial intelligence
    • Experience in participation in robot competitions
  • Machine learning
  • Robotics
  • Manipulation
  • Reinforcement learning
  • Task & Motion planning

Robot software or hardware engineering

We would like accepted interns to be involved in robot software or hardware engineering, such as programming, designing robot environments to launch robot systems, or hardware maintenance.
We also highly encourage interns to publish system papers or OSS libraries as achievements during this internship.

  • Required skills and experience

    At least one of the following:

    • Research or development experience in ROS
    • Research or development experience in robot hardware design
    • Research or development experience in sesing design
  • Preferred skills and experience
    • Experience in participation in robot competitions
  • Machine learning
  • Robotics
  • Manipulation
  • Robot system integration
  • Hardware design
  • OSS development

A mechanism and system of a high-speed soft robot

We will develop a mechanism or system of a high-speed soft robot. In particular, we will develop a computational design of mechanisms, such as cable-driven systems, linkage systems, and cam mechanisms and additive manufacturing, such as 3D printing. We will also develop a robotic system that quickly responds to an object and moves fast, such as state estimation and prediction of a fast-moving object and motion planning in a short time.

  • Required skills and experience

    Experience in robotic mechanism design or robotic system design

  • Preferred skills and experience
    • Robot competition
    • Experience in team-based development
    • Publication in robotics (IROS, ICRA, etc.)
    • Experience of receiving an award from an academic society or a scholarship
    • Experience of coding in ROS, Python, or C++ in the development of a robot
  • Robotics
  • Mechanics design
  • Robotic system development

Video analysis of a ball game using machine learning

We are developing a technology to observe the behavior of humans and estimate their intention and desire. In this project, we will apply machine learning techniques to video analysis of a ball game, such as table tennis, extract the actions of players, and analyze the mechanism of decision making and interactions of multiple players.

  • Required skills and experience

    Research experience using deep learning

  • Preferred skills and experience
    • Experience of winning a prize of a machine learning competition such as Kaggle
    • Experience of research using Git/Github and Docker
    • Experience of receiving an award from an academic society or a scholarship
    • Publication in artificial intelligence (CVPR, ICCV, ECCV, NeurIPS, ICML, ICLR, TPAMI, etc.)
  • Machine learning
  • Robotics
  • Computer vision & graphics
  • Human sensing
  • Sports robot

Action generation of a robot to entertain humans in a game

We will develop a method for generating actions of a robot to entertain humans in a game, such as table tennis and air hockey.

  • Required skills and experience

    (Any of the following)

    • Research experience in HAI, HRI, or HCI
    • Research experience using deep learning
    • Research experience in robotics
    • Robot development
  • Preferred skills and experience
    • Publication record in human-computer interactions (HAI, HRI, RO-MAN, CHI, SIGGRAPH, etc.)
    • Management of projects and code using git/GitLab/GitHub
    • Experience of coding in ROS, Python, or C++ in the development of a robot
    • Experience in team-based development
    • Knowledge related to robotics (e.g., control, optimization, HCI, computer vision, etc.)
  • Robotics
  • Interaction
  • Robotic system development
  • Sports robot

Learning dynamic nonprehensile manipulation

We will develop a method for learning dynamic nonprehensile manipulation, such as dynamic re-orientation of a grasped object, hitting an object, and catching a flying object. A robot utilizes information from sensors for learning, such as a camera, a force sensor, and a tactile sensor.

  • Required skills and experience

    Research and development experiences using deep learning

  • Preferred skills and experience
    • Knowledge related to robotics (e.g., control, optimization, HCI, computer vision, etc.).
    • Publication record in robotics or artificial intelligence (IROS, ICRA, ICML, NeurIPS, ICLR, etc.)
    • Experience of receiving an award from an academic society or a scholarship
    • Robot competition or a machine learning competition
    • Management of projects and code using git/GitLab/GitHub
  • Machine learning
  • Robotics
  • Manipulation
  • Multi-modal

Enhancing remote communication via physical avator robots

Removing the restriction in the physical workplace may resolve many social problems. However, remote work is still not a widely-accepted working style even under the current social situation. This project tackles this problem through robot-mediated human-to-human communication. Interns are expected to work with the mentors to submit a paper for an international conference in HCI/HRI.

  • Required skills and experience
    • Experience in user studies for dialog analysis
    • Experience in participating any full-remote projects
  • Preferred skills and experience
    • Experience in JavaScript / Python Flask implementation
  • Interaction
  • Human sensing

Multi-modal eyes image prediction

We will develop research on multimodal eye estimation and prediction methods and write papers for EMBC and other international conference in biomedical engineering.

  • Required skills and experience
    • Knowledge and experience in deep learning
    • Knowledge and experience in git, docker and vscode
  • Preferred skills and experience
    • Knowledge and experience in Vision&Language or multimodal analysis
    • Medical Knowledge about eyes and genes
  • Signal processing
  • Multi-modal
  • Human sensing
  • Biological signal

Effect analysis of real-time information visualization on web meeting

For a multi-person web meeting held at Zoom etc., we will design and carry out experiments using tools that visualize real-time information (amount of conversation, facial expressions, etc.) related to communication. In addition, the experiment will verify the effect of facilitating exchanges and changes between conference participants. The research results will be submitted to international conferences in the field of interaction such as CHI.

  • Required skills and experience
    • Interested in human-computer interaction
  • Preferred skills and experience
    • Experience in conducting user-participatory experiments such as dialogue analysis
    • Experience in paper submission to CHI
  • Interaction
  • Dialog analysis

Research on improving machine learning performance by improving image datasets

According to the concept of "data centric AI", improving the quality of datasets, such as label consistency, often contributes significantly to improving the performance of machine learning models. This research aims to verify the effectiveness of such a data-centric approach in the field of computer vision and to publish paper in a related international conference.

  • Required skills and experience
    • Python
    • Knowledge and experience in deep learning and image processing
  • Preferred skills and experience
    • Experience in implementing and experimenting with object detection/segmentation models on large-scale datasets
  • Machine learning
  • Computer vision & graphics
  • Dataset

Research on fusion understanding of image/video and natural language

While there are tons of research on machine learning for understanding images and natural language, deep learning has led to the commoditization of modules from each other, and research combining multiple modalities is also increasing. In this project, we will develop research on fusion understanding of image/video and natural language and write papers for relevant top international conferences.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Knowledge and experience in computer vision
  • Computer vision & graphics
  • Natural language processing
  • Multi-modal
  • Vision and language

Research on learnable discrete information processing

Research to modify specific computational modules by making them machine-learnable is underway (e.g., differentiable rendering) to include them in a deep learning pipeline. This project will develop research on making discrete information processing learnable and writing papers for international machine learning conferences, such as ICLR, ICML, and NeurIPS.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Mathematical knowledge and ability to formulate methods for machine learning and deep learning
  • Machine learning
  • Optimization

Research on high-dimensional black box optimization

Black-box optimization, such as Bayesian optimization, is subject to computational overheads as the number of parameters to be optimized increases. In this project, we will develop research on high-dimensional black-box optimization and write papers for top international conferences in machine learning or journals.

  • Required skills and experience
    • Python, Github, Docker
  • Preferred skills and experience
    • Knowledge and experience with black box optimization such as Bayesian optimization
  • Machine learning
  • Blackbox optimization
  • Bayesian optimization

Representation learning for structured data

Pre-training models based on supervised/self-supervised learning using large amounts of data are widely used in machine learning for natural language and images. The concept of foundation models is gaining ground. In this project, we will conduct research and development on representation learning of data with unique structures other than images and natural language and write a paper aiming at a journal such as Nature and Science families.

  • Required skills and experience
    • Python, Github, Docker
  • Preferred skills and experience
    • Knowledge and experience in representation learning with images/natural language.
    • Knowledge and experience in machine learning with point clouds/graphs.
    • Mathematical knowledge of machine learning and deep learning and ability to formulate equations
  • Machine learning
  • Representation learning
  • Point cloud processing
  • Graph processing

Research on law discovery from observed data

Research on causal analysis on time-series data and explanatory AI is progressing to make some predictions while clarifying the laws between data. In this project, we will conduct research and development on methods from a new angle for discovering such laws and write papers aiming for acceptance in top international conferences in machine learning or journals. The internship is mainly aimed at Ph.D. students and is expected to last at least three months.

  • Required skills and experience
    • Python, Github, Docker
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Knowledge and experience in signal processing
  • Machine learning
  • Data mining

Research on data2text

Machine translation is a technology that converts one input text into another. There is also growing research on data2text, which generates captions and news headlines when the input is data such as images or economic data. This project will develop research on data2text for table data and a particular form of texts and write papers aiming for acceptance in top international conferences in the machine learning field or journals.

  • Required skills and experience
    • Python, Github, Docker
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Knowledge and experience in signal processing
  • Machine learning
  • Data mining

Research on human-in-the-loop machine learning

Machine learning research that incorporates humans in machine learning and makes efficient use of feedback from humans is expanding. In this project, we will conduct research and development on human-in-the-loop machine learning research and write papers aiming for publication in top international conferences in machine learning and interaction or journals.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in human-computer interaction
  • Machine learning
  • Interaction
  • Human-in-the-loop

Multimodal understanding of specialized documents

Understanding technical documents such as papers and patents require understanding data, including structured text and diagrams. It requires efforts that go beyond the framework of conventional natural language processing. In this project, we will conduct research and development on a multimodal understanding of such specialized documents and write papers for international conferences in related fields.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Mathematical knowledge and formulation ability in machine learning and deep learning
  • Machine learning
  • Computer vision & graphics
  • Natural language processing
  • Multi-modal
  • Document understanding

Research on machine learning for few training data

Target tasks can be achieved in modalities or domains where a pre-trained foundation model exists by finetuning it on a small number of data. On the other hand, more advanced machine learning techniques are required for tasks not in such modalities or domains. In this project, we will conduct research and development on such small-data machine learning and write a paper aiming at top international conferences in machine learning and computer vision.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in computer vision
  • Machine learning
  • Computer vision & graphics
  • Transfer learning/domain adaptation

Machine learning applications to multimodal biological data

The approach for biological data processing is shifting from inference based on the understanding of either time-series data or medical images in isolation to the fusion of multimodal data with different attributes and structures. In this project, we will conduct research and development on the application of machine learning to such multimodal biological data and write a paper for an international conference in the related field. We welcome applications from undergraduate students.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing
    • Knowledge and experience in computer vision
    • Knowledge and experience in signal processing
  • Computer vision & graphics
  • Signal processing
  • Multi-modal
  • Human sensing
  • Biological signal

Radiance Field Optimization

Neural Radiance Field (NeRF) is widely used for general volumetric scene representation. In this project, we research the modeling of radiance fields and the optimization method of the model and write papers for publication in top international conferences in the field of computer vision.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in 3D deep learning
    • Knowledge and experience in munerical optimization
  • Computer vision & graphics
  • Numerical optimization

Research on machine learning of physics simulation

We will conduct research and development on the application of machine learning to physics simulations and write papers in related fields (Nature/Science, Physical Review, SC/HPCG, etc.). We will solve various problems of physics simulation, especially multi-scale properties from the quantum and atomic level to the earth and space level using machine learning.

  • Required skills and experience
    • Knowledge and experience in either deep learning or physics simulation
  • Preferred skills and experience
    • Pytorch, Python
    • Github, Docker
  • Machine learning
  • Physics simulation

Generate diverse motions grounded to language directions for robots

In conventional robot programming, a programmer needs to calibrate robot motions one by one for their environment. This project aims to develop a group of technologies to generate robot motions from directions written in natural language and observation of the environment around robots. Our initial focus is a methodology to transfer human activity to robots by extracting primitive actions. Interns are expected to work with the mentors to submit a paper for a top-tier conference in computer vision or robotics.

  • Required skills and experience
    • Knowledge and experience in deep learning
    • Basic knowledge in either computer vision or natural language processing
  • Preferred skills and experience
    • Experience in action recognition/detection with deep learning
    • Experience in robot control with ROS
  • Robotics
  • Computer vision & graphics
  • Natural language processing
  • Human sensing

ECG classification using deep learning

We will conduct research and development on ECG classification using deep learning and write a paper for related international conferences.

  • Required skills and experience
    • Knowledge and experience in deep learning
    • Knowledge and experience in git, docker and vscode
  • Preferred skills and experience
    • Knowledge and experience in ECG
  • Machine learning
  • Signal processing
  • ECG analysis
  • Biological signal

Development of avatar dataset collection system and collection of avatar data and its evaluation

We will develop an avatar dataset collection system, collect avatar data, evaluate those data, and write a paper for those achievements.

  • Required skills and experience
    • Knowledge and experience in character design
  • Preferred skills and experience
    • Knowledge and experience in deep learning
  • Computer vision & graphics
  • Natural language processing
  • Dataset

Automatic game playtest using deep reinforcement learning

You will study and develop a deep reinforcement learning algorithm for automatic game playtest. Concretely, we aim at developing an algorithm that is efficient and generalizes over different games, leveraging existing movies of game bugs and glitches.

  • Required skills and experience
    • Basic knowledge on reinforcement learning
    • Experience of implementing a deep learning algorithm with deep learning libraries such as PyTorch and Tensorflow
  • Preferred skills and experience
    • Interest on games
    • Experience of implementing deep reinforcement learning algorithms
    • Research experience in STEM (any field)
    • Experience of receiving an award from an academic society or a scholarship
  • Machine learning
  • Reinforcement learning
  • Game

Unsupervised manipulation primitive learning

We will develop a method for acquiring general robotic manipulation skills without reward function or demonstration.

  • Required skills and experience
    • Research experience using deep learning
    • Research experience in robot learning
  • Preferred skills and experience
    • Publication record in human-computer interactions (ICRA, IROS, CoRL, ICML, NeurIPS, ICLR, etc.)
    • Management of projects and code using git/GitLab/GitHub
    • Experience of coding in ROS, Python, or C++ in the development of a robot
    • Experience in team-based development
    • Knowledge related to robotics (e.g., control, optimization, HCI, computer vision, etc.)
  • Machine learning
  • Robotics
  • Manipulation

AI for Science

You will work on AI research that accelerates and automates research and development itself. You will participate in partial projects in the realization of AI scientists who can formulate research claims, run experiments, analyze the results, and write papers in an interactive co-evolution with human researchers.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing, computer vision, and data science
    • Mathematical knowledge and formulation ability in machine learning and deep learning
  • Machine learning
  • Interaction

Procedure understanding using cooking videos

Vision and Language technologies are extending their target from still images to movies, and their scope from simple moves to complex operations, such as cooking and manufacturing. You will be engaged in Vision and Language tasks that promote understanding of manuals and operations using videos of humans preparing food based on recipes. This project is a joint research with Cookpad Inc.

  • Required skills and experience
    • Python, Github, Docker
    • Knowledge and experience in deep learning
  • Preferred skills and experience
    • Knowledge and experience in natural language processing, and computer vision
    • Mathematical knowledge and formulation ability in machine learning and deep learning
  • Natural language processing
  • Multimodal understanding
  • Computer vision

Conditions

Term: From three months. Start and end dates can be adjusted. Some projects accept short-term interns within a month.
Hours: Full time or part time (3 days a week, etc. negotiable), 45 minute breaks, weekends and holidays off.
Location: At Hongo office or from home (determined flexibly based on social situations and request). Full remote work from home or other remote locations is possible upon request. We will ensure your health and safety when you come to work in our office.
Salary: Full-time monthly salary of 240,000 yen to 480,000 JPY. Hourly rate is applied for part time. Social insurance and other benefits are provided according to working conditions. Transportation and accommodation expenses are fully covered. Lending of a personal computer and full support for other expenses necessary for research activities.
Language: Japanese or English (English-only communication is also fine.)
Others: Two or more mentors with extensive research experience will provide in-depth support for each project. Computational resources (DGX, other GPU workstations, ABCI) and robotic facilities (robot arms, various sensors, 3D printers, motion capture systems, other prototyping and experimental equipment) are available.

How to apply

Please contact internships@sinicx.com with (1) your CV and (2) the topic(s) you would like to work from the list above. We will first screen each application based on those information.

Those who pass the above screening will be interviewed remotely. Please prepare slides or other materials to introduce past research and development activities and achievements.

Please contact us at least three months in advance if you need a visa to enter Japan.