- Job Type: Full-Time
- Function: Data Science
- Industry: AI
- Post Date: 10/14/2021
- Website: indigo.ai
- Company Address: Via Torino, 61, Milano, Lombardia , IT, 20123
About IndigoIndigo.ai is a full-service conversational AI studio, busy designing and building beautiful chatbots, language technologies and conversational experiences.
Indigo.ai is on a mission to enhance communication between people and businesses. We developed an artificial intelligence platform that uses machine learning to automate conversations between businesses and customers via chat.
About the Ai Team
In the Ai team we are the R&D soul of the company. We are engineers and mathematicians with experience in Machine Learning and a deep understanding of Natural Language Processing. We try to investigate every problem starting from the theoretical and modeling base, reaching the development and production deployment in a lean and effective way.
In the last 4 years NLP faced a revolution represented by the introduction of the Transformer architecture, a network designed to process sequential inputs using attention mechanisms. Among NLP models adopting the Transformer architecture, BERT affirmed as a groundbreaking and flexible alternative. Thanks to its two-steps training, this network demonstrated being able to reach high quality performances in any NLP task.
A limit that affects Language Models consists in the fact that they learn using textual information exclusively. If on one hand humans gather information from the external world using vision, audio signals and so on, on the other NLP models are only able to gather information by “reading”. An attempt to break this paradigm is represented by the Vokenization approach, published last year by two researchers of Chapel Hill university. The idea of Vokenization is to pre-train a BERT model to predict both correct words in a sentence and correct images associated with words. For this reason, this visually-supervised BERT network can be considered a multimodal model gathering information both from text and images.
With this project we want to assess if the visually-supervised training proposed with Vokenization is a valid approach to bridge the gap between human and artificial intelligence. In particular we want to explore the sphere of commonsense reasoning tasks, a field in which classical Language Models have trouble and that we believe can obtain sensible improvements thanks to visual supervision.
Here is a set of possible experiment in the field of commonsense reasoning that are in target for the objective of this project:
- Physical commonsense reasoning (Forbes et al., 2019): analysis of the capabilities of a model to assess relationships between objects (ex. a house is bigger than a person)
- Commonsense reasoning on object attributes (Da and Kasai, 2019): analysis of capabilities of models to assign right attributes to objects (ex. zebra, is an animal)
- Diagnostics from human language experiments (Ettinger, 2020): probing language models on psycholinguistic tests
We underline that the experiments listed above are just a proposal, and the optimal set of experiments and tests to conduct will be defined during the internship period as part of the project.
The skills we believe a successful candidate should have are:
- Knowledge of algorithms and data structure, Object Oriented Programming
- Python (or C++ and will to learn Python)
- Mathematical and Statistical knowledge
- Understanding the theory behind main machine learning algorithms
- Experience in a research project
- PLUS: Deep Learning experience
- PLUS: Experience in the Natural Language Processing field
- Duration: 6-9 months
- Location: fully remote (with possible brainstorming sessions in Milan according to pandemic rules)
- Contract: full-time or part-time curricular internship with relative remuneration
- University: affiliation to a specific university is not required. For this reason, we leave it to the candidate to find a Professor who acts as an internal referent for the graduation.