Designer-centered reinforcement learning
In video games, nonplayer characters, bots, and other game agents help bring a digital world and its story to life. They can help make the mission of saving humanity feel urgent,
Read moreCategory: Artificial intelligence
Speller100: Zero-shot spelling correction at scale for 100-plus languages
At Microsoft Bing, our mission is to delight users everywhere with the best search experience. We serve a diverse set of customers all over the planet who issue queries in over 100 languages. In search we’ve found about 15% of queries submitted by customers have misspellings. When queries are misspelled, we match the wrong set of documents and trigger incorrect answers, which can produce a suboptimal results page for our customers. Therefore, spelling correction is the very first component in […]
Read moreResearch Collection – Shall we play a game?
From a research point of view, games offer an amazing environment in which to develop new machine learning algorithms and techniques. And we hope, in due course, that those new algorithms will feed back not just into gaming, but into many other domains. Beyond the very technical machine learning techniques themselves, gaming is an environment in which we can explore the relationship between AI and people, and see how they can work in partnership. It’s a very rich environment in […]
Read moreThree mysteries in deep learning: Ensemble, knowledge distillation, and self-distillation
Under now-standard techniques, such as over-parameterization, batch-normalization, and adding residual links, “modern age” neural network training—at least for image classification tasks and many others—is usually quite stable. Using standard neural network architectures and training algorithms (typically SGD with momentum), the learned models perform consistently well, not only in terms of training accuracy but even in test accuracy, regardless of which random initialization or random data order is used during the training. For instance, if one trains the same WideResNet-28-10 architecture […]
Read moreVinVL: Advancing the state of the art for vision-language models
Humans understand the world by perceiving and fusing information from multiple channels, such as images viewed by the eyes, voices heard by the ears, and other forms of sensory input. One of the core aspirations in AI is to develop algorithms that endow computers with a similar ability: to effectively learn from multimodal data like vision-language to make sense of the world around us. For example, vision-language (VL) systems allow searching the relevant images for a text query (or vice […]
Read moreMicrosoft DeBERTa surpasses human performance on the SuperGLUE benchmark
Natural language understanding (NLU) is one of the longest running goals in AI, and SuperGLUE is currently among the most challenging benchmarks for evaluating NLU models. The benchmark consists of a wide range of NLU tasks, including question answering, natural language inference, co-reference resolution, word sense disambiguation, and others. Take the causal reasoning task (COPA in Figure 1) as an example. Given the premise “the child became immune to the disease” and the question “what’s the cause for this?,” the […]
Read moreResearch at Microsoft 2020: Addressing the present while looking to the future
Microsoft researchers pursue the big questions about what the world will be like in the future and the role technology will play. Not only do they take on the responsibility of exploring the long-term vision of their research, but they must also be ready to react to the immediate needs of the present. This year in particular, they were asked to use their roles as futurists to address pressing societal challenges. In early 2020, as countries began responding to COVID-19 […]
Read more‘Seeing’ on tiny battery-powered microcontrollers with RNNPool
Computer vision has rapidly evolved over the past decade, allowing for such applications as Seeing AI, a camera app that describes aloud a person’s surroundings, helping those who are blind or have low vision; systems that can detect whether a product, such as a computer chip or article of clothing, has been assembled correctly, improving quality control; and services that can convert information from hard-copy documents into a digital format, making it easier to manage personal and business data. All […]
Read moreMPNet combines strengths of masked and permuted language modeling for language understanding
Pretrained language models have been a hot research topic in natural language processing. These models, such as BERT, are usually pretrained on large-scale language corpora with carefully designed pretraining objectives and then fine-tuned on downstream tasks to boost the accuracy. Among these, masked language modeling (MLM), adopted in BERT, and permuted language modeling (PLM), adopted in XLNet, are two representative pretraining objectives. However, both of them enjoy their own advantages but suffer from limitations. Therefore, researchers from Microsoft Research Asia, […]
Read more