Natural Language Processing

A recurrent unit that can run over 10 times faster than cuDNN LSTM

sru SRU is a recurrent unit that can run over 10 times faster than cuDNN LSTM, without loss of accuracy tested on many tasks. Average processing time of LSTM, conv2d and SRU, tested on GTX 1070 For example, the figure above presents the processing time of a single mini-batch of 32 samples. SRU achieves 10 to 16 times speed-up compared to LSTM, and operates as fast as (or faster than) word-level convolution using conv2d. Reference: Simple Recurrent Units for Highly […]

Read more
PyTorch

Model summary in PyTorch similar to model.summary() in Keras

Keras style model.summary() in PyTorch Keras has a neat API to view the visualization of the model which is very helpful while debugging your network. Here is a barebone code to try and mimic the same in PyTorch. The aim is to provide information complementary to, what is not provided by print(your_model) in PyTorch. Usage pip install torchsummary or git clone https://github.com/sksq96/pytorch-summary from torchsummary import summary summary(your_model, input_size=(channels, H, W)) Note that the input_size is required to make a forward […]

Read more
PyTorch

A collection of optimizers for PyTorch compatible with optim module

torch-optimizer torch-optimizer — collection of optimizers for PyTorch compatible with optim module. Simple example import torch_optimizer as optim # model = … optimizer = optim.DiffGrad(model.parameters(), lr=0.001) optimizer.step() Installation Installation process is simple, just: $ pip install torch_optimizer Documentation https://pytorch-optimizer.rtfd.io GitHub https://github.com/jettify/pytorch-optimizer    

Read more
PyTorch

A PyTorch implementation of EfficientNet and EfficientNetV2

EfficientNet PyTorch A PyTorch implementation of EfficientNet and EfficientNetV2 (coming soon!) Quickstart Install with pip install efficientnet_pytorch and load a pretrained EfficientNet with: from efficientnet_pytorch import EfficientNet model = EfficientNet.from_pretrained(‘efficientnet-b0’) Updates Update (April 2, 2021) The EfficientNetV2 paper has been released! I am working on implementing it as you read this 🙂 About EfficientNetV2: EfficientNetV2 is a new family of convolutional networks that have faster training speed and better parameter efficiency than previous models. To develop this family of models, […]

Read more
Deep Learning

PyTorch Implementation of Differentiable ODE Solvers

PyTorch Implementation of Differentiable ODE Solvers This library provides ordinary differential equation (ODE) solvers implemented in PyTorch. Backpropagation through ODE solutions is supported using the adjoint method for constant memory cost. For usage of ODE solvers in deep learning applications, see reference [1]. As the solvers are implemented in PyTorch, algorithms in this repository are fully supported to run on the GPU. Installation To install latest stable version: pip install torchdiffeq To install latest on GitHub: pip install git+https://github.com/rtqichen/torchdiffeq Examples […]

Read more
SVG

Conversor de arquivos svg para react-native utilizando python

svg-react-native-converter Conversor de arquivos svg para react-native utilizando python. 🚀 Technologies Technologies that I used to develop this application 💻 Getting started Requirements Clone the project and access the folder $ git clone https://github.com/cesarzxk/svg-react-native-converter.git Follow the steps below # For run the code(or double click): python ./main.py 🤔 How to contribute Make a fork of this repository # Fork using GitHub official command line # If you don’t have the GitHub CLI, use the web site to do that. $ […]

Read more
Facebook NLP Research
NLP

Getting to Production with Few-shot Natural Language Generation Models

July 29, 2021 By: Peyman Heidari, Arash Einolghozati, Shashank Jain, Soumya Batra, Lee Callender, Ankit Arun, Shawn Mei, Sonal Gupta, Pinar Donmez, Vikas Bhardwaj, Anuj Kumar, Michael White Abstract In this paper, we study the utilization of pretrained language models to enable few-shot Natural Language Generation (NLG) in task-oriented dialog systems. We introduce a system consisting of iterative self-training and an extensible mini-template framework that textualizes the structured input data into semi-natural text to fully take advantage of pre-trained language […]

Read more
Facebook NLP Research
NLP

Text-Free Image-to-Speech Synthesis Using Learned Segmental Units

August 2, 2021 By: Wei-Ning Hsu, David Harwath, Tyler Miller, Christopher Song, James Glass Abstract In this paper we present the first model for directly synthesizing fluent, natural-sounding spoken audio captions for images that does not require natural language text as an intermediate representation or source of supervision. Instead, we connect the image captioning module and the speech synthesis module with a set of discrete, sub-word speech units that are discovered with a self-supervised visual grounding task. We conduct experiments […]

Read more
Facebook NLP Research
NLP

SUPERB: Speech Understanding and PERformance Benchmark

August 30, 2021 By: Shu-wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Lai, Kushal Lakhotia, Yist Y. Lin, Andy T. Liu, Jiatong Shi, Xuankai Chang, Daniel Lin, Tzu-Hsien Huang, Wei-Cheng Tseng, Godic Lee, Darong Liu, Zili Huang, Annie Dong, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, Hung-yi Lee Abstract Using self-supervised learning methods to pre-train a network on large volumes of unlabeled data followed by fine-tuning for multiple downstream tasks has proven vital for advancing research in natural language representation learning. However, […]

Read more
Deep Learning

Deep Unsupervised 3D SfM Face Reconstruction Based on Massive Landmark Bundle Adjustment

This repository shows two tasks: Face landmark detection and Face 3D reconstruction, which is described in this paper: Deep Unsupervised 3D SfM Face Reconstruction Based on Massive Landmark Bundle Adjustment. Installation Clone the repository. install dependencies. pip install -r requirement.txt Running a pre-trained model Download landmark pre-trained model at GoogleDrive, and put it into FaceLandmark/model/ Run the test file python Facial_landmark.py Running a pre-trained model Download face 3D reconstruction pre-trained model at GoogleDrive, and put it into FaceReconstruction/checkpoints/ Run the […]

Read more
1 … 493 494 495 496 497 … 928