August 18, 2021 NLP

Don’t Sweep your Learning Rate under the Rug- A Closer Look at Cross-modal Transfer of Pretrained Transformers

July 23, 2021 By: Danielle Rothermel, Margaret Li, Tim Rocktäschel, Jakob Foerster Abstract Self-supervised pre-training of large-scale transformer models on text corpora followed by fine-tuning has achieved state-of-the-art on a number of natural language processing tasks. Recently, Lu et al. (2021) claimed that frozen pretrained transformers (FPTs) match or outperform training from scratch as well as unfrozen (fine-tuned) pretrained transformers in a set of transfer tasks to other modalities. In our work, we find that this result is, in fact, […]

August 18, 2021 NLP

Many-Speakers Single Channel Speech Separation with Optimal Permutation Training

Abstract Single channel speech separation has experienced great progress in the last few years. However, training neural speech separation for a large number of speakers (e.g., more than 10 speakers) is out of reach for the current methods, which rely on the Permutation Invariant Training (PIT). In this work, we present a permutation invariant training that employs the Hungarian algorithm in order to train with an O (C 3) time complexity, where C is the number of speakers, in comparison […]

August 18, 2021 Transformer

DeepViT: Towards Deeper Vision Transformer

DeepViT This repo is the official implementation of “DeepViT: Towards Deeper Vision Transformer”. The repo is based on the timm library (https://github.com/rwightman/pytorch-image-models) by Ross Wightman Deep Vision Transformer is initially described in arxiv, which observes the attention collapese phenomenon when training deep vision transformers: In this paper, we show that, unlike convolution neural networks (CNNs)that can be improved by stacking more convolutional layers, the performance of ViTs saturate fast when scaled to be deeper. More specifically, we empirically observe that […]

August 18, 2021 Asyncio

A Prometheus Python client library for asyncio-based applications

aioprometheus aioprometheus is a Prometheus Python client library for asyncio-based applications. It provides metrics collection and serving capabilities, supports multiple data formats and pushing metrics to a gateway. The project documentation can be found on ReadTheDocs. Install $ pip install aioprometheus A Prometheus Push Gateway client and ASGI service are also included, but their dependencies are not installed by default. You can install them alongside aioprometheus by running: $ pip install aioprometheus[aiohttp] Prometheus 2.0 removed support for the binary protocol, […]

August 18, 2021 Tutorial

A free, online learning platform to make quality education accessible for all

Oppia Oppia is an online learning tool that enables anyone to easily create and share interactive activities (called ‘explorations’). These activities simulate a one-on-one conversation with a tutor, making it possible for students to learn by doing while getting feedback. In addition to developing the Oppia platform, the team is also developing and piloting a set of free and effective lessons on basic mathematics. These lessons are targeted at learners who lack access to educational resources. Oppia is written using […]

August 18, 2021 Flask

Flask-Rebar combines flask, marshmallow, and swagger for robust REST services

Flask-Rebar Flask-Rebar combines flask, marshmallow, and swagger for robust REST services. Features Request and Response Validation – Flask-Rebar relies on schemas from the popular Marshmallow package to validate incoming requests and marshal outgoing responses. Automatic Swagger Generation – The same schemas used for validation and marshaling are used to automatically generate OpenAPI specifications (a.k.a. Swagger). This also means automatic documentation via Swagger UI. Error Handling – Uncaught exceptions from Flask-Rebar are converted to appropriate HTTP errors. Example from flask import […]

August 18, 2021 Deep Learning

High-Performance Large-Scale Image Recognition Without Normalization

NFNet Pytorch Implementation This repo contains pretrained NFNet models F0-F6 with high ImageNet accuracy from the paper High-Performance Large-Scale Image Recognition Without Normalization. The small models are as accurate as an EfficientNet-B7, but train 8.7 times faster. The large models set a new SOTA top-1 accuracy on ImageNet. NFNet F0 F1 F2 F3 F4 F5 F6+SAM Top-1 accuracy Brock et al. 83.6 84.7 85.1 85.7 85.9 86.0 86.5 Top-1 accuracy this implementation 82.82 84.63 84.90 85.46 85.66 85.62 TBD All […]

August 18, 2021 Segmentation

Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers

SETR – Pytorch Since the original paper (Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers.) has no official code,I implemented SETR-Progressive UPsampling(SETR-PUP) using pytorch. Original paper: Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers. Vit The Vit model is also implemented, and you can use it for image classification. Usage SETR from SETR.transformer_seg import SETRModel import torch if __name__ == “__main__”: net = SETRModel(patch_size=(32, 32), in_channels=3, out_channels=1, hidden_size=1024, num_hidden_layers=8, num_attention_heads=16, decode_features=[512, 256, 128, 64]) t1 = torch.rand(1, 3, […]

August 18, 2021 API

Python SDK for the Buycoins API

Buycoins Python Library This library provides easy access to the Buycoins API using the Python programming language. It provides all the feature of the API so that you don’t need to interact with the API directly. This library can be used with Python 3.6+ Installation You can install this package using pip: pip install –upgrade buycoins Introduction Primer The library is structured around the concept of a type, so everything is a type. All date quantities are specified as timestamps. […]

August 18, 2021 Graph

Graphical tool to make photo collage posters in python

PhotoCollage Graphical tool to make photo collage posters PhotoCollage allows you to create photo collage posters. It assembles the input photographs it is given to generate a big poster. Photos are automatically arranged to fill the whole poster, then you can change the final layout, dimensions, border or swap photos in the generated grid. Eventually the final poster image can be saved in any size. The algorithm generates random layouts that place photos while taking advantage of all free space. […]

« 1 … 523 524 525 526 527 … 943 »