Deep Learning, NLP, NMT, AI, ML
1cak – is Indonesian web that provide lot of fun. Endpoint Lol -> 10 Recent stored posts on databaseExample: https://onecak.azurewebsites.net/?lol Shuffle -> Select random posts from databaseExample: https://onecak.herokuapp.com/?shuffle=5 Info This app is subject of experiment, you may found some weird thing or even useless code.If so, please ignore or do PR.Thanks. GitHub https://github.com/dickymuliafiqri/onecak
Read more
Official repository for the paper Robust High-Resolution Video Matting with Temporal Guidance. RVM is specifically designed for robust human video matting. Unlike existing neural models that process frames as independent images, RVM uses a recurrent neural network to process videos with temporal memory. RVM can perform matting in real-time on any videos without additional inputs. It achieves 4K 76FPS and HD 104FPS on an Nvidia GTX 1080 Ti GPU. The project was developed at ByteDance Inc.
Read more
tinygrad For something in between a pytorch and a karpathy/micrograd This may not be the best deep learning framework, but it is a deep learning framework. Due to its extreme simplicity, it aims to be the easiest framework to add new accelerators to, with support for both inference and training. Support the simple basic ops, and you get SOTA vision extra/efficientnet.py and language extra/transformer.py models. We are working on support for the Apple Neural Engine. Eventually, we will build custom […]
Read more
See article Clarifying exceptions and visualizing tensor operations in deep learning code and TensorSensor implementation slides (PDF). One of the biggest challenges when writing code to implement deep learning networks, particularly for us newbies, is getting all of the tensor (matrix and vector) dimensions to line up properly. It’s really easy to lose track of tensor dimensionality in complicated expressions involving multiple tensors and tensor operations. Even when just feeding data into predefined Tensorflow network layers, we still need to […]
Read more
Performer – Pytorch An implementation of Performer, a linear attention-based transformer variant with a Fast Attention Via positive Orthogonal Random features approach (FAVOR+). Install $ pip install performer-pytorch Then you must run the following, if you plan on training an autoregressive model $ pip install -r requirements.txt Usage Performer Language Model import torch from performer_pytorch import PerformerLM model = PerformerLM( num_tokens = 20000, max_seq_len = 2048, # max sequence length dim = 512, # dimension depth = 12, # layers […]
Read more
TabNet : Attentive Interpretable Tabular Learning This is a pyTorch implementation of Tabnet (Arik, S. O., & Pfister, T. (2019). TabNet: Attentive Interpretable Tabular Learning. arXiv preprint arXiv:1908.07442.) https://arxiv.org/pdf/1908.07442.pdf. Easy installation You can install using pip by running:pip install pytorch-tabnet Source code If you wan to use it locally within a docker container: git clone [email protected]:dreamquark-ai/tabnet.git cd tabnet to get inside the repository CPU only make start to build and get inside the container GPU make start-gpu to build and […]
Read more
higher is a library providing support for higher-order optimization, e.g. through unrolled first-order optimization loops, of “meta” aspects of these loops. It provides tools for turning existing torch.nn.Module instances “stateless”, meaning that changes to the parameters thereof can be tracked, and gradient with regard to intermediate parameters can be taken. It also provides a suite of differentiable optimizers, to facilitate the implementation of various meta-learning approaches. Full documentation is available at https://higher.readthedocs.io/en/latest/. Python version >= 3.5 PyTorch version >= 1.3 […]
Read more
(Generic) EfficientNets for PyTorch A ‘generic’ implementation of EfficientNet, MixNet, MobileNetV3, etc. that covers most of the compute/parameter efficient architectures derived from the MobileNet V1/V2 block sequence, including those found via automated neural architecture search. All models are implemented by GenEfficientNet or MobileNetV3 classes, with string based architecture definitions to configure the block layouts (idea from here) Models Implemented models include: I originally implemented and trained some these models with code here, this repository contains just the GenEfficientNet models, validation, […]
Read more
PyTorch Sparse This package consists of a small extension library of optimized sparse matrix operations with autograd support. This package currently consists of the following methods: All included operations work on varying data types and are implemented both for CPU and GPU.To avoid the hazzle of creating torch.sparse_coo_tensor, this package defines operations on sparse tensors by simply passing index and value tensors as arguments (with same shapes as defined in PyTorch).Note that only value comes with autograd support, as index […]
Read more
PyTorch Scatter This package consists of a small extension library of highly optimized sparse update (scatter and segment) operations for the use in PyTorch, which are missing in the main package.Scatter and segment operations can be roughly described as reduce operations based on a given “group-index” tensor.Segment operations require the “group-index” tensor to be sorted, whereas scatter operations are not subject to these requirements. The package consists of the following operations with reduction types “sum”|”mean”|”min”|”max”: In addition, we provide the […]
Read more