GPU

Massively parallel self-organizing maps: accelerate training on multicore CPUs, GPUs, and clusters

Somoclu is a massively parallel implementation of self-organizing maps. It exploits multicore CPUs, it is able to rely on MPI for distributing the workload in a cluster, and it can be accelerated by CUDA. A sparse kernel is also included, which is useful for training maps on vector spaces generated in text mining processes. Key features: Fast execution by parallelization: OpenMP, MPI, and CUDA are supported. Multi-platform: Linux, macOS, and Windows are supported. Planar and toroid maps. Rectangular and hexagonal […]

Read more
PyTorch

A PyTorch library for decentralized deep learning across the Internet

Hivemind: decentralized deep learning in PyTorch Hivemind is a PyTorch library for decentralized deep learning across the Internet. Its intended usage is training one large model on hundreds of computers from different universities, companies, and volunteers. Key Features Distributed training without a master node: Distributed Hash Table allows connecting computers in a decentralizednetwork. Fault-tolerant backpropagation: forward and backward passes succeed even if some nodes are unresponsive or take toolong to respond. Decentralized parameter averaging: iteratively aggregate updates from multiple workers […]

Read more
Distributed

A Python distributed computing library for modern computer clusters

Distributed Computing for AI Made Simple This project is experimental and the APIs are not considered stable. Fiber is a Python distributed computing library for modern computer clusters. It is easy to use. Fiber allows you to write programs that run on a computer cluster level without the need to dive into the details of computer cluster. It is easy to learn. Fiber provides the same API as Python’s standard multiprocessing library that you are familiar with. If you know […]

Read more
Machine Learning

A high performance and generic framework for distributed DNN training

BytePS is a high performance and general distributed training framework. It supports TensorFlow, Keras, PyTorch, and MXNet, and can run on either TCP or RDMA network. BytePS outperforms existing open-sourced distributed training frameworks by a large margin. For example, on BERT-large training, BytePS can achieve ~90% scaling efficiency with 256 GPUs (see below), which is much higher than Horovod+NCCL. In certain scenarios, BytePS can double the training speed compared with Horovod+NCCL. Performance We show our experiment on BERT-large training, which […]

Read more
Tool

A lightweight tool for submitting Python functions for computation within a Slurm cluster

What is submitit? Submitit is a lightweight tool for submitting Python functions for computation within a Slurm cluster.It basically wraps submission and provide access to results, logs and more.Slurm is an open source, fault-tolerant, and highly scalable cluster management and job scheduling system for large and small Linux clusters.Submitit allows to switch seamlessly between executing on Slurm or locally. An example is worth a thousand words: performing an addition From inside an environment with submitit installed: import submitit def add(a, […]

Read more
PyTorch

A PyTorch extension library for high performance and large scale training

FairScale FairScale is a PyTorch extension library for high performance and large scale training.This library extends basic PyTorch capabilities while adding new SOTA scaling techniques.FairScale makes available the latest distributed training techniques in the form of composablemodules and easy to use APIs. These APIs are a fundamental part of a researcher’s toolbox asthey attempt to scale models with limited resources. FairScale was designed with the following values in mind: Usability –  Users should be able to understand and use FairScale […]

Read more
Deep Learning

praudio: Audio preprocessing framework for Deep Learning audio applications

praudio praudio provides objects and a script for performing complex preprocessing operations on entire audio datasets with one command. praudio is implemented having Deep Learning audio/music applications in mind. Operations are carried out on CPU. Preprocessing can also be run on-the-fly, for example, while training a model. The library uses librosa as an audio processing backend. How do I install the library? You can install praudio both with pip via PyPi, and by cloning the praudio repo from GitHub. For […]

Read more
Eyetracking

Unified Categorization for Eyetracking in Python

Categorization for Eyetracking in Python This repository was developed for Peter König’s Neurobiopsychology Lab at the Institute of Cognitive Science, Osnabrück. Its aim is to provide easy access to different automated gaze classification algorithms and to generate a unified, simplistic, and elegant way of handling Eyetracking data. Currently available gaze classification algorithms are: NSLR-HMM: Pekkanen, J., & Lappi, O. (2017). A new and general approach to signal denoising and eye movement classification based on segmented linear regression. Scientific reports, 7(1), […]

Read more
Distributed

A fast and reliable distributed task processing library for Python 3

A fast and reliable distributed task processing library for Python 3. Fork of dramatiq.io Installation If you want to use it with RabbitMQ $ pipenv install ‘remoulade[rabbitmq]’ or if you want to use it with Redis $ pipenv install ‘remoulade[redis]’ Quickstart Make sure you’ve got RabbitMQ running, then create a new file calledexample.py: from remoulade.brokers.rabbitmq import RabbitmqBroker import remoulade import requests import sys broker = RabbitmqBroker() remoulade.set_broker(broker) @remoulade.actor def count_words(url): response = requests.get(url) count = len(response.text.split(” “)) print(f”There are {count} […]

Read more
1 … 504 505 506 507 508 … 947