Tool

A lightweight tool for submitting Python functions for computation within a Slurm cluster

What is submitit? Submitit is a lightweight tool for submitting Python functions for computation within a Slurm cluster.It basically wraps submission and provide access to results, logs and more.Slurm is an open source, fault-tolerant, and highly scalable cluster management and job scheduling system for large and small Linux clusters.Submitit allows to switch seamlessly between executing on Slurm or locally. An example is worth a thousand words: performing an addition From inside an environment with submitit installed: import submitit def add(a, […]

Read more
PyTorch

A PyTorch extension library for high performance and large scale training

FairScale FairScale is a PyTorch extension library for high performance and large scale training.This library extends basic PyTorch capabilities while adding new SOTA scaling techniques.FairScale makes available the latest distributed training techniques in the form of composablemodules and easy to use APIs. These APIs are a fundamental part of a researcher’s toolbox asthey attempt to scale models with limited resources. FairScale was designed with the following values in mind: Usability –  Users should be able to understand and use FairScale […]

Read more
Deep Learning

praudio: Audio preprocessing framework for Deep Learning audio applications

praudio praudio provides objects and a script for performing complex preprocessing operations on entire audio datasets with one command. praudio is implemented having Deep Learning audio/music applications in mind. Operations are carried out on CPU. Preprocessing can also be run on-the-fly, for example, while training a model. The library uses librosa as an audio processing backend. How do I install the library? You can install praudio both with pip via PyPi, and by cloning the praudio repo from GitHub. For […]

Read more
Eyetracking

Unified Categorization for Eyetracking in Python

Categorization for Eyetracking in Python This repository was developed for Peter König’s Neurobiopsychology Lab at the Institute of Cognitive Science, Osnabrück. Its aim is to provide easy access to different automated gaze classification algorithms and to generate a unified, simplistic, and elegant way of handling Eyetracking data. Currently available gaze classification algorithms are: NSLR-HMM: Pekkanen, J., & Lappi, O. (2017). A new and general approach to signal denoising and eye movement classification based on segmented linear regression. Scientific reports, 7(1), […]

Read more
Distributed

A fast and reliable distributed task processing library for Python 3

A fast and reliable distributed task processing library for Python 3. Fork of dramatiq.io Installation If you want to use it with RabbitMQ $ pipenv install ‘remoulade[rabbitmq]’ or if you want to use it with Redis $ pipenv install ‘remoulade[redis]’ Quickstart Make sure you’ve got RabbitMQ running, then create a new file calledexample.py: from remoulade.brokers.rabbitmq import RabbitmqBroker import remoulade import requests import sys broker = RabbitmqBroker() remoulade.set_broker(broker) @remoulade.actor def count_words(url): response = requests.get(url) count = len(response.text.split(” “)) print(f”There are {count} […]

Read more
Debugger

Full-screen console debugger for Python

Its goal is to provide all the niceties of modern GUI-based debuggers in a more lightweight and keyboard-friendly package. PuDB allows you to debug code right where you write and test it–in a terminal. If you’ve worked with the excellent (but nowadays ancient) DOS-based Turbo Pascal or C tools, PuDB’s UI might look familiar. Here’s a screenshot: You may watch a screencast, too. Features Syntax-highlighted source, the stack, breakpoints and variables are all visible at once and continuously updated. This […]

Read more
Debugger

An improbable web debugger through WebSockets

Description wdb is a full featured web debugger based on a client-server architecture. The wdb server which is responsible of managing debugging instances along with browser connections (through websockets) is based on Tornado. The wdb clients allow step by step debugging, in-program python code execution, code edition (based on CodeMirror) setting breakpoints… Due to this architecture, all of this is fully compatible with multithread and multiprocess programs. wdb works with python 2 (2.6, 2.7), python 3 (3.2, 3.3, 3.4, 3.5) […]

Read more
Websocket

In-process service that will accept unix domain socket connections and present the stacktraces

Manhole is in-process service that will accept unix domain socket connections and present the stacktraces for all threads and an interactive prompt. It can either work as a python daemon thread waiting for connections at all times or a signal handler (stopping your application and waiting for a connection). Access to the socket is restricted to the application’s effective user id or root. This is just like Twisted’s manhole. It’s simpler (no dependencies), it only runs on Unix domain sockets […]

Read more
Debugger

Debugger capable of attaching to and injecting code into python processes

DISCLAIMER: This is not an official google project, this is just something I wrote while at Google. What this is Pyringe is a python debugger capable of attaching to running processes, inspecting their state and even of injecting python code into them while they’re running. With pyringe, you can list threads, get tracebacks, inspect locals/globals/builtins of running functions, all without having to prepare your program for it. What this is not A “Google project”. It’s my internship project that got […]

Read more
1 … 506 507 508 509 510 … 949