Deep Learning, NLP, NMT, AI, ML
Исходные данные из API Госдумы РФ извлекаются с помощью Apache Nifi и приземляются в хранилище Clickhouse. Репозиторий состоит из двух частей Data transformation – преобразование данных, приземлённых из API в 3НФ с помощью dbt Telegram бот Архитектура системы GitHub View Github
Read more
Created covid data pipeline using PySpark and MySQL that collected data stream from API and do some processing and store it into MYSQL database. Tools used : PySpark , MySQL Procedure Fetch latest data from API using requests & pandas module of python. Apply some data processing and filtering to generate summarized information. Store that summarized information into database using MySQL. To build above pipeline i had used pyspark {IMPORTANT} Before move to the execution part please read below sentences […]
Read more
This is an implementation of the following paper: Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, Vikas Chandra. Federated Learning with Non-IID DataarXiv:1806.00582. Paper TL;DR: Previous federated optization algorithms (such as FedAvg and FedProx) converge to stationary points of a mismatched objective function due to heterogeneity in data distribution. In this paper, the authors propose a data-sharing strategy to improve training on non-IID data by creating a small subset of data which is globally shared between all the […]
Read more
Scientific Visualization: Python + Matplotlib Nicolas P. Rougier, Bordeaux, November 2021. The Python scientific visualisation landscape is huge. It is composed of a myriad of tools, ranging from the most versatile and widely used down to the more specialised and confidential. Some of these tools are community based while others are developed by companies. Some are made specifically for the web, others are for the desktop only, some deal with 3D and large data, while others target flawless 2D rendering. […]
Read more
GiantMIDI-Piano [1] is a classical piano MIDI dataset contains 10,854 MIDI files of 2,786 composers. The curated subset by constraining composer surnames contains 7,236 MIDI files of 1,787 composers. GiantMIDI-Piano are transcribed from live recordings with a high-resolution piano transcription system [2]. Here is the demo of GiantMIDI-Piano: https://www.youtube.com/watch?v=5U-WL0QvKCg Transcribed MIDI files of GiantMIDI-Piano can be viewed at midis_preview directory. Download GiantMIDI-Piano Method 1 (suggested) Follow disclaimer.md to agree a disclaimer and download a stable version of GiantMIDI-Piano (193 MB). […]
Read more
Introduction This is a collection of python-based sample code that interact with the Ryze Tello drone. Project Description This toolkit contains three sample programs based on tello sdk and python2.7,including Single_Tello_Test, Tello_Video, and Tello_Video (With_Pose_Recognition). There is also a program file named tello_state.py. In Single_Tello_Test,You can design a series of command combinations by writing a txt script to let tello execute a series of actions you have designed. This program can also be used as a command set test tool […]
Read more
Automate Excel via PowerShell without having Excel installed. Runs on Windows, Linux and MAC. Creating Tables, Pivot Tables, Charts and much more has just become a lot easier. Open ImportExcel as a remote repo in VS Code, without cloning it. CI System Environment Status Azure DevOps Windows Azure DevOps Windows (Core) Azure DevOps Ubuntu Azure DevOps macOS
Read moreDonkeycar is minimalist and modular self driving library for Python. It is developed for hobbyists and students with a focus on allowing fast experimentation and easy community contributions. Quick Links Use Donkey if you want to: Make an RC car drive its self. Compete in self driving races like DIY Robocars Experiment with autopilots, mapping computer vision and neural networks. Log sensor data. (images, user inputs, sensor readings) Drive your car via a web or game controller. Leverage community contributed […]
Read morecve-search is a tool to import CVE (Common Vulnerabilities and Exposures) and CPE (Common Platform Enumeration) into a MongoDB to facilitate search and processing of CVEs. The main objective of the software is to avoid doing direct and public lookups into the public CVE databases. Local lookups are usually faster and you can limit your sensitive queries via the Internet. cve-search includes a back-end to store vulnerabilities and related information, an intuitive web interface for search and managing vulnerabilities, a […]
Read more