Author: Deep Learner

Uncategorized Leave a comment

DeepSpeed powers 8x larger MoE model training with high performance

Today, we are proud to announce DeepSpeed MoE, a high-performance system that supports massive scale mixture of experts (MoE) models as part of the DeepSpeed optimization library. MoE models are an emerging class of sparsely activated models that have sublinear compute costs with respect to their parameters. For example, the Switch Transformer consists of 1.6 trillion parameters, while the compute required to train it is approximately equal to that of a 10 billion-parameter dense model. This increase in model size […]

Read more
processing

A Python Framework for Large-Scale SAR Satellite Data Processing

pyroSAR A Python Framework for Large-Scale SAR Satellite Data Processing. The pyroSAR package aims at providing a complete solution for the scalable organization and processing of SAR satellite data: Reading of data from various past and present satellite missions Handling of acquisition metadata User-friendly access to processing utilities in SNAP and GAMMA Remote Sensing software Formatting of the preprocessed data for further analysis Export to Data Cube solutions Head on over to readthedocs for installation instructions, examples and API reference. […]

Read more
Analysis

Scalable analysis of image and time series analysis in python

thunder Thunder is an ecosystem of tools for the analysis of image and time series data in Python. It provides data structures and algorithms for loading, processing, and analyzing these data, and can be useful in a variety of domains, including neuroscience, medical imaging, video processing, and geospatial and climate analysis. It can be used locally, but also supports large-scale analysis through the distributed computing engine spark. All data structures and analyses in Thunder are designed to run identically and […]

Read more
Observability

A toolbox for processing earth observation data with Python

eo-box eobox is a Python package with a small collection of tools for working with Remote Sensing / Earth Observation data. Package Overview So far, the following subpackages are available: eobox.sampledata contains small sample data that can be used for playing around and testing. eobox.raster contains raster processing tools for extracting raster values at given (by vector data) locations, window- / chunk-wise processing of multiple single layer rasterfiles that do not fit in memory, e.g. calculating virtual time series and […]

Read more
Image Processing

A Python package for delineating nested surface depressions from digital elevation data

lidar lidar is Python package for delineating the nested hierarchy of surface depressions in digital elevation models (DEMs). It is particularly useful for analyzing high-resolution topographic data, such as DEMs derived from Light Detection and Ranging (LiDAR) data. GitHub repo: https://github.com/giswqs/lidarDocumentation: https://lidar.gishub.orgPyPI: https://pypi.org/project/lidarConda-forge: https://anaconda.org/conda-forge/lidarOpen in Colab: https://gishub.org/lidar-colabFree software: MIT license lidar is a Python package for delineating the nested hierarchy of surface depressions in digital elevation models (DEMs). In traditional hydrological modeling, surface depressions in a DEM are commonly treated […]

Read more
Geoscience

The whitebox Python package is built on WhiteboxTools

whitebox-python The whitebox Python package is built on WhiteboxTools, an advanced geospatial data analysis platform developed by Prof. John Lindsay (webpage; jblindsay) at the University of Guelph’s Geomorphometry and Hydrogeomatics Research Group. WhiteboxTools can be used to perform common geographical information systems (GIS) analysis operations, such as cost-distance analysis, distance buffering, and raster reclassification. Remote sensing and image processing tasks include image enhancement (e.g. panchromatic sharpening, contrast adjustments), image mosaicing, numerous filtering operations, simple classification (k-means), and common image transformations. […]

Read more
Tool

ArcGIS Python Toolbox for WhiteboxTools

ArcGIS Python Toolbox for WhiteboxTools. This repository is related to the ArcGIS Python Toolbox for WhiteboxTools, which is an ArcGIS frontend of a stand-alone executable command-line program called WhiteboxTools. WhiteboxTools-ArcGIS is an ArcGIS Python Toolbox for WhiteboxTools, an advanced geospatial data analysis platform developed by Prof. John Lindsay (webpage; jblindsay) at the University of Guelph’s Geomorphometry and Hydrogeomatics Research Group. WhiteboxTools can be used to perform common geographical information systems (GIS) analysis operations, such as cost-distance analysis, distance buffering, and […]

Read more
Google Earth

Get Landsat surface reflectance time-series from google earth engine

Google Earth Engine data extraction tool. Quickly obtain Landsat multispectral time-series for exploratory analysis and algorithm testing Online documentation available at https://loicdtx.github.io/landsat-extract-gee A python library (API + command lines) to extract Landsat time-series from the Google Earth Engine platform. Can query single pixels or spatially aggregated values over polygons. When used via the command line, extracted time-series are written to a sqlite database. The idea is to provide quick access to Landsat time-series for exploratory analysis or algorithm testing. Instead […]

Read more
Function

A C/Python library with vectorized geometry functions

PyGEOS PyGEOS is a C/Python library with vectorized geometry functions. The geometry operations are done in the open-source geometry library GEOS. PyGEOS wraps these operations in NumPy ufuncs providing a performance improvement when operating on arrays of geometries. Note: PyGEOS is a very young package. While the available functionality should be stable and working correctly, it’s still possible that APIs change in upcoming releases. But we would love for you to try it out, give feedback or contribute! What is […]

Read more
Maps

Create Interactive maps with your geodataframe

geopatra Geopatra extends geopandas for interactive mapping and attempts to wrap the goodness of amazing mapping libraries like Folium, Plotly, Kepler.gl, hvplot etc for rapidly creating interactive maps with Geodataframes You can already create interactive maps easily with geopandas and Folium/ Plotly/ Kepler.gl. Geopatra is merely meant to make this easier and is more geared towards ease and currently does not support complex maps or intricate style control Installation Everything is always a pip away pip install geopatra Basic Usage […]

Read more
1 … 555 556 557 558 559 … 984