Visualizing the Loss Landscape of Neural Nets
loss-landscape
This repository contains the PyTorch code for the paper
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer and Tom Goldstein. Visualizing the Loss Landscape of Neural Nets. NIPS, 2018.
An interactive 3D visualizer for loss surfaces has been provided by telesens.
Given a network architecture and its pre-trained parameters, this tool calculates and visualizes the loss surface along random direction(s) near the optimal parameters. The calculation can be done in parallel with multiple GPUs per node, and multiple nodes. The random direction(s) and loss surface values are stored in HDF5 (.h5) files after they are produced.
Setup
Environment: One or more multi-GPU node(s) with the following software/libraries installed:
Pre-trained models:
The code accepts pre-trained PyTorch models for the CIFAR-10 dataset.
To load the pre-trained model correctly, the model file