Deep Learning

State-of-the-art deep learning and self-supervised learning algorithms for tabular data using PyTorch

deep-table implements various state-of-the-art deep learning and self-supervised learning algorithms for tabular data using PyTorch.

Design

Architecture

As shown below, each pretraining/fine-tuning model is decomposed into two modules: Encoder and Head.

Encoder

Encoder has Embedding and Backbone.

  • Embedding makes continuous/categorical features tokenized or simply normalized.
  • Backbone processes the tokenized features.

Pretraining/Fine-tuning Head

Pretraining/Fine-tuning Head uses Encoder module for training.

Implemented Methods

Available Modules

Encoder – Embedding

  • FeatureEmbedding
  • TabTransformerEmbedding

Encoder – Backbone

  • MLPBackbone
  • FTTransformerBackbone
  • SAINTBackbone

Model – Head

Model – Pretraining