A huggingface transformers implementation of Transformer Memory as a Differentiable Search Index

A huggingface transformers implementation of Transformer Memory as a Differentiable Search Index, Yi Tay, Vinh Q. Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, Tal Schuster, William W. Cohen, Donald Metzler

Requirements: python=3.8 transformers=4.17.0 datasets=1.18.3 wandb

Note: This is not the official repository.

Goal of this repository

Reproduce the results of DSI Large, Naive String Docid, NQ10K. According to Table 3 in the original paper, we should have [email protected]=0.347,[email protected]=0.605

Step1: Create NQ10K training (indexing) and validation datasets

cd data/NQ
python3 create_NQ_train_vali.py

Step2: Run training script

cd back to the root directory and run:

we use

To finish reading, please visit source site