lucidrains

Implementation of ETSformer, state of the art time-series Transformer, in Pytorch

An implementation of masked language modeling for Pytorch, made as concise and simple as possible

Implementation of Invariant Point Attention, used for coordinate refinement in the structure module of Alphafold2, as a standalone Pytorch module

A denoising diffusion probabilistic model (DDPM) tailored for conditional generation of protein distograms

Implementation of Chroma, generative models of protein using DDPM and GNNs, in Pytorch

Implementation of STAM (Space Time Attention Model), a pure and simple attention model that reaches SOTA for video classification

Implementation of π-GAN, for 3d-aware image synthesis, in Pytorch

Implementation of the GBST block from the Charformer paper, in Pytorch

Implementation of Long-Short Transformer, combining local and global inductive biases for attention over long sequences, in Pytorch

A simple to use pytorch wrapper for contrastive self-supervised learning on any neural network

Implementation of ProteinBERT in Pytorch

Implementation of Nyström Self-attention, from the paper Nyströmformer

Pytorch implementation of the hamburger module from the ICLR 2021 paper "Is Attention Better Than Matrix Decomposition"

Implementation of Uniformer, a simple attention and 3d convolutional net that achieved SOTA in a number of video classification tasks, debuted in ICLR 2022

A Pytorch implementation of Global Self-Attention Network, a fully-attention backbone for vision tasks

Implementation of Gated State Spaces, from the paper "Long Range Language Modeling via Gated State Spaces", in Pytorch

Implementation and replication of ProGen, Language Modeling for Protein Generation, in Jax

Implementation of Lie Transformer, Equivariant Self-Attention, in Pytorch

GPT, but made only out of MLPs

Implementation of Uformer, Attention-based Unet, in Pytorch

Implementation of NWT, audio-to-video generation, in Pytorch

Implementation of Marge, Pre-training via Paraphrasing, in Pytorch

Implementation of trRosetta and trDesign for Pytorch, made into a convenient package, for protein structure prediction and design

Implementation of the video diffusion model and training scheme presented in the paper, Flexible Diffusion Modeling of Long Videos, in Pytorch

Implementation of RQ Transformer, proposed in the paper "Autoregressive Image Generation using Residual Quantization"

Implementation of Differentiable Sign-Distance Function Rendering - in Pytorch

Implementation of a Transformer that Ponders, using the scheme from the PonderNet paper

Implementation of Hierarchical Transformer Memory (HTM) for Pytorch

Implementation of Discrete Key / Value Bottleneck, in Pytorch

Implementation of Hourglass Transformer, in Pytorch, from Google and OpenAI