CyberZHG

Implementation of BERT that could load official pre-trained models for feature extraction and prediction

Attention mechanism for processing sequential data that considers the context for each timestamp.

Some exercises and problems in Introduction to Algorithms 3rd edition.

Transformer implemented in Keras

RAdam implemented in Keras & TensorFlow

A wrapper layer for stacking layers horizontally

Implementation of XLNet that can load pretrained checkpoints

Load GPT-2 checkpoint and generate texts

Transformer-XL with checkpoint loader

Position embedding layers in Keras

Graph convolutional layers

Layer normalization implemented in Keras

AdaBound optimizer in Keras

Lookahead mechanism for optimizers in Keras.

Learning rate multiplier

Concatenate word and character embeddings in Keras