Megatron-LM源码系列(六)：Distributed-Optimizer分布式优化器实现Part1

2024-07-08 14:37:33

以下是30个以上比较好的中文 BERT 系列模型的 Github 源码： 1. BERT-Base, Chinese: https://github.com/google-research/bert/blob/master/multilingual.md#chinese-pre-trained-models 2. BERT-WWM-Ext, Chinese: https://github.com/ymcui/Chinese-BERT-wwm 3. BERT-WWM-Ext-finetune, Chinese: https://github.com/ymcui/Chinese-BERT-wwm/tree/master/finetune 4. RoBERTa-wwm-ext-large, Chinese: https://github.com/ymcui/Chinese-RoBERTa-wwm-ext 5. BERT-wwm-ext-multilingual: https://github.com/ymcui/BERT-wwm-ext 6. ALBERT-base, Chinese: https://github.com/brightmart/albert_zh 7. ALBERT-tiny, Chinese: https://github.com/brightmart/albert_tiny_zh 8. ALBERT-tiny-finetune, Chinese: https://github.com/brightmart/albert_tiny_zh/tree/master/finetune 9. ALBERT-xlarge, Chinese: https://github.com/brightmart/albert_zh/tree/master/albert_xlarge 10. ERNIE-v1.0, Chinese: https://github.com/PaddlePaddle/ERNIE 11. ERNIE-v2.0, Chinese: https://github.com/PaddlePaddle/ERNIE/tree/v2.0 12. ERNIE-Baidu, Chinese: https://github.com/baidu/ERNIE 13. GPT, Chinese: https://github.com/openai/gpt-2 14. GPT-2, Chinese: https://github.com/openai/gpt-2 15. XLNet, Chinese: https://github.com/ymcui/Chinese-XLNet 16. XLNet-Mid, Chinese: https://github.com/ymcui/Chinese-XLNet/tree/master/mid_data 17. XLNet-Large, Chinese: https://github.com/ymcui/Chinese-XLNet/tree/master/large_data 18. XLM-R, Chinese: https://github.com/ymcui/XLM-RoBERTa 19. Chinese-BART, Chinese: https://github.com/ymcui/Chinese-BART 20. Chinese-BART-finetune, Chinese: https://github.com/ymcui/Chinese-BART/tree/master/finetune 21. MT-DNN, Chinese: https://github.com/namisan/mt-dnn 22. MASS, Chinese: https://github.com/microsoft/MASS 23. T5, Chinese: https://github.com/google-research/text-to-text-transfer-transformer 24. DAE, Chinese: https://github.com/thunlp/DAE 25. DAE-finetune, Chinese: https://github.com/thunlp/DAE/tree