pre-training

Awesome resources for in-context learning and prompt engineering: Mastery of the LLMs such as ChatGPT, GPT-3, and FlanT5, with up-to-date and cutting-edge updates.

chatbot prompt language-modeling prompt-toolkit cot pre-training language-understanding prompt-learning prompt-tuning in-context-learning llm prompt-engineering chain-of-thought prompt-based-learning chatgpt chatgpt-api large-language-model prompt-design

UpdatedDec 26, 2024
Jupyter Notebook

LirongWu /awesome-graph-self-supervised-learning

Star1.4k

Code for TKDE paper "Self-supervised learning on graphs: Contrastive, generative, or predictive"

machine-learning deep-learning transfer-learning representation-learning unsupervised-learning data-augmentation graph-neural-networks self-supervised-learning pre-training pretext-task

UpdatedAug 15, 2024

zjunlp /KnowLM

Star1.3k

An Open-sourced Knowledgable Large Language Model Framework.

deep-learning models instructions english chinese llama lora language-model reasoning bilingual pre-training pre-trained-model pre-trained-language-models gpt-3 deepspeed instruction-following large-language-models instruction-tuning knowlm instructie

UpdatedJan 11, 2025
Python

yzhuoning /Awesome-CLIP

Star1.2k

Awesome list for research on CLIP (Contrastive Language-Image Pre-Training).

clip pre-training contrastive-learning

UpdatedJun 28, 2024

Tencent /TencentPretrain

Star1.1k

Tencent Pre-training framework in PyTorch & Pre-trained Model Zoo

natural-language-processing model-zoo pytorch classification bart chinese gpt pegasus ner clue albert bert fine-tuning roberta elmo pre-training gpt-2 t5 unilm xlm-roberta

UpdatedAug 4, 2024
Python

SalesforceAIResearch /uni2ts

Star1.1k

Unified Training of Universal Time Series Forecasting Transformers

machine-learning deep-learning time-series transformers forecasting representation-learning time-series-forecasting pre-training pre-trained-models universal-forecasting

UpdatedFeb 24, 2025
Jupyter Notebook

microsoft /Oscar

Star1k

Oscar and VinVL

vqa image-captioning oscar vision-and-language pre-training image-text-search vinvl

UpdatedAug 28, 2023
Python

qingsongedu /Awesome-TimeSeries-SpatioTemporal-LM-LLM

Star1k

A professional list on Large (Language) Models and Foundation Models (LLM, LM, FM) for Time Series, Spatiotemporal, and Event Data.

timeseries forecasting machinelearning deeplearning autoscaling large-models anomalydetection rca pre-training foundation-models large-language-models

UpdatedDec 22, 2024

brightmart /bert_language_understanding

Star962

Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN

nlp text-classification question-answering document-classification transfer-learning fasttext language-model textcnn attention-is-all-you-need self-attention transformer-encoder bert-model pre-training language-understanding

UpdatedJan 1, 2019
Python

ChenRocks /UNITER

Star791

Research code for ECCV 2020 paper "UNITER: UNiversal Image-TExt Representation Learning"

transformers pytorch vision-and-language pre-training

UpdatedJun 30, 2021
Python

jackroos /VL-BERT

Star741

Code for ICLR 2020 paper "VL-BERT: Pre-training of Generic Visual-Linguistic Representations".

pytorch representation-learning bert vision-and-language self-supervised-learning pre-training iclr2020 vl-bert

UpdatedMay 22, 2023
Jupyter Notebook

nancheng58 /Awesome-LLM4RS-Papers

Star652

Large Language Model-enhanced Recommender System Papers

survey awesome-list recommender-system gpt papers pre-training prompt-tuning large-language-models llm

UpdatedFeb 14, 2025

princeton-nlp /LLM-Shearing

Star595

[ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning

nlp efficiency pruning llama pre-training llm llama2

UpdatedMar 4, 2024
Python

cxcscmu /Craw4LLM

Star588

Official repository for "Craw4LLM: Efficient Web Crawling for LLM Pretraining"

crawler web-crawler crawling web-crawling pre-training pretraining large-language-models llm

UpdatedFeb 24, 2025
Python

Shen-Lab /GraphCL

Star580

[NeurIPS 2020] "Graph Contrastive Learning with Augmentations" by Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen

self-supervised-learning pre-training graph-neural-network contrastive-learning

UpdatedJul 17, 2024
Python

acbull /GPT-GNN

Star492

Code for KDD'20 "Generative Pre-Training of Graph Neural Networks"

graph-neural-networks self-supervised-learning graph-representation-learning pre-training

UpdatedJul 25, 2024
Python

Improve this page

Add a description, image, and links to thepre-training topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with thepre-training topic, visit your repo's landing page and select "manage topics."

Learn more

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly