分布式表示学习

研究目标

1. 基于PyTorch底层API，设计容纳翻译模型（Trans系列）的分布式表示学习算法，以支持大规模知识图谱分布式表示学习
2. 权衡计算通信代价，优化训练过程，在准确率不降低（或少量降低）的前提下，减少训练时间，提升效率
3. 在真实知识图谱（DBpedia，Wikidata等）上进行对比实验，验证算法的准确性、高效性和可扩展性

相关论文

综述：

1. Knowledge Graph Embedding: A Survey of Approaches and Applications (TKDE 2017)

模型：

1. Translating Embeddings for Modeling Multi-relational Data (NIPS 2013)
2. Knowledge Graph Embedding by Translating on Hyperplanes (AAAI 2014)
3. Learning Entity and Relation Embeddings for Knowledge Graph Completion (AAAI 2015)
4. Knowledge Graph Embedding via Dynamic Mapping Matrix (IJCNLP 2015)
5. Differentiating Concepts and Instances for Knowledge Graph Embedding (EMNLP 2018)

系统：

1. Scaling Distributed Machine Learning with the Parameter Server (OSDI 2014)
2. 可扩展机器学习的并行与分布式优化算法综述 (软件学报 2017)
3. Angel: a new large-scale machine learning system (National Science Review 2018)
4. PYTORCH-BIGGRAPH: A LARGE-SCALE GRAPH EMBEDDING SYSTEM (SysML 2019)
5. PSGraph: How Tencent trains extremely large-scale graphs with Spark? (ICDE 2020)
6. AliGraph: A Comprehensive Graph Neural Network Platform (KDD 2019)
7. PyTorch Distributed: Experiences on Accelerating Data Parallel Training (VLDB 2020)
8. 图嵌入算法分布式优化 (软件学报 2020)
9. EDGES: An Efficient Distributed Graph Embedding System on GPU clusters (TPDS 2020)
10. DGL-KE: Training Knowledge Graph Embeddings at Scale (SIGIR 2020)
11. TORCHKGE: KNOWLEDGE GRAPH EMBEDDING IN PYTHON AND PYTORCH (Arxiv 2020)

State of the art工作

• PyTorch-Biggraph

github 教程 手册

• PS-Graph

• AliGraph

• DGL-KE

github: 教程: https://towardsdatascience.com/optimize-knowledge-graph-embeddings-with-dgl-ke-1fff4ab275f2

工具包

• PyTorch

github 教程

• OpenKE

github

• TorchKGE

github

• DGL

github

国内外相关课题组

1. 北京大学 崔斌教授组 : 主页链接