Exploring the Benefits of Training Expert Language Models over Instruction Tuning

被引：0
|
作者：
Jang, Joel [1 ]
Kim, Seungone [1 ]
Ye, Seonghyeon [1 ]
Kim, Doyoung [1 ]
Logeswaran, Lajanugen [2 ]
Lee, Moontae [2 ,3 ]
Lee, Kyungjae [2 ]
Seo, Minjoon [1 ]
机构：
[1] KAIST, Korea, Republic of
[2] LG AI Research, Korea, Republic of
[3] University of Illinois, Chicago, United States
来源：
Proceedings of Machine Learning Research | 2023年 / 202卷
关键词：
Compilation and indexing terms; Copyright 2024 Elsevier Inc;
D O I：
暂无
中图分类号：
学科分类号：
摘要：
Computational linguistics
引用
收藏
页码：14702 / 14729
相关论文
共 50 条
  • [1] Exploring the Benefits of Training Expert Language Models over Instruction Tuning
    Jang, Joel
    Kim, Seungone
    Ye, Seonghyeon
    Kim, Doyoung
    Logeswaran, Lajanugen
    Lee, Moontae
    Lee, Kyungjae
    Seo, Minjoon
    arXiv, 2023,
  • [2] Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Models
    Mekala, Dheeraj
    Nguyen, Alex
    Shang, Jingbo
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: ACL 2024, 2024, : 10456 - 10470
  • [3] Exploring the effectiveness of instruction tuning in biomedical language processing
    Rohanian, Omid
    Nouriborji, Mohammadmahdi
    Kouchaki, Samaneh
    Nooralahzadeh, Farhad
    Clifton, Lei
    Clifton, David A.
    ARTIFICIAL INTELLIGENCE IN MEDICINE, 2024, 158
  • [4] OCTOPACK: INSTRUCTION TUNING CODE LARGE LANGUAGE MODELS
    Muennighoff, Niklas
    Liu, Qian
    Zebaze, Armel
    Zheng, Qinkai
    Hui, Binyuan
    Zhuo, Terry Yue
    Singh, Swayam
    Tang, Xiangru
    von Werra, Leandro
    Longpre, Shayne
    arXiv, 2023,
  • [5] GraphGPT: Graph Instruction Tuning for Large Language Models
    Tang, Jiabin
    Yang, Yuhao
    Wei, Wei
    Shi, Lei
    Su, Lixin
    Cheng, Suqi
    Yin, Dawei
    Huang, Chao
    PROCEEDINGS OF THE 47TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL, SIGIR 2024, 2024, : 491 - 500
  • [6] Phased Instruction Fine-Tuning for Large Language Models
    Pang, Wei
    Zhou, Chuan
    Zhou, Xiao-Hua
    Wang, Xiaojie
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: ACL 2024, 2024, : 5735 - 5748
  • [7] BioInstruct: instruction tuning of large language models for biomedical natural language processing
    Tran, Hieu
    Yang, Zhichao
    Yao, Zonghai
    Yu, Hong
    JOURNAL OF THE AMERICAN MEDICAL INFORMATICS ASSOCIATION, 2024, 31 (09) : 1821 - 1832
  • [8] The Benefits of Expert Instruction in Microsurgery Courses
    Paladino, Joseph
    Gasteratos, Konstantinos
    Akelina, Yelena
    Marshall, Brittany
    Papazoglou, Lysimachos G.
    Strauch, Robert J.
    JOURNAL OF RECONSTRUCTIVE MICROSURGERY, 2021, 37 (02) : 143 - 153
  • [9] Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models
    Luo, Gen
    Zhou, Yiyi
    Ren, Tianhe
    Chen, Shengxin
    Sun, Xiaoshuai
    Ji, Rongrong
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023), 2023,
  • [10] Demystifying Instruction Mixing for Fine-tuning Large Language Models
    Wang, Renxi
    Li, Haonan
    Wu, Minghao
    Wang, Yuxia
    Han, Xudong
    Zhang, Chiyu
    Baldwin, Timothy
    PROCEEDINGS OF THE 62ND ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, VOL 4: STUDENT RESEARCH WORKSHOP, 2024, : 86 - 93
12345