A Framework for Agricultural Intelligent Analysis Based on a Visual Language Large Model

被引:1
|
作者
Yu, Piaofang [1 ,2 ]
Lin, Bo [1 ,2 ]
机构
[1] Zhejiang Univ, Sch Software Technol, Ningbo 315048, Peoples R China
[2] Zhejiang Univ, Binjiang Inst, Innovat Ctr Informat, Hangzhou 310053, Peoples R China
来源
APPLIED SCIENCES-BASEL | 2024年 / 14卷 / 18期
关键词
visual language large model; cross-modal fusion; image recognition; agricultural knowledge understanding;
D O I
10.3390/app14188350
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
Smart agriculture has become an inevitable trend in the development of modern agriculture, especially promoted by the continuous progress of large language models like chat generative pre-trained transformer (ChatGPT) and general language model (ChatGLM). Although these large models perform well in general knowledge learning, they still have certain limitations and errors when facing agricultural professional knowledge about crop disease identification, growth stage judgment, and so on. Agricultural data involves images and texts and other modalities, which play an important role in agricultural production and management. In order to better learn the characteristics of different modal data in agriculture, realize cross-modal data fusion, and thus understand complex application scenarios, we propose a framework AgriVLM that uses a large amount of agricultural data to fine-tune the visual language model to analyze agricultural data. It can fuse multimodal data and provide more comprehensive agricultural decision support. Specifically, it utilizes Q-former as a bridge between an image encoder and a language model to achieve a cross-modal fusion of agricultural images and text data. Then, we apply a Low-Rank adaptive to fine-tune the language model to achieve an alignment between agricultural image features and a pre-trained language model. The experimental results prove that AgriVLM demonstrates great performance in crop disease recognition and growth stage recognition, with recognition accuracy exceeding 90%, demonstrating its capability to analyze different modalities of agricultural data.
引用
收藏
页数:15
相关论文
共 50 条
  • [31] A large language model framework to uncover underreporting in traffic crashes
    Arteaga, Cristian
    Park, JeeWoong
    JOURNAL OF SAFETY RESEARCH, 2025, 92 : 1 - 13
  • [32] Bidirectional Planning for Autonomous Driving Framework with Large Language Model
    Ma, Zhikun
    Sun, Qicong
    Matsumaru, Takafumi
    SENSORS, 2024, 24 (20)
  • [33] A Visual Based Framework for the Model Refactoring Techniques
    Stolc, M.
    Polasek, I.
    2010 IEEE 8TH INTERNATIONAL SYMPOSIUM ON APPLIED MACHINE INTELLIGENCE AND INFORMATICS, 2010, : 77 - 82
  • [34] A LANGUAGE-BASED GENERATIVE MODEL FRAMEWORK FOR BEHAVIORAL ANALYSIS OF COUPLES' THERAPY
    Chakravarthula, Sandeep Nallan
    Gupta, Rahul
    Baucom, Brian
    Georgiou, Panayiotis
    2015 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP), 2015, : 2090 - 2094
  • [35] Collective intelligent toolbox based on linked model framework
    Thanh Binh Nguyen
    Wagner, Fabian
    JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2014, 27 (02) : 601 - 609
  • [36] A large language model framework for literature-based disease-gene association prediction
    Li, Peng-Hsuan
    Sun, Yih-Yun
    Juan, Hsueh-Fen
    Chen, Chien-Yu
    Tsai, Huai-Kuang
    Huang, Jia-Hsin
    BRIEFINGS IN BIOINFORMATICS, 2025, 26 (01)
  • [37] Large language model based framework for automated extraction of genetic interactions from unstructured data
    Gill, Jaskaran Kaur
    Chetty, Madhu
    Lim, Suryani
    Hallinan, Jennifer
    PLOS ONE, 2024, 19 (05):
  • [38] LLM-BRC: A large language model-based bug report classification framework
    Du, Xiaoting
    Liu, Zhihao
    Li, Chenglong
    Ma, Xiangyue
    Li, Yingzhuo
    Wang, Xinyu
    SOFTWARE QUALITY JOURNAL, 2024, 32 (03) : 985 - 1005
  • [39] Elicitron: A Large Language Model Agent-Based Simulation Framework for Design Requirements Elicitation
    Ataei, Mohammadmehdi
    Cheong, Hyunmin
    Grandi, Daniele
    Wang, Ye
    Morris, Nigel
    Tessier, Alexander
    JOURNAL OF COMPUTING AND INFORMATION SCIENCE IN ENGINEERING, 2025, 25 (02)
  • [40] IPM-AgriGPT: A Large Language Model for Pest and Disease Management with a G-EA Framework and Agricultural Contextual Reasoning
    Zhang, Yuqin
    Fan, Qijie
    Chen, Xuan
    Li, Min
    Zhao, Zeying
    Li, Fuzhong
    Guo, Leifeng
    MATHEMATICS, 2025, 13 (04)