A Framework for Agricultural Intelligent Analysis Based on a Visual Language Large Model

被引:1
|
作者
Yu, Piaofang [1 ,2 ]
Lin, Bo [1 ,2 ]
机构
[1] Zhejiang Univ, Sch Software Technol, Ningbo 315048, Peoples R China
[2] Zhejiang Univ, Binjiang Inst, Innovat Ctr Informat, Hangzhou 310053, Peoples R China
来源
APPLIED SCIENCES-BASEL | 2024年 / 14卷 / 18期
关键词
visual language large model; cross-modal fusion; image recognition; agricultural knowledge understanding;
D O I
10.3390/app14188350
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
Smart agriculture has become an inevitable trend in the development of modern agriculture, especially promoted by the continuous progress of large language models like chat generative pre-trained transformer (ChatGPT) and general language model (ChatGLM). Although these large models perform well in general knowledge learning, they still have certain limitations and errors when facing agricultural professional knowledge about crop disease identification, growth stage judgment, and so on. Agricultural data involves images and texts and other modalities, which play an important role in agricultural production and management. In order to better learn the characteristics of different modal data in agriculture, realize cross-modal data fusion, and thus understand complex application scenarios, we propose a framework AgriVLM that uses a large amount of agricultural data to fine-tune the visual language model to analyze agricultural data. It can fuse multimodal data and provide more comprehensive agricultural decision support. Specifically, it utilizes Q-former as a bridge between an image encoder and a language model to achieve a cross-modal fusion of agricultural images and text data. Then, we apply a Low-Rank adaptive to fine-tune the language model to achieve an alignment between agricultural image features and a pre-trained language model. The experimental results prove that AgriVLM demonstrates great performance in crop disease recognition and growth stage recognition, with recognition accuracy exceeding 90%, demonstrating its capability to analyze different modalities of agricultural data.
引用
收藏
页数:15
相关论文
共 50 条
  • [41] An Intent-based Networks Framework based on Large Language Models
    Fuad, Ahlam
    Ahmed, Azza H.
    Riegler, Michael A.
    Cicic, Tarik
    2024 IEEE 10TH INTERNATIONAL CONFERENCE ON NETWORK SOFTWARIZATION, NETSOFT 2024, 2024, : 7 - 12
  • [42] Knowledge graph construction for intelligent cockpits based on large language models
    Dong, Haomin
    Wang, Wenbin
    Sun, Zhenjiang
    Kang, Ziyi
    Ge, Xiaojun
    Gao, Fei
    Wang, Jixin
    SCIENTIFIC REPORTS, 2025, 15 (01):
  • [43] Research on Intelligent Grading of Physics Problems Based on Large Language Models
    Wei, Yuhao
    Zhang, Rui
    Zhang, Jianwei
    Qi, Dizhi
    Cui, Wenqian
    EDUCATION SCIENCES, 2025, 15 (02):
  • [44] Simulation Analysis for intelligent scheduling model of large tasks
    Wang LiXin
    Wang WeiJing
    PROCEEDINGS OF THE 2015 INTERNATIONAL CONFERENCE ON AUTOMATION, MECHANICAL CONTROL AND COMPUTATIONAL ENGINEERING, 2015, 124 : 1302 - 1307
  • [45] Intelligent smelting process, management system: Efficient and intelligent management strategy by incorporating large language model
    Fu, Tianjie
    Liu, Shimin
    Li, Peiyu
    FRONTIERS OF ENGINEERING MANAGEMENT, 2024, 11 (03) : 396 - 412
  • [46] LUPA: A Framework for Large Scale Analysis of the Programming Language Usage
    Vlasova, Anna
    Tigina, Maria
    Vlasov, Ilya
    Birillo, Anastasiia
    Golubev, Yaroslav
    Bryksin, Timofey
    2022 MINING SOFTWARE REPOSITORIES CONFERENCE (MSR 2022), 2022, : 398 - 402
  • [47] Large Language Model for Automating the Analysis of Cryoprotectants
    Ashikhmina, Mariia S.
    Zenkin, Artemii M.
    Ivanova, Anastasia O.
    Pavlishina, Irina R.
    Orlova, Olga Y.
    Pantiukhin, Igor S.
    Skorb, Ekaterina V.
    JOURNAL OF CHEMICAL INFORMATION AND MODELING, 2024,
  • [48] An Information Security Situation Analysis Model of Intelligent Electric Grid based on Large Data
    Yang Ying
    Meng Huiping
    Dang Fangfang
    Yan Lijing
    PROCEEDINGS OF THE 2017 4TH INTERNATIONAL CONFERENCE ON MACHINERY, MATERIALS AND COMPUTER (MACMC 2017), 2017, 150 : 512 - 515
  • [49] Framework-based qualitative analysis of free responses of Large Language Models: Algorithmic fidelity
    Amirova, Aliya
    Fteropoulli, Theodora
    Ahmed, Nafiso
    Cowie, Martin R.
    Leibo, Joel Z.
    PLOS ONE, 2024, 19 (03):
  • [50] Utility-Based Precoding Optimization Framework for Large Intelligent Surfaces
    Bjornson, Emil
    Sanguinetti, Luca
    CONFERENCE RECORD OF THE 2019 FIFTY-THIRD ASILOMAR CONFERENCE ON SIGNALS, SYSTEMS & COMPUTERS, 2019, : 863 - 867