OpenROAD-Assistant: An Open-Source Large Language Model for Physical Design Tasks

Large language models (LLMs) have shown significant potential in serving as domain-specific chatbots. Recently, these models have emerged as powerful tools for chip design, providing both natural language responses and script generation for domain-specific inquiries. Previous work has demonstrated the effectiveness of LLMs in assisting with physical design automation; however, these approaches often rely on proprietary tools, APIs, technologies, and designs. As a result, access to these models is extremely limited, particularly for new chip designers who could greatly benefit from a design assistant. This paper introduces OpenROAD-Assistant, an open-source chatbot for OpenROAD that relies only on public data and responds to queries in either prose or Python script using the OpenROAD APIs. OpenROAD-Assistant leverages the Llama3-8B foundation model and employs retrieval-aware fine-tuning (RAFT) to respond to physical design-specific questions for OpenROAD. Notably, OpenROAD-Assistant outperforms other foundational models such as ChatGPT3.5, ChatGPT4, Code Llama, Claude3, and other ablation study baselines on the measured metrics (pass@kappa for scripting and BERTScore/BARTScore for question-answering). OpenROAD-Assistant achieves a 77% pass@1 score, 80% pass@3 score for scripting, and it achieves a 98% BERTScore and 96% BARTScore on question-answering.