Grafting technique helps to reduce soil-borne diseases, overcome the obstacles of cropping cultivation, increase yield and fruit quality. But grafting requires precise technique; the proficiency of operation will also directly affect the productivity and the survival rate of grafted seedlings. With the wide application of grafting technology, grafting robots were developed continually. However, the current grafting robots could not realize the production automation completely, and it is necessary for them to grade seedlings by hand and match different diameters of the rootstock and scion, and it is urgent to realize grading automatically before grafting. So, the machine vision system of the grafting robot will be designed to grade grafting seedling automatically to improve grading accuracy, reduce labor and improve the quality and speed of grafting effectively. Firstly, the hardware structure of the grading system was designed based on machine vision, which includes selection of Japanese CIS industrial cameras, whose type is UXGA, with a resolution of 1600 x 1200, a frame rate of 20fps, 1/1.8CCD, and 8mm lens and so on. Then using proper lighting resource, the algorithm of the image acquisition and segmentation about tomato grafting classification was analysed using the VC++ software and OpenCV tools. Taking the tomato seedlings for the research object, the statistical methods that gray value accumulation of pixels horizontally extracting stem diameter of seedlings features were used in this paper. Thus the proper joint point was detected and determined according to the middle point of straight and the smooth part of pixels' curve. Finally, the grading system software was designed and debugged, and 100 tomato seedlings were acquired and processed using the visual grading system for tomato seedling and classified by A, B, C, D levels, the results show that the system classification success rate of 96%, because of irregular blade barrier stalks or stalks suddenly bend, the four seedlings grading failure. The results also show the grading system was designed reasonably and performed reliably.