Identifying the total number of fruits on trees has long been of interest in agricultural crop estimation work. Yield prediction of fruits in practical environment is one of the hard and significant tasks to obtain better results in crop management system to achieve more productivity with regard to moderate cost. Utilized color vision in machine vision system to identify citrus fruits, and estimated yield information of the citrus grove in-real time. Fruit recognition algorithms based on color features to estimate the number of fruit. In the current research work, some low complexity and efficient image analysis approach was proposed to count yield fruits image in the natural scene. Semi automatic segmentation and yield calculation of fruit based on shape analysis is presented. Color and shape analysis was utilized to segment the images of different fruits like apple, pomegranate obtained under different lighting conditions. First the input sectional tree image was converted from RGB colour space into the colour space transform (i.e., YUV, YIQ, or YCbCr). The resultant image was then applied to the algorithm for fruit segmentation. After it is applied Morphological Operations which is enhanced image then execute Blob counting method which identify the object and count the number of it. Accuracy of this algorithm used in this thesis is 82.21% for images that have been scanned.