DEVELOPMENT OF A PROXIMAL MACHINE VISION SYSTEM FOR OFF-SEASON WEED MAPPING IN BROADACRE NO-TILLAGE FALLOWS

Weeds are among the most significant and costly environmental threats in Australian agriculture. Weeds compete with crop plants for moisture, nutrients and sunlight and can have a detrimental impact on crop yields and quality if uncontrolled. The distribution, size, density and species of the weeds are often heterogeneous in the cropping land. Instead of uniformly spray the same type of herbicide to the whole farm land, selective spray can reduce the herbicide usage therefore can reduce the serious problems of herbicide resistance, soil damage and food safety. This study describes a weed mapping method which could be used for broadacre no-tillage fallow weed management. The weed maps have the potential to be used as powerful herbicide prescription maps for spot spray. The weed mapping is realized by the machine vision technologies which including image acquisition, image stitching and photomosaic processing. The sampling points are continuous and the interpolation methods are used at the minimum levels. The experiment result shows that this weed mapping method can map weed under limited conditions.


INTRODUCTION
Weeds often grow in aggregated patches of varying size or in stripes along the field borders and along the direction of cultivation. It has been recognized widely that there are financial and environmental benefits from spraying only weed patches, rather than the entire fields or by adjusting application rates according to weed density (Cepl and Kasal, 2010;Gerhards, 2010;Rew and Cousens, 2001;Sharif and Mollick, 2013). Land managers should set weed management priorities before taking the action of the weed control. The weed control plan can be made based on the information of the weeds including the location, density, species and size.
Weed mapping is a valuable tool for optimizing resource utilization in the management of weed control efforts.
Weed map can be used as a powerful tool to help understanding the distribution of weeds and monitoring the spread of established weeds and the effectiveness of control programs (Gutjahr and Gerhards, 2010;Slaughter et al., 2008). Weed mapping could also be used to create a prescription map for a spray operation (Greg, 2001). Mapping weed distribution prior to a weed control program and then mapping in the subsequent years will provide an indication of the program's effectiveness.
More than half of the Australian cropping area is zero-tillage or no-tillage (Edwards et al., 2012) and the most commonly used weed control method in no-tillage fallow land is to uniformly spray the same type of herbicide regardless the spatial population of the weeds (Beckie and Tardif, 2012). Using weed mapping method to realize site-specific weed manage and selective spray Science Publications JCS in the no-tillage fallow is significant. However the tasks of collecting the necessary data and making weed maps to meet weed management objectives while remaining accurate and cost effective are quite challenging. There are many weed mapping methods have been developed in the previous researches, however it still lacks a proper weed mapping approach which can accurately and efficiently map weeds for the weed management in the broadacre no-tillage cropping lands.
The objective of the proposed research is to develop a proximal Machine Vision System (MVS) for off-season weed mapping in broadacre no-tillage fallows. The developed weed maps would provide accurate density and distribution information of the weeds in the field therefore the weed maps could be used for more efficient weed control management at reduced herbicide usage. The structure of this study is organized as following: Section 2 reviews and compares the existing weed mapping methods. Section 3 introduces the material and methods used in this proposed research. The weed mapping methods include three aspects which are image acquisition, image stitching and photomosaic processing to provide the necessary information on the weed maps. Section 4 discusses the result of the proposed weed mapping method and section 5 presents the conclusion and future works of this research.

Literature Review of the Weed Mapping Methods
Weed mapping includes two aspects: data collection and data processing. Data collection is a series of works including taking photos of weeds and recording weed species, growing stage and position. Data processing involves the works of processing the collected data and presenting the processed data in the manner of weed maps.
Traditionally, data collection or called weed sampling was done manually, using random sampling techniques by a weed expert walking through a field. The experts were equipped with cameras and GPS and the weed maps were made by inputting the collected data into Geographic Information System (GIS). The manual approach is labor intensive and expensive therefore the measurements are reduced to a minimum which may lead to an over or under estimation of weed competition (Horstmeier, 1997;Nashiki et al., 2006;Rew and Cousens, 2001). Recently, many precise and powerful weed mapping techniques to automatically and continuously determine in-field variation of weed seedling populations were developed. From the view of the data collection platforms, the weed mapping methods can be categorized into two types: remote weed sensing and ground based weed sensing or called proximal weed sensing.
The remote sensed images can be acquired by aircraft, UAV (unmanned aerial vehicle) and satellite platforms (Samseemoung et al., 2012;Swanson, 2010;Thorp and Tian, 2004). One of the main advantages of remote sensing is the speed at which weed infestation maps can be generated, while remote sensing have the limitation to capture higher resolution image for detailed data analysis. The typical weed sensors for remote sensing are multi-spectral meters or hyperspectral meters and CCD cameras. The resolution of the images captured at the different platforms from satellites to UAV is 4 to 1 m approximately at the altitude of 680 to 1 km. Some smaller size and low altitude UAV can grab images at the altitude of 10 to 20 m with the resolution of 5 to 10 mm (Himstedt et al., 2012;Lopez-Granados, 2011;Samseemoung et al., 2012;Sui et al., 2008). Another disadvantage of remote sensing is that the platforms such as satellite or aircraft are heavily dependent on the weather conditions (cloud cover, atmospheric distortion). The timing window of data acquisition and delivery to the end users could be a few days or weeks (Lopez-Granados, 2011).
A finer resolution of the sensor, however, is required to detect lower density weed population and smaller size weed seedling. Therefore, many proximal weed sensing method were developed to detect weeds in the near range. Tang (2002) developed a MVS to map weeds and crops using a 3-CCD camera. The camera was mounted at a height of 3.35m with the resolution of 2×2 mm and the FOV 1.3×1 m. The images were taken with the chemical sprayer moving forward at the speed of 0.6 km h −1 . Gerhards and Christensen (2003) used three digital bi-spectral cold mirror cameras mounted in the front of the sprayer to detect and mapping weeds. The images were taken every 2 meters when the sprayer was moving at a speed of 7-8 km h −1 . They built up herbicide treatment map based on the collected data and the weed mapping also helps to understand weed-crop interactions and population dynamics of weed species through more than ten years study (Gerhards, 2010;Gerhards et al., 1993;Gerhards and Oebel, 2006). A proximal weed mapping Science Publications JCS system was developed by Sui et al. (2008) to measure weed intensity and distribution in a cotton field. The system includes WeedSeeker PhD600 sensors (Sui et al., 2008) and a GPS receiver with the dynamic position accuracy of 0.1-0.3 m. The travel speed of the sprayer was about 9.7 km h −1 and data were collected every second. The weed mapping system was evaluated in a commercial cotton field over 2 years and the test result showed that this weed mapping system had provided a useful tool for weed control project. Wiles (2011) developed a software to quantify weed cover in fallow fields with digital images. The camera was mounted on a vehicle with 1.2 m above the ground to collect images with the resolution of 2048×1360 pixels. The driving speed was 8-11 km h −1 and images were acquired every 4s so images represented approximately 1.0 square meter and were taken 8 to 6 meters apart. GPS coordinates were collected continuously at 2 sec intervals while images were collected. A map can be generated from 1000 images and 5000 GPS coordinates in 30 min. Weed cover was estimated with 90 to 96% accuracy for images. Zhang et al. (2012) investigated the effects of the weed seasonal variability by groundlevel weed sensing and mapping using hyperspectral images in the visible and NIR range (384-410 nm). With the image acquisition rate set at 7 frames sec −1 and moving speed of 36 mm sec −1 , the camera was set up to have the field of view 2.5 mm (in the travel direction) by 108mm (across the seed line) at a height of 203 mm. The camera scene was illuminated by a controlled light source. The disputed regions by various weed species were presented as weed map. Silva Junior et al. (2012) developed a machine vision system to map the weed density in a 0.8 hectare experimental field. They used both colour and near infrared cameras with the resolution of 480×640 pixels and have the field of view 4.9×6.5 m. Images were taken every 6.5 m approximately.
The platforms and the sensors play the important role for weed mapping. Table 1 makes a summary of the commonly used platforms and sensors for weed mapping in the recent researches. The platforms and the sensors decide the quality of the data and the resolution of the weed map. Resolution directly decides the quality of the weed mapping. Studies have demonstrated that the potential infestation or target area decreases greatly as the resolution becomes finer. Wallinga et al. (1998) estimated the herbicide usage of an idealized patch sprayer, which could detect and spray all units of area containing G. aparine plants. At a resolution of 4 m 85% of the field would be sprayed, at 1 m resolution only 41% would be sprayed, while at 0.5 m resolution only 26% needed herbicide application.
Beside the platforms and the sensors, another important factor which determines the accuracy of the weed mapping is the interval of the sampling points and the interpolation methods. No matter what type of weed mapping methods, to collect continuous and precise data is one of the most challenging issues. In most studies, discrete weed mapping was applied in a regular sampling grid. The field between the grid points may remain unsampled and interpolation techniques are used to estimate presence or density at the unsampled area. The assumption is made that in the broad-scale trends, the widely spaced sampling points are continued between sample locations. Interpolation methods used for weed mapping include linear triangulation, polynomial and kriging interpolation (Rew and Cousens, 2001). While the interpolation is a tradeoff between the accuracy of the weed map and the speed of data collection, the patches of weeds occurring between sampling points are often missed and even the most advanced mathematical interpolation methods will not be able to compensate for this.
In summary of the literature review of the weed mapping method, there are wide range of weed mapping methods have been developed and the type of platforms, weed sensors, resolution, sampling speed and sampling interval and the interpolation method are quite different. The weed mapping methods chosen to create the map are often be influenced by the end uses, the target species and farming system. There is a tradeoff between the accuracy, the speed and the cost when choosing the weed mapping method. For the weed management of the no-tillage broadacre cropping land, it lacks suitable weed mapping methods which can accurately and efficiently provide weed distribution information in the fields. The gaps exist on three aspects. Firstly, it lacks of proper weed sensing and data processing methods to distinguish the weeds from the no-tillage background. Secondly, the sampling speed is too slow to meet the requirements of the broadacre farming fields. At last, the resolution of the data is not fine enough to provide necessary detail information of the weeds. This proposed research focuses on fill the gaps in the three aspects.  (Himstedt et al., 2012;LOPez-Granados, 2011;Sui et al., 2008), 5: (Samseemoung et al., 2012), 6: (Tang, 2002), 7: (Gerhards, 2010), 8: (Sui et al., 2008), 9: (Zhang et al., 2012

MATERIALS AND METHODS
The objective of the proposed project is to develop a proximal Machine Vision System (MVS) for off-season weed mapping in broadacre no-tillage fallows. In the notillage cropping land, beside the soil, the background also includes plenty of straws, gravels and other dry plants. The authors studied most of the proximal weed sensing method using machine vision system and the result shows that there are no green plant segmentation methods, which have been developed in the previous researches, are suitable for the detection of green plants in the no-tillage farming environment under the natural sun light conditions. A new green plant image segmentation method called Hue-NIR-R method has been developed by the authors in the previous works. Hue-NIR-R method uses the combination of the indices of hue in HSI colour space, red in RGB colour space and near infrared to separate the green plants from the background and the experiments shows that this method is more suitable for the no-tillage background. For the detail information of the Hue-NIR-R method for green plant segmentation, the readers can refer the publication of Liu et al. (2013). At the current stage, the specific objectives include three aspects: (1) Develop an image acquisition method which is suitable for the broadacre cropping land with no-tillage background and different sunlight conditions. The speed of the image acquisition should be quick enough to meet the requirements of the broadcare cropping land. (2) Develop an image stitching method to stitch the collected images to high resolution photomosaic which can give overall view of the cropping lands.
(3) Develop an image analytical method to process and analyse the high resolution photomosaic and present the necessary information of weed on the weed map. This section presents the weed mapping methods of the three aspects in detail.

Image Acquisition
In the previous works of Liu et al. (2013), the cold mirror system was used as weed sensor for image acquisition. With the purpose of making the MVS work reliably in the natural farming environment, the cold mirror system was replaced by the JAI AD-130GE camera. AD-130GE is a prism-based 2-CCD progressive area scan camera capable of simultaneously capturing visible and near-infrared light spectrums through the same optical path using two individual channels (Fig.  1). The first channel has a Bayer mosaic colour image that captures visible light at 400 to 700 nm, while the second channel has a monochrome sensor for capturing near infrared light at 750 to 1000 nm. The images from the two channels are pairwise registered. The camera can capture images at the rate of 31 frames per second with the full resolution of 1296 (h) ×966 (v). The lens LM4NC3 is a wide angle lens with the angle of view 64.5×49.2 degree. The focal length is 4 mm and the iris range is 1.8 to 16. In natural outdoor lighting conditions, direct sunlight could cause plant leaves with glaring surfaces thus causing saturated pixels (Tang, 2002). A polarizing filter was used to reduce part of the glare. The AD-130GE camera and a notebook were connected to a Gigbit switch through the Ethernet cables. JAI SDK software combined with Matlab 2012b image acquisition tool box were used for data collection (Fig. 2).
The images were collected on 20 May, 2013 and 01 Aug, 2013 in Roseworthy test field in South Australia and both of the days were sunny. The camera was mounted on a frame which was fixed on the top of a vehicle as shown in Fig. 2 left. The height of the camera was place at 2.35m and lens was orthotropic to the ground. The field of Field Of View (FOV) was 2965×2151 mm and the pixel resolution was 2.29×2.23 mm. The parameters need to be adjusted during image acquisition are focus and exposure time. Focus need to be adjusted manually before the starting of the image acquisition. AD-130GE supports the exposure time from 11.49 us to 31760us and it support auto exposure functions. For the purpose of check the image response to different exposure time and iris, the exposure time was adjusted manually during image acquisition depending on the weather condition. However the parameters would be automatically adjustable once the research scheme is fully developed.

JCS
The image acquisition speed was set to 5 and 10 frames sec −1 at different tests. For the purpose of image stitching, it needs approximately half of the adjacent frames to be overlapped from each other therefore the maximum image acquisition speed at 10 Hz is 38.7 km h −1 . The vehicle was driven at the speed of 5, 10 and 30 km h −1 to test the image quality. At 30 km h −1 the image quality decreases slightly with blur. The captured visible videos and near infrared videos were saved on hard disk in uncompressed AVI format for further processing.

Introduction of Image Registration and
Image Stitching Tang (2002) and Tang and Tian (2008) developed a MVS to measure the plant spacing uniformity by using the image stitching method and this method indicate the possibility that image stitching method could be used for mapping weeds in an accurate way. There are wide image stitching methods developed in the recent years and it is necessary to make an introduction of the image stitching technology. However this topic is too big to be covered in this study therefore this study provides only a brief review of the image registration and image stitching technologies.
Using the image stitching technologies, images can be 'stitched' into a seamless photomosaic which can be used to produce digital maps, satellite photos and beautiful ultra wide angle panoramas. Based on the review, the procedures of the image stitching algorithm are summarized to five steps: (1) Feature detection and description; Salient and distinctive objects (closedboundary regions, edges, contours, line intersections, corners) are manually or automatically detected. The information of the neighbourhood of the detected feature points are described as the form of descriptors which make the feature points distinctive from each other. (2) Feature matching; the pairwise correspondence between the two set of points using the descriptors of the features will be determined in this step. (3) Image transform model estimation; The type and parameters of the socalled mapping functions, aligning the target image with the reference image, are estimated. The parameters of the mapping functions are computed by means of the established feature correspondence in step 2. (4) Image transformation; the reference image and target image are transformed into a same coordinate by the means of the mapping functions. (5) Image interpolation and image blending; the values of the pixels in the stitched image are estimated by using the value of the pixels in the reference image and target image.
Image registration is the core task of image stitching algorithm and it covers 1 to 4 steps of image stitching algorithm. Image registration is a method of aligning two images into the same coordinate system, so that the aligned images can be directly compared, combined and analysed. Image data may be multiple photographs, data from different sensors, from different times, or from different viewpoints. One of the images is referred to as the reference image or the base image and another image is referred to as the target image or the sensed image (Brown, 1992;Goshtasby, 2005).
Image registration algorithms can be classified into feature-based and area-based (Goshtasby, 2005). Featurebased methods establish a correspondence between a number of salient features such as closed-boundary regions, edges, contours, line intersections, corners. Knowing the correspondence between a number of salient features in images, a transformation is then determined to map the target image to the reference images, thereby establishing point by point correspondence between the reference and target images (Goshtasby, 2005;Myronenko and Song, 2010). Featurebased matching methods are typically applied when the local structural information is more significant than the information carried by the image intensities. They allow registering images of completely different nature and can handle complex between-image distortions. The common drawback of the feature-based methods is that the respective features might be hard to detect or unstable in time. The crucial point of all feature-based matching methods is to have discriminative and robust feature descriptors that are invariant to all assumed differences between the images. Area-based methods, sometimes called correlation-like methods or template matching, do not need the step of feature detection. Instead of extracting the salient features, these methods merge the feature detection and matching processes. The correspondences are calculated using a predefined window or even the whole image. The classical region-based methods, like cross-correlation or the Fourier transform, are based on matching of image intensities (Fan, 2011). From the geometric point of view, only shift and small rotation between the images are allowed when using area-based methods. Feature-based methods are often used in applications where the images contain enough details that are distinctive and easy to be detected. However, if the image is not rich in detail, or the detail is unsharp and hard to be detected, one may resort to area-based algorithms.

JCS
In this proposed research, because feature-based image registration is more robust to image rotation and distortion and the images used have distinctive features, the feature-based image registration is selected for the image stitching algorithm. Bay et al. (2008) presented a novel scale and rotation-invariant detector and descriptor called SURF (Speeded-Up Robust Features). SURF method is of combination of feature detection and feature description. SURF can distinguish invariant features from images that can be used to perform reliable matching between different views of an object or scene and it outperform the previous feature detection and feature description methods including phase-based local features (Carneiro and Jepson, 2003) and SIFT (Lowe, 2004;Mikolajczyk and Schmid, 2005).
With the matched feature points in the reference image and target image, the image transformation model, which defines how the target image is transformed to align with the reference image, can be built up. The image transformation model is defined by a mapping function which can be simply expressed as a matrix multiplication (Dubrofsky, 2009). The mapping function of affine transformation, which is one of the most common image transformation types, can be defined as: There are many techniques are available that can estimate the transform model based on the matched points on two images such as minimum spanning trees (Zahn, 1974) and convex hull edges (Goshtasby and Stockman, 1985). Goshtasby (1986) develop a mapping method which is outperforming the previous works by dividing the images into triangular regions then a linear mapping function is obtained by registering each pair of corresponding triangular regions in the images. However, not all the matched points can be used for the model estimation because local descriptor matching may produces false matches. The Random Sample Consensus Algorithm (RANSAC) proposed by Fischler and Bolles (1981) is a general parameter estimation approach designed to cope with a large proportion of outliers in the input data. RANSAC is capable of interpreting and smoothing data containing a significant percentage of gross errors and is thus ideally suited for applications in automated image registration based on the data provided by error-prone feature detectors. Iteratively, RANSAC picks a random subset of matches from the putative match list and a transform model can be built up based on the matches. Then, the transform model is tested against all the other correspondences in the putative match list. Correspondences that fit the model are considered as hypothetical inliers. Those that do not fit are considered as hypothetical outliers. After a fixed number of iterations, the model with highest number of hypothetical inliers is selected. A disadvantage of RANSAC is that it may require many iterations before it can find a correct hypothesis (Bhattacharya and Gavrilova, 2012). Torr and Zisserman (2000) developed a new robust estimator called Maximum Likelihood Estimation Sample Consensus (MLESAC) which is a generalization of the RANSAC estimator. It adopts the same sampling strategy as RANSAC to generate putative solutions, but chooses the solution that maximizes the likelihood rather than just the number of inliers. It is demonstrated that the method gives the results equal or superior to those of the previous approaches. The MLESAC is selected as the image transform model estimation method in the automatic image stitching algorithm. The MLESAC can find the image transformation parameter T. With the matrix T, the target image and the reference image can be transformed into a same coordinate.

Automatic Image Stitching of Colour Images and Binary Images
Section 3.2.2 introduces the algorithm of the automatic image stitching of the colour images and the binary images. The Matlab 2012b image processing tool box and 2013a computer vision tool box are used for image processing. The saved video frames of visible images and near infrared images were further processed to make the photomosaic. When images are taken with wide-angle lenses, it is often necessary to model lens distortions such as radial distortion. This kind of distortion can be rectified by the corresponding Science Publications JCS rectifying functions (Woods et al., 1993). LM4NC3 lens has the in-built function that eliminates the fish eye distortion and the distortion is less than 5%. By observation of the images, it is found that this distortion can be ignored.
The developed algorithm can automatically stitch both of the colour images and the near infrared images into two high resolution photomosaic. The procedure of the image stitching algorithm including 6 steps described as below and the flow chart of the image stitching algorithm is shown in Fig. 3.

Step 1: Feature Detection and Description
The first frame in the visible video is selected as the reference image and the next frame as the target image. SURF method was applied to both of the reference image and target image to find the feature points. Maltlab detect SURF Feature and extract Features are two functions to implement the SURF method. The function returns the feature descriptors which are vectors with the length of 64. The descriptors include all the information including the position, direction and neighbourhood intensity information of the detected features.
The feature point searching in the reference image is limited in a predefined region which is at the top of the reference image and the size of the region is the size a single video frame. This has two advantages. Firstly, the predefined region is the maximum possible overlap region and all the matched feature points in the reference image should be located in this region. Instead of searching the feature points in the whole reference image, just to search the feature points inside the predefined region makes the feature points meaningful. From another aspect, all the computation of the feature searching is limited in the predefined area which is a constant size therefore it can reduce the computation time.

Step 2: Feature Matching
This step is to find the putatively matched points of the reference image and the target image. Euclidean distance matching is a popular point matching method for image registration. Euclidean matching method finds the Euclidean distance between the two N dimension feature point vectors and the sum of all such distances for a given pair of images gives the overall image proximity measure. The lower the sum, the greater is the image match (Chew et al., 1997).
Let the first feature point in the target image be DT 1 and the number k feature point in the reference image be DR k , where k = 1,2, 3…n, n is the total number of feature points of the reference image. The task is to find the DR k which matches to DT 1 . Each feature point has a corresponding descriptor which is a 64 dimension vector, therefore the descriptor of DT 1 can be presented as DT 1 (1), DT 1 (2)…DT 1 (64) and the descriptor of DR k can be presented as DR k (1), DR k (2)…DR k (64). The Euclidean distance E of DT 1 and DR k is: where, k = 1,2…n is the number of the feature points. If the number k feature point of the reference image returns the minimum value of E, the number k feature point of the reference image matches to the first feature point of the target image. Use the same method to find all the putatively matched feature points of the reference image and target the image.

Step 3: Image Transform Model Estimation
Image transform model estimation is realized by the estimate Geometric Transform function of Matlab. This function is an integration of the image mapping method developed by Goshtasby (1986) and image transform model estimation method developed by Zisserman and Hartley (2003). The function returns a transformation structure TFORM which is the parameter defines the mapping function of the reference image and target image. Fig. 4 shows an example the putatively matched point in step 2 and the inliers after image transform model estimation.

Step 4: Image Transformation
Once the image transformation model has been built up, the target image can be transformed to the coordinate of the reference image by using the imtransform function. Imtransform function can transform the image to a new coordinate according to the transformation structure TFORM. First of all, the imtransform function in Matlab and the TFORM structure are used to transform the target image to the coordinate of the reference image. Now the two images are in the same coordinate while the pixels of target image may outside the coordinate of the reference image (Fig. 5a). A new coordinate which can hold both of the reference image and the target image is built up and the reference image and target image are transformed into the new ordinate as shown in Fig. 5b.

Fig. 5. Transform the images into the same coordinate
Step 5: Image Interpolation and Image Blending In short, the image interpolation and image blend decide how the colour or grey level is presented in the stitched photomosaic and these technologies are usually used to remove the artificial seams in the stitched photomosaic. There are many image interpolation and image blending technologies available and depending on their complexity, the more accurate the more computation they need (SpecTIR, 2013).
The purpose of the image stitching algorithm in this proposed research is to make a high resolution photomosaic which can be further processed to present the information of the weeds, hence the pixle's geometrical position in the stitched photomosaic is critical while the visible seams in the photomosaic do not affect the accuracy of the weed map. In this algorithm, the intensity value of the pixels in the overlapping regions is the value of the pixel of the reference image. Even though this algorithm could leave visible seams on the stitched photomosaic, it is computation efficient and keeps the original information of the videos.

Step 6: Stitch Binary Images
One of the new features of the developed image stitching algorithm is that it can automatically and simultaneously stitch both of the colour im.age and the binary image. The developed algorithm can generate two high resolution photomosaic, one is colour photomosaic and another is binary photomosaic. These two photomosaic is used interactively to compute and present the information of the weeds and this will be introduced in 3.2.3.
Firstly, the reference image and the target image are processed to generate two binary images using the Hue-NIR-R green plant segmentation method which has been

Photomosaic Processing to Present the Weed Distribution
The stitched colour photomosaic and binary photomosaic provide a convenient way for image processing, analysing and representation. The image processing and analysing are implemented in the binary photomosaic and the results are presented in the colour photomosaic. The weed are presented as white regions in the binary photomosaic therefore the weed mapping task is to find all the position of the white regions in the binary photomosaic and show the relative information in the colour photomosaic. The photomosaic processing methods all depend on the user's requirements and the specific conditions of the farming fields. The detailed processing methods are not covered in this study, while as an example, section 4.3 discusses the photomosaic processing method to make the weed maps. The whole procedure of the photomosaic processing work can be summarized as the Fig. 6.

Evaluation of Image Segmentation Algorithm
The algorithm of image segmentation has been developed in the previous work by Liu et al. (2013). This image segmentation method is compared with other three methods which have been widely used in the existing researches, they are: (1) NDVI method using NDVI index. This method has been widely used for remote green plant sensing (Gonzoearth, 2011); (2) NIR-R method using the index NIR-R which is the intensity of red subtracted from the intensity of near infrared. This method was used by Gerhards and Christensen (2003) to successfully map the weeds; (3) Hue method using the index of Hue, which was used by Golzarian (2009) for no-tillage wheat crop monitoring. Six sample images ( Table 2) are chosen to present different background and sunlight environments. All the image segmentation methods use the Otsu's method (Otsu, 1979) to automatically find the threshold and generate the binary images.
The segmented results are compared using both the visual perception and the type error estimation. The error types used are defined as type 1 error and type 2 error and this error evaluation method has been used for citrus and crop detection (Golzarian, 2009;Li et al., 2011;. Type 1 error is defined as the probability of the background pixels being classified as the weeds. Type 2 error is defined as the probability of the weeds being excluded as the background pixels. The total error is the weighted sum by the foreground and background respectively. The automatic segmented images are compared with the manually processed images template and the result is show in Fig. 7-9. The manually processed template is a weak tool for the evaluation of the error, however the primary error can still be quantified in the dominant type error. The type 1 error of Hue-NIR-R method is less than 5% which outperforms the other methods. The type 2 error of the Hue-NIR-R method is less or equals the other three methods. In the images with the uniform sunlight condition (image 1 to image 5), the total error of the Hue-NIR-R method is less than 10% which outperforms the other methods. For the image 6, all the methods have higher total error because part of the image has shadow. The partial shadow significantly decreases the image quality due to that the shadow area in the image lacks of proper exposure. The objects in the shadow have no information for the image segmentation which causes the errors. Some specific algorithm could resolve the problem of the partial shadow (Golchin et al., 2013), however with the consideration of the computation time, using artificial illumination which can provide uniform light density in the field of view of the camera is more feasible.

Evaluation of the Image Stitching Algorithm
Image stitching algorithm plays the most important role of this weed mapping method. The imaging stitching algorithm uses feature-based image registration method therefore the number of the matched feature points determine the reliability of the weed mapping. To check if the algorithms can detect enough feature points, 200 images are selected as sample images and the images are separated into four groups to present four different types of conditions. Each group include 50 images which are sequenced video frames. The background of the first group is soil (includes less dry plants), the second group is soil and gravels, the third group is dry straws and the last group is dry straws and shade. The 4 groups of image are stitched into 4 photomosaic and the number of the putative matched feature points after feature matching and the number of inliers after image transform model estimation are recorded and the results are shown in Fig. 10 and 11. Figure 10 shows that for each stitching, there are 4000 to 10000 putatively matched feature points can be found and Fig. 11 shows that after image transform model estimation there are less than 90 inliers left. The image transformation model estimation needs the minimum 3 inliers therefore if the inliers less than 3 the image cannot be stitched. This curve can explain the testing result. In the testing, the first two group images (red and green curve) can be stitched properly while the last two group images (yellow and blue curve) failed.

JCS
Since enough putatively matched feature points can be found in all the four types of images, one of the possible reasons could cause ill image transform model is the distribution of the feature points in the images (Yan et al., 2006). Good feature points should scatter evenly on the whole image while if the feature points gather to clusters in certain regions of the image, it could lost accuracy for image transform model estimation. In order to observe the distribution of the feature points, four images are randomly selected from the four types of groups and the inliers are marked by different colour to show the distribution the inliers (Fig. 12). Figure 12 shows that if the images have the background of soil with few dry plants and gravels, the inliers scatter evenly in the whole images and this contribute to the higher possibility of the right image transform models. While if the background has many straws, instead of scattering evenly, the inliers tend to gather together alone the lines which are the rows of the straws. The clustered feature points could create a wrong image transformation model and the reference image and the targe image could not be registered.
In summary of the experiment result, SURF method can detect enough putatively matched feature points in the images captured in the natural fallow environment under different background and sunlight conditions. Based on these putatively matched feature points, the image could be stitched properly under limited conditions. If the number of the inliers after the image transform model estimation is less than 3 or the distribution of the inliers is uneven, the images could not be stitched properly. Under this condition, the region based image registration method could resolve this problem and this will be studied in the further work.

Discussion of the Photomosaic Processing to Make the Weed Maps
This section discusses the photomosaic processing to make the weed maps. There are many types of weed maps and the information shown on the weed maps all depend on the user's requirements and the specific conditions of the farming fields. Just as an example, this section discusses three kinds of information presented as three layers on the weed maps and they are: layer 1: Binary weed map, layer 2: Colour weed map and layer 3: Weed density map (Fig. 13).
At first, a Cartesian coordinate is established with the origin at the top left corner of the photomosaic. In this Cartesian coordinate, the unit of the distance changed from pixels to meter. The binary photomosaic and the colour photomosaic can be processed to make three layers of the weed maps. Layer 1 is the binary weed map which gives the clearest information of the weeds (white pixels) against the background (black pixels). Layer 2 is the colour weed map which provides the true scenario of the cropping land in colour images. By the processing of the binary photomosaic, many detail information of the weed can be found such as the boundaries and the centroids of the weeds and these important information can be shown in the colour photomosaic. Layer 2 provides users a convenient way to check the detected weeds in the colour photomosaic. Layer 3 is the weed density map which shows the density of the 'green' in each grid with a certain size. The example shown in Fig.  13 has the grids with the size of 23×23 cm which is close to the range of spraying of the herbicide nozzle. Layer 3 gives uses a clear view of the weed density distribution in the field and it could be used as a powerful tool for making the weed control plans. Layer 3 also could be used as an herbicide prescription map for spot spray action if the GPS position of each grid is available.
Use the method described in section 3.2.2, all the videos can be processed to corresponding photomosaic and the whole map of the cropping land can be built up. When doing the image acquisition, it is difficult to drive the vehicle alone a very straight line and this could cause errors between the rows in the weed maps. The adjacent rows could have overlaps or gaps and this could cause over count the weeds or missing the weeds (Fig. 14). The best method to avoid this type of error is to control the route of the image acquisition and makes the route as straight as possible. If the error cannot be ignored, the interpolation methods can be applied to estimate the weed density in the missing areas. In this specific case of this research, the missing area is at minimum level hence it is ignored.
The accuracy of the weed map should be evaluated from two aspects. The first aspect is the accuracy of the information of the individual weed, such as the size and shape. The second aspect is the accuracy of the position of the weeds. The error of the shape and size of the weeds should be evaluated using the same method described in 4.1 while due to the difficulty of making a standard template for checking the type 1 and type 2 error, this method is not feasible in this specific case. A weak evaluation method is used by human expert's visual perception. A sample video was selected with the background of soil, few dry plant and gravels and 100 frames was stitched to map the weeds in 45 m long fallow. The stitched photomosaic is checked with the original video frames by expert's visual perception. The photomosaic can provide the density and shape information of the weeds with some acceptable errors.
The errors occur at some of the overlap area of the photomosaic where pixels of the weeds cannot be registered properly and this changes the shape of the weeds. The error could up to 5 to 10 pixels. In the image processing, the regions smaller than 3 by 3 pixels are considered as noise and are removed therefore the weeds smaller than 3 x 3 pixels, which is 6.9 by 6.9 mm in real field, cannot be detected. To check the accuracy of the position of the weeds, first of all, the Cartesian coordinate of the photomosaic needs to be transformed to the geographical coordinate and then the position of the weeds in the geographical coordinate should be checked with the true GPS data collected in the fields. The transformation of Cartesian coordinate to geographical coordinate is the future work which needs to be done at the next stage and is not covered in this study.

CONCLUSION
This study introduces and discusses a proximal weed mapping method based on the machine vision technologies. Most of the current weed mapping systems use discrete sampling and interpolation method to make weed maps and the accuracy of the weed maps are quite different depending on the weed sensors, sampling and data processing methods. The objective of this proposed research is to develop a weed mapping method which could map weeds in an accurate and efficient manner for the weed control in broadacre no-tillage fallows. The developed weed mapping method includes three aspects: (1) image acquisition; (2) automatic image stitching of colour images and binary images; (3) photomosaic processing to present the information of weeds. The main feature of this weed mapping method is that the image stitching algorithm can provide the continuous information of the weed in the field and interpolation method is used at the minimum level or not necessary. The relative position of the weeds is calculated in the stitched photomosaic. Compared with the normal weed mapping methods which totally depend on the GPS to record the position of the sampling point, this mapping method provides a more accurate way to present the position of the weeds. Another contribution of this weed mapping method is that it has the potential to be used for the measurement of the plant spacing Science Publications JCS uniformity where the accuracy of the distance is critical (Tang and Tian, 2008).