Low Cost Shooting Simulator Based on a Single Board Computer

: This paper proposes the camera on weapon system used as the shooting simulator. The single board computer (Raspberry Pi module) is used as the main processor. The OpenCV is implemented on the embedded system to perform the image processing task. The aim of the proposed camera on weapon system is to detect the position of the markers on the target and the laser spot. The detected coordinates are sent to the personal computer using a wireless communication provided by a low power Zigbee protocol. The performance of the proposed system using two camera systems (the Web-Camera and the Raspberry Pi-Camera) are evaluated. Both systems achieve the higher performances in the shooting detection which is greater than 85%, the target detection error which is lower than 0.8% and the small displacement on the arrangement of the camera and the laser pointer which is lower than one pixel. The Raspberry Pi-Camera system has a fast processing speed of 27 fps, while the processing speed of the Web-Camera system is 17 fps.


Introduction
Shooting simulator is a common instrument used for shooting practice.The most popular one employs the vision system to capture the laser spot emitted by the shooter on the target.There are two techniques called single fixed stationary system and camera on the weapon system (Ladha et al., 2010).In the first technique, a single camera is placed in front of the target.The camera is connected to the computer for further processes.While in the second method, the camera is mounted on the weapon (Soetedjo et al., 2013;2014;Ladha et al., 2010;2013).The latter one has an advantage i.e., allowing the multi shooters shot on a single target.
On both systems, the most challenging task is to detect the laser spot and the target by the camera.To detect the laser spot, (Lapointe and Godin, 2005) employed the thresholding method in the intensity images.The RGB color thresholding methods were employed in (Liang and Kong, 2006;Soetedjo et al., 2013).While the HSV color thresholding techniques were employed in (Kim et al., 2007;Widodo et al., 2012).
The previous works employed the Web-Camera or USB camera connected to the computer.Then the image processing techniques are implemented on the computer.This arrangement is suitable for the single fixed camera system.However, it becomes impractical to be implemented on the camera on weapon system, due to the USB cable connection between the camera on the weapon and the computer.To overcome the limitation, our previous work (Soetedjo et al., 2014) proposed the embedded camera system (CMUCam4) equipped with the wireless system.However, since the embedded microcontroller is employed, it could do a simple color thresholding technique only.
In this study, we extend our previous work by adopting a single computer board (Raspberry Pi) as the embedded system.Using this approach, the perfomance of image processing task could be increased, while maintaining the simplicity of the hardware system.To provide the flexibility of the weapon against the wired connection, a wireless communication is introduced between the embedded system and a personal computer.Two types of the camera systems, i.e., the Web-Camera and the Raspberry Pi-Camera are tested in the experiments.

System Archictecture
Figure 1 illustrates the architecture of proposed system.The main components are: (a) The shooting target, (b) the camera on weapon system, (c) the computer system.The shooting target consists of a target screen where the image is projected on it by a LCD projector.The camera on weapon system is a gun (artificial gun) equipped by the camera and the single board computer system.The computer system is a personal computer used to monitor and calculate the shooting score.The wireless modules are employed on the camera on weapon system and the computer system for providing the wireless communication.
The camera on weapon system as illustrated in Fig. 1 consists of five components, i.e., single board computer, camera, laser pointer, wireless module and battery.The operation principle is described in the following.When the shooter aims the gun into the target, a camera installed on the gun captures the image of target.Once the shooter shoots, the laser beam is emitted from the gun to the target.

Fig. 1. System architecture
The captured image is processed by a single board computer for locating the target and laser beam on the image.Then the coordinates of detected target are sent to the computer wirelessly.The personal computer processes the detected coordinates by computing the homography transform and locating the laser hit point.

Hardware Module
The proposed camera on weapon system is implemented on a low cost hardware as described in the following.Two things are considered for implementing the hardware, i.e., low cost and the components are available commercially.
The hardware module consists of the following components: • Camera: Web-Camera and Raspberry Pi-Camera • Laser Pointer: Red laser pointer which is usually used for aiming on the air gun • Single Board Computer: Raspberry Pi module • Wireless Module: Xbee module • Battery: Power bank for smart phones Fortunately, those components and modules could be mounted easily on PS3 pistol which used as the artificial gun.Therefore the camera on weapon system could be assembled easily.

Camera and Laser Pointer
Two types of camera systems are developed.The first one employs the Logitech HD Pro Webcam C920 (Fig. 2a) and the second one employs the Raspberry Pi-Camera module (Fig. 2b).The Logitech Webcam is equipped with USB interface and the popular device driver.The important feature of the camera is the Carl Zeiss lens which is adopted for providing the brighter image.The other consideration is the dimension and physical arrangement which are suitable for mounting on the gun.The Webcam provides a still image with resolution of 4492×3328 pixels (15 megapixel).
The Raspberry Pi-Camera is a camera module designed for connecting to the Raspberry Pi using CSI (Camera Serial Interface) connector.The CSI bus offers the high data rates.The camera sensor has a still image resolution of 2592×1944 pixel (5 megapixel).Many laser pointers are available in the market.The common laser pointer is the one used for presentation purpose.Another type of laser pointer is the one used in the shooting sport or hobby (assembled on the air gun).In this research, we employed the second one.It has the wavelength of 532 nm and the power of 5 mW.

Raspberry Pi
The Raspberry Pi is a credit-card sized computer based on ARM processor.The recent model (Raspberry Pi Model B) has 512 MB RAM and an ARM1176JZF-S 700 MHz processor.There are two USB ports and one Ethernet port.The module is equipped with a RCA video out and HDMI out for interfacing to display monitor, audio out, SDCARD slot, General Purpose Input Output (GPIO) and micro USB power input.The module is suitable to be mounted on the gun.
One USB port is used for connecting to the USB camera, while the another one is used for connecting to the USB Wifi dongle.It is noted here that the wifi module is used for debugging only.The wireless connection between the Raspberry Pi and a personal computer employs the Xbee module as described in the next section.

Wireless Module
One benefit of our proposed camera on weapon system is the wireless connection between the gun and a personal computer.This wireless connection provides the flexible handling of the gun.The shooter is able to hold the gun freely.There is no restriction caused by the cable connection.There are two wireless modules employed in the system, i.e., XBee module and USB Wifi module.The Xbee module is the main wireless connection used by the system.The USB Wifi module is used for monitoring the Raspberry Pi remotely.While the Xbee module is employed to transfer the data from the gun to the personal computer.
The Xbee Pro Series 2 as illustrated in Fig. 3a is employed.It provides the Zigbee protocol which allows a very simple and reliable wireless communication.The module offers the low power consumption.
The Xbee extension shield as illustrated in Fig. 3b is adopted for attaching the Xbee module to the Raspberry Pi via GPIO pins.The extension shield is equipped with an RGB LED, which is utilized for shooting purpose as explained in the next section.

Software Module
Raspberry Pi requires the Linux based operating system.Fortunately, there are many packages available on Internet.The most popular one is Raspbian, which is adopted in the proposed system.The program is written with C++ language for accessing the camera, the Xbee module and performing the image processing tasks.Then it converts RGB color image to HSV color image.After color conversion, the red color thresholding is applied to detect the markers.The markers are four red solid circles painted on the border of target image.The markers are used to calculate the homography transform for correcting the camera view, due the LCD projection and the camera position on the gun.
When the four markers are detected by the camera, it will inform the shooter that he/she could start to fire the laser on the gun.This information is indicated by turningon the Blue LED on the gun.If the four markers are not detected, the Blue LED is turned-off, then the camera will capture the image again until the markers are detected.
Once the shooter fires the laser, the algorithm will detect the laser spot on the target using the red color thresholding similar to the marker detection.If it verifies that the laser spot is detected, then the Red LED is turnedon.Finally the coordinates of four markers and laser spot are sent to the personal computer wirelessly.It is worthy to note that by sending the coordinates only, instead of a whole image, the high speed communication is not required.Therefore the wireless communication between the camera on weapon systen and the personal computer could be developed easily using the Zigbee protocol.

Marker and Laser Detection
When capturing a rectagular target image using a camera, the captured image might be distorted by the LCD projection or the camera rotation.In our application where the camera is mounted on the weapon, the camera moves along the gun.Therefore the above distortion could not be avoided.
To provide the correct image projection, the homography transform should be computed.Our approach employs four known points for computing the homography matrix as employed in (Widodo et al., 2012;Ladha et al., 2013).Four points are taken from the center coordinates of four red markers as illustrated in Fig. 5.To speed up the process, the homography transform is computed on the personal computer.While the camera on weapon system detects the coordinates of four markers and sends them wirelessly.
In this research, the most popular computer vision library called Open CV is employed to perform the image processing tasks.Fortunately, the OpenCV is also availabe for the Raspberry Pi running under Raspian operating system.
Detecting the markers involves two steps i.e., the color thresholding and the contour finding.The color thresholding is used to find the binary objects (blobs) according to the certain color criteria.In this case the hue, saturation and value of the HSV color space.If the color of pixels satisfy the criteria, they are painted in the white color, otherwise the pixels are the black color.After the black and white image is obtained, the countours of detected objects (white objects) are found.To verify that the contours are the red markers, the numbers and the areas of contours are checked.Namely, the numbers of contours should be four and the areas of contours should fall in a certain ranges defined empirically.
To detect the laser spot, the similar red color thresholding is employed.Since the area of laser spot is smaller compared to red marker, the laser spot could be distinguished easily.Further verification is done by calculating the position of laser spot in the image, i.e., the position of laser spot should fall inside the bounding box determined by four red markers.

Monitoring Software
In our proposed system, although the single board computer is attached on the weapon, the shooting monitoring is carried out by a personal computer.This approach has several benefits such as: (a) The coach could monitor or examine the trainee's shooting performance effectively; (b) The shooting hit points could be recorded for further analysis; (c) The GUI application software could be developed attractively on the personal computer; (d) Since data is sent wirelessly, the monitoring computer could be placed far away from the shooting room, e.g., in the coach room.
In our research, the monitoring software is developed to display the detected target and the laser spot and to compute the homography transform.The goal is to find the normalized target image, i.e., projecting the detected markers and laser spot into the reference target image as illustrated in Fig. 6.
The reference target image is an image of the shooting target.The image contains the target picture such as human, animal, circle, etc. and four red markers as illustrated in Fig. 6a.This image is projected by the LCD projector into the target screen.As described previously, the camera on weapon system does not send the whole captured image to the personal computer.Instead, only the detected coordinates of the four markers and the laser spot are sent.The coordinates are received by the personal computer and displayed as illustrated in Fig. 6b.To reconstruct the detected target image, the homography transform is employed.Let x is the coordinates of point in the reference target image.The coordinates of point in the reconstructed image (x') is expressed as: where, H is the homography matrix, which is solved using four points of the markers.All points in the reference image are mapped to the new image using Equation 1.The resulted image called as the reconstructed target image is illustrated in Fig. 6c, where the target picture is not a rectangular, but it appears as a parallelogram conforms with the detected markers.
Using the reconstructed target image, we could display the laser spot on the target picture as the real captured image, although the whole captured image does not sent to the personal computer.However to locate the laser spot or score the hit point accurately, the laser spot should be mapped onto the reference target image.Fortunately, it could be done easily using Equation 1 by inverting the homography matrix H.The resulted image is called the normalized target image as illustrated in Fig. 6d.

Experimental Results
To verify the proposed system, several experiments are conducted.Two types of camera systems are examined during experiments.The first type employs the Logitech Web-Camera, while the second one employs the Raspberry Pi-Camera as desrcibed previously.
During the experiments, the Raspberry Pi is operated remotely through a Putty software using the Wifi connection.This method allows the image captured by the Raspberry Pi to be sent to the personal computer for comparison purposes.
The objectives of experiments are to measure the performance of proposed techniques, which comprise of the following three factors.First is the success rate of shooting detection.It is measured by firing the weapon several times until ten shots are detected.The Success Rate (SR) is defined as: The shooting is considered as detected when the Blue LED and the Red LED are turned-on.
Second is the target detection error, which is defined as the discrepancy between the reference target image and the normalized target image.It measures how accurate does the system detect the target and perform the projection into the reference target.The Target detection Error (TE) is computed using the following Equations 3-7: Third is the deviation of the camera displacement.Since the camera and the laser pointer are installed on the weapon in a fixed position, the position of detected laser spot on the camera's coordinate should be constant.This arrangement could be used to detect the laser spot directly.Although in our approach we don't use this approach, the measurement of camera displacement is conducted to verify the laser spot detection as follow.In the experiments, we record the coordinates of detected laser spots using the techniques described before, for all weapon shots.Then the mean and the standard deviation are calculated for all collecting data.By assuming that there are no displacement between the camera and the laser pointer during the experiments, the standard deviation could be used to measure the accuracy of laser detection.
Ten reference target images as illustrated in Fig. 7 are tested during the experiments.For every target, two shooting scenario are examined.In the first scenario, the shooter holds the weapon in a fixed position and shots a fixed hit point on the target several times.While in the second scenario, the shooters moves the weapon to shot the random hit points on the target.

Detection Using the Web-Camera
Figure 8 illlustates the shooting detection results using the Web-Camera for fixed hit point and random hit points scenarios.The average Succes Rates (SRs) are given in Table 1.The average succes rate is calculated using Equation 2 for all ten reference target images.From the table, it is obtained that SR of the fixed hit point scenario is better than the one of the random hit points scenario.In the random hit points scenario, when the shooter moves the weapon to shot the different hit point, the shooter must wait until the Blue LED is ON before firing.It ensures that the four markers are being detected.Thus the lower SR of the random hit points scenario is caused by the laser spot detection error, not by the marker detection.From the observations, this error is affected by the different position of the laser spot on the target images that yields the different color of laser spot.For instance, the color of laser spot captured by the camera varies when the laser hits the black, blue, white colors of the background.
The experimental results of target detection errors using the Web-Camera for fixed hit point scenario and random hit points scenario are illustrated in Fig. 9 and 10 respectively.In the figures, TE-TL, TE-TR,TE-BL, TE-BR and TE are the detection errors as defined by Equation 3-7.The target detection errors for both scenarios are plotted again in a graph as illustrated in Fig. 11.While the average target detection errors for both scenario are given in Table 2.    2, it is clearly shown that the target detection errors for both scenarios are almost the same.Compared to the previous laser detection error, where the random hit points scenario yields the greater error, the target detection error is not affected by the shooting scenarios.It could be realized from the fact that in the target detection method, both scenarios use the fixed markers.Further it could be stated that the target detection method achieves a very small error about 0.6%.
Figure 12 and 13 illustrate the x-coordinate and ycoordinate of the detected laser spot using the Web-Camera respectively.From the figures, it is obtained that both scenarios yield the small deviation on detecting the laser spot.By observing the mean and standard deviation given in Table 3, our proposed Web-Camera on the weapon sytem shows a good accuracy on detecting the laser spot.Further, the results verify that the arrangement of the camera and the laser pointer provides a very small displacement error.

Detection Using the Raspberry Pi-Camera
The shooting detection results using the Raspberry Pi-Camera are illustrated in Fig. 14, while the average succes rates are given in Table 4.These results are similar to the system using the Web-Camera, where the average success rate of the fixed hit point scenario is better than the one of the random hit points.The average success rates for both systems are almost the same.
The target detection errors of the system using the Raspberry Pi-Camera are illustrated in Fig. 15, 16 and 17.Both scenarios show a small error as illustrated in Fig. 17 clearly.The average target detection errors are also very small (about 0.8%, Table 5).It is almost similar to the one using the Web-Camera.
Figure 18 and 19 illustrate the x-coordinate and ycoordinate of the detected laser spot using the Raspberry Pi-Camera respectively.The coordinates are different from the Web-Camera system.It is caused by the different installations of the camera on the weapons.Observing the figures, both systems have a small deviation below than one pixel.
The means and the standard deviations given in Table 6 could be interpreted as the accuracy of the laser spot detection.Similar to the Web-Camera system, the Raspberry Pi-Camera system shows a good accuracy.

Results of Reconstructed Target Images
As discussed previously, our proposed technique is able to recontruct the captured image by utilizing the detected coordinates of the markers and laser spot.To speed up the process, both camera systems use the resolution of 320×240 pixels.Using this resolution, the frame rates of 17 fps and 27 fps are achieved for the Web-Camera and the Raspberry Pi-Camera systems respectively.

Conclusion
The camera on weapon system was developed for detecting the laser spot on the shooting target.The Raspberry Pi module is employed as the embedded system to perform the image processing tasks.To avoid the cable connection between the gun and the personal computer, the wireless communication using Zigbee protocol is employed.
The experimental results show that the proposed system is effective to detect the shooting target and the laser spot with the high succes rate, the low target detection error and the small displacement between the camera and the laser pointer.
In future, we will extend the research for handling the complex shooting scenario.Further the shooting analyzer software will be developed.

Fig. 6 .
Fig. 6.(a) Reference target image (b) Detected marker and laser spot (c) Reconstructed target image (d) Normalized target image TL , TE TR , TE BL , TE BR are the target detection errors calculated from top-left, top-right, bottom-left, bottom-right markers respectively; TLX(Y)n, TRX(Y)n, BLX(Y)n, BRX(Y)n are x(y)-coordinates of top-left, top-right, bottom-left, bottom-right markers of the normalized target image respectively; and TLX(Y)r, TRX(Y)r, BLX(Y)r, BRX(Y)r are x(y)-coordinates of top-left, top-right, bottom-left, bottom-right markers of the the reference target image respectively.

Fig. 8 .
Fig. 8. Shooting detection results using the Web-Camera

Fig. 13 .
Fig. 13.Y-coordinate of detected laser spot using the Web-Camera Figure 20 and 21 illustrate some reconstructed target images of the Web-Camera and the Raspberry Pi-Camera systems respectively.The figures on the left columns are the recontructed target images, while the ones on the right columns are the captured images by the Raspberry Pi.The figures show that the reconstructed target images are almost similar to the captured images, in this case the detected positions of the markers and the laser spot.

Fig. 20 .
Fig. 20.Reconstructed target images of the Web-Camera system

Table 1 .
Average SR using the Web-Camera

Table 2
Fig. 12. X-coordinate of detected laser spot using the Web-Camera