An Evaluation of Thermal Imaging Based Respiration Rate Monitoring in Children

Problem statement: An important indicator of an individual’s health i s respiration rate. It is the average number of times air is inhaled and exha l d per minute. Existing respiration monitoring methods require an instrument to be attached to the pati nt’s body during the recording. This is a discomfort to the patient and the instrument can be dislodged from its position. Approach: In this study a novel noncontact, thermal imaging based respiration rate measurement method is developed and evaluated. Facial thermal videos of 16 children (ag e: Median = 6.5 years, minimum = 6 months, maximum = 17 years) were processed in the study. Th e recordings were carried out while the children rested comfortably on a bed. The children’s respira tion rates were also simultaneously measured using a number of conventional contact based methods. Re ults: This allowed comparisons with the thermal imaging method to be carried out. The image capture rate was 50 frames per second and the duration of a thermal video recording was 2 min per child. The th ermal images were filtered and segmented to identif y the nasal region. An algorithm was developed to aut ma ically track the identified nasal area. This re gion was partitioned into eight equal concentric segment s. The pixel values within each segment were averag ed to produce a single thermal feature for that segmen t of the image. A respiration signal was obtained b y plotting each segment’s feature against time. Conclusion: Respiration rate values were automatically calculated by determining the number of oscillation s i the respiration signals per minute. A close correlation (coefficient = 0.994) was observed betw e n the respiration rates measured using the therma l imaging method and those obtained using the most ef fective conventional contact based respiration meth od.


INTRODUCTION
Existing respiration rate monitoring devices provide an estimate of respiration rate due to the complexities associated with measuring this physiological parameter (Murthy and Pavlidis, 2005a;2005b). Respiration monitoring devices can be classified in different ways, depending on their sensing types and the manner in which they are applied. A classification approach is by considering whether the sensing device makes contact with the subject's body (i.e., a contact method) or the measurements could be performed at a distance (i.e., a noncontact method). In the following two sections, a review of contact and noncontact respiration monitoring methods and the ongoing research to further develop noncontact methods are provided.
Contact respiration rate monitoring: Exhaled air is significantly warmer, contains more CO 2 and has more humidity as compared with inhaled air. These differences can be detected by suitable sensors to estimate the value of respiration rate. Temperature based respiration rate measurements use a nasal or oronasal thermistor. This gives a semi-quantitative estimate of respiration rate, but it suffers from a high incidence of thermistor displacement (Storck et al., 1996). The volume of exhaled air measured by a pressure transducer provides a mean by which respiration rate can be obtained (Shneerson, 2005;Lee-Chiong, 2006). The air pressure can be measured via nasal cannulae, mouthpiece or facemask. A limitation of this method is that some subjects may not feel comfortable with a pressure sensor (Tobin, 1988) and the collector can affect respiration by increasing deadspace. There is also the phenomenon of artificially increasing the respiratory rate for a period as the individual is aware of the measuring device.
A CO 2 sensing respiration rate measurement method has been reported by Folke et al. (2002). They indicated that subtle design changes in the collecting device could cause large differences in the sensor's performance.
Respiration process produces a distinct sound that can be detected using a suitable microphone placed near respiratory airways or over the throat. A respiration rate measurement system using this principle was reported by Werthammer et al. (1983).The sounds generated by snorting, speaking, crying, coughing negatively affected the operation of the system. A microphone placed on the neck enabled a respiration signal to be produced and provided a miniaturised wearable 2 respiration monitoring system (Corbishley and Rodriguez-Villegas, 2008).
Another respiration rate measurement technique is based on detecting the movements of the chest or abdominal wall using pressure sensing bands. Two bands can be used. The thoracic band is placed around the top of the thorax and passes under the arm pits. The abdominal band is placed over the abdomen at the level of the umbilicus. The bands are made from extendible/deformable conducting materials (either a very fine wire or a thin foil) to allow conductivity be maintained during stretching (Lee-Chiong, 2006;Blom, 2004). The changes in the lungs' volume alter the pressure detected by the bands and thus a respiration signal is produced.
Oximetry (SpO 2 ) based respiration rate measurement uses red or infrared frequencies to determine the percentage of hemoglobin in the blood that is saturated with oxygen. This percentage is called blood saturation or SpO 2 (KMS, 2006). A SpO 2 meter can simultaneously display the SpO 2 level as well as respiration rate.
A number of studies have demonstrated the feasibility of extracting respiration rate from plethysmograms. For example, in a study involving 14 infants, the frequency spectra of their plethysmograms indicated respiration rate related peaks (Wertheim et al., 2009).
Respiration signal can also be extracted from the electrocardiogram (ECG). The process is known as ECG-Derived Respiration (EDR) monitoring. It is believed that the ECG signal is affected by the respiratory cycle. The changes in the heart position relative to the measuring electrodes and the changes in the lungs' volume are reported to produce this effect (Mazzanti et al., 2003). An EDR monitoring has been reported using a single-channel ECG that did not have to be a pre-cordial lead (Ding et al., 2004). Respiration rate monitoring during sleep has been performed by analysing the amplitude and duration of the R-wave and QRS area of the ECG (Furman et al., 2005).
Noncontact respiration rate monitoring: In contact respiration monitoring, the subjects can be inconvenienced by having the sensing device in contact with their body. Also, the sensing device can be dislodged, interrupting the monitoring. There are also hygiene issues arising from attaching devices to the patients. Therefore, the development of noncontact respiration monitoring methods has been considered in a number of studies. A critical review of noncontact respiration monitoring in medical care is provided in (Folke et al., 2003). Ultrasound, radar, microwave, video image processing, optical image processing and thermal image processing were approaches reported to achieve noncontact respiration monitoring.
A respiratory signal has been produced by analysing the internal diaphragm motion using two dimensional ultrasound (Xu and Hamilton, 2006). In another study two ultrasound sensors were positioned in front of a subject's face at opposite locations (Hok et al., 1996). The variations in the differential transit time of the two ultrasound signals, reflected from the face, were processed to monitor respiration.
A low power 2.4 GHz Doppler radar with proprietary hardware and software enabled respiration rate measurement (Droitcour et al., 2009). The radar transmitted toward the subject's thorax at a distance of about one meter and the chest movements were detected by comparing the transmitted and received signals.
It has been possible to monitor respiratory related chest wall motions of subjects in bed, covered by a soft comfortable bedding, using a microwave system attached to the ceiling (Uenoyama et al., 2006). Breathing has also been detected using a continuous wave single frequency microwave sensor (Dei et al., 2009).
In a study fiber grating was used to produce bright infrared spots on subjects, while a charge-coupled device camera was used to capture the scene of the spots (Aoki et al., 2001). Respiration rate was monitored by processing the resulting patterns.
In an investigation by our group, a high speed desktop web camera, connected to a computer, was used to record respiration related chest and abdomen movements (Tan et al., 2010). Every two successive recorded images were subtracted to detect chest movements and to produce a respiration signal. Using this approach, the value of respiration rate could be obtained automatically in real time, but the subjects had to wear a striped shirt and needed to remain still during the recording. Chekmenev et al. (2005) used a thermal camera consisting of a focal plane array for a long-wave infrared (6-15 µm) sensor. They measured the temperature changes around the neck, carotid vessel complex and the nasal regions. These regions were identified manually in the recorded images. ECG and the respiration rate signals were extracted from the recorded images by performing wavelet transform. An infrared imaging based respiration rate monitoring is also reported by Zhu et al. (2005). A tracking algorithm that could follow three facial features was developed. These features were identified manually from the first image of the recorded video. Two features represented the warmest regions of the face. These were the areas between the bridge of the nose and the inner part of each eye (i.e., the periorbital region). Underneath these two features, the coolest region was identified. This was the apex of the nose. A respiration signal was extracted from this region and was used to produce respiration rate.
In this study, a method to automatically track the skin surface centred on the tip of the nose (respiration region of interest, ROI) in the thermal images was developed. A feature extracted from this region enabled a respiration signal to be produced. In the following sections, the methodology and results are discussed.

MATERIALS AND METHODS
Twenty children were enrolled for the study at a local hospital. The parents of the children were informed of the nature of the experiments and consented for their children's data to be used in the study. All thermal video recordings were carried out in parallel with a number of conventional contact based respiration monitoring methods. This enabled a comparison to be carried out between the thermal imaging method and the conventional contact methods. The conventional contact respiration monitoring methods either measured the temperature variations between the inhaled and exhaled air using a nasal thermistor, or detected thorax and abdomen movements using pressure sensing bands, or measured nasal pressure and transcutaneous CO 2 . During the thermal recordings, the nasal thermistor partially covered the upper lip, but this could not be avoided. The thermal video recordings were performed with the child resting comfortably in a bed. The recordings did not cause any form of distress to the children. Some children were awake during the recordings while others were sleeping. The recording room temperature was about 20° centigrade.
Of the 20 children included in the study, 16 breathed via nose, while the rest either breathed via the mouth. The 16 children that breathed via nose were included in this study, as the algorithm described in this study did not work as well when children breathed through mouth (further work is needed to deal with this issue).
The thermal sensitivity of the camera used in the study (i.e., FLIR A40) was 0.08° Kelvin. The camera was fixed on a tripod in front of the subject at a distance of about one meter. The camera settings were: the emissivity = 0.92°, reflected temperature = 15°C and relative humidity = 50%.
Images were recorded at 50 frames per second. The average respiration rate in an adult is about 15 cycles per minute. In infants, the average respiration rate could be as high as 60 cycles per minutes. Therefore, the image capture rate was sufficiently high. The duration of each recording was 2 min. This produced 6000 thermal images (i.e., 120 sec *50 images). The recorded images were processed off-line using the Matlab image processing tool box. The operations involved for processing the images are explained in the next sections. The time taken to process the images for one subject was about 20 min currently the image processing tasks are being optimised to reduce this time and to allow for real-time processing.
In order to generate a respiration signal and respiration rate, the thermal images had to be processed. A discussion of the operations involved is provided in Al-Khalidi et al. (2010). However, in the current study the ROI tracking method has been further improved to make it more robust. The operations involved to achieve this tracking are briefly described next.
The recorded thermal images were enhanced by using the median filter. This reduced their noise. The filter had a size of 5. The Prewitt operator was used to segment the facial boundaries from the image background. The Prewitt operator (Gonzales et al., 2004) is an edge detection method that searches for the edges in both the horizontal and vertical directions. The masks used were: x y 1 1 1 1 1 1 G 0 0 0 G 0 0 0 1 1 1 1 1 1 The masks were convolved with the images. The application of this operator to detect a subject's boundary is shown in Fig. 1.
After extracting the subject's boundary, the two warmest facial regions within the facial region were located. These warmest regions were the points where the corners of the eyes meet nose (Fig. 2).The coldest point underneath these warmest regions then represented tip of the nose. The area around the tip of the nose was considered as the respiration region. This tracking process however sometimes failed as during significant head movements a wrong warmest point was identified (Fig. 3).
The following changes were then made in order to further improve the tracking algorithm.
The images were thresholded to separate the subject's boundary from the image background. This operation was performed by considering the facial temperature distribution. The temperature of the image background was relatively lower than the temperature of a subject's head. A suitable threshold was 30°C and therefore this operation was performed as Eq. 1: where, f(x,y) and g(x,y) represent image pixels prior and after the thresholding operation respectively.
The Prewitt edge detection scheme was used to identify the boundary of the subject's head in the thresholded images. The Prewitt masks were convolved with the images. Figure 4 shows an example of a subject's boundary obtained by performing this operation.
The next task was to select the image area covered by the subject's face. This was achieved by automatically superimposing an ellipse on each filtered image. The location and size of the ellipse were determined by performing the following tasks: • The highest (x max ) and lowest (x min ) pixels' locations of the head boundary in the vertical image direction were identified and the centre between these two locations (i.e., x 0 ) was determined • Centred at x 0 , the head boundary points in the horizontal image direction were identified, providing y min and y max . Then, the centre (y 0 ) between y min and y max was calculated • The ellipse semi-major axis (a, i.e., the longer axis) and the semi-minor axis (b, i.e., the shorter axis) were calculated by determining x 0 and y 0 to x min and y min Eq. 1 respectively and dividing the values by 2 • The ellipse's position and axes' lengths were determined using the ellipse Eq. 2: (2) Figure 5 shows the position of an ellipse on a thermal image. The next task was to identify and track a circular region centred on the tip of the nose (i.e., the respiration region of interest, ROI). To achieve this, initially the image area enclosed by the ellipse was automatically searched to identify the warmest point. This region corresponded to a small area between the bridge of nose and the inner aspect of the eye. The image section enclose within the ellipse, underneath the warmest region, was searched to locate the coolest region. The skin surface temperature of this region is most affected by respiration through the nose. The reason for first identifying the warmest region and then coolest region underneath it was to reduce the possibility of a wrong region being chosen, especially when there were large head movements. The coolest value within this region was determined. This value corresponded to the tip of the nose.
A circle was placed on the tip of the nose automatically. The radius of the circle was 1/16th of the number pixels in the vertical direction of the image. As the recordings were carried out with the subjects always at a fixed distance from the camera, the circle radius was constant. The circle covered the tip of the nose and the upper lip. The circled region (i.e., the respiration region of interest, ROI) was used to obtain the respiratory signals.
A process described by Saatchi et al. (2009) was applied to the ROI. Using this process, the circle was divided into eight equal concentric segments as shown in Fig. 6 The segmentation of the ROI into eight parts allowed for a more detailed temperature analysis to 5 is performed. The pixel values within each segment were averaged to obtain a single value representing that segment (other feature selection and extraction methods were also considered, but the averaging of pixel values proved most effective).
The above processes were repeated for each of the 6000 images (i.e., 120 sec recording duration 50 images per second) recorded from each subject. The averaged pixel values of the eight segments were then plotted against time to produce eight respiratory signals. In order to determine the respiration rate automatically, the respiration signals were digitally filtered using a 5th-order Butterworth filter with a cut-off frequency of 1 Hz. This cut-off frequency was sufficiently low for the respiration signal to be smoothed for further processing. It was sufficiently high to allow 60 cycles per minute to be detected. The peak-to-peak duration of each respiration cycle (T sec) was then determined. The respiration rate (in cycles per minute) was determined by first producing the average of all respiration cycles for each signal. Then, the reciprocal of this value was multiplied by 60 to obtain respiration rate in cycles per minute. This process is illustrated in Fig. 7.

RESULTS AND DISCUSSION
The respiration signals obtained from a child are shown in Fig. 8. For this child, the signals from segments 1, 2, 3, 7 and 8 are more recognisable than those from the other segments. This is because either the corresponding areas were affected more by respiration, or the areas were more clearly detected by the camera. The respiration rate obtained from segments 1, 2, 3, 7 and 8 was 27 cycles per minute. Figure 9 shows the respiration signal produced by averaging pixel values from all eight segments of the identified respiration region of interest. Because some segments do not provide a well defined respiration signal, averaging pixels across all eight segments produces a distorted respiration signal and therefore this signal was not used in our study. This observation highlights the importance of segmenting the ROI and using the best segment for obtaining the respiration rate value.
The respiration rate values obtained using the thermal imaging method was compared with those 6 obtained using the conventional contact based methods. The contact methods were based on temperature detection of the inhaled and exhaled air using a nasal thermistor, thorax and abdomen movement's detection using pressure sensing bands, nasal pressure and transcutaneous CO 2 . These contact based respiration monitoring methods operated in parallel with the thermal imaging method. The clarity (i.e., the ease that the respiratory cycle could be visually observed) from respiration signals produced by the each contact based method varied in different children and therefore, the specific contact based method that provided most distinguishable respiration signal for each child was visually selected and used. The contact methods only produced a respiration signal (i.e., not the actual value of respiration rate). In order to determine the respiration rate for contact methods, the number respiration cycles in the relevant signal had to be manually counted. Figure 10 shows the respiratory signals obtained using a, thoracic and Abdominal belts for the child shown in Fig. 6. The value of respiration rate manually determined from these signals was 27 cycles per minute. Table 1 shows the respiration rate results for the sixteen children included in the analysis. The table compares the respiration rates obtained using thermal imaging against those obtained from the most effective contact method for each child. The average respiration rate values across all children using thermal imaging and the selected contact method were 20.7 and 21.0 cycles per second respectively. The correlation coefficient (ρ), obtained using Eq. 3, for the two sets of respiration rate measurements was 0.994 (3 decimal places): The variables x and y represent the measurements from the two methods and x and y are their respective means. This indicates that the respiration rate values from the two approaches were closely correlated. Figure 11 shows a plot of respiration rate values obtained using the thermal imaging method against those obtained from the most effective contact respiration rate monitoring method. A close correlation between the thermal imaging and contact based respiration rate measurements is evident.
In order to show the variations of respiration rate over a time interval, a moving average of respiration rate values was produced. This involved averaging respiration rates within a predefined window containing five successive respiration cycles and then repeating this operation by moving the window forward by one cycle. An example of the results obtained using this approach is shown in Fig. 12. Although the average respiration rate for this plot is 27 cycles per second, there is a considerable variation around its mean value. Figure 13 shows the corresponding plot for all sixteen children. The average respiration rates for the children calculated automatically were: 23.3, 17.4, 21.6, 22.0, 16.7, 15.0, 15.0, 20.0, 18.9, 17.9, 19, 33, 15.4, 27.0, 24.0, 22 cycles per minute respectively.
The amount of the fluctuations shown in these plots may have useful clinical information in determining the consistency of respiration rate. Figure 14 shows the magnitude frequency spectrum of a typical respiratory shown in Fig. 8 (signal from the first segment of ROI).  The highest peak in the frequency spectrum occurs at 0.45 Hz. This corresponds to a respiration rate of 0.45 cycles per sec or 27 cycles per minute. This value corresponds well with that obtained directly from the respiration signal by determining the number of cycles per minutes. Figure 15 shows the magnitude frequency spectrum of the respiration signals obtained from all sixteen children (the signals were obtained from a segment of the ROI that provided clearest respiration signal).The highest peak in these frequency spectra occurred at 0. 39, 0.30, 0.35, 0.38, 0.28, 0.25, 0.26, 0.30, 0.31, 0.30, 0.316, 0.64, 0.26, 0.43, 0.40 and 0.37 Hz respectively. These corresponded to respiration rates of 23.4, 18.0, 21.4, 22.8, 16.8, 15.0, 15.5, 18.0, 18.6, 18.0, 19.0, 38.0, 15.6, 26.0, 24.0 and 22.0 cycles per minute.
The thermal imaging method developed in this study provided a noncontact method of determining respiration rate. However, there are a number of issues that require further development. There include: • The image processing was off-line due to the extent of calculations required. It is advantageous for the respiration rate to be determined in real time. This is currently being investigated. The software used for processing the images was Matlab. It took about 15 min to process the images for one subject. However, with optimizing the algorithms, this time can be reduced substantially • The current method operated well when respiration was through the nose only (not via the mouth or both). This is because the method could not automatically identify the mouth region. The approach requires further enhancement to be able it deal with both nasal and mouth breathings • The current method cannot handle very large head movements or when the subject is wearing glasses. When there are very large head movements, the region associated with respiration moves very close to the edge of the image, introducing errors in automatic detection of the respiration region. Glasses hide facial features. Again, further improvements are currently being sought to deal with these limitations • In this study, a commercial thermal camera (Flir A40) was used. A customised infrared measurement device, suitable for respiration monitoring, is currently being developed by our group. This would significantly reduce the cost of the instrument and makes the system more portable.

CONCLUSION
In this study a thermal imaging based method to automatically monitor the value of respiration rate was developed and its effectiveness was evaluated by applying it to sixteen children.
The method determined the skin surface temperature changes centered on the tip of the nose and tracked the region effectively. There was a close correlation between the respiration rate values obtained using the thermal imaging method and those obtained using the most effective contact based conventional respiration monitoring method.
The study indicated that thermal imaging was suitable for monitoring respiration rate. The main advantage of thermal imaging method is that, it is noncontact. Existing methods are contact based, thus requiring an instrument to be attached to the subject's body, causing discomfort.
The thermal imaging method explained in the study has a number of limitations. The study is currently ongoing to deal with these issues.