Comparison with the Results of other Regression Models
To demonstrate performance superiority of GRNN model proposed in this study, two other models, that is, linear regression and logistic regression, are developed for the same data and the results are compared and shown in Tables 5a and 5b for the training and testing sets, respectively. From Tables 5, one can see clearly that GRNN model performs significantly better than both linear and logistic regression models do.
Comparison with Results of other Studies
To better appreciate our model performance, results from our model are further compared with those from other studies (See Table 6). While all three other studies show relatively lower sensitivity (ranging from 74% to 85%), ours has higher sensitivity (88% and 95.7% for testing and training respectively). It is worth noting that since no requirement for sensitivity and specificity is given, the comparison among different studies is difficult.
Table 6. The comparison among various studies
|
|
FEIN |
Kapuniai |
Crocker |
This study |
This study |
|
|
st al |
et al |
et al |
(Training group) n=75 |
(Testing group) n=42 |
|
Method |
 |
 |
Logistic regression |
GRNN |
GRNN |
|
Sensitivity |
74% |
78"% |
85% |
95.7% |
88.0% |
|
Specificity |
93% |
67% |
61% |
50.0% |
52.9% |
|
AHI predicted |
 |
 |
 |
 |
 |
|
Probability cutoff-point |
NA |
NA |
 |
 |
 |
# According to the original respective authors, their methods were based on derivation from ascribing a point value to a number of clinical characteristics that have been indicative for OSAS.
Comments on Sensitivity and Specificity
It is well understood that designing a diagnostic system to achieve both high sensitivity and specificity is almost impossible in real-world applications. Thus striking a best trade-off between the two is the most practical solution. Obviously choosing such trade-off between sensitivity and specificity is problem-dependent. That is, there are no such thing as good numbers for sensitivity and specificity.
For instance, for diagnosing clinically significant ankle fractures, the Ottawa Ankle Rule had a specificity of only 50% but a sensitivity of 100%. Henceforth, not all patients will meet the decision rule criteria, but in their case of those who do, the necessity for an ankle radiograph can be disregarded. The Ottawa Ankle Rule has a diagnostic gray zone of about 70%, but in the field testing, it is estimated that the rule has reduced the necessity for ankle radiography by 30% (Siellet al) [20].
For another example, due to the severe morbidity associated with Obesity Hypoventilation Syndrome (OHS), some researchers selected a highly sensitive threshold of serum bicarbonate level. A threshold of 27mEq/l had a sensitivity of 92% and a specificity of 50%. Merely 3% of patients with a serum bicarbonate level <27 mEq/l had hypercapnia compared to 50% with a serum bicarbonate >27 mEq/l. In their conclusion, OHS is common in severe OSA. A normal serum bicarbonate level excludes hypercapnia and an elevated serum bicarbonate level should prompt clinicians to measure arterial blood gases. [21, 22]
There are several other instances in the literature of sleep medicine. For example, an Italian version of the Epworth sleepiness scale (ESS) is an easy-to-use form useful for preliminary screening of daytime sleepiness level in specialist settings. Noticeably, the (ESS) cut-off scores associated with the best sensitivity and specificity were set to be 12 and 17. For the 5-min multiple sleep latency test (MSLT) cut-off, sensitivities were 87% and 47% for cutoff scores of 12 and 17 respectively, andspecificities of 39% and 74%. For the 8-min MSLT cut-off, sensitivities were 84% and 49%, and specificities of 50% and 88%. [22]
In this paper our model achieves the sensitivity of 95.7% and specificity of 50.0 % for the training set, while 88.0%, and 52.9%, respectively, for the testing set. Specificity of equal or greater than 50% while sensitivity is higher than 85% indicates that our model developed in this paper is reasonable. Our sensitivity and specificity results are based on the decision threshold of 0.40, which can be tuned if costs associated with false positive and false negative errors are known.
Discussion
It is true that the widely recognized definition of sleep apnea has the round numbers of AHI cutoff = 5, 10, and 15. Conversely, our data did indicate that using AHI of 13 as cutoff is better than using 15. A brief review of the following mechanisms may help to understand the issues that we shall advance latter on.
The Reason Why We Selected AHI Cut Off of 13 Events per Hours is as Follows
While the accepted AHI cut offs for defining OSA severity is somewhat arbitrary, as suggested by the current authors and others, using an AHI cut off of 13 instead of 15 events per hour seems ‘unusual’. Nevertheless, this was used merely based on our previous clinical experience.
Furthermore, our selection of the criterion is clinically and not statistically. What is the actual clinical meaning of the two events represented by a cutoff of 13 (more precisely, 13.21) and not 15 is as follows.
Clinically, we do observe a better sleep efficiency (%) for those patients whose AHI were < 13.21 but not for those whose AHI > 13.21 (p=0.05). On the contrary, when we use AHI cutoff of 15, there is no difference between AHI <15 and > 15 these two groups. It can thus be stated that the employing of AHI cutoff of 13 is based on clinical finding of sleep efficiency (%) The employing of AHI cutoff 13 is not based on this GRNN modeling here.
OSA was defined by a polysomnographical AHI cut-off that has been considered by investigators to be clinically important. The cutoff of 13 for AHI does fall into the current definition of sleep apnea. Although the value was derived from the ROC, its clinical significance was particularly for symptomatic elderly patients. In our model, with AHI = 13 is obvious for the reason as follows. The cutoff value of 13 for AHI falls into the upper range of mild sleep apnea. In addition, there are the clinical significances of any particular cut off points between AHI 10 and 20 9. In fact, AHI=15 has never been the absolute and sole point of cut-off.
It all dependents on the nature of disease, and other clinical factors involved along with the setup of the device used in the measurement. For example, the cut-off value of estimated AHI set at 17, instead of 15, was optimal for the differentiation between patients with or without sleep related breathing disorder using the Lifescreen Apnea software from Holter ECG as an accurate, specific and sensitive method for the detection and classification of obstructive and mixed SRBD 19. According to their estimated AHI, 50 (68%) patients were correctly diagnosed. The ROC analysis showed high accuracy of SRBD detection using Est. AHI: AUC – 0.91 with sensitivity – 91.2%, specificity – 87.5%, PPV – 88.6%, and NPV -88.9%.
Cost Effectiveness Assessment of our Model
As shown in the section of RESULTS, with the decision threshold of 0.40, our GRNN model achieves the sensitivity of 95.7% and specificity of 50.0 % for the training set, as well as 88.0%, and 52.9% respectively. If a PSG sleep study were performed only in subjects for whom the model predicted an AHI > 13, the number of PSG required for the 42 subjects in our testing set would have been decreased from 25 to 22, a 12% (3/22) reduction in the number of PSG taken. To assess true cost effectiveness of our model, a cost-benefit analysis on the numbers of false positive an false negative case is needed, which can take into account the impact of quality of life on the subject (patient) and the family, and the financial impact on the community. in our model, the specificity attained of >=50% when sensitivity is high (>=88.0%), it is definitely indicative that the methodology used in our model is geared up for prime time.
It is important to note that our study conducted in this paper here is not a CPAP treatment study,even though it is indeed on diagnostic PSG. We intend to develop into a CPAP treatment study in future, with our aim in cost-effective purpose and are looking forward in seeing a potential savings.
With this model, we can determine if a subject with complaints of poor sleep and with an AHI<13 requires a CPAP treatment, which leads to an earlier therapic action without the need of a diagnostic PSG. .
Limitations of Study
Some clinical variables, such as TST, used in this model for estimating AHI, may require some device to measure the sleep time as accurate if possible. There is an exception to the aforesaid. For example, sleep latency, which has been removed as an independant (input) variable after the initial trial in this model, may not be accurately measured by actigraphy, not to mention by subjective reports. By comparison, data of the total sleep time except sleep latrency is some what easier to be measured than sleep latency per se. Devices such as simple monitors using web-camera with a home personal computer (PC) can serve the purpose. They are not expensive. Hence such measurements for TST are cost-eefective.
As compared to the portable home monitoring for OSA, the disadvantages follow. Generally, the set-up to be used at the examinees’ homes using portable recording with an ambulatory monitoring system recording nasal/oral airflow (thermistor) without electroencephalogram (EEG) or nasal airway pressure or pleural pressure measurements. By contrast, it is most important to note that in this context, the recording system that are described here for the ambulatory monitoring system indeed lacks to detect upper airway resistance.
On the other hand, the moderate specificity depicted in our model may suggest that when we use this model, the chance of co-applying other expensive extra tests is not indicated.
Although the economics of these additional screening, web-camera and P C, may not be disregarded, the expense of setting up a web-camera with the P C is really minimal, since often home P C is preexisting.
Using TST as a variable in this model is meaningful for the reason as follows. First of all, TST has been one of the ‘good’ (friendly) variables selected by the algorism used in this model. Nevertheless, one might still argue that it might be ‘meaningless to use a PSG variable such a TST to predict OSA’. At first glance, such a statement sounds ‘plausible’. However, it is neither logical nor acceptable. Unlike other sleep medicine variables, TST is one of the few that can be measured easily and precisely with web-camera and a home P C at home instead of the Sleep Medicine Laboratory. Second, whoever believes that PSG and actigraphy, other than PSG, are the only two precise ways in obtaining a record of an individual’s TST may be misleading. Third, TST, unlike sleep latency, can be used as an easily obtaintable clinical datum.
Conclusions
Generalization of this GRNN model over other populations in general medicine is possible. It all depends on the various combinations of influence among age, gender, weight, and BMI. They may attribute a significant explanatory power for the AHI, above and beyond what has been explained by the samples in the current study.
