Advertisement

Clinical evaluation of the rapid nucleic acid amplification point-of-care test (Smart Gene SARS-CoV-2) in the analysis of nasopharyngeal and anterior nasal samples

Open AccessPublished:December 30, 2021DOI:https://doi.org/10.1016/j.jiac.2021.12.027

      Abstract

      Introduction

      Smart Gene is a point-of-care (POC)-type automated molecular testing platform that can be performed with 1 min of hands-on-time. Smart Gene SARS-CoV-2 is a newly developed Smart Gene molecular assay for the detection of SARS-CoV-2. The analytical and clinical performance of Smart Gene SARS-CoV-2 has not been evaluated.

      Methods

      Nasopharyngeal and anterior nasal samples were prospectively collected from subjects referred to the local PCR center from March 25 to July 5, 2021. Two swabs were simultaneously obtained for the Smart Gene SARS-CoV-2 assay and the reference real-time RT-PCR assay, and the results of Smart Gene SARS-CoV-2 were compared to the reference real-time RT-PCR assay.

      Results

      Among a total of 1150 samples, 68 of 791 nasopharyngeal samples and 51 of 359 anterior nasal samples were positive for SARS-CoV-2 in the reference real-time RT-PCR assay. In the testing of nasopharyngeal samples, Smart Gene SARS-CoV-2 showed the total, positive and negative concordance of 99.2% (95% confidence interval [CI]: 98.4–99.7%), 94.1% (95% CI: 85.6–98.4%) and 99.7% (95% CI: 99.0–100%), respectively. For anterior nasal samples, Smart Gene SARS-CoV-2 showed the total, positive and negative concordance of 98.9% (95% CI: 97.2–99.7%), 98.0% (95% CI: 89.6–100%) and 99.0% (95% CI: 97.2–99.8%), respectively. In total, 5 samples were positive in the reference real-time RT-PCR assay and negative in the Smart Gene SARS-CoV-2 assay, whereas 5 samples were negative in the reference real-time RT-PCR assay and positive in the Smart Gene SARS-CoV-2 assay.

      Conclusion

      Smart Gene SARS-CoV-2 showed sufficient analytical performance for the detection of SARS-CoV-2 in nasopharyngeal and anterior nasal samples.

      Keywords

      1. Introduction

      Since its emergence in late 2019, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has spread worldwide [
      • World Health Organization
      Coronavirus disease (COVID-19) weekly epidemiological update and weekly operational update.
      ]. A quick and accurate diagnosis is essential for the clinical management of patients and infection control measures [
      Centers for Disease Control and Prevention
      Guidance for SARS-CoV-2 point-of-care and rapid testing.
      ]. The expanding pandemic has increased the demand for laboratory tests and large-scale testing. Facility-based platforms using a high-capacity automated testing system are advantageous for meeting this demand. However, the time to perform the test and transport the samples to the laboratory are barriers to obtaining results in a timely manner.
      A point-of-care (POC) test, which does not require extensive equipment or skilled technicians and which provides results in a short period, is beneficial for overcoming the disadvantages of large-scale laboratory testing. POC tests are helpful in situations like outbreaks in long-term care facilities, where an immediate diagnosis is needed [
      Centers for Disease Control and Prevention
      Guidance for SARS-CoV-2 point-of-care and rapid testing.
      ]. Antigen tests are quicker and easier to use than nucleic acid amplification tests (NAAT) and are widely used as POC tests. However, a recent systematic meta-analysis of antigen test performance in real-world settings revealed that sensitivity ranges from 28.9% to 98.3% according to demographic features, viral load, and symptom state [
      • Dinnes J.
      • Deeks J.J.
      • Berhane S.
      • Talor M.
      • Adriano A.
      • Davenport C.
      • et al.
      Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection.
      ]. Thus, rapid and more reliable diagnostic tools are required.
      Smart Gene SARS-CoV-2 (Mizuho Medy Co., Ltd., Tosu City, Saga, Japan) is a novel test kit for Smart Gene, an automated molecular testing platform that utilizes the quenching probe (QProbe) method. Because of its small size and fast turnaround time, Smart Gene can be installed in clinics and used for POC testing platforms, and has been used to diagnose pediatric respiratory infection of Mycoplasma pneumoniae [
      • Kakiuchi T.
      • Miyata I.
      • Kimura R.
      • Shimomura G.
      • Shimomura K.
      • Yamaguchi S.
      • et al.
      Clinical evaluation of a novel point-of-care assay to detect Mycoplasma pneumoniae and associated macrolide-resistant mutations.
      ]. In this prospective study, we evaluate the clinical performance of Smart Gene SARS-CoV-2 for nasopharyngeal and anterior nasal samples collected at a local PCR center.

      2. Methods

      In this study, we enrolled participants and collected samples at the PCR center in Tsukuba Medical Center Hospital (TMCH) between March 25 and July 5, 2021, which intensively collected nasopharyngeal samples for the PCR analysis of SARS-CoV-2 [
      • Takeuchi Y.
      • Akashi Y.
      • Kato D.
      • Kuwahara M.
      • Muramatsu S.
      • Ueda A.
      • et al.
      Diagnostic performance and characteristics of anterior nasal collection for the SARS-CoV-2 antigen test: a prospective study.
      ,
      • Takeuchi Y.
      • Akashi Y.
      • Kato D.
      • Kuwahara M.
      • Muramatsu S.
      • Ueda A.
      • et al.
      The evaluation of a newly developed antigen test (QuickNavi™-COVID19 Ag) for SARS-CoV-2: a prospective observational study in Japan.
      ,
      • Kurihara Y.
      • Kiyasu Y.
      • Akashi Y.
      • Takeuchi Y.
      • Narahara K.
      • Mori S.
      • et al.
      The evaluation of a novel digital immunochromatographic assay with silver amplification to detect SARS-CoV-2.
      ,
      • Kiyasu Y.
      • Akashi Y.
      • Sugiyama A.
      • Takeuchi Y.
      • Notake S.
      • Naito A.
      • et al.
      A prospective evaluation of the analytical performance of GENECUBE® HQ SARS-CoV-2 and GENECUBE® FLU A/B.
      ,
      • Kiyasu Y.
      • Takeuchi Y.
      • Akashi Y.
      • et al.
      Prospective analytical performance evaluation of the QuickNavi™-COVID19 Ag for asymptomatic individuals.
      ]. Subjects referred by 59 primary care facilities and a local public health center, as well as TMCH healthcare workers with suspected SARS-CoV-2 infection based on symptoms or a known contact history with COVID-19 confirmed/suspected patients were prospectively enrolled.
      For the evaluation of nasopharyngeal samples, we obtained one additional nasopharyngeal sample for the evaluation of Smart Gene SARS-CoV-2, as previously described [
      • Takeuchi Y.
      • Akashi Y.
      • Kato D.
      • Kuwahara M.
      • Muramatsu S.
      • Ueda A.
      • et al.
      The evaluation of a newly developed antigen test (QuickNavi™-COVID19 Ag) for SARS-CoV-2: a prospective observational study in Japan.
      ,
      • Kurihara Y.
      • Kiyasu Y.
      • Akashi Y.
      • Takeuchi Y.
      • Narahara K.
      • Mori S.
      • et al.
      The evaluation of a novel digital immunochromatographic assay with silver amplification to detect SARS-CoV-2.
      ]. For the evaluation of anterior nasal samples, we obtained two additional anterior nasal samples, as previously described [
      • Takeuchi Y.
      • Akashi Y.
      • Kato D.
      • Kuwahara M.
      • Muramatsu S.
      • Ueda A.
      • et al.
      Diagnostic performance and characteristics of anterior nasal collection for the SARS-CoV-2 antigen test: a prospective study.
      ], for the Smart Gene SARS-CoV-2 and reference real-time RT-PCR assays. The Sponge swab™ (NIPRO, Osaka, Japan), equipped with a Smart Gene SARS-CoV-2 kit, was used for obtaining samples for the Smart Gene SARS-CoV-2 assay and the FLOQ swab™ (Copan Italia S.p.A., Brescia, Italy) was used for obtaining samples for in-house reverse transcription (RT)-PCR and reference real-time RT-PCR assays.
      All subjects provided their consent for enrollment. If multiple samples were collected from the same subject, they were all treated as individual samples. The ethics committee of TMCH approved the present study (approval number: 2021–008).

      2.1 Smart Gene SARS-CoV-2 assay

      The evaluation of Smart Gene SARS-CoV-2 was performed using fresh samples according to the manufacturer's instructions, as described in the package insert. An overview of the test procedure is shown in Fig. 1. Briefly, the collected sample with the sponge swab was suspended with 1 mL of extraction reagent solution in a vial, and the suspended sample was dripped into the cartridge. The cartridge was inserted into the analyzer and the fully automated PCR analysis was performed after a few touch-steps. A PCR result was considered “positive” when the Ct value was ≤45.
      Fig. 1
      Fig. 1Smart Gene SARS-CoV-2 assay workflow
      (A) Suspend the collected sample in the extraction reagent solution vial. (B) Drop the suspended sample on the cartridge. (C) Set cartridge into the analyzer and touch the start button on the screen. (D) The analyzer automatically performs nucleic acid extraction and amplification steps, and reports the test result and the Ct value within 40–60 min.

      2.2 In-house RT-PCR assay for SARS-CoV-2

      The swabs were suspended in 3 mL of Universal Transport Medium™ (UTM™) (Copan diagnostics, Brescia, Italy). Purification and RNA extraction from UTM™ samples were performed with magLEAD® 6gC (Precision System Science, Chiba, Japan). The purified samples were evaluated by GENECUBE® and GENECUBE® HQ SARS-CoV-2 as an in-house RT-PCR assay [
      • Kiyasu Y.
      • Akashi Y.
      • Sugiyama A.
      • Takeuchi Y.
      • Notake S.
      • Naito A.
      • et al.
      A prospective evaluation of the analytical performance of GENECUBE® HQ SARS-CoV-2 and GENECUBE® FLU A/B.
      ].

      2.3 Reference real-time RT-PCR assay for SARS-CoV-2

      After used for the in-house PCR, the purified samples were frozen at −80 °C and transferred to Mizuho Medy for the reference real-time RT-PCR assay. The N1 and N2 primer/probe set (Nihon Gene Research Laboratories, Miyagi, Japan) were employed for the reference real-time RT-PCR assay, as suggested by the “Manual for the Detection of Pathogen 2019-nCoV Ver. 2.9.1” issued by the National Institute of Infectious Diseases of Japan (NIID) [
      • Shirato K.
      • Nao N.
      • Katano H.
      • Takayama I.
      • Saito S.
      • Kato F.
      • et al.
      Development of genetic diagnostic methods for detection for novel coronavirus 2019(nCoV-2019) in Japan.
      ]. The reference real-time RT-PCR assays were performed on a LightCycler® Nano System (Roche Diagnostics, Rotkreuz, Switzerland) using One Step PrimeScript™ III RT-qPCR Mix (Takara Bio, Kusatsu, Japan) with the following cycling conditions: reverse transcription at 52 °C for 5 min and 95 °C for 10 s, and 45 cycles at 95 °C for 5 s and at 60 °C for the 30s. A PCR result was considered “positive” when the Ct value was ≤40 [
      • Shirato K.
      • Nao N.
      • Katano H.
      • Takayama I.
      • Saito S.
      • Kato F.
      • et al.
      Development of genetic diagnostic methods for detection for novel coronavirus 2019(nCoV-2019) in Japan.
      ]. The absolute viral copy number was determined by serially diluted RNA control targeting the N2 gene of SARS-CoV-2 (Nihon Gene Research Laboratories). For the comparison between the Ct values of Smart Gene SARS-CoV-2 and the reference real-time RT-PCR assay, we used the Ct values of N2 gene.

      2.4 Validation of discordant cases

      For discordant samples that were negative by Smart Gene SARS-CoV-2 and positive by the reference real-time RT-PCR assay, the stored UTM samples were re-evaluated by Smart Gene SARS-CoV-2. Three hundred microliters of each UTM sample were added to the extract reagent solution, and subsequent operations were performed according to the manufacturer's instructions. These measurements were performed in duplicate. For discordant samples that were positive by Smart Gene SARS-CoV-2 and negative by the reference real-time RT-PCR assay, each stored UTM sample was evaluated by GeneXpert® for SARS-CoV-2 (Cepheid, Sunnyvale, CA, USA) according to the manufacturer's instructions.

      2.5 Limits of detection of Smart Gene SARS-CoV-2 and reference real-time RT-PCR

      The limits of detection (LOD) of Smart Gene SARS-CoV-2 and reference real-time RT-PCR were evaluated using the AccuPlex SARS-CoV-2 Verification Panel (SeraCare Life Sciences, Inc., Milford, MA, USA), and negative nasopharyngeal swabs.
      For sample preparation, negative nasopharyngeal swabs were obtained from 4 healthy volunteers and were suspended into either 3 mL of UTM for the reference real-time RT-PCR assay or 1 mL of extraction reagent solution of Smart Gene SARS-CoV-2. Each suspended sample was pooled, and a total of 2 pooled samples for each solution were prepared. Then, each pooled solution was divided into 4 groups by adding diluted AccuPlex SARS-CoV-2 Verification Panel (Supplementary Fig. 1), and negative samples were also prepared. The viral concentration for UTM was 1824 copies/mL, 912 copies/ mL, 456 copies/mL, 228 copies/ mL, and 0 copy/mL respectively.
      For evaluation of the LOD, a total of 24 replicate analyses (8 for pooled sample 1, 8 for pooled sample 2, 8 for UTM) for the reference real-time RT-PCR assay, and 12 replicate analyses (4 for pooled sample 1, 4 for pooled sample 2, 4 for extraction reagent solution) for Smart Gene SARS-CoV-2 were performed for each viral concentration. The N2 primer/probe set of the NIID method was used for the reference real-time RT-PCR assay. The lowest viral load at which a 100% detection rate was achieved was considered to be the LOD of the assay.

      2.6 Statistical analyses

      The 95% confidence intervals (CIs) of total, positive and negative concordance rate between Smart Gene SARS-CoV-2 and reference real-time RT-PCR were calculated using the Clopper and Pearson method. The correlation of Ct values between the reference real-time RT-PCR assay and Smart Gene SARS-CoV-2 was evaluated using Spearman's rank correlation coefficient. The Ct values between nasopharyngeal and anterior nasal samples were compared using Wilcoxon rank sum test. P values < 0.05 were considered statistically significant. All statistical analyses were conducted using the R 4.0.3 software program (www.r-project.org).

      3. Results

      During the study period, 1150 samples were collected, of which 791 were nasopharyngeal samples and 359 were anterior nasal samples (Table 1a, b, c). Sixty-eight of 791 (8.6%) nasopharyngeal samples and 51 of 359 (14.2%) anterior nasal samples were positive for SARS-CoV-2 by the reference real-time RT-PCR assay. There were significant differences between the Ct values (N2 gene) of nasopharyngeal samples and anterior nasal samples (median Ct values: 21.4 vs 25.4, p = 0.004) (Supplementary Fig. 2). All of the results of the in-house RT-PCR assays were in concordance with the results of the reference real-time RT-PCR assays.
      Table 1aThe concordance rate between Smart Gene SARS-CoV-2 and reference real-time RT-PCR using nasopharyngeal samples and anterior nasal samples.
      Reference real-time RT-PCR
      PositiveNegative
      Smart Gene SARS-CoV-2Positive1145
      Negative51026
      Positive concordance rate (%)95.8 (90.5–98.6)
      Negative concordance rate (%)99.5 (98.9–99.8)
      Total concordance rate (%)99.1 (98.4–99.6)
      Data in parentheses are 95% confidence intervals.
      RT-PCR reverse transcription polymerase chain reaction, SARS-CoV-2 severe acute respiratory syndrome coronavirus 2.
      Table 1bThe concordance rate between Smart Gene SARS-CoV-2 and reference real-time RT-PCR using nasopharyngeal samples.
      Nasopharyngeal samplesReference real-time RT-PCR
      PositiveNegative
      Smart Gene SARS-CoV-2Positive642
      Negative4721
      Positive concordance rate (%)94.1 (85.6–98.4)
      Negative concordance rate (%)99.7 (99.0–100)
      Total concordance rate (%)99.2 (98.4–99.7)
      Data in parentheses are 95% confidence intervals.
      RT-PCR reverse transcription polymerase chain reaction, SARS-CoV-2 severe acute respiratory syndrome coronavirus 2.
      Table 1cThe concordance rate between Smart Gene SARS-CoV-2 and reference real-time RT-PCR using anterior nasal samples.
      Anterior nasal samplesReference real-time RT-PCR
      PositiveNegative
      Smart Gene SARS-CoV-2Positive503
      Negative1305
      Positive concordance rate (%)98.0 (89.6–100)
      Negative concordance rate (%)99.0 (97.2–99.8)
      Total concordance rate (%)98.9 (97.2–99.7)
      Data in parentheses are 95% confidence intervals.
      RT-PCR reverse transcription polymerase chain reaction, SARS-CoV-2 severe acute respiratory syndrome coronavirus 2.

      3.1 Results of the limit of detection tests using recombinant SARS-CoV-2

      The results of the LOD tests using recombinant SARS-CoV-2 are shown in Table 2 and the detailed data are summarized in Supplementary Table 1. For the current LOD evaluation, the reference real-time RT-PCR LOD showed positive results in all spiked samples down to 912 copies/mL. Smart Gene SARS-CoV-2 provided positive results in all samples with viral concentration corresponding to 456 copies/mL in UTM samples for the reference real-time RT-PCR assay.
      Table 2Summary of the results of the LOD test for the reference real-time RT-PCR and Smart Gene SARS-CoV-2 assays.
      Viral loads of UTM samples (Copies/ mL)Reference real-time RT-PCR
      For the LOD evaluation, a total of 24 replicate analyses (8 for pooled sample 1, 8 for pooled sample 2, 8 for UTM) for the reference real-time RT-PCR assay, and a total of 12 replicate analyses (4 for pooled sample 1, 4 for pooled sample 2, 4 for extraction reagent solution) for the Smart Gene SARS-CoV-2 assay were performed for each viral concentration.
      (NIID method, N2 gene)
      Smart Gene SARS-CoV-2
      For the LOD evaluation, a total of 24 replicate analyses (8 for pooled sample 1, 8 for pooled sample 2, 8 for UTM) for the reference real-time RT-PCR assay, and a total of 12 replicate analyses (4 for pooled sample 1, 4 for pooled sample 2, 4 for extraction reagent solution) for the Smart Gene SARS-CoV-2 assay were performed for each viral concentration.
      positive/ test (Detection rate)positive/ test (Detection rate)
      182424/24 (100)12/12 (100)
      91224/24 (100)12/12 (100)
      45622/24 (91.7)12/12 (100)
      22821/24 (87.5)7/12 (58.3)
      00/24 (0)0/12 (0)
      ND not detected, NIID National Institute of Infectious Diseases, RT-PCR reverse transcription polymerase chain reaction.
      a For the LOD evaluation, a total of 24 replicate analyses (8 for pooled sample 1, 8 for pooled sample 2, 8 for UTM) for the reference real-time RT-PCR assay, and a total of 12 replicate analyses (4 for pooled sample 1, 4 for pooled sample 2, 4 for extraction reagent solution) for the Smart Gene SARS-CoV-2 assay were performed for each viral concentration.

      3.2 Performance evaluation of Smart Gene SARS-CoV-2

      The results of the performance evaluation of Smart Gene SARS-CoV-2 and the reference real-time RT-PCR assay are compared in Table 1a, Table 1b, Table 1ca–c. For nasopharyngeal samples (Table 1b), the total, positive, and negative concordance rate between the two assays were 99.2% (95% CI: 98.4–99.7%), 94.1% (95% CI: 85.6–98.4%), 99.7% (95% CI: 99.0–100%), respectively (Table 1b). For anterior nasal samples (Table 1c), the total, positive, and negative concordance rate between the two assays were 98.9% (95% CI: 97.2–99.7%), 98.0% (95% CI: 89.6–100%), and 99.0% (95% CI: 97.2–99.8%), respectively (Table 1c). For the overall evaluation of nasopharyngeal and anterior nasal samples (Table 1a), the total, positive and negative concordance rate between the two assays were 99.1% (95% CI: 98.4–99.6%), 95.8% (95% CI: 90.5–98.6%), and 99.5% (95% CI: 98.9–99.8%), respectively.
      The detailed evaluations of samples for which the Smart Gene SARS-CoV-2 and reference real-time RT-PCR assays showed discordant results are summarized in the Supplementary Tables 2a and 2b

      3.3 Comparison of the Ct value between Smart Gene SARS-CoV-2 and reference real-time RT-PCR

      The comparison of the Ct values of Smart Gene SARS-CoV-2 and the reference real-time RT-PCR assay (N2 gene) are plotted in Fig. 2. The calculation of Spearman's rank correlation coefficient revealed a significant correlation (R = 0.81, p < 0.001) between the reference RT-PCR and Smart Gene SARS-CoV-2 assays. The median Ct value was 29.0 (IQR: 25.0–34.8) for Smart Gene SARS-CoV-2 and 23.0 (IQR: 19.6–28.5) for the reference real-time RT-PCR assay (N2 gene). The median difference in Ct values between Smart Gene SARS-CoV-2 and the reference real-time RT-PCR assay (N2 gene) was 6.0 (IQR: 3.9–7.8). The Ct value of each sample is listed in Supplementary Tables 3a and 3b and Supplementary Fig. 3.
      Fig. 2
      Fig. 2Comparison of the cycle threshold (Ct) values between the Smart Gene SARS-CoV-2 and reference real-time RT-PCR assays (N2 gene). Spearman's rank correlation coefficient (R) between the two tests was 0.81. N = 114.

      4. Discussion

      The comparison between the Smart Gene SARS-CoV-2 and reference real-time RT-PCR method of the National Institute of Infectious Diseases (NIID) demonstrated that Smart Gene SARS-CoV-2 has equal analytical performance in the detection of SARS-CoV-2 in fresh nasopharyngeal samples and anterior nasal samples. This was also confirmed through the LOD evaluation. A strong correlation was indicated between the Ct value of Smart Gene SARS-CoV-2 and the reference real-time RT-PCR assay. The median difference of Ct values between the Smart Gene SARS-CoV-2 and reference real-time RT-PCR assays was 6.0.
      Through the comparison with 1150 samples, the results of the Smart Gene SARS-CoV-2 and reference real-time RT-PCR assays deviated in 10 samples. For 5 samples with positive reference real-time RT-PCR results and negative Smart Gene SARS-CoV-2 results, GeneXpert® showed positive results; thus, the Smart Gene SARS-CoV-2 assay was considered to have provided false-negative results. For 5 samples with negative reference real-time RT-PCR results and positive Smart Gene SARS-CoV-2 results, 3 of the 5 samples were anterior nasal samples, which were obtained from patients with SARS-CoV-2 infection that was confirmed by a positive nasopharyngeal real-time RT-PCR assay; thus, the reference real-time RT-PCR of anterior nasal samples was considered to have provided false-negative results. Some factors may have caused the inconsistency between the two molecular assays can be explained. The swabs used for the Smart Gene SARS-CoV-2 and the reference real-time RT-PCR assays were obtained separately. Other variables such as the possibility of a non-specific reaction in the Smart Gene SARS-CoV-2 assay, RNA extraction procedures, the interval from sample collection and testing, and randomness in low virus load samples may have influenced the results [
      • Afzal A.
      Molecular diagnostic technologies for COVID-19: limitations and challenges.
      ].
      Currently, several POC-type molecular assays have been available worldwide [
      • Dinnes J.
      • Deeks J.J.
      • Berhane S.
      • Talor M.
      • Adriano A.
      • Davenport C.
      • et al.
      Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection.
      ], and their sensitivities and specificities were generally as high as above 90% [
      • Dinnes J.
      • Deeks J.J.
      • Berhane S.
      • Talor M.
      • Adriano A.
      • Davenport C.
      • et al.
      Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection.
      ,
      • Hansen G.
      • Marino J.
      • Wang Z.X.
      • Beavis K.G.
      • Rodrigo J.
      • Labog K.
      • et al.
      Clinical performance of the point-of-care cobas liat for detection of SARS-CoV-2 in 20 minutes: a multicenter study.
      ,
      • Moran A.
      • Beavis K.G.
      • Matushek S.M.
      • Ciaglia C.
      • Francois N.
      • Tesic V.
      • et al.
      Detection of SARS-CoV-2 by use of the cepheid xpert xpress SARS-CoV-2 and roche cobas SARS-CoV-2 assays.
      ]. Our study indicated that Smart Gene SARS-CoV-2 has comparable diagnostic performance to other POC molecular tests [
      • Dinnes J.
      • Deeks J.J.
      • Berhane S.
      • Talor M.
      • Adriano A.
      • Davenport C.
      • et al.
      Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection.
      ,
      • Hansen G.
      • Marino J.
      • Wang Z.X.
      • Beavis K.G.
      • Rodrigo J.
      • Labog K.
      • et al.
      Clinical performance of the point-of-care cobas liat for detection of SARS-CoV-2 in 20 minutes: a multicenter study.
      ,
      • Moran A.
      • Beavis K.G.
      • Matushek S.M.
      • Ciaglia C.
      • Francois N.
      • Tesic V.
      • et al.
      Detection of SARS-CoV-2 by use of the cepheid xpert xpress SARS-CoV-2 and roche cobas SARS-CoV-2 assays.
      ] despite having modest turnaround time (Table 3). In the evaluations of LOD and clinical performance, Smart Gene SARS-CoV-2 provided almost identical results to the reference real-time RT-PCR.
      Table 3Comparison of four POC type commercial molecularly based tests to detect SARS-CoV-2.
      Product nameGeneXpert
      GeneXpert system GX-II model.
      ID Nowcobas LiatSmart Gene
      Size(W × D × H) and weight of instrument163 × 297 × 307 mm 6.5 kg207 × 194 × 145 mm 3.0 kg114 × 241 × 190 mm 3.8 kg152 × 343 × 300 mm 6.0 kg
      Hands-on-time1min5min1min1min
      Examination time<45min<10min20min<60min
      Method of amplificationRT-PCRNEARRT-PCRRT-PCR
      Display of quantitative resultYesNoNoYes
      RT-PCR reverse transcription polymerase chain reaction, NEAR Nicking Enzyme Amplification Reaction, SARS-CoV-2 severe acute respiratory syndrome coronavirus 2, GeneXpert GeneXpert® for SARS-CoV-2 (Cepheid, Sunnyvale, CA, USA), ID Now ID Now SARS-CoV-2 (Abbott, Chicago, USA), cobas Liat The cobas Liat system (Roche Diagnostics Corporation, Basel, Switzerland), Smart Gene Smart Gene SARS-CoV-2 (Mizuho Medy Co., Ltd., Tosu City, Saga, Japan).
      a GeneXpert system GX-II model.
      As summarized in Table 3, the advantages of Smart Gene SARS-CoV-2 are its small size and the least hands-on time with three-step manipulation of the POC type molecular examination platforms. Another feature of the system is to display Ct values of samples. The appropriate interpretation of Ct values is important in clinical contexts. The Ct values may predict not only the viral loads in samples but also the infectivity or disease severity of patients with SARS-CoV-2 [
      • Rabaan A.A.
      • Tirupathi R.
      • Sule A.A.
      • et al.
      Viral dynamics and real-time RT-PCR Ct values correlation with disease severity in COVID-19.
      ]. The current evaluation observed the linear Ct value correlation (p values < 0.001) between Smart Gene SARS-CoV-2 and the reference real-time RT-PCR (national standard method in Japan). Nevertheless, the provided Ct values should be carefully read. The Smart Gene SARS-CoV-2 showed 6 points higher Ct values in median than the reference RT-PCR (N2 gene), and marked differences existed in some samples (Supplementary Tables 3a and 3b and Supplementary Fig. 3).
      Our study was associated with some limitations. First, we did not analyze the sequences of the viruses detected in this study. Thus, the effect of genetic mutation on the performance of Smart Gene SARS-CoV-2 was not evaluated. Second, this study was conducted in a limited season and area, and it may be necessary to verify whether similar results can be obtained throughout the year or in another area. Finally, the reference real-time RT-PCR assay used frozen stored purified extraction, and such storage and transportation methods may have affected the test results.
      In conclusion, current study showed that Smart Gene SARS-CoV-2 had sufficient analytical performance in the detection of SARS-CoV-2 in nasopharyngeal and anterior nasal samples.

      Author contribution

      Contributor Yoshihiko Kiyasu drafted the manuscript and performed the statistical analysis. Masato Owaku revised the manuscript and assisted each author in coordinating their work. Yusaku Akashi and Yuto Takeuchi revised the manuscript. Shigeyuki Notake, Atsuo Ueda, and Koji Nakamura, Kenji Narahara, Sunao Mori, and Takashi Nagano were in charge of manipulating the measuring equipment. Kenji Narahara, Sunao Mori, and Takashi Nagano also drafted the manuscript. Hiroichi Ishikawa and Hiromichi Suzuki supervised the project. All authors contributed to the writing of the final manuscript.

      Declaration of competing interest

      Mizuho Medy provided fees for research expenses and provided Smart Gene SARS-CoV-2 free of charge. Masato Owaku, Kenji Narahara, Sunao Mori, Takashi Nagano are employed by Mizuho Medy, the developer of Smart Gene SARS-CoV-2. Hiromichi Suzuki received a consultation fee from Mizuho Medy.

      Acknowledgments

      We thank Yoko Ueda, Mio Matsumoto, Suwako Kikuchi, Kaoru Kuriiwa, Masaomi Matsubayashi, Yumiko Tanaka, Mika Yaguchi, Shoko Yoshiwara, Asami Sugie, Shiori Kanoya, Kodai Tayama, Kazuya Ishiguro, Toshiki Yoshizawa, Norihiko Terada, Naoki Tanimura and the staff in the Department of Clinical Laboratory of Tsukuba Medical Center Hospital for their intensive support of this study. We thank all of the medical institutions for providing their patients' clinical information.

      Appendix A. Supplementary data

      The following is the Supplementary data to this article:

      References

        • World Health Organization
        Coronavirus disease (COVID-19) weekly epidemiological update and weekly operational update.
        • Centers for Disease Control and Prevention
        Guidance for SARS-CoV-2 point-of-care and rapid testing.
        2021
        • Dinnes J.
        • Deeks J.J.
        • Berhane S.
        • Talor M.
        • Adriano A.
        • Davenport C.
        • et al.
        Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection.
        Cochrane Database Syst Rev. 2021; 3CD013705https://doi.org/10.1002/14651858.CD013705.pub2
        • Kakiuchi T.
        • Miyata I.
        • Kimura R.
        • Shimomura G.
        • Shimomura K.
        • Yamaguchi S.
        • et al.
        Clinical evaluation of a novel point-of-care assay to detect Mycoplasma pneumoniae and associated macrolide-resistant mutations.
        J Clin Microbiol. 2021; 59e0324520https://doi.org/10.1128/JCM.03245-20
        • Takeuchi Y.
        • Akashi Y.
        • Kato D.
        • Kuwahara M.
        • Muramatsu S.
        • Ueda A.
        • et al.
        Diagnostic performance and characteristics of anterior nasal collection for the SARS-CoV-2 antigen test: a prospective study.
        Sci Rep. 2021; 11: 10519https://doi.org/10.1038/s41598-021-90026-8
        • Takeuchi Y.
        • Akashi Y.
        • Kato D.
        • Kuwahara M.
        • Muramatsu S.
        • Ueda A.
        • et al.
        The evaluation of a newly developed antigen test (QuickNavi™-COVID19 Ag) for SARS-CoV-2: a prospective observational study in Japan.
        J Infect Chemother. 2021; 27: 890-894https://doi.org/10.1016/j.jiac.2021.02.029
        • Kurihara Y.
        • Kiyasu Y.
        • Akashi Y.
        • Takeuchi Y.
        • Narahara K.
        • Mori S.
        • et al.
        The evaluation of a novel digital immunochromatographic assay with silver amplification to detect SARS-CoV-2.
        J Infect Chemother. 2021; 27: 1493-1497https://doi.org/10.1016/j.jiac.2021.07.006
        • Kiyasu Y.
        • Akashi Y.
        • Sugiyama A.
        • Takeuchi Y.
        • Notake S.
        • Naito A.
        • et al.
        A prospective evaluation of the analytical performance of GENECUBE® HQ SARS-CoV-2 and GENECUBE® FLU A/B.
        Mol Diagn Ther. 2021; 25: 495-504https://doi.org/10.1007/s40291-021-00535-5
        • Kiyasu Y.
        • Takeuchi Y.
        • Akashi Y.
        • et al.
        Prospective analytical performance evaluation of the QuickNavi™-COVID19 Ag for asymptomatic individuals.
        J Infect Chemother. 2021; 27: 1489-1492https://doi.org/10.1016/j.jiac.2021.07.005
        • Shirato K.
        • Nao N.
        • Katano H.
        • Takayama I.
        • Saito S.
        • Kato F.
        • et al.
        Development of genetic diagnostic methods for detection for novel coronavirus 2019(nCoV-2019) in Japan.
        Jpn J Infect Dis. 2020; 73: 304-307https://doi.org/10.7883/yoken.JJID.2020.061
        • Afzal A.
        Molecular diagnostic technologies for COVID-19: limitations and challenges.
        J Adv Res. 2020; 26: 149-159https://doi.org/10.1016/j.jare.2020.08.002
        • Hansen G.
        • Marino J.
        • Wang Z.X.
        • Beavis K.G.
        • Rodrigo J.
        • Labog K.
        • et al.
        Clinical performance of the point-of-care cobas liat for detection of SARS-CoV-2 in 20 minutes: a multicenter study.
        J Clin Microbiol. 2021; 59: e02811-e02820https://doi.org/10.1128/JCM.02811-20
        • Moran A.
        • Beavis K.G.
        • Matushek S.M.
        • Ciaglia C.
        • Francois N.
        • Tesic V.
        • et al.
        Detection of SARS-CoV-2 by use of the cepheid xpert xpress SARS-CoV-2 and roche cobas SARS-CoV-2 assays.
        J Clin Microbiol. 2020; 58: e00772-20https://doi.org/10.1128/JCM.00772-20
        • Rabaan A.A.
        • Tirupathi R.
        • Sule A.A.
        • et al.
        Viral dynamics and real-time RT-PCR Ct values correlation with disease severity in COVID-19.
        Diagnostics. 2021 Jun 15; 11: 1091https://doi.org/10.3390/diagnostics11061091