Protocol

This SR was conducted following the 27 guidelines and the flow diagram structure provided by the PRISMA statement ^{[11] [12] [13]} as well as the recommendations of the Cochrane Collaboration to perform an orderly selection of papers according to the proposed protocol ^[14]. All this enhanced the quality and presentation of the analysis and brought transparency to the paper selection process.

Search strategy

Between 1 May and 19 June 2020, an electronic search was carried out in the COCHRANE, IEEE, BVS/ LILACS, MEDLINE/PUBMED, and WEB OF SCIENCE databases in order to find any papers published in the January 2010-June 2020 period, when most of the development and research work around the use of VR technology applied to motor and cognitive rehabilitation ^[9] took place. Additionally, the search was performed in English and Spanish, without a geographic delimitation with the purpose of obtaining a global overview.

The search terms in Spanish consisted of combinations of keywords found in the Descriptores en Ciencias de la Salud (DeCS) thesaurus: lesión de médula espinal, traumatismo de médula espinal, tetraplejia, cuadriplejia, paraplejia, realidad virtual, realidad aumentada, terapia de exposición mediante realidad virtual, juego de video, rehabilitación, and rehabilitación neurológica. The search terms in English included terms found in the Medical Subject Headings (MeSH) thesaurus: spinal cord injury, spinal cord trauma, tetraplegia, quadriplegia, paraplegia, virtual reality, augmented reality, virtual reality immersion therapy, virtual reality exposure therapy, video game, rehabilitation, and neurorehabilitation. The final structure of each database search strategy was adjusted based on syntax, logical operators, tags, and relevant qualifiers ^{[15] [16] [17] [18]}.

Eligibility criteria

The inclusion criteria were defined based on the PICOS model (Participants, Interventions, Comparators, Outcomes, Study Design) ^[13]. The review took into account both controlled and randomized controlled clinical trials ^{[14] [19]} conducted on human subjects that provided empirical evidence of the efficacy of neurorehabilitation of motor and cognitive functions through virtual reality therapies (VRT) applied as a complement or not to physical, occupational, or mixed therapies, and that allowed for a comparison of an intervention group with a control group. Quasiexperimental (before-and-after) studies were also considered. The patient population included both male and female subjects with SCI in the 18 to 85 age range, with traumatic and non-traumatic, complete and incomplete injuries, regardless of their ASIA grades or time since injury.

The studies were required to contain this information: population sample, assisted limbs and SCI characteristics, objectives, and aspects of the rehabilitation intervention; VR technology type, effects, and application method; duration, frequency, and acceptability; assessment of the effect of the intervention compared to conventional therapies; validation or development of VR software or devices; and intervention efficacy indicators.

Any systematic reviews, meta-analyses, lectures, abstracts, or studies that were duplicated or not designed to assess the clinical efficacy of motor or cognitive rehabilitation on SCI, that did not focus on VR as a therapeutic intervention, or that used electrical stimulation that could have influenced its own results of the technology under study were excluded. Similarly, studies that did not fit the defined study time period and were not written in English or Spanish were also excluded.

The selection of potentially relevant studies was performed in three stages. The first stage focused on eliminating duplicate records; the second stage centered on exclusion of papers according to their title and abstract; finally, the third stage consisted of a fulltext analysis. The last two stages were guided by inclusion and exclusion criteria ^[14].

Data extraction

Two electronic worksheets were created in order to obtain detailed information from the studies selected in an organized manner. The first worksheet contained demographic and clinical information about the patient population: mean age, sex, cause and level of SCI, ASIA impairment grade, type of injury, time since injury, sample distribution across study groups, and their respective dropout rates. The second worksheet summarized the characteristics of the selected studies: authors’ names, country and year of publication, study design, and characteristics of the VR technology used, information on interventions, sessions, outcome measures, and conclusions.

Risk of bias assessment

The internal validity of the controlled clinical trials was assessed using the PEDro scale criteria ^[20]. For before-and-after studies without a control group, the National Institute of Health Quality Assessment Tool was applied ^[21].

Taking the registered level of evidence as a reference, the internal validity was rated as poor, fair, good, or excellent for scores in the 0-4, 5-6, 7-8, and 9-10 ranges, respectively, for PEDro. Comparatively, for the before-and-after studies, it was rated as poor, fair, or good in the 0-4, 5-6, and 7-10 ranges, respectively. Studies rated as with good methodological quality and low risk of bias were identified with a score higher than the mean of 5, while those with scores lower than the mean of 5 were rated as with poor methodological quality and high risk of bias.

Paper search and selection process

A total of 353 papers were found, 218 of which remained after eliminating duplicates. Of these, only 207 could be downloaded. During the second stage of the selection process, 179 papers were ruled out based on their title and abstract, and only 28 moved forward to the final stage for full-text analysis, where another 17 were excluded, resulting in 11 papers finally selected for the SR. Figure 1 depicts the paper selection process listing the exclusion criteria in each stage based on the PRISMA statement ^[12].

Figure 1 PRISMA flow diagram of the paper search strategy and selection process. 

A larger number of papers were found from India and Spain, each represented 27.27% of the studies selected, while Italy, South Korea, Switzerland, Taiwan, and Australia each represented only 9.09% of these. Over the entire search time period, no papers were found for the years 2013 and 2019. English was the main language used in 90.91% of the studies selected

Risk of bias assessment of selected papers

The results of the controlled clinical trials ^{[22] [23] [24] [25] [26] [27] [28] [29] [30]} and the before-and-after studies ^{[31] [32]} are presented in Tables 1 and 2, respectively. According to the PEDro scale, the score for the risk of bias of the controlled clinical trials was 7.22±1.30, and in the case of the before-and-after studies, the National Institute of Health score was 7.5±0.7. This indicates that the selected papers have a low risk of bias and a high methodological quality associated with the evidence they provide.

Ethical statement

All studies reported that, before the start of the intervention, patients were asked to sign an informed consent letter after they were informed, in writing and verbally, of the experimental procedures in the study protocol. The interventions were authorized by the ethics committees of the participating hospitals ^{[23] [24] [25]}, universities ^{[27] [28]}, sites ^{[26] [27] [29][30] [31]} or state agency ^[32], with the exception of one ^[22], which did not report anything in this regard. Only 3 studies ^{[23] [25] [32]} adapted their protocols to the Helsinki Declaration of the World Medical Association.

Design of selected studies

Of the 11 selected papers, 9 were controlled clinical trials (8 randomized ^{[22] [23] [24] [25] [26] [27] [28] [29]} and 1 non-randomized ^[30]), and 2 were before-and-after studies ^{[31] [32]}.

Controlled clinical trials were those in which the subjects had been allocated to a control group (CG) or an intervention group (IG) and which conducted a prospective analysis. If such allocation had been made through randomization, then the trial was considered a randomized, controlled clinical trial. Additionally, the design of before-and-after studies included subjects that received the same treatment without a control group ^[14].

Out of the 9 controlled clinical trials, 8 worked with parallel groups, and only 1 ^[27] worked with crossover groups. Only 5 of the 8 randomized controlled clinical trials were blinded and reported the randomization technique used: opaque envelopes containing sequential numbers ^[23], computer-generated random numbers ^[26], the “randbetween” function in Excel ^[27], Random Allocation 2.0 software ^[28], and an unspecified fixed randomization method ^[29]. In the case of the non-randomized study ^[30], patients were assigned to the IG and CG based on their demographic characteristics, which is not considered an adequate randomization method by PEDro ^[20] and the Cochrane Collaboration ^[14].

As far as blinding is concerned, 4 controlled clinical trials and 2 before-and-after studies had no blinding ^{[22] [24] [25] [27] [31] [32]}. Moreover, of the 5 controlled clinical trials, 2 were single-blinded to the assessors ^{[28] [29]}, and 3 were double-blinded to both the subjects and the assessors ^{[23] [30]} or to the therapists and the assessors ^[26].

It should be noted that only 3 papers ^{[23] [29] [32]} included a long-term follow-up on the effects of VRT on subjects over time.

Population characteristics

All papers reported their inclusion and exclusion criteria for subject eligibility, and only 9 ^{[23] [25] [26] [27] [28] [29] [30] [31] [32]} indicated their recruitment sources.

The total population sample analyzed was made up of 243 patients with SCI, with 4 studies ^{[22] [25] [26] [28]} having a 1:1 distribution between the IG and the CG. Only 2 randomized controlled trials ^{[27] [29]}, 1 controlled clinical trial ^[30], and 1 before-and-after study ^[32] reported 1 or 4 dropouts in both groups ^{[27] [29]} or only in the intervention group ^{[30] [32]}, with a total of 15 patient dropouts. The mean age of the total population analyzed was 40.25 years for both groups, with a majority of men (78.68%) in the reported population (n=197) in 9 studies (see Table 3).

Traumatic injuries were found in 86% of the population analyzed (n=150) by 8 studies ^{[23] [24] [25] [26] [27] [29] [31][32]} and only 1 study ^[30] reported both traumatic and non-traumatic injuries without providing precise data. As to the level of injury, all studies (n=209) but one ^[22] observed that 50.72% of injuries were located in the cervical spine, 44.97% at the thoracic level, and 4.31% in the lumbar spine. Most studies worked with recently injured patients (< 6 months), and 4 other studies ^{[28] [29] [31] [32]} worked with patients who had had their injuries for more than one year.

Most SCI were incomplete (58.37%) in the reported population (n=209), and only 1 study ^[26] did not report on this. Regarding the level of impairment of all patients (n=167), according to the ASIA scale, most had grade A injuries (47.31%), followed by grade B (23.35%), grade D (15.57%), and grade C (13.77%). Two studies ^{[22] [26]} reported, with no details, that they had worked with certain ASIA grades, and one ^[31] did not address this issue.

The clinical characteristics of each population group nalyzed are presented in Table 4.

Characteristics of virtual reality

The studies used different types of commercial VR technologies, including video game consoles, specialized VR peripherals and systems designed for rehabilitation, as well as devices developed by the research teams (see Table 5.

Only one study used a commercial, fully-immersive VR system (Oculus Go VR headset) ^[27], while the other 10 studies used non-immersive VR systems ^{[22] [23] [24] [25] [26] [28] [29] [30] [31] [32]}. Four of them ^{[22] [26] [28] [29]} were based on Nintendo and Sony commercial consoles, with their video games (mainly sports and recreational) and compatible peripherals that, in few cases, were reported to be adapted for use in rehabilitation of certain impairment grades (for instance, only one mentioned attaching the Wiimote controller to the hand with bandages or a glove in cases of weak grip ^[29]). Additionally, Rhetoric system ^[30] only used Microsoft Kinect peripheral. Other studies used motion sensing devices for rehabilitation such as YouKicker (with four wireless accelerometers) ^[32], TOyRA ^{[23] [25]} (based on five wireless inertial sensors), and CyberGlove ^[24] (with 22 resistive bend sensing technology and vibrotactile stimulators). One study developed a driving simulator ^[31] using an adapted real car mounted on a one-axle tiltable platform to virtually control acceleration and braking.

Regarding those studies where their VRT used commercial consoles along with their video games, only one ^[26] explicitly mentioned the adaptation of their virtual environments for rehabilitation purposes without providing further details. On the other hand, those studies where commercial video games were not used as virtual environment, two of them were developed by the research teams ^{[24] [32]} and only one specified the use of Unity 3D ^[32] as graphic engine for its development. In the case of the Rhetoric ^[30] and TOyRA ^{[23] [25]} systems, their games were designed for neurorehabilitation purposes by specialized teams of Rehametrics and INDRA systems companies, respectively. It should be noted that the design of the environments for TOyRA was developed on the basis of therapeutic guidelines for SCI rehabilitation, while for another study ^[32] it was mentioned that the therapists participated in the design of the clinically virtual exercises. In addition, the fully-immersive VR system ^[27] relied mainly on the use of pre-recorded videos from National Geographic which were not properly virtual environments and without any possibility of patient interaction. Details on the orientation of the different virtual environments in rehabilitation of balance control, ADL autonomy, motor function, psycho-emotional health and driving skills are included in Table 5.

The VRT was administered using avatars in virtual environments ^{[22] [26] [28] [29]}, mirroring of movements made through an avatar ^{[23] [25] [30]}, limb control in a first-person virtual environment ^{[24] [32]}, activities of daily living (ADL) in virtual everyday spaces ^{[23] [24] [25] [26]}, projection of pre-recorded 360º real-life natural environments ^[27] and driving skills training ^[31]. It should be noted that in the TOyRA system, the avatar could be personalized based on the patient’s anthropomorphic traits which increases the patient’s sense of presence in the virtual environment.

Intervention characteristics

All studies ^{[22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32]} provided information on the baseline assessment carried out prior to the start of VRT. In controlled clinical trials, the IG was treated with a mixed therapy including VRT ^{[22] [23] [24] [25] [26] [27] [28] [29] [30]} comprising several VR technologies, along with traditional rehabilitation therapy (TRT) ^[22] or conventional therapy (CT) ^{[23] [24] [25] [26] [28] [27] [29] [30]}, based on occupational and/or physical therapy and in certain cases, supported by other treatments ^[27]. Patients assigned to the CG were administered the same treatment as those in the IG but without the VR component. The before-and-after study protocols only had one group of subjects on which they experimented with VRT independently ^{[31] [32]}.

Most VRT, TRT, or CT activities focused on balance control ^{[22] [24] [26] [28] [30] [32]}, ADL autonomy ^{[23] [24] [25]}, other functional limb movements ^{[29] [32]}, psycho-emotional health ^[27], and driving skills ^[31]. Only 2 studies based their CT on each subject’s goals or needs ^{[27] [30]}.

It should be highlighted that in 5 studies, VRT aimed at improving upper limb (UL) function ^{[23] [24] [25] [28] [29]}; in 1 study, VRT aimed at improving lower limb (LL) ^[32] function, and in 4 studies, VRT aimed at improving both UL and LL ^{[22] [26] [30] [31]} function. Some therapies also included trunk movements ^{[22] [24] [26] [28] [30]}.

In most cases, VRT was supervised by an occupational therapist ^{[23] [30]}, a physical therapist ^{[22] [28] [30] [31]}, or several therapists ^{[26] [32]}.

The number of VRT sessions varied from 3 to 80, with a frequency between 2 and 20 times per week over a total VRT period of 2 to 12 weeks, mostly with 30-minute sessions. Most of the studies were conducted at specialized centers or hospital departments ^{[23] [24] [25] [27] [28]}, or rehabilitation centers ^{[22] [26] [29] [30] [31]}, while only 1 study ^[32] reported that VRT was administered at the patient’s home and set up by a therapist.

It should be pointed out that in 7 studies ^{[23] [24] [25] [26] [29] [30] [32]}, the level of difficulty of the virtual games or activities was adjusted (increasing/reducing speed or number of repetitions, changing object appearance, and interaction parameters) depending on the level of progress achieved by patients in the various VRT exercises. It is worth noting that one intervention ^[31], instead of including gradually increasing levels of difficulty according to the patient’s progress, created a more challenging scenario that brought together all the isolated activities done in previous scenarios. The details of the interventions analyzed are included in Table 5.

Outcome measures

The various scales, indices, or instruments used to evaluate the effects of interventions on UL and LL motor function, balance, functional independence in ADL, and pyscho-emotional health associated with depression are presented in Table 5 according to the VRT applied in each study and where at least two or more combinations were used. Only 3 studies ^{[23] [29] [32]} assessed the level of acceptance and motivation using patient satisfaction scales after the intervention.

Other studies came up with new outcome measures, such as the normalized trajectory length to measure movement trajectory precision improvement, and repeatability to measure movement precision improvement ^[24]. Similarly, kinematic variables ^[25] or the simulator’s own aspects ^[31] were used to evaluate the effects of VRT.

Effects of virtual reality therapy

There were 7 controlled clinical trials ^{[23] [24] [25] [26] [27] [28] [30]} that did not report significant differences between both intervention groups at baseline, which means that at the start of the studies, patients were in similar functional conditions. The effects of VRT are consolidated in Table 6. From then onwards, VRT was understood as being a mixed or as an individual therapy.

Only 9 papers reported a statistically significant difference in VRT (p<0.05) in balance control ^{[22] [26] [28] [30] [32]}, motor function ^{[25] [26] [29] [31] [32]}, and psychological aspects ^[27]. Furthermore, 3 studies assessed the effect size of the interventions, which was found to be between medium and large, with Cohen’s d values between 0.41 and 0.84, and an η² between 0.21 and 0.95 ^{[23] [26] [27]}. Similarly, the level of satisfaction ^{[23] [29]} and motivation ^[32] were assessed, and one study ^[26] subjectively observed a high level of interest and enthusiasm in patients during VRT.

Side effects of virtual reality

There were no adverse or side effects reported, such as motion sickness, vertigo, muscle pain, or spasticity ^{[23] [24] [29] [30] [32]}. Only one study ^[27] reported significant signs of depression in its IG immediately after VRT. Another study ^[29] reported difficulty in holding the Wii-mote controller due to weak grip in seven patients, and another one mentioned ^[30] that during the intervention there were some cases of back pain and orthostatic hypotension that were controlled and subsided with medication, allowing the therapy to continue as they were not deemed serious side effects.

Limitation of the studies

Some of the controlled clinical trials indicated that their design did not include any type of blinding ^{[22] [24] [25] [27]}, concealed allocation ^{[22] [24] [25] [30]}, or even randomization ^[30]. In the before-and-after studies ^{[31] [32]}, it was clear that there was no blinding as they did not include a CG in their design, which implied certain bias in the results.

Regarding subjects and protocols, it was not possible to generalize results given the limitations observed, such as small sample size ^{[23] [24] [25] [28] [29] [30] [32]}, with particular SCI characteristics ^{[23] [26] [28] [29] [32]} and very few variations in demographic data ^[27], as well as the fact of having single selection sources ^[26].

As far as interventions are concerned, the identified limitations included: the short duration of intervention ^{[23] [24] [27] [29] [31]}; most studies did not follow-up on the results, except for 3 of them ^{[23] [29] [32]}; only 1 study explicitly recognized having included a small set of virtual exercises as a limitation ^[30]; the weight of any other health condition that could have influenced the results was not considered ^[27]; and the diversity of interventions did not make it possible to set guidelines on intensity, dosage, and duration of VRT ^[30].

As to the limitations derived from the VR technology, the need to have more sophisticated and modern software (commercial or especially designed games) and hardware (consoles, controllers, headsets, sensors, graphic cards) was identified for VRT to be more effective ^{[26] [30]}. The Wii Fit system especially stood out, because as it is a black box system, it was not possible to monitor several game parameters included in the VRT, thus limiting its contribution to improving balance control ^[22].

Three studies pointed out limitations in terms of interaction with VR devices, which could have affected the obtained results. Such was the case with the Nintendo Wii console ^{[28] [29]}, where the patient required some prior training before using it.

Patients with acute SCI could not use it because they could not maintain balance, hold the controller, or even generate motion, especially in the case of higher cervical injuries. The third study identified predefined VR scenarios (360º recordings) as a limitation since patients could not interact with them ^[27].

Future development

Based on the above-mentioned limitations, the subjects’ profiles, and the intervention protocol, a suggestion was made to use a broader patient sample ^{[23] [24] [25] [26] [27] [29] [30] [31] [32]}, with different SCI characteristics than the ones included in the studies ^{[26] [29] [31] [32]}, involving several recruitment sites ^[30], and including more VRT sessions ^{[23] [24]} to better identify VRT benefits, the system’s critical characteristics, and the virtual exercises that achieved better results ^[24].

The before-and-after studies ^{[31] [32]} along with one controlled clinical trial ^[30] proposed to carry out future interventions based on a blinded, randomized, controlled clinical trial with long-term follow-up design ^{[30] [31] [32]} to reduce the risk of bias and monitor the sustainability of the intervention by means of a longitudinal study to validate the level of skill improvement achieved ^[31].

Regarding types of VR devices, a proposal was made to conduct studies that combine VRT (such as TOyRA) with robotic devices (Amadeo) for telerehabilitation based on VR motion capture systems ^[25] and develop new VR devices for rehabilitation with a focus on ADL or relevant exercises/skills designed to simulate more realistic situations with higher levels of difficulty according to different SCI levels ^[31].

For interventions that used mixed VRT ^{[22] [24] [29] [30]}, a proposal was raised to conduct new studies to assess the impact of VRT individually, compare the results of home-based therapies with those achieved at the clinic ^[32], and develop methods to exercise systematized balance control with a focus on SCI level and sitting balance control ^[28].

Main findings, quality of evidence and potential application

This SR included a total sample of 243 subjects from the 11 papers analyzed, out of which 155 experimented with VRT (63.78%). There was a similar number of paraplegic (69) and tetraplegic (67) patients, and 14 patients did not complete the intervention. Patients treated with VRT mostly had ASIA grade A/B impairment (42.20% and 20.18%), followed by ASIA grades C/D (16.51% and 21.10%), with an absence of motor function and little or no sensory perception below the neurological level of injury, with a major focus on UL.

Three types of studies were taken into consideration: randomized controlled clinical trials ^{[22] [23] [24] [25] [26] [27] [28] [29]}, one non-randomized controlled clinical trial ^[30], and before-and-after studies ^{[31] [32]}. Over half of the studies ^{[23] [26] [28] [29] [30] [31] [32]} had a good or excellent level of evidence with low risk of bias and achieved statistically significant results ^{[26] [28] [29] [30] [31] [32]} with regard to VRT.

The controlled clinical trials exhibited an adequate level of evidence given that they were mostly randomized ^{[22] [23] [24] [25] [26] [27] [28] [29]}, with 6 of them having an adequate sample size ^{[22] [23] [26] [27] [28] [29]}. Seven ^{[23] [24] [25] [26] [27] [28] [30]} did not show any confounding bias upon observing a similar baseline functional condition among patients. Regarding blinding, half of them did not describe any blinding method ^{[22] [24] [25] [27]}, while the remaining half that reported having used a double-blind ^{[23] [26]} or single-blind ^{[28] [29]} method reduced their detection bias upon including blinded assessors.

Given their methodological limitations, the beforeand-after studies and the controlled clinical trials had to be considered with caution as they had a certain selection bias for not presenting any randomization method ^[30] or control groups ^{[31] [32]}, which could have affected the VRT evidence obtained.

Most trials aimed at a non-immersive use ^{[22] [23] [24] [25] [26] [28] [29] [30] [31] [32]} of VR, and only 1 was fully-immersive ^[27]. All of them showed a statistically significant improvement with VRT (p<0.05) in balance control ^{[22] [26] [28] [30] [32]}, motor function ^{[25] [26] [29] [31] [32]}, and psychological aspects ^[27]. The effect size of the interventions ^{[23] [26] [27]} was determined to be between medium and large.

Regarding the type of VRT, although in some controlled clinical trials ^{[23] [24] [25] [29] [30]} no statistically significant differences were observed between the VRT + CT intervention compared to the CT, significant improvements were found in the IG in different functional parameters (Table 6. Additionally, in the remaining 4 trials ^{[22] [26] [27] [28]}, statistically significant differences were observed in the IG that were reflected in a greater improvement compared to the CG. The sum of both results suggests that VRT can be an important adjunct/complementary instrument for CT when considering the benefits derived from the different protocols, particularly in terms of balance control, motor function, and patients’ moods.

Moreover, even though the evidence was methodologically limited, it could be added that the beforeand-after studies ^{[31] [32]} also showed positive effects of VRT when applied individually to improve LL motor function and driving skills, which speaks to the convenience of using VRT along with CT.

Furthermore, the findings ^{[23] [25] [29] [32]} suggest considering VRT as a complement to CT that can be used at home and not just in a hospital setting to extend the time of therapy sessions and to monitor and/or maintain the results achieved after rehabilitation. In this regard, the need for future studies to compare the effects of VRT in both settings was highlighted.

Comparatively, 4 controlled clinical trials ^{[22] [24] [29] [30]} underlined that it was not possible to visualize the effect of VRT separately, because it was applied together with CT. Thus, a proposal was made to conduct studies with the same methodological quality focused on interventions centered on VRT alone, with a broader and more diverse sample of SCI patients.

Efficacy of virtual reality therapy

Most studies showed a good level of evidence for the use of VR technology applied to the rehabilitation of SCI patients based on a detailed description of its application, movements made, and patient interaction with the virtual environment. There was only one exception, where information was inferred from the system used ^[29].

Non-immersive VR was the most commonly implemented type of VR based on games with a scoring system and a set duration, whose level of difficulty was determined based on the progress recorded ^{[22] [23] [25] [26] [28] [29] [30] [32]}.

Other study ^[32] mentioned that the type of VR used was augmented reality, which would require virtual objects to be projected onto real world surfaces. However, when the VR technological description was analyzed, it was clear that they had used a non-immersive system that displayed the virtual environment and objects on a screen.

It was interesting to see how fully-immersive VR technology ^[27] used to study patients’ moods only focused on projecting videos of real landscapes without any means of interaction with the virtual environment, given the technological capabilities of the device. Although an improvement was achieved in patients' psycho-emotional health, it will be necessary to gather more evidence on the effects of fully-immersive technology on VRT, related with physiotherapy and occupational therapy exercises, to analyze and determine the impact of this type of VR on the rehabilitation of SCI patients. Despite the technological differences inherent to each VR technology, such as the degree of immersivity and sensorimotor interaction, previous literature has shown different benefits of the use of the full-immersive VR and some of them points that full-immersive VR outperforms non-immersive ^{[33] [34] [35]}.

Although all the interventions applied different immersive and non-immersive VR, some significant benefits were identified that can be generalized as follows: VR stimulated a wide set of functional movements similar to those of the real world; it promoted independence and self-confidence; it improved psycho- emotional health; it increased patient motivation and engagement with the rehabilitation process; and it promoted VRT with no side effects. Such evidence made it possible to confirm the potential use of VR technology as an effective tool for rehabilitation of SCI patients.

The efficacy of VR was confirmed based on a number of indicators reported by the studies. Firstly, there was positive feedback (visual and auditory) about therapeutic sessions designed around games that included a scoring system to help patients come up with new self-improvement strategies within certain timeframes and levels of difficulty established by the devices and/or adjusted by therapists, which made it possible to follow-up on the progress made.

In this regard, it was observed that the level of difficulty in each round prompted patients to make specific movements in an intensive and repetitive manner, which stimulated the subsequent recovery of motor function as a result of triggering neuroplasticity mostly in the motor cortex of the brain ^{[23] [24] [26] [28] [29] [32]}.

Secondly, real-time visual feedback was another indicator of efficacy of VR environments for motor function recovery ^{[26] [28] [29]}; that is, there was visual feedback of the patient’s proper movement execution within the game’s virtual environment with the use of an avatar through activation of certain areas of the motor cortex, leading to improved recovery ^[32]. There was also activation of the mirror neuron system and motor cortex of the brain by enabling recovery in spatial orientation and balance ^[30]. Furthermore, visualizing the patient’s movements in real-time made by the avatar through mirror vision produced a feeling of control and realism with a similar positive effect on motor function ^[23].

Thirdly, given the importance of patients’ motivation as an indicator of efficacy, it is interesting to see that this factor was assessed only by 3 studies ^{[23] [29] [32]}, with high levels of satisfaction and motivation achieved during VRT. However, even when such an evaluation was not considered, another study ^[26] indicated that patients showed a high level of interest and enthusiasm towards VRT, and another one ^[30] observed that patients were open to develop new self-improvement strategies as a result of their experience with VRT. In summary, given the limited evidence on the level of patient acceptance of VRT, it is suggested that future research should include instruments to evaluate this aspect as well.

The findings from the above-mentioned evaluations showed that patients’ satisfaction and motivation with VRT promoted a higher level of commitment, adherence, active engagement, and dedication to the rehabilitation process. Personal motivation due to VR was promoted by feelings of increased curiosity, self-confidence, self-driven exploration, and imagination which led to a greater enjoyment of the CT in addition to functional improvements ^{[23] [29] [32]}. Moreover, social motivation that led to the development of self-improvement and self-esteem were promoted through competition as a result of the social interaction of patients based on the level of difficulty and scoring of VR games ^{[23] [29]}.

As it was confirmed in other studies ^{[33] [34] [36]}, these key elements are not induced by CT, which limits their level of effectiveness by showing low level of attendance and adherence to the training exercises, thus limiting the level of intensity necessary to achieve patient recovery.

This increase in motivation may be related to the influence of the video games used in VRT with a scoring and reward system (positive feedback) ^{[29] [34]}, and that it decreases the perception of effort ^[37], which boost active patient participation and therefore increases adherence and commitment to the rehabilitation therapy.

In addition, some studies ^{[35] [38]} have confirmed that VRT promotes a deep motivation in patients, which improves their subjective initiative and commitment to actively complete various rehabilitation exercises, thus creating a virtuous circle that improves their functional recovery.

Some studies included in the SR underlined that the VRT was the most effective tool for improving neuroplasticity and subsequent recovery of motor function in SCI patients through intense and repetitive task-oriented practice by increasing exercise therapy time expressed as time dedicated to practice (dosage) compared to CT ^{[36] [38]}, which may enhance functional recovery ^{[23] [26] [29] [30] [32] [34]}. However, it is important to point out that other studies report that the evidence of neuroplasticity as a result of training in VR is currently modest and more research it is needed ^[38]. Also, it should be noted that the relation between dosage and achieving functional recovery is currently an unsolved issue in rehabilitation studies ^{[30] [32] [36]}, where the need for further evidence is highlighted.

There is some additional evidence resulting from the authors’ perceptions during the interventions. For instance, when patients immersed themselves in VRT game activities, they forgot about certain fears that could have affected their performance vis-à-vis the objectives of the CT activities ^[26]. The projected images had a positive impact on the patients’ moods and reduced the perception of pain from the SCI ^[27], which reaffirmed the convenience of using VRT in conjunction with CT ^{[28] [30]}.

The importance of the familiarization process between the patient and the VR technology is a key element to achieve the objectives of the rehabilitation programs as it allows the patients to be engaged with the VRT and to perform the training exercises in a more effective way, thus promoting their active participation and motivation. In this regard, some studies have highlighted ^{[26] [28] [33] [37]} the limitations of commercial VR systems to fit the needs of SCI patients, since these are designed to be used by people without motor or cognitive impairments. Additionally, previous studies ^{[34] [38] [39]} have confirmed the importance of patients perceiving a greater immersion in the virtual environment rather than in the real world, which is related to the software and hardware characteristics.

Therefore, it is necessary that research teams seek to adapt existing devices to provide better grip and manipulation of the peripherals of VR systems ^{[29] [33]}, or still better to design new devices that allow an improved handling and capture of the movements made by patients according to their motor and cognitive skills, along with an appropriate calibration according to their neurological conditions ^[29], which will increasing their interaction with the virtual environment, thus a better immersive experience.

On the other hand, another important element that allows patient familiarization is the sense of presence, which is related to the patients' subjective experience of feeling inside the virtual environment and is able to active brain mechanisms underlying sensorimotor integration as well cerebral networks regulating focused attention promoting patients to perform rehabilitation programs and patients’ movement recovery ^{[34] [37]}. Since this also depends on the characteristics of the VR system itself ^[37], it is important that the games are able to imitate both in execution and visually the task-oriented activities and the virtual scenario, as well as to increase the sensory feedback through the peripherals, which allows a higher level of realism when interacting with 3D objects in real-time, for instance the realistic haptic feedback achieved by the CyberGlove compared to the simple feedback offered by most commercial consoles ^{[24] [33]}. In addition, as already pointed out by some studies included in this review, the patient's embodiment in the virtual avatar, either in one part (first person) or the whole body (third person), has a key role in developing a sense of presence in the virtual environment, as it allows the sensation that the actions performed belong to the patient ^[34]. In this sense the non-commercial devices achieved better results, especially TOyRA system ^{[23] [25]} by achieving an avatar based on the patients’ anthropometric data.

In addition, it would be important to include induction training programs ^{[30] [32]} that make it possible to be familiar with the VR system and interact with greater confidence within the virtual environment. Therefore, all the elements that allow familiarization increase the acceptance of the system by the patient as he/she feels that real world movements are performed within the virtual environment, which allows to obtain adequate functional improvements by providing a better transfer of skills to the real world and open the possibility of continuing rehabilitation programs at home ^{[23] [33]}.

Fourthly, another efficacy indicator was the ability to gradually reduce the help required from the therapist as therapy progressed, leading to more patient independence to choose the VRT activities designed in accordance with the rehabilitation goals ^{[23] [29]}. Nevertheless, some authors reported that VRT had been more effective with the help of a therapist ^[32].

Fifthly, potential home-based rehabilitation and telerehabilitation were other key efficacy indicators of the use of VR technology based on commercial consoles and other devices. Having more intuitive, smaller sized, home-based systems could possibly increase the amount of time devoted to rehabilitation following therapy administered at the hospital or rehabilitation center. In this context, it should be stressed ^[23] that patients stated their interest in using the virtual system at home and would suggest its use to other patients with the aim of potentially creating an online gaming network to promote more socializing among patients and to extend therapy time.

Moreover, home-based VR rehabilitation could have a positive impact on reducing costs, time, effort, and travel of patients to the clinics, with systems being adapted to patients’ needs, particularly those with a high SCI level ^{[29] [32]}. Based on the aforesaid, it would be necessary to have data capture systems imbedded in the consoles or in future rehabilitation dedicated devices that are capable of sending information of the activities performed at home to the therapist for analysis ^{[25] [32]}.

It is important to emphasize that some features of the VR devices posed some potential limitations to the efficacy of VRT, for instance: the virtual environment used by the TOyRA system did not represent execution of ADL in a realistic fashion ^[23]; the commercial consoles or peripherals used in the interventions were relatively obsolete given the ongoing technological advances seeking to make VR more efficient and easy to use in rehabilitation; and the consoles or peripherals could not be easily modified (black box), thus preventing adaptation to patients’ functional needs ^{[22] [26] [28] [29] [30]}.

It should be noted that only one paper reported having used a graphic engine (Unity 3D) ^[32], given its relevance to the creation of virtual environments and the effect of VR on rehabilitation. Using a graphic engine implied a significant investment in terms of time and practice in order to create an appealing virtual environment where the patient can properly interact. Therapists collaborated on this design effort ^{[23] [25] [32]}.

Finally, this SR identified some evidence of the impact of VRT in other neurological disorders, which confirms some of the findings for SCI, since the impaired motor function is one of its common consequences. For instance, some reviews have been identified concerning the use of VR technology applied to Stroke ^{[38] [39] [40] [41]}, Parkinson’s Disease ^{[35] [42] [43]}, Multiple Sclerosis ^{[37] [44]}, Cerebral Palsy ^[45] and Traumatic Brain Injury ^[46], which have reported positive effects of VRT improving balance control ^{[35] [40] [42] [43] [44] [45] [46]}, UL or LL motor function ^{[37] [38] [39] [40] [41] [45]}, gait ^{[35] [40] [42] [43] [44] [46]} and cognitive function ^{[37] [42] [46]}. A systematic review ^[47] showed positive impacts in the treatment of phantom limb pain due to a greater immersive experience based on the use of the mirror therapy which induces the perception that the amputated limb is performing the tasks. However, the authors are inconclusive regarding the efficacy of VR and Augmented Reality therapy and continues to need further research with higher quality evidence.

As is the case of the interventions with SCI patients, the role of VR as rehabilitation tool in these neurological disorders is still under discussion, although some studies reported positive results when VR was used as a complementary therapy which improved ADL performance and quality of life. These studies provided evidence points toward the advantages of increased motivation, confidence, engagement, and increase the intensity of movement based in repetitive and task-oriented with multisensory feedback which is needed for promoting neuroplasticity ^{[35] [37] [39] [40] [43] [45] [46]}.

These results align with the main findings on the use of TRV in patients with SCI and for future development could be important to follow research on how the VR devices had been designed to these pathologies, including the greater use of VR full immersive and multisensory feedback ^[33].

In this regard, an interesting feature showed in a study applied to Stroke ^[40] was the introduction of a specific type of rehabilitation visual feedback named “virtual teacher”, which can be displayed during every task repetition and that shows the correct execution movement of the training exercise so that the patient can imitate it allowing real-time visual comparison between a patient’s execution and the virtual teacher’s execution of a movement. The incorporation of this characteristic into the VR systems for SCI could improve the motor performance quality promoting motor adaptation via supervised learning mechanism.

Limitations and future work of the systematic review

One of the main limitations was not having a larger number of papers providing evidence on the use of augmented reality and fully-immersive VR technology as a rehabilitation tool.

Patient sample heterogeneity as well as the application of different types of VR technology and VRT protocols made it impossible to generalize the results and applied therapies, and since this SR was guided by the Cochrane recommendations ^[14], it was deemed advisable not to do a meta-analysis.

Moreover, no controlled clinical trials were found to focus on the use of VRT individually in the IG compared only with the CT. This could be based on the fact that there is not much research on this, and it was not possible to access a more specialized, fee-based database.

The authors of this SR consider it necessary to update this SR in the future to largely include blinded, randomized, controlled clinical trials with long-term follow-up to reduce the risk of bias and follow-up on the sustained effects of the intervention.