This was a cross-sectional descriptive study that examined within-session performance characteristics from a web-based motor skill task (Super G) and their association with AD risk factors among adults recruited from the MindCrowd study.

Participants were recruited in May of 2021 through MindCrowd, a web-based research study launched in 2013 to crowdsource demographic, medical history, lifestyle, and cognitive data to identify risk factors of AD [22,23]. Emails were sent through MindCrowd to a subset of 662 individuals who met our inclusion or exclusion criteria and were older than 45 years, who had previously provided a dried blood spot or saliva sample for APOE genotyping (see section “AD Risk Factors and Other Participant Characteristics From MindCrowd” for details) and had provided consent to be contacted for future studies. A hyperlink was included in the email that directed individuals to the Super G game website, on which participants digitally provided consent (approved by the Arizona State University institutional review board study no. 13081). Of the 662 individuals emailed, 54 participants (age: mean 62.39, SD 7.4 years; female: n=39) appropriately registered and completed all 75 trials of the game, equating to an 8.1% response rate. The mean (SD) time between MindCrowd data collection (specifically verbal learning, see section “AD Risk Factors and Other Participant Characteristics From MindCrowd” for details) and Super G data collection was 5.9 (1.4) years.

The Super G game was developed in Unity 5.3.1 and is hosted on Hostinger. Thus, participants were not required to download an app or program to their device. Super G was developed as a gamified version of a seminal motor skill paradigm [24] and has been validated against the original version [21]. The goal of Super G is to help an astronaut explore as many planets as possible within the game’s solar system. There are 16 planets to visit but only 75 attempts to reach them all. Participants use the left and right arrow keys on their keyboard to move the astronaut onto a planet. However, the game uses a rate control mechanism that may not be immediately apparent to participants [21]. Specifically, pressing the right arrow key applies a constant positive force to move the astronaut toward the planet, while pressing the left arrow key applies an equal negative force away from the planet. Since the virtual environment lacks gravity or drag, any force applied will result in a constant velocity until a negative force is applied to slow it down. Thus, participants must learn to apply negative force at the right time and for the right duration to land the astronaut on each planet.

Each trial in Super G begins with the astronaut positioned on the left side of the screen on the initial start planet. The target planet is located on the opposite side of the screen to the right (Figure 1A). The trial lasts 4.5 seconds, but the astronaut cannot leave the start planet until 1.5 seconds have elapsed, as indicated by the disappearance of the blue atmosphere around the start planet (Figure 1B). If the astronaut leaves too early, the trial resets and the astronaut is returned to the initial position on the start planet. Once the blue atmosphere disappears, the participant has 3 seconds to land the astronaut on the target planet. To achieve a successful landing, the astronaut must stay within the boundary of the target planet for 1 continuous second. If successful, a reward tone plays, and fireworks erupt from the target planet (Figure 1C). After a successful landing, Super G repositions at the start position, and the previously landed planet becomes the start planet, with a new target planet appearing in its place (Figure 1D). In the event of a failed landing, the planets for the next trial remain the same, and astronaut reappears at the start position. All 75 trials are completed within a single session for a total session length of approximately 6 minutes.

Cursor position and key press data from each trial of Super G were collected at 100 Hz, along with high scores, time and date of each trial. From these data, four performance variables were calculated. First, scaling ratio (SR) represents the ratio of negative force (duration of left button press) applied to Super G over positive force (duration of right button press), where values equal to 1 indicate equal scaling of forces. Second, time of reversal (TR) is a measure of how well participants execute the timing of their movements by identifying when the left arrow key press occurs during the trial, where higher values are reversals that occur later in the trial. Successful trials require equal scaling of forces (ie, SR close to or equal to 1) and a reversal timed at the midway point of the trial, although we note that there can be trade-offs between the SR and TR to still allow for some degree of success in the task. Third, total time in the target (TinT) planet is the time that the cursor stayed within the target planet, with higher values representing better performance. Fourth, response time (RT) is a measure of how soon the cursor left the home planet after the disappearance of the start planet’s atmosphere, and lower values equate to a faster RT.

Although these performance metrics do strongly correlate with one another, each represents distinct phases of individual motor skill acquisition, as it is possible to have a fast RT with a low TinT if either timing of cursor reversal or scaling of forces is not well executed. Overall, average TinT across all 75 trials is the primary measure of performance since it directly represents the task goal and performance of the task as it relates to execution of SR and TR. Average RT across all 75 trials was considered as a secondary measure of performance because it represents the earlier stage of skill acquisition as participants first need to anticipate when to exit the start planet prior to execution of their acquired movement strategy. Average TinT and RT across the 75 trials are used as the Super G metrics of individual performance due to our prior work [21], which demonstrated that average TinT and RT well describe individual within-session change in performance and delayed retention.

Super G data were harmonized with MindCrowd through merging of hashed email addresses linked to MindCrowd data and used as the login for the Super G game. Super G variables were then merged with participant age, sex, level of education, verbal learning (measured via paired associates learning [PAL]) score [25], simple visual reaction time (svRT), and APOE genotype data, which were available in MindCrowd. The PAL is a verbal learning task that measures the ability to remember the associations between different word pairs. Specifically, participants are visually presented with 12 word pairs, with each word pair presented separately and at 2-second intervals. Participants were then presented with the first word of each pair and then used their keyboard to type the missing word. This procedure was repeated for 2 additional trials. The maximum score on the PAL is 36 (12 words across 3 trials). The svRT task measured the median reaction time across 6 trials, in which participants had to press any keyboard button as soon as a target stimulus appeared on their screen, with reaction time for each trial recorded in milliseconds. Median svRT was used due to the skewness of the distribution of reaction time in each participant’s individual data (skewness svRT median 1.2 vs skewness svRT mean 4.1). The svRT test was used in this study as a control variable only. Participants were classified as APOE ε4 carriers or noncarriers. Carriers were defined as individuals who had either 1 or 2 copies of the ε4 allele, and noncarriers were defined as individuals with 0 copies of the ε4 allele. This was based on prior genotyping from biospecimen collection via self-administered saliva or dried blood spot kits that were mailed to the participants by MindCrowd and then processed by the Translational Genomics Research Institute. Details regarding biospecimen collection and genotyping can be found here [26]. Complete visualization of APOE ε4 carrier status with svRT, PAL, and age can be viewed in Figure S1 in Multimedia Appendix 1.

Wilcoxon rank sum tests were used due to a difference in sample sizes between APOE ε4 carriers (n=23) and noncarriers (n=31), since uneven sample sizes may lead to inaccurate and disproportionate estimates of variance of each group and violate assumptions of parametric testing. This determined whether there were differences in age, PAL score, svRT, and Super G performance between the 2 groups. Chi-square tests were used to determine whether there was a difference in proportion based on sex, race, education, and ethnicity between carriers and noncarriers. To provide broader context of the verbal learning (PAL) and reaction time (svRT) of this sample, percentile scores were calculated for each participant adjusted for their age, sex, and level of education. This allowed for better contextualization of the relative performance on each measure between carriers and noncarriers relative to the entire MindCrowd cohort (ie, if participants in this study are under- or overperformers compared with what would be expected to a random sample across the entire cohort).

Separate linear and logistic multivariable regression analyses, depending on outcome variable type, were used to test the relationship between Super G performance and the dependent variables, that is, AD risk factors. Specifically, different models were constructed to identify which Super performance metrics are related to specific AD risk factors, while controlling for possible confounding effects of the other AD risk factors and participant characteristics. Multivariable logistic regression was used for the dependent variable of APOE ε4 carrier status (where carriers were coded as “true” and noncarriers as “false”), along with the primary and secondary measures of Super G performance (TinT and RT), and control variables of age, sex, PAL score, hour of day Super G played, level of education, and svRT. This approach controlled for the other factors and was repeated with sex as the dependent variable (whereby male was coded as “true” and female as “false”) while switching APOE ε4 carrier status to a control variable. Multivariable regression was used when the dependent variable was PAL score, with a primary independent variable of mean Super G performance (TinT, RT, SR, or TR), and control variables of age, sex, APOE ε4 carrier status, hour of day Super G played, level of education, and svRT. The same approach was repeated with age as the dependent variable while switching PAL score to a control variable. In addition, to control for the potential delay between initial PAL scores and Super G measurement, PAL-adjusted scores were also generated and analyzed. Individual PAL scores were adjusted based on the amount of time between their PAL and Super G measurement, given previously reported expected decline in PAL based on age reported by Talboom and colleagues [26]. This resulted in an expected 0.2-point decline in PAL for every year between the initial PAL and current Super G measurement. The formulation of each model can be viewed in the following equations:

Participants could play Super G at any time throughout the day, the variable of hour played (measured with a 24-hour clock rather than a 12-hour clock) was transformed using a cosine function to ensure that adjacent hours 0 and 23 were close together. This is visualized in Figure S2 in Multimedia Appendix 1. All numeric variables were standardized (age, svRT, PAL, hour played, and Super G Performance) to be centered at their respective mean and divided by their respective SD. Thus, a 1-unit change in the results of these variables with respect to all reported odds ratios (ORs) and beta coefficients represents a 1 SD change in either the outcome or the performance metric. This allowed for better relative comparison between all independent variables within each model. To detect multicollinearity, the variance inflation factor was calculated, and any variable with a variance inflation factor of >5 was removed. Outliers were identified using Cook’s distance and removed if their distance was >1.

The study protocol was reviewed and approved by the office of Research Integrity and Assurance at Arizona State University (approval number STUDY00013081). To participate, participants needed to provide digital informed consent. All data collected from this experiment were deidentified for privacy and confidentiality. Participants were not compensated for their time in this study.

Overall, there were 23 APOE ε4 carriers (20 heterozygotes and 3 homozygotes) and 31 noncarriers. Between-group comparisons (carriers vs noncarriers) using the Wilcoxon rank sum test demonstrated that groups did differ by age (W=471; P=.04), with noncarriers being an average of 4.3 years older than carriers (carriers=59.9 years, noncarriers=64.2 years). For all other control variables, there were no observed group differences (Table 1).

Within the APOE ε4 carrier status logistic regression (model 1) there was a significant association between APOE ε4 carrier status with Super G RT (OR 0.12, 95% CI 0.02-0.44; P=.006) (Figure 2A). Thus, a participant with a Super G RT that was 1 SD below the mean would have an 88% increase in odds of being an APOE ε4 noncarrier than carrier. Visual comparison between all Super G performance metrics and APOE ε4 can be visualized in Figure S3 in Multimedia Appendix 1. All other control variables, including PAL (P=.62) and svRT (P=.29), were not significantly related to APOE ε4 carrier status. Within the participant age linear regression (model 3), Super G TinT was significantly associated with age (β −3.97, 95% CI −6.64 to −1.30; P=.005) (Figure 2C) and within the participant sex logistic regression (model 2), TinT was also associated with sex (OR 32.03, 95% CI 3.74-1192.61; P=.01) (Figure 2B), with lower TinT values (poorer performance) associated with being older and being female, respectively. This is consistent with our earlier work in other cohorts [27]. Visual comparison between all Super G performance metrics and sex can be visualized in Figure S4 in Multimedia Appendix 1.

Given that previous research in MindCrowd has demonstrated a main effect of sex on verbal learning (as measured with the PAL test) [26,28], and that TinT is also strongly linked with participant sex (Figure 2B), within the PAL linear regression (model 4), a sex-by-TinT interaction was also included to best model PAL [26]. Results from the PAL model demonstrated that age was associated with PAL (β −3.37, 95% CI −6.34 to −0.4; P=.03), indicating that older age was associated with lower PAL scores. There was also a main effect of participant sex (β_Male −7.73, 95% CI −14.98 to −0.45; P=.04). This result indicates that male participants scored an average of 7 points lower on the PAL than females. There was a significant sex-by-TinT interaction (β_Male:TinT 6.77, 95% CI 0.34-13.19; P=.04). This result indicates that males with a mean TinT 1 SD greater than the group mean are associated with an increase of 6.77 points on the PAL compared with females with the same TinT performance (Figure 2D). Furthermore, in the PAL adjusted regression (model 5), where PAL scores are modified based on the time between PAL and Super G measures, the results are nearly identical to the raw scores (β_age −3.41, 95% CI −6.38 to −0.44; P=.03; β_Male −7.57, 95% CI −14.81 to −0.33; P=.04; β_Male:TinT 6.72, 95% CI 0.31-13.14; P=.04; Figure S5 in Multimedia Appendix 1). The results of this adjusted analysis provide evidence that the delay between PAL and Super G measures does not significantly impact the observed relationship between PAL and TinT given participant sex.