The university’s ethics committee of La Salle Higher Center for University Studies determined that, given the design of the study, formal ethics committee review and approval were not required and therefore granted an exemption, authorizing the study to be conducted without ethics committee approval. All students signed an informed consent document before participating, which explained the objectives of the study, the confidentiality of the data, and the voluntary nature of their participation. The database was encoded using an alphanumeric code, and no personal data was recorded. No compensation was provided to the participants.

This study used an exploratory sequential mixed design, combining qualitative and quantitative methods for the development, construction, and validation of the CuSAERS. The first qualitative phase, which included a literature review, semistructured interviews, expert content validation, cognitive interviews, and a pilot test, has been previously published [25]. This first phase had been aimed at identifying the dimensions of satisfaction perceived by students with regard to educational activities carried out on social media, such as interaction with the teacher, the quality of the content, and collaboration between colleagues.

The second phase focused on the psychometric evaluation of the CuSAERS. During this stage, the questionnaire was administered to a representative sample of bachelor’s and master’s students to assess its psychometric properties, including validity and reliability.

The sample selected on a nonprobabilistic basis consisted of 160 bachelor’s and master’s degree students in physiotherapy from a Spanish university. Of these, 110 were bachelor’s students and 50 were master’s students. All participants were aged 18 years or older and enrolled in official academic programs. As an inclusion criterion, participants had to have previously participated in educational activities developed on social media, specifically on X or Instagram.

The master’s students participated in an academic activity designed on Instagram. This activity consisted of the critical analysis of therapeutic exercises published in video format on this platform. Students watched the videos, identified technical errors, proposed modifications, and justified their proposals based on scientific evidence. This task not only fostered autonomous and critical learning but also promoted the use of social media as a professional learning tool. The students’ responses and observations were subsequently discussed in a digital forum supervised by the teacher, who acted as moderator and interlocutor, offering specific and targeted feedback.

The bachelor’s students participated in an activity developed on X. This activity included a virtual debate on aspects related to chronic pain. Each student had to perform prior research on the assigned topic and present their position in the form of X threads, using technical but accessible language. The debates were enriched with bibliographical references and the integration of specific hashtags to facilitate the monitoring of the discussions. The teacher’s involvement focused on moderating the debate, posing critical questions to deepen the arguments, and providing direct feedback on the content and quality of the interactions.

Both activities were designed to maximize students’ active participation, integrating the use of social media as an innovative pedagogical resource. The strategies used sought to foster critical, technical, and communicative skills in digital contexts, thus contributing to more dynamic and meaningful learning.

The normality of the data was assessed through a comprehensive analysis that included both statistical tests and graphical methods, providing an in-depth understanding of the distribution of the items in the CuSAERS. The results of the Kolmogorov–Smirnov test indicated that the data significantly deviated from a normal distribution (P<.001), confirming that the variables did not meet the assumption of normality. This finding was consistent with skewness and kurtosis values, which suggested deviations from perfect symmetry and distributions far from the ideal mesokurtic shape. Most of the items showed negative skewness, indicating a tendency for participants to give higher scores on the satisfaction scale, while predominantly platykurtic kurtosis reflected lower concentration at the scale’s extremes.

The visual analysis complemented these statistical evaluations. Q–Q plots and histograms demonstrated that the data did not align with the theoretical normal curve, confirming the presence of biases and lighter tails in the distribution of scores. Additionally, the influence of outliers was explored using boxplots, which revealed greater dispersion in specific items, such as Item 8 and Item 12. These items, which had the highest SDs, reflected the heterogeneity in the participants’ perceptions regarding these specific aspects of the questionnaire.

The descriptive analysis of the items showed means and medians centered around values close to 3, reflecting a tendency toward neutral or moderately positive responses. SDs ranged between 0.752 and 1.046, demonstrating differences in the dispersion of responses. Items with the highest dispersion were Item 8 and Item 12, whereas Item 16 presented the lowest variability.

Regarding distribution, most items exhibited negative skewness, suggesting a slight tendency toward higher scores, except for Item 8, which showed skewness close to 0. Platykurtic kurtosis predominated, indicating lower concentration of responses at the extremes, except for some items, such as Item 3 and Item 4, which displayed more leptokurtic distributions.

In terms of internal consistency, the item-total correlations were heterogeneous, ranging from −0.001 (Item 16) to 0.539 (Item 10). Items 11 and 16 stood out for having near-zero or negative item-total correlations, suggesting a lower contribution to the instrument’s consistency. Cronbach α and McDonald ω coefficients ranged between 0.645 and 0.716, and between 0.674 and 0.755, respectively, indicating moderate reliability overall. The highest reliability values were observed for Items 11 and 16, although these items had low item-total correlations, which might imply atypical behavior for these items (Table 1).

The data were suitable for factor analysis (KMO=0.754; Bartlett’s χ²₄₅=685; P<.001). An exploratory factor analysis with oblimin rotation supported a 3-factor solution explaining 61.9% of the variance (sum of squared loadings: 3.07, 1.56, 1.56; 30.7%, 15.6%, 15.6%). Interfactor correlations were small (r=0.06-0.20).

Factor 1 (perception of learning) showed high primary loadings for Item 10 (0.821), Item 4 (0.820), Item 3 (0.803), Item 7 (0.748), and Item 9 (0.695), with uniqueness ranging from 0.318 to 0.518 (Table 2).

Overall model fit was acceptable (RMSEA=0.058; 90% CI 0.00-0.100; TLI=0.961; χ²₁₈=27.9; BIC=–63.4; P=.06). All retained items met the a priori loading threshold (≥0.40), and cross-loadings were minimal (Table 2). The scree plot and parallel analysis supported the retention of three factors: the first 2 observed eigenvalues were clearly above the simulated distribution, the third was approximately at the simulated threshold, and from the fourth onward, the observed eigenvalues fell below the simulated ones (Figure 1). We note that Item 8 presented a boundary (Heywood) estimate—an occurrence that can arise in small samples and oblique solutions—so this third factor should be considered provisional pending confirmatory testing and item redevelopment in an independent, larger sample.

Consistent with the scree plot and parallel analysis (Figure 1), we retained the 3-factor solution as the primary model (Table 3). The 2-factor alternative, although parsimonious, collapses conceptually distinct domains and does not map cleanly onto the theorized constructs (Table S1 in Multimedia Appendix 1). The 4-factor alternative introduces a psychometrically weak structure: after applying the a priori retention rule (primary loading ≥0.40), one factor is left with a single indicator (Table S2 in Multimedia Appendix 1), which is not recommended for latent constructs. Taken together, the 3-factor model offers the best balance between empirical support and theoretical interpretability.

Table 4 presents the descriptive statistics and reliability coefficients for the evaluated scales. The overall mean for the CuSAERS was 2.93 (SD 0.503), with moderate internal consistency coefficients for both Cronbach α=0.743 and McDonald ω=0.787. Among the subscales, perception of learning showed the highest internal consistency (α=0.882; ω=0.884) and a mean of 3.06 (SD 0.662), indicating a positive perception of learning.

Concurrent validity was assessed using Spearman correlations between the CuSAERS scores and the ASS, as well as prior experience with social media (Table 5). The results showed that the total CuSAERS score had a moderate positive correlation with the ASS (ρ=0.59; P<.001), supporting its validity as an instrument to evaluate the perception of educational activities on social media. Additionally, significant positive correlations were observed between the ASS and the subscales perception of learning (ρ=0.44; P<.001), task satisfaction (ρ=0.31; P<.001), and self-realization (ρ=0.29; P<.01), indicating a consistent relationship between academic satisfaction and the dimensions evaluated by the CuSAERS.

On the other hand, prior experience with social media showed a weak positive correlation with the CuSAERS total score (ρ=0.27, P<.001), as well as with perception of learning (ρ=0.20; P<.01) and self-realization (ρ=0.23; P<.01). However, no significant correlations were observed with the task satisfaction subscale (ρ=0.14; P>.05). These results suggest that, although prior experience with social media has a positive relationship with overall perception and some specific aspects evaluated by the CuSAERS, its influence on dimensions such as motivation is limited (Figure 2).

The discriminant validity of the CuSAERS was evaluated by comparing groups of students based on their academic level (bachelor’s or master’s). These comparisons were conducted using the nonparametric Mann-Whitney U test, and the effect size was calculated using rank-biserial correlation. The statistical results and graphical inspection through box and violin plots reveal clear differences between the 2 groups, supporting the CuSAERS’s ability to discriminate between students with different educational levels (Figure 3).

For the total CuSAERS score (Figure 3A), master’s students had significantly higher scores (U=1113; P<.001), with a median of 39 (IQR 37-41), compared with the bachelor’s group, whose median was 34 (IQR 31-37). Box and violin plots showed greater dispersion in the bachelor’s group, whereas scores for the master’s group were more compact and centered on higher values.

The perception of learning subscale (Figure 3B) also showed significant differences (U=1064; r=0.613; P<.001), with higher scores and less dispersed distributions in the master’s group (median 18, IQR 16-18). The plots reveal greater variability in the bachelor’s group, with individual cases at considerably low levels.

For the task satisfaction (Figure 3C) and self-realization (Figure 3D) subscales, significant differences were found (U=2006; r=0.271; P=.005). For task satisfaction, the master’s group showed higher scores with less dispersion (median 9, IQR 9-10.8), whereas for self-realization, the master’s group’s scores were also higher (median 6, IQR 4.25-6), in contrast to the greater dispersion and lower values observed in the bachelor’s group (Table 6).

The results also indicated differences between students with and without prior experience in social media. The group with experience obtained higher scores across all subscales and in the total CuSAERS score, although the differences were only significant for the total score (U=2514; r=0.2045; P=.03). For the perception of learning and self-realization, although statistical significance was not reached (P>.05), trends toward higher scores were observed in the group with experience (Table 7).

Results for the motivation facet—considered at the content-validation stage but not retained by the final EFA—are provided in Table S3 in Multimedia Appendix 1 for completeness. These results suggest that familiarity with the use of social media might positively influence overall perceptions of educational activities, although its impact on specific subdimensions is more limited.

The present findings support CuSAERS as a preliminary instrument for assessing student satisfaction in social media–mediated learning environments. In its 3-factor configuration—perception of learning, task satisfaction/environment, and self-realization—the scale shows acceptable reliability at the total-score level and coherent relations with external criteria, while some facets remain candidates for refinement. Construct validity, analyzed through EFA, supported a 3-factor solution that aligns with prior conceptualizations of satisfaction in digital educational settings [18,19] and with the multidimensional notion of academic satisfaction [30-32].

Among the 3 identified factors, perception of learning and self-realization showed the strongest relations with external criteria, consistent with previous findings highlighting the importance of acquiring new skills and the sense of achievement in digital learning experiences [5,7]. These components moderately correlated with the ASS, supporting the concurrent validity of the CuSAERS. In line with research linking satisfaction to performance and retention, students who perceived greater learning and personal growth also tended to report higher overall academic satisfaction [33,34]. Socioemotional elements and social support are key in online contexts; recent work shows that emotional support and effective communication through social media increase student satisfaction and motivation [35].

Psychometrically, internal consistency was acceptable overall. The perception of learning dimension exhibited the highest internal consistency, reflecting its stability. Although motivation emerged at the content-validation stage as a relevant facet, the present exploratory analyses did not retain a stable motivation factor. This facet remains a target for item redevelopment and confirmatory evaluation in future studies. Notably, recent studies suggest that visually rich platforms (eg, Instagram) can enhance motivation, whereas text-centric platforms (eg, X) may require higher digital literacy for comparable satisfaction [36,37]. These considerations are congruent with standard scale-development practice, where less stable content is refined after initial validation [26].

Discriminant analyses revealed significant differences between bachelor’s and master’s students, with higher scores among the latter on most subscales. Explanations may include greater academic and professional experience and stronger self-management skills among master’s students [3,23].

By contrast, bachelor’s students engaged in an X-based debate on chronic pain, a context in which lower academic maturity and less experience with educational uses of social media may temper satisfaction [2,9,15,38-41]. Importantly, the bachelor group’s scores were moderately positive, indicating overall acceptance of social media–based activities, while calling for replication in other health disciplines.

Regarding platform differences (X vs Instagram), platform use was not randomized and tasks differed in nature; therefore, any platform contrasts are descriptive and noncausal. Prior literature suggests that Instagram’s audiovisual format may favor motivation and participation [8,42,43], whereas X is powerful for debate but may demand greater digital literacy [11,16].

The moderate correlation between CuSAERS scores and ASS scores further supports the instrument’s concurrent validity, indicating that satisfaction with social media activities is related to overall academic satisfaction [27,28]. Prior experience with social media showed weak-to-moderate associations with satisfaction, suggesting that technological familiarity helps but does not solely determine satisfaction; interaction quality, content relevance, social presence, and the teacher’s facilitating role also matter [13,14,19,21,36,44].

Taken together, the findings support CuSAERS as a preliminary instrument with acceptable reliability at the total-scale level and a provisional self-realization subscale. The scale can inform formative pedagogical reflections, whereas policy or high-stakes decisions should await confirmatory replication and measurement-invariance testing.

This study used a nonprobabilistic, single-institution sample, which limits generalizability [45]. Future work should include larger, multisite samples spanning diverse programs and contexts [46,47]. Platform use was not randomized: students in the master’s degree used Instagram and students in the bachelor’s degree used X, and activities differed, so platform effects are confounded by academic level and task; future studies will randomize platform assignment or use within-subject counterbalanced designs with covariate-adjusted models.

Next steps will subject the 3-factor structure to CFA, test configural, metric, and scalar measurement invariance (degree level, gender, age), and conduct cross-validation in an independent, multi-institutional sample [48]. The motivation facet will undergo item redevelopment prior to confirmatory testing. Beyond psychometrics, it will be useful to relate CuSAERS to outcomes such as academic performance, motivation types, peer and teacher interaction, and participation in learning communities [49].

This study provides initial evidence that the CuSAERS is a promising, early-stage instrument for assessing student satisfaction with learning activities delivered over social media. In its 3-factor configuration—perception of learning, task satisfaction/environment, and self-realization—the scale showed acceptable reliability at the total-score level and coherent associations with an established academic satisfaction measure. Group contrasts by academic level were observed, but platform-related differences should be interpreted cautiously given the nonrandomized design.

CuSAERS should be regarded as an early-stage measure with promising properties. Its routine use will benefit from CFA, measurement-invariance testing, and multisite replication, alongside ongoing item refinement. Within the broader trend toward digitization in higher education, tools such as CuSAERS can inform formative pedagogical decisions and help tailor social media–based activities to student needs, while definitive applications should await confirmatory evidence.