Statistical concerns, invalid construct validity, and future recommendations
-
Hamid Sharif-Nia
,
Long She
,
Jason Osborne
,
Ozkan Gorgulu
,
Fatemeh Khoshnavay Fomani
,
Amir Hossein Goudarzian
Abstract
The need for accurate measurement of constructs and understanding of diverse phenomena has prompted researchers to investigate the validity and reliability of developed measures in a continuous effort to improve our understanding of their psychometric properties. However, at times, these endeavors are flawed by serious methodological inadequacies. In this article, we discussed one example of a psychometric study that appears to be deeply flawed, providing some recommendations for remediating the issue.
1. Mokkink LB, Terwee CB, Knol DL, Stratford PW, Alonso J, Patrick DL, et al. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: A clarification of its content. BMC Medical Research Methodology. 2010;10(1):22.
2. Oliveira MR, Gomes Ade C, Toscano CM. QUADAS and STARD: evaluating the quality of diagnostic accuracy studies. Rev Saude Publica. 2011;45(2):416-22.
3. Shamsalinia A, Ghadimi R, Chafjiri RT, Norouzinejad F, Pourhabib A, Ghaffari F. Nutrition self-efficacy assessment: designing and psychometric evaluation in a community-dwelling elderly population. Journal of Health, Population and Nutrition. 2019;38(1):1-10.
4. Kaiser HF. A second generation little jiffy. 1970.
5. Osborne JW, Banjanovic ES. Exploratory factor analysis with SAS: Sas Institute; 2016.
6. Costello AB, Osborne J. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical assessment, research, and evaluation. 2019;10(1):7.
7. Osborne JW. Best practices in exploratory factor analysis: CreateSpace Independent Publishing Platform; 2014.
8. Tabachnick BG, Fidell LS. Using Multivariate Statistics: Pearson; 2013.
9. Plonsky L. Advancing Quantitative Methods in Second Language Research: Taylor & Francis; 2015.
10. Osborne JW, Fitzpatrick DC. Replication analysis in exploratory factor analysis: What it is and: Why it makes your analysis better. Practical Assessment, Research and Evaluation. 2012;17(15):1-8.
11. Pahlevan Sharif S, Naghavi N, Sharif-Nia H. Concerns regarding the validity of nutrition self-efficacy questionnaire among Iranian elderly population. Journal of Health, Population and Nutrition. 2022 Mar 25;41(1):11.
12. Koran J. Indicators per factor in confirmatory factor analysis: More is not always better. Structural Equation Modeling: A Multidisciplinary Journal. 2020;27(5):765-72.
13. Shi D, Maydeu-Olivares A, Rosseel Y. Assessing fit in ordinal factor analysis models: SRMR vs. RMSEA. Structural Equation Modeling: A Multidisciplinary Journal. 2020;27(1):1-15.
14. Lose G, Klarskov N. Why published research is untrustworthy. International Urogynecology Journal. 2017;28(9):1271-4.
2. Oliveira MR, Gomes Ade C, Toscano CM. QUADAS and STARD: evaluating the quality of diagnostic accuracy studies. Rev Saude Publica. 2011;45(2):416-22.
3. Shamsalinia A, Ghadimi R, Chafjiri RT, Norouzinejad F, Pourhabib A, Ghaffari F. Nutrition self-efficacy assessment: designing and psychometric evaluation in a community-dwelling elderly population. Journal of Health, Population and Nutrition. 2019;38(1):1-10.
4. Kaiser HF. A second generation little jiffy. 1970.
5. Osborne JW, Banjanovic ES. Exploratory factor analysis with SAS: Sas Institute; 2016.
6. Costello AB, Osborne J. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical assessment, research, and evaluation. 2019;10(1):7.
7. Osborne JW. Best practices in exploratory factor analysis: CreateSpace Independent Publishing Platform; 2014.
8. Tabachnick BG, Fidell LS. Using Multivariate Statistics: Pearson; 2013.
9. Plonsky L. Advancing Quantitative Methods in Second Language Research: Taylor & Francis; 2015.
10. Osborne JW, Fitzpatrick DC. Replication analysis in exploratory factor analysis: What it is and: Why it makes your analysis better. Practical Assessment, Research and Evaluation. 2012;17(15):1-8.
11. Pahlevan Sharif S, Naghavi N, Sharif-Nia H. Concerns regarding the validity of nutrition self-efficacy questionnaire among Iranian elderly population. Journal of Health, Population and Nutrition. 2022 Mar 25;41(1):11.
12. Koran J. Indicators per factor in confirmatory factor analysis: More is not always better. Structural Equation Modeling: A Multidisciplinary Journal. 2020;27(5):765-72.
13. Shi D, Maydeu-Olivares A, Rosseel Y. Assessing fit in ordinal factor analysis models: SRMR vs. RMSEA. Structural Equation Modeling: A Multidisciplinary Journal. 2020;27(1):1-15.
14. Lose G, Klarskov N. Why published research is untrustworthy. International Urogynecology Journal. 2017;28(9):1271-4.
| Files | ||
| Issue | Vol 11 No 1 (2024): Winter | |
| Section | Commentary(s) | |
| DOI | https://doi.org/10.18502/npt.v11i1.14938 | |
| Keywords | ||
| construct validity; exploratory factor analysis, confirmatory factor analysis | ||
| Rights and permissions | |
|
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
How to Cite
1.
Sharif-Nia H, She L, Osborne J, Gorgulu O, Khoshnavay Fomani F, Goudarzian AH. Statistical concerns, invalid construct validity, and future recommendations. NPT. 2023;11(1):16-21.
