I-DEEL: Inter-Disciplinary Ecology and Evolution Lab
  • Home
  • People
  • Research
  • Publications
  • Blog
  • Open Science
    • Registrations
    • Registered Reports
    • Published Protocols
    • Preprints
    • EDI
    • Other
  • Opportunities
  • Links

Say goodbye to fixed- and random-effects meta-analyses

27/10/2022

0 Comments

 
By Yefeng Yang

As I have been doing more surveys on meta-analytic practices in many disciplines and re-analysing more published meta-analysis (MA) papers, there is one “recommendation” that is growing stronger and stronger in my brain. That is, we should say goodbye to traditional fixed- and random-effects MAs and conduct our MAs using advanced methods like multilevel and multivariate models because meta-analytic datasets are often multilevel and multivariate in nature. Doing so can make sure you properly handle statistical issues like dependency, and heteroscedasticity, resulting in more robust parameter estimations and inferences. My main argument is that in the “worst-case” scenario, where your dataset does not have a complex structure thereof, these advanced models will automatically reduce into a normal fixed- and random-effects models, all with similar (or identical) results to those expected. More importantly, applying advanced methods can help you decompose variances (Figure 1) and separate correlations of true effects from observed effects (Figure 2), delivering new biological insights. I can see the between-study heterogeneity and correlation are overestimated in many published meta-analyses using fixed-and random-effects models.
Picture
Figure 1. Imaginary example of hierarchical data structure.
Although these advanced methods are good, there are (at least) three remarks worth noting here. First, all your models should be built strictly based on predefined questions (e.g., a priori hypotheses). Second, before applying these models, you need to correctly understand the statistical theory behind them. Otherwise, you very likely disseminate misleading information if you published results from them. Third (but not the last), do not use complex models to fit a small-sample-size dataset. This is especially true for multivariate models, which are often heavily parameterized (even overparameterized). So, always do (at least some basic) model checking (e.g., likelihood profile, convergence) to ensure stability of your model fitting.
Picture
Figure 2. Joint probability distribution (bivariate normal joint density). Photo source: Multivariate normal distribution. (2022, October 16). In Wikipedia. https://en.wikipedia.org/wiki/Multivariate_normal_distribution
​As I have been knowing more about statistics, I realised that many methods are just a special form of a more general framework. For example, (two-sample) Student t-test is a special form of ANOVA, which is a special form of linear regression, which is a special form of generalized linear model or linear mixed model, which is a special form of generalized linear mixed model, which is a special form of the generalized additive mixed model. In the same vein, fixed-effect MA is a special form of random-effects MA, which is a special form of a multilevel or multivariate model. I can imagine that one might disagree with “say goodbye to fixed- and random-effects meta-analyses”. For example, fixed-effects MA can still provide valid inferences if limiting your results to the included studies (e.g., conditional inference). I acknowledge this is true as long as you are not goanna generalize results beyond the included studies. I know asking people to resort to complex methods is difficult because people like easily-understandable tools - just think about P-value. I am always open and happy to see different ideas. Lastly, all the above claims only represent my personal intuition and opinion (I might extend it into a paper in future). They might be wrong and do not necessarily speak for my lab’s attitudes toward meta-analyses.
0 Comments

    Author

    Posts are written by our group members and guests.

    Archives

    May 2025
    April 2025
    March 2025
    January 2025
    December 2024
    November 2024
    October 2024
    September 2024
    August 2024
    July 2024
    June 2024
    May 2024
    April 2024
    March 2024
    February 2024
    January 2024
    December 2023
    November 2023
    October 2023
    September 2023
    August 2023
    July 2023
    June 2023
    May 2023
    April 2023
    March 2023
    February 2023
    January 2023
    December 2022
    November 2022
    October 2022
    September 2022
    August 2022
    July 2022
    June 2022
    May 2022
    April 2022
    March 2022
    February 2022
    January 2022
    December 2021
    November 2021
    October 2021
    September 2021
    August 2021
    July 2021
    June 2021
    April 2021
    March 2021
    February 2021
    January 2021
    December 2020
    November 2020
    August 2020
    July 2020
    June 2020
    April 2020
    December 2019
    November 2019
    October 2019
    September 2019
    June 2019
    April 2019
    March 2019
    February 2019
    January 2019
    December 2018
    November 2018
    October 2018
    September 2018
    August 2018
    July 2018
    June 2018
    May 2018
    March 2018
    January 2018
    December 2017
    October 2017
    September 2017
    July 2017
    June 2017
    May 2017
    April 2017
    March 2017
    January 2017
    October 2016
    August 2016
    July 2016
    June 2016
    May 2016
    March 2016

    Categories

    All

    RSS Feed

HOME
PEOPLE
RESEARCH
PUBLICATIONS
OPEN SCIENCE
OPPORTUNITIES
LINKS
BLOG

Created by Losia Lagisz, last modified on June 24, 2015