As I have been doing more surveys on meta-analytic practices in many disciplines and re-analysing more published meta-analysis (MA) papers, there is one “recommendation” that is growing stronger and stronger in my brain. That is, we should say goodbye to traditional fixed- and random-effects MAs and conduct our MAs using advanced methods like multilevel and multivariate models because meta-analytic datasets are often multilevel and multivariate in nature. Doing so can make sure you properly handle statistical issues like dependency, and heteroscedasticity, resulting in more robust parameter estimations and inferences. My main argument is that in the “worst-case” scenario, where your dataset does not have a complex structure thereof, these advanced models will automatically reduce into a normal fixed- and random-effects models, all with similar (or identical) results to those expected. More importantly, applying advanced methods can help you decompose variances (Figure 1) and separate correlations of true effects from observed effects (Figure 2), delivering new biological insights. I can see the between-study heterogeneity and correlation are overestimated in many published meta-analyses using fixed-and random-effects models.
By Yefeng Yang As I have been doing more surveys on meta-analytic practices in many disciplines and re-analysing more published meta-analysis (MA) papers, there is one “recommendation” that is growing stronger and stronger in my brain. That is, we should say goodbye to traditional fixed- and random-effects MAs and conduct our MAs using advanced methods like multilevel and multivariate models because meta-analytic datasets are often multilevel and multivariate in nature. Doing so can make sure you properly handle statistical issues like dependency, and heteroscedasticity, resulting in more robust parameter estimations and inferences. My main argument is that in the “worst-case” scenario, where your dataset does not have a complex structure thereof, these advanced models will automatically reduce into a normal fixed- and random-effects models, all with similar (or identical) results to those expected. More importantly, applying advanced methods can help you decompose variances (Figure 1) and separate correlations of true effects from observed effects (Figure 2), delivering new biological insights. I can see the between-study heterogeneity and correlation are overestimated in many published meta-analyses using fixed-and random-effects models. Although these advanced methods are good, there are (at least) three remarks worth noting here. First, all your models should be built strictly based on predefined questions (e.g., a priori hypotheses). Second, before applying these models, you need to correctly understand the statistical theory behind them. Otherwise, you very likely disseminate misleading information if you published results from them. Third (but not the last), do not use complex models to fit a small-sample-size dataset. This is especially true for multivariate models, which are often heavily parameterized (even overparameterized). So, always do (at least some basic) model checking (e.g., likelihood profile, convergence) to ensure stability of your model fitting. As I have been knowing more about statistics, I realised that many methods are just a special form of a more general framework. For example, (two-sample) Student t-test is a special form of ANOVA, which is a special form of linear regression, which is a special form of generalized linear model or linear mixed model, which is a special form of generalized linear mixed model, which is a special form of the generalized additive mixed model. In the same vein, fixed-effect MA is a special form of random-effects MA, which is a special form of a multilevel or multivariate model. I can imagine that one might disagree with “say goodbye to fixed- and random-effects meta-analyses”. For example, fixed-effects MA can still provide valid inferences if limiting your results to the included studies (e.g., conditional inference). I acknowledge this is true as long as you are not goanna generalize results beyond the included studies. I know asking people to resort to complex methods is difficult because people like easily-understandable tools - just think about P-value. I am always open and happy to see different ideas. Lastly, all the above claims only represent my personal intuition and opinion (I might extend it into a paper in future). They might be wrong and do not necessarily speak for my lab’s attitudes toward meta-analyses.
0 Comments
by Samantha Burke
After over two years of lockdown, I had the opportunity to leave Australia to attend the Ecological Society of America’s (ESA) joint conference with the Canadian Society for Ecology and Evolution (CSEE). This conference marked my first time presenting an oral talk outside of UNSW. While it was exciting to share my research with others, I found learning about others’ research and networking with new people to be an equally exciting experience. As my projects consist of systematic-like research, I was thrilled to see ESA created an entire session dedicated to meta-analysis in ecology. Ecologists are relatively new to conducting meta-analysis of their data, so this session was well-attended and directed conversation towards improving meta-science while it’s still in its early stages in ecology. These talks were all excellent and highlighted the upcoming importance and challenges of conducting systematic-like research in ecology and evolutionary biology. In addition to meeting new people, I was able to connect with researchers I already knew. While in Montreal, I was able to meet I-DEEL’s newest post-doc, April Martinig, in person. April has been working remotely for the past few months, so it was great to attend her presentation on her previous work examining predator-prey interactions in culvert animal passages. As a Canadian citizen, she knew of the best places to go in Montreal, and we chatted over a delicious vegan lunch. We should all look forward to the research she’ll conduct with I-DEEL. I also had the opportunity to meet members of the Society for Open, Reliable, and Transparent Ecology and Evolutionary biology (SORTEE), of which I’m a member. Even though I went to Canada intending to attend the ESA conference, SORTEE members attending the conference gathered for a mini meetup in Montreal. The society was able to reach out to more ecologists at the conference, and many people came to the meetup to hear firsthand what SORTEE is all about. If you’re interested, please check out a previous blog post by Rose O’Dea and the SORTEE website. Attending a conference was such a privilege, especially one as diverse as ESA’s 2022 Conference. I look forward to continuing to share my work and learn from others. By Losia Lagisz 13 - 19 August 2022 has been a very busy and fun week – a week at ESEB (European Society for Evolutionary Biology) Congress in Prague, Czech Republic.
Big thanks to the organisers of ESEB2022 and we hope to be able to attend the next one – ESEB2025 to be held in Barcelona, Spain!
by Kyle Morrison In today’s world Ireland is famous for vibrant cities, cosy pubs and cold Guinness but, in a simpler time – before us humans got involved, it was once the land of giant deer, grey wolves and grizzly bears. Although, some of these animals can be seen elsewhere, a few sadly cannot and were never seen again. Here I provide you five of the coolest animals that ever roamed the Emerald Isle.
Image sources:
by Lorenzo Ricolfi The Italian version of Charles Darwin's The Origin of Species opens with a preface by Luca and Francesco Cavalli-Sforza. They are two of the four children of Luigi Luca Cavalli-Sforza, an Italian geneticist, academic, researcher, and professor emeritus at Stanford University in California, who died in 2018 and became known for his research activities in population genetics. He was also involved in anthropology and history in his studies of human migration. Since I could not find the English version of the preface anywhere, I would like to translate and summarize it in this article. Therefore, the following text is a summary and translation of the preface written by Luca and Francesco Cavalli-Sforza. Translation: "It is said that when Laplace, the great French astronomer, presented Napoleon with a copy of his Celestial Mechanics, in which he described universal gravitation and advanced hypotheses on the formation of the solar system, Napoleon remarked: "Mr Laplace, they tell me that you have written this big book on the design of the universe, without ever mentioning its Creator". "This is a hypothesis I did not need", replied Laplace. When Napoleon, amused, reported this conviction to the mathematician Lagrange, he exclaimed: "What a beautiful hypothesis! It explains a lot! ". Two hundred years later, modern texts on astronomy continue to describe the behaviour of celestial bodies without the need for a God creator. In science, no unnecessary hypotheses are introduced to explain events. While no one nowadays argues about divine intervention in the history of the cosmos, a similar question resurfaces from time to time in biology. Since Darwin's time, the theory of evolution has made enormous progress and can explain a great deal of the history of life. Today, our relationship with primates is no longer in question. It has been proven beyond any reasonable doubt. Nevertheless, it still meets with the most vigorous resistance from the ultra-conservative fringes of Baptist Christians (a powerful political force in the south of the United States) and ultra-orthodox Jews. On the other hand, it does not seem to create any difficulties for either Catholicism or Islam. What is questioned today is whether evolution is sufficient to explain the extraordinary complexity of life: how is it possible that living beings have developed such a variety of forms? How can an organ such as the eye have achieved its extreme complexity only under natural forces? Someone says there must be an Intelligent Design guiding the history of life, intervening in the mechanisms of evolution (with a view to some goal, it is assumed, but this is not stated). The Intelligent Design movement was born as a political fact in the United States; it is promoted by foundations financed by ultra-conservative billionaires and engaged in specific activities, such as supporting those who sue state schools to have the biblical account of creation taught alongside the theory of evolution as an equal alternative. The extreme right-wing label with which the movement was born does not help its spread in Europe, where there has been enough ideology. The absence of scientific arguments makes it fiddly to counter directly. An organism can only live if it interacts with its living environment to obtain food and can only pass on its DNA to the next generation if it becomes an adult and reproduces. However, the environment is constantly changing. Only those who remain 'adapted' to their environment can continue to live. Natural selection acts by automatically filtering, like a rigid sieve, the best types to survive and reproduce, environment by environment and circumstance by circumstance. The theory of evolution by mutation and natural selection says precisely this: living species evolve under the impetus of chance and necessity. Darwin's theory of evolution provides an excellent key to interpreting what we see around us and deepening our knowledge of the molecules that make life possible." - End of translation.
Science and religion have always had harsh disagreements about explaining the existence of the observable universe from the earliest known periods through its subsequent large-scale evolution (of both abiotic and biotic factors). My opinion is that science should not be concerned with the beliefs of others if the views of others do not limit science. But, at the same time, religions should help scientists find the right path following moral rules and ethics. Both science and religion are great powers that give humankind its singularity. Therefore, they should work together to make our species more just, educated and happy. by Shinichi Last week, the I-DEEL lab gathered to have a farewell party for Cat who worked on the "PFAS project" for the last 2 years. This project is our lab’s first research synthesis project in environmental sciences, and Cat played a major role. Now she is in Europe and travelling around the world for the next several months (detoxifying PFAS, I presume).
We also welcomed 4 new PhD students to our lab: Lorenzo, Kyle, Coralie and Jess. Lorenzo will further synthesize the PFAS literature while Kyle will work on the pesticide pollution literature. Coralie will develop new meta-analytic tools, working with Prof David Warton. Jess, who did Honours degree with us already, will apply deep learning methods to Australian wildlife image data, working with Prof Richard Kingsford, people from Taronga Zoo, and NSW Wildlife and National Parks. This is going to be a huge variety of research work - just like the food on the table (see picture above - this is a potluck party where everybody brings a dish!). As they say: “Variety is the spice of life”. I am very much looking forward to what the future will bring to I-DEEL! By Patrice Pottier Being vegan for nearly five years, I have noticed drastic changes in the accessibility and fanciness of vegan food. The days when people thought vegan food only consist of salads and seeds are far behind! Plant-based foods can take all shapes and forms, and I guarantee you that you may not be able to tell some meals are vegan in a blind taste. Let me introduce you to 10 vegan restaurants you must try in Sydney. Forget the old dry veggie patty - I guarantee you won’t be skeptical about vegan food after trying those places.
This are, of course, only a short sample of the amazing range of options Sydney has to offer. Want to find more vegan places? Check out HappyCow – an app that list vegetarian and vegan restaurants worldwide.
I hope you enjoy this culinary discovery! 😊
Anyway, adaptation is a capacity that plays an essential role in evolutionary biology; it is a dynamic process that adapts organisms to their environment, improving their evolutionary fitness. Similarly, but on a different time scale, an individual's acclimatization capacity to a change in its environment enables it to maintain fitness across various environmental conditions. My name is Lorenzo Ricolfi, and, like anyone who has survived these two years of the pandemic, I struggle every day to acclimatize to change. The COVID-19 pandemic has dramatically upset our habits and daily routines. Moreover, it has presented us with a tough challenge: to cope with dramatic and sudden changes. I lived my life in Italy, studying and working as a researcher at the University of Rome, until January 2020, when I took a plane to Brisbane. Study and work followed each other without a gap year, and I needed a breath of fresh air and an adventure before returning to Italy six months later. It was a good plan. Well, it never came true. The World Health Organization declared the outbreak a Public Health Emergency of International Concern on the 30th of January 2020, 21 days after my landing in Australia. On the 31st of January, two Chinese tourists in Rome tested positive for the virus, and Italy was the first country in Europe to be affected by the pandemic. A month and a half later, the Italian army vehicles had to transport the dead out of the city of Bergamo as its crematorium struggled to cope. This disaster happened only a week after the World Health Organization officially declared COVID-19 a pandemic. That was the situation. Australia at that time was in a bubble of its own, far removed from what was happening overseas. I was reading the news on the web, and it all seemed absurdly surreal. Virus? Wheezing and difficulty breathing? Social distancing? Masks? It was hard to assess and assimilate the news with reason and objectivity. And I had to take a decision now and immediately: return to my country or keep staying in Australia. How could I take such a decision lightly? There were many factors to consider and the implications too. Italy was in full lockdown, and although I was worried about my loved ones, I decided to stay, not knowing when I would return. I would return when the situation improved and the pandemic had passed. Days turned into weeks and weeks into months, and I began to need work. I held about ten different jobs in the time that followed. I worked as a dishwasher, a waiter, a kitchen hand, a warehouse worker, a driver, a delivery guy and a carpenter. I had never done any of these jobs before in my life. Months turned into years, and I realized that I wasn't coming back anytime soon. Australia closed its borders. There were no more planes in the sky. Suddenly, my life was completely different, and the sense of nostalgia was strong. If I couldn't go home, I wanted at least to get back to what I was passionate about. I decided to take the English exam necessary to apply for a research project at the university level. I studied and passed the exam with flying colours.
Meanwhile, while surfing the websites of various Australian universities, I found an exciting laboratory at the University of New South Wales in Sydney. As luck would have it, the lab was looking for a PhD student with my background. I immediately got in touch and, after devising a research proposal that matched my interests and the knowledge and skills of the lab, I applied for a scholarship from the Australian government to cover the PhD. It is now the beginning of April 2022, it has been two years and three months since I landed in Australia, and I started my PhD a couple of months ago. The pandemic situation has improved thanks to vaccines, although the pandemic is not over. And I have still never returned home. Over the last two years, the changes in my life have been massive, but I am thrilled with where they have taken me, even though they were unplanned and presented me with some callous times and challenges. The point of all of this is that although we constantly try to categorize, order, and simplify reality, it is permeated by the chaos in which change is the engine. We need order and stillness in our environment and minds, but we cannot avoid change. Instead, we must learn to be flexible enough to shape ourselves without breaking or losing our identity. It is a challenging game based on compromise and sometimes on acceptance and letting go. The pandemic has abruptly put reality before us, where not everything goes as planned. But it also reminded us of one thing: there is nothing wrong with that. Plans in life are necessary, but their implementation is not to achieve a state of happiness. Instead, an idea and a plan can evolve into something completely different. This turning point may initially be seen as a failure, a crack in the wall of our lives. However, it is only after time that we realize that the plan was but one of many steps, rather than the dividing line between success and failure. by Losia Lagisz Removing duplicated records can be cumbersome. When collating bibliographic records from multiple literature databases both the total number of records and the proportion of duplicates can be high making manual removal of duplicates extremely time-consuming. Manual resolution of each set of potentially duplicated records is required when using reference managers such as Zotero or EndNote, and especially a screening platform Rayyan (note that deduplication algorithms available in all these are reasonably good at detecting (flagging) duplicating records (exact and non-exact duplicates), but not perfect, so combining different approaches is recommended anyway). Here, I present an efficient workflow in which records from multiple sources (literature databases) are combined in Rayyan (https://rayyan.ai/), then automatically deduplicated using an R script (www.r-project.org), and finally uploaded into Rayyan again for the final round of deduplication and screening. Importantly, apart from Rayyan and R no other software is needed (but, at any stage, you can import/export lists of records into your reference manager to see the records or convert file formats). I assume you are already quite familiar with Rayyan and R. The workflow: 1. Gather the bibliographic files. Download lists of bibliographic references (with abstracts) from databases used to run the literature searches. Most of the time, exporting thema as a .ris file would work best. Rayyan has guidelines for the most commonly used databases on its upload page (see the screenshot below). ![]() 2. Upload files into Rayan. Create a new project in Rayyan and upload all files into it. This will create a combined list of records. 3. Run deduplication algorithm in Rayan (optional). This will give you an idea on how many duplicated records you have in the combined set of records (if less <200 you may want to resolve them manually in Rayyan). To run the algorithm, press a “Detect duplicates” button close to the top right corner of the view with the list of combined references in Rayyan. 4. Export combined list of records from Rayan. This will create one .csv with all references in the same format. To export the records, press a “Export” button close to the top right corner of the view with the list of combined references in Rayyan. In the pop-up window select “All” and “CSV” format (you can include all the fields listed below these options). Note that Rayyan will send you a link via email to download a compressed file. After decompressing, rename the .csv file to something usable (e.g., "FILENAME.csv") and place it in your R project folder. 5. Upload combined .csv file into R.
Load the R packages needed: library(tidyverse) # https://www.tidyverse.org/ library(synthesisr) # https://CRAN.R-project.org/package=synthesisr library(revtools) # https://revtools.net/ dat <- read.csv("FILENAME.csv") #load the file dim(dat) #see the initial number of uploaded references 6. Prepare data for deduplication in R. We will deduplicate by comparing titles. Before doing so, it is good to tidy them up by bring them to the same case, removing extra white spaces and punctuation. We save these “processed” titles in a new column. dat$title2 <- stringr::str_replace_all(dat$title,"[:punct:]","") %>% str_replace_all(.,"[ ]+", " ") %>% tolower() # Removing all punctuation and extra white spaces 7. Remove exact title matches in R. This step uses processed titles to create a new smaller list of references with exact duplicates removed. It will save computational time for the next step (detection of non-exact duplicates). dat2 <- distinct(dat, title2, .keep_all = TRUE) #reduce to records with unique titles (removes exact duplicates) dim(dat2) #see the new number of records #View(arrange(dat2, title2)$title2) #an optional visual check - sorted titles 8. Deduplicate by fuzzy matching the remaining titles in R. This step uses string distances to identify likely duplicates - it may take a while for long lists of references. duplicates_string <- synthesisr::find_duplicates(dat2$title2, method = "string_osa", to_lower = TRUE, rm_punctuation = TRUE, threshold = 7) #dim(manual_checks) #number of duplicated records found #View( review_duplicates(dat2$title2, duplicates_string) # optional visual check of the list of duplicates detected. If needed, you can manually mark some records as unique (not duplicates) by providing their new record number from duplicates_string (duplicates have the same record number), e.g. #new_duplicates <- synthesisr::override_duplicates(duplicates_string, 34) dat3 <- extract_unique_references(dat2, duplicates_string) #extract unique references (i.e. remove fuzzy duplicates) dim(dat3) #new number of unique records 9. Prepare the data for exporting from R. Modify the data frame into a format that can be imported to Rayyan (the files saved as .bib or .ris for .csv files cannot be directly uploaded to Rayyan due to some formatting changes happening during processing them in R). This is done by first selecting only the key columns, saving them into a BibTex format (.bib file) and them changing the record labels into the desired format. dat3 %>% select(key, title, authors, journal, issn, volume, issue, pages, day, month, year, publisher, pmc_id, pubmed_id, url, abstract, language) -> dat4 #select the key columns write_refs(dat4, format = "bib", file = "FILENAME_deduplicated.bib") #save into a bib file readLines("FILENAME_deduplicated.bib") %>% stringr::str_replace( pattern = "@ARTICLE", replace = "@article") %>% writeLines(con = " FILENAME_deduplicated.bib") #fix the record labels and save again as a .bib file 10. Import deduplicated records into Rayyan. Create a new project in Rayyan and import the modified .bib file. Run the algorithm for detecting duplicates in Rayyan (see Point 3 above). This will reveal potential duplicates that were below the similarity threshold used in R (or have lots of formatting differences). These will need to be resolved manually in Rayyan (usually it is not a big number and some will require human intelligence to tell what counts as a real “duplicate”). After resolving these duplicates you are ready to start screening your deduplicated records in Rayyan. Note: Unfortunately, record fields with authors and keyword information (and many other fields) are stripped from the original records in the above workflow, mostly by Rayyan. For this reason, records exported from Rayyan are usually not suitable for direct use in bibliometric analyses. But, at least, you can claim that your screening of bibliographic records in Rayyan was blinded to the authors’ identity. by Hamza Anwer One would have heard the following quote at some stage in their life: “The journey of a thousand miles begins with the first step”. However, you don’t hear too much relating to how that journey went or how it ended. Chapter 1 (2018) - Rollercoaster When I first started my PhD, I was armed with an optimism that one feels when they hear that quote, a sense of dipping into the unknown willingly. I wasn’t quite sure what to expect. My first year could only be described as a rollercoaster. It felt like it flew and there were plenty of moments where I thought I would not make it out alive. The learning curve was a lot steeper than I thought and I had to master it, fast, otherwise I would be left behind. I look back now, and I can’t understate how valuable that year was. I learnt a lot about myself and was able to sharpen my skillset as well as my mentality and approach. I learnt to drop bad habits and develop new ones. I took criticism on board and told myself to see every opportunity, good and bad, as a learning opportunity. It set the stage for a very exciting Chapter 2. Chapter 2 (2019) - Foundation This was the year everything really started to take shape. I had a good sense of direction and the work I achieved in this year really set the ultimate foundation. While I had several other tasks in the background, my major milestone was completing my zebrafish work. It was long. It was gruelling. Nailing down the tiniest details was crucial to ensure everything didn’t fall apart. There was no room for significant error although I came horrendously close several times. Whilst the process was by no means smooth, the times in the lab during this year were some of my favourites. I had a great support group and grew in confidence each day. Thankfully, I was able to complete my experiments before we entered the era of Covid-19. Out of the frying pan and into the fire I went. Chapter 3 (2020) – Knuckling down My time at Garvan Institute of Medical Research was over. Covid-19 was sweeping across the world and soon enough, we had entered the lockdown phase. Working from home and online meetings became the norm. I embraced every moment. I looked at it as an opportunity for growth. I had to adapt and focus on moving forward. I embraced a good rhythm while working from home but there was still plenty of work that needed to be done, even when I got back to campus and was grinding away at my desk. I had completed the foundations, but it was all about putting it together in coherent pieces of writing to be published. Slowly but surely, it was all coming together but I was 3/4 into my PhD and still had not published a single paper or completed a single chapter. It was worrying but I pressed on. I still had time. I just needed to use my time wisely. I had everything I needed. I just needed to finish strong. Chapter 4 (2021) – Finish line “There are better starters than me, but I'm a strong finisher.” -Usain Bolt
I think this quote by Usain Bolt really sums up my PhD. If I was running a marathon, my start would be akin to one stumbling and landing face first on the ground. However, knowing I can make up for lost ground if I push that extra bit harder. Which is what I did. I started the year strong with my first publication (can read about that here). The first publication formed the first chapter of my thesis. Slowly but steadily, I worked on putting more components together. The next achievement was publishing my meta-analysis which would form the final chapter of my thesis as well as my general introduction. Two chapters down, two to go. The final two chapters were the most difficult because they comprised the bulk of my zebrafish data. In addition, they complimented one another so I had to ensure a lot of things were consistent. The final few months were intense, but after plenty of deliberation, I managed to piece all the components together. I could finally see the finish line. It looked glorious. Fast forward to 2022, my thesis has been successfully submitted. The final two chapters have been sent in for publication. I am working full-time at Cure Brain Cancer Foundation and am better placed than ever. To finish, I can’t emphasize enough how important my lab was in this journey. From my supervisors to lab members, everyone played a crucial part in where I am now. I thank you all. “The way a team plays as a whole determines its success. You may have the greatest bunch of individual stars in the world, but if they don't play together, the club won't be worth a dime” – Babe Ruth |
AuthorPosts are written by our group members and guests. Archives
August 2023
Categories |