by Losia Lagisz

Managing research projects is not easy. It gets even more complicated if there is more than one person involved. Not only many project participants need to know what needs to be done and by whom, but also what others just finished and what they are currently working on. This applies especially to meta-analyses.

Robust meta-analytic process requires at least 2 people involved in screening and extracting data. The other steps of the process (planning, searching, analysing and writing up) also greatly benefit from having more than one person involved. Large teams are not uncommon and the whole process can drag for months and even years. Both large and small teams need efficient communication and job management tools to get a good quality output in reasonable time.

There a few communication and job management software tools that worked for us, so far. For general, lab-level, across the projects and teams, task management we use a Todoist (basically an online “to-do list”), and for communication we use Slack. Last year, we started using Workstreams add-on for Slack, which appeared to be a good tool for managing systematic review and meta-analytic projects.

Workstreams have at least three big advantages:

1) Workstreams are integrated with Slack – so, no new sign-ups, setting and all the hassle of new system. Slack channels can be used to display alerts on what is happening with the tasks and you can also use shortcuts directly from Slack to crate tasks and change their status.

2) Workstreams are visual – the system it is based on a concept of a Kanban board (see the screenshot of one of our taskboards, below). Rather than just displaying the list of tasks that need to be done, it has three columns (you can add more and customise). The default columns are: planned, in progress and finished – you can drag the tasks between the columns, as needed. The tasks can also be assigned to people, labelled, priority reordered, commented on, have attached files, etc. So, Workstreams have quite a lot of functionality, but they are also easy and intuitive to use.

3) Workstreams are free - at least the basic version. The PRO version is quite pricey (user/month $9.99) and it comes with a few more functions, of which only one is really useful – setting deadlines and reminders (however, we can go around this problem by using /remind bot function from Slack).

You can learn more about Workstreams here. From my experience, if used consistently by the team members, Workstreams (with Slack) is a good tool for managing research projects such as systematic reviews and meta-analyses (and other projects), especially if you are not a big fan of traditional (even digital) to-do lists.

Last week (14-15 Nov), I went to Melbourne for a workshop (“From Replication Crisis to Credibility Revolution”). The workshop was hosted by my collaborator and “credibility revolutionary” Fiona Fidler.

I suspect many workshops and mini-conferences of this nature are popping out all over the world as many researchers are very much aware of “reproducibility crisis”. But what was unique about this one is its interdisciplinary nature; we had philosophers, psychologists, computer scientists, lawyers, pharmacologists, oncologists, statisticians, ecologists and evolutionary biologists (like myself).

I really like the idea of calling “reproducibility crisis” “credibility revolution” (hence the title). A speaker at the workshop, Glenn Begley, wants to call it “innovation opportunity” (he wrote this seminal comment for Nature). What a cool idea! And these re-namings make things a lot more positive than a bit of doom-and-gloom feel of “replicability crisis”. Indeed, there are a lot of positive movements toward Open Science and Transparency, happening to remedy the current ‘questionable’ practice.

Although I live in Sydney, I was also in Melbourne early last month (4-5 Oct) for a small conference. This is because Tom Stanly invited me over, as an evolutionary biologist, to give a talk on meta-analysis to a bunch of economists who love meta-analyses. To my surprise, I had a lot of fun chatting with meta-enthusiastic economists.

Tom is not only an economist but also a credibility revolutionary, like Fiona. He has asked me to invite ecologists and evolutionary biologists to comment on his blog about a credibility revolution. It is an excellent read. And if you can make comments to join the conversation, Tom will appreciate it a lot, and get conversations going. Disciplines need to unite together to make this revolution successful or make the most of this innovation opportunity. So, join the credibility revolution!

by Rose O'Dea

Measuring fish. In my short research career this is the activity I’ve done the most. ​Whether measuring mosquitofish or zebrafish, the method has been:

1. Photograph fish next to a scale
2. Rename photo with the ID of the fish and the date
3. Measure photo using ImageJ

Here I share my tricks to renaming photos using R.
​

Step 3: Generate a file with the name of each fishTo link the marking of the fish to their records, we save a file that contains three pieces of information: the tank ID, the fish ID, and the marking of each fish. The file is exported from our master records, and requires a bit of manipulation so that we only have the records of the marked fish in our particular tanks. In this example we are only renaming photos from the tank called “DTG03”.

# note that working directory is already set to our project folder (this is automatic if you use an R Project)
# importing excel document
library(readxl)
marking <- read_excel("Fish Records.xlsx", sheet = 1)
# renaming columns to remove "::" (which filemaker uses this to distinguish related table fields)
names(marking) # before

## [1] "fish_group_ID"
## [2] "individual_fish main 3::fish_indiv_ID"
## [3] "individual_fish main 3::marking"

names(marking) <- gsub(".*:", "", names(marking)) names(marking) # after
## [1] "fish_group_ID" "fish_indiv_ID" "marking"

# finding fish in the tank "DTG03", which we are renaming
# the problem is Filemaker doesn't export the repeating records for fish_group_ID, so we need to
# take the first instance of DTG03, and every subsequent NA until the next tank record
marking <- marking[which((marking$fish_group_ID == "DTG03") == T):nrow(marking),] # first record of DTG03
marking <- marking[1:(which(!is.na(marking$fish_group_ID)==T)[2]-1),] # removing everything not in DTG03

Step 4: Import the old file names, and create a corresponding vector of new file names.

old <- list.files("./original_DTG03_15_03_17") head(old, 5)

## character(0)

# Taking the markings of these fish so that they can be matched to their unique IDs
photomark <- gsub(".JPG", "", old) head(photomark, 5)

## character(0)

# Using indexing to check we're using `match` correctly
marking$marking[match(photomark, marking$marking)] == photomark # all match

## logical(0)

# saving JPEG names of new photos
new <- paste0("15_03_17_", marking$fish_indiv_ID[match(photomark, marking$marking)], ".JPG")
head(new, 5)

## [1] "15_03_17_.JPG"

Now we should have two vectors of equal length for the old and new names of photos:

length(old) == length(new)

## [1] FALSE

If this isn’t TRUE, then we need to go back and work out why. Usually one of the photos has been mislabeled, but this can also catch bigger mistakes such as missing or misplaced fish.


Step 5: Rename photos.First we will create an empty destination folder where we want the photos to go: in this case it’s called photos_to_measure. We then add the file path to the old and new file names.

oldfile <- paste0("./original_DTG03_15_03_17/", old) head(oldfile, 5)

## [1] "./original_DTG03_15_03_17/"

newfile <- paste0("./photos_to_measure/", new) head(newfile, 5)

## [1] "./photos_to_measure/15_03_17_.JPG"

file.rename(from = oldfile, to = newfile)

​## [1] TRUE

​Note that this function file.rename will transfer the photos from the old folder into the new folder, leaving nothing behind (i.e., something’s gone wrong if the old folder isn’t empty):

​Not just for fish photos!

Whenever you want to rename a bunch of files, file.rename could save you lots of time.


R is found in tRial and eRRoR

file.rename is a base function. Whenever I discover that something I’ve been doing in R for ages can be more easily done using a simple base function, I’m alternately amazed and infuriated. Before I found file.rename I renamed thousands - thousands! - of photos using a looping method that is more complicated and slower.

library(jpeg)
oldfile <- paste0("./original_DTG03_15_03_17 copy/", old)
newfile <- paste0("./photos_to_measure/", new)
for(i in 1:length(newfile)){
    writeJPEG(image = readJPEG(oldfile[i]), target = newfile[i], quality = 1)
}

I don’t regret the time it took me to discover file.rename, for a couple of reasons. The loop allows you to specify the file size of the renamed image by specifying quality on a scale form 0 to 1 (see ?writeJPEG). In our database we sometimes store photos of important fish, and to keep the file size lower it’s useful to export these photos with quality = 0.3. More importantly, R is about the journey as well as the destination. That small piece of code I wrote in 2016 - which took at least an hour of trial and error! - was the first loop I ever wrote and used regularly. My inelegant code taught me a lot, and watching it rename photos was immensely satisfying. Still, we need to pick our R battles, and I hope file.rename will leave you more time for your own.

by Susi Zajitschek
​​
​The fast pace of life that rules our modern world can have profound effects on our wellbeing. Burnout and depression have unfortunately become common in many professions, and mental health problems are also accumulating in academia. While it seems to be a topic that isn’t often talked about – sometimes it seems “we” (scientists) are not supposed to feel unhappy – after all we are following our chosen part, directing our own research questions – and are getting paid for it, too … (I won’t be getting into why science is important now, and ways to defend it to sceptics – that is a topic for a future blog post!)

​Stress can be avoided (at least to some degree). And here I mean – take measures to stay on track, not avoid working! Procrastination will not solve your problem, but merely add guilt and more stress later on.

• Being organised and able to prioritise (and re-prioritize!) can help seeing the big picture while dealing with the small steps that will take you in the right direction. On a weekly basis, setting daily tasks and anti-procrastination deadlines for the not-so preferred tasks (such as writing up that one experiment that just didn’t show clear-cut results) can help. Having a “what do I want to achieve within the next 5 years?” plan that also identifies not only your overall goals, but also the skills that you need to acquire, and will identify misalignments early. That way you may avoid having to wonder why you wasted a large chunk of your life on something you actually don’t really want to do.

 

• Should stress and anxiety become overwhelming after all, try mediation and relaxation techniques. Even a few minutes of just breathing can help calm down and de-clutter your mind, even if in the heat of that moment it may seem like you might be wasting your time. Nonetheless, stress clouds your thinking, and mediation helps to focus your mind. Not taking the time to re-set is like being the wood-cutter who says “I don’t have time to sharpen my axe, there’s too much wood to cut”!

 

• Work-life balance: Brooding over your projects all day and night may seem admirable, but sacrificing social contacts and hobbies for work can negatively impact your mental health. It is important to shut off sometimes, relax, do something you love! Solutions to problems often present themselves after a break. Having hobbies and friends will help with that!

 

• Physical exercise and sufficient sleep are well-known factors contributing to mental health, so make sure to work out from time to time, and don’t sacrifice sleep on a regular basis (sometimes deadlines will require an all-nighter, but don’t let this become a habit – consider ramping up your organisational and time management skills instead!).

 

• And last but not least – attitude… This of course is a life-long task to accomplish, much easier said than done, and deeply philosophical. But a positive outlook and seeing learning opportunities in the most adverse of events make you resilient, and not to let stress direct your life. Here some quotes from famous people on life attitude: 

We can complain because rose bushes have thorns, or rejoice because thorn bushes have roses.
​Abraham Lincoln

Whether you think you can or can’t – you are right. 
Henry Ford

​Life is a shipwreck, but we must not forget to sing in the lifeboats.   
Voltaire

by Rose and Losia
​
Originally published as an online "Behind the Paper" article at https://socialsciences.nature.com/users/177900-rose-o-dea/posts/39046-slow-and-steady-our-meta-analysis-on-greater-male-variability-in-academic-performance

The paper, authored by us (Rose and Losia) and our co-authors Michael and Shinichi, is freely available in Nature Communications: https://bit.ly/2N2ymie.

The seed for this paper was planted five years ago. Michael was sat at Losia and Shinichi’s kitchen table in Dunedin, New Zealand, listening to Shinichi describe a new meta-analysis technique he’d been developing. This technique could test for differences in variability between two groups, while controlling for differences in means. Michael suggested we use this technique to test the greater male variability hypothesis.
It would be easy. Losia found a recently published meta-analysis that tested for gender differences in average school grades; we’d do a re-analysis. It would be a simple project for Michael’s honours student, Rose, to learn some meta-analysis basics; she was joining Shinichi’s new lab in Sydney, Australia, at the beginning of 2015. The paper would be finished (we all thought) within a year, before Rose started a PhD. But it took that long just to get the data, never mind analysing and writing up our results.

The descriptive statistics of studies included in the original meta-analysis were not available. So, we downloaded the papers and re-extracted the necessary data, and contacted the authors of papers that hadn’t reported descriptive statistics. We also updated the original systematic search to find papers published since mid–2011. Losia and Rose each screened over 3,000 abstracts (thanks abstrackr), and by the start of 2016 we finally had our comprehensive dataset.

But by then, this was a side-project. Not part of anyone’s grant or thesis, it was often relegated to the bottom of our priority lists. Having each other as co-authors was essential: the project never fully stagnated because there was someone else to keep it moving. Had we been doing the project alone it would probably lay abandoned.
We were also challenged by our ignorance of the field – we are evolutionary biologists and behavioural ecologists, not social scientists or psychologists. It took time to find, and read, studies that informed our introduction and discussion. Often we were missing jargon that was the key to unlocking this information. Here we are thankful to popular science communicators for bridging these gaps (e.g. the term ‘occupational segregation’, which appears in our introduction, was learnt on an episode of Freakonomics Radio), and to psychologists who commented on earlier drafts.

by Dony Indiarto

Earlier this year, a team of researchers published a study on Indonesia’s Bajau people that made a highlight in evolutionary biology. Bajau people have been widely known as extraordinary divers. The study unveiled that these people have significantly bigger spleens compared to those live in the nearby farming communities. Bajau people spend most of their days freediving and hunting underwater for generations. The researchers also identified 25 distinctive genes that are plausibly associated with the sea gipsies’ unique lifestyle, as reported in Cell. No doubt, this research has revealed crucial evidence on human adaptation under extreme condition. Scientifically, there has been no flaw or concerns about the research and its findings. Nevertheless, the publication has been disconcerting for some Indonesian scientists, who revealed concerns over western scientists’ behaviours.

Helicopter science
The story above reflects a bigger social phenomenon, the so-called ‘helicopter science’, where foreign scientists come and collect data for a short period of time in a particular local area. They then go back home, analyse and write their research in their institution, often disconnected from the local contexts, from the local knowledge, and from those who have been focusing their research in longer term, often for their entire career. Two potential misses: (i) the research results might be biased due to small samples and also lack of local contexts; (ii) without true collaboration with local scientists, if any, long-term research for covering breadth and depth of exciting findings will not be feasible. The study may bring some progress to science, but by leaving behind the local researchers, the knowledge and capacity gaps among the scientific community will be widened, primarily due to lack of funding and resources to support scientific research in developing countries. The problem is magnified specifically for basic natural science that needs advanced technology and expensive lab works, such as in genetic studies.

This issue often happens in biodiversity-rich developing countries, where data are scarce and badly needed, but funding some particular research topics are not yet a priority. These countries are often alleged for enforcing complicated procedures and unclear regulation in facilitating and hosting foreign researchers and in permitting samples to be taken out of the country. While this is factual, it does not tell us the complete story. Cultural and language barrier can also create misunderstanding and unmet expectation between scientists. The front desk officers who implement the procedures are blind to the reason behind the regulation. Local counterparts are often employed as logistic support or leg workers and ended up having narrow space, if any, in participating in doing science. Not unlike Trump’s current policy for balanced trade, developing countries’ protection policy can retaliate back by imposing stricter regulations for research by foreign scientists. This is also similar to meta-analysis scientists who are called as “research parasites” by other scientists who collect primary data. As a result, this tit for tat will slow down new scientific discoveries in under-researched areas.
 
Call for inclusive science
“Helicopter science” is just one form of implicit biases and inequalities that do exist and persist in the scientific community. The wall between countries in doing science should be removed through collaboration driven by ethics.  The scientific partnership should be built on mutual respect and benefit. Of course, this is easier said than done. Scientists in developing countries are hindered by limited financial and institutional support, technical skills, as well as exposure to the advances in scientific discoveries. Practical or applied research is often more relevant for the locals than basic research, and therefore the collaboration in such areas will be truly fruitful.

On the other hand, local scientists should actively engage in reducing “helicopter science” practice by being pro-active. One good example is from African scientists who have taken a step forward by declaring the ethical handling of samples for genomic studies. Like everything else, the sociology of science will be rich and flourished when diversity is cherished.  In reaching so, collaborative research has to bring benefit to all parties.

Now there is no one way to give a good presentation, but below I have compiled some tips and words of wisdom I collected throughout the years on how to make presentations an enjoyable experience for yourself and your audience too.
 
Be prepared. Be really prepared

My first and foremost tip for anyone who is uncomfortable with public speaking is to plan ahead so you can have enough time to prep. I am an anxious person, I literally schedule a two-week reminder for when I have to give a talk, so I have enough time to think about the audience, the story, the format and style.
 
Hello, meet Practice and Feedback

When it comes to really nailing a talk, practice and feedback are the key. Although too much practice can also be a bad thing! You don’t want to come across as too scripted or even bored because you have practiced too much! Testing out your talk helps you getting into the flow of speaking in front of other people, it also helps you to identify parts of the talk where you are stumbling on, etc. Recording yourself is also another good way to do this.
 
Organise a practice session with your lab group, your office buddies and other students that might be presenting at the same event! Seeing other people’s talks can also spark ideas for your own. Practice with people that you feel comfortable to receive (and give) honest, constructive feedback, because that is the only way to improve. I find practicing with people that are not directly in my own field invaluable. I practice with my writing group which consists of a bacteria bioinformatician, a marine biologist, a plant geneticist, an avian evolutionary biologist and a invertebrate ecologist and honestly… this is where I get the best feedback. Explaining concepts to a broader audience helps you communicate your ideas more effectively. If you can explain a highly specialised jargon-y concept to someone not working in your field, then you are winning at science communication!

Slides are your sidekick

Depending on what type of speaker you are, the amount of text on the slide is really up to you. If you need some bullet points as prompts, then do it! But let’s all remember a time where we painfully watched someone read off their slides… again balance is important!
 
I never use animations. There is just too much room for error when transferring your files between different operating systems etc. However, I do use multiple slides to sequentially pop up elements on my slide. Like a flipbook, each slide is almost the same of the previous slide but with some small adjustments. (See here for some examples). I end up with a lot more slides but now I have fail-proof animations! Another trick is to export and transfer your slides as a .pdf so all your customised text and figures don’t move because of incompatibilities between different computers
 
Sequential ‘pop ups’ help you guide your audience’s attention to exactly what you want them to focus on. Do all the work for your audience by walking them through your argument step-by-step. Never assume they know what you mean. I use a lot of boxes to conceal parts of graphs, I cover my data, and first explain the axes, I even put some made-up data to show them how to interpret these when I finally reveal my own graph. Here’s one of my recent presentation as an example
 
Questions time is like a box of chocolate

The presentation, you can practice, you know what will happen. But with question time…it is a mixed bag. One trick, I’ve been told, is to anticipate what kind of questions you might get.
Think about: “If I watched my talk, what would I ask?” Practice sessions can help you identify these as well. My supervisor, Dan, even tries to strategically guide the audience to ask a particular question by providing less detail during the talk(!). Having an expectation of the range of questions you might be asked allows you think of a response in advance and even prepare some additional slides that you can tack on to the end of your presentation to really address the question thoroughly. Genius!
 
I wanna talk like you

One thing I find really helpful is to think back to a memorable presentation you’ve seen and identify what made that presentation so effective. I find myself thinking back to a presentation by an old lecturer A/Prof Greg Holwell. He effortlessly conversed with the audience with stunning photographs on his slides. I think of Dr. Camilla Whittington who had complete control of the whole room while she posed a pressing problem in her field and seamlessly addressed it with her own research. I am reminded of a keynote by Dr. Anne Gaskett, whose personality and casual presenting style made her research even more engaging. Drawing ideas from those that inspire you will help you develop your own style. Channel their confidence and composure!
 
Everyone responds to public speaking differently. Some thrive in the arena, while others just need “fake it before they make it”. I, for one, definitely fall in the latter category. However, over the many conferences I’ve been to I have managed to see presentations as a fun thing to do where you can show off your own work! Remember to enjoy the process and know that you are the expert of your own work. 

Conference visit and talk in Paris  (https://cee2018.sciencesconf.org/)

I landed in Paris on 18th of April (with a whole-Paris public transport strike waiting there to welcome me), so I could only make the latter part of the third day of the conference. Last two days of the conference I attended had the most exciting sessions for me and our research group. Talks were under two major categories: (a) discussions on how can technology contribute to evidence synthesis, and (b) presentations / demonstrations of software tools that assist evidence synthesis. Some key discussion points were: (i) using standard formats and tools to report systematic review process, (ii) visualisation and mapping of bibliographies, (iii) automating abstract / full-text screening using text mining, and (iv) potential future uses of machine learning in evidence synthesis.

My talk for the conference extended the fourth discussion point listed above, and my talk was titled “Deep Learning - Potential Uses in Systematic Reviews”. This was my first conference talk without presenting a paper, so it was a whole new experience for me to present something as a short-lecture format. The talk started by introducing the basics of machine learning and deep learning, then the potential uses of the introduced techniques in different stages of systematic reviews. My talk went very well, I got some overwhelming response (and also some good questions). The best comment I got was simply: “I learned from your talk!”. After a few tech talks, there was a technical discussion panel and I could be a part of that, and I enjoyed it. Machine learning was one of the hot topics during the discussions and introduction of proper database systems to keep track of evidence syntheses was equally popular.

Hackathon in Stockholm (https://evidencesynthesishackathon.com/)

After the conference, my next destination was evidence synthesis hackathon in (magnificent) Stockholm, to which I (and some other participants) carried some ideas from the Paris conference. There were hackathon participants who came from all over the word, some from extremely popular places for technology (look up the list of participants here). Some of them were hackathon enthusiasts (who try to participate in every possible hackathon), tech-gurus from industry (well known companies), computer scientists (from well known universities), and academics who mastered evidence synthesis. A computer science (>=80%) and evidence synthesis (<=20%) hybrid like me could fit in well, fortunately.

At the beginning, we all gathered in for a round-table discussion and the hackathon organisers were presenting the open problems in evidence synthesis that we ‘hackers’ could help to solve, and we all pitched in our ideas / plans as well. The whole set of participants then divided into 4 main streams according to the sub-task within evidence synthesis they were trying to ‘hack’: (i) reporting of evidence synthesis processes, (ii) improving search queries, (iii) data extraction from pdf articles and indexing the coded documents, and (iv) data visualisation using systematic maps.

Four participants, including me, were working on the third stream listed above. Our aim was to implement a software tool that assists efficient content extraction from pdf documents during data coding. The most significant barrier during this task was that the pdf files are embedded in highly abstract and protected manner. The main contribution of the prototype application we prepared was accessing contents in pdf articles selectively. The tool was developed around ReactJS JavaScript framework, therefore, it is suitable for deploying as a web application, in a local virtual web-server in a desktop environment or in a centrally hosted web-server. This application takes in a CSV file with fields to be extracted as headers, and loads pdf files from a server folder. The content coding can be performed using a right-click menu that brings up list of fields, and then saves the fields, selected contents / values from the pdf and any user comments back to the CSV file as a new row per single pdf.

Hackathon concluded with us presenting and demonstrating our implementations, concluding discussion session, and a nice dinner outing with the whole group. I mostly value the opportunity to meet and talk with people of different career and academic backgrounds, and the things I could learn from and share with them.

Acknowledgments
Shinichi for nominating me for both events, introducing me to the key people, and supporting my conference visit and the talk. Neal Haddaway at the Stockholm Environment Institute and Martin Westgate at the Australian National University (https://evidencesynthesishackathon.com/organisers/), for organising the hackathon and the technical discussion sessions at the conference, inviting me for both events and funding my visit to the hackathon