Systematic reviews can be beautiful when seen through the eyes of a comic strip artist

I recently illustrated key parts of a Cochrane systematic review on selective versus routine episiotomies. See the illustration in full on my website. Jani Ruotsalainen, who was unfortunate enough to see early versions of it (and even commented them to everyone’s great benefit), told me that others could be interested by further details on how and why I made this illustration. Hopefully by following his suggestion people who actually know something about illustration and research can swiftly bury my work and replace it with something good.

selective-episiotomy-focus-on-methods-sample

A snippet of the illustration explaining the systematic review process.

First, I’d like to begin with what I believe is most important about this illustration:

I HAVE NO IDEA IF IT IS MORE CONFUSING THAT HELPFUL.

Yes you read that right and maybe that’s not the best thing I should say about my work but I don’t intend on only saying positive things about it.

Indeed, every single panel of my illustration can be interpreted in many different ways by different readers (or by the same reader on different occasions, me included)*. Not all panels are as straightforward as I want them to be and there are a number of things that could be improved or done differently (such as involving end-users in the creation process, offering the review author’s perspective, regular updates or actually telling a story of how the systematic review emerged and how it fits with current knowledge). While uncertainties are entirely normal when testing non-standard approaches to research dissemination, I think it should be emphasized with this illustration. Epidemiology is a serious domain (even if I keep making jokes about it) and associated misunderstandings can have large consequences.

sample university hospitals.png

Some readers may think all 5 studies took place in the same hospital or that studies took place in single hospitals. Others may think they were only written in university hospitals. Many may see something else than a hospital (a factory perhaps?).

Now…Why did I go ahead and spend two days drawing non-stop (a pretty bad idea, I am aware, but motivation can come and go very fast) and over 35+ hours** working on an illustration of a systematic review?

Well, there are plenty of reasons.

The first one is that I did it so that I could have more friends who can fake understanding what I talk about***. The main reason I illustrated a systematic review is that I care about people being able to live happy and fulfilling lives and this has led me to health research, epidemiology, Cochrane and systematic reviews. Many healthcare guidelines and decisions are now based on systematic reviews and I find it critical that people understand some of their key elements. While I had already seen some graphical representations of systematic reviews. I felt they were still too abstract. I wanted to make it slightly more real and wanted to point out that illustrating a systematic review was entirely feasible. Finally, it was a personal challenge (am I crazy enough to do it?) and a way to (hopefully) get better at illustrating scientific studies (then maybe one day we can end up with more user-friendly epidemiology textbooks?).

As a bit of context and disclaimer, I also draw French comic strips (sometimes translated to English) on health research, all of which can be read online for free. While I try to keep things as simple as possible (or so I fool myself to believe) for anyone interested, the comics are primarily aimed at health professionals, with an educational intent and so was this illustration. In my opinion it is an illustration of parts of a systematic review and I would not call it something else.

So how did I do it?

I read a systematic review, summarized it, made some drawings, published it and gracefully accepted being praised for my heroic work.

I followed these specific steps:

  1. Search for and identify a key study (ideally a systematic review)
  2. Read, appraise the study and summarize its key points
  3. Send written summary to study authors for feedback
    • Update/modify if/as needed
  4. Illustrate the written summary with drawings
  5. Send illustration to study authors for feedback
    • Update/modify if/as needed
  6. Publish
  7. Review & update the illustration (in a perfect world)

Let us now examine each of these steps in a little more detail. Please feel free to emulate my recommendations/suggestions however you see fit. This process shouldn’t be set in stone and unchangeable.

1. Search for and identify a key study

The first step involves finding a study to illustrate. There are many ways to do this but I usually find ideas of systematic reviews to illustrate through a specialized alert system or by browsing the titles of recently published Cochrane reviews in the Cochrane Library. The selection process is highly subjective; as I try to choose the systematic reviews that I believe are sufficiently simple to require limited background explanations and that seem relevant to a broad audience (a decision which unfortunately means rare diseases are unlikely to be represented).

cochrane current issue.png

The current (February 2018) issue of Cochrane reviews. Found [here].

Consequently systematic reviews that are not Open Access (OA) are excluded. This is partly for legal reasons (copyright) but also because I want those reading my illustration to be able to check what my work is based on and because I strongly value OA science. Besides, I hope to encourage researchers to publish their work OA by mainly illustrating their studies. I know this makes no difference but I choose to believe a different narrative.

2. Read the study and summarize its key points

I then read the systematic review I found to check if my initial assessment was appropriate and I assess how rigorously it was done by using the AMSTAR II checklist.  This tells me if the authors followed good practices or not and to what extent. Systematic reviews with too low AMSTAR II scores or that lack a key AMSTAR II element are excluded (I said the selection was highly subjective). Again, I don’t want to illustrate studies that have been poorly done and that could be unreliable (unless the whole point is to show what probably shouldn’t be done) and I hope illustrating good ones encourages good practices (more delusions?).

I then briefly summarize the systematic review with 42 or so sentences with <30 words (I’m not drawing a book after all). This summary must, if possible, use the least amount of jargon (although I do include some by design and also by forgetting what is jargon and what is not) and either offer a visual or a verbal explanation otherwise. There must be some details about the context of the review, the methods used, when the searches were done, the results (I often select some of them or the primary outcome for brevity), conflicts of interest & financial sources and a link to the study.  Depending on my intentions, the summary may be more focused on some aspects of the study than others. In the selective episiotomy illustration my focus was on the methods and PICO framework, not the outcomes.

selective-episiotomy-written-summary-sample

An early draft of the selective episiotomy systematic review. Note the subtle changes in wording with the final illustration and the removal of some contents.

3. Send written summary to study authors for feedback

I then send an email to the corresponding author(s) of the review with the written summary included, along with some explanation of what I’m doing. If I get no answer whatsoever in the following month (as is often the case) I usually drop the project for fear of making momentous mistakes. If I get comments I try to update or modify my written summary accordingly (although I’m open to disagreeing). This can go on for some time (months or more) depending on the availability of authors, how frequently they answer/read their emails, how specific the comments are (and how easy the modifications) and how fast I make changes. I don’t know how much time the review authors spend looking at my illustrations. Although contact with the authors is an important step I follow to ensure fidelity of the illustration with the study, I am aware they may end up biasing it.

4. Illustrate the written summary with drawings

Here is a 5-minute video which may be more explicit than any explanation I may offer:

[Brief description of the CookieScience comic strip creation process (January 2018)]

Briefly put: I launch Photoshop CS3, open a premade template, add rows of squares to have similar spacing between sentences and drawings (the original intention was to make it easier if one day my drawings were to be printed, but I think those spaces are actually useless). I then type the sentences (either one or two per “row” depending on their length and what space I estimate they will require) and I directly draw one of the visual representation alternatives that comes to my mind when I read and think about the sentence. On rare occasions, I may look for some pictures on Google (such as on mediolateral episiotomies) for reference so that I stay accurate. I’m using a cheap Wacom Intuos Comic pen to simulate using a pencil (which is highly unreliable and requires me to restart my laptop 50% of the time to make it work…).

Photo1107.jpg

I may occasionnally draw outside. Unpractical, but I could hardly find something better than combining comics, health research and nature.

I then add some color where I find that it makes sense (again, a subjective decision) to make some elements stand out more (such as countries or studies with a randomized design) or to distinguish them (episiotomies in grey, selective episiotomies in white). The only rule is that colors must be kept simple (besides, doing this reasonably well would take me ages).

selective-episiotomy-with-and-without-color-sample

Some may find it easier to understand the process by manipulating the raw files, which can be found here.

5. & 6. Send illustration to study authors for feedback and publish

Finally, I send an email to the corresponding author(s) of the review with the illustration attached for feedback. As before, I then make changes where needed. Once the authors are satisfied with my changes I tell them when I will publish the illustration on my website and give them its URL link. All that is left once published is to send a tweet from my Twitter account.

I rarely promote my illustrations any further (I hate advertising as I have a hard time saying things I draw are any good and consider that if my drawings aren’t shared they weren’t that great to begin with****).

Some thoughts for the future

If I had to think about what to do next with this illustration, I would start by asking a sample of health professionals and non-health professionals what they think of it (while keeping in mind that some may find it difficult to understand combining illustrations/comics and scientific research) and collect suggestions for improvements/changes (especially those lovely criticisms!). I would then create three new versions (or actually I would ask actual artists to make them), make them into leaflets and compare them with a different leaflet (as similar as possible but with less or no drawings) in a trial. Important outcomes would also have to be defined together with end-users and stakeholders. Should these consist of: comprehension, dissemination of leaflets, contents remembered six months later, percent of the leaflet read, satisfaction, health outcomes, understanding of risks/benefits, or some combination of the above?

A more detailed documentation of the creation process (such as how the systematic review was selected and appraised, why some outcomes were shown and others not, etc.) may at some point end up being required for increased transparency and to help distinguish illustrations done following a rigorous process from those that may arise from less rigorous authors but this is likely to increase production costs and the time required to create the illustrations. I am also concerned this could stifle creativity and make it very difficult to illustrate studies. But this concern is likely premature because standards for scientific illustrations and tools to appraise them just aren’t available yet.

Assuming early results show positive and sufficiently important results (a threshold which may be up for debate) I would then love to see international collaborations of artists, non-health-professionals and researchers working on illustrations of research studies and their findings. Or competitions where the goal is to produce an illustration or a single drawing (with a CC-BY license!) which aims to explain a scientific concept/result/study that as many people as possible can understand and remember. I would also be curious to see speed-painting methods applied to illustrations of research studies and further derivatives such as: interactive illustrations, illustrations where the contents change depending on the users, living comic strips explaining the current state of research on a specific topic, etc…

But there is no need to rush; I am not keen to see illustrations of systematic reviews end up on a list of practices widely adopted yet later shown ineffective or harmful due to excessive early hype.

Acknowledgments

I would like to specifically thank the authors and Jani for their comments on my illustration as it is fairly uncommon for authors to answer my requests for feedback. I realize it may take some (valuable) time to check for inaccuracies or offer feedback and there are plenty of other more important things to do. Thanks a lot! Your help was well appreciated.

About the author

Martin Vuillème received very limited formal training in epidemiology and didn’t believe much of what his teachers taught him anyway. He received no funding for this illustration or previous ones. He hopes commercial, academic and personal interests never end up plaguing scientific illustrations and actually doesn’t like cookies that much. He is aware of novelty effects.

*Some could say words alone are also equally subject to interpretations.

** Many of which could be avoided with automation, extra tea and templates.

***Actually there may be more than an ounce of truth in this statement.

****If I get too attached to my work I could risk ignoring its critics and failures.

 

Comments from the editor (Jani)

Wow. Double wow. What the hell, let’s up the stakes straight away with triple wow with extra whipped cream and sugar on top! I mean not only does this guy understand our Cochrane cryptic secret language of systematic reviews but he can: 1) translate the essence of our complicated scientific product into something that doesn’t lull the reader to sleep within minutes, 2) draw elegantly, 3) explain in writing what he does for the benefit of others, and 4) sprinkle all of the above with a hefty wallop of humour! I bet he’s also good looking and plays Mozart on the grand piano from memory whilst blindfolded. I hate the bastard already. But seriously, this is just what the doctor ordered we need in Cochrane. If only we could foster more talents like his by organizing a practical workshop in which he could share his particular brand of magic live to inspire others. No wait. We can! Let’s invite the guy to the Cochrane Colloquium in Edinburgh in September 2018 and organize a workshop!

Advertisements