I was once tasked with designing a qualitative study that asked administrators, nurses, and doctors about their accountability. Not an easy thing to do, as this subject is very complex and each group would see accountability very differently. In the end, having the interview participant draw out a diagram or edit a prepared one greatly helped (more here and here). Diagrams work well as a data collection method because they sit between the open-endedness of a visual and the linear flow of text (more here and here) and borrow the benefits (and disadvantages) of both sides. Cochrane Blogshots, which consist of a few lines of text summing up a Cochrane Review’s Plain Language Summary and an image, seemed to me to also sit on a spectrum between text on one side and the visual on the other.
When looking at the impact of blogshots on Twitter, it seemed reasonable to compare them to the two opposite sides of the spectrum, i.e. just text and just an image. Looking at engagement on Twitter, some say that tweets with images get up to 150% more retweets and 89% more favorites and infographics being 40 times more likely to share on social networks. So where would blogshots fit into this? Would blogshots borrow the benefits of text and pictures and be ‘better’ than both? Or would they borrow both the benefits and disadvantages of both and be in the middle?
Tasked with a quarter to look into blogshots deeper, we set up a pilot study to test how well blogshots performed. The long term plan was to hand off a tried and tested concept to other Cochrane groups to scale-up. We had five data sets each with a tweet with just text, a tweet containing a blogshot, and a tweet containing a picture. We controlled for a variety of things:
- Order of presentation: Perhaps the audience responds more favourable to the first tweet they see. Hence we changed the order of the three tweets for each dataset.
- Time of day: All tweets were be sent at the same time: 9pm AEST, which is 12noon BST and 7am EDT.
- Day of the week: Traffic varies depending on the day of the week. All tweets from the same data set were sent on the same day of the week.
- Content of tweet: standardized wording and standard #cochraneevidence hashtag were used. We did not use any topic-based hashtags or tag any Twitter account.
- Link to Review: We used a new and different link for each of the 15 tweets so that we could track who is clicking through to the review.
- Accompanied material: Each of the five data sets were tweeted without an image, with a picture, and with a blogshot. The picture used was the same one used in the blogshot.
- Topic: Interest in a tweet/picture/blogshot may be mostly driven if someone is interested in the subject matter. Working with Cochrane UK, we selected general interest reviews from their Evidence for Everyday Health Choices blogshot series to maximize general appeal.
- Blogshot quality: Cochrane UK has taken the lead in blogshot production. We used only blogshots created by Cochrane UK to ensure an equal quality standard.
- Only English: Blogshots are translated into many languages. Given the scale of this study we focused on English blogshots.
And so out went 15 tweets on the main Cochrane twitter account: five with just text, five with text and an image, and five with text and a blogshot.
Here are the results (highest scores highlighted):
C: clicks, L: likes, RT: retweets
Our small-scale study suggests that sharing a blogshot on average almost doubles the click-throughs, likes, and retweets that you get over a text-only tweet. However, sharing a related picture with text in a tweet on average triples the click-throughs and likes, and doubles retweets.
We hope that the Cochrane Community views this as a ‘proof of concept’ or ‘pilot stage’ study on which to base larger studies. This may be done by several people or groups working together and combining evidence, or several smaller studies done separately. Ideally it would also be good to have a sample of non-English tweets and perhaps a sample from a specialized audience, like a particular Review Group or Field.
It should also be noted that these measures really only look at engagement i.e. that people took notice of the tweets and then performed an action which makes them visible to us. It is hard to know if people understood the information within them or if they did something with that information offline. Unpacking the difference between engagement versus uptake and how it relates to impact is definitely a whole blog post in itself!
Overall I think the lesson is that presenting Cochrane visually is important. Every Cochrane-related post on social media should have an image accompanying it. This can be a photo or a blogshot, but the important thing is to include something other than just text. This should be standard practice across all Cochrane accounts.
CEAD looks forward to hearing from you on your thoughts about this pilot study and other plans to share Cochrane evidence visually.
Cochrane Internal Communications and Content Officer