Tweet, tweet + blogshot, or tweet + picture: results from a Twitter pilot study

I was once tasked with designing a qualitative study that asked administrators, nurses, and doctors about their accountability. Not an easy thing to do, as this subject is very complex and each group would see accountability very differently. In the end, having the interview participant draw out a diagram or edit a prepared one greatly helped (more here and here). Diagrams work well as a data collection method because they sit between the open-endedness of a visual and the linear flow of text (more here and here) and borrow the benefits (and disadvantages) of both sides. Cochrane Blogshots, which consist of a few lines of text summing up a Cochrane Review’s Plain Language Summary and an image, seemed to me to also sit on a spectrum between text on one side and the visual on the other.

When looking at the impact of blogshots on Twitter, it seemed reasonable to compare them to the two opposite sides of the spectrum, i.e. just text and just an image. Looking at engagement on Twitter, some say that tweets with images get up to 150% more retweets and 89% more favorites and infographics being 40 times more likely to share on social networks. So where would blogshots fit into this? Would blogshots borrow the benefits of text and pictures and be ‘better’ than both? Or would they borrow both the benefits and disadvantages of both and be in the middle?

Tasked with a quarter to look into blogshots deeper, we set up a pilot study to test how well blogshots performed. The long term plan was to hand off a tried and tested concept to other Cochrane groups to scale-up.  We had five data sets each with a tweet with just text, a tweet containing a blogshot, and a tweet containing a picture. We controlled for a variety of things:

  • Order of presentation: Perhaps the audience responds more favourable to the first tweet they see. Hence we changed the order of the three tweets for each dataset.
  • Time of day: All tweets were be sent at the same time: 9pm AEST, which is 12noon BST and 7am EDT.
  • Day of the week: Traffic varies depending on the day of the week. All tweets from the same data set were sent on the same day of the week.
  • Content of tweet: standardized wording and standard #cochraneevidence hashtag were used. We did not use any topic-based hashtags or tag any Twitter account.
  • Link to Review: We used a new and different link for each of the 15 tweets so that we could track who is clicking through to the review.
  • Accompanied material: Each of the five data sets were tweeted without an image, with a picture, and with a blogshot. The picture used was the same one used in the blogshot.
  • Topic: Interest in a tweet/picture/blogshot may be mostly driven if someone is interested in the subject matter. Working with Cochrane UK, we selected general interest reviews from their Evidence for Everyday Health Choices blogshot series to maximize general appeal.
  • Blogshot quality: Cochrane UK has taken the lead in blogshot production. We used only blogshots created by Cochrane UK to ensure an equal quality standard.
  • Only English: Blogshots are translated into many languages. Given the scale of this study we focused on English blogshots.

And so out went 15 tweets on the main Cochrane twitter account: five with just text, five with text and an image, and five with text and a blogshot.

blogshot

Here are the results (highest scores highlighted):

blogshot-study-results-clearer

C: clicks, L: likes, RT: retweets

Our small-scale study suggests that sharing a blogshot on average almost doubles the click-throughs, likes, and retweets that you get over a text-only tweet. However, sharing a related picture with text in a tweet on average triples the click-throughs and likes, and doubles retweets.

We hope that the Cochrane Community views this as a ‘proof of concept’ or ‘pilot stage’ study on which to base larger studies. This may be done by several people or groups working together and combining evidence, or several smaller studies done separately. Ideally it would also be good to have a sample of non-English tweets and perhaps a sample from a specialized audience, like a particular Review Group or Field.

It should also be noted that these measures really only look at engagement i.e. that people took notice of the tweets and then performed an action which makes them visible to us. It is hard to know if people understood the information within them or if they did something with that information offline. Unpacking the difference between engagement versus uptake and how it relates to impact is definitely a whole blog post in itself!

Overall I think the lesson is that presenting Cochrane visually is important. Every Cochrane-related post on social media should have an image accompanying it. This can be a photo or a blogshot, but the important thing is to include something other than just text. This should be standard practice across all Cochrane accounts.

CEAD looks forward to hearing from you on your thoughts about this pilot study and other plans to share Cochrane evidence visually.

Muriah Umoquit
Cochrane Internal Communications and Content Officer

Advertisements

4 thoughts on “Tweet, tweet + blogshot, or tweet + picture: results from a Twitter pilot study

  1. I’ve been interested in those blogshots for quite some time… Couldn’t those results also be somewhat related to a novelty-effect? And then, should all tweets have a picture, would that benefit potentialy get lost?

    Like

    • Good point Martin. It is true that a lot of the fuss about social media is still the excitement about the shiny new toy. Most often it is quite hard to evaluate return-on-investment especially when disseminating science like we do. What we can do is 1) follow the example of this study by Muriah and friends to see if we see any effects and/or 2) support/lobby for a larger scale study. I can personally say that since publishing this post I have added photos to nearly all my tweets but sadly there hasn’t been an overwhelming rush to retweet my superlative insights (*sigh*).

      Like

  2. Indeed! And even when dissemination is successful the message might get distorted or you might get more viewers but the same amount of people actually reading. This reminds me very much of criticisms of altmetrics. Hard to tell what they’re really measuring, if it’s important and what to do with it.

    A hypothesis would be that Cochrane tweet readers see tweets with pictures as surrogate markers for higher quality content, more important content or content that is made for a broader audience. And your current followers might not care that much about pictures. Qualitative study might help clarify that (at least from biased self-reports, if such work hasn’t yet been done somewhere).

    I wonder how much this is linked to Twitter’s algorithm favoring some type of tweet over others.

    Like

    • Good points!
      I thought the novelty and clearness of the Blogshots was going to make them the clear winner for likes and retweets and probably not for click-throughs as a high-level overview of the content is given. However, we didn’t see that. I don’t think an algorithm could differentiate between a jpeg of a picture or a jpeg of a blogshot (perhaps it does favor these over just text tweets though) so there is definitely something going on. A qualitative study about how people interact with health evidence on social media and what they get out of it would be fabulous. We have one PhD student from Cochrane Oral Health who may be looking at this further.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s