Deep Impact [Factor] - PowerPoint Presentation

Slide1

Deep Impact [Factor]Altmetrics from a PhD perspective

Jon TennantPhD student, Imperial College LondonSeeking employmentSlide2

Who am I..?Palaeontologist by day, none of your business by night

Interested in open science and science communicationNot a huge fan of legacy publishershttp://orcid.org/0000-0001-7794-0218@

Protohedgehoghttp://blogs.egu.eu/network/palaeoblog/Slide3
Slide4

What are Altmetrics?

A compliment to traditional metrics (e.g., citation indices)Enhance how we understand the re-use of researchHighly variable – a ‘basket’ of metrics

Four different categories:Scholarly activity (e.g Mendeley)

Social activity representing brief engagementsScholarly commentary (expert blogs, Wikipedia)Mass media coverage

Savage (2015) Scientists in the

Twitterverse

, Cell, 162(2), 233-234

Most publishers now employ some sort of

altmetric

measuring systemSlide5

The scope of

Altmetrics

Development of a ‘persona’ for research and researchersMeasures aspects of social influenceRecognises need for public dissemination of researchThe role of social media in communicating scienceMost effective platforms for distributing research

http://errantscience.com/blog/2015/07/08/how-to-measure-a-scientist/Slide6

Online science communication now kind of a big deal

Each aspect can be tracked and measured

Bik and Goldstein (2013) An introduction to social media for scientists, PLOS Biology, 11(4), e1001535Slide7

Altmetrics as a part of Open ScienceSlide8

Altmetrics as an essential tool for ECRs

Blogging, tweeting etc. all now part of mainstream scienceHow do you measure the success of these?Part of the ‘open science’ toolkit for researchersGoes beyond traditional measures of ‘impact’An integral part of 21

st Century scienceSlide9

Impact factor disease

http://occamstypewriter.org/scurry/2012/08/13/sick-of-impact-factors/

http://journal.frontiersin.org/article/10.3389/fnhum.2013.00291/abstractSlide10

HEFCE metrics reportSlide11

The rising metrics tideSlide12

The HEFCE metrics reportEveryone is sceptical about the use of metrics in assessment

Metrics can’t replace peer-review as the primary mode of assessmentAssessment requires a combination of qualitative and carefully selected quantitative indicatorsMetrics can be used to compliment narrative case studiesThe responsible use of metrics (which academics are great at..)

RobustnessHumilityTransparencyDiversityReflexivitySlide13

What does the science say?

arXiv articles show positive correlation between rapid citations and downloads and Twitter mentions (Shuai et al., 2012, PLOS ONE)Highly tweeted articles 11x more likely to be cited

(Eysenbach, 2011, JMIR)Social media therefore either increases citations, or reflects qualities that predict citations, such as societal impactTwimpact factor! A ‘real-time’ impact metric?

Different ‘flavours’ of ‘impact’ that capture different aspects of tools and audiences (Priem et al., 2012, arXiv)Positive but weak correlations between altmetrics and citations (

Costas et al., 2014

arXiv

) (

early view

)

They capture different elements of the research dissemination processSlide14

Scopus article-level metrics

Citation count and percentile benchmark

Field-Weighted Citation Impact

Mendeley readership count and benchmark

Count of scholarly commentary (e.g., blog posts, Wikipedia)

Count and benchmark of social activity (e.g., Twitter, Facebook)

Total count of additional metrics and link to see breakdown by sourceSlide15

Do Altmetrics measure ‘quality’?

What is ‘high quality’ research?What about journal and publisher brands, citation metrics, impact factors (ugh..), peer-review procedures..?

How do you measure inspiration? Education? Creativity? Innovation?How do Altmetrics fit into this?Is ‘quality’ even measurable..?http://trianglesci.org/2015/05/15/the-qualities-of-quality/The hyper-dimensionality of research

Academic impact (e.g., originality rigour)vs societal impact (i.e., reach)Slide16

Afterthoughts/coffee discussion points

Altmetrics don’t solve our problem about assessing ‘impact’ of researchBut they do help us to think more about its societal reachGo beyond what traditional metrics show about research re-useHave to use metrics responsibly

The reality of academiaNational Information Standards Initiative – altmetrics for software and data? (details)What can ECRs do?Sign DORA! (12,300 individuals, 570 organisations)

Discourage use of impact factorsEncourage use of altmetrics and social mediaInfluence performance metrics at an institutional levelSlide17

Thanks for listening!