AGENDA
Science 2.0 Integrating and opening the whole research cycle for everyone
Diagnostics for a new accountability of Science From
DORA, ‘bad bibliometrics’ and a few suggestions
A new level: Author profiling
… to
Rankings and the Ranking Web (Webometrics)
Accountability Metrics 2.0 New Stakeholders
Citizen Science Blurring the lines Formal & Informal Communication
INTEGRATING
the whole research cycle and its stakeholders
OPENING
the whole set of data, tools, results and
metrics
Open Data
Big Data = ∑Open Data
SCIENCE 2.0
Data-intensive Big Open Data
‘Globalization’
Sharing everything with everybody
Digital natives scientists
Transparency Open Access
A humble suggestion
∑
Hint: Badly needed Easy access to more powerful computer infrastructures Big Science is no longer the only
provider of Big Data
Evolving from Bibliometrics: Current scenario
SWOT Analysis
STRENGTHS WEAKNESSES
THREATS OPPORTUNITIES
OPEN ACCESS Big Data Social Networks BIBLIOMETRICS
Tested tools and indicators
BAD
BIBLIOMETRICS DORA Declaration
ALTMETRICS Trusteness, Meaning
Interpretation
Requirements for a
quantitative-based assessment
Multi-
source New bibliometric sources, specially from developing countries Non-bibliometric sources too
Multi-
dimension Involving all aspects of the research cycle
Involving all facets of communication and impact Better
attribution Of authors roles and contributions
Of resources, funding and policies involved Excellence-
guided More relative indicators and ranks
Journal-Level METRICS
Infamous IF SJR, SNIP
Article-level METRICS
CPP, MNCS PLoS ALM
University RANKINGS
Leiden, Scimago NTU, URAP
Author PROFILES
GS Citations ResearcherID ResearchGate
Changing the focus
I-Metrics
WEBOMETRICS Number of times the URL of a document/author webpage
is linked from another webpage
ALTMETRICS Number of times the elements are mentioned (shared) in websites, wikis,
blogs, social bookmarks and networks (incl. Twitter)
or search engines
USAGEMETRICS Number of times the
document is
read/visited/downloaded from its publishing place (incl.
websites) BIBLIOMETRICS
Number of times the bibliographic record (identifier) of a paper/book is
cited in another similar formally published paper
Document-level metrics
Title
Source:Journal/Book/Patent Publication Year
Citations
Identifier: URL/DOI/Handle Subject/Tags
Author-level metrics
Author(s) Institution (Affiliation) Publication Year Discipline/Tags Web profiles Web 2.0 profiles
About changing the altmetrics organization model
Problems
An ugly name (webtwometrics?)
Even worst: Confusing tangled set of mixed value tools
A rose is
a rose Bibliometrics to bibliometrics: Mendeley, ResearchGate, F1000, …
Tool-related naming: Twittermetrics, wikimetrics, …
Segregate
Usage is a very rich environment: Visits, visitors, downloads, …
Usagemetrics!
Another humble suggestion
Bibliometrics issues: (Incomplete) Diagnostics
Bibliometrics issues: The (lack of) Context
PAPERS CITATIONS
PATENTS ALT
DISCIPLINE BIASES GEOPOLITICAL
BIASES
Bibliometric Issues: The Sources
Bibliometric Issues: The Coverage
Bibliometric Issues: The Authorship Attribution
Three suggestions for bibliometric community
Two sets
Size dependent indicators, but ..
add size independent indicators too!
Authorship
Kill the full count!
Pure fractional count? Move to ‘one size fits all’: From 100% to 50% (?)
Ranks
Composite indicators?
A controversial suggestion
* Papers signed by 2 or more authors
And perhaps, even more controversial
Adding author-level metrics
PUBLIC AUTHOR PROFILES
CITATION (& MENTION) BASED METRICS
BETTER AUTHOR ATTRIBUTTION
A model?
A preliminary assessment
The good
One shop
Bibliometric, altmetric (usagemetrics) indicators
Relative indicator
The ugly
Composite indicator
No API
A second model?
Another assessment
The good
Public, free
Huge coverage
The ugly Incomplete (voluntary)
Only bibliometric indicators
Free text for disciplines and affiliations No API
Easy to manipulate
Institutional CRIS as another candidate model
CRIS
The good
Public, free
Complete institutional (self-sourcing) coverage
The ugly
No standards for comparison purposes
Basic indicators
For bureaucrats (and librarians)
‘Stick and carrot’
Adding relative indicators
Isidro F. Aguillo 56 432 7.71 2 (h>=38)
Author/Discipline-Country 2.18% 3.78% 105% 5.26%
About Composite Indicators: The Rankings
High School Exercise
Cut & Paste Exercise
Bad Bibliometrics
No Composite indicator
Ranking Web: Mission, Vision
MULTISOURCE Bibliometrics
Altmetrics Webometrics
True GLOBAL Coverage 24,000 HEIs
OPEN
Governance to Society to Industry Access Data A MODEL (1:1)
for the
Composite Indicator
ONE RANKING
Unique for the all the missions PERFORMANCE
Web presence: The Whole University
VISIBILITY
Web Impact: Huge, global and diverse audiences
IMPACT
On the colleagues and beyond
COMMITMENT
An feasible way to measure quality in first mission
OPENNESS
Institutional repositories are the treasure of the University
EXCELLENCE Top research
TRANSPARENCY Fighting unethical
INTERNATIONALIZATION Attracting talent
3624 166
290 20 987
2
8311 84
3798 16
705 3 6172
209
23887
500
WR January’15Current coverage of the Ranking Web
Sources for the Ranking (2.0) Web
Biblio- metrics
Webo- metrics
Composite Indicator
Excellence indicator: Top 10% cited papers in 21 disciplines
Number of webpages under common web domain
ACADEMIA FACEBOOK LINKEDIN MENDELEY
RESEARCHGATE SCRIBD
SLIDESHARE TWITTER
WIKIPEDIA (all)
WIKIPEDIA (English) YOUTUBE
Mentions in
SCIMAGO
Alt- metrics
MAJESTIC / AHREFS
Number of backlinks and referring domains to the common domain
Number of (open) documents
GOOGLE SCHOLAR
More facts about the Ranking Web (Webometrics)
authority
Edited by public-funded research body, not a private for-profit company
Scientific
Evolving methodology, not guided for stability
target It ranks universities, not websites
Popularity (visits, visitors) are not taken into account Web design (usability) is irrelevant for the ranking currency
Ranks are based on current data, collected every 6 months
Metrics 2.0
BETTER PROFILES MORE METRICS CUSTOMISATION GLOCAL TRANSPARENCY Recovering metrics prestige
Hint: Leiden Declaration (in prep) by the European Network of Indicator Designers
Questions?
SaCSbeoAAAAJ 124106
isidro-f-aguillo A-7280-2008
0000-0001-8927-4873
IsidroAguillo 3481717
@isidroaguillo
6507380175
