Bibliometrics

Services / Bibliometrics

Services

Reference & Research Services
ILL
Bibliometrics
Author Services
Brown Bag Seminars

BIBLIOMETRICS

Bibliometrics and citation analysis is one of the services the NOAA Central Library offers to NOAA researchers and staff and is one way we are able to illustrate NOAA's status as producer of world-class research. This page provides a guide to understanding and getting started with bibliometric measures. A printable version of this guide is available for download.

For information on other services we provide NOAA researchers please see the Author Services page.

For a searchable bibliography of NOAA articles please see the NOAA Peer-Reviewed Articles Bibliography.

bibliometrics

Bibliometrics 101

What are bibliometrics?

Bibliometrics are the quantitative analysis of academic publications. Using academic publications as a data source, bibliometric analysis attempts to provide a better understanding of how research is produced, organized, and interrelated. It also attempts to evaluate academic publications and sets of publications based on the number of citations these publications have received. Bibliometrics and citation analysis is one way we are able to illustrate NOAA's status as producer of world-class research.

Why are bibliometrics important?

Bibliometrics can help illustrate the impact of a scholarly publication or group of publications in the greater research community and can support application for grants and research funding.
When used with other methods such as peer review, bibliometrics is a useful tool in evaluating the research output of programs and researchers.
Bibliometrics can be used to identify research strengths and gaps in research and inform decisions about future research.

Limitations of bibliometrics

Bibliometric indicators are imperfect and can only be used for their specific purpose. For example: Journal Impact numbers can only be used to evaluate journals and not the papers in those journals.
Citation counts and behaviors vary among fields and over time and comparisons between subject areas should be avoided. Additionally, articles need at least two years to accumulate enough citations to provide meaningful analysis.
Most bibliometric datasets have skewed distributions and as such an average is not necessarily representative of the whole. Providing median or other metrics in addition to mean helps resolve this problem.
Many metrics are easy to manipulate such as increasing H-Index through self-citation and these metrics should be used with care and a full awareness of the potential for manipulation.
Bad data makes for bad bibliometrics; to produce a quality analysis it is important to start with a clean dataset of high quality and adequate size. Using data from a reliable source such as Web of Science makes this much easier.
There’s no way to definitively know what indicators actually measure. For instance, high citation rates don’t necessarily correlate to credibility and an article may be highly cited because the research it presents is being questioned.

Types of Metrics

Publication Metrics

Publication metrics are used to analyze the research output of an individual, group or institution and include number of publications as well as:

Publications per:

Year or Quarter

Office, lab or division

Journal Title

Subject Area

Author

Funding Agency

Type of Publication

Citation Metrics

Citation metrics are used to help determine the relative quality and impact of a single article or the research output of an individual, group or institution using citation counts.

Citation metrics include:

Citation Rate
Citations Received Per Year
Mean Citation Rate
Median Citation Rate

H-Index is a metric calculated using publication and ciation rates and can help determine relative quality of an author's work based on both productivity and impact. An author's h-index is equal to the number of papers that author has published (n) that have received at least n citations; if an author has published 32 papers and 5 of those papers have received at least 5 citations, the author has an h-index of 5.

Keep in mind that h-index cannot be compared across fields due to variations in citation conventions and habits and is inherently biased towards older authors who have not only published more works but had the time to accumulate more citations.

Analysis of Citing Articles

Performing a publication analysis on the publications that cite a given article or set of articles can provide insight into how articles are being used by the greater scientific community. Tools like Web of Science make performing such an analysis relatively simple and adds helpful context.

Percentile Ranks

An article's percentile rank is calculated based on how it compares to other papers published in the same year in the same field based on citation counts. In Web of Science, an article that received that received enough citations to rank it in the top one percent of papers published in the same year and field is considered a Highly Cited Paper. Percentile Ranks can be calculated for individual publications or for the output of an entire program or organization and have the advantage of being useful to compare impact across fields of study.

NOTE: Articles require at least 2 years to accumulate enough citations for article-level bibliometric indicators to be reliable.

Network Analysis

Network analysis examines and visualizes the relationships between publications based on authorship, citations, or common terms.

There are three types of bibliometric networks:

Collaboration Networks show colloborations between authors, institutions, or groups (such as NOAA line offices). Collaboration network maps illustrate how various parties are working together.
Sematic Networks or word co-occurance network maps show the most commonly co-occuring words in the titles, abstracts or keywords of articles in a given set. These networks are useful in showing how how potentially diverse areas of research interrelate and overlap.
Citation Networks illustrate relationships between publications based on citations. There are three types of citation networks: direct (connecting articles that cite each other); co-citation (connecting articles cited by the same publications); and bibliographic coupling (connecting articles that are cited by the same publications). These networks can be useful in illustrating emerging research trends and "hidden" research communities.

Journal Metrics

Journal metrics seek to illustrate relative importance of a journal within its field. These metrics are not appropriate for evaluating articles published in those journals or the researchers who write those articles.

Basic Journal Indicators include:

Journal Impact Factor (JIF)
Eigenfactor
Scimago Journal & Country Rank (SJR)
Cited Half-Life

Ongoing Bibliometrics Projects at NCL

The NOAA Central Library has number of ongoing projects including an effort to track and analyze all peer-reviewed articles published by NOAA authors dating back to 2012.

Learn more about the NOAA-Authored Publications Tracking project

NOAA-Authored Publications

Project Overview

In December 2011, the library was tasked with assisting OAR in reporting the number of peer-reviewed publications produced by all NOAA authors per quarter in a consistent and reproducible manner. Using Web of Science as our primary data source, the library continues to identify and manage NOAA publications for this purpose.

In November 2012, we performed a preliminary analysis of the 1,800 publications that had been collected for fiscal year 2012. The analysis aimed both to summarize the nature of the publications produced and to begin the discussion about the types of analyses that would be useful to the agency. These discussions are ongoing.

A bibliography of NOAA-authored peer-reviewed publications is publicly available and updated quarterly.

Reports and Publications

An Analysis of Publications by NOAA Authors Published in FY2017 (pdf)

An Analysis of Publications by NOAA Authors Published in FY2016 (pdf)

Learn more about the Ocean Exploration and Research Bibiography

Ocean Exploration and Research Bibliography

Since 2001, Office of Ocean Exploration and Research (OER) has funded multiple Ocean Explorer expeditions and maintains an interactive Digital Atlas of all of the cruises sponsored by the Ocean Explorer program.

The data gathered on these expeditions has resulted in the publication of over 800 peer-reviewed journal articles, reviews, and notes. To provide access to these publications and to highlight the research resulting from the Ocean Explorer program, the NOAA Central Library have developed this bibliography. A citation analysis of these publications is produced quarterly and the most recent edition is available as a PDF.

This bibliography is available organized by author or expedition.

Upon request, we also offer services including:

Compiling metrics for your publications;
Producing custom analyses for labs and programs;
Providing training on bibliometric theory, practice, and tools;
Assisting in tracking trends in research fields.

If you’re interested in learning more about our program, would like to schedule a tutorial, or are interested in a custom portfolio analysis, contact Sarah Davis or Jamie Roberts.

Bibliometrics and You

NOAA staff have access to ORCiD or ResearcherID, both of which offer basic metrics as well as a way to distinguish your published work. Additionally, authors can set up a profile in Google Scholar to track who is citing your publications an other author metrics. For more information, visit the Google Scholar Citations page.

For more information on using these and other tools to create your own metrics, contact Sarah Davis or Jamie Roberts.