Find spikes in vector

Find spikes in a numeric vector using the algorithm of Whitaker and Hayes (2018). Spikes are values in spectra that are unusually high or low compared to neighbours. They are usually individual values or very short runs of similar "unusual" values. Spikes caused by cosmic radiation are a frequent problem in Raman spectra. Another source of spikes are "hot pixels" in CCD and diode arrays. Other kinds of accidental "outliers" can be also detected.

Usage

find_spikes(
  x,
  x.is.delta = FALSE,
  height.threshold = 10,
  z.threshold = 5,
  k = 20,
  spike.direction = "both",
  return.numeric = FALSE,
  na.rm = FALSE,
  max.spike.width = NA
)

Arguments

x

numeric vector containing the data.

x.is.delta

logical Flag indicating whether x contains differences or original values.

height.threshold

numeric The minimum height of spikes expressed relative to the median amplitude of the baseline local variation of x.

z.threshold

numeric Modified local \(Z\) values larger than z.threshold are detected as boundaries of spikes.

k

integer width of median window used for smoothing; must be odd

spike.direction

character Controls the direction of spikes to be detected. Accepted arguments are "up", "down", "both".

return.numeric

logical If TRUE a numeric vector is returned and otherwise a logical one.

na.rm

logical indicating whether NA values should be stripped before searching for spikes.

max.spike.width

integer The width of the widest spike to be detected, NA puts no limit.

Value

An integer vector of the same length as x. Values that are 0, +1 or -1 corresponding to no-spike, upwards-spike, and downwards-spike in the data. Conversion to logical with as.logical() results in a vector with TRUE for spikes and FALSE otherwise.

Spike detection

Spikes are detected based on a modified \(Z\) score calculated from the differenced spectrum. The \(Z\) threshold used should be adjusted to the characteristics of the input and desired sensitivity. The lower the threshold the more stringent the test becomes, with shorter spikes being detected.

The algorithms assume a consistent step size for the underlying independent variable, e.g., wavelength or time, and should not be applied if the data do not fulfil this assumption, at least approximately. As find_spkikes() operates on a single vector, checking this remains the responsibility of calling functions or methods such as spikes() and despike().

The algorithm uses running differences to detect abrupt changes in value, compared to an estimate of the baseline variation of the differences, approximating a baseline \(Z\) from MAD and a baseline value from the median differences. Currently, a single estimate of MAD is used but running medians, when possible, as baseline. This comparison detects running differences that are unusually large, in most cases signalling a transition between values near the baseline and far from it, in both directions.

Transitions into- and out of spikes are distinguished based on the median of the non-differenced values, as a descriptor of the data baseline. As for the median of the differences, a running median is used when possible.

This function thus detects the start and end of each spike, and distinguishes upward and downward spikes.

k is the width in number of observations of the window used for running median smoothing to extract the baseline. A value several times the width of the broader spike but narrow enough to track broader peaks needs to be manually set in most cases.

With na.rm = TRUE, NA values are omitted before searching for spikes and set to 0L in the returned vector.

If all spikes are guaranteed to be one observation-wide and either going up or down from the baseline, it is possible to detect them based purely on the z.threshold by passing height.threshold = NA and either spike.direction = "up" or spike.direction = "down", which ensures very fast computation.

Parameters of the algorithm need to be adjusted depending on the data, so inspection of returned values is needed together with adjustment by trial and error of suitable values for z.threshold, height.threshold, and k.

Parameter max.spike.width searches for too wide spikes in the output of the algorithms described above and ignores them. This is possibly redundant, but maintained for partial backwards compatibility.

References

Whitaker, D. A.; Hayes, K. (2018) A simple algorithm for despiking Raman spectra. Chemometrics and Intelligent Laboratory Systems, 179, 82-84. doi:10.1016/j.chemolab.2018.06.009 .

See also

Other peaks and valleys functions: find_peaks(), get_peaks(), peaks(), replace_bad_pixs(), spikes(), valleys(), wls_at_target()