| Description |
This package provides implementations of several spectral (FFT,DCT)
saliency algorithms for images.
There are several methods (multichannel_method) to calculate the
multichannel saliency:
'fft': by default the same as 'fft:whitening'
'fft:whitening' Uses spectral whitening to calculate the saliency of
each channel separately and then averages the result.
'fft:residual' Uses the spectral residual to calculate saliency of
each channel separately and then averages the result.
'dct' Uses DCT-based image signatures to calculate saliency
of each channel separately and then averages the
result.
'quat:fft': Converts the image into a quaternion-based
representation, uses quaternion FFT/IFFT operations.
'quat:dct' Converts the image into a quaternion-based
representation, uses quaternion DCT/IDCT operations.
'quat:dct:fast' Same as 'quad:dct', but with a fixed image
resolution of 64x48 and uses optimized .mex files for
faster calculation.
Usage examples:
- spectral_saliency_multichannel(imread(..image path...))
or as an example for other color spaces (e.g. ICOPP, Lab, ...)
- spectral_saliency_multichannel(rgb2icopp(imread(..image path...)))
If you use any of this work in scientific research or as part of a
larger software system, you are requested to cite the use in any
related publications or technical documentation. The work is based
upon:
B. Schauerte, and R. Stiefelhagen, "Predicting Human Gaze using
Quaternion DCT Image Signature Saliency and Face Detection," in IEEE
Workshop on the Applications of Computer Vision (WACV), 2012.
Notes:
- The implementation of the quaternion-based approach requires the
quaternion toolbox for Matlab (QTFM).
- I kept the implementations as focused and simple as possible and
thus they lack more advanced functionality, e.g. more complex
normalizations. However, I think that the provided functionality is
more than sufficient for (a) people who want to get started in the
field of visual attention (especially students), (b) practitioners
who have heard about the spectral approach and want to try it, and
(c) people who just need a fast, reliable, well-established visual
saliency algorithm (with a simple interface and not too many
parameters) for their applications.
- GBVS and Itti require the original GBVS Matlab implementation by
J. Harel (see http://www.klab.caltech.edu/~harel/share/gbvs.php)
For more details on the method see:
[1] X. Hou and L. Zhang, "Saliency Detection: A Spectral Residual
Approach", in CVPR, 2007.
(original paper)
[2] C. Guo, Q. Ma, and L. Zhang, "Spatio-temporal saliency detection
using phase spectrum of quaternion fourier transform," in CVPR,
2008.
(extension to quaternions; importance of the residual)
[3] X. Hou, J. Harel, and C. Koch, "Image Signature: Highlighting
sparse salient regions," in PAMI, 2011.
(uses DCT-based "image signatures")
[4] B. Schauerte, and R. Stiefelhagen, "Predicting Human Gaze using
Quaternion DCT Image Signature Saliency and Face Detection," in IEEE
Workshop on the Applications of Computer Vision (WACV), 2012.
(extension to quaternions; spectral saliency and face detection;
evaluation of spectral saliency approaches on eye-tracking data;
achieved the currently best reported results on the CERF/FIFA
eye-tracking data set and Toronto/Bruce-Tsotsos data set)
It has been applied quite a lot during the last years, e.g., see:
[5] B. Schauerte, B. Kuehn, K. Kroschel, R. Stiefelhagen, "Multimodal
Saliency-based Attention for Object-based Scene Analysis," in
IROS, 2011.
("simple" multi-channel and quaternion-based; Isophote-based
saliency map segmentation)
[6] B. Schauerte, J. Richarz, G. A. Fink,"Saliency-based
Identification and Recognition of Pointed-at Objects," in IROS,
2010.
(uses multi-channel on intensity, blue-yellow/red-green opponent)
[7] B. Schauerte, G. A. Fink, "Focusing Computational Visual
Attention in Multi-Modal Human-Robot Interaction," in Proc. ICMI,
2010
(extended to a multi-scale and neuron-based approach that allows
to incorporate information about the visual search target
to facilitate joint attention in spoken human-robot interaction)
However, the underlying principle has been addressed long before:
[9] A. Oppenheim and J. Lim, "The importance of phase in signals,"
in Proc. IEEE, vol. 69, pp. 529-541, 1981.
|
