\documentstyle{article}
\def\texhome{/home/rolf/tex}

\def\r#1{\right#1}
\def\opcompl{{\cal C}}
\def\opsingle{{\cal S}}
\def\opdouble{{\cal D}}
\def\opinhib{{\cal I}}
\def\opcomb{{\cal E}}
\def\oporient{{\cal O}}
\def\opall{{\cal A}}

\flushbottom

\title{Object representations and flexible matching methods}

\author{Rolf P.\ W{\"u}rtz\\
Institut f{\"u}r Neuroinformatik\\
Ruhr-Universit\"at Bochum\\
D-44780 Bochum\\
Germany\\
Email:  Rolf.Wuertz@neuroinformatik.ruhr-uni-bochum.de\\
http://www.neuroinformatik.ruhr-uni-bochum.de/ini/PEOPLE/rolf/
}

\date{}

\begin{document}
\maketitle
% 
% 
% 
% {\vspace*{-6.0cm}
% \noindent {\begin{tabular}{lp{11cm}}\sf From:& \sf Proceedings of 
%     the Third Asian Conference on Computer Vision, 
%     Hong Kong, January 1998,
%     Lecture Notes in Computer Science, Springer Verlag, 1998.
%   \end{tabular}}\par 
% \vspace*{4.5cm}}

\section{Introduction}

As a startup I posed the following three theses in order to supply
discussion material and to excuse some lack of mathematical rigor in
my presentation.
\begin{enumerate}
\item computer vision can not exist without reference to human vision.
\item Human vision is a weird thing.
\item Due to 1) and 2) rigorous mathematical models should not be expected.
\end{enumerate}
The reason for 1) is that the question if any computer vision algorithm
has solved its task must be decided by a human observer. 2) has been 
underpinned by some ad hoc demonstrations.

\section{Background invariant face recognition}
The processing of a retinal grey-level image in the primary visual
cortex can be modeled by a wavelet transform based on complex-valued
Gabor functions~\cite{transcomp,diss}.  The single wavelet is
parameterized by its two-dimensional spatial frequency vector.  The
responses of all spatial frequencies of some fixed length form a {\em
frequency level}, which assigns a small feature vector to all image
points on an appropriate sampling grid.  This pyramidal arrangement
has the advantage that all responses in the stored model which are
influenced by the background can be discarded, but the object (which
here is the face without hair) still represented well enough for
recognition. The image to be analyzed consists of a full pyramid, a
presegmentation is not required.

Now, the pyramidal representations of image and model can be matched
using the following modules:
\begin{enumerate}

\item The {\em coarse localization} of the counterpart of the model in
the image is done by global template matching of the vectors
of Gabor amplitudes on the lowest frequency level. This is
not very expensive, because the resolution is low, and yields
a first rough correspondence mapping. 

\item Mappings acquired using only the amplitudes of the Gabor
responses are not very precise, because the fine geometrical
information resides in the phases. On the other hand, the
phases or the full complex responses are not suitable for
template matching because they depend strongly on the
sampling grid. Therefore, a {\em local phase matching} has been
implemented that enhances the accuracy of amplitude-based
mappings. This can be done in parallel on all model points.

\item In order to cope with occlusion problems, it must be possible
to {\em exclude points} from the mapping. This is done on the basis
of poor similarity, which is also possible in parallel on all
image points.

\item Finally, any mapping can be refined by local template
matching with amplitudes from the next higher frequency
level. For this, the model is split up into several patches
that independently search for correspondences in an area
defined by the coarse mapping already known.

\end{enumerate}

The procedures just described yield a correspondence map on every
frequency level.  Each of them can be
used for recognition in the following way. An incoming image is
matched to every model out of a database. Some global similarity is
calculated from the actual feature similarities and the distortion of
the mapping.  The model with the highest similarity is the recognized
one; the recognition is {\em significant} if the distance of the
highest similarity to the distribution of all similarities exceeds a
suitable threshold. This scheme has already been applied successfully
in~\cite{transcomp}. 

In the coarse-to-fine hierarchical matching procedure described above
the notion of significance can been exploited to build a hierarchical
recognition scheme: If a recognition is significant on one frequency
levels, the higher ones need not be evaluated. 

The work from this section is described in~\cite{pyramid}, and in a
much more detailed way in~\cite{diss}. Local phase matching has been
exploited for the tracking of facial points in~\cite{avbpa97}.



\section{Hierarchical dynamic link matching}
Conventional neural networks try to solve the problem of object
recognition in a single step by building a stimulus-response system
that codes its result as cell activities.  We take a different
approach assuming that recognition is an active process with temporal
dynamics and results in an ordered state. I presented a structure of
neuronal layers, interconnected by dynamic
links~\cite{bunsen,nn,transcomp} that solves the correspondence
problem between two images and thus constitutes an important building
block for a model of recognition.

Images as well as stored models are represented as Gabor pyramids.
This allows the dynamics to proceed from coarse to fine scale and
reduces the sequential processing time inherent in the
concept. Invariance under background changes is also made possible.

On the lowest frequency level a single blob of activity moves across
the image and model layer, respectively. Dynamic links between these
layers are initialized to the (highly ambiguous) feature similarities.
Links grow or decline according to a combination of feature similarity
and correlated activation. This enforces correct neighborhood
relationships in addition to feature similarity. On the higher levels
the established correspondences are refined by several blobs in
parallel.

I presented an improved version of the dynamical system
from~\cite{diss} and show with examples of human faces that it evolves
from an unordered link distribution to an ordered state where only
corresponding point pairs are connected by strong
links. Correspondences between sample points are population-coded by a
set of neighboring links. The work presented is described partly
in~\cite{palmas95,diss}, a complete description is still in
preparation.

\section{Model matching using symbolic edge graphs}
Labeled graph matching is a method used successfully for, e.g.,
invariant face recognition~\cite{transcomp,bunchgraph} (see also Jan
Vorbr{\"u}ggen's talk. In that system from ~\cite{transcomp}, vertices
are assigned local texture elements and edges carry the geometrical
information about the relative locations.  Applicability of that
method is limited to richly structured or textured objects like
faces. Objects with homogeneous surfaces do not provide the sort of
vertex labels required there and can only be matched using their
outlines.

In this final section, I presented an object recognition system based
on symbolic graphs with object corners as vertices and outlines as
edges. Corners are determined in a robust way by a multiscale
combination of an operator modeling cortical end-stopped
cells~\cite{icann97}. Graphs are constructed by line-following between
corners.  Model matching is then done by finding subgraph isomorphisms
in the image graph. The complexity is reduced by adding labels to
corners and edges. The choice of labels makes the recognition system
invariant under translation, rotation, and scaling. The work is described
in~\cite{accv98}

\section*{Remark}

Postscript files
for~\cite{accv98,avbpa97,palmas95,diss,pyramid,icann97} can be
retrieved from\\ {\sf
http://www.neuroinformatik.ruhr-uni-bochum.de/ini/PEOPLE/rolf/publications.html},
the one for~\cite{bunsen} from\\ {\sf
http://www.neuroinformatik.ruhr-uni-bochum.de/ini/VDM/publications/top.html}.\\
All references also exist on old-fashioned paper.


%\bibliographystyle{plain}
%\bibliography{own,malsburg}

\begin{thebibliography}{10}

\bibitem{transcomp}
Martin Lades, Jan~C.\ Vorbr{\"u}ggen, Joachim Buhmann, J{\"o}rg Lange,
  Christoph von~der Malsburg, Rolf~P.\ W{\"u}rtz, and Wolfgang Konen.
\newblock Distortion invariant object recognition in the dynamic link
  architecture.
\newblock {\em IEEE Transactions on Computers}, 42(3):300--311, 1993.

\bibitem{accv98}
Tino Lourens and Rolf~P.\ W\"urtz.
\newblock Object recognition by matching symbolic edge graphs.
\newblock In {\em Proceedings of the Third Asian Conference on Computer Vision,
  Hong Kong, January 8 - 11, 1998}, Lecture Notes in Computer Science. Springer
  Verlag, 1998.
\newblock In press.

\bibitem{avbpa97}
Stephen~J. McKenna, Shaogang Gong, Rolf~P.\ W\"urtz, Jonathan Tanner, and
  Daniel Banin.
\newblock Tracking facial feature points with {G}abor wavelets and shape
  models.
\newblock In Josef Big{\"u}n, Gerard Chollet, and Gunilla Borgefors, editors,
  {\em Proceedings of the First International Conference on Audio- and
  Video-based Biometric Person Authentication Crans-Montana, Switzerland, March
  1997}, volume 1206 of {\em LNCS}, pages 35--42. Springer Verlag, 1997.

\bibitem{bunsen}
Christoph von~der Malsburg.
\newblock Nervous structures with dynamical links.
\newblock {\em Ber. Bunsenges. Phys. Chem.}, 89:703--710, 1985.

\bibitem{nn}
Christoph von~der Malsburg.
\newblock Pattern recognition by labeled graph matching.
\newblock {\em Neural Networks}, 1:141--148, 1988.

\bibitem{bunchgraph}
Laurenz Wiskott, Jean-Marc Fellous, Norbert Kr{\"u}ger, and Christoph von~der
  Malsburg.
\newblock Face recognition by elastic bunch graph matching.
\newblock {\em IEEE Transactions on Pattern Analysis and Machine Intelligence},
  19(7), 1997.

\bibitem{palmas95}
Rolf~P.\ W{\"u}rtz.
\newblock Building visual correspondence maps --- from neural dynamics to a
  face recognition system.
\newblock In Roberto Moreno-D{\'\i}az and Jos\'e Mira-Mira, editors, {\em Brain
  Processes, Theories and Models}, pages 420--429. MIT Press, November 1995.

\bibitem{diss}
Rolf~P.\ W{\"u}rtz.
\newblock {\em Multilayer Dynamic Link Networks for Establishing Image Point
  Correspondences and Visual Object Recognition}, volume~41 of {\em Reihe
  Physik}.
\newblock Verlag Harri Deutsch, Thun, Frankfurt am Main, 1995.

\bibitem{pyramid}
Rolf~P.\ W{\"u}rtz.
\newblock Object recognition robust under translations, deformations and
  changes in background.
\newblock {\em IEEE Transactions on Pattern Recognition and Machine
  Intelligence}, 19(7):769--775, July 1997.

\bibitem{icann97}
Rolf~P.\ W\"urtz and Tino Lourens.
\newblock Corner detection in color images by multiscale combination of
  end-stopped cortical cells.
\newblock In Wulfram Gerstner, Alain Germond, Martin Hasler, and Jean-Daniel
  Nicoud, editors, {\em Artificial Neural Networks -- ICANN '97}, volume 1327
  of {\em Lecture Notes in Computer Science}, pages 901--906, Berlin Heidelberg
  New York, 1997. Springer Verlag.

\end{thebibliography}


\end{document}






