(click each sequence to see more details)
(click each sequence to see more details)
In the literature, people from both graphic and vision communities always have keen interests in modeling complex motion patterns like dancing fire, wavy water and dangling cloth. A wide spectrum of models have been proposed to account for these motion phenomena. Fig.1 tries to map these models in a 1D axis according to the number of parameters that a model memorizes from the observed image sequence. At one extreme is the physics-based models. This type of models are very parsimonious, and they explain the motion of the underlying systems by physics with few parameters. At the other extreme is image-based models, which remember every pixels of the system and reproduce new sequences by cut-and-paste techniques.
Despite their success for realistic image synthesis, both models are not friendly or not suitable for image analysis and are perhaps pretty far from the mechanisms used in human visual perception. It is believed that a generic motion model, adopted by human vision, must lies somewhere between the two extremes. We vaguely call it the perception-based model. This model should be able to disclose the “meaningful” moving objects in a motion sequence, so as to answer the question as the title of this subsection.
2. A Perceptual-based Generative Model for Complex Motion.
The success of motion modeling lies in how to cooperate the image modeling and dynamic modeling well.
Early psychology studies including Julesz's texton concept and Marr's primal sketch scheme shed light to our image representation. They argue that human are prone to perceive the structural part of a scene made up by image primitives or primal sketches, and tend to ignore details less structured. Our perceptural-based generative method is based on this observation. A given motion sequence I[0,t] is represented by a series of attributed graphs G[0,t] by a primal sketch model [1]. Each vertex/node in the graph corresponds to a primal sketch in a frame. This primal sketch model with the attributed graph representation captures not only the image photometric properties, but also the geometric properties.
While traveling in spatial-temoporal domain, the graphs of river, fire, and cloth evolve with both continuous movement and abrupt structural changes. Modeling the abrupt structural changes or topological changes is considered challenging in the literature. Our generative method aims to take care of both the motion cases. We assume the motion of the graphs is driven by two types of forces: (1) drifting force acting on each vertex which is Brownian motion; (2) topological operators acting on subgraphs and thus change the graph structure (topology). Thus the system is depicted as a three-layer hidden Markov model (HMM) as shown in Fig. 2.
Under this framework, we have the joint probability for an image sequence I[0,t], the hidden graph representation G[0,t] and the external force field F[0,t],
This probability model includes the following four components:
Transfers an input image into an attributed graph representation using the primal sketch model [1]. Each vertex of the graph is a scaled and oriented image patch selected from a dictionary. The graph connects and align these patches.
Characterizes the deformation of the attributed graph.
Specifies the motion dynamics of these vertices and their interactions in the coupled Markov chains..
Interprets the graph topological changes over time.
Given an observed image sequences I[0,t], we want to achieve two objectives:
- Infer the hidden graph structures G[0,t].
- Fit all parameters 'theta' of the image model, dynamic model, and topological model.
We adopt the EM like stochastic gradient algorithm as our learning process to approximate the global optimal 'theta'. It iteratively sampling the hidden variables, update the parameters. The computation is realized by data driven Markov Chain Monte Carlo techniques.
