Controlling robotic systems with factor graphs

Our group is attempting to model neural dynamics in the cortex as networks of relations. More specifically, we are working on undirected graphical models such as factor graphs. These models are well studied and have successfully been applied to sensory tasks such as handwritten digit recognition. However, no one has yet attempted to use them to generate motor output.
We are using simple robotic systems such as a small pendulum to investigate in which way factor graphs can be applied to generate meaningful motor output.
Traditional factor graphs can effectively be used to predict the state of non-observed variables when the states of a sufficient amount of variables are given.
Unfortunately, it is difficult to model the influence of time using factor graphs as the state of variables will change over time. This poses a problem if we want to control a robotic system: If the present state and the desired state of a robotic system are known, it might happen that there would be no motor action that corresponds to the desired transition. Instead, a sequence of actions might be necessary to approach the goal. Such a sequence would require a planning procedure, which would need to predict the change of the system in time.
Therefore, we will investigate how factor graphs can be extended such that they can be used to compute actions which require planning.