Probabilistic Relational Model

Probabilistic relational model (PRM) (Friedman et al., 1999; Getoor et al., 2001) describes a probabilistic formulation for a relational data base. It integrates Bayesian network with the database structure representation relational model (Ullman & Widom, 1997). PRM is a milestone in the development of statistical relational learning. Koller and Pfeffer (1997) proposed object-oriented Bayesian network, which extends the Bayesian network with the concepts ofclasses,objects, and theirattributes. The model can be viewed as the initial work of PRM. Koller and Pfeffer (1998) introduced the probabilistic frame-based system, which combines the frame-based knowledge representation with Bayesian network to model organizational structure of a large complex domain. It provides more expressive power than traditional Bayesian network. With these early researches, Friedman et al. (1999) developedprobabilistic relational model, which is a compact and effective language to describe a statistical formulation over a typed relational domain. A PRM models the probabilistic uncertainty over the attributes of objects and relationships between objects.

2.3. SRL MODELS 13

(a) (b)

(c) (d)

Figure 2.2: An example of PRM over school domain from (Getoor et al., 2001). (a) Relational schema specifying the classes, descriptive attributes and reference slots. (b) Dependency structure and local probability model. (c) An example skeleton instantiating objects and relationships. (d) Ground Bayesian network which is obtained by applying the PRM template in (b) to the example skeleton in (c).

14 CHAPTER 2. STATISTICAL RELATIONAL LEARNING

An attribute of an object depends on not only other attributes of the same object, but also the attributes of related objects. PRM provides a new perspective for data mining.

Probabilistic relational model is motivated from Bayesian network (BN). A BN is a graphical model to encode the probabilistic dependencies between variables, which pro- vides an elegant formalism for representing and reasoning probabilistic uncertainty. The major advantage is that it exploits the underlying structure of the domain knowledge to represent the joint distribution in an effective way. However, BN lacks the concepts of ob- jects and relationships. In many real-world applications, the domain of interest typically consists of objects, their attributes and relationships between them. This kind of under- lying information can not be captured by a traditional Bayesian network. Generally, BN pre-processes the data into a flat representation, and then, the probabilistic dependencies are learned and reasoned. It is obvious that some important patterns are missing in the procedure of flattening the data. In addition, there is an important assumption in BN, i.e. the samples are independently and identically-distributed (i.i.d.). However, in more cases than not, the samples are linked together into a ground network via relationships. The information about relationships is helpful in making decision/prediction. For exam- ple, in a social network, the friendship between two persons influences the frequency and mode of communication between them. PRM is a framework integrating relational logic to overcome these limitations in a compact and natural way.

A probabilistic relational model consists of three components: relational schema, de- pendency structure and local probability model. Figure 2.2 shows an example on school domain. A relational schema describes data structure of the domain of interest. It con- sists of a set of classes, e.g. Student and Take. Each class is associated with a set of descriptive attributes and a set of reference slots. A descriptive attribute represents a particular property of objects in the class, e.g. Student.Intellignce, which specifies in- telligence of a student. A reference slot describes a relationship between two classes, e.g. Take.Student, which specifies an instance in the class Take is related with an in- stance in the class Student. Figure 2.2(b) shows dependency structure and local prob- ability model for the running example. The probabilistic dependencies are specified by the solid directed arcs, e.g. the arc from Student.Intellignce to Take.Grade specifies the fact that student’s grade depends on his intelligence. An attribute can depend on other attributes of the same class, or the attributes of the related classes, e.g. a probabilistic parent of Take.Grade is Student.Intelligence, which is an attribute of the class Student which is related to the class Take. The local probability model can be a conditional probabilistic table for a discrete attribute, or a conditional probabilistic density func- tion for a continuous attribute. In the school example, the local probability model is

P(T ake.Grade|Student.Intelligence, Course.Dif f iculty).

A PRM is a probabilistic template on the domain of interest. It will be replicated on a particular skeleton. A skeleton specifies a possible relational structure of the domain and is an instantiation of objects and relationships for a schema. Figure 2.2(c) shows an example skeleton. Note, that the particular values of attributes in a skeleton can be unknown. Applying dependency structure defined by a PRM to an example skeleton, we obtain a ground Bayesian network, e.g. Figure 2.2(d), which represents the joint probability over all attributes and relationships in the skeleton. The probabilistic inference is finally

2.3. SRL MODELS 15

(a) (b)

(c)

Figure 2.3: An example of DAPER model on the school domain from (Heckerman et al., 2004). (a) DAPER model. (b) Instantiated objects and relationships. (c) Ground Bayesian network. All information propagates to the attribute of interest, i.e. George’s grade at the course Geo101. The grey arrows show the procedure of probabilistic inference.

performed on the ground network.

In summary, PRM integrates the relational database model with Bayesian network. An attribute can probabilistically depend on not only the attributes of the same object, but also the attributes of related objects. PRM is an important contribution in the development of statistical relational learning.