Visualizing Graphical Probabilistic Models
Chih-Hung Chiang*, Patrick Shaughnessy, Gary Livingston, Georges Grinstein
Department of Computer Science, University of Massachusetts Lowell, Lowell, MA01854
ABSTRACT
Complex probabilistic models are difficult to evaluate not only for consistency with the domain theory but also for novelty and significance. In this paper we describe the current visual representations of Bayesian networks and then present an implementation containing enhancements to these visualizations by integrating several well known techniques. We then apply our approach in an exploration of graphical models and use Bayesian networks as an illustration.
Keywords: Graphical probabilistic model, Bayesian network, visualization
1.
INTRODUCTION
For many applications, merely learning or manually developing probabilistic models and then quantitatively evaluating them is not enough. These models also need to be explored qualitatively in order to identify portions of them that are consistent with the domain theory, portions which are inconsistent with the domain theory, and portions which represent potentially novel and significant discoveries. Current methods for visualizing probabilistic models exist for a variety of areas which include for example geo-spatial data and flow visualization (Pang et al.1). We present interactive methods for visualizing the probabilities contained in graphical models and use Bayesian networks to illustrate these techniques. Bayesian networks (Pearl2) are graphic structures for representing probabilistic relationships among variables and for performing probabilistic inference with those variables. They have proven to be a valuable tool for encoding, learning and reasoning with probabilistic relationships. A Bayesian network consists of two primary components; The qualitative component is a directed acyclic graph (DAG) in which each node represents an attribute of the data and the edges represent the dependencies among the attributes. The quantitative component is a set of conditional probability distributions which give the probabilities for the value of each node given the value of its parents.
Figure 1 shows a Bayesian network with three Boolean attributes and their corresponding conditional probability tables (CPT). In the network given in Figure 1, attribute A has a 50% chance of having the value T, given no other information, and attribute C has a 37.5% chance of having the value T if A’s value is T and a 62.5% chance of having the value T if A’s value is F. When A and C are both T or both F, B has a 50% chance of having the value T, and when A and C have different values, B has a 90% chance of having the value T. Table 1 shows the joint probability distribution for the Bayesian network in Figure 1.
Many model analysis issues can benefit from visual analysis. Card et al3 describe numerous information visualization techniques. Thearling et al.4 emphasizes that visualizing a model should allow a user to understand, discuss and explain the logic behind the model, gaining a user’s trust. We hypothesize thus that information visualization techniques should prove useful for harnessing Bayesian networks for modeling and analyzing the underlying data.
The visualization methods presented in this paper are implemented in an interactive visualization tool for the exploratory analysis of Bayesian networks. This tool allows users to explore the cause-effect relationships represented in the conditional probability tables using simple and interactive manipulations of the visual representations, thus facilitating the discovery of meaningful causal features.
Figure 1. Simple Bayesian network with CPTs.
Table 1. Joint probability distribution for the network in Figure 1
a b c P(A=a AND B=b AND C=c) T T T 0.9375 T T F 0.28125 T F T 0.09375 T F F 0.03125 F T T 0.28125 F T F 0.09375 F F T 0.03125 F F F 0.09375
2.
RELATED WORK
For the past twenty years, much research has focused on developing Bayesian networks (learning the structure of the network), and performing inferences. For example, Friedman et al. 5 used Bayesian networks to analyze expression data. Different software packages serve different purposes with some tools focusing on learning while others emphasize inference.
Murphy6 did a survey of thirty four popular software packages for graphical models based on several features of which the following are the most relevant to our work:
1. Does the package support continuous random variables and does it support sampling? 2. Does the package support undirected graphs?
3. Does the package learn CPTs and the structure of the network? 4. Does the package support utility/action nodes?
Murphy’s analysis motivated the development of the Bayes Net Toolbox (BNT) 6. BNT is an open-source widely used Matlab package for directed graphical models. One of the strengths of BNT is that it offers a variety of inference and learning algorithms which can be used for different kinds of models.
Omitted from Murphy’s survey was whether or not a package provided support for visualizing Bayesian networks. Table 2 is the result of a survey we performed on the Bayesian network visualization of some of the most popular software packages. All of the chosen tools provide visualization of the edges in the models. Some of these tools (Hugin Expert7, BayesBuilder8, Netica9 and GeNIe/SMILE10) use a probability distribution with bar charts within the node, providing an overview of the probability distributions of the values of all nodes at a glance (Figure 2), while other tools only show the node as a simple geometric object (e.g., circle). GeNIE/SMILE (figure 3) also provides an interface for users to display bar chart or pie chart distributions of the probabilities for a selected column. While these tools provide visualizations of the probabilities of a node’s values, none provide graphical views of the entire conditional probability tables.
Table 2. Comparison software for visualizing Bayesian network
Name Visual
representation of the model
Graphical visualization of all node’s probability distribution of values
Graphical visualization of conditional probability tables
Bayes Net Toolbox Yes No No
Hugin Expert Yes Bar charts No
BayesBuilder Yes Bar charts No
WinMine Yes No No
BayesianLab Yes No No
Netica Yes Bar charts
MSBNx Yes No No
Analytica Yes No No
GeNIe/SMILE Yes Bar charts Displays a pie chart distribution of the probabilities for the selected column
Figure 2. The bar chart visualization of the probability distribution in each node in Netica
Figure 3. The pie chart visualization for the selected column in CPT in GeNIE/SMILE
A major issue of the elicitation of numerical parameters in Bayesian networks is navigation through large CPTs. Wang et al. 11 developed two user interface tools, CPTREE (Conditional Probability Tree) and the sCPT (shrinkable Conditional Probability Table) that aim at improving navigation through large CPTs and at improving the interactive assessment of discrete conditional probability distributions. The CPTREE is a tree view of a CPT that allows users to shrink any of the conditioning parents while sCPT is a table view of a CPT with a shrinkable structure for any dimension of the table. Both reduce the size of the table displayed on the screen and allow a user to efficiently navigate through CPTs. GeNIE/SMILE has incorporated these navigation techniques into their package
Despite the large number of software tools available for the modeling of Bayesian networks, as Elmqvist and Tsigas12 state, the work performed on causal relation visualization has been surprisingly low. They proposed the use of partitioned polygons with color-coded segments to show the dependencies between the variables. Zapata-Rivera et al. 13 developed a Bayesian network visualization tool (VisNet), which uses temporal order, color, size, proximity and animation to visualize the cause-effect relationship, marginal probability and probability propagation. However, none of these tools provide for visualization of the conditional probability tables. This is the focus of our work and the next two sections discuss the visual representations of causality and conditional probability tables.
3.
VISUALIZATION OF CAUSALITY
Graphs with nodes and directed edges are widely used to model dependency relationships (Heckerman et al.14) and Bayesian networks use directed acyclic graphs to represent causalities. Nodes represent variables, and directed edges represent direct probabilistic dependences. Under these circumstances, the layout of the DAG plays an important role in the depiction of causality. For instance, the temporal order of the nodes could offer an intuitive notion of the cause-effect relationships. Moreover, the use of visualization attributes such as color and size in the representation of the nodes and edges can also provide valuable information about the networks.
3.1 Layout of Bayesian Networks
Although a number of software tools have been created to build and visualize Bayesian networks, Marriott and Moulder15 mention in their study that the layout provided by these Bayesian networks visualization tools is generally poor.
DAG layout is a well studied area (Sugiyama et al.16). Our focus was not on creating a new DAG drawing algorithm, but instead on finding a layout algorithm for an acyclic directed graph G= (V, E), with nodes V and edges E, with design rules based on the following aesthetic principles.
• Use temporal order to build a hierarchical structure for the cause-effect relationships • Minimize the number of edge crossings
• Keep the edges short and keep the drawing area as small as possible without compromising the readability of the network
One popular approach in graphical drawing is the hierarchical approach (Battista et al.17). It has many variants for drawing DAGs and matches our design rules very well. Figure 4 illustrates the three steps we used to build the network.
Figure 4. Steps in building the layout of the networks
Step 1: layer assignment. During this step, the nodes are assigned to layers L1, L2, L3,…,Lh based on their cause-effect relationships. Overall, the layer number for node j will be larger than node i if there is an edge between parent node i and child node j. We find the longest path first and assign an incremental layer number to each node when it goes through the path. Nodes not appearing on the path are assigned a layer number based on relationships with the nodes already having the layer number.
Step 2: crossing reduction. Nodes in each layer are ordered to reduce the number of edge crossings. We use a heuristic method to find the optimal orderings, building on the approach of Ganser et al. 18.
Step 3: x- and y-coordinate assignment. After node layer assignment and ordering, the layer number is used to assign the y-coordinate and the ordering is used to assign the x-coordinate.
3.2 Visualization of Nodes and Edges
As mentioned earlier, the nodes in Bayesian networks represent variables of the data and often these variables have additional properties. Some are obvious, such as the number of parent variables and the number of children. Some properties, such as the name of the variable, can easily be added to the display. However there are some properties which are hidden or not so obvious. We use color and size to visualize these hidden properties. For edges, we use the thickness and color to highlight properties related to causal relationships, such as the correlation value and the computed confidence level.
4.
VISUALIZATION OF CONDITIONAL PROBABILITY TABLES
We believe our most significant contribution is the visualization of conditional probability tables. Currently there are two common methods for embedding conditional probability tables into the graph, both with limitations.
The first method uses the formal mathematical descriptions of conditional probability and puts these descriptions either beside or under the node they belong to. There are some problems related to this implementation, the most obvious one being the size of the descriptions, making it difficult to put all the descriptions into the graph when the variables have multiple values. Even with a modest number of nodes and a small number of values, these visualizations quickly become crowded.
The second common method for putting conditional probability into the graph is the use of separate probability tables. These views vary from one table representing the conditional probabilities for each node to one giant table, a compact representation of joint probability distributions via conditional independence lying on the side of the graph which shows the conditional probabilities for all nodes.
Besides the size issue for the approaches given above, neither provides an easy nor effective way to quickly perceive conditional relationships between two variables.
In our method, we use colormaps (or heatmaps) to visualize the conditional probabilities. Its character eliminates the major limitations described above. The colormap, a common visualization, popular in gene expression and microarray data analysis, is an alternative to table visualization. Instead of displaying the table value directly, it uses an icon, glyph or color to visually represent cell values.
One of the important advantages of colormaps, and the main reason we use colormaps instead of tabular or mathematical descriptions, is the efficiency of visual quantitative comparisons. A colormap view of conditional probability tables quickly provides a user with the overall context, including all other correlations not currently visible in the usual table view or with mathematical descriptions. Size clearly is another benefit; the size for each colormap table cell being smaller than the corresponding table cells with numeric values. The difference is even more pronounced when colormap displays are compared to mathematical descriptions.
Figure 5 show an example of correlations between parent and child nodes from strongly negative on the left side to strongly positive on the right side.
Figure 5. Colormap visualization of a CPT
5.
IMPLEMENTATION
Our Bayesian network visualization package includes two major components; one is a Bayesian model learning toolbox which implements Bayesian network learning algorithms, and the other is our visualization tool, BayesViz, specifically targeted for Bayesian networks. It is written in Java and based on the Universal Visualization Platform (UVP) 19. UVP is a general-purpose platform for building numerous visualization and analysis applications. It is composed of a central framework and a large number of plug-in tools allowing us to focus on the design of the visualizations and thus results in a rapid implementation of experimental tools.
Figure 6 shows the visualization process. The Bayesian model learning toolbox constructs the Bayesian network structure and computes the conditional probability tables based on the learning algorithms the user selects. The learned causality relationships and CPT information are passed to the visualization tool. A user can interact with the visualization tool for more detailed information about the model with, for example, a probing panel showing the corresponding numeric values when the user hovers the mouse over a colormap.
6.
EXAMPLE
Figures 7 and 8 show BayesViz’s visualization of a model generated using random hill-climbing on 50 genes from the yeast dataset presented by Spellman et al20. Figure 5 presents the inferred network with edges colored by correlation coefficient (green indicates a negative correlation coefficient and red a positive coefficient) and colormap tables representing the conditional relationships between the values of parent and child nodes. A strong positive correlation between a parent and a child may be recognized by a colormap with strong yellow lower-left to upper-right banding, and strong negative correlations between a parent and a child is signaled by strong yellow upper-left to lower-right banding. For example, the colormap for the edge from parent YNL031C to child YBR010W (A in Figure 7) indicates a strong positive correlation. With this visualization method, positive correlations may be quickly identified by looking for red edges and negative correlations by green edges. The quality of the correlations is identified by viewing the banding in the colormaps. This view thus provides for the quick identification of the types and strengths of the dependencies between parent and child nodes. This type of analysis is useful to a scientist in identifying relationships for more detailed analysis or experimentation and helps in suggesting new hypotheses to be tested.
The automatically generated layout of the network can be adjusted by dragging nodes so that any undesired choices made by the layout algorithm can be easily overcome and the sizes and color schemes of the conditional probability tables can be adjusted (B in Figures 7). The color and size of the nodes and edges can be made to depend on the network properties selected by the user allowing such properties to be easily explored.
In Figure 8, we see the colormaps for the conditional probability tables for the nodes. Each row indicates a combination of parent conditions, and the rows are sorted by values of the “first parent” (selected arbitrarily), and then the values of the second parent, and so forth. The changes in the values for the first parent are indicated by a small gap in the colormaps. This allows a quicker recognition of how changes in one parent affect the conditional dependencies between the remaining parent’s values and the child’s values. For instance, the colormap for the node YLR049C, shown in detail with the associated probabilities in C in Figure 8, shows that the conditional relationship between the second parent and the YLR049C varies considerably with different values for the first parent.
7.
CONCLUSION
The ability to analyze a model inferred from data by a learning system is important. Some portions of the model may be incorrect or irrelevant, and interactive visualization provides a powerful tool supporting this analysis. We have presented an integrated Bayesian Network analysis and interactive visualization that uses layering, color, and colormaps. We’ve used this implementation in a microarray gene expression analysis activity and suggest that these methods allow for the strength and quality of conditional relationships to be quickly identified and analyzed.
REFERENCES
1. A. T. Pang, C. M. Wittenbrink, S. K. Lodha, (1996). Approaches to Uncertainty Visualization, Technical Report, UCSC-CRL-96-21 University of California, Santa Cruz.
2. J. Pearl, (1997). Graphical Models for Probabilistic and Causal Reasoning in The Computer Science and Engineering Handbook, A. Tucker, Editor. CRC Press: Boca Raton, FL. p. 699-711.
3. S. K. Card, J. D. Mackinlay, B. Shneiderman, (1996). Readings in Information Visualization: Using Vision to Think. Morgan Kaufmann.
4. K. Thearling, B. Becker, D. DeCoste, B. Mawby, M. Pilote, D. Sommerfield, (2002). Visualizing Data Mining Models, in Information Visualization in Data Mining and Knowledge Discovery, Georges G. Grinstein, Usama Fayyad and Andreas Wierse, Editor. Mogran Kaufmann: San Francisco, CA.
5. N. Friedman, I. Nachman, M. Linial, D. Pe'er, (2000). Using Bayesian Networks to Analyze Expression Data. Journal of Computational Biology. 7(3-4): p.601-620.
6. K. P. Murphy, (2001). The Bayes Net Toolbox for Matlab in the 33rd Symposium on the Interface. Costa Mesa, California.
7. Hugin Expert, Infosys Technologies Limited. http://www.hugin.com
8. M. Nijman, E. Akay, W. Wiegerinck, SNN Nijmegen, BayesBuilder. http://www.snn.ru.nl/nijmegen 9. Netica, Norsys Software Corp. http://www.norsys.com
10. GeNIE/SMILE, Decision Systems Laboratory, University of Pittsburgh. http://genie.sis.pitt.edu
11. H. Wang, M. J. Druzdzel, (2000). User Interface Tools for Navigation in Conditional Probability Tables and Elicitation of Probabilities in Bayesian Networks. In Proceedings of the Sixteenth Annual Conference on Uncertainty in Artificial Intelligence (UAI-2000). San Francisco, CA, USA.
12. N. Elmqvist, P. Tsigas, (2003). Causality Visualization Using Animated Growing Polygons in IEEE Symposium on Information Visualization. Seattle, Washington, USA.
13. Juan-Diego Zapata-Rivera, E.N., Jim E. Greer, (1999). Visualization of Bayesian Belief Networks in IEEE Visualization’99.
14. D. Heckerman, D. M. Chickering, C. Meek, R. Rounthwaite, C. Kadie, (2000). Dependency Networks for Inference, Collaborative Filtering, and Data Visualization. Journal of Machine Learning Research, 1: p. 49-75.
15. K. Marriott, P. Moulder, L. Hope, C. Twardy, (2005). Layout of Bayesian Networks in the 28th Australian Computer Science Conference. The University of Newcastle, Australia: Estivill-Castro, Ed.
16. K. Sugiyama, S. Tagawa, M. Toda, (1981). Methods for Visual Understanding of Hierarchical System Structures. IEEE TRANSACTIONS on Systems, Man, and Cybernetics, 11: p. 109-125.
17. G. D. Battista, P. Eades, R. Tamassia, I. G. Tollis, (1999). Graph Drawing, Algorithms for The Visualization of Graphs. Prentice Hall.
18. E. R. Gansner, E. Koutsofios, S. C. North, K. Vo, (1993). A Technique for Drawing Directed Graphs. IEEE Transactions on Software Engineering, 19(3): p. 214-230.
19. A. G. Gee, H. Li, M. Yu, M. B. Smrtic, U. Cvek, H. Goodell, V. Gupta, C. Lawrence, J. Zhou, C. Chiang, G. G. Grinstein, (2005). Universal visualization platform. In SPIE Visualization and Data Analysis. San Diego, California. 20. P. T. Spellman, G. Sherlock, M. Q. Zhang, V. R. Iyer, K. Anders, M. B. Eisen, P. O. Brown, D. Botstein, and B. Futcher, (1998). Comprehensive Identification of Cell Cycle-regulated Genes of the Yeast Saccharomyces cerevisiae by Microarray Hybridization. Molecular Biology of the Cell, 9: p. 3237-3297.
