Predictive Uncertainty Estimators for Sum-Product Networks

Mar 1, 2020

Funding Agency

CNPq grant 304012/2019-0 (ended)

Coordinator

Denis Deratani Mauá

Participants

  • Denis Deratani Mauá
  • Julissa G. Villanueva, PhD candidate at IME-USP
  • Renato Lui Geh, MSc student at IME-USP
  • Decio L. Soares, MSc student at IME-USP
  • Jonas Gonçalves, Undergraduate student at IME-USP

Abstract

Deep models have recently obtained impressive results in a wide range of tasks and domains, from computer vision to image and text processing to diagnosis and automated reasoning. Sum-product networks are special type of neural networks targeted at the representation of high-dimensionality probability distributions. Notably, the structure of a sum-product network (i.e., its graph) encodes a number of probabilistic independence assertions. In addition, each node of the network computes a valid distribution over its part of the variables. Such probabilistic semantics facilitates debugging, allows a wide range of queries and data (including missing and noisy data) to be handled properly and efficiently, and enables efficient structure learning algorithms.

In many applications, it is desirable to have not only a prediction (e.g., the probability of observing a certain object, say an image), but a confidence measure about its prediction. As sum-product networks are learned from data, the probabilities they calculate can be overly confident or too sensitive to hyperparameters on regions where data was scarce, conflicting or too noisy. In this research project, we plan on following two possible approaches to estimating the uncertainty of a sum-product network prediction. The first approach estimates uncertainty by analyzing the effect that small perturbations of the data have on the prediction. The second approach considers the variability of predictions among different networks (learned from the same data).

This research investigates predictive uncertainty estimators for sum-product networks.

Productivity Scholarhip Sum-Product Networks Tractable Probabilistic Models
Denis D. Mauá
Denis D. Mauá
Associate Professor

I research computational aspects of probabilistic reasoning.

Related

Publications

Tractable Classification with Non-Ignorable Missing Data Using Generative Random Forests
Tractable Mode-Finding in Sum-Product Networks with Gaussian Leaves
Cautious Classification with Data Missing Not at Random using Generative Random Forests
Missing data present a challenge for most machine learning approaches. When a generative probabilistic model of the data is available, an effective approach is to marginalize missing values out. Probabilistic circuits are expressive generative models that allow for efficient exact inference. However, data is often missing not at random, and marginalization can lead to overconfident and wrong conclusions. In this work, we develop an efficient algorithm for assessing the robustness of classifications made by probabilistic circuits to imputations of the non-ignorable portion of missing data at prediction time. We show that our algorithm is exact when the model satisfies certain constraints, which is the case for the recent proposed Generative Random Forests, that equip Random Forest Classifiers with a full probabilistic model of the data. We also show how to extend our approach to handle non-ignorable missing data at training time.
Fast And Accurate Learning of Probabilistic Circuits by Random Projections
Learning Probabilistic Sentential Decision Diagrams under Logic Constraints by Sampling and Averaging
Probabilistic Sentential Decision Diagrams (PSDDs) are effective tools for combining uncertain knowledge in the form of (learned) probabilities and certain knowledge in the form of logical constraints. Despite some promising recent advances in the topic, very little attention has been given to the problem of effectively learning PSDDs from data and logical constraints in large domains. In this paper, we show that a simple strategy of sampling and averaging PSDDs leads to state-of-the-art performance in many tasks. We overcome some of the issues with previous methods by employing a top-down generation of circuits from a logic formula represented as a BDD. We discuss how to locally grow the circuit while achieving a good trade-off between complexity and goodness-of-fit of the resulting model. Generalization error is further decreased by aggregating sampled circuits through an ensemble of models. Experiments with various domains show that the approach efficiently learns good models even in very low data regimes, while remaining competitive for large sample sizes.
Efficient Algorithms for Robustness Analysis of Maximum A Posteriori Inference in Selective Sum-Product Networks