Automatic Learning of Sum-Product Networks

Undergraduate Research Project (Iniciação Científica)


Back home Student: Renato Lui Geh
Advisor: Denis Deratani Mauá

Abstract

Sum-product networks (SPNs) are probabilistic graphical models that model a tractable probability distribution. Proposed in 2011, SPNs compute exact inference in linear time in the number of edges of its graph if certain criteria are met [1]. SPNs have shown many interesting properties, such as a higher representability of functions the deeper its architecture whilst maintaining its efficiency [2]. Other interesting theoretical properties include a generalization of SPNs to any semiring with disjoint product [3]. With regards to applications, SPNs have shown impressive results in various tasks, such as protein folding [4], modeling speech [5], image classification and completion [1,6,7], activity recognition [8] and modeling natural language [9]. In this undergraduate research project, we implement various state-of-the-art learning algorithms.

Objectives

The objectives of this undergraduate research project is to study Sum-Product Networks and implement state-of-the-art SPN learning algorithms. Namely the following:

The code is free and open-source, and is available as part of the GoSPN inference and learning framework for Sum-Product Networks.

Acknowledgements

This undergraduate research project is financed by CNPq grant #800585/2016-0.

References

  1. Hoifung Poon, Pedro Domingos. "Sum-Product Networks: A New Deep Architecture". 2011. Uncertainty in Artificial Intelligence (UAI 2011). Ed. 27.
  2. Olivier Delalleaum, Yoshua Bengio. "Shallow vs. Deep Sum-Product Networks". 2011. Advances in Neural Information Processing Systems (NIPS 2011). Ed. 24.
  3. Abram L. Friesen, Pedro Domingos. "The Sum-Product Theorem: A Foundation for Learning Tractable Models". 2016. International Conference on Machine Learning (ICML 2016). Ed. 33.
  4. Abram L. Friesen, Pedro Domingos. "Recursive Decomposition for Non-convex Optimization". 2015. International Conference on Artificial Intelligence (IJCAI 2015). Ed. 24.
  5. Robert Peharz, Georg Kapeller, Pejman Mowlaee, Franz Pernkopf. "Modeling Speech with Sum-Product Networks: Application to Bandwidth Extension". IEEE International Conference on Acoustics, Speech, and Signal Processing (ICAASSP 2014). Ed. 39.
  6. Robert Gens, Pedro Domingos. "Learning the Structure of Sum-Product Networks". International Conference on Machine Learning (ICML 2013). Ed. 30.
  7. Aaron Dennis, Dan Ventura. "Learning the Architecture of Sum-Product Networks Using Clustering on Variables". Advances in Neural Information Processing Systems (NIPS 2012). Ed. 25.
  8. Mohamed R. Amer, Sinisa Todorovic. "Sum-Product Networks for Activity Recognition". IEEE Transactions on Pattern Recognition and Machine Intelligence (TPAMI 2015).
  9. Wei-Chen Cheng, Stanley Kok, Hoai Vu Pham, Hai Leong Chieu, Kian Ming Chai. "Language Modelling with Sum-Product Networks". Annual Conference of the Speech Communication Association (INTERSPEECH 2014). Ed. 15.