[BANANA] LA/OPT Seminar: Nov 15, 2006

[Gene H Golub]

Linear Algebra and Optimization Seminar
Fall 2006
Gates Building, Room 104
David Wolpert
Wednesday, November 15, 4:15pm
Probability collectives and Supervised Learning

There are two major fields that analyze distributed systems: statistical 
physics and game theory. These fields can be re-expressed in a way that 
makes them mathematically identical. Doing so allows us to combine 
techniques from them, producing a hybrid formalism. That hybrid is called 
Probability Collectives (PC).

As borne out by numerous experiments, PC is particularly well-suited to 
black-box optimization and associated problems in distributed control. The 
core idea is that rather than directly optimize a variable of interest x, 
often it is preferable to optimize an associated probability distribution, 
P(x). That optimization can be done either via Monte Carlo Optimization 
(MCO) or, under certain circumstances, in closed form.

Recently is was realized that one can map MCO into a supervised machine 
learning problem. This means that all the powerful techniques of 
supervised learning can be used to improve MCO. As a special case, those 
techniques can be used to improve the optimization of P(x) in a PC-based 
optimizer. In this way the techniques of supervised learning can be 
leveraged to improve black-box optimization and distributed control.

In this talk I review PC. I also illustrate the identity between MCO and 
supervised learning using PC. In particular, I present results showing how 
cross-validation can be used to adaptively set an annealing schedule for 
the optimization of P(x), and to adaptively modify the complexity of P(x). 
I also illustrate the benefit of bagging in PC.


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
I AM HAVING TROUBLE WITH MY MAILER.
PLEASE SEND ALL FUTURE MAIL TO ME AT
golub at stanford.edu.
Thanks. Gene



Gene Golub,  Fletcher Jones Professor of Computer Science
Gates 2B
Computer Science Dept
Stanford University
Stanford, CA 94305
USA

Office Phone: 650 723 3124
Home Phone: 650 323 0105
FAX:  (650) 618 2767
Mobile: 650 796 5402


-++**==--++**==--++**==--++**==--++**==--++**==--++**==
This message was posted through the Stanford campus mailing list
server.  If you wish to unsubscribe from this mailing list, send the
message body of "unsubscribe linear_algebra_optimization" to majordomo at lists.stanford.edu