If it has better value than current state then accept

Info icon This preview shows pages 22–28. Sign up to view the full content.

View Full Document Right Arrow Icon
If it has better value than current state, then “accept the transition,”  that is, use successor state as current state; Otherwise, do not give up, but instead flip a coin and accept the  transition with a given probability (that is lower as the successor is  worse). So we accept to sometimes “un-optimize” the value function a little  with a non-zero probability.
Image of page 22

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
23 Boltzmann’s statistical theory of gases In the statistical theory of gases, the gas is described not by a  deterministic dynamics, but rather by the probability that it will be in  different states.   The 19th century physicist  Ludwig Boltzmann  developed a theory  that included a probability distribution of temperature (i.e.,  every  small region of the gas had the same kinetic energy).   Hinton, Sejnowski and Ackley’s idea was that this distribution might  also be used to describe neural interactions, where low temperature   T  is replaced by a small noise term  T (the neural analog of random  thermal motion of molecules). While their results primarily concern  optimization using neural networks, the idea is more general.
Image of page 23
24 Boltzmann distribution At thermal equilibrium at temperature T, the  Boltzmann distribution  gives the relative  probability that the system will occupy state A vs.  state B as: where E(A) and E(B) are the energies associated with  states A and B. ) / ) ( exp( ) / ) ( exp( ) ( ) ( exp ) ( ) ( T A E T B E T B E A E B P A P = - - =
Image of page 24

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
25 Simulated annealing Kirkpatrick et al. 1983: Simulated annealing  is a general method for making likely the  escape from local minima by allowing jumps to higher energy states. The analogy here is with the process of  annealing used by a  craftsman in forging a sword from an alloy . He heats the metal, then slowly cools it as he hammers the blade  into shape.   If he cools the blade too quickly the metal will form patches of different  composition; If the metal is cooled slowly while it is shaped, the constituent metals will  form a uniform alloy.
Image of page 25
26 Simulated annealing in practice - set T - optimize for given T - lower T (see Geman & Geman, 1984) - repeat
Image of page 26

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
27 Simulated annealing in practice - set T - optimize for given T - lower T (see Geman & Geman, 1984) - repeat Geman & Geman (1984):  if T is lowered sufficiently slowly (with  respect to the number of iterations used to optimize at a given T),  simulated annealing is guaranteed to find the global minimum.
Image of page 27
Image of page 28
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern