Genetic Annealing (option METHOD/PMETHOD=GA) is a cross between the genetic algorithm and simulated annealing. It’s mainly the genetic algorithm (thus tracks a population of parameter sets) which are mutated at each point, but, instead of only picking the “fittest” (that is, replacing one parameter set only if the mutated one produces a better function value) uses an annealing schedule so it might replace it with one that’s not as good.

 

An “iteration” is a pass through the algorithm at a given temperature. Within each, it does several batches of mutations. Within a single batch of mutations, each member of the population is mutated and tested to see if it should be replaced. After a batch of mutations, the control parameters for the mutations are adjusted to try to keep the percentage of acceptances at around 50%. (The crossover scale is increased if the acceptance rate is below the target and decreased if it’s above it).

 

METHOD=GA (and PMETHOD=GA, which is what is more typically used) are available on BOXJENK, CVMODEL, DLM, FIND, GARCH, MAXIMIZE and NLLS. Each of those instructions has a T0 option to control the initial temperature. For a maximum likelihood function (all of the above but BOXJENK (without the MAXL option) and NLLS), the default is the maximum of 5 and the square root of the number of parameters, which seems to work fine in practice. For non-linear least squares functions, it’s 20% of the function value at the initial guesses. The ITER (for METHOD) and PITER (for PMETHOD) options control the number of “iterations”.

 

The other controls are in the NLPAR instruction. The ones common to simulated annealing are  ASCHEDULE, which controls the rate at which the temperature is lowered per iteration. The default is .85. AOUTER controls the number of times the mutation batches are run per iteration. This is a multiplier on the number of parameters. The default is 3.0—however, a minimum of 25 are done, regardless of the number of parameters.

 

The ones common to the genetic algorithm are POPULATE (multiplier on the number of parameters for the size of the population) and MUTATE (method of mutation). The other two genetic method controls (CROSSOVER and F) are used, but have values which adapt to the situation. (F is changed to try to keep the acceptance rate at 50%).

 

If you have 15 parameters, a single iteration would (with the default settings) take 3.0 x 15 loops over the mutation batch, each of which would have 6.0 (POPULATION default multiplier) x 15 function evaluation. It will take (roughly) the same number of function evaluations per iteration as simulated annealing if the AOUTER times the POPULATION value for genetic annealing is the same as AOUTER times AINNER for simulated annealing. (Simulated annealing has the extra “inner” loop, but that’s over the number of parameters, while genetic annealing loops over the population size, which is considerably larger). In comparison with the regular genetic algorithm, it takes that entire outer loop within an iteration (AOUTER x the number of parameters times through that), so requires much more computation, but then it does a broader scan of the parameter space.