cover of episode Hyperparameter optimisation in differential evolution using Summed Local Difference Strings, a rugged but easily calculated landscape for combinatorial search problems

Hyperparameter optimisation in differential evolution using Summed Local Difference Strings, a rugged but easily calculated landscape for combinatorial search problems

2023/7/12
logo of podcast PaperPlayer biorxiv bioinformatics

PaperPlayer biorxiv bioinformatics

Frequently requested episodes will be transcribed first

Shownotes Transcript

Link to bioRxiv paper: http://biorxiv.org/cgi/content/short/2023.07.11.548503v1?rss=1

Authors: Pannu, H. S., Kell, D. B.

Abstract: We analyse the effectiveness of differential evolution hyperparameters in large-scale search problems, i.e. those with very many variables or vector elements, using a novel objective function that is easily calculated from the vector/string itself. The objective function is simply the sum of the differences between adjacent elements. For both binary and real-valued elements whose smallest and largest values are min and max in a vector of length N, the value of the objective function ranges between 0 and (N-1) x (max-min) and can thus easily be normalised if desired. This provides for a conveniently rugged landscape. Using this we assess how effectively search varies with both the values of fixed hyperparameters for Differential Evolution and the string length. String length, population size and generations for computational iterations have been studied. Finally, a neural network is trained by systematically varying three hyper-parameters, viz population (NP), mutation factor (F) and crossover rate (CR), and two output target variables are collected (a) median and (b) maximum cost function values from 10-trial experiments. This neural system is then tested on an extended range of data points generated by varying the three parameters on a finer scale to predict both median and maximum function costs. The results obtained from the machine learning model have been validated with actual runs using Pearsons coefficient to justify the reliability to motivate the use of machine learning techniques over grid search for hyper-parameter search for numerical optimisation algorithms. The performance has also been compared with SMAC3 and OPTUNA in addition to grid search and random search.

Copy rights belong to original authors. Visit the link for more info

Podcast created by Paper Player, LLC