TY - JOUR
T1 - Benchmarks for evaluating optimization algorithms and benchmarking MATLAB derivative-free optimizers for practitioners' rapid access
AU - Li, Lin
AU - Saldivar, Alfredo Alan Flores
AU - Bai, Yun
AU - Chen, Yi
AU - Liu, Qunfeng
AU - Li, Yun
PY - 2019/6/14
Y1 - 2019/6/14
N2 - MATLAB ® has built in five derivative-free optimizers (DFOs), including two direct search algorithms (simplex search, pattern search) and three heuristic algorithms (simulated annealing, particle swarm optimization, and genetic algorithm), plus a few in the official user repository, such as Powell’s conjugate (PC) direct search recommended by MathWorks ® . To help a practicing engineer or scientist to choose a MATLAB DFO most suitable for their application at hand, this paper presents a set of five benchmarking criteria for optimization algorithms and then uses four widely adopted benchmark problems to evaluate the DFOs systematically. Comprehensive tests recommend that the PC be most suitable for a unimodal or relatively simple problem, whilst the genetic algorithm (with elitism in MATLAB, GAe) for a relatively complex, multimodal or unknown problem. This paper also provides an amalgamated scoring system and a decision tree for specific objectives, in addition to recommending the GAe for optimizing structures and categories as well as for offline global search together with PC for local parameter tuning or online adaptation. To verify these recommendations, all the six DFOs are further tested in a case study optimizing a popular nonlinear filter. The results corroborate the benchmarking results. It is expected that the benchmarking system would help select optimizers for practical applications.
AB - MATLAB ® has built in five derivative-free optimizers (DFOs), including two direct search algorithms (simplex search, pattern search) and three heuristic algorithms (simulated annealing, particle swarm optimization, and genetic algorithm), plus a few in the official user repository, such as Powell’s conjugate (PC) direct search recommended by MathWorks ® . To help a practicing engineer or scientist to choose a MATLAB DFO most suitable for their application at hand, this paper presents a set of five benchmarking criteria for optimization algorithms and then uses four widely adopted benchmark problems to evaluate the DFOs systematically. Comprehensive tests recommend that the PC be most suitable for a unimodal or relatively simple problem, whilst the genetic algorithm (with elitism in MATLAB, GAe) for a relatively complex, multimodal or unknown problem. This paper also provides an amalgamated scoring system and a decision tree for specific objectives, in addition to recommending the GAe for optimizing structures and categories as well as for offline global search together with PC for local parameter tuning or online adaptation. To verify these recommendations, all the six DFOs are further tested in a case study optimizing a popular nonlinear filter. The results corroborate the benchmarking results. It is expected that the benchmarking system would help select optimizers for practical applications.
KW - optimization methods
KW - heuristic algorithms
KW - evolutionary computation
KW - benchmark testing
KW - particle filters
U2 - 10.1109/ACCESS.2019.2923092
DO - 10.1109/ACCESS.2019.2923092
M3 - Article
SN - 2169-3536
VL - 7
SP - 79657
EP - 79670
JO - IEEE Access
JF - IEEE Access
ER -