A robust combined trust region–line search exact penalty projected structured scheme for constrained nonlinear least squares

We present a combined trust region–line search structured algorithm for solving constrained nonlinear least-squares problems. The approach is based on adaptive structured scheme due to Mahdavi-Amiri and Bartels of the exact penalty method of Coleman and Conn for nonlinearly constrained optimization problems. The structured adaptation also makes use of the ideas of Nocedal and Overton for handling quasi-Newton updates of projected Hessians. For robustness, we propose a new penalty parameter updating strategy and a specific line search strategy within the trust region, taking account of the least-squares objective and special structured considerations for the approximate projected least-squares Hessians. Our added usage of a second-order correction step in the global phase, combined trust region–line search strategy, and, of course, the new adaptive penalty parameter updating strategy helps in speeding up the global iterations and reaching the asymptotic phase confidently. We discuss the details of our implementation and provide comparative results of the testing of our programs and three nonlinear programing codes from KNITRO on test problems (both small and large residual) from Hock and Schittkowski, Lukšan and Vlček, Biegler et al. and some randomly generated ones due to Bartels and Mahdavi-Amiri. The numerical results obtained by our approach showed a clear local two-step superlinear convergence rate, and as compared to the results obtained by Mahdavi-Amiri and Bartels, the best methods tested in the collection of Hock and Schittkowski, three methods in KNITRO, a recent nonlinear least-squares method of Bidabadi and Mahdavi-Amiri and two algorithms provided by fmincon in MATLAB, would indeed confirm the practical significance of our adaptive penalty update scheme, combined trust region–line search strategy, and special structured considerations for the approximate projected least-squares Hessians.