Abstract--In this paper, a new iterative method is proposed to solve the generalized Hamilton-Jacobi-Bellman （GHJB） equation through successively approximate it. Firstly, the GHJB equation is converted to an algebraic equation with the vector norm, which is essentially a set of simultaneous nonlinear equations in the case of dynamic systems. Then, the proposed algorithm solves GHJB equation numerically for points near the origin by considering the linearization of the non-linear equations under a good initial control guess. Finally, the procedure is proved to converge to the optimal stabilizing solution with respect to the iteration variable. In addition, it is shown that the result is a closed-loop control based on this iterative approach. Illustrative examples show that the update control laws will converge to optimal control for nonlinear systems. Index Terms--Generalized Hamilton-Jacobi-Bellman （HJB） equation, iterative method, nonlinear dynamic system, optimal control.
