Solving differential equations via artificial neural networks: Findings and failures in a model problem

In this work, we discuss some pitfalls when solving differential equations with neural networks. Due to the highly nonlinear cost functional, local minima might be approximated by which functions may be obtained, that do not solve the problem. The main reason for these failures is a sensitivity on initial guesses for the nonlinear iteration. We apply known algorithms and corresponding implementations, including code snippets, and present an example and counter example for the logistic differential equations. These findings are further substantiated with variations in collocation points and learning rates.