If you're trying to solve a real world problem I think you have to tread very carefully, at the early stages, in the way your formulate it mathematically. Once you've formulated it, it's easy to imagine that this is now the problem you need to solve and forget about the original problem. You can then spend a long time making progress on solving the formulation, but fail to realise that you're actually stuck in a rut and that there's another formulation of the original problem that works much better.

I had the problem of trying to find optimal (in some sense) paths for moving a large number of vehicles from various A's to various B's. It seemed natural to pick one vehicle and then optimise for that. The problem of finding the optimal path could then be written as something like a calculus of variations problem. At this point I was now trapped in a particular formulation. Trying to solve the calculus of variations problem led to an ODE, but instead of initial conditions it had both initial and terminal conditions. You can discretise time and use a standard ODE solving method, but you can't simply step forward through time from the start as you don't know, for any choice of starting path, whether it will end the right way. It turns into a messy search problem. And then I had the prospect of solving this problem reliably for a large number of paths.

Eventually I realised I had to break out of my original formulation. Instead of solving for one entire path at a time it works better to solve for many paths incrementally and simultaneously. If you can solve and tabulate the solutions to the problem of getting from each A to each B over the time period s to t, and you can solve from the short time period from s-ds to s, then you can combine these to solve for the entire table of solutions from s-ds to t. Now it's just recursion. This is classic dynamic programming, which was in fact originally invented to solve this very problem in the case where the vehicles were missiles. It can be incredibly easy compared to solving millions of ODEs with initial and terminal conditions.

I think I basically recapitulated the history of control theory, which is unsurprising as I was getting my problem solving clues from web pages on control theory and colleagues who had some experience with related problems. The former approach is the Pontryagin principle approach [1] and the latter is the Hamilton-Jacobi-Bellman approach [2] (which is similar to reinforcement learning). The former approach is probably a good choice if there's a chance you can solve the equations exactly and get a simple formula. The latter approach seems to work better when the quantity you're optimising is determined by big tables of numbers.

[1]

https://en.wikipedia.org/wiki/Pontryagin%27s_maximum_principle[2]

https://en.wikipedia.org/wiki/Hamilton%E2%80%93Jacobi%E2%80%93Bellman_equation