Using finite differences to solve an equations is quick and easy, especially for ODEs. On the boundaries it is not possible to use the same scheme as in the bulk and hence the boundaries need to be treated separately. I have read several times that, at the boundary, one can use a lower order finite difference approximation. However, I could never find a rigorous source and it might have been for a specific equations, so here a quick try.
Let’s use the Poisson equation
The finite difference scheme inside the domain is 4th order and the only thing changed is the order of the finite difference scheme at the boundary

Norm of the Residue for different grid spacing. The order of the finite difference scheme at the boundary determines the order of the whole finite difference scheme.
Using a lower order finite difference approximation order at the boundary lowers the overall order of the scheme. While this is not a full treatment of the topic it shows that in general, using lower order at the boundary will affect the overall order of your system.