This work concerns the development of high order accurate finite difference techniques for the discretization of hyperbolic systems that are first order in time and second order in space. Its motivation comes from numerical relativity where many current formulations of the Einstein equations have this form. It pays special attention to the case where some first order derivatives are approximated with non-centered finite difference operators. This case is important because in numerical simulations of black holes using the BSSN formulation, it has been observed that a one-point off-centering of the derivatives from the Lie terms is essential for the accuracy and the lifetime of the simulation. In the particular case of the shifted wave equation -a simple but very powerful tool in numerical relativity -it is shown here analytically that, although off-centered operators have larger truncation errors than the centered ones, there are indeed situations where off-centering is beneficial for accuracy. This system and yet another one- dimensional model (BSSN in spherical symmetry) are also investigated with respect to the numerical treatment of boundaries.