This thesis is concerned with the linear-quadratic optimal control and model order reduction (MOR) of large-scale linear time-varying (LTV) control systems. In the first two parts, particular attention is paid to a tracking-type finite-time optimal control problem with application to an inverse heat conduction problem and the balanced truncation (BT) MOR method for LTV systems. In both fields of application the efficient solution of differential matrix equations (DMEs) is of major importance. The third and largest part deals with the application of implicit time integration methods to these matrix-valued ordinary differential equations. In this context, in particular, the rather new class of peer methods is introduced. Further, for the efficient solution of large-scale DMEs, in practice low-rank solution strategies are inevitable. Here, low-rank time integrators, based on a symmetric indefinte factored representation of the right hand sides and the solution approximations of the DMEs, are presented. In contrast to the classical low-rank Cholesky-type factorization, this avoids complex arithmetic and tricky implementations and algorithms. Both low-rank approaches are compared for numerous implicit time integration methods.