In this paper we discuss several variants of the acceleration technique known as Iterated Defect Correction (IDeC) for the numerical solution of initial value problems for ODEs. A first approximation, computed by a low order basic method, is iteratively improved to obtain higher order solutions. We propose new versions of the IDeC algorithm with maximal achievable (super-)convergence order twice as high as in the classical setting. Moreover, if the basic numerical method is designed for a special type of ODEs only, as is the case for many geometric integrators, the idea of classical IDeC is not applicable in a straightforward way. Our approach enables the application of the defect correction principle in such cases as well.