Starting from the well-known factorization of linear ordinary differential equations, we define the generalized Loewy decomposition for a ${\cal D}$-module. To this end, for any module $I$, overmodules $J\supseteq I$ are constructed. They subsume the conventional factorization as special cases. Furthermore, the new concept of the module of relative syzygies $Syz(I,J)$ is introduced. The invariance of this module and its solution space w.r.t. the set of generators is shown. We design an algorithm which constructs the Loewy-decomposition for finite-dimensional and some kinds of general ${\cal D}$-modules. These results are applied for solving various second- and third-order linear partial differential equations. Several extensions of this work are indicated.