Scattering of acoustic phonons by linear defects is studied in the continuum limit of elasticity theory and the scattering matrix induced by the change in the phase of a phonon that passes round a defect in a closed contour is calculated. It is shown that on the background of a screw dislocation and for negative Frank angle of a disclination phonon modes containing components with kinetic angular momentum $\mu$ satisfying the inequality $0<|\mu|<1$ are singular, namely, near the defect line such components can increase unboundedly as $\rho^\mu$, where $\rho$ is the distance to the line of the defect. In the presence of singular modes, the curvature of the gauge group $G=SO(3)\rhd T(3)$, which is concentrated on the defects, leads to transitions between different polarizations. The topological interaction plays a leading role in the case when the phonon wavelength is much greater than the scattering length corresponding to scattering by the short-range potential of the defect core and, thus, it is most important in problems with long-range correlations.