Well so is torture

I’m not sure if I understand right, but I think that is the default behavior of apt to install the version available via repository over externally installed packages even if it is a version downgrad.

Long time since I used Ubuntu or other Deb based Linux distributions, but you can pin packages at a specific version with apt. Quick search says apt-mark hold package-name should do the trick. Maybe consult the man page of apt-mark or search a little more by yourself. this link suggests using dpkg for pinning the package version.