This is kind of the "fashionable" thing to complain about today. Much like people believing that everyone runs either Gnome, (Unity,) KDE, XFCE or LXDE and that no other desktops matter.

First of all I would say that GUI frontends aren't really a part of the package manager, hence I think those don't really matter in this discussion, and I think it is a bit unfair to count those alongside package managers.

The problem with saying that there are too many package managers is that it is like saying there are too many Linux distros.

Because of file system layout differences, differences in init scripts, differences in how current the distros want to stay, differences in if the want to do source or binary packages, differences in if they think it is the role of the package manager to handle dependencies, differences in if they want to do rolling release or separate release versions... and so on every distro would still need to have it's own set of repositories which would negate a lot of the benefits of having a single package manager since packages would still not be cross-compatible.

This means that as long as there is distro choice it seems impossible to ever make packages fully cross-compatible to the point where a user could install a package on any distro. A lot of packages would work if you put in a lot of user-interaction for where things are on your computer and you do packaging like sta.li where everything is statically linked so there are no dependencies... but it wouldn't work anyway if the package relies on a certain version of the kernel, X11 or a driver etc as statically linking those wouldn't really make sense.

Since there is no one way to structure a Linux filesystem, a Linux init system or a Linux release schedule you will never really fix that issue if you do not want to cut Linux down to one single distro.