A "Connection" handles all interaction between two nodes. No argument there.

A "ConnectionManager" handles all the housekeeping to do with the connections - whether there's a limit on connections, whether there's external API that influences behaviour, whether there's resource allocation to split between them and the policy for that, etc. etc. Basically all the things that a more holistic view of the system demands.

There's nothing wrong with a Manager-type class, as long as its task is management of that type. It shouldn't conflate the connection functionality as its own, I agree with that much, but there are two very distinct concepts being modelled here, by different classes.