In general, the complexity ime is at least,
1. It can make builds non-deterministic (meaning potentially unreproducible, which is table stakes for most "release" builds today)
2. It can make Link time code optimization (LTO) hard(er)
3. It requires tracking of changes (to know when to relink), that means either some external system or integration with the compiler
At least 3. introduces a good chunk of complexity. There is also the minor issue that modern linkers are fast enough-ish without incremental linking.