Partner who has kids with exes

Hi guys anyone in the same situation as me? I have a partner I love more than anything and 2 lovely boys of whom I’m proud to say I am their step dad. I want what’s best for them and for my partner but I really struggle with their dads/ her exes turning up and treating my/our house like they can just come in and make themselves at home. It really gets under my skin and they both slag each other off to me and my partner. One of them only comes every other week on a Sunday for about 2 hrs and the other spends a little more time here, comes 2 times a week and spends time with his son and has him at weekend’s. I don’t like him coming so much. Is that bad? I feel like I’m in a relationship with at least 1 other person(mostly because one of the dads hardly sees his son) however.. both of them give the minimum when it comes to child support. I’m the wage earner in our household, i have a good job earning really good money, me and my partner share money so I pay for a lot of stuff. More importantly I father them more than their biological dads. Both of them really [censored] me off, my partner deals with when they come around and is very easy going about it. I feel like I just have to go with it even if it’s not what I want. Anyone feel my pain or am I being a selfish and unreasonable [censored]?