I think the issue lies in the mind.
The buddha once said when a person walks across the grass they can unknowingly kill countless beings, and as to blame, they are innocent. But if you walk across the grass and purposefully step on beings killing them on purpose, then you are to blame. If you walk across the grass unknowingly killing beings you have not committed and evil act. But, if you purposefully kill them, then you have willfully committed an evil act.
I think the only issue is in the mind
One time I was going to the store to get some chocolate after work. A man rode his bike into the center of the road, part of his bike was not all the way in the median. He was looking at the other half of the road to see when he could cross. I was looking at him feeling worried that he would jet out in front of my car. Suddenly a car going over the speed limit hit him because part of his bike was not all the way in the median. I watched as he flew through the air and rag dolled and bike parts were raining down. It was in slow motion I felt like I could get out of the car and catch him. He died instantly. I often felt the guilt in my mind, what part did I have in that mans death because he had to wait for traffic in that spot in order to cross. There is some guilt there. Although I had no way to know a car would be speeding that day and had no way to know that was going to happen. I though that man had no idea that today he was going to die when he woke up. The cross walk was only a short way up. If only he'd used the cross walk. But then again in Arizona people get hit in crosswalks almost every day. This just reminded me of that situation. Although I know it wasn't my fault. I still wish I would have known and could have warned him. Although he was a stranger it didn't feel like it in that moment. As I seen him flying through the air I seen him as someone's son, someone's loved one. It was personal. I know it wasn't my fault but I feel bad none the less.
I guess it is up to each individual person whether it is in them or not to care for another being. If it does not bother them then they will not suffer because of that decision to not step in. If they have a conscience and depending on the situation if it could be helped or not, the person has to rely on their own conscience. I guess I think it is up to each individual and how their mind is set. And, there can be any myriad of conditions that may or may not make it possible for you to step in. There is a lot to calculate and all must be considered.