But you forget that a huge majority of the humans that once lived were born in the last century.

We were less than 0.2 billions in 200 AC, one billion in 1804, two in 1927 and we are 8.3 billions now.

For most of the 8 millions of years walking on this planet we were like "a couple thousands of people" or so, just a blip in the entire history. The impact of this long dead people on statistics comprising our whole species would be negligible, or really low at best.