It's not so simple.
There is a responsability towards the aikido people who still have not the ability to discern what is reliable information and what is bs. If bs remains unadressed, silence means assent.
OTOH, I would not like to see downvoting posts as an alternative to debate. Mob justice, censorship and anonymous bandwagoning are no substitutes for discussion.
Wouldn't want people to be able to block a post by downvoting, but there could be a tally box or someting showing percentage approving and disapproving, maybe.