Peter A Goldsbury
I have taught the English language to Japanese over the years and I believe that hearing is just as culture-specific a skill as speaking and reading.
As a gigantic fwiw, I think there is little doubt of this. As a person who has had hearing problems my whole life it is amazing the things I can "mis-hear" in conversation. People living with those with hearing loss have to learn a few things. One is to face the person who is hard of hearing when you speak -- we gather a tremendous amount of information visually when people speak. Another is that there are extremely subtle cues in speech that are critical to "understanding". It isn't enough that the sound is close unless the person is actively trying to figure out what something sounds like. In normal conversation if any of the most subtle of things are off you literally don't "hear" it. So considering the variety of quirks, mannerisms, etc. that are culture/region/whatever specific it should not be surprising that there can be remarkably subtle misunderstanding that render someone unable to "hear" what someone else is saying even if on reflection you realize they are very close in sound.
Those who are hard of hearing have constant problems with misunderstandings. My wife and daughter are both very good at being my "translator" when we're at a restaurant or something like that. The wait staff will ask me something and my wife and/or daughter almost always automatically rephrase it for me if I have any hesitation at all. They'll also correct me if I obviously misunderstood. The critical point here is that I "heard" what I thought I heard. It just isn't always what they said. The combination of facial expression, sounds, body language, everything comes together in the brain to form the "language" we hear. So I literally hear better when I can see someone's face. When I take out my hearing aids or they're in the shop, I have to "relearn" to read faces better. Lacking the improved hearing the high end digital hearing aids give I need to relearn reading faces, context, etc. So I hear a lot better if I know someone and have experience listening to them, especially without my hearing aids in. A new person, a noisy environment, and if I can't see their face clearly... Hearing aids or not I have a heck of time understanding a word they're saying. Or I might think I understood but I'll have often heard it completely wrong. Sometimes comically so.
Recently at a restaurant a waitress asked me if I needed a new fork. I picked up the fork and said "No, I"m fine". She looked puzzled and walked away. My wife told me she actually asked if I wanted a refill on my coke. Fork. Coke. Hmmm, I would have sworn I heard fork. And as a matter of fact I *did* hear fork. It's just not what she said... My brain filled in the rest. Maybe I happened to be holding the fork and wasn't thinking about the drink. I wasn't facing her so I didn't see her face when she spoke. So my brain filled in all the blanks in real time. And came up with something that made sense given context and whatever I happened to be thinking. It just wasn't what she asked...
The understanding of spoken language is a remarkably subtle thing on many levels. Every language (and dialect and region and...) has their own specific quirks and habits. If you're not familiar with them and immersed in them sufficiently you will have problems.