I usually press and release the brakes slowly, so I never noticed it before. Recently, my spouse and her passenger mentioned it because they were in stop and go traffic, constantly pressing and releasing the brake pedal.
Is this a serious issue? I don't notice anything else going on other than the hissing sound when I press and release the brake pedal. Brakes still work, fluid in the master cylinder still just under the max line.

Some searching points to the brake booster, but this seems to happen on both press and release of the brake pedal, not just pressing. Maybe there's some rubber boot or something with a hole that's releasing air?

Sounds like this video: