I've changed the logic to only show the warning pop-up when you've got a part input field focused.
I'm happy with this because it turns out that warning messages to do with invalid answers are always repeated in the feedback messages anyway, and those are shown regardless of the exam's feedback settings, so the student doesn't miss out on anything. When you focus an input field, the warning box is shown.
This should mean that only one warning box is shown at a time, and it can be dismissed by moving out of the input box or by pressing Escape.