So, I've seen two work arounds for this:
1: Select phone app, press menu, select Call Settings, then de-select "Auto screen off during callss" which turns of the proximity sensor. Of course it turns it off for all calls, not just voicemail, which could be a bit of a battery drain.
2. If the primary issue is disappearing keypad just when entering your voicemail pin, then you can set up to auto-enter pin when you dial voicemail.
a - In contacts, set up voicemail # as *86,,pin# where pin is your voicemail password.
b - Select phone app, press menu, select Call Settings, then scroll to Voicemail settings, input Voicemail # as *86,,pin# where pin is your voicemail passcode.
Note that if you use the SwiftKey3 keyboard, there won't be a comma...look for a "pause" button instead.