Did this ever happened to you that you got a voicemail, and you know who sent it and you don't want to hear the voicemail, so you just tried to clear it but it never go away?
Well, that happened to me so many times and that notification icon made me pissed off. But I've found the solution so I thought why not share with other people aswel.
Hold that icon and slide it down.
Click and hold for a second on the "New voicemail" tab as you see in the picture above.
When you click on the "New voicemail" tab you will see a new tab called "App info" like the picture below, and click on the "App info".
Now click on "Clear data" as you can see in the picture below.
Now you should be able to see pop up window saying "All of this application's data will be deleted permanently. This includes all files, settings, accounts, databases etc." click ok.