Try to use antiseptic mouth wash to rinse mouth. They hurt much less after the rinse. After about 3,4 hours later, you may feel they come back again,then do the rinse again.

This problem is caused by bacteria.I found this method helps a lot. Usually they began to heal the second day, for me. Any brand of antiseptic mouth wash, I used a pretty cheap brand.

Taking Vb should help the long way.

Good luck.