Goosebumps on skin that won't go away?

Last Friday, I got goosebumps on my face and arms out of nowhere. I have dry, sensitive skin and trying my best to take care of it by exfoliating 2x a week, using jojoba oil for skin moisturizing and witch hazel soap for face and body and now these goosebumps formed. has anyone ever experience this problem and know the cause of it? Will it ever go away?