Both words are typically pronounced with a /p/ sound, but only "dumpster" has a /p/ sound in the word it was derived from, which is obviously "dump".
English borrowed the word "hamster" from German, where it used to be "hamastra". The fact that the /m/ and the /s/ weren't touching back then is important, because the existence of the /p/ in the first place is a result of them touching after the vowel between them was deleted. The sound /m/ is voiced (your vocal cords vibrate while you make it) and bilabial (your lips come together to make it), while the sound /s/ is voiceless (your vocal cords don't vibrate while you make it) and coronal (your tongue touches the front part of your mouth to make it). Saying these two sounds next to each other requires complicated changes in the positioning of parts of your mouth, so /p/ serves as a bridge sound between them, because it maintains the bilabial shape of /p/ but is voiceless like the /s/ following it. You can absolutely say "hamster" without a /p/ sound if you really try, and it probably was when "hamastra" first became something like "hamstra". However most English speaking people nowadays do say it with a /p/ because it is easier to make a relatively slow transition through /mps/ than it is to go straight from /m/ to /s/.
This applies to the transition between other voiced and voiceless sounds in English as well. A lot of people say "prince" and "prints" both with a /t/, "months" often has a dental /t/ sound before the /θ/ (spelled <th> in English), and "strengths" often has a /k/ before the /θ/.
TL;DR: The /p/ sound in dumpster was always there and the /p/ in hamster wasn't and still isn't for everyone. It exists because it's physiologically a little easier to say "hampster" than "hamster".
Actual explanation incoming:
Both words are typically pronounced with a /p/ sound, but only "dumpster" has a /p/ sound in the word it was derived from, which is obviously "dump".
English borrowed the word "hamster" from German, where it used to be "hamastra". The fact that the /m/ and the /s/ weren't touching back then is important, because the existence of the /p/ in the first place is a result of them touching after the vowel between them was deleted. The sound /m/ is voiced (your vocal cords vibrate while you make it) and bilabial (your lips come together to make it), while the sound /s/ is voiceless (your vocal cords don't vibrate while you make it) and coronal (your tongue touches the front part of your mouth to make it). Saying these two sounds next to each other requires complicated changes in the positioning of parts of your mouth, so /p/ serves as a bridge sound between them, because it maintains the bilabial shape of /p/ but is voiceless like the /s/ following it. You can absolutely say "hamster" without a /p/ sound if you really try, and it probably was when "hamastra" first became something like "hamstra". However most English speaking people nowadays do say it with a /p/ because it is easier to make a relatively slow transition through /mps/ than it is to go straight from /m/ to /s/.
This applies to the transition between other voiced and voiceless sounds in English as well. A lot of people say "prince" and "prints" both with a /t/, "months" often has a dental /t/ sound before the /θ/ (spelled <th> in English), and "strengths" often has a /k/ before the /θ/.
TL;DR: The /p/ sound in dumpster was always there and the /p/ in hamster wasn't and still isn't for everyone. It exists because it's physiologically a little easier to say "hampster" than "hamster".