As with all jewelry questions - it's complicated.
Most advice will tell you it comes down to your own skin tone, and whether you are a warm, neutral, or cool. And contrast is the key ingredient. Silver, when it's not reflecting tons of light (as jewelry likes to do), is grayish and cool toned. It tends to stand out and look "better" on warm tones. Cool tones, you guessed it, lack that warm hue - and gold stands out. Neutral contrasts with all metal types, and so works "better".
Dissatisfied with what you have been given? The tone of our hands is usually different than the tone of our face and different still from our decolletage. You know how you're not supposed to test foundation on your hand? Well, you can use the same logic. Simply hold your hand up to your face (back of hand facing forward) you should be able to see the difference.
For more objective comparison, here's a handy visual with some people you are probably used to seeing:
I say "better" in quotes because - while this advice does follow color theory - there are plenty of other reasons for jewelry decisons. Also, with different karats of gold, there are different alloys. So, 10k could be a pale 10k and would contrast with warm skin more than pure 24k gold.
Most importantly, make your jewelry decisions for yourself, your preferences and not other people, and wear what you want.