Turn off the gas to the stove. While you don't need to turn on the burner to clean the ignitor, it's better to be safe than sorry.
Remove the burners from the stove or -- if you have a different model -- lift up the cooktop to access the ignitors.
Dip the tip of your brush into rubbing alcohol, and gently shake the brush to remove any excess liquid.
Place the head of the brush on the ignitor.
Gently scrub the ignitor with the brush. Make sure to get underneath the ignitor too. The rubbing alcohol should help loosen the dirt. Once the ignitor is clean, let it dry completely before replacing the stovetop and burners.