Use a combination of a brush, a vacuum and/or some compressed air and a cloth to clean the speaker cone. Make sure the metal fillings don't get into anything else on your work surface. Make sure no loose particles remain on the side that interfaces with the gasket.

When re-installing the speaker, align the contacts with their respective holes but don't press them in yet.

Place the speaker into its front panel location and press it in using a rocking downward pressure till seated. Gently apply pressure around the edges to ensure a good seal.

Use your needle nose to press the contacts into their respective holes.