I find the noise painful. And it's also distracting. When I'm at a noisy reading I can't concentrate and follow the words in the Megillat Scroll of Esther. We're supposed to listen to every word, following the words written on the page.
If you insist that it's important to "stamp out Haman's name," then do it right, only when his name is said. That means that young children, too young to read the text shouldn't have groggers, noisemakers, rattles, drums, hammers etc.
Prepare for the holiday by reading Megillat Esther, especially with the kids. Mark Haman's name on the text. It is a book without the Name of G-d, so there isn't any problem. Take responsibility for your children. If they make noise at the wrong time, take away the noisemaker, and make these rules clear before Purim.