The stuff you want OUT needs to have a place to go. Remove the shields and let the bad stuff go away. You should also blow it out as well to get solid stuff out.
Now, since you’re planning to run it dry, it’s your choice to replace the shields or not. I always recommend doing so, but I’ll let you make your own decision regarding that. It won’t make a difference in performance.
You have something bad inside the bearing. In this case, it’s called “lube”.
You want to soak your bearing in acetone to get rid of the lube.
If you remove the shields of the bearing BEFORE soaking it in the acetone, when you shake the bearing in the acetone, it will get into more spaces and places, removing more acetone.
When you remove the bearing from the acetone, nearly all of the lube will come OUT of the bearing, IF it has a place to go easily. If you have the shields in place, it will be harder for the lubricant to come out. If you have the shields removed, it should go away with the draining acetone.
This isn’t really complicated. It’s just proper practice.
You wouldn’t take a bath with your clothes on, right? Why are you cleaning your bearing with the shields on?
Then, use those office dusters to blow out additional stuff, which will also dry the bearing faster as well as remove solid contaminants that the acetone doesn’t break down and remove.