SUre. It will be fine. Better to remove that stuff so that it doesn't hold water and make a place for pests.

Cheers,
BD