Yes, many species are only fragrant when the pollinator is active. Many white flowers are night fragrant due to moth pollinators. Actually, it is the reverse. Flowers that are pollinated by moths, which are night active, don't need color therefore they tend to be white.

I have a few plants that the fragrance changes at different times of the day...maybe in nature they have more than one pollinator. There are many species without a fragrance though.