I agree with Tony, it sounds like thrips. I know several people here in Florida that had a severe go-round with chilli thrips recently. I use a couple of systemic insecticides to keep thrips at bay.