My attache bag is black, my daypack is OD green. I prefer the solid colors myself, but that's just a personal preference.
The ACU pack, with or without a brightly-colored rain cover, will still be attracting some attention. Then again, how much attention depends on where you're going or where you might wind up.
I guess it depends on where you're planning to be with it. In the middle of L.A. or NYC I'd imagine a few people would notice. In a rural area or small town, I doubt anyone would care.