There are always colorblind people among the audience and readers. There should be more than ten colorblinds in a room with 250 people (50% male and 50% female). There is a good chance that the paper you submit may go to colorblind reviewers. Supposing that your paper will be reviewed by three white males (which is not unlikely considering the current population in science), the probability that at least one of them is colorblind is whopping 22%!
Okabe, Masataka and Kei Ito. University of Tokyo (2002). Articles>Accessibility>Graphic Design>Color