Magnets will wreak havoc on a computer hard drive or tape media, but I believe the SD card is encoded in such a way that a magnet will not mess with it....... at least not as easily......It is not magnetically encoded media.
A little testing may have to be done to prove it one way or the other.