way to unselect one object when selecting several objects with a marquee?

When you draw a marquee around several objects and you want to eliminate one thing from the selection, is there a way to unselect just one object from being selected without having to unselect and reselect everything else?

Thanks,
Ellen