White holes are basically black holes in reverse: They eject all the matter sucked into a black hole. We've never observed one, and most astrophysicists suspect they don't exist - all because they're not the exact opposite of black holes.
White holes are, on a theoretical level, simply a recognition of the fact that, if time flows in one direction, physics says it can just as easily flow in the other direction. What does that mean? Well, a black hole doesn't just describe an object in space - it also describes a progression in time towards a singularity in the future. If a black hole has a singularity in its future, then physics says there should also be an object with a singularity in its past, and that's a white hole. (For a more detailed backgrounder on white holes, check out this post from a couple months back.)