It's not just the jet stream, the rotation of the earth plays a big role, too. If you're flying against it, your trip will take less time than if you were flying in the same direction.