Wherever you have windows, you often find curtains, and the door curtain is no exception. If there is a window in your door, you probably have a door curtain to cover it.
Heat is just another type of light, so a door window often lets as much heat out as it lets in light. A curtain provides a barrier, keeping the warmth where you need it. Insulate the entire house, but do not just block draughts.