To increase energy efficiency, heat pump dryers and membrane dryers have been proposed to replace conventional fossil fuel dryers. Both conventional and heat pump dryers require substantial energy for condensing and reheating, while "active" membrane systems require vacuum pumps that are insufficiently developed. Lower temperature dehumidification systems make efficient use of membrane energy recovery ventilators (MERVs) that do not need vacuum pumps, but their high heat losses and lack of vapor selectivity have prevented their use in industrial drying. In this work, we propose an insulating membrane energy recovery ventilator for moisture removal from drying exhaust air, thereby reducing sensible heat loss from the dehumidification process and reheating energy. The second law analysis of the proposed system is carried out and compared with a baseline convective heat pump dryer. Irreversibilities in each component under different ambient temperatures (5-35 degrees C) and relative humidity (5-95%) are identified. At an ambient temperature of 35 degrees C, the proposed system substantially reduces sensible heat loss (47-60%) in the dehumidification process, resulting in a large reduction in condenser load (45-50%) compared to the baseline system. The evaporator in the proposed system accounts for up to 59% less irreversibility than the baseline system. A maximum of 24.5% reduction in overall exergy input is also observed. The highest exergy efficiency of 10.2% is obtained at an ambient condition of 35 degrees C and 5% relative humidity, which is more than twice the efficiency of the baseline system under the same operating condition.