It is case by case, although you can start by going out for very brief separations, like to throw the trash out or get the paper at the end of the driveway..and then build up slowly, maybe limiting freedom by putting a gate up and confining just to the kitchen.
Again, recall that if you've used the crate properly then they are not aversive to their time in it IF they are exercised before and they're not in there too long, say, more than 3-4 hrs... If he chews up the bedding, try other things...we use a couple of bath mats and AJ is fine with them, esp. since you can tuck them around the pan....and give him a Kong toy stuffed with frozen peanut butter for entertainment.
I understand your concern about the crate behavior, but it is symptomatic of some other cause, not the crate itself, and you should figure out what it is before letting him roam the house, the results of him showing the same distress in the house are incomprehensible!