Many may find they cannot because of health issues, layoffs or because they have to care for spouses or parents