Retention of in-home caregivers to resolve insufficient labor issues has become the primary obstacle in realizing the home care model of “Aging in Place.” This study seeks to determine the factors affecting retention of in-home caregivers by examining their working environment, organization culture, family life, and personal viewpoint. The data were collected in 2018 from home care assistants working in the long-term home care service units in Changhua County. A total of 220 valid responses were obtained from the sample of 350 participants. To test the research hypotheses, hierarchical regression analysis was adopted. The results showed that: (a) the working environment has a negative impact on the tendency to stay; (b) internal and external factors have a positive impact on the tendency to stay; (c) work-family conflict has a negative impact on the tendency to stay; and (d) prosocial behavior has a positive impact on the tendency to stay. The findings of the study provide several management implications: 1. reduce the uncertainty of the environment factor in the work area; 2. enhance the internal and external incentive satisfaction; 3. reduce work-family conflict; 4. prosocial behavior should be cultivated and encouraged during the recruitment process.