Working memory is one of the cognitive brain mechanisms which manages instant information storage and processing. This article discusses the development of Chinese listening and reading skills under the influence of working memory. 71 foreign students were recruited in our research, whose working memory capacity was evaluated with a reading span task. A listening and reading comprehension test were later implemented to evaluate the students’ Chinese listening and reading skills along with the effect of inferential complexity on their working memory load. The results were analyzed using three-way mixed design ANOVA and linear regression, which demonstrated a correlation between listening and reading skills and working memory for students with an intermediate level of Chinese proficiency but not for those with a high level of Chinese proficiency. Furthermore, we found that higher inferential complexity of the context requires a higher working memory load, which results in a weaker listening and reading performance. Based on our findings, we propose a number of class activities for a more efficient use of working memory and various approaches to reduce the inferential complexity of course materials.