In 2011, the Dutch psychologist Diederik Stapel has been revealed for fabricating and manipulating data for his research publications. In 2015, the Open Science Collaboration published an article on “Science” journal and then demonstrated that less than 50% of the classical psychological experiments can be replicated, highlights the instability of the knowledge in psychological science, thereby allowing the academic community to reflect and adjust some new methods in statistical analysis and advocate perspective on open science. Furthermore, this reflection also influences the current social and behavior science on research design and data analytic method, especially in the area of recent physical education. In order to understand the full scope of this impact, the current article will be illustrated such background of the current replication crisis, and then discussed some issues in research design in which focus on power analysis and sample sizes estimation, and explained the current status in physical education. Furthermore, the new statistical analysis such as p-curve will be introduced, and then discussed the Registered Replication Report and open science perspective. Finally, their applications in physical education will be discussed.