Towards a framework for the design of quantitative experiments: Human-computer interaction and accessibility research

Abstract

Many students and researchers struggle with the design and analysis of empirical experiments. Such issue may be caused by lack of knowledge about inferential statistics and suitable software tools. Often, students and researchers conduct experiments without having a complete plan for the entire lifecycle of the process. Difficulties associated with the statistical analysis are often ignored. Consequently, one may end up with data that cannot be easily analyzed. This paper discusses the concept sketch of a framework that intends to help students and researchers to design correct empirical experiments by making sound design decisions early in the research process. The framework consists of an IDE, i.e., Integrated (statistical experiment) Development Environment. This IDE helps the user structures an experiment by giving continuous feedback drawing the experimenter’s attention towards potential problems. The output of the IDE is an experimental structure and data format that can be imported to common statistical packages such as JASP in addition to providing guidance about what tests to use.