Create QualitativeSimulation

Initial Draft of Qualitative Simulation Standard

docs/drafts/QualitativeSimulation

@@ -0,0 +1,90 @@
+# Qualitative Simulation 
+<standard name="Qualitative Simulation">
+
+*<desc>"A study that involves predominately qualitative analysis of direct observations of humans taking part in a lab-based simulation</desc>*
+
+
+## Application 
+
+This standard applies to empirical research that meets the following
+conditions.
+
+-   Human participants take part in a simulated task, typically in a contrived or lab-based environment, alone or in groups
+-   Researchers directly observe (and often record) participants taking part
+-   Observations are analyzed using qualitative coding
+
+Protocol Analysis (AKA protocol studies) are a common type of Qualitative Simulation
+
+## Specific Attributes 
+
+### Essential Attributes
+<checklist name="Essential">
+
+<intro>
+
+
+<method>
+
+- [ ]   describes the environment in which the simulation took place
+- [ ]   describes the events, activities, tasks, or objects comprising the simulations    
+- [ ]   describes the qualitative coding procedures
+
+<results>
+
+- [ ]   presents a clear chain of evidence from observations to findings
+
+<discussion>
+
+
+<other>    
+
+</checklist>
+
+### Desirable Attributes
+<checklist name="Desirable">
+
+- [ ]	  provides supplemental materials such as task materials, coding schemes, coding examples, decision rules, or extended chain-of-evidence tables
+- [ ]   includes debriefing sessions with participants 
+- [ ]   cross-checks observations against statements made my participants during debriefing sessions
+- [ ]   describes how prior understandings of the phenomena were managed and/or influenced the research
+- [ ]   EITHER: evaluates an a priori theory (or model, framework, taxonomy, etc.) using deductive coding with an a priori coding scheme based on the prior theory    
+     OR: synthesizes results into a new, mature, fully-developed and clearly articulated theory (or model, etc.) using some form of inductive coding (coding scheme generated from data)
+- [ ]   includes autoreflection; i.e., researchers reflect on how their own possible biases may have influenced the research
+</checklist>
+
+### Extraordinary Attributes
+<checklist name="Extraordinary">
+
+- [ ]   uses a team-based approach; e.g., multiple raters with analysis of inter-rater reliability (see the [IRR/IRA Supplement](https://github.com/acmsigsoft/EmpiricalStandards/blob/master/Supplements/InterRaterReliabilityAndAgreement.md))
+- [ ]   published a protocol beforehand and made it publicly accessible (see the [Registered Reports Supplement](https://github.com/acmsigsoft/EmpiricalStandards/tree/master/Supplements)) 
+</checklist>
+
+## General Quality Criteria 
+
+Qualitative simulations should be evaluated using qualitative validity criteria
+such as credibility, multivocality, reflexivity, rigor and transferability (see **Glossary**).  
+
+## Antipatterns 
+
+-  Overstating the degree to which the simulation justifies inferences about participants' cognitive states or cognitive processes. The whole point of a qualitative simulation is often to get insight into cognitive states and processes, but these insights should be presented with caution.
+-  Attempting a qualitative simulation from a positivist epistemological stance. There is no coherent positivist basis for qualitative research in an artificial context. Qualitative simulations make more sense from a realist or interpretivist epistemological stance.  
+
+## Invalid Criticisms 
+
+-   Does not present quantitative data. That's why it's called a "qualitative" simulation.
+-   The task or environment are artificial. The whole point of a qualitative simulation is to study an artificial task in a controlled environment. 
+-   Lack of internal validity. Internal validity is a positivist criterion.
+
+## Suggested Readings 
+
+Mathew Miles, A Michael Huberman and Saldana Johnny. 2014. *Qualitative
+data analysis: A methods sourcebook*. Sage.
+
+Sarah J. Tracy. 2010. Qualitative Quality: Eight "Big-Tent" Criteria for
+Excellent Qualitative Research. *Qualitative Inquiry*. 16, 10,
+837–851. DOI:
+[10.1177/1077800410383121](https://doi.org/10.1177/1077800410383121)
+
+## Exemplars 
+
+Mohanani, Rahul, Paul Ralph, Burak Turhan, and Vladimir Mandić. 2021. How templated requirements specifications inhibit creativity in software engineering. *IEEE Transactions on Software Engineering* 48, 10, 4074-4086. DOI: [10.1109/TSE.2021.3112503](https://doi.org/10.1109/TSE.2021.3112503)