While in a true IPA model, we would be using themes generated from the
data set itself to make sense of the data, in this paper, we have used pre-existing definitions of the seven 21st century skills (and empathy) outlined
in ASB’s model of learning (Figure 2) as our codes for analysis.
Purvi and Deepika read through more than five hundred student
reflections written by over two hundred students from 6th-12th grades
between September 2013 and April 2015. We also reviewed student
generated artefacts like Google docs, Facebook pages, websites,
presentations, etc. to look for evidence of 21stcentury skills.
While reading, we pulled out direct excerpts from the reflections, as well
as took snapshots of student artefacts, and plugged them into a
spreadsheet. Each quote or artefact was tagged with one or more
21stcentury skill using the definitions provided in Figure 2. To increase the
reliability of this analysis, both authors tagged the same data set
independently and then came together to discuss their differences, thus
increasing the robustness of our inferences. By the end of our analysis,
we had tagged 113 student reflections and artefacts.
Overall, we went through three rounds of coding. To start with, we
determined the keywords to use in coding for 21st century skills based on
the definitions in Figure 2. We coded a few reflections together for
practice. The first complete round one was done individually. In round
two, we compared our codes and discussed each piece of evidence for
similarities and differences. During these discussions, it became apparent
that we had sometimes strayed from the definitions given in Figure 2,
bringing our own interpretations of the 21stcentury skills to the coding
process. We also realized that a similar set of questions kept recurring
during our discussions in round two. Figure 3 explains the questions that
emerged during our analysis and the assumptions we made to aid us in
the coding process. Based on these discussions, we decided to conduct
one more round of coding where we minutely re-examined each piece of
evidence against the ASB definitions to check whether the data were truly
indicative of the 21stcentury skill as defined by ASB. This cyclical process
led to a high inter-rater reliability.