Tuesday, July 7, 2015

Some notes on course design and assignments

Since adopting 3dgamelab as a teaching tool, I have had the opportunity to think about assignment design. Below are some that turned out to be quite engaging for students:

Design a midterm

This was inspired from a brainstorming session about student self-assessment. Only a person who has a strong grasp about the course material can design appropriately calibrated midterm questions. Such questions have to address core concepts and do so in a way that compels the test taker to use them. So I figured, why not give the students the task of designing a few midterm questions? I instruct them to do fourish multiple choice questions and two short answer questions. The assignment works really well - not a few students design better questions than I would have, and the exercise quickly reveals which students are still struggling with the material.

Swapping papers

One of the problems of academic paper writing exercises for students is to get the needed feedback to improve. One of the problems for instructors is to get papers with relatively elementary issues, like grammar, punctuation and the like - writing issues that are really on the secondary level of schooling. At post-secondary, instructors (in my experience) would prefer to deal with the structure of the paper, or the strength of the argument and research. 

This assignment is designed to address both of the above problems. It has students switch papers with each other before handing in the draft to the instructor. They read a peer's paper and comment on it constructively - what was strong, what can be improved? For the student, it has the benefit of having them reflect on the writing process as a reader, so when they go back to their own paper they can use that perspective more effectively during editing. Also, they get some feedback from a peer who can hopefully catch the most glaring grammar issues. The instructor on average gets papers of higher quality.

Flip the classroom

First, I decided to remove the planned lecture series from the course design. Since I was using a webtool to deliver the course assignments anyway, I felt that any material that would have been presented through a traditional lecture series could just as well be done through YouTube-videos. Thus I started my channel. I upload all the lecture material there. In my experience, the material presented through the traditional 50 minute lecture can be presented through a 10-15 minute video just as well. The shorter time frame requires being more concentrated with the information and the fact that students can re-watch the video an infinite number of times effectively avoids the problem of wondering whether students will remember what you said in a lecture six weeks ago.

These videos become mini-assignments for students to watch and then I use the classroom time for exercises that compel the students to use the concepts introduced in the videos. The time saved through this method cannot be overestimated.

The role-plays

And that's where the role-plays come in. These are design to provide a simulation of the institutions and political contexts the students are studying in the readings and are inspired by edu-larps (educational live action role-playing) and Model UN. In short, this is game-based learning. Students take the roles of politicians, journalists, civil servants or clients and act within the incentive structure of the institution in question. The debriefs they have to do after each simulation shows how effective the tool is. These are some of the models designed (there is more):

Introduction to Comparative politics

  • US Congress - pass a budget
  • UK House of Commons - Question Period
  • German Parliament - build a government coalition post-elections
  • French Unitary State - build a budget in the contest between regional, departmental and municipal governments who all want state funding

Introduction to Canadian politics

  • First Minister Conference - interaction between federal and provincial governments re: Senate reform
  • The Newsroom - an exercise in media news triage
  • An election campaign - pitch your platform and then win the televised debate. Next time I do this one, I would like to broadcast the debate live through my YouTube-channel to give the students a group of voters, too.
In my last incarnations of this, I took a cue from Sheldon's The Multiplayer Classroom, and made the role-playing continuous. That is, students took the roles that they played through the entire term, from five different "character classes", like in computer games. For Comparative Politics, these were Politician, Civil Servant, Journalist, Activist, and Academic. The idea is for each of these to have their own motivations and goals, and together, the students create a story line that probes the concepts at hand. I will be exploring this model further in the future.

Tuesday, June 23, 2015

Race and demography in Sweden

A debate about racial demography is developing in Sweden. This is the background - recently, the Swedish government decided to create a national institute against racism and extremist violence. This task has been assigned to Gothenburg University. It has drawn a range of criticism from scholars exploring racism.

Among other issues, critics have pointed out that all the working group that has been appointed to develop the centre are all white and none of them have specialized in the study of racism. The critics argue the omission of non-whites and racism scholars is rather remarkable given the group's mission. They describe the appointment process as columbusing, i.e. where a group of power-holders "discover" something that was already known outside their own circle.  The obvious comparison is, what would the reaction be if a centre was set up for the study of gender equality, but the working group to develop this centre were all males without any experience of gender studies?

There was an immediate reaction to this critique, particularly from professor Rothstein. For context, he is one of the most luminary Swedish political scientists, with a highly distinguished publishing record. It's not possible to get an undergraduate degree in political science in Sweden without reading some of his work. He considers the critics' attention to the racial background of the appointees deeply troubling. He then turns to the time in the 1930s when the Swedish government asked the government of Nazi Germany to introduce the stamp "J" in the passports of its Jewish citizens. He argues that the attention anti-racists give to the racial or ethnic background of applicants or employees amounts to copying the Nazi German model.

Shortly afterwards, an influential editorial by Teodorescu (on the political right - these issues often become politically polarized quickly) came out essentially repeating Rothstein's position. Her development of the argument essentially lands in the conclusion that we should stop speaking about racial or ethnic categories all together, because doing so reinforces racism and exclusion.

The argument is bizarre for anyone who has worked in the field of combating ethnic discrimination for any period of time. One of the primary tools for tracking processes of discrimination is to chart the demography of employees in the workforce. This is routinely being done in both private and public organizations in several countries, notably the United States and Canada.

A typical example of how such tools are implemented is diversity management (yes, it is loaded with controversy, but that goes beyond this post). A classic problem is that recruitment processes can be unintentionally discriminatory, when a homogeneous group sets up standards or criteria for recruitment, they might "rig the game" in a way that ends up excluding relevant competencies and experiences in such a way that the homogeneity is maintained. When that happens, there is a risk that the organization becomes characterized by group-think and develop blind-spots where important perspectives are missed. To prevent this, human resource departments can do internal surveys of applicants and employees to make sure that the demography of these groups is not too homogeneous. If it is, there is likely something wrong in the recruitment procedure. This is an example of such a survey, which is now a regular tool in the appointment of academic positions in the United States:

There is then a question for those who argue that all discussions on the charting of racial or ethnic demographics are problematic: How could anyone develop practical tools for preventing discrimination without being able to measure the outcomes of human resource practices? Is the survey above really racist, or an equivalent to Nazi German practices? I find that position difficult to sustain, and it becomes really troublesome when the logic leads to the position that talking about racism is racist.

Saturday, June 13, 2015

Barriers to teaching innovation in post-secondary education

I recently went to the CPSA (Canadian Political Science Association) annual conference in Ottawa to present the paper "Gamifying Political Science", which describes my experiences of using game design tools to improve the learning experiences of my students. The method shows great promise and a mounting body of evidence suggests that it is a highly effective pedagogical tool. The paper ends with a discussion on the likelihood of game design tools being adopted in political science teaching more widely and identifies a series of barriers on both individual and institutional level that gives us little reason to expect its adoption any time soon.

Some of those barriers include a lack of familiarity with pedagogical techniques or technological tools among individual professors, as well as a shortage of time which prevents instructors from learning about how to use new pedagogical approaches. On the institutional level, departments rarely invest in structures to support teaching development, since they prioritize production - i.e. research - instead.

Importantly, the paper notes that teaching is not incentivized as part of the career path (indeed, one CPSA-panelist who remarked that devoting time to become a good teacher can be 'career suicide', which is quite revealing). I discuss some of the mechanisms in the paper and would like to add a couple of reflections here. These notes are based on experiences and observations from working in this field and should be seen as points of departure for further discussion.

The first concerns the lack of career incentives for good instructors in the academic world, or at least in my field - political science. Universities are, in the public debate, often said to have two core tasks, research and teaching. However, in my experience, teaching seems to be treated as a core task in name only. For example, as a graduate student, the most prestigious research awards (there is more than one) are set at about CAD  40 000. There was only one award for teaching excellence available to us, set at CAD  1 000, so the career infrastructure is made already at that stage. Tenure is, of course, a research position. Those who are passionate about teaching are confined to sessional positions, which pay much less and have no job security. There is no way to advance from there as an instructor. Also, institutions generally have few tools for evaluating teaching quality. The only one I've seen is the student course evaluations, which have been much criticized, but no one seems to be interested in using that critique to develop better tools. An institution that doesn't pay its instructors stable and sufficient salaries and doesn't even bother to evaluate whether the activity holds a high quality should hardly claim that activity as part of its core mission. The situation calls to mind Brunsson's piece (2003) Organized hypocrisy, where he describes how it is rational for organizations to say one thing and then do another to satisfy contradictory demands.

This institutional structure is reflected on the individual level. In my experience, academics seem oddly defensive about teaching methods. Common reactions have included comments like "...but I like lecturing". That is all well and good, but instructors are entering the classroom for the undergraduate students, and to do their duty by them. Instructors should thus do what research shows is effective and efficient to facilitate student learning. I have yet to find any study showing that the lecture series is the most optimal teaching tools. On the contrary, there is a growing body of evidence showing that other techniques are more effective, but that evidence seems to have little effect on practices.

Of note, the physicist Eric Mazur famously remarked on how tenured Harvard professors, who certainly know everything about the scientific method when it comes to their research, for some reason discards that same scientific method entirely when it comes to teaching. As Donald Clark notes in his keynote address, "hardly anybody who teaches in a university believes in the application of the scientific method to teaching and learning". The situation is thus like this: the same profession that a) continuously and routinely exposes itself to critique through peer review and b) keeps emphasizing the importance of the scientific method to students, suddenly lets go of these principles entirely when it comes to how to design and deliver a course. That is quite problematic.

All is not doom and gloom, though. There are interesting initiatives out there. Quest University was literally built from scratch with the intention of providing undergraduate students with the best possible learning experience. It has small classes - max. 25 students and the very interesting block system for how courses as structured: students study one subject at a time, instead of several courses in parallel, which allows instructors more liberty in terms of how to dispose of their time. Also, no departments. Professor Helfand presents it here. The Teaching Professor network in the US is also interesting. It organizes a conference for disseminating best teaching practices.

Such initiatives are important, but more is needed to institutionalize substantive incentives for teaching excellence in the academic career track. I would argue that universities owe their undergraduates to commit to and make that change.

Monday, May 25, 2015

Gamification and game-based learning

I mentioned in a previous post that I have had some interesting experiences with teaching methods over the last couple of years. I felt frustrations when I was using the design that is standard operating procedure for most post-secondary courses: a lecture series, possibly with discussion seminars graded based on participation and then a couple of exams and a graded paper. I did not feel that the assessment forms really were sufficient for optimizing the learnings for my students nor the model provided me with sufficient feedback to know how to facilitate their learning.

Then I discovered 3dgamelab, and everything changed. Since 2013, it is the design I use, and shown in this video

It has three main advantages, in my experience:

  • formative assessment - This is different from summative assessment (i.e. a typical exam - write the exam, get a grade, move on). In formative assessment, the student will produce a learning artifact of some kind, but if it doesn't meet the requirements, I will send it back. Thus, the student learns through that feedback.
  • increased student engagement through active learning - I have dispensed with the lecture series (my lectures are now available on YouTube), and use the classroom time for students to do labs or simulations, keeping them active.
  • increased student agency - students can choose from a smorgasbord of assignments and complete them at a time of their choice.

The effects have been palpable, and it seems to me that the scholarship on teaching methods generally provide support for the veracity of the model. Next week, I'm presenting a paper on my experiences. Here is the abstract:

Gamification consists of the introducing game mechanics into activities to engage users with motivation beyond what is normally expected. Properties include introducing points accumulation, badges, levels, leaderboards, challenges or quests, customization, economies, avatars, and role-play. When introduced into education in the form of game-based learning, such elements can raise student motivation as well as decision-making capacity and cognitive development as students are given increased control over their learning path. In 2013, I changed my political science course design using the learning platform 3dgamelab. The experience was transformative on many levels, including learning achievements and the instructor-student relationship. This paper describes those experiences for the purpose of demonstrating the utility of game-based learning when teaching political science in higher education. It provides an overview of the literature on game-based learning in education and the significance of some integral components of a course design using 3dgamelab, including elements like active learning, the flipped classroom and formative grading. This literature provides important context for the experiences I have made transforming a lecture based course design to a game-based one. Five courses have now been delivered using this framework at the 200 and 300-level in Comparative and Canadian politics, with class sizes varying between 8 and 75 students. The paper identifies opportunities for instructors as well as potential sources of problems and how to re-think how political science education can be delivered in a more engaging way.

Sunday, May 24, 2015

New topics

I will be adding comments on two areas to this blog. One is about public administration. Most of the research on immigrant integration focuses on the human capital of the newcomers (what do they bring in terms of competence, education, experience), immigrant rights (how encompassing should they be), the size of the welfare state, or public policy (for example, does the host country have official multiculturalism or not). However, very little attention is given to how the programs that are supposed to facilitate immigrant integration are organized - who is doing what, with what mandate and funding? Those are all questions of public administration and management.

Secondly, I will add my reflections on teaching in political science on the post-secondary level. I taught my first course in 2010 and in 2013, I switched to gamification and game-based learning. That prompted me to think a lot about how teaching is done in this field. I'm presenting a paper on the subject at the coming annual CPSA Canadian Political Science Association in June in Ottawa. That will be quite exciting!

Thesis: Completed

I am finally getting back to this blog. It is done - the thesis has been successfully defended. Here is a 1 minute synopsis of it:

Friday, June 1, 2012

Measuring discrimination with a questionnaire?

A new report from Timbro, the Swedish right-wing think tank, has caused some commentary. The author argues that discrimination does not explain why immigrants face challenges on the Swedish labour market. This argument rest on the observation that employers in other countries (Canada, Denmark, Germany) answer a question from World Value Survey concerning privileging native born job seekers over foreign born such more negatively than Swedish employers do.

While I have to admit that I was surprised that Canadian employers would be presenting such answers to the question, the report, in my opinion, has a fundamental methodological flaw, and that is its reliance on the World Value Survey for its conclusions. This survey is a quantitative survey of values, taken in from across the world (as the name implies). It could be argued the aggregate data of the Survey gives us some idea about cultural differences across the world (though it is possible to question that, too, to some extent - after all, how do you develop distinctly separate categories of "traditional" and "secular-rational" values?) but I'm not sure that it is a useful in a smaller sample studie like this.

More importantly, it is inherently quantative, and I'm not at all convinced that quantitative methods can reveal a whole lot about discrimination in practice. At best, the report manages to show that Swedish employers have a self-image of not being discriminatory, but from there, it's a bit of a leap to conclude that they will not act in a discriminatory fashion in practice. Indeed, unconscious discrimation, i.e. acts that are not intended as discriminatory, but result in discriminatory outcomes because of ignorare, or prejudice that is assumed to be common sense, cannot be easily probed using a questionnaire.

Qualitative methods, like in-depth interviews, participant observation and so on would be more appropriate, and every time such methods have been employed in Sweden, significant discriminatory behaviour has been observed.