Why you should do design based research (DBR)

In this post I’ll review and respond to two useful posts by Rebecca Hogue – Why you shouldn’t do Educational Design Research Part 1  and Part 2.

First of all I think the title is quite misleading, in that my reading of it provides amble reasons for doing DBR, in line with the literature. Perhaps I should allow a bit more literally license, but I expected the posts to be attacks on DBR itself as a mythology for doctoral student research. However, the series does highlight contexts and understandings which are inconsistent with DBR and thus is of significant value.


My second problem, is that Rebecca urges the readers to first read her article:

Hogue, R. J. (2013). Epistemological Foundations of Educational Design Research. Paper presented at the World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education.

Unfortunately, she only provides the link to the article in the EdIT database, which charges for access. I was able to access it through my University, but I urge Rebecca and others to NOT submit her work to closed sources (if possible) and to post the article in open fashion if you must give copyright away. Let the copyright police get after you. The worst that can happen is a cease and desist order, unless the copy owner can proof significant economic loss from the infringement.

The first post notes the underlying epistemology of the DBR research paradigm. Pragmatism  supports and defines DBR, action research, many “mixed methods designs” and some types of grounded theory. In the post, Rebecca does a great job of explaining the differences in research design, research questions and methodologies that do not create a ‘mismatch’, with the overarching research paradigm. She gives a good example of why students interested in the nature, phenomena or experience of learning, should be using interpretive paradigms and not pragmatic. One can easily provide additional examples of valid research questions that really are only a match with positivist or critical research paradigms.  Both her article and this post provide amble evidence for the particular epistemological underpinnings (Pragmatism) that are associated and she argues, and I agree, are  required of a DBR research project.

She goes on talk about the output of a “design” as itself being a valid research output. I also agree, but always urge my DBR students (from my own pragmatic world-view) to provide emperical evidence (qualitative and quantitive) that their design has or has not worked – that is as important a contribution to the research as is the design.

The second post is more “pragmatic” and gives 3 potential “show stoppers” which can derail a DBR research thesis.

The first deals with time and notes that a dissertation takes time and the problem should still be around when committee, ethics and a zillion other delays have been surmounted. I would add the converse, that DBR studies almost always involve multiple iterations. This is fine for an established academic to build a research agenda upon, but is nearly impossible for a DBR student who wants to graduate with the decade they begin!!  In science a PhD student can often do a small piece of a research agenda being enacted within their supervisor’s “lab”. Unfortunately, in education due to funding problems and lack of clear and concrete identification and action on particular problems, doctoral students often have to create and define their own research problem and context.

Jan Herrington et al have addressed this issue in more detail see:

Herrington, J., McKenney, S., Reeves, T., & Oliver, R. (2007). Design-based research and doctoral students: Guidelines for preparing a dissertation proposal. Paper presented at the Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications, Vancouver. Retrieved from http://researchrepository.murdoch.edu.au/6762/

As an aside, note how the above article was published and likely owned by EdIT but there are 5 other versions (including the uRL above) of the article (mostly in University databases) available.

A final constraint that I would as is that the intervention must be feasible and cost effective. If one is designing for example a mobile app, one needs the resources to actually create a compelling app that likely needs to run on multiple operating systems. Without resources and time, the design may fail, not because it isn’t an effective design, just that it can’t be built within the contexts of the environment in which it is designed to operate. DBR is pragmatic in more than just the epistemological sense of the word.


Great Firewall of China

I’m on research and study leave (aka Sabbatical) this year and I see that I have been ignoring my blog as well as a number of other “normal responsibilities”. But I have been learning and enjoying. After a 6 week road trip through Eastern Canada and the USA, my wife Susan and I  are just ending a 4 week visit to China.

Our VERY gracious hosts for this trip have been the faculty and students in Distance Education and Educational Technology at Beijing Normal University. Education universities here in China still use (in English) the rather old fashioned term “Normal University” – (not implying that other universities are not normal, nor that the education ones are more normal in the modern sense of the word). Beijing Normal University BNU was founded in 1902 and is China’s 2nd oldest and one of the 3 or 4 most highly respected Universities in China.

I was charged withdoing lectures in 6 classes for PhD and Masters students, 2 lectures for Faculty and students and along the way, accepted invitations to talk at 3 other Universities. In addition I was asked to do a literature review on Interaction in Online Learning. Given the overwhelming interest in various types and modes of interaction in online education in the literature and in my own career, I didn’t think this should be too great a problem. With the help of a grad student from Canada, we did Google Scholar searches for the 10 most cited articles in each of the past 10 years, that included the words Interaction and “distance education” or “online education” or elearning in the title. We then began classifying them by types of interaction (student-student, student content etc.), methodology, context and tried to get a sense of results of recent scholarship. To increase efficiency we stored our spreadsheet of results and first outlines of the paper on DropBox.

Well, major surprises when I attempt to to continue the work in China.  I had heard that some Google services weren’t available, so I changed my browsers to use Microsoft Bing for searches -which worked OK, but much less coverage from Google Search). But I began to realize how Gogglized my life had become. Fortunately Google Mail works most of the time, but Google Scholar was also disabled along with Maps, Image search, Google Books, Google Earth, YouTube and most everything else Google that I use.  Wikipedia lists 2,701 banned sites but I am told that sites come and go with irregular frequency and certainly no accountability. I was particularly sad to loose Google Scholar because I have it set to let me access all of the full text works from closed works that are not available on the Internet but are available to folks like myself with university access to a number of journal databases. I am able to logon to my university library account directly, but when this hotel internet, (shared with MANY University offices) gets used during daylight hours, Internet speed gets VERY slow.

I knew that Google and the Chinese government had a major dustup, but I was surprised to see how many other services were blocked. no Twitter, no Slideshare, No DropBox, No FaceBook,  and likely a number of other services. For the first week, I couldn’t access CBC.com but now it is available – perhaps the Chinese Government  has gotten over the outrageous behariour of either our Prime Minsiter or Jian Ghomeshi, even if I haven’t!)

I also came to realize how Googlized other aspects of my life have become. As Editor Erimitus of IRRODL.COM, I was very surprised to find that this open access journal is basically unusable here in China. We had installed an automatic translator app, in large part becuase of the growing interest in China and many other developing countries in distance education research. But I had forgotten that it used Google Translate (banned). Further investigation found that we used Google analytics, google API’s that are built into the Open Journal System we use and one other Google service – on each page view!  As result the IRRODL site works SLOWLY, one has to wait while it calls and eventually times out on 4 different calls to banned services, making it functionally useless. sigh…

Most of new Chinese friends are aware of the problem, but have a number of standard responses. First, the blocked services have invigorated a number of Chinese social networks and commercial services. Many of these web services such as wechatRenrenDouban  and Jiepang  have millions of users (they have achieved critical mass) and arguably are as good or better than English language services. Secondly most academics use their library databases and seem quite resigned (no protests in the streets here) to doing without some of the systems that have become part of my personal learning network. Finally, there are MANY services which provide services for $5-10 month. I asked a friend if they were not worried that the government would come down on them for bypassing their control systems. He was quite confident that the government didn’t mind, as they were doubtlessly monitoring his VPN access and getting the potential miscreants using fewer services makes their monitoring job easier!

So, it has a been a great visit to China. We’ve seen many of the top tourist sites, squeezed into quite a few over crowded buses, subways and elevators and had many conversations with fascinating students, academics and ordinary Chinese- well at least those who speak English.

I can understand the Chinese motivation to get out of the domination of new media by Western (and mostly USA) services. We’ve been struggling with that in Canada for decades. But heavy handed blocking seems to make academics compete in research endeavours with one English language arm tied behind their back!

African Council for Distance Education 2014

Zambezi Valley
Zambezi Valley

I was honoured to be invited to do a keynote talk at the 4th conference of ACDE in Victoria Falls in Zimbabwe. After sitting up for 2 nights on a plane (42 hour journey) I was very glad to reach the Elephant Hills hotel and a soft bed. The hotel overlooks the Zambezi River and the constant myst of the Victoria Falls can be seen about 3 km away.

Mist rising from Victoria Falls
Mist rising from Victoria Falls

The next day I took a tour of the Falls, and they did not disappoint. They reminded me a lot of Niagara – maybe not so tall, but wider and the same deafening roar as millions of gallons of water churn over the cliff. Victoria Falls Victoria Falls

I did my talk the next day entitled ” Using Open Scholarship to Leapfrog Traditional Educational Barriers And it went OK, but the elaborate formal greetings and pomp of the opening ceremonies, meant that my time was really constrained- though I did manage to squeeze in a joke and give away a copy of one of our open access, Athabasca University Press, Issues in Dist. Educ. series books. The afternoon was spent at a very interesting workshop present by UNESCO and Fred Muller in which he challenged the Open Universities of Africa to embrace and develop MOOC applications- rather than fear them as we seem to do.  I was very impressed with over 200 MOOCs put together by 13 European OpenUpEd collaborators as a service- not profit see openuped.eu.

Generally the African Open Universities are focussed on quality production of print packages and support (tutorials, testing etc) in local learning Centres. They suffer from the same prejudice from educated elites, and the faculties of traditional universities, but of course their costs are much lower. It is clear to all that sufficient campus universities will never be built to accommodate the large and growing demand for higher education in Africa.  Many of  the presenters presented compelling cases for more support, but also presented evidence of the changes that their programs are making in the lives of disadvantaged students.  All the participants seem to have a sense that they should be using more net based technologies, but judging from the general absence of laptops by all but a few of the conference delegates, I think that net access and literacy is an issue not only for students but for faculty as well. This has been a very short trip, but the kindness of new friends and of the Zimbabwe people I have met here at the hotel is long!

Greewich Connect connects with us on a number of levels

I was pleasantly surprised to see a recent conference paper (reference below) by folks at the University of Greenwich who are reporting their first year results from a project (Greenwich Connect) that is designed to induce a variety of open and social programs to the university teaching and learning communities.  The surprise was pleasant because we have been attempting a similar project here at Athabasca University- known as the Athabasca Landing. However, it was unpleasant to read that the challenges they are facing are very similar to our own and some days they seem intractable.

The contrast between contexts is as striking as the similarities in challenges. Greenwich is a 2-campus University in the UK, while Athabasca is a 100% online and distance education institution in Canada. However we both share a passion “to enhance the connectedness of learners at a curricula and teaching and learning level” .

Greenwich Connect is a 2 year, $750,000 initiative funded and championed by their Vice Chancellor (President). (see project description) Continue reading

Where is Higher Education’s Digital Dividend?

One doesn’t need to devour political or economic analysis, listen to experts or even chat with one’s friends to realize that the Internet has changed the way we produce and consume information and the myriad ways in which we communicate. Blogs, wikis and Facebook walls have granted to each of us –a multimedia printing press with global delivery capacity – at VERY low cost. Similarly we can engage in audio, video or text conversations with politicians, relatives, co-workers or “followers” at VERY low cost.

Given that education works by nurturing interactions and communication among and between teachers, students and content, it would seem logical that the costs of education, like its component interactions would also have drastically reduced in cost. However, this is not the case. Despite the possibility of a digital dividend students, in every country, are being met with heavy increases in the cost of education. Continue reading

Does teaching presence matter in a MOOC?

A recent study of a Coursera MOOC is really interesting in that it implemented a random assignment of student to 2 conditions – one with no teacher interaction with the students and the other with teacher and teacher assistant interaction in forums. The study is

Tomkin, J. H., & Charlevoix, D. (2014). Do professors matter?: using an a/b test to evaluate the impact of instructor involvement on MOOC student outcomes. Paper presented at the Proceedings of the first ACM conference on Learning@ scale conference. Retrieved from http://dl.acm.org/citation.cfm?id=2566245

The study concluded that teacher presence had no significant relation to course completion, most badges awarded, intent to register in subsequent MOOCs or course satisfaction.  This is of course bad news for teacher’s unions and those convinced that a live teacher must be present in order for significant learning to occur. However, the findings is predicted by my Interaction Equivalency Theory in which I argue that if one of the three forms of student interaction (student-student, student-teacher, student content) is at a high level, the other two can be reduced or even eliminated.  Adding additional forms of interaction may increase satisfaction (though it seems not to have done so in this experiment), but it most certainly also increases costs and thus decreases accessibility.

Tomkin and Charlevoix argue “The results of this study broadly support the connected learning model, at least for these motivated, educated participants. The absence of the professor did not impact the activity of the forums – the participants did generate their own knowledge in this arena. It should be stressed that this MOOC was highly structured, so an alternative explanation is that the enhanced machine interactivity that MOOCs provide relative to textbooks, or older styles of distance learning, may be sufficient to stimulate student engagement. ” p. 75

I see this as one the few tangible outcomes of the “digital dividend” that actually results in cost savings to students.  The student-teacher interaction was morphed into student-content interaction through the digital videos. The study shows there was student-student interaction, however in no teacher interaction MOOC, this interaction was both stimulated and supported by the students themselves.

I’m discouraged by the ever increasing costs of higher education and most notably our incapacity to scale higher education to meet needs (and the capacity) of students in developing countries. I believe we have a moral obligation to help all students become proficient life-long learners who are capable of learning with or without active teacher presence – despite the potential impact on our own employment.


Online 3 Minute Thesis Contest at Athabasca

I’m intrigued by this “speed dating” approach for disseminating and promoting thesis research. A thesis is a LOT of work, and results are usually buried in 150+ page tomes – thus the need for new scholars to be able to present their work succinctly and efficiently.  The 3 minute thesis (originally developed at University of Queensland Australia) seemed like an ideal format for developing communications skills and confidence and be fun for both contestants and the audience.  Other 3 minute thesis contests have been held F2F, however, Athabasca graduate students are located around the globe (literally) and so we needed to use a distributed platform to host the event. Of course, the organization of the event also had to be easy and inexpensive so as to fit into my busy schedule and budget as well.

In this post I detail how this, to my knowledge, world’s first online 3 minute thesis contest worked, with a hope that it inspires similar contests. Continue reading

A Publishing Primer for Education Grad students

Ask any academic, and they will get into a long discourse about the value of publishing scholarly work, the politics of doing it, the challenges and the outlets.  Although you haven’t asked, I’d like to share my own ideas with particular relevance to publishing work related to distance education.

By way of background, I’ve been in the publish or perish business (as a full time academic) for the past 20 years. During that time I’ve published (solely or in collaboration) over 60 articles and have had my share of rejections as well (ouch!). I also have been the editor of IRRODL for the past 10 years, and so have been involved in the review and production of over 500 articles and many more rejections!

Why Publish?  If tenure or a promotion is at stake, the answer to this question is obvious. If not, publishing allows you the opportunity to share your work on an international scale. You’ve worked long and hard on a project and not only does your work likely warrant celebration and dissemination, the publication begins building your global academic career and increases your social capital, that you can cash in for a whole variety of rewards.  Publication also insures that your work preservers. It is a great treat when you get old (like myself) to revisit some of your earlier work – without having to find a machine that reads 5 ¼ floppy disks! Finally,  a quality review process, will show you how to improve the article and thus directly lead to increased capacity to express yourself in this format. A video addition to this post for an OER course.

Continue reading