Why you should do design based research (DBR)

In this post I’ll review and respond to two useful posts by Rebecca Hogue – Why you shouldn’t do Educational Design Research Part 1  and Part 2.

First of all I think the title is quite misleading, in that my reading of it provides amble reasons for doing DBR, in line with the literature. Perhaps I should allow a bit more literally license, but I expected the posts to be attacks on DBR itself as a mythology for doctoral student research. However, the series does highlight contexts and understandings which are inconsistent with DBR and thus is of significant value.


My second problem, is that Rebecca urges the readers to first read her article:

Hogue, R. J. (2013). Epistemological Foundations of Educational Design Research. Paper presented at the World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education.

Unfortunately, she only provides the link to the article in the EdIT database, which charges for access. I was able to access it through my University, but I urge Rebecca and others to NOT submit her work to closed sources (if possible) and to post the article in open fashion if you must give copyright away. Let the copyright police get after you. The worst that can happen is a cease and desist order, unless the copy owner can proof significant economic loss from the infringement.

The first post notes the underlying epistemology of the DBR research paradigm. Pragmatism  supports and defines DBR, action research, many “mixed methods designs” and some types of grounded theory. In the post, Rebecca does a great job of explaining the differences in research design, research questions and methodologies that do not create a ‘mismatch’, with the overarching research paradigm. She gives a good example of why students interested in the nature, phenomena or experience of learning, should be using interpretive paradigms and not pragmatic. One can easily provide additional examples of valid research questions that really are only a match with positivist or critical research paradigms.  Both her article and this post provide amble evidence for the particular epistemological underpinnings (Pragmatism) that are associated and she argues, and I agree, are  required of a DBR research project.

She goes on talk about the output of a “design” as itself being a valid research output. I also agree, but always urge my DBR students (from my own pragmatic world-view) to provide emperical evidence (qualitative and quantitive) that their design has or has not worked – that is as important a contribution to the research as is the design.

The second post is more “pragmatic” and gives 3 potential “show stoppers” which can derail a DBR research thesis.

The first deals with time and notes that a dissertation takes time and the problem should still be around when committee, ethics and a zillion other delays have been surmounted. I would add the converse, that DBR studies almost always involve multiple iterations. This is fine for an established academic to build a research agenda upon, but is nearly impossible for a DBR student who wants to graduate with the decade they begin!!  In science a PhD student can often do a small piece of a research agenda being enacted within their supervisor’s “lab”. Unfortunately, in education due to funding problems and lack of clear and concrete identification and action on particular problems, doctoral students often have to create and define their own research problem and context.

Jan Herrington et al have addressed this issue in more detail see:

Herrington, J., McKenney, S., Reeves, T., & Oliver, R. (2007). Design-based research and doctoral students: Guidelines for preparing a dissertation proposal. Paper presented at the Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications, Vancouver. Retrieved from http://researchrepository.murdoch.edu.au/6762/

As an aside, note how the above article was published and likely owned by EdIT but there are 5 other versions (including the uRL above) of the article (mostly in University databases) available.

A final constraint that I would as is that the intervention must be feasible and cost effective. If one is designing for example a mobile app, one needs the resources to actually create a compelling app that likely needs to run on multiple operating systems. Without resources and time, the design may fail, not because it isn’t an effective design, just that it can’t be built within the contexts of the environment in which it is designed to operate. DBR is pragmatic in more than just the epistemological sense of the word.


African Council for Distance Education 2014

Zambezi Valley
Zambezi Valley

I was honoured to be invited to do a keynote talk at the 4th conference of ACDE in Victoria Falls in Zimbabwe. After sitting up for 2 nights on a plane (42 hour journey) I was very glad to reach the Elephant Hills hotel and a soft bed. The hotel overlooks the Zambezi River and the constant myst of the Victoria Falls can be seen about 3 km away.

Mist rising from Victoria Falls
Mist rising from Victoria Falls

The next day I took a tour of the Falls, and they did not disappoint. They reminded me a lot of Niagara – maybe not so tall, but wider and the same deafening roar as millions of gallons of water churn over the cliff. Victoria Falls Victoria Falls

I did my talk the next day entitled ” Using Open Scholarship to Leapfrog Traditional Educational Barriers And it went OK, but the elaborate formal greetings and pomp of the opening ceremonies, meant that my time was really constrained- though I did manage to squeeze in a joke and give away a copy of one of our open access, Athabasca University Press, Issues in Dist. Educ. series books. The afternoon was spent at a very interesting workshop present by UNESCO and Fred Muller in which he challenged the Open Universities of Africa to embrace and develop MOOC applications- rather than fear them as we seem to do.  I was very impressed with over 200 MOOCs put together by 13 European OpenUpEd collaborators as a service- not profit see openuped.eu.

Generally the African Open Universities are focussed on quality production of print packages and support (tutorials, testing etc) in local learning Centres. They suffer from the same prejudice from educated elites, and the faculties of traditional universities, but of course their costs are much lower. It is clear to all that sufficient campus universities will never be built to accommodate the large and growing demand for higher education in Africa.  Many of  the presenters presented compelling cases for more support, but also presented evidence of the changes that their programs are making in the lives of disadvantaged students.  All the participants seem to have a sense that they should be using more net based technologies, but judging from the general absence of laptops by all but a few of the conference delegates, I think that net access and literacy is an issue not only for students but for faculty as well. This has been a very short trip, but the kindness of new friends and of the Zimbabwe people I have met here at the hotel is long!

A problem with educational research publishing is that most of the most highly rated peer-reviewed journals are closed access, and though most are accessible to me through our library, I try as much as possible not to contribute to journals that are not available as open access. Especially in education, there are too many potential readers in schools, universities in developing countries and ordinary Canadians who just don’t have access to expensive closed publications. Thus, I strongly support the recent boycott of Elsevier (largest journal publisher in the world) but I extend a personal boycott it to all closed research publications (with the occasional exception).
However a year ago, I had an idea to research the impact of what for us in education was a new and i think very promising research methodology known as design-based research (DBR). The DBR model matches researchers with teachers in real educational contexts, to develop interventions (pedagogical technical, administrative etc.) and then test them in real contexts using a variety of qualitative and quantitative techniques and then extracting broader design principles explaining how and why the intervention works or fails to improve teaching or learning.

I was fortunate enough to recruit Atahabasca University doctoral student Julie Shattuck and together we analysed the 5 most widely cited DBR articles over each of the ten years since the methodology was first promoted. You are welcome to read the results published this week

Terry Anderson and Julie Shattuck (2012) Design-Based Research : A Decade of Progress in Education Research? Educational Researcher, 41: 16-25,http://edr.sagepub.com/content/41/1/16.full

Once the article was written we were faced with the question of where to publish. I thought the article would have considerable interest -especially in the US, as this is where the majority of cited DBR articles were published. Thus, I wanted to go beyond the open and distance education audience that I normally write for. I immediately thought of Educational Researcher. This is arguably and usually cited as the most widely read and prestigious peer reviewed journal in the educational world (ISO Impact factor of 3.774), so I really didn’t think we would get accepted.
However, although Educational Researcher is published by Sage (a commercial publisher) it is sponsored by The American Educational Research Association (world’s largest professional educational group) and is distributed in paper to all its members and most importantly freely in PDF format on the web. So the prospect of a very large audience in a very prestigious journal and close to open access publication was irresistible.

The submission process was very picky and exacting, with editors demanding very strict adherence to APA format, page length etc. before they would even send it to review. Not to my surprise, two months later, it was returned with requests for revisions which I must say all reasonable and likely to improve the paper. However, the returned manuscript must go through a second review, so we didn’t get our hopes up. Then last summer, to our surprise, a very quick second review and minor suggestions for revisions and we were accepted!! 5 months later everyone is able to read the article online and I await my paper copy in the mail!
So thanks to Julie, the reviewers and editors at Educational Research and celebration time!!

AuPress to expand open access online learning publications

I am a big supporter of Open Access presses – largely because they serve potential readers without means or capacity to purchase books and as importantly, because they increase the readership and dissemination of ideas.

Athabasca University Press (AUPress) was Canada’s first open access, scholarly press, and provides all of its books for free download in PDF format and of course sells paper copies. These paper copies are offered for sale from the AUPress site, on Amazon and in epub format via sonybookstore. The download statistics for books and individual chapters are impressive and paper sales are about the same as scholarly publications from commercial or non open access scholarly publications.

For example my own edited book “Theory and Practice of Online Learning has been downloaded well over 90,000 times, read online by a large number of google book readers of the 20% offered at this site for free, and sales of over 1300 books. AUPress does pay royalties (about the same % of sales as commercial publishers). Interestingly I also got a small check from Copywrite Canada, from Universities who are paying for including chapters in reading packages- even though the students could download them for free!

I had a meeting with AUPress staff yesterday and we discussed ramping up production and promotion of the Issues in Distance Education series for which I serve as series editor. The series currently has 5 titles and 2 more “in press’.

If any readers are interested in producing a volume for this series, I hope you will contact me or the Press for author’s guidelines and further details. Like all AUPress books, each volume must survive two rigourous peer reviews. We are developing new guidelines for editors of edited volumes. The current practice is to accept publications only after the complete draft manuscript is submitted. This is problematic when an editor is trying to solicit chapter contributions and has no guarantee that the Press will accept the completed volume. However, an editor can communicate that the volume is being readied and hopefully published in open access format by AUPress, but there is no guarantee that any individual chapter or the whole book will survive the review process. The upside of this process is that a completed chapter or a book, can likely find an outlet someplace, even if fails AUPress’s review.

So please forward this post to any potential DE, online learning or even blended learning author wannabes and check out, download, or if you can afford it, order an AUPress book!

Next Phase for Tom Reeves

It was a very pleasant honor to be able to attend the retirement celebration and Design Based Research conference Georgia that honoured the  noted ed tech researcher, philosopher and activist – Tom Reeves.

The event was a very palatable love in for one of the most respect and loved academics in America. Tom has for 30 years been a professor of Instructional Technology at the University of Georgia. He has authored many articles, chapters and books, but is most known for his compelling presence as a speaker and advocate for meaningful research in education.

The conference featured a two day design based seminar  and conference for grad students and researchers focused on methods and results of design-based research. There were key note talks by Jan Harrington (Australia), Susan McKenny (Netherlands), Mike Spector, Michael Hannifin and Tom (University of Georgia).  There was also interesting mini sessions with scholars from Australia, Italy, South Africa  and from across the USA.

I was pleased to hear of many cases where design research is being used n local contexts from higher education, to classrooms, to professional education – with promising results. I also was busy scribbling references and resources (notably the 2007 Introduction to Design Based Research  booklet.) Tom’s own talk was both memories and inspiration as he demolished the focus on “rigour” that has marked the pendulum swing in the US as encapsulated in the No Child Left Behind and What Works thrusts of the Bush administration. Instead Tom championed, with many humorous and touching examples, the need for research that is relevant and makes a difference to real educators and most importantly real students.

There was a number of references for adding a fifth concurrent phase to design based research models based on a growing effort at understanding and promoting adoption of the design intervention in formal educational contexts.

I couldn’t help getting into a bit of a scrum with noted Ed tech author and publisher Michael Spector. Spector is the editor of the prestigious Journal Educational technology Research and Development, and the Handbook of Educational Technology, thus he is well positioned to present a “how to publish in Ed tech ” seminar. His talk ended with a listing of the 10″best” research journals in the ed tech field. However, only one of them was open access (Kinshuk’s Journal of Educational Technology & Society ). He dismissed all the other online journals as being of low quality. I had to jump up and dispute this claim and point out the growing list of Ed tech journals listed in the Directory of Open Access Journals. I further noted that as long as senior scholars like himself, keep reinforcing the value and sending papers to closed publications, the majority of people on the earth will continue to be  denied access to these works. It is especially relevant in that much of Tom’s talk focused on the lack of impact of ed tech research on real teachers and schools – perhaps because researchers who publish in closed journals effectively deny access to their work to the teachers who need and support their work.

The event ended in a tribute banquet, at which many of Tom’s current and former colleagues, friends and over 30 PhD students thanked Tom for the remarkable influence he has had on their lives.  The event featured Jazz,  blues and folk musicians and not just a few embarrassing stories dredged up from Tom’s past. Tom has traveled and spoken at all of the major educational conferences around the world, and there are few countries who cannot claim to have been influenced and blessed by Tom’s words and presence. Finally, I had my first tastes of southern grits and other southern delicacies as Tom and wife Trish opened their home for a farewell brunch.

Tom promises not to disappear in retirement and I doubt he will! I think he still has a few corny jokes and more than a few insights left for anyone looking for a keynote speaker at their next conference.

All the best Tom and thanks for a great career long, contribution to “research that matters”.