Every human undergoes a unique, life-long process of learning. Mostly, this process is informal, even accidental. Occasionally we choose to enter more structured learning environments – classrooms and seminars, schools and colleges – intentionally designed by more experienced individuals to foster, evaluate, and represent our progress. All of these experiences, from coincidental encounters to bureaucratized degree programs, make us who we are.

Everyone involved in this process has decisions to make. Individuals engaged in learning make choices constantly:

Should I read this, attend this lecture, read this blog post? Which school should I attend, course should I take, major should I pursue? What activities make me a better writer, problem solver, critical thinker, team member? How best to learn content, develop a skill, foster a habit of mind? Should I undertake this task, persist, give up? What is more important to me; validated formal education or authentic experience? How should I decide?

Faculty and staff charged with creating learning environments face their own array of questions:

Who are our students? What can they do when they arrive? What are shared learning goals for students in this environment? How will we, and they, know they have been achieved? How effectively do different activities foster learning for different learners? How can we effectively motivate each student to undertake appropriate activities? How might the learning goals of individual students extend beyond a base of shared goals? How should I decide?

Faculty administrators charged with supporting the creation of learning environments have choices to make at another level:

How should we decide whether to admit an applicant? What kinds of experiences should all students share? How do we make a campus into an effective learning community? What kinds of formal learning environments are most effective for different purposes? What kinds of learning require less formal environments? What should we require before awarding a degree? How should available resources be deployed? How should I decide?

When individuals make decisions in a reflective way, they draw on experience; both their own and, as they can, that of others. They also apply models of the world, a kind of shared informal theory. Most often this is simply tradition – a perception of common wisdom. In everyday life, experience and tradition are enough. For some kinds of questions, we have learned to consult experience and theory in a very special and powerful way: through organized, scientific research.

The role of research in establishing an evidence-basis for education

Today, few doubt the importance of scientific research for guiding our interaction with the physical world. When we want to design a plane, a cell phone system, or medical imaging device, we turn without hesitation to the results of organized research. In the humane sciences, the response is less clear. We allow some health care decisions to be informed by research, usually as interpreted by our doctors, but are much less inclined to turn to the research community in deciding on personal behavior, social policy, or education.

There have been good reasons for this hesitation. In the past, it was difficult to systematically observe a process as complex and distributed as education. Each of us learned primarily from our own experience and created learning environments by tradition; replicating the best of what we’d seen ourselves. What systematic research was done often seemed environmentally specific, especially when it contradicted the common wisdom established by our personal experience.

How could eliminating lecture lead to greater learning? That’s the only way I ever learned!

Over the last decade, the ground has shifted beneath our feet. The emergence and rapid spread of digitally mediated education makes it possible to observe and learn from the practice of education at scales unimagined a decade ago. It’s now possible for students, faculty, and administrators to inform their answers to all of the questions posed above using the methods of observational, and sometimes even experimental, research. Should they? If so, what needs to change to make this practical?

Learning from experience: using research to improve education

At its core, education research is nothing more than an intentional, systematic approach to learning from experience. In everyday life, we observe what happens to come along, evaluate it according to preexisting schemas, remember what happens to stick, and most important, remain unaware of what we don’t see. We learn from our own experience: a relatively narrow, individual evidence-basis for action. In research, we seek out specific phenomena to observe, measure them with carefully calibrated instruments, record and curate the results as data, and explicitly seek the novel and unexpected. Consulting the results of research, we can each learn from the experience of many, establishing a broad, perhaps representative, basis for action. Framed in this way, the attractions of using research to improve education seem obvious. We should do it all the time. Yet there are objections…

What prevents the use of research in education?

First, there are concerns about measurement. Educational interactions observed by an experienced individual are interpreted through a complex and often tacit set of expert filters. Such an individual expert uses direct experience to shape their understanding of education. Is it possible to for non-human instruments to capture what matters in education? Can they do this with fidelity adequate for understanding? What we measure and how we measure it will be central to determining whether a system of large scale data collection can meaningfully complement direct expert observation.

Second, there are concerns about generalizability. Education is both deeply social and essentially personal. Each learner is unique, engaging from birth in a set of learning experiences shared by no other individual. Experienced teachers understand this. When they can, they use a rich combination of information and experience to tailor their interactions with students. To what extent can observations made of diverse individuals in one context inform our understanding of learning for other individuals in other contexts? Where we conduct our research and how we probe its generalizability will be central to our ability to effectively learn from the experience of all.

Third, there are concerns about personal privacy. To learn from the experience of all, information about individual experiences must be shared. In today’s world, we all expect to keep some aspects of our lives private: indeed many countries have codified privacy as an inalienable human right. At the 2014 Asilomar Convening,[1] respecting this right to privacy was considered central to the ethical conduct of learning research in higher education. If we are to continuously conduct richly measured, generalizably broad research on learning, we must find ways to appropriately protect every individual’s right to privacy.

Measuring more important things in consistent ways

In the past, the measurements widely available for understanding education were quite limited in depth. The student record of the 20th century was created to support bureaucratic processes, rather than research. Enrollment records, grades, credit hours, and degrees are primarily accounting measures, eliding over most of the richness and individual variability of the educational experience.

Despite this, colleges and universities have invested enormous resources ‘instrumenting’ their campuses to measure these things. Curriculum committees and accreditation authorities police credit hours. Instructors maintain careful records of student enrollment and invest substantial expert effort summarizing a rich body of student work to report grades. Departments, schools, and colleges generate requirements, enforced by degree auditors. Registrar’s offices maintain permanent records of it all. As we ponder possible futures, we should not forget this enormous commitment. Our institutions have made major investments to gather and curate educational data like this for a century and more.

These transcripts are not, I would insist, useless for research. Many important realities of education have only been recognized through systematic research examination of traditional student records. Nevertheless, they form a thin, poorly conceived portrait of what happens in education, falling far short of the rich observations regularly made by expert instructors in the classroom. For 100 years and more, traditional transcripts have been the only data which all institutions of higher education systematically record. If we are serious about using research to improve education, we must do better.

Measuring everywhere and openly sharing what we learn

In the past, education studies have usually been conducted in restricted environments, most often a few classes on a single campus. This is especially true of studies which rely on data collected beyond the traditional student record. Given the enormous diversity apparent across the landscape of higher education, limitations like this encourage persistent doubts about generalizability.

Accepted habits of institutional secrecy have been one of the principal barriers to conducting studies across a wider range of environments. Faculty members and institutions resist external examination and comparison so strongly that they regularly hesitate to examine themselves. Our limited ability to collect rich beyond-the-transcript data across many courses and institutions has allowed doubts about generalizability to fester. If we are to test the generalizability of research findings, expectations of institutional secrecy and data sharing will have to change.

Rethinking the meaning of privacy protection

Within the practice of higher education, identifiable information about individuals has always been maintained and, when appropriate, shared with members of the learning community. The fact that students choose to join college and university learning communities, which exist to improve their lives, makes this openness seem natural. Faculty members know their students individually and evaluate their academic work in rich and often deeply personal ways. Admissions officers, academic advisors, and degree auditors review student transcripts holistically, selecting them for admission, guiding their progress, and approving their degrees. Students share personal information with these school officials because doing so is essential to their educational goals.

Protecting the personal privacy of individuals is deeply embedded in the practice of higher education. It is also encoded in law. In 1974, the Federal Education Rights and Privacy Act was passed to ensure that students (and when students are minors, their parents) can access, review, and correct records maintained by schools. FERPA rules generally require permission for release of student records, but include essential exceptions, both for school officials with a legitimate academic interest and for organizations conducting studies on behalf of the schools.

The Belmont Report of 1978 articulated a set of ethical principles intended to guide research involving human subjects. It led to the passage of 45 CFR 46, and the creation of Institutional Review Boards. These regulations specifically exempted most educational research from IRB oversight, including:

46(101)(b)(1) Research conducted in established or commonly accepted educational settings, involving normal educational practices, such as (i) research on regular and special education instructional strategies, or (ii) research on the effectiveness of or the comparison among instructional techniques, curricula, or classroom management methods.

Since the emergence of the Belmont Report and the establishment of Institutional Review Boards, the world of research in higher education has too often been treated as completely separate from practice. Higher education administrators and faculty members use a combination of tradition and personal experience to make choices about educational practices used with students every day. We might describe these choices as ‘clinical’. The only constraints on them emerge from traditional community standards as enforced by curriculum committees and accreditation boards. Occasionally educators choose to elevate their examination of an educational approach to the status of research – searching beyond their personal experience for generalizable patterns, learning from the experience of many, and sharing those openly with their community of peers. But this approach remains rare.

If we’re serious about using research to continuously improve education we have to recognize that everyone in our communities should be able to learn from the experience of all, end practices of institutional secrecy, and break down the separation between research and practice in university life.

Can we create a higher education community which learns continuously, collectively, from experience?

The key to creating a higher education community which learns continuously, collectively, from experience is to break down the perceived divide between research and practice. In recent years, the US Institute of Medicine has called for the creation of a learning health care system, “in which knowledge generation is so embedded into the core of the practice of medicine that it is a natural outgrowth and product of the healthcare delivery process and leads to continual improvement in care.”[2] If we’re to take full advantage of the transformative promise of research in improving education, we must work to create an analogous learning higher education system, in which knowledge generation is embedded into the core of educational practice.

How to do this?

There are technical challenges. To support a learning educational system we must do a better job of measuring what matters. Thoughtfully recording and curating digital traces of student behavior and engagement, along with the authentic product of their work (as learning management systems are beginning to do) would go a long way toward this goal. Learning to combine clickstreams and pageviews with problems solved, papers written, and projects completed should help us to create richer representations of these higher level goals. Achieving this needs to be a major research goal.

We must also protect privacy. The goal is to allow everyone to learn from the collective experience of the past – not to explore the particular history of any individual. The tools we create to support decision making by students, faculty, staff, and administrators must be built so that they provide access to extensive information while protecting personal privacy. This is not an impossible task, but it is one we must make a central goal from the start.

The greatest challenges are cultural. To make higher education a learning system, everyone involved is going to have approach their work in new ways. Higher education practitioners of all kinds – faculty, staff, and administrators – will have to rethink how they make decisions. Personal experience, anecdote, and tradition will have to give way to learning from the experience of all; to research at all its levels. Students too will need to change. They will need to accept the importance of participation in a learning education system, freely sharing their experience with others.

Conclusions: creating a learning higher education community

Everyone involved in education has questions about what to do and how to do it well. The best way to answer those questions is by learning from the experience of everyone who has come before. To make this happen, we have to take advantage of the information age in which we find ourselves and dissolve the walls between research and practice in education. We need to gather, refine, understand, curate, protect, and attend to the extensive data produced by modern educational systems. We must build tools which help everyone in our community use this data to answer the questions they have. And it’s not enough to do this just occasionally, when we’re doing “capital R” research. We need to do it all the time, as part of everyday practice.

When everyone involved in education is able to learn from the experience of all, we’ll have become the kind of inclusive, efficient learning community we aspire to be.

  1. See http://asilomar-highered.info/.
  2. LeighAnne Olsen, Dara Aisner, and J. Michael McGinnis, The Learning Healthcare System: Workshop Summary (IOM Roundtable on Evidence-Based Medicine) (Washington DC: National Academies Press, 2007).