Passport to Knowledge
Year Two Evaluation Report
August 1996 through July 1997
Live from Marsand Live from Antarctica 2
Center for Children and Technology
Education Development Center
96 Morton Street
New York, New York 10014
Table of Contents
One Framing Questions
Two Survey Forms used in Year Two
Three Postcards and Registrations
Four Science Assessment Standards for Student Work
Five Background Materials on Passport to Knowledge
Passport to Knowledge is an innovative educational program that focuses on middle school science learning by combining video broadcast events with extensive print and on-line resources as well as interactive applications of the Internet. The project is funded by the National Science Foundation (NSF) and the National Aeronautics and Space Administration (NASA). Each year, the project offers teachers at least two different multiple media "field trips" that range in length from several weeks to some that span the entire school year and beyond. Annually thousands of teachers use at least some of Passport to Knowledge’s components.
As part of a three year National Science Foundation grant under the Instructional Materials Development program, Passport to Knowledge has contracted with the Center for Children and Technology of the Education Development Center to evaluate the project. This report presents the findings gathered during the second year of the three years.
The purpose of the evaluation is both formative and summative: it describes who is using the program, how they are using it, and what is the project’s impact on teaching and learning. The evaluation involves developing and implementing a comprehensive array of documentation and evaluation strategies to provide critical feedback to the Passport to Knowledge staff as their curriculum development work evolves. At the same time, we are providing information and analysis to help the staff, the project funders, and the participants assess the project’s impact on student learning and teacher professional development.
This past year Passport to Knowledge produced Live from Antarctica 2 and Live from Mars. The research with teachers this past year reveals the following key points:
This evaluation report covers the second year of Passport to Knowledge’s three year grant from the National Science Foundation for Instructional Material Development. As part of this grant, the Center for Children and Technology at the Education Development Center is conducting an assessment of the challenges faced by Passport to Knowledge and its achievements in meeting those challenges. The work will culminate in a final summative report due in Year Three. However, as shown in this interim progress report, there is already clear evidence of the project’s impact on teachers and learners in a wide variety of situations.
Before presenting the details of the evaluation, it is important to set Passport to Knowledge’s work in a larger context. As this report will document, Passport to Knowledge is attempting to transform significant aspects of science education for middle school students. It is doing this by developing innovative instructional materials and experiences for students and placing them within a multiple medium communications infrastructure that leverages the respective power of television, video, and the Internet as educational tools.
At the same time, it is using this infrastructure to foster an engaged professional community that centers on teachers but also includes scientists, engineers, parents and other community members. This ambitious undertaking puts Passport to Knowledge squarely in the middle of the intersection of two major forces that are working to reshape American education. One force is the exponential increase in the purchase of technology in general, and computer-based telecommunications technology in particular, for use in education. The second force affecting education today is the widespread public interest in school improvement, the pressure for accountability and results, and the active debate in our society on how best to achieve substantial reform of our schools. These two forces need to be examined in more detail.
The United States, and in fact most countries around the world, are in the midst of a major investment to increase the use of technology in K-12 education. Whether one measures it in terms of financial commitment, or the re-alignment of human resources and energies within schools, or by the heightened public expectations around what technology will do for our children, the scale of this investment is enormous. During the 1996-1997 school year, the period covered by this report, there was significant acceleration in the number of teachers and students gaining access to the Internet.
When Passport to Knowledge began working on its NSF IMD grant in 1995, the National Center for Educational Statistics surveyed schools for the first time to find that 35% of public schools (mostly secondary) had some form of Internet access. In the short time from that survey to their latest data from the fall of 1996, access has grown to 64% of public schools reporting Internet access. The percentage of elementary and middle schools in comparison to secondary schools with access has increased, as has the level of access within classrooms within those schools (from only 3% to over 14% of classrooms). The gains have continued unabated since that survey and perhaps even accelerated in the past year. And this change in schools only mirrors the changes in connectivity taking place in the larger society in which schools are situated: business, community agencies, and homes are also connecting at a rapid rate.
Recent reports demonstrate this penetration. QED’s 1997 report on technology penetration in U.S. public schools for 1995-96 (using sampling techniques that are somewhat inadequate given the rapid rate of investment in many states), estimates that 98% of schools have access to computers (with an average student to computer ratio of 10 to 1), 97% have VCRs and 76% have cable TV access. Over 85% of the schools have access to computers with multimedia capabilities and 54% have CD-ROM drives. Satellite penetration has grown to just under 20% of our schools.
Spending on educational technology infrastructure increased at the local, state and federal level. This year marked the beginning of the President’s Technology Literacy Challenge program, which provides states with significant funds to direct to local education authorities for planning and implementing technology. This builds upon other sources of systemic funding being used to apply technology within education, such as the Goals 2000 block grants and Title One funds, as well as the significant funding allocations made by many state legislatures this year that explicitly target technology expenditures for schools.
1996 also saw the creation of the Universal Service Program as part of the Telecommunications Reform Act. This program, through its "E-rate" discounts for schools, is already having a profound effect in spurring planning and investments, even though its funds will not be available until 1998 at the earliest.
While there are growing concerns that all this rapid investment and desire for quick fixes might precede, or even hinder, the necessary planning required to ensure its optimal impact, all indications seem to point out that we are only near the bottom of a parabolic curve. The trends toward growing technology investment and the demands for a return on these investments that tail immediately behind them will continue to accelerate.
Coupled with this push for technology infusion is the national discourse on school improvement. This manifests itself in the work done on curriculum standards at local and state levels which in turn has benefited from the national work done by professional organizations such as the American Association for the Advancement of Science’s Project 2061 Benchmarks for Science Literacy and the National Research Council’s National Science Education Standards. This work has evolved from building a consensus on what students should know and be able to do at various levels to specific state and local descriptors that can be used for as the basis for curriculum and teaching decisions and for student assessment of progress. Forty-nine out of the fifty states have some sort of standards in place or under development.
Technology is often cast as a potential means, sometimes as even a necessary component, of successful educational reform. As stated in a recent monograph reporting TERC’s work on the Model Schools Partnership:
"Familiarity with the use of technology represents a necessity for our schools today; but it also represents an opportunity for educators to open new vistas for student learning. The technology that is now becoming available inexpensively – and will soon be ubiquitous – provides unparalleled opportunity to update our model of education. These technologies provide powerful tools that will enable us to expand learning opportunities for students and to support teachers in learning and adopting new professional skills."
Even materials that focus exclusively on connecting schools to local and wide area networks give credence to the role this technology can play in school change. The potential of networking for schools and districts has been depicted in many places. For instance, in NCSA’s report on their NSF NIE work, the authors state:
"Once districts and building networks connect to the Internet, the opportunities grow even larger. Students can share their work with others around the world, providing them access to diverse cultures and perspectives that they would not encounter in everyday experiences. Information is available on the world Wide Web that can provide students access to materials such as scientific journals and up-to-date research data that can take years for textbooks to offer. The Web can also provide access to mentors and experts that would not normally be accessible to children."
There are some voices of caution among the general noise and enthusiasm surrounding this technological magic bullet. Many observers note that technology, by itself, will do little to change or impact education. It is only when the technology is applied within the framework of specific teaching and learning at the district, school, and classroom levels that we begin to see educational transformations. Where it works, technology use is supported within schools by broader change strategies with ample teacher professional development that includes development within the classroom and during ongoing instruction. This often means focusing on the students as part of the technology integration process rather than just as ultimate beneficiaries of it at some point further down the road.
We now know that it is the particularities of the individual teaching and learning contexts – the orientations and activities of the teachers, students, schools, and families – that make the difference in desirable impacts from technology. Whatever else is effective, it is not the educational technologies per se. The social contexts in which they are utilized are all important. It is the blend of the technology, the content and the teaching strategies.
In light of these developments, projects like Passport to Knowledge can contribute more than just the specific instructional materials they have been funded to develop. They need to be examined for what we can learn about large-scale, implementation models for these types of resources and activities. Few projects in existence today address content, process and pedagogy in an integrated fashion. The scale of Passport to Knowledge, in terms of widespread availability and utilization in numerous contexts, is more characteristic of a national testbed or a design experiment implemented on a national scale. It is a concerted effort to address educational reform at a critical point in the teaching and learning process: the teacher and students interactions with stimulating real-world scientific endeavors. It uses a coordinated mixture of technologies to accomplish this but it also encourages the active participation of teachers and students as more than recipients of an instructional delivery system. It is this combination using various delivery formats while developing a human infrastructure to support its teaching and learning that makes PtK an interesting endeavor.
At the same time, it is important to acknowledge that Passport to Knowledge modules are not intended to be comprehensive whole year or multi-year curricula in science nor are they meant to entirely replace existing instruction. However they are providing concrete, direct experiences for both teachers and students in the meaningful utilization of education technology that fundamentally changes science teaching and learning in participating classrooms. The effectiveness of this project can and should be evaluated based on the learning environments it assists teachers in creating and the capabilities it provides to do things one could not do otherwise, rather than considering effectiveness as a function of the technology itself.
Passport to Knowledge’s significance lies in part in its widespread availability for immediate use in classrooms today. It does not require that the infrastructure be fully realized before teachers and students can learn how to use it. As enumerated herein, teacher teams at the building level, individual teachers, and even parents at home are choosing to participate in Passport to Knowledge. They are utilizing its resources to aid in their own professional development and they are improving the instructional opportunities for the learners for whom they are responsible. The numbers that detail PtK’s audience are impressive. Beneath those statistics lies the equally compelling story of PtK’s impact on individual classrooms and teachers. One of the most articulate and widely disseminated such story is that of Rhonda Toon, a public school teacher from Lamar County, Georgia as published earlier this year in Business Week:
At my rural Georgia school, over 60% of the students receive free or reduced-price lunch, and many of them know little of the world beyond our county. Textiles still play a part in the local economy, but mill closings have devastated many families.
My task as a teacher is enormous. How do I expose these children to the wonders and opportunities available to them? How do I keep bright, talented children focused on education?
One way has been to use technology. For six years, I have had the Internet in my classroom. I have never received any formal computer training. I did what many people do: I purchased a home computer and began to see the classroom applications it could have. But to get the Internet to my classroom, I had to write grant proposals, beg, and borrow…
… The program that has brought the most change in my classroom is Passport to Knowledge (PTK), sponsored by NASA and the National Science Foundation. Kids get to know working researchers. They read their journals online, have their questions answered, and watch researchers on closed-circuit TV from such places as Antarctica, aboard aircraft flying in the stratosphere, or at the Jet Propulsion Laboratory.
PTK includes hands-on student activities. My students have constructed aircraft to hold eggs and dropped them from cherry-pickers to simulate the work of NASA engineers. They have submersed their hands in icy water to study the effects of the cold at the South Pole. The PTK crew has helped me to become a better teacher. But most important, they have helped me show rural kids in Georgia that they can become scientists…
…This past year, as my concern over the need for technology integration has grown, I have gone a step further, leaving the classroom to become regional coordinator of the Gordon Georgia Youth Science & Technology Center. I will train teachers to use technology in their lessons.
This country has plenty of willing and able teachers, but they need resources. I know what it is like to have kids come into my room and not be able to name a single scientist. And I have cried when I have had students--after participation in the PTK projects--list not only the names of the scientists they met through this program but their classmates as well. They now see themselves and each other as scientists.
America needs a scientifically literate populace. The use of technology can help us achieve this goal.
Ms. Toon’s expression is notable but it is not unique, as a search on the growing archive of messages posted to the various discussion lists used within PtK modules will quickly reveal.
It is within this broader context that our evaluation takes on added interest: our task is not just to document the reach and the impact of Passport to Knowledge modules, such as this year’s two offerings, Live from Mars and Live from Antarctica 2, although that certainly is part of it. In the report that follows, we are presenting the initial work to draw out the larger lessons that can be gleaned from Passport to Knowledge’s position. Even at this interim stage in the three year evaluation effort, it is clear that Passport to Knowledge is doing something that has implications for all who anxious to see that the opportunities arising from this historic convergence of public investment, high expectations, and desire for serious improvement in education are not squandered.
This evaluation is a work in progress. Evaluating the impact of any technology on learning is challenging and evaluating it within a project like Passport to Knowledge, which is be used by many different types of teachers, in a wide range of grade levels, who are appropriating it in so many diverse learning situations, is all the more so.
This report, along with its final successor produced in year three of the evaluation, add to the work in overcoming these challenges to evaluation by addressing two critical areas as identified in the OERI January 1997 report "The effectiveness of using technology in K-12 Education: A preliminary framework and review." One area is distinguishing between the "hype", assertions, hopes and expectations, and data-based research findings that tell us what really works and what does not. Both in the formative feedback provided to the PtK staff throughout the year, and in the analyses contained here, our focus has been on addressing this issue. Secondly, it is often noted that technology’s impact on learning is difficult because there are few settings in which it is used optimally.
The accessibility of the Passport modules and their voluntary use by teachers in so many different settings offers a research window that goes beyond the interesting but limited-in-scale projects that characterize many of the current innovative technology applications to education. The need for this research has been underscored in recent studies and policy papers. The ETS study and the newly released Presidential report conducted exhaustive reviews of research on the impact of educational technology and have found only a small number of studies that have tackled the more pedagogically complex uses of technology. Both reports contain calls for new and ongoing evaluations of cutting edge technology projects while acknowledging the methodological and practical challenges facing such work. It is our intention that the evaluation of Passport to Knowledge will make a contribution to addressing these needs both through its findings and through the techniques we are evolving.
|Foreward||Year 2 Report Introducation||Who Is Using P2K?||How Is P2K Being Used||Student Outcomes|