Evaluation of NSF IMD Grant

CONTRIBUTIONS WITHIN DISCIPLINE

As an ESIE project, what is the "discipline" within which P2K made its contributions? At the broadest and most important level it can be defined as "science education reform." Secondarily, P2K's arena of endeavor is the prototyping and testing of new types and techniques of communication (or ICTs, "Information and Communication Technologies") in education. Third and finally, it is in the use of authentic, ongoing scientific research, and the work of actual researchers at real locations, as the context for secondary science education.

Recent studies of student learning (NAS/NRC NAP, 1996; AAAS, 1993) recommend that:

A follow-up study for the National Academy of Sciences (Elmore et al., 1997) found considerable evidence that high quality instructional materials embodying these criteria can and do improve students' conceptual understandings. This is the "current base of knowledge and pedagogy" within which P2K operates and within which its contributions should be understood. Previous sections of this Report have presented detailed quantitative and qualitative data bearing on these issues: this section summarizes P2K's contributions in broader terms.

A combination of evaluation data and anecdotal evidence showed that P2K materials improved student learning outcomes in both process skills and content mastery in critical ways:

  1. by engaging student interest, open and positive attitudes towards SMET resulted, thereby fostering a classroom climate in which real learning took place.
  2. By presenting current content in a real-world context, basic scientific principles became relevant and comprehensible.
  3. The use of multiple media connected with students with varied learning styles, while also empowering many students who experienced difficulty with traditional instruction.
  4. By watching scientists at work, on-camera and on-line, students saw first-hand the importance of the scientific method (broadly defined) and were able to parallel some of the procedures in their own classrooms.
P2K showed that learning key but otherwise abstract concepts in real-world context was feasible and effective. During the years of the NSF grant, P2K and its LIVE FROM specials developed a unique "interface" or model through which teachers and their students could experience ongoing, authentic scientific research which P2K transformed into "teachable moments" for middle school science. Infrared astronomy (LIVE FROM THE STRATOSPHERE), or the design of autonomous robots is, of course, the work of Ph.D. scientists, yet P2K was able to relate appropriate middle school science concepts and skills, such as the electromagnetic spectrum or wheels, gears and axles, to cutting edge research. NSF and other awardees have spent considerable time prototyping what have become known as "Student-Scientist Partnerships" (see TERC et al's report) but P2K demonstrated that less formal relationships between students and working researchers can still produce powerful learning experiences. (See teacher comments above, and following, on LIVE FROM MARS.)

Furthermore, P2K's model showed that instead of staging expensive "expeditions" like those of the JASON project (costing in excess of $4 million per year), or sending adventurers like those bicyclists who are the backbone of Classroom Connect's "Quest" projects (MayaQuest, AfricaQuest, AustraliaQuest, etc.), which are on-line events featuring one-time only excursions, it is possible to turn ongoing research at places like NSF's Palmer or McMurdo Stations in the Antarctic, into meaningful, replicable and sustainable teaching and learning opportunities. The use of real world contexts, and real scientists as interlocutors and roles models for students, is both less expensive and more authentic than those other models.

The P2K model not only proved effective in exposing students to specific content, but also in revealing the character, habits of mind, and procedures of contemporary scientific research. One teacher reported the reactions of his students to the scientists they saw at work in Antarctica or at NASA's JPL:

"I learned that scientists work hard, not only with their hands but with their minds.
Scientists don't just sit in the lab all day.
When they plan to put a spacecraft on another planet, they must think of everything!
These projects made me realize that scientists are more than the Far Side "white lab coats and beakers" stereotype.
They have to learn to work together as a team.
I learned that sometimes it is hard to get information to other people.
I learned how they use their tools and knowledge to overcome their problems.
I have a greater respect for those who give up portions of their lives to live in desolate places to do research.
They have to organize a lot of data so it can be accurately used for reference.
I learned that they don't always get the recognition they deserve."

Tim McCollum wrote about his students' reactions to LIVE FROM THE HUBBLE SPACE TELESCOPE, "The success of any lesson can often be measured by insights and sometimes profound comments from our students. In the following sample, all done by 13 and 14-year-old-students, I think you'll know what I mean. They all had a very positive impact. Such things as, 'I've learned that there's a lot more to astronomy than just looking through a telescope.' Or my favorite one, 'I used to think astronomers were nerds with no life; now I think they're pretty cool!' 'I felt our class was really part of the decisions made around the world, I felt we played an important part in the larger scheme of things.' And finally, 'Astronomy is not just looking at stars, it's looking at a much broader topic, an endless universe.' Those are pretty incredible thoughts from the minds and hearts of 13 and 14-year-olds."

P2K believes that at least some of the students who saw the LIVE FROM videos, or interacted on-line, will have had an experience that launches them on a career in science, or at least opens them to an appreciation of engineering and math that would otherwise be lacking. Many of today's researchers date their interest in science to watching Apollo 11 land on the Moon. James Thomson, pioneer of stem cell research, traces his passion for science to a more mundane conversation with an uncle, who was an enthusiastic rocket scientist for NASA: "Imagine, they pay me to do this!" (Time, August 20, 2001, p27) As can be seen in the LIVE FROM THE STRATOSPHERE Teacher's Guide, with its verbatim quotes from some of the KAO astronomers, through P2K students saw that scientists don't just work 9-5, and aren't just motivated by pay checks: they saw, perhaps for the first time, just as did James Thomson, that you could have a career and raise a family by doing something challenging, energizing and fun. We hope-and clearly only time will produce data to document this-that in years to come some of the USAP researchers working in the Antarctic, or using the nation's great observatories at Kitt Peak and elsewhere to study stars and galaxies, will date their first realization that this could be their career to seeing a researcher on camera on "The Ice" answer a student's question about a penguin, or-high above Earth in the middle of the night-respond with accessible information about why Jupiter glows in IR light.

P2K has also made contributions in terms of its model. The use of the "P2K Advocates" to spread information through workshops was described above. (Activities and Findings/Outreach.) NASA JPL sent one of its outreach staff to attend the LIVE FROM MARS summer workshop, and has since implemented a very similar program of Solar System Educators, combining face to face workshops with year-long, on-line support. NASA Quest continues to use "BIOgraphies", "Researcher Q&A", and "Field Journals" as part of its ongoing on-line projects, innovations developed first by P2K. Many Advocates and PBS stations have created local and regional EFTs, attributing their activities to participation in P2K/LIVE FROM and seeing its successes. In summary, during its IMD grant period, P2K both delivered substantive science content and demonstrated a model for 21st Century science education.

CONTRIBUTIONS TO OTHER DISCIPLINES

The use of public broadcasting and open access to the website meant that P2K/LIVE FROM was used productively by audiences beyond its initially-targeted middle school base.

Susan Cowles, a national leader in Adult Literacy, used both LIVE FROM ANTARCTICA 2 and LIVE FROM MARS with her classes. She found the inherent interest of the subject-matter motivated reading: just as happened with middle school learners, the real-world context triggered the mastery of important new skills. Her work was documented by NCREL in its "Captured Wisdom" videos and CD-ROM. Susan reports:

"The CDs and video tapes are used in a lot of trainings for adult literacy instructors. They are used so much that now I am recognized in workshops and hotel lobbies ...I'm glad that teachers are seeing the opportunities for using such materials in adult basic skills classes.

In addition to teaching, I maintain a collection of on-line resources in Science & Numeracy for the National Institute for Literacy. http://literacynet.org/sciencelincs My plan is to have a sub-page on materials about Antarctica, and I do plan to link to the LFA2 materials... Finally, adult literacy programs operate in the nation's correctional systems... I know from direct teaching experience that expeditions such as those with TEA and Passport to Knowledge really work with adult learners..."

P2K/LIVE FROM programs also proved effective with students attending "alternative" high schools. (See comments from Deb Regal, CO, and Jo Lynne Roberts, WA, cited above.)

While less in number than public school teachers, and not P2K's targeted constituency, EDC's survey data also showed that as many as 5% of the larger lists (such as LIVE FROM MARS) were home-schoolers, for whom the provision, via distance learning, of current content and pedagogy, proved important and appealing.

Lastly, materials originally developed and field-tested under the NSF grant are included in the PASSPORT TO Modules which have been licensed by SAFETY-NET, and are currently (2001-2002) being used by this new Star Schools organization. ("For the first time in the history of the United States Department of Education Star Schools Program, a five-year grant was awarded to support distance learning initiatives for the juvenile offender population and the adults who serve them. The Justice Distance Learning Consortium (JDLC) is a partnership between the Texas Youth Commission, the New York State Office of Children and Family Services and the Florida Department of Corrections. This Star Schools initiative will provide accessibility, resources and opportunities for students and teachers to use distance learning to literally redefine the educational process in correctional institutions. JDLC will develop and support a technology-based network, SAFETY-NET (Systems Applications For Educating Troubled Youth Network) for the juvenile offenders and staff in 63 educational facilities located in the three states. SAFETY-NET will represent the integration of two technologies: satellites and computers. JDLC will extend the use of both current and previously developed programming by re-broadcasting selected programs, re-purposing instructional video and print-based materials for use via secure Intranet and video-server technology, and by providing juvenile offenders and correctional educational staff with access to satellite-delivered programs from selected educational resources. All of this will be delivered via the SAFETY-NET network. From the SAFETY-NET website, safety-net.org.")

P2K has just (September 2001) heard that SAFETY-NET is now being distributed nationally, further extending the potential reach and utilization of these NSF-supported materials.

CONTRIBUTIONS TO HUMAN RESOURCE DEVELOPMENT

All P2K/LIVE FROM programming attempted to show a science community as "much like America" as possible. Comments from NSF's External Auditor, cited above, and from teachers such as Rhonda Toon and Lynn Hammond (also, see above), specifically refer to the inclusion of both male and female researchers. During LIVE FROM ANTARCTICA, P2K included a taped report on, and live interaction, with Sridhar Anandakrishnan, one of the few (if not the only) researcher of color in the USAP (in 1994-1995.) The selection of uplink classrooms also consciously reflected a diversity of backgrounds and ethnicities. Again during LIVE FROM ANTARCTICA, a classroom on the Rio Grande in Harlingen, Texas, in a relatively poor school district was chosen to interact with UT Austin researcher Ian Dalziell on location in the Transantarctic Mountains. To prepare for the broadcast, the students and their teacher (supported by P2K staff and consultants) researched why Dalziell believes that back in time Antarctica and Texas were connected, until plate tectonics drew them apart. Excellent, informed questions resulted and the national audience saw high-quality participation from the primarily Hispanic-American students.

Deaf students from Gallaudet High School explained via sign language and their teacher's translation how the Internet leveled the playing field in terms of their participation, and asked questions of Sridhar in the Antarctic. During LIVE FROM THE STRATOSPHERE, high school senior, Brian Scott, an African-American, served as a student host, interviewing researchers and passing along student questions. Teachers reported their students were impressed, and wanted to emulate his success. Kuiper Airborne Observatory telescope operator Juan Rivera wrote fascinating Journals about his all-important work maintaining the instrument, and what it took to recover when there were technical problems.

On-camera and on-line P2K made it a goal to select researchers who could be role models, as well as expert commentators, and-we believe-succeeded.

CONTRIBUTIONS TO RESOURCES FOR RESEARCH AND EDUCATION

We have previously reported (in "Activities and Findings/Training and Development") numerous national, state and local awards won by many teacher participants in P2K/LIVE FROM. In addition to the psychic boost to the teachers, many of these awards also included cash prizes. Most of the teachers applied their winnings to educational resources to benefit students in future years. Scituate, MA, teacher Charlie Lindgren was able to use his students' performance in LIVE FROM THE HUBBLE SPACE TELESCOPE to secure a $10,000 grant to support the ongoing Weathergate project, which now reaches teachers across the nation with meteorological science and activities. Numerous teachers, such as Tim McCollum, Detlef Johl, Ginny Dexter and others indicated that P2K projects helped secure additional computers and/or connectivity. Peggy Motes, from the Muncie Schools Planetarium reported, that:

"The Muncie Community Schools Planetarium was the first site wired for the Internet in our public school district... Through my activities with P2K, my professional growth and student participation has resulted in the complete renovation of the MCS Planetarium. The programs at the MCS Planetarium continue to benefit from the awards and grants we have received due to my participation in these on-going P2K educational projects. I know I personally found the "LIVE FROM MARS" workshop in Washington, D.C., useful because there I learned about the "Red Rover" Project. When I returned to Muncie, I was selected as an Outstanding Indiana Educator and received the once in a lifetime professional award from the Lilly Endowment, The Teacher Creativity Fellowship. During this fellowship, I set up the first "Red Rover" site in Indiana. Using the "Red Rover" Project, other teachers and students have studied about the surface of Mars and learned how to design robotic vehicles."

Not only teachers, but also students gained attention, validation and material rewards through their participation in P2K/LIVE FROM. Ginny Dexter, from a comparatively small and remote community in Northern California, reported to EDC's interviewer that over the summer of 1997, her students watched the landing of Pathfinder on Mars. "Well, as you know, everyone in the country was watching the landing, but these kids had the background to share with their parents. Kids listened to the debates afterwards too, 'rounded pebbles on Mars... What is the only way you can get a rounded pebble?' Her students had answers." That knowledge led to their success in a local competition: "...what we have done through P2K made students want to be life long learners... (her) local education store held an essay contest, 'What happened on Mars this summer.' It was a big deal. All of her students entered. They revised and revised and revised those papers. Ginny said that by the time they were done they never wanted to see those papers again. But Ginny promised, 'One of you will win.' Sure enough, one of her students did. Suddenly all of the revisions made sense to them. This was yet another way Ginny evaluated their understanding. Again, another incentive for her students to show what they had learned."

In a very real sense, P2K provides a model through which to transform the ongoing research activities of scientists across America and around the world into meaningful science content appropriate for secondary school students. P2K brought the wider world into classroom in ways resulting in genuine learning. Rosemary Thiebaut (7th grade science, Wyoming) said that: "The most important aspect of the program for me was having the support and focus to learn how to integrate the technology into my classroom. I gained this support from teachers in the discuss group." Through this technology, real world research and researchers and engineers spoke to her students: "It was very different (to other projects) because we had the sense, through the videos and the on-line resources, that we were talking to the experts; we were a part of what was going on at the moment. I can remember listening to my kids, and the kids knew Donna Shirley (then head of NASA's Mars Exploration Directorate, and one of NASA's most prominent female engineers) and what she does at NASA. This is (a hallmark) of the project. Having access to that kind of expertise as a single teacher and having it be easy to give that to the students makes this unique and the top of the line in terms of what I want from curriculum materials."

CONTRIBUTIONS BEYOND SCIENCE AND ENGINEERING

EDC's Year 3 report stated that "Few curriculum projects in science or any other discipline area are voluntarily picked up and adapted both above and below the middle school target as has been consistently done with these modules. In fact, PASSPORT modules are used outside school settings entirely-they consistently attract interested adults who are participating in adult education programs or are following along out of self-interest. This popularity range makes PASSPORT TO KNOWLEDGE rare, and worthy of study for that reason alone. The link forged by PASSPORT between what is going on in the classroom and what is going on in the larger world ensures that the relevance of science instruction is clear to the students." EDC continued, "Few projects in existence today address content, process and pedagogy in an integrated fashion. The scale of PASSPORT TO KNOWLEDGE, in terms of widespread availability and utilization in numerous contexts, is more characteristic of a national test bed or a design experiment implemented on a national scale."

P2K achieved national recognition for its innovations and successes:

The National Academy's Science Education Standards begins with a comment from Richard Feynman:

"The world looks so different after learning science. For example, trees are made of air, primarily. When they are burned, they go back to air, and in the flaming heat is released the flaming heat of the Sun which was bound in to convert the air into tree. And in the ash is the small remnant of the part which did not come from air, that came from the solid earth, instead. These are beautiful things, and the content of science is wonderfully full of them. They are very inspiring, and they can be used to inspire others." (NSES p. viii. Italics added.)

We think that during its initial 3-year NSF IMD grant, P2K helped students and teachers appreciate not just science content and how research is done, but also exposed them to science as a way of knowing how the world works, and appreciating the universe around us. We believe, and trust this Final Report has documented, that P2K/LIVE FROM not only landed-in the words of the Chinese proverb-some big fish, but that it helped teachers and students learn how to fish. We believe we achieved the specific goals of our original proposal, learned more than we could ever have expected about what really works in real schools with real teachers and real students... and adapted our project accordingly. PASSPORT TO KNOWLEDGE thanks NSF for its support, and hopes NSF now has data to document how the P2K/LIVE FROM model can, in turn, continue to support NSF's own goals.

Activities Findings Outreach Contributions