Evaluation of NSF IMD Grant

P2K's 1994 proposal to NSF ESIE/IMD (ESI-9452769) began with a comment from Lewis Thomas: "The worst thing that has happened to science education is that the great fun has gone out of it... Science is high adventure... the chance to catch close views of things never seen before, the shrewdest maneuver for discovering how the world works." (Thomas, 1981, emphases added.) Over the 3 years of the grant, P2K made learning high adventure, shared "things never seen before," and helped teachers excite and inform students about how the world works.

Specifically the proposal set the following goals:

  1. Provide 6-8 "electronic field trips" (EFTs) for elementary, middle and high school teachers and students to locations across America and around the world
  2. Reach students and teachers in schools across America in a high profile project demonstrating (to students, teachers, parents and policy-makers the effectiveness of new tools, technologies and pedagogical strategies to redesign, support and sustain science education in all our schools
  3. Develop hands-on inquiry based activities for the classroom, and contribute current content, tied to the National Academy's/NRC's NSES and AAAS/Project 2061 "Benchmarks" and state guidelines, to the existing science curriculum
  4. Encourage teachers and students to navigate in a rich and rewarding telecomputing environment
  5. Show that careers in science are possible goals to male and female students of all ethnic backgrounds
  6. Add visual context to computer network interaction, and
  7. Evaluate changes in student knowledge of, and interest in, specific content and concepts.
Results data cited hereunder comes from the project's External Evaluation by the Center for Children and Technology of the Education Development Center, Inc. (EDC, NYC and Newton, MA), from internal work by project staff, and from the External Auditor, Carl Pennypacker (HANDS-ON UNIVERSE), suggested by the NSF program officer, working with Jodi Asbell-Clarke, TERC. This section of the Final Report summarizes "Activities" corresponding with each goal: the following section provides statistical and other data bearing on "Findings", again organized by Goal.

GOAL: Provide 6-8 "electronic field trips" for elementary, middle and high school teachers and students to locations across America and around the world

Between 1995 and 1998, P2K created and distributed 7 electronic field trips (EFTs): LIVE FROM THE STRATOSPHERE (focusing on NASA's Kuiper Airborne Observatory and infrared astronomy), LIVE FROM THE HUBBLE SPACE TELESCOPE, LIVE FROM ANTARCTICA 2, LIVE FROM MARS (1995-6), LIVE FROM MARS (1996-7), LIVE FROM THE RAINFOREST and LIVE FROM THE POLES. In all, the EFTs comprised some 27 hours of broadcast television, aired over PBS stations and NASA-TV, as well as being distributed as ITFS or closed-circuit programming by school districts and educational networks. In addition, an 8 hour teacher training workshop for LIVE FROM MARS (a "Virtual Teacher Workshop" or V-CON) was organized in Washington, DC, and carried live by NASA-TV, with interaction from 43 sites in the USA and 16 international locations, facilitated via the Internet. Each EFT was supported by companion hands-on activities and websites: P2K refers to each suite of integrated multiple media materials as a "Module." Locations featured in the EFTs included NSF's Palmer Station off the Antarctic Peninsula; the heart of the Amazon rainforest (close to Manaus, Brazil); NASA's Jet Propulsion Laboratory, Pasadena, CA; The "Imaginarium" Science Center in Anchorage, AK (during LIVE FROM THE POLES, produced in support of NSF's National Science and Technology Week, 1997, whose theme was "Polar Connections"); the National Museum of Natural History and the National Air and Space Museum (both part of the Smithsonian Institution), and the Space Telescope Science Institute, Baltimore, MD.

GOAL: Reach students and teachers in schools across America in a high profile project demonstrating to students, parents and policy-makers the effectiveness of new tools to redesign, support and sustain science education in all our schools

Broadcast live or re-run on taped, the video programs aired on more than 260 PBS stations, and 50 state- or regional-education networks, thereby reaching more than 95% of all schools and homes in the nation. Based on PBS carriage statistics and other measure, P2K estimates that from 1.5 to 2 million students and teachers viewed and participated in each Module. The LIVE FROM MARS online mail list had some 4,300 individual subscribers, primarily individual educators. Portals to other educational mail lists were also subscribers, thereby reaching many more participants.

Articles and/or news reports referring to these EFTs, all duly crediting NSF for its role in the project, were featured in the national media, including ABC, CBS and CNN, the Washington Post, New York Post, Atlanta Constitution, Chicago Tribune, and local newscasts and papers.

Both NSF and NASA cited PASSPORT TO KNOWLEDGE as an example of the successful integration of research with education and outreach in internal documents and reports to Congress. (NSF/USAP FY 1997 Budget Report to Congress; NASA Office of Space Science Report, "Implementing Education and Public Outreach Strategy.")

P2K staff or participating teachers gave Keynotes and/or Workshops at national conferences such as ISTE's 1996 Tel-Ed, and many regional Science Teacher Association meetings, either through project staff, or via "P2K Advocates." (Additional details are provided in "Outreach Activities.")

P2K was a finalist in the Education category of the National (later the "Global") Information Infrastructure Awards for both 1996 and 1997, and was the winner of the 1997 EdNet "Hero" Award for Excellence in Educational Materials.

GOAL: Develop hands-on inquiry based activities for the classroom, and contribute current content, tied to NSES and AAAS/Project 2061 "Benchmarks," and state guidelines, to the existing science curriculum

Working with classroom teachers, education researchers and scientists actively engaged in field and lab. studies, P2K created or adapted some 178 hands-on activities published in a series of Teacher's Guides and/or online in HTML or PDF formats. All necessary blackline masters of student worksheets and teacher handouts were provided. The Activities were correlated with NSES and Benchmarks, using McREL's formulation, and were designed to illuminate and extend the research seen on camera or read about on-line. For example, during LFA2, marine biologist Robin Ross, seen on camera during the first program, collaborated with P2K to adapt an activity developed by educational consultant Meredith Olson observing the behavior of brine shrimp as an analog to krill behavior in the Antarctic. Arizona State University researcher Tad Day worked with P2K staff to create an original activity measuring UV-B's effects on plants. The project's multimedia Kit provided UV and control filters, and Tad and colleagues were subsequently seen working on their experiment at an island off Palmer Station, while students in Hawaii compared their results, during a live broadcast. Tad and the students interacted, on camera, about their respective findings. With such substantive learning resources included in each EFT, the Modules serve as 6-8 week curriculum "replacement units" (as TERC uses the term) for the middle school science curriculum, rather than being intended as either comprehensive full-year instruction, or supplemental enrichment. Each Module also suggests interdisciplinary connections from science to math, social studies, language arts, computers and technology and other classes.

GOAL: Encourage teachers and students to navigate in a rich and rewarding telecomputing environment

For many teachers P2K was a first experience with integrating online materials into regular instruction, and functioned as a kind of working in-service on the capabilities of the Internet. (It should be emphasized that in 1994-1995, the Internet and the World Wide Web were still relatively inaccessible and unknown to classroom educators.)

Teachers who have Used Other On-line Projects
 
LFA2 (N=389)
LFM (N=923)
LFRF (N=168)
None
66.4%
52.75%
28.3%
One
5.0%
8.85%
13.8%
A few
19.5%
23.68%
41.3%
Many
9.1%
9.81%
16.7%

Specific findings follow in the next section of the Report.

GOAL: Show that careers in science are possible goals to male and female students of all ethnic backgrounds

From 85-90% of teachers reported that P2K increased student excitement about scientific content, and in careers in science and technology, and gave a clear or some understanding about the topics covered

Rhonda Toon, a Presidential Teacher from Georgia, and now a technology facilitator for her region, wrote in Business Week:

... The program that has brought the most change in my classroom is Passport to Knowledge (PTK), sponsored by NASA and the National Science Foundation. Kids get to know working researchers. They read their journals online, have their questions answered, and watch researchers on closed-circuit TV from such places as Antarctica, aboard aircraft flying in the stratosphere, or at the Jet Propulsion Laboratory… The PTK crew has helped me to become a better teacher. But most important, they have helped me show rural kids in Georgia that they can become scientists...

GOAL: Add visual context to computer network interaction

Based on one of the PI's role as a Senior Producer and Director of Carl Sagan's COSMOS series and several NOVA specials, broadcast TV was always intended as a major and unique component of the project. It should be emphasized, however, that TV was but one element: hands-on activities, on-line interactions, assessment and teacher support were co-equal elements of a model that emphasized the integrated use of print, on-line and video.

GOAL: Evaluate changes in student knowledge of, and interest in, specific content and concepts

Student learning outcomes are clearly the single most important set of results for any NSF supported ESIE project. P2K's IMD grant period was carefully monitored by the Center for Children and Technology of the Education Development Center Inc., and yearly reports were submitted to ESIE. In addition, the Year 2 report and intermediate survey data has been published online at an open site, to invite teacher review and comment:

see http://passporttoknowledge.com/storm

(select EDUCATORS in the bottom navigation bar, then click through to RESULTS.)

EDC's reports included both qualitative and quantitative data, derived from a methodologically-innovative set of measures developed to adequately assess the Multiple Media format, the relatively large numbers of participants and wide-ranging content of P2K. EDC also provided formative input to PASSPORT TO KNOWLEDGE, resulting in the integration of assessment into the hands-on and other Activities suggested for the more recent Modules. Taken together the results show that P2K achieved the goals of its IMD proposal, and-if anything-delivered more than promised. (See "Findings" in the following section of the report for statistics and details.)

EDC developed measures during the 3 years of evaluation which they (and NSF's External Auditor, quoted at length in the following section) believe are state-of-the-art, and adequate to the multidimensional character of P2K.

Activities Findings Outreach Contributions