A S S E S S M E N T I N F O R M A T I O N
How is Passport to Knowledge being used?
As stated in the previous section, this year saw an increase in the number of teachers who used PtK in their classrooms that used all three major components. While many adults and students who are unconnected to a PtK classroom utilization visit the web site as their sole PtK experience, most teachers indicate that they do view the video programs and access or download some or all of the teachers guide. Video Over 65% of the members of the e-mail lists who are classroom teachers indicate that they watched the associated PtK videos. Taped viewing (vs viewing the live broadcast) ranged from approximately 54% of the viewing for the LFA2 Program 1 "Oceans, Ice and Life" to 73% for the second LFM program "Cruising Between Planets." Again these differences are explained best by the slightly higher appeal of the LFM module for upper grades and the need for using tape in situations where the teacher uses PtK with more than one class period. LFM and LFA2 programs were carried by PBS stations and by cable companies who carry NASA TV. Across the various broadcasts, teachers used PBS or NASA TV in roughly even numbers and together these two sources accounted for over 80% of the audience with dedicated local or state educational networks (redistributing one of these two feeds) or obtaining a video cassette accounting for the rest. Over 75% of the respondents indicate that they are saving the tapes for repeat use at a later time. More teachers are indicating that they are downloading the teachers guide from the web site vs. ordering the hard copy or obtaining it in the full kit. Most teachers indicate that they have used at least one activity from the teachers guide before viewing the video (for instance, teachers indicated that had used on average 1.62 activities associated with the LFM "Cruising Between Planets" before they saw the program). The majority indicated that they used more than one as a follow up to a program. The activities in the guide continue to be highly rated by teachers both in their evaluation rankings and in their comments.
Table Twelve: Which activities did you do with your class prior to watching "Cruising Between Planets? N=579
On-line Over 80% of all survey respondents indicate that they have visited the associated web site and almost 60% say that they visit frequently (several times per month). The majority also says that they are using the web resources and/or the messages posted to the e-mail lists directly with their students as an instructional resource. We used the surveys to determine which features of the web site and which activities in the teachers guide were being used. Table Thirteen: Which of the following LFM components have you used? N=579
Table Fourteen: Which areas on the LFM Web site do you find useful? N=579
For the Live from Antarctica 2 program we asked similar questions. Table Fifteen: Which aspects of LFA2 did you use? N=78
The additional questions on design were asked because LFA2’s web site tried to address the different audiences using the site through providing somewhat different site views for these audiences. It also used frames to help clarify what was available on the site. While the web site has gained dramatically as a tool utilized by PtK participants, the mailing lists remain the central on-line tool for most users. This fits in with our experience with other projects. While people will visit a site when they need to get information or to catch up on something, they prefer to have those aspects that they want to see or participate in on a regular basis delivered to their desktop. E-mail is still a preferred tool for many. Others have some access to the web but e-mail is just more convenient for them. Both Live from Mars and Live from Antarctica 2 have several active lists. Table Sixteen summarizes the lists, purpose and subscriber numbers as of March 1997 for the two modules. Note that all of these lists with the exception of the PtK Advocates list, are open to subscription by any one and all of them are archived for reading on the web site. Because this last point allows non-subscribers to keep up with message postings, subscription numbers do not reflect total readership since some participants prefer not to receive these messages in their e-mail. Also some very popular K-12 web sites and e-mail lists such as the Global Schoolhouse Network serve as reflectors for the PtK lists. In these cases, one subscriber account is picking up messages and posting them to a reflector list that can have several hundred subscribers who do not show up in the numbers below.
Table Sixteen: Mail list subscribers
How are the Lists Used?
All messages posted to these lists have been coded by the evaluation staff first by which module they are dealing with, then by type of author and finally by category of content. Module classifications consist of: LFM (Live from Mars), LFA2 (Live from Antarctica 2), LFA (Live from Antarctica (original), LFS (Live from the Stratosphere) and LHST (Live from the Hubble Space Telescope). Author-type codings are mutually exclusive (one code per message): Staff, Expert, Advocate, Teacher, Student, Home Schooler (adult), Other Adult. It should be noted however that on most of the lists, and for most of their message postings, Advocates are participating as teachers (which they all are). So in any type of analysis, the Teacher category needs to have the Advocates added in to get an accurate representation of what teachers are doing with the lists. Content codings are not mutually exclusive. One message can have multiple codes assigned to it. Codes are: Testimonial, Critique, Suggestion, Project info, Response, Request, Resource sharing, Technical question, Content question, Teaching question, Student work sharing, Assessment-related, Sponsored activity, Teacher generated activity, Feedback to staff, Feedback to experts, Feedback to teachers, Feedback to students, Explicit reference to video program(s), Explicit reference to Teachers Guide, Off task Messages from the lists are collected by the evaluation staff as they are posted. A coding template is attached to the top of each message and codes are assigned after reading the message. Coded messages are converted to HTML format, stored on a web server and indexed using a full-text search engine. This permits staff to search on various combinations of codes or any string of text within the message header or body. This tool can be used to look at responses to particular initiatives, activities or events. Below in Table Seventeen is a general analysis of the teacher discussion lists for both modules. Table Seventeen: Live from Mars Discuss LFM messages N = 867 messages posted as of 5/18/97 Author Type Percent of Messages Staff 24.57% Expert 2.77% Advocate and Teachers 68.74% Student 0.12% Home Schooler 1.04% Other Adult 2.65& Content Type Testimonial 9.23% Critique 0.92% Suggestion 4.5% Project Information 12.57% Response 31.26% Request 11.19% Resource Sharing 38.41% Technical Question 4.96% Content Question 1.73% Teaching Question 2.08% Dissemination Question 0.46% Student Work Sharing 3.34% Assessment Related 0.81% Sponsored Activity 10.38% Teacher Generated Activity 7.83% Feedback to Staff 4.5% Feedback to Students 0.46% Explicit Reference to Video 9.46% Explicit Reference to Guide 5.07% Off Task 6.57% For Live from Antarctica 2 Discuss LFA2, the message count was 123 messages. Author Type Percent of Messages Staff 35.8% Expert 0.8% Advocate and Teachers 59.3% Student 0.8% Home Schooler 2.4% Other Adult 0.8% Content Type Testimonial 25.2% Critique 0.81% Suggestion 4.88% Project Information 18.7% Response 34.96% Request 19.51% Resource Sharing 20.33% Technical Question 2.44% Content Question 1.63% Teaching Question 4.88% Dissemination Question 0% Student Work Sharing 6.50% Assessment Related 0% Sponsored Activity 4.88% Teacher Generated Activity 0% Feedback to Staff 4.07% Feedback to Students 0% Explicit Reference to Video 37.4% Explicit Reference to Guide 12.2% Off Task 2.44%
Perceptions of Passport Modules In terms of curriculum integration, over 50% of the respondents indicated that they were teaching about space (in the case of LFM) or Antarctica (LFA2) and the overwhelming majority ranked the content as either closely related or related to the content they are currently teaching. For instance, within the 579 respondents to the Live From Mars video survey that closely followed the broadcast of the "Cruising Between Planets" program, we looked at grade level and program relevancy to the what was being taught in the classroom this year. Table Eighteen: How relevant was this LFM program to the content you are teaching? (N= 579)
Each survey that was used to generate the data in the tables also had some open-ended questions for respondent feedback. These responses are gathered in Appendix Five and sorted by module and into relevant utilization categories. They provide additional information on the perceptions of PtK by teachers (and parents, in some cases) who are utilizing it.
Teacher Case Studies We are augmenting the survey data with a two-year tracking study of 24 teachers. The pool was built from those teachers responding to either the postcards or the on-line registrations collected during the fall of 1996. All teachers who responded to either the postcard survey or the on-line registration forms were classified on two dimensions. First we grouped them by previous experience with using on-line curriculum projects in their teaching. Categories were no previous experience, one or two on-line projects, many. The second grouping was experience with Passport to Knowledge and contained categories for no previous experience, used one prior module, used more than one prior module. This two dimensional grouping created a 3 by 3 matrix. However, there were no teachers that occupied the cell for no previous utilization of on-line curriculum projects and high previous experience with PtK. Three teachers were randomly selected from each of the remaining 8 cells to create a pool of 24 teachers who will be tracked over the next two years. This study will help show professional growth and change as these teachers continue or increase their involvement with Passport to Knowledge. It will also give us some insight as to why teachers drop out or decrease their involvement. Data is being gathered through telephone interviews. The results of this will be used in our final report in Year Three. In the interim, some of these teachers are interesting to use as small case studies that can shed additional light on how teachers are using Passport to Knowledge in their classrooms. Ten of these cases are described below. The final report will have an expanded case study section which will include some of these teachers from the tracking study but will also include teachers who are exemplars of different types of curriculum integration and best practices. Our focus for the interviews with the tracking pool this year, which were conducted in the winter and spring of 1997, was on how the two PtK modules were integrated into the ongoing science curriculum. Is PtK bringing in new content or is it supplanting or augmenting existing units. Our questions looked at which components were used successfully or not so successfully and why this happened. We wanted to get an idea of the time frame or period over which use occurred. We also wanted to understand what PtK use looks like under varying technology access infrastructures and how the differing levels of teacher educational telecommunications experience noted in the surveys affected its implementation. At the same time, we wanted to see if PtK use helps support or justify classroom technology integration. A third grouping of questions looked at what students produced as a way of accessing how PtK impacts what students know and can do. What kinds of assessments are being done by these teachers? Although most teachers are excited to participate in projects like Passport to Knowledge, their ability to participate depends on whether the materials can be integrated into a very "packed" science curriculum. These teachers’ stories illustrate that PtK modules fit into various curricula in a variety of ways
|