Follow-up Survey of Physics Student Attitudes on Differing Modes of Access to Problem Solutions.

by

Edw. S. Ginsberg, Dept. of Physics, U.Mass.Boston, MA 02125-3393,
& Regina M. Panasuk, College of Education, U.Mass.Lowell, MA 01854.

Abstract

Use of end-of-chapter textbook problems for teaching and assessing concepts in introductory physics is widespread, as is restriction of student access to the solutions (Ginsberg, Edw. S., Regina M. Panasuk, & Simon George, 1997). We recently surveyed student attitudes on access to problem solutions (Ginsberg, Edw. S. & Regina M. Panasuk, 1998), sampling the enrollment in a large calculus-level course offered in spring 1996. For most of the course, student access was restricted to assigned problem solutions, placed on library reserve after discussion in class and collection of homework. During the last two weeks, however, solutions to all textbook problems in three chapters were made available. We seized the opportunity to survey student attitudes toward these altered circumstances, collecting responses from 128 students. The survey instrument contained two multiple-choice questions, one open-ended question, five Likert-scale questions, and space for comments. Results confirm a definite preference for freer access to solutions. Responses to direct-comparison questions suggest enhanced student engagement with problems, a desirable learning indicator.

Introduction

Instructors of the large introductory calculus-level physics course, used in the first survey of student attitudes (Ginsberg & Panasuk, 1998), restricted student access to solutions to textbook problems according to standard practice. That is, solutions to specific problems, assigned weekly for homework & classroom discussion, were placed on library reserve after student's papers were collected for grading. However, during the final two weeks of classes, instructors felt there was insufficient lead time to introduce the material in the last three chapters covered, while allowing for subsequent homework & classroom discussion, so this practice was abandoned. Instead, copies of the solutions to all of the textbook problems for these chapters were placed on library reserve. The only significant restriction on the use of the solutions was the demand on student time necessary for a library trip & check-out of reserve material. A follow-up survey was conducted to assess student reaction to the altered circumstances for access to problem solutions. We report the results in this paper. As stated in our previous paper, these student reactions relate to various aspects of college-level education for adult learners, such as learner control (Hiemstra & Sisco, 1990), self-direction (Candy, 1991), participation (Galbraith, 1990), & self-confidence (Usher, 1986).

Survey Instrument and Sample

The short follow-up survey instrument contained two multiple-choice, one open-ended, & three Likert-scale questions specifically related to student use of the solutions to problems in the three chapters covered during the last two weeks of the course. Two additional Likert-scale questions, one related to the standard access allowed earlier in the course, & one to unrestricted access in general, were included to verify attitudes sampled by the first survey instrument (Ginsberg & Panasuk, 1998). The complete questionnaire is reproduced in the Appendix.
The follow-up survey was administered at the last scheduled lecture of the semester. We collected 128 questionnaires, representing the surviving class attendance of roughly a third of the total enrollment of approximately 375. Of the number surveyed, 35 respondents, or 27%, reported (in Question #1) never using the available solutions. Therefore, their answers to the questions specifically related to usage during the final two weeks (Questions #2, 3, 5 - 7) are probably not significant. In passing, it may be noted that this percentage is lower than the 38% who reported never using the solutions on the first survey earlier in the course (Ginsberg & Panasuk, 1998). Data was tabulated both with & without this subset of non-users. In addition, about 82 respondents participated in both the first & follow-up surveys, as indicated by the truncated student ID numbers written on the questionnaires (there were a few duplicate ID numbers on each survey), so that a comparison of data between surveys is possible for this subset. For example, 21 students, identifiable on the first survey as non-users of the available solutions, had become users by the time of the follow-up survey.

Results

        The number of responses to the multiple-choice answers in Questions #1 & 2, & the number of responses to Question #3, are indicated in curly brackets on the survey instrument in the Appendix. The number following the slash, for Questions #2 & 3, is the number of responses excluding the never-used subset from Question #1. The results from Questions #1 show that 73% of the respondent sample made the effort to check-out the solutions from library reserve at least occasionally during the final two weeks of the course, when access to all the solutions for problems in three chapters was otherwise unrestricted. Nearly 44% used the solutions, in one way or another, for half or more of the problems studied. These percentages may be compared to the 62% at least occasional users, & 30% half or more users recorded in the first survey (Ginsberg & Panasuk, 1998), when access to solutions to only the assigned problems was restricted until after the collection of the homework & quizzes. Another way to characterize the frequency of usage data is to report the percentage in the occaisional & never categories. This is 56% during the final two weeks compared to 70% earlier in the course. Results from Question #2 show that among the user subset, 71% consulted the solutions to get help when stuck doing a problem, 68% to check work afterwards, 24% to get started, & 11% to explore problems beyond those assigned. There were too many uncontrolled variables in this survey to permit drawing any definitive conclusions from these percentages, but an increased level of student engagement with homework problems, attendant upon the freer mode of access to solutions, is indicated.
        The high percentage of respondents in the user subset who circled the second multiple-choice response in Question #2 (71%) is consistent with the written responses to Question #3, an open-ended question specifically addressing student's attitude towards the relative freedom of access to problem solutions for the last three chapters covered in the course. Slightly more than two thirds of respondents in the user subset answered this question, & a few of those who didn't, included equivalent information in comments elsewhere on the instrument. Also, some of the non-user subset of respondents wrote answers to this question, which we treated as general comments, suggesting why they chose not to utilize the problem solutions. A clear majority of the written responses (48 out of 63) indicated some degree of positive attitude derived from freer access to the solutions to the last three homework assignments. The remaining 15 out of 63 responses were not contrary, but dealt with other aspects variously related to the question.
        Student responses mentioned one or more of the following perceived benefits: having the solutions available while actively working on a problem helped them to learn more, to better understand the concepts involved, to work more thoroughly & productively on a greater number of assigned problems, with less wasted time & frustration, & to feel more confident, self-reliant, & better prepared for classes & quizzes. These responses reflect several aspects of learner self-direction, such as: a willingness & capacity to manage one's own learning in an environment allowing some effective control & personal autonomy by the learner (Tennant & Pogson, 1995); an occasion for students to think about their own learning, & to discover how they learn best (Usher, 1986); the opportunity for students to select an effective learning strategy from among alternatives (Candy, 1991). The following paragraph of excerpts illustrates these student sentiments.
        "Before this method was instituted, I was unable to see what I was doing wrong." [2744 - these numbers locate the source of a quotation in the database]; "I feel that I have learned & understood the material more fully, being able to consult the answers while I was doing it." [7238]; "I found it easier to learn on my own ... & I was more likely to do all the assigned problems, rather than abandoning the ones I couldn't get in frustration." [0681]; "Able to understand the context of the materials." [2087]; " ... If stuck on a problem, I didn't just stop & wait ... [until] discussion, but rather continued with the assistance of the problem solutions." [2694]; "When I got stuck, I used the solutions to get over the bump, instead of spending hours & hours on one problem." [9118]; "I learned more while doing the homework & I saved time." [5830]; "I was able to get through all the problems & understand the methods of solving ... [them]." [1439]; "I was not stuck as often ... didn't get as frustrated ... could move along & make progress." [0378]; "It's a lot more helpful, because I can get help whenever I want ..." [8572]; "I was able to do my homework with confidence instead of frustration." [2153]; "I knew how to do the problems before I went to discussions, rather than learning it two minutes before I took the quiz." [6086]; "I feel more self-sufficient." [0426]; "-was more prepared -easier to understand lecture" [0.07]; "...[greater] efficiency & acceleration in this class!" [0440]. (Samples other than positive are: "Study guides are addictive. I didn't get to that point, but easily could have." [0987]; "...The solution book doesn't really help me since I came to office hours." [0915]; "...If it were available since the beginning of the semester, it might of [sic] made a difference." [7372].)
        Table 1 displays the number of responses to the Likert-scale questions, for each point on the scale, from the entire sample of questionnaires, & from just the subset of users of the solutions during the last two weeks of the course. The average Likert-scale variable is also shown. The percent response distributions are plotted in Figure 1, using the entire sample for Questions #4 & 8, but just the user subset for Questions #5 - 7. Inclusion of the non-user subset in the distributions for Questions #4 & 8 makes little difference in the percentages, since these questions address attitudes which changed little from those probed by the first survey (Ginsberg & Panasuk, 1998). (Compare response distributions for Questions #4 & 8 from this follow-up survey with Questions #5 & 9 from the first survey.) Inclusion of the non-user subset in the analysis of Questions #5 - 7 does not affect our qualitative interpretations, but does lessen the impact of the numerical differences in the response distributions, since the varied access to solutions available at different times during the course was irrelevant for non-users.

TABLE 1. Number of Responses for Likert-scale Questions (with/without non-user subset).
_______________________________________________________________________________

LIKERT-

SCALE

QUES. #4 QUES. #5 QUES. #6 QUES. #7 QUES. #8

RESPONSE
_______________________________________________________________________________

1	23/17	9/3	21/9	49/28	5/2
2	24/20	8/4	10/6	19/17	7/6
3	35/25	33/20	43/31	28/20	11/8
4	13/10	29/25	25/22	12/11	18/11
5	17/10	33/30	15/14	6/6	74/55

_______________________________________________________________________________

TOTAL

112/82

112/82 114/82

114/82 115/82

_______________________________________________________________________________

AVERAGE 2.8/2.7

3.6/3.9 3.0/3.3

2.2/2.4 4.3/4.4

_______________________________________________________________________________

FIGURE CAPTION.

FIGURE 1. Percent response distributions for Likert-scale questions, with non-user subset for Questions #4 & #8, without non-user subset for Questions #5 to #7.


        Questions #4 & 5 directly compare student attitudes toward the standard practice of restricting problem solutions, in effect throughout most of the semester, with their attitudes toward the final two weeks of relatively free access to the solutions for problems in three chapters. The preference for freer over restricted access is unmistakable. Among the user subset, for example, there was 67% agreement to 9% disagreement for Question #5, whereas only 24% to 45% for Question #4. The reasons students give for this preference are probably the same as the perceived benefits mentioned above in the summary of their responses to Question #3. Those comments indicate that the availability of solutions, while students are actively engaged in learning through problem solving, provides them with immediate guidance or confirmation for understanding the concepts & methods involved. Students also believe that they have more control & flexibility in scheduling time for learning, are able to use their time more efficiently, experience less frustration, feel more self-confident, & are better prepared for classes & exams.
        Questions #6 & 7 queried the effect of access to solutions on the number of problems students studied. Among the user subset, there was 44% agreement to 18% disagreement for Question #6. Evidently, having solutions available allowed students to contemplate a greater number of their assigned problems. This is consistent with responses to Question #3, which suggested that freer access to solutions promoted more productive use of study time. For Question #7, however, there was 21% agreement to 55% disagreement, that is, only a minority of students took the time to explore additional problems which were not assigned. (The size of this minority, compared to the number of students who circled the fourth multiple-choice response in Question #2, which gives similar information, may only reflect the difference in the type of question.) We hesitate to speculate on the full significance of these figures, without knowing the corresponding data for the time period when access to solutions to unassigned problems was denied, but 10 to 20% of active students doing extra work isn't bad.
        Question #8, essentially the same as Question #9 on the first survey instrument (Ginsberg & Panasuk, 1998), addressed students' attitude toward totally unrestricted access to problem solutions. The result was the same, an overwhelming majority in favor of unrestricted access, 80% agreement to 10% disagreement, in either the entire sample or the user subset. Evidently, the desire of students to exercise freedom & responsibility themselves, in managing their own learning processes, did not change.
        About 30% of the respondent sample used the space provided in the survey instrument for general comments, & as noted above, a few general comments were included in answers to some of the questions. Most of these remarks reinforce attitudes already mentioned in connection with specific questions, but some raised other interesting issues. Thus, the largest fraction of comments (over 60%) reiterate the majority response to Question #8. Some (about 20%), particularly among the non-user subset, noted that the time expenditure required for trips to & from the Science & Engineering library was a hardship, or didn't fit into busy schedules. A few (less than 10%) expressed attitudes like those reported on our survey of physics instructors (Ginsberg et al., 1997). For example: "Solutions are a good thing & should be available, but ... not ... until after the assignment is turned in." [9344]; "Having a solution manual would make it easier to not do problems myself!" [6275]. While some students share these sentiments with instructors, however, they also believe that the responsibility for using problem solutions should be theirs. For example: "...If a student wishes to see solutions before the due date, [he or she] should have the option of buying a book." [2839]; "College is a time when restriction of personal liberties (including the freedom to browse the solution manuals) is not an effective tool in teaching..." [0440].

CONCLUSION.

        The results of this follow-up survey of 128 science & engineering students, administered to those attending at the conclusion of an introductory, calculus-level physics course, clearly demonstrate students' preference for unrestricted access to textbook problem solutions, confirm attitudes observed in a larger sample of students surveyed earlier in the same course (Ginsberg & Panasuk, 1998), & suggest that freer access probably has some pedagogic advantages.
        * In a pair of Likert-scale questions offering a direct comparison (Questions #4 & 5), a 67% majority of students favored the freer access available during the final two weeks, whereas only a 24% minority favored the standard restricted access in effect previously in the course.
        * Consistent with the earlier survey results, an overwhelming 80% majority of students felt they could learn better by owning a complete solution manual to all textbook problems.
        * An increased percentage of students utilized the homework problem solutions in some way, 73% when access was freer compared to 62% when restricted. Moreover, a large fraction of these students, 44%, reported studying a greater number of assigned problems when freer access to solutions was permitted.
        * Numerous student written responses state that freer access to problem solutions contributes to a deeper understanding of physical concepts & problem solving methods, to a more efficient use of study time, to fewer feelings of frustration, & to a greater sense of accomplishment.
        These attitudes, together with the enhanced level of student engagement with homework assignments, indicate possible pedagogical advantages of free access to problem solutions. Whether these perceived benefits can be verified in actual classroom studies & learning outcome assessments will be the subject of further research.

ACKNOWLEDGEMENT.

        We thank William Skocpol & his colleagues at Boston University for the opportunity to conduct this survey, &, of course, the Physics 211 students who participated. We are also grateful to Brian Clarridge, from the Center for Survey Research at UMassBoston, for advice on formulating the survey instrument.

REFERENCES.

        Candy, P. C. (1991). _Self-Direction for Lifelong Learning_. San Francisco: Jossey-Bass Publishers.
        Galbraith, M. W. (1990). "Attributes & Skills of an Adult Educator". In Michael Galbraith (ed.) _Adult Learning Methods_. Malabar, FL: Krieger Pub. Co.
        Ginsberg, Edw. S., Regina M. Panasuk, & Simon George, (1997). _Survey of Physics Instructor Attitudes on Student Access to Problem Solutions_. Electronic Journal of Science Education [Online]. Available: http://unr.edu/homepage/jcannon/ejse/ejsev2n2.html [1997, December].
        Ginsberg, Edw. S., & Regina M. Panasuk, (1998). _Survey of Physics Student Attitudes on Access to Problem Solutions_. Electronic Journal of Science Education [Online]. Available: http://unr.edu/homepage/jcannon/ejse/ejsev2n3.html [1998, March].
        Hiemstra, R. & Sisco, B. R. (1987). _Individualizing Instruction: Making Learning Personal, Empowering, & Successful_. San Francisco: Jossey-Bass Publishers.
        Tennant, M., & Pogson, P. (1995). _Learning & Change in tne Adult Years: A Developmental Perspective_. San Francisco: Jossey-Bass Publishers.
        Usher, R. (1986). "Adult Students & Their Experience: Developing a Resourse for Learning". _Studies in the Education of Adults_, 18(1), 24-34.
        

APPENDIX. Survey Instrument, with Multiple-choice & Fill-in Question Response Data.
_______________________________________________________________________________

BU SOLUTION MANUAL FOLLOW-UP QUESTIONNAIRE - SPR.'96

Please Read: This questionnaire is designed to gather information about how best to provide you with access to the solutions to problems in your textbook. Your answers & comments are important for this project. Thanks for responding.

Enter the last four digits of your student number: __ __ __ __ .
(This questionnaire is strictly confidential & will not affect course grades in any way. Some identification is necessary only for correlation of results with other statistical data.)

Instructions: Please answer the following questions both by circling the most appropriate numbered response, and by writing answers & comments in the spaces provided.

1. During the last two weeks, how frequently did you consult the solution to a problem in chapters 15, 16, & 9 (or a xerox copy thereof) on reserve in the library?

1. for almost every problem you studied					{15}
2. for a majority of those problems					{21}
3. for about half of the problems					{20}
4. for an occasional problem						{37}
5. never								{35}

2. During the last two weeks, how have you used these solutions? (Circle all that apply.)

1. to get started doing a problem					{23/22}
2. to get help when stuck doing a problem				{67/66}
3. to check a method or answer after doing a problem			{65/63}
4. to explore a problem which was not assigned				{10/10}

5. other (please describe)______________________________________________{2/2}__
_______________________________________________________________________________

3. Please describe any change in your approach to, or the outcome from, studying problems in chapters 15, 16, & 9 during the last two weeks, compared to earlier in the course (for example, as a result of the change in scope & availability of the solutions).________________________________________________
_______________________________________________________________________________
_______________________________________________________________________{68/63}_
_______________________________________________________________________________
_______________________________________________________________________________







Please answer the questions on the other side of this page.


_______________________________________________________________________________

Instructions continued: Please indicate disagreement or agreement with the statements below, on a scale where 1 = completely disagree, 2 = disagree, 3 = neutral, 4 = agree, & 5 = completely agree.

								AGREE--->
								<--DISAGREE

4. The way the solutions to only the assigned problems were made
available earlier in the semester is about right for my way of

learning the concepts behind the problems.

1 2 3 4 5

5. The way the solutions to all the problems in chapters 15, 16, &
9 were made available during the last two weeks is about right

for my way of learning the concepts behind the problems.

1 2 3 4 5

6. During the last two weeks, I studied a relatively greater number
of the assigned homework problems from chapters 15, 16, & 9, than

from chapters covered earlier.

1 2 3 4 5

7. During the last two weeks, I studied a relatively greater number
of problems from chapters 15, 16, & 9, which were not assigned,

than from chapters covered earlier.

1 2 3 4 5

8. I could learn better if I owned my own copy of the complete
solution manual to all the problems in the textbook, which I

could use whenever & however I wished.

1 2 3 4 5

_Comments_:(Use the space below to record further thoughts about the subject of this questionnaire.)









Did you follow the instructions & complete all of the questions on both sides of this page? Thanks for your cooperation.
_______________________________________________________________________________