The 2000 National Doctoral Program Survey Released October 17, 2001
Tell a friend!

Survey Sample

The National Doctoral Program Survey is an observational study, not a controlled experiment. Participants were self-selected, and as a result their responses may not fully reflect the opinions of the entire graduate student population. Despite the self-selection, there is important evidence that the concerns expressed represent widely held student opinions rather than a small but outspoken set of negative voices.

1. Our survey reached a broad cross section of the graduate student population. The demographics of the survey participants, after controlling for discipline, are similar to the demographics of graduate students and recent cohorts of Ph.D.s (see below).
2. The vast majority of students expressed satisfaction with their programs (81%) and with their advisors (86%). Most would recommend their programs to others (80%).
3. A pilot survey using similar methods, when compared to a survey conducted using more traditional methods, showed very similar findings. The 1999 PhDs.org Graduate School Survey, on which our survey is based, shared a number of questions with a recent national study conducted at the University of Wisconsin-Madison and funded by the Pew Charitable Trusts (Golde and Dore, 2001). The Golde study obtained responses from 42.3% percent of all students surveyed: doctoral students in their third year or beyond in 11 disciplines at 27 leading research universities. Student assessments of their educational experiences as measured in the Madison study were similar to those in the PhDs.org study (see below). In fact, the comparison of the two suggests that our results, if anything, may be positively biased, as the responses in the Golde study were nearly uniformly more negative than those in the PhDs.org study.

Sample Demographics

We can compare the demographics of our sample to those of the survey population using data from the NSF's Graduate Students and Postdoctorates in Science and Engineering: Fall 1999. The comparisons are imprecise, as the NSF data do not disaggregate doctoral students from masters students, but they serve as a useful first approximation.

The primary mechanism used to publicize the survey was viral marketing. Once participants completed a survey, they were given the opportunity to forward a survey announcement to their colleagues and friends. In addition, participants were prompted for the email addresses of their department chairs and deans. A description of the survey, an invitation to participate, and a sample survey announcement were sent to the chairs and deans whose addresses we obtained in this fashion.

The viral publicity mechanism introduces a source of significant variability into the sampling process. Small actions by individuals can have large effects on the sample. For example, suppose a respondent sends an announcement to a friend in another department. The friend then completes a survey and provides the email address for the department chair. An announcement is then sent to the dean, who then forwards it on to all students at the institution. Thus, a single announcement sent by a participant can trigger the sending of hundreds or thousands of new announcements. The end result is dense sampling in some programs or disciplines, and sparser or no sampling in others.

The first two rows of the table above illustrate this variability. While the proportions of responses from the various disciplines approximates the fractions of students in each discipline, we see that our response rate in engineering is significantly lower than in other disciplines.

The viral variability does not appear to have had a large affect on other characteristics of the respondents, however. We see comparable sample and population demographic characteristics in the remaining sets of rows. The discrepancy in U.S. citizens in the physical sciences is largely due to the lower rate of participation by computer sciences, a field with a higher than average fraction of non-U.S. citizen students. The smaller percentages of underrepresented minority students in our sample are likely due in part to differences in the data: the NSF's numbers are for U.S. citizens and permanent residents, while ours are for U.S. citizens only.

The National Doctoral Program Survey is based on an earlier pilot project, the 1999 PhDs.org Graduate School Survey. A number of the questions in this pilot study were drawn from the work of Golde and Dore. Below we compare responses for questions that are common to this pilot survey and the Pew-funded Golde/Dore Survey on Doctoral Education and Career Preparation [1]. The Golde/Dore survey was conducted using more traditional methods and with the cooperation of the universities involved. Their response rates are on the order of 40%, much higher than those for the pilot. Despite differences in populations (the Golde/Dore work focuses exclusively on current students at a selected group of 27 institutions) and minor differences in the wording of questions, the results are qualitatively quite similar. In fact, somewhat surprisingly, the responses to the Golde/Dore survey are nearly uniformly more negative than those for this pilot survey, suggesting that our results may be slightly positively biased. (Table reproduced from the PhDs.org Graduate School Survey)

[1] Golde, C.M. & Dore, T.M. (2001). At Cross Purposes: What the experiences of doctoral students reveal about doctoral education (www.phd-survey.org). Philadelphia, PA: A report prepared for The Pew Charitable Trusts.

[2] Golde, C.M., personal communication, 2001.