 |
 |
Saturday, Nov 7
|
|
|
|
PSID User Guide
For a panel study, issues of nonresponse bias and representativeness
of the sample are crucial. Maintenance of high response rates,
careful cleaning of the data, and monitoring of data quality are all
high priorities for the PSID. Over the years, the Board and staff of
the PSID have encouraged a number of studies of PSID data quality.
The results of these analyses have provided reassuring evidence about
the validity of the data and the absence of substantial nonresponse
bias.
RESPONSE RATES
As noted at the end of the "Field Procedures" chapter, the PSID takes
a number of special measures to try to ensure high follow-up response
rates. Annual response rates have been exceedingly high in every year
except the first. In 1968, the PSID's first year, 76 percent of
sampled families were successfully interviewed. In 1969, interviews
were attempted with the heads of family units containing adults who
were members of 1968 interviewed families. The response rate in 1969
was 88.5 percent. Since 1969, annual response rates have ranged
between 96.9 and 98.5 percent. With a minor exception in 1990, no
attempt has been made to recontact attriters from previous years.
Even small attrition from wave to wave cumulates over time. As of
1988, the response rate for individuals who lived in 1968 households
was 56.1 percent. The level of cumulative response is sufficiently
low to raise concerns, and this has prompted direct investigation of
possible attrition biases.
UNICON STUDY
In 1982, the Unicon Research Corporation was commissioned by the
National Science Foundation to conduct comparisons of the descriptive
characteristics of individuals who had attrited and those still
remaining in the panel, and to estimate a series of models of
earnings, labor supply and migration using data from early panel
waves to see if subsequent attriters differed from respondents in
behavioral terms. We quote directly from their results (Becketti et
al., 1988; 490-491):
"In this article we examined the dynamics of participation in the
PSID and considered whether attrition has affected the
representativeness of the PSID. We found some observable variables
that are correlated with attrition, but these variables explain only
a negligible portion of the attrition in the PSID. We found no
compelling evidence that attrition (or entry) has any effect on
estimates of the parameters of the earnings equations we studied.
The 1968 PSID is quite unlike the population of the United States if
we use the CPS as a benchmark..1 Weighting the PSID with the weights
supplied by ISR goes a long way toward making the PSID sample
resemble the CPS sample. While there are statistically significant
differences in the empirical distributions of observable
characteristics, most of these differences are of no practical
significance or can be explained by known differences in coding of
answers across the two surveys. For some variables, particularly
income and education, there is some reason to believe that the
reports in the PSID may be more accurate than those in the CPS. At
any rate, the PSID participants behave almost identically,
conditional on their observed characteristics, to participants in the
CPS."
LILLARD-WAITE STUDY OF MARITAL HISTORIES
As part of a larger study of marriage and divorce, Lee Lillard and
Linda Waite conducted an analysis of the quality of panel and
retrospective marital histories in the PSID. Again, we quote directly
from their report (Lillard and Waite, 1989: 252-253):
"Our comparison of panel and retrospective histories produced a
detailed picture of the agreements and disagreements between the two.
To summarize briefly, we found substantial levels of agreement on
marital status as of the first survey interview, and substantial
agreement on the occurrence of the first marriage. We found that the
dates of first marriage matched best for those who were either
married as of the initial interview or who married during the survey
in the most typical pattern--living at home until marriage and then
moving out. For these people dates from the panel and retrospective
histories matched very well indeed.
Disruptions also appeared to be captured well by both types of
histories, although we do observe disruptions in the panel that are
not reported in the retrospective history and respondents report a
substantial number of disruptions that the panel history misses. For
those disrupted by both histories, dates of disruption match within a
year for three-quarters; we suspect the other quarter are reporting
on two different events.
...[O]n balance this data set is among the very best for studying the
beginnings and ends of marriages. The large sample at all ages, the
long panel period, the wealth of other information, and the multiple
measures of the events in question all make the PSID an excellent
source of information on marriage and divorce."
CURTIN, JUSTER, AND MORGAN STUDY OF WEALTH
As part of a general assessment of the quality of wealth data from
surveys presented at the 1988 NBER Conference on the Measurement of
Saving, Investment and Wealth, Curtin, Juster and Morgan (1988)
evaluated wealth data gathered in the 1984 wave of the PSID, the 1983
Survey of Consumer Finances, and the 1984 Wealth Supplement to the
Survey of Income and Program Participation (SIPP). A number of
quality dimensions were investigated: sample and questionnaire
design, response rates and nonresponse bias, ability to represent the
upper tail of the income and wealth distribution, the size of
measurement error, the importance of item nonresponse and
imputations, and the degree to which the household survey adequately
represents national wealth. They conclude (p. 544) that:
- Measured against the standards set by previous household wealth
surveys, all three of these data sets stand up quite well. They do
not differ substantially among themselves when it comes to measuring
total wealth and the distribution of wealth in the great bulk of the
U.S. population.
- The unique design characteristics of the SCF[Survey of Consumer Finances]
give it the highest overall potential for wealth analysis
of the three data sets examined....Comparing PSID to SIPP, one gets a
mixed picture, but, in general, PSID had the advantage. Although its
basic sample design is less well suited to measuring wealth than SIPP
(because it oversamples low-income families, for whom wealth holdings
are relatively unimportant), its general descriptive characteristics,
taking SCF as the benchmark, look to be closer to actual population
characteristics than are those of SIPP. Although PSID is not able to
describe the details of wealth holding nearly as well as SIPP because
of its highly aggregated nature, its measurement error
characteristics look to be consistently better than are those of
SIPP. The PSID has a lower item nonresponse rate than SIPP and thus
less need to construct imputed values, and it appears to be a
somewhat closer match to external control totals."
Research papers periodically provide additional data on the
representativeness of the PSID sample. In an article on PSID data
quality Duncan and Hill (1989) compared 1980 official program totals
and PSID reports of aggregate transfer income of various types. They
found that the PSID accounted for 92 percent of income from the Aid
to Families With Dependent Children (AFDC) program, 84 percent of
Supplemental Security Income and 85 percent of Social Security
income. As a frame of comparison, Current Population Survey reports
for calendar year 1979 show that the CPS accounts for about 77
percent of AFDC, 69 percent of Supplemental Security Income and 91
percent of Social Security (U.S. Bureau of the Census, 1983, Table
A-2, p. 216). The Census Bureau's Survey of Income and Program
Participation does considerably better than the CPS in matching up
with program aggregates than the CPS, accounting for about 79 percent
of Aid to Families With Dependent Children, 94 percent of
Supplemental Security Income and 101 percent of Social Security (U.S.
Bureau of the Census, 1985, Table D-3, p. 47).
As part of an analysis of the consequences of teenage childbearing,
Duncan and Hoffman (forthcoming) compared high school graduation and
marriage rates of black and white women in the PSID (at age 25) and
Current Population Survey (at ages 25-29). Although there is some
tendency for modest but persistent differences in some of these rates
(e.g., black marriage rates are higher in the CPS than PSID; white
schooling rates are somewhat higher in the PSID than CPS), the trends
for both racial groups track fairly closely over the two decades.
As part of a research proposal submitted to the National Institute on
Aging, Ken R. Smith compared the mortality experience of the PSID
sample from 1968 to 1984 with life tables for the U.S. taken from
1980 Vital Statistics sources. He found close agreement in the
five-year survival rates calculated from the two sources.
A crucial component of the quality of data from any survey such as
the PSID is the validity of responses to the questions posed. To
investigate this, the National Science Foundation, at the urging of
the Board of Overseers, funded a two-wave validation study of the
PSID instrument. Attempting to validate responses from actual PSID
respondents was judged too costly, so the strategy adopted was to
secure the cooperation of a large firm, interview a sample of workers
(about 500) from that firm using the PSID instrument and then,
whenever possible, to check carefully the responses recorded in the
interviews against actual company records.
Evidence from the validation study sample (detailed in Bound et al.,
1989) shows that the amount of measurement error in cross-sectional
reports of annual earnings is rather low, with the ratio of
error-to-total variance ranging from .15 to .30, depending on the
year of measurement. Error in reports of annual work hours is higher
(.28 to .37), while error in reports of hourly earnings, obtained by
dividing annual earnings by annual hours, is disturbingly high (.67
to .69).
Although annual earnings were reported fairly reliably, it was also
discovered that workers with lower-than-average earnings tended to
overreport and high-wage workers to underreport their earnings--a
covariance almost always assumed to be zero in measurement error
models. This covariance reduced from 18 to 24 percent the biasing
effects due to errors in measuring earnings when earnings is a
right-hand independent variable. Mean-reverting error also produced
biases to right-hand side variable coefficients when annual earnings
is a dependent variable that ranged from 10 to 17 percent. The
restricted variability of true earnings from the single-company
sample probably leads to an overstatement of these biases.
Furthermore, the validation data set also showed a surprisingly small
decrement to reliability when going from cross-sectional measures of
earnings level to panel measures of annual earnings change--there was
more "news" than "noise" when earnings were differenced over either
one- or four-year intervals.(Note). Reliability was also fairly high in
panel reports of change in annual work hours. Indeed, apparently
turbulent employment conditions produced cross-sectional reports of
earnings and hours in one of the survey waves that were less reliable
than the corresponding change measures.
The company sample also provided validation for retrospective reports
over a two-and-a-half year period of spells of nonemployment with the
firm. It showed that only one-third of the spells of nonemployment
appearing in company records were reported in the interviews. Shorter
and more distant spells were less likely to be reported, although the
fraction of presumably salient longer and more recent spells
unreported still exceeded one-third. Furthermore, the incidence of
reporting error appeared to be correlated with typical right-hand
measures such as age and schooling. Thus, all of the ingredients for
coefficient bias due to measurement errors would appear to be present
in unemployment event-history data.
In 1984 the PSID began coding information on labor-force status and
program participation on a monthly basis. As has been found in other
studies where the measurement period (e.g. month) is less than the
length of the reference period (e.g. year), observed transitions tend
to concentrate at the beginnings and ends of the reference period.
Hill (1987) compared the PSID with the Survey of Income and Program
Participation (SIPP) in terms of the disproportionate concentration
of transitions at the 'seam'. Perhaps because of the PSID's longer
reference period, he found the extent of seam problems appreciably
greater in the PSID than in SIPP data--especially for Food Stamp
recipiency.
Taking advantage of overlap in the 1984 and 1985 PSID reference
periods, Hill used dual reports of employment status for the same
month to examine individual characteristics associated with 'seam
amplifying' and 'seam attenuating' inconsistencies. Age and race were
found to be very strong predictors of seam amplifying
inconsistencies--with Blacks and older individuals having
significantly higher rates of concentration of transitions at seams.
Gender and income, on the other hand, were the sole significant
predictors of 'seam attenuating' inconsistencies--with high income
females exhibiting a greater propensity for this type of response
error.
The extent to which these types of response errors affect the
estimated parameters of event-history models has not been fully
worked out. Hill and Hill (1986), however, have demonstrated that
with SIPP data whether the week in question is a 'seam week' is by
far the most important predictor of transitions from unemployment and
the existence of excessive seam (or insufficient within wave)
transitions has profound impacts on the estimated survival functions.
Interestingly, despite a smaller sample size, the chi-square
goodness-of-fit statistic of the proportional-hazards model was more
than twice as high for the PSID as for the SIPP. This suggests that
the original over-sampling of low-income and minority households in
the PSID has notable analytic advantages.
Taken as a whole, these different studies examine a variety of
aspects of data quality, and the general results are supportive of
the PSID data being valid and not subject to major nonresponse bias.
Still, an analyst of any data set should be sensitive to
possibilities of low validity or nonresponse bias for his or her
particular analysis.
1. AUTHOR'S NOTE:
This refers to the deliberate oversampling of low-income families in
the PSID's initial wave.
2. AUTHOR'S NOTE:
These design characteristics include a long questionnaire focused on
components of wealth and an oversample of high-income households.
3. AUTHOR'S NOTE:
Bound and Krueger (1989) report a similar finding for the March 1977
and 1978 Current Population Survey, using Social Security earnings
records for those same individuals for validation.
4. AUTHOR'S NOTE:
There is no evidence, however, that the measurement errors in
measures such as earnings are higher in the PSID than in other
surveys. Indeed, the PSID's substantial editing and across-wave
consistency checking should make measurement errors of this type less
problematic than in surveys not following such procedures.
5. AUTHOR'S NOTE:
This result held in a CPS-Social Security validation study as well
(Bound and Krueger, 1989).
| |
Institute for Social Research |
University of Michigan |
Privacy |
Conditions of Use
|
