Evaluating the effectiveness of Washington state repeated job search services on the employment rate of prime-age female welfare recipients

Abstract

This paper uses an unbalanced panel dataset to evaluate how repeated job search services (JSS) and personal characteristics affect the employment rate of the prime-age female welfare recipients in the State of Washington. We propose a transition probability model to take into account issues of sample attrition, sample refreshment and duration dependence. We also generalize Honoré and Kyriazidou’s [Honoré, B.E., Kyriazidou, E., 2000. Panel data discrete choice models with lagged dependent variables. Econometrica 68 (4), 839–874] conditional maximum likelihood estimator to allow for the presence of individual-specific effects. A limited information test is suggested to test for selection issues in non-experimental data. The specification tests indicate that the (conditional on the set of the confounding variables considered) assumptions of no selection due to unobservables and/or no unobserved individual-specific effects are not violated. Our findings indicate that the first job search service does have positive and significant impacts on the employment rate. However, providing repeated JSS to the same client has no significant impact. Further, we find that there are significant experience-enhancing effects. These findings suggest that providing one job search services training to individuals may have a lasting impact on raising their employment rates.

Executive summary

This paper uses an unbalanced panel dataset to evaluate the effects of repeated job search services (JSS) on the employment rates of the prime-age female welfare recipients in the state of Washington. The JSS are the main services provided by the WorkFirst program under the Federal Temporary Assistance for Needy Family (TANF) program in the state of Washington. Since the average annual expenditure per recipient of TANF in 1998 was $12,363 and the total annual expenditure was $830 million to the State of Washington, policy makers are particularly interested in finding out how effective the JSS program is and whether repeatedly participating in JSS could fundamentally change the labor market outcomes.

We propose a transitional probability model of finding employment or staying on employment as a means to take into account issues arising from sample attrition, sample refreshment and duration dependence. Being state dependent, employment or non-employment, a transition probability model accommodates dynamics in a simple format. We estimate our transitional probability model under the assumptions of (1) conditional independence (CIA) (namely, participation in JSS can be considered exogenous conditional on the confounding variables) and (2) no individual-specific effects. We also generalize Honoré and Kyriazidou’s (2000) conditional maximum likelihood estimator to allow for the presence of individual-specific effects. A limited information test is suggested to test for selection issues in non-experimental data. The specification tests indicate that the assumptions of no selection due to unobservables and/or no unobserved individual-specific effects are not violated.

We find that only the first Job Search Services had positive and statistically significant impacts on employment rates. The probabilities of employment are increased by about 3.6% for the non-employed and by 3.4% for the employed. Repeating JSS does not appear to raise the probability of employment. We also find that each additional quarter of non-employment reduces the probability of employment by 2.9% and each additional quarter of employment raises the probability of employment by 2.3%. The “experience-enhancing” effect together with the finding that the first JSS does raise the probability of employment appears to provide empirical support for requiring TANF recipients to engage in employment-related activities, and a focus on short-term less expensive job search activities can be beneficial. However, repeating JSS does not appear to yield any additional benefit. It appears that for those who have already taken one JSS, perhaps other training programs, such as long-term human capital augment activities could be more beneficial rather than prodding them to repeatedly taking JSS.

1. Introduction

This paper uses an unbalanced panel dataset to evaluate the effects of repeated job search services (JSS) on the employment rates of the prime-age female welfare recipients in the state of Washington. The JSS are the main services provided by the WorkFirst program under the Federal Temporary Assistance for Needy Family (TANF) program in the state of Washington. The criterion for an individual to be on TANF is the income level. The TANF program was established in 1996 by the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) to replace the Aid to Families with Dependent Children (AFDC) program. The TANF requires all adults who receive cash welfare assistance to work or to engage in employment-related activities or face sanctions. The WorkFirst program was initiated in 1997 using block grants from TANF. TANF recipients are required to participate in the main activities of JSS that may last up to 12 continuous weeks and include classroom instructions on how to find jobs; pre-employment training; high-wage/high-demand training, etc. A TANF recipient is also allowed to participate in the JSS repeatedly with the approval of case managers. However, since the average annual expenditure per recipient of TANF in 1998 was $12,363 and the total annual expenditure was $830 million to the State of Washington, policy makers are particularly interested in finding out how effective the JSS program is and whether repeatedly participating in JSS could fundamentally change the labor market outcomes.

A central issue for using non-experimental data for program evaluation is the non-randomness of program participants and non-participants (e.g. see the survey of Friedlander et al. (1997)). The measurements may be subject to various selection bias (e.g., Angrist et al. (1996), Heckman and Robb (1985), and Rosenbaum and Rubin (1983)). With panel data outcomes of the same individual before and after the treatment could be observed. Moreover, panel data provides information on duration of employment and unemployment spell which is considered important but is often not available in cross-sectional data (e.g., Heckman et al. (1999)).

Contrary to the early studies using panel data (e.g., Bassi (1984), Ashenfelter and Card (1985), Heckman and Hotz (1989), Hotz et al. (2006) and Hotz et al. (2002)), our dataset is not a balanced panel data. A significant feature of our data is that clients entered and left the TANF program at different time periods. There is also the issue of right censoring because the data period ends at the last quarter of 2000. Table 1 shows the frequency of new entrants of our sample at each quarter from the second quarter of 1998 to the last quarter of 2000. For instance, only about 3.4% clients have been observed over the complete sample period. In this paper, we propose a transitional probability model of finding employment or staying on employment as a means to take into account issues arising from sample attrition, sample refreshment and duration dependence. Being state dependent, employment or non-employment, a transition probability model accommodates dynamics in a simple format. Under the assumption that conditional on certain regressors, the transition probability from one state at time period t − 1 to another state at time period t is homogeneous, the observed sample provides some information on the unknown parameters characterizing the transition probability as long as the time series observations for a given individual exceed two and there are sufficient mixtures of samples in all four possible states. There is no need to be concerned about when an individual entered and exited the sample. Moreover, from the estimated transitional probabilities, it is possible to trace out an individual’s path of employment over time and duration of employment.

Table 1.

Frequency distribution of new entrants in each quarter

Quarter	Freq.	Percent	Cum.
1998.II	1 224	3.40	3.40
1998.III	2 233	6.21	9.61
1998.IV	2 920	8.12	17.73
1999.I	3 050	8.48	26.21
1999.II	3 420	9.51	35.71
1999.III	3 935	10.94	46.65
1999.IV	3 793	10.54	57.19
2000.I	4 070	11.31	68.51
2000.II	4 065	11.30	79.81
2000.III	3 755	10.44	90.25
2000.IV	3 509	9.75	100.00
Total	35 974	100.00

We estimate our transitional probability model under the assumptions of (1) conditional independence (CIA) (namely, participation in JSS can be considered exogenous conditional on the confounding variables) and (2) no individual-specific effects. Diagnostic tests are then proposed to examine the validity of these assumptions.

Our findings show that only the first job search services have positive and statistically significant effects on the employment rate regardless of whether one is initially employed or non-employed. However, providing repeated JSS to the non-employed clients or to those who are already employed has no statistically significant impacts on the probability of finding employment or staying employed. Furthermore, the probability of employment is also influenced by the duration in employment or non-employment, family factors, education level, geographic and local labor market conditions as well as other welfare services.

Section 2 introduces the model and Section 3 presents the estimation results. Diagnostic checking procedures are proposed and conducted in Section 4. Conclusions are given in Section 5. Detailed descriptions of our data are presented in Appendix.

2. The model

Most empirical investigations evaluating government training programs using experimental and non-experimental data do not consider the effects of the repeated job trainings. Considering sequential treatments from an intertemporal optimization framework is very complicated. Most literature follow the lead of Robins (1986) and Gill and Robins (2001) by treating sequential treatments as some form of sequential randomization (e.g. Lechner and Miquel (2001), Lechner (2004)). Under the assumption of some form of weak or strong dynamic conditional independence assumption (e.g. Lechner (2004)), the participations of the treatment are essentially treated as exogenous. Unfortunately, the demand on the data to measure the effects of different treatment sequences is huge and complicated. For instance, if there are two periods, there are four possible states for the two-period sequences, (0, 0), (1, 0), (0, 1) and (1, 1), where 1 indicates receiving treatment in that period, and 0 not. If there are n periods, then there will be 2ⁿ possible states for the n-period sequences. Matching estimates will have to be computed for each possible state to control the confounding effects of observables that vary across individuals and over time. Therefore, for the n-period data, ${\sum }_{t=1}^n{2}^t$ matching estimates will have to be computed. Such a huge number of measurements might fail to convey a clear picture to policy makers. Therefore, in this paper we propose separating the timing effects and treatment effects.

2.1. A transitional probability model for the outcomes

In this section we propose a transitional probability framework to take into account issues of sample attrition, sample refreshment and duration dependence. Let y_it be the binary indicator that takes the value 1 if the ith client is employed at time t and the value 0 otherwise, t = t_i, t_i + 1, …, T_i, where t_i and T_i denote the first period and the last period that client i is observed. We assume that y_it depends on the previous state, y_i,t−1, previous JSS treatments, and strictly exogenous socio-demographic variables, x_it. Conditional on s = y_i,t−1 = 0 or 1, the potential state of employment is given by

$$y_{it}^{1s\ast }={\alpha }_i^{1s}+{\mathbf{x}}_{it}^{\prime }{\mathit{\beta }}^{1s}+{\mathbf{D}}_{it}^{\prime }{\mathit{\gamma }}^{1s}+{\epsilon }_{it}^{1s},s=0,1,$$ (1)

and the potential state of unemployment is given by

$$y_{it}^{0s\ast }={\alpha }_i^{0s}+{\mathbf{x}}_{it}^{\prime }{\mathit{\beta }}^{0s}+{\mathbf{D}}_{it}^{\prime }{\mathit{\gamma }}^{0s}+{\epsilon }_{it}^{0s},s=0,1,$$ (2)

where D_it is a 4 × 1 vector that takes the value ${\mathbf{D}}_{it}^{\prime }=(0,0,0,0),{\mathbf{D}}_{it}^{\prime }=(1,0,0,0),{\mathbf{D}}_{it}^{\prime }=(1,1,0,0),{\mathbf{D}}_{it}^{\prime }=(1,1,1,0)$, and ${\mathbf{D}}_{it}^{\prime }=(1,1,1,1)$ if the ith individual at time t received no JSS, 1 JSS, 2 JSS, 3 JSS and 4 or more JSS before time period t, respectively.¹ We group 4 or more JSS into 4 JSS dummy because over 98% of clients took not more than 4 JSS treatments (Table 2). Let

Table 2.

Frequency distribution of number of JSS taken

Number of JSS	Freq.	Percent	Cum.
0	8,856	24.62	24.62
1	13,446	37.38	61.99
2	8,968	24.93	86.92
3	3,075	8.55	95.47
4	1,058	2.94	98.41
5	353	0.98	99.39
6	148	0.41	99.81
7	45	0.13	99.93
8	18	0.05	99.98
9	6	0.02	100.00
10	1	0.00	100.00
Total	35,974	100.00

$$\begin{array}{l}{y}_{it}^{s\ast }=y_{it}^{1s\ast }-y_{it}^{0s\ast }\\ ={\alpha }_i^s+{\mathbf{x}}_{it}^{\prime }{\mathit{\beta }}^s+{\mathbf{D}}_{it}^{\prime }{\mathit{\gamma }}^s+{\epsilon }_{it}^s,s=0,1,\end{array}$$ (3)

where ${\alpha }_i^s={\alpha }_i^{1s}-{\alpha }_i^{0s}$, β^s = β^1s − β^0s, γ^s = γ^1s − γ^0s. Combining $y_{it}^{1\ast }$ and $y_{it}^{0\ast }$ into one equation yields

$$y_{it}^{\ast }={\alpha }_i^0(1+{\delta }_i{y}_{i,t-1})+{\mathbf{x}}_{it}^{\prime }({\mathit{\beta }}^0+\mathbf{b}{y}_{i,t-1})+{\mathbf{D}}_{it}^{\prime }({\mathit{\gamma }}^0+\mathbf{g}{y}_{i,t-1})+{\epsilon }_{it},$$ (4)

where ${\alpha }_i^0{\delta }_i={\alpha }_i^1-{\alpha }_i^0$, b = β¹ − β⁰, g = γ¹ − γ⁰, and ${\epsilon }_{it}={\epsilon }_{it}^1{y}_{i,t-1}+{\epsilon }_{it}^0(1-y_{i,t-1})$. We assume

$$y_{it}=\{\begin{array}{cc}1& \text{if}{y}_{it}^{\ast }>0,\\ 0& \text{if}{y}_{it}^{\ast }\le 0.\end{array}$$ (5)

We assume that

• A1The error ε_it follows an independent type I extreme value distribution.
• A2
  Conditional on y_i,t−1 and ${\mathbf{x}}_i^{\prime }=\left({\mathbf{x}}_{i,t_i}^{\prime },\dots ,{\mathbf{x}}_{i,T_i}^{\prime }\right)$, the distribution of ε_it and D_it are independent (Conditional Independence Assumption (CIA) (Rosenbaum and Rubin, 1983) or Ignorable Treatment Assignment Assumption (Heckman and Robb, 1985; Holland, 1986)),²

  $${\epsilon }_{it}\perp D_{it}\mid y_{i,t-1},{\mathbf{x}}_i.$$ (6)
• A3Conditional on y_i,t−1 and x_i, ${\alpha }_i^{1s}={\alpha }_i^{0s}=c$ so there are no unobserved individual-specific effects.
  Under A1–A3,

  $$\begin{array}{l}Pr(y_{it}=1\mid {\mathbf{x}}_{it},y_{i,t-1})=\frac{exp(c(1+\delta y_{i,t-1})+{\mathbf{x}}_{it}^{\prime }({\mathit{\beta }}^0+\mathbf{b}{y}_{i,t-1})+{\mathbf{D}}_{it}^{\prime }({\mathit{\gamma }}^0+\mathbf{g}{y}_{i,t-1}))}{1+exp(c(1+\delta y_{i,t-1})+{\mathbf{x}}_{it}^{\prime }({\mathit{\beta }}^0+\mathbf{b}{y}_{i,t-1})+{\mathbf{D}}_{it}^{\prime }({\mathit{\gamma }}^0+\mathbf{g}{y}_{i,t-1}))}\\ =F_{it},\text{for}t=t_i+1,\dots ,T_i,i=1,\dots ,N.\end{array}$$ (7)

  However, the lagged value of y before t_i is unobserved. We follow Heckman (1981) to approximate the initial state by

• A4
  $$Pr(y_{i{t}_i}=1\mid {\mathbf{x}}_i,D_{i{t}_i})=\frac{exp\{\overline{c}+{\overline{\mathbf{x}}}_i^{\prime }{\mathbf{b}}^{\ast }+D_{i{t}_i}^{\prime }{\mathit{\gamma }}^{\ast }\}}{1+exp\{\overline{c}+{\overline{\mathbf{x}}}_i^{\prime }{\mathbf{b}}^{\ast }+D_{i{t}_i}^{\prime }{\mathit{\gamma }}^{\ast }\}}=P_{i{t}_i},$$

  where ${\overline{\mathbf{x}}}_i=\frac{1}{(T_i-t_i+1)}{\sum }_{t_i}^{T_i}{\mathbf{x}}_{it}$, c̄, b^* and γ^* are parameters to be estimated.

  Under A1–A4, the likelihood function of (y_iti, y_i,(ti+1), …, y_iTi), i = 1, …, N is equal to

  $$L=\prod _{i=1}^N\prod _{t=t_i}^{T_i}{F}_{it}^{y_{it}}{(1-F_{it})}^{1-y_{it}}{P}_{i{t}_i}^{y_{i{t}_i}}{(1-P_{i{t}_i})}^{(1-y_{i{t}_i})}.$$ (8)

Because all the parameters vary depending on the outcome in the previous period, one can obtain a full set of estimates by obtaining first one subset, then the other in two separate maximizations. That is, the maximum likelihood estimator (MLE) can be obtained by maximizing two separate likelihood functions of the binary response models, one conditional on y_i,t−1 = 1, the other on y_i,t−1 = 0.

2.2. Evaluation of the conditional and unconditional impacts

We are interested in two questions. First, how do repeated JSS and personal characteristics affect the employment rate of non-employed and employed, respectively? Second, what are the unconditional impacts of JSS and other characteristics on the employment rate if an individual is randomly drawn from welfare recipients (i.e., regardless of an individual’s employment status)?

Let $d_{it}^m=1$ if the ith individual received exactly m JSS before time period t and after t and 0 otherwise. We define the treatment effect of the mth JSS for individual i conditional on $d_{it}^{m-1}=1$, y_i,t−1 = s, s = 0, 1 as changes of probability of employment,

$${\mathrm{\Delta }}_{it}^{sm}(\mathbf{x},d_t^m=1)=Pr(y_{it}=1\mid y_{i,t-1}=s,\mathbf{x},d_{it}^m=1)-Pr(y_{it}=1\mid y_{i,t-1}=s,\mathbf{x},d_{it}^{m-1}=1).$$

The average treatment effect (ATE) of the mth JSS of TANF recipients conditional on last period’s employment s, s = 0, 1, is defined as

$$\text{ATE}({\mathrm{\Delta }}^{ms})=\int [Pr(y_{it}=1\mid y_{i,t-1},\mathbf{x},d_{it}^m=1)-Pr(y_{it}=1\mid y_{i,t-1},\mathbf{x},d_{it}^{m-1}=1)]f(\mathbf{x})\mathrm{d}\mathbf{x},$$ (9)

where f (x) denotes the probability density function of x of TANF recipients. The average treatment effect of the treated (TT) of the mth JSS for those employed, y_i,t−1 = 1, or non-employed, y_i,t−1 = 0 conditional on $d_{it}^{m-1}=1$, is defined as

$$\text{TT}({\mathit{\Delta }}^{ms})=\int [Pr(y_{it}=1\mid y_{i,t-1},\mathbf{x},d_{it}^m=1)-Pr(y_{it}=1\mid y_{i,t-1},\mathbf{x},d_{it}^{m-1}=1)]f(\mathbf{x}\mid d_t^{m-1}=1)\mathrm{d}\mathbf{x}.$$ (10)

The ATE of the mth JSS is the mean impact of the mth JSS if a random TANF client is assigned m JSS given he has already received (m − 1) JSS. The TT of the mth JSS is the mean impact of the mth JSS if the same selection rule for assigning treatment applies to the future. If x_it is randomly drawn, TT (Δ^ms) or ATE (Δ^ms) can be approximated by taking the sample average of the predicted probabilities of those who have $d_{it}^{m-1}=d$, d = 0, 1. We can also evaluate the impact of one-unit change of x_ij on P_ist, s = 0, 1, by ∂P_ist/∂x_ij if x_ij is continuous, where P_ist = Pr (y_it = 1|y_i,t−1 = s, x). The population impact of a one-unit change of x_j is $\int \frac{\partial P_{\mathit{ist}}}{\partial x_{ij}}\mathrm{d}F(x_i)$. Assuming that our sample is a random sample, this impact can be approximated by $\frac{1}{N}{\sum }_i\frac{\partial P_{\mathit{ist}}}{\partial x_{ij}}$.

The transitional probability framework also allows one to trace out an individual’s dynamic path of P_it from its initial state, following the rule of

$$\left[\begin{array}{c}{P}_{it}\\ 1-P_{it}\end{array}\right]=\left[\begin{array}{cc}{P}_{i1,t-1}& (1-P_{i1,t-1})\\ P_{i0,t-1}& (1-P_{i0,t-1})\end{array}\right]\left[\begin{array}{c}{P}_{i,t-1}\\ 1-P_{i,t-1}\end{array}\right],$$

where P_it denotes the marginal or unconditional probability of employment. Since the transition probability depends on x, an individual’s dynamic path depends on her past, current and future x. We also evaluate the impact of x on the long-run marginal probability of employment regardless of an individual’s employment status by letting ${\pi }_i^1=F_i^0/[1-F_i^1+F_i^0]$, where $F_i^s$ denotes the probability of employment given the previous state is s, where s = 0, 1, and are evaluated at the time series mean of x_it, x̄_i.

3. Findings

In this section we provide estimates of model (7) using panel data from Washington State female TANF recipients between the age of 25–35 from the second quarter of 1998 to the third quarter of 2000.³ Following the suggestion of the literature (e.g., Ashenfelter and Card (1985), Friedlander et al. (1997) and Hotz et al. (2006)) we consider the following types of socio-demographic factors as possible explanatory variables that determine an individual’s current employment status: (i) previous participation in the WorkFirst program such as JSS, alternative services (AS) (for clients who could not participate in JSS directly due to problems like drug abuses and family violence), and previous participation in post-employment services (PS) (for clients who have got at least part time jobs); (ii) duration dependence such as number of quarters employed or non-employed; (iii) earnings history; (iv) family information such as number of adults, number of children, age of the youngest child, marital status; (v) race and ethnicity such as dummies for whites, blacks and Hispanic; (vi) education dummy such as grade 12 and above dummies; (vii) local economy such as local non-employment rate; and (viii) geographic and time dummies. A full description of these variables are presented in Table 3 and the descriptive statistics are provided in Table 4.⁴

Table 3.

Variable definitions

Variable category	Variable name	Definitions
WorkFirst participation	First JSS	Indicator for whether the first Job Search Services (JSS) has been taken before period t.
	Second JSS	Indicator for whether the second JSS has been taken before period t.
	Third JSS	Indicator for whether the third JSS has been taken before period t.
	Four or more JSS	Indicator for whether the individual has taken four or more JSS before period t.
	Previous total AS	Total number of Alternative Services (AS) taken before period t.
	Previous total PS	Total number of Post-employment Services (PS) taken before period t.
Employment history	Previous non-employment quarters	Total unemployed quarters before period t.
	Previous employment quarters	Total employed quarters before period t.
Earnings history	Pre-WorkFirst earnings	Earnings before the quarter that client entered the WorkFirst, measured in dollars.
	Previous wage rate	Wage rate of previous job.
Family	Number of adults	Number of adults in the assistance unit.
	Number of kids	Number of children in the assistance unit.
	Age of the youngest kid	Age of the youngest child in the Assistance Unit. Calculated based on the first quarter that WorkFirst began, 1997.IV.
	Married	Marital status. 1 indicates married.
Racea	Whites	Race indicator. 1 indicates client is white.
	Blacks	Race indicator. 1 indicates client is black.
	Hispanics	Race indicator. 1 indicates client is Hispanics.
Education	grade 12	Binary indicator with 1 indicates client’s highest grade higher than 12.
Geographic	Region 1–Region 5	Location indicator. Region i indicates client is from Region i, i = 1, …, 5
Local economy	Unemployment rate	The unemployment rate of the county that the client was located at time t.

^aAsian and Native American are the omitted category.

Table 4.

Descriptive statistics

Variable	Mean	Min	Max
First JSS	0.630	0	1
Second JSS	0.219	0	1
Third JSS	0.073	0	1
Four or more JSS	0.025	0	1
Number of kids	2.407	0	12
Age of the youngest kid	5.064	0	18
Previous total AS	0.645	0	9
Previous total PS	0.127	0	5
Pre-WorkFirst earnings	321.535	0	20 110
Previous non-employment quarters	1.463	0	11
Previous employment quarters	0.871	0	10
Number of adults	1.183	0	4
Married	0.182	0	1
Whites	0.669	0	1
Hispanics	0.115	0	1
Grade 12	0.139	0	1
Region 1	0.141	0	1
Region 2	0.141	0	1
Region 3	0.077	0	1
Region 4	0.209	0	1
Region 5	0.219	0	1
Unemployment rate	5.602	2.566	15.871
Previous wage rate	570.241	0	15 734

Tables 5 and 6 (columns (1)) provide the estimates of model (7) with all JSS variables and socio-demographic variables included. For those who are non-employed we note that the first job search services have significant impacts on the probability of employment (at 5% level), while the rest of the job search services have insignificant impacts; and second, the longer an individual stays non-employed, the less likely she will be employed (the estimated coefficient for duration of non-employment is −0.16). These two results put together make a strong case for the state to provide job search services to non-employed individuals quickly to get them out of non-employment. The negative coefficient for number of alternative services previously taken may be viewed as capturing the attributes of a client that are deemed not employable since people who take alternative services are people who have problems of substance abuse or domestic violence.

Table 5.

MLE estimations for initially non-employed clients

	(1)	(2)	(3)
First JSS	0.203*** (0.038)	0.189*** (0.036)	0.244*** (0.049)
Second JSS	−0.074 (0.051)		−0.086 (0.067)
Third JSS	0.101 (0.084)		0.094 (0.108)
Four or More JSS	0.188 (0.127)		0.193 (0.127)
Number of kids	−0.018 (0.014)	−0.018 (0.014)	−0.017 (0.014)
Age of the youngest kid	0.018*** (0.004)	0.018*** (0.004)	0.019*** (0.004)
Previous total AS	−0.091*** (0.021)	−0.096*** (0.020)	−0.090*** (0.021)
Previous total PS	0.130** (0.062)	0.130** (0.062)	0.130** (0.062)
Pre-WorkFirst Earnings	0.000*** (0.000)	0.000*** (0.000)	0.000*** (0.000)
Previous non-employment quarters	−0.160*** (0.018)	−0.152*** (0.014)	−0.131*** (0.029)
Previous employment quarters	0.129*** (0.035)	0.134*** (0.034)	0.120 (0.074)
Number of adults	−0.167*** (0.047)	−0.167*** (0.047)	−0.168*** (0.047)
Married	−0.225*** (0.051)	−0.224*** (0.051)	−0.225*** (0.051)
Whites	−0.166*** (0.039)	−0.166*** (0.039)	−0.163*** (0.039)
Hispanics	0.076 (0.055)	0.074 (0.055)	0.075 (0.056)
Grade 12	0.190*** (0.046)	0.190*** (0.046)	0.189*** (0.046)
Region 1	0.219*** (0.056)	0.217*** (0.056)	0.227*** (0.057)
Region 2	0.178*** (0.063)	0.179*** (0.063)	0.174*** (0.064)
Region 3	0.295*** (0.068)	0.294*** (0.068)	0.296*** (0.072)
Region 4	−0.049 (0.059)	−0.046 (0.059)	−0.048 (0.062)
Region 5	−0.019 (0.050)	−0.019 (0.050)	−0.016 (0.053)
Unemployment rate	−0.022** (0.010)	−0.022** (0.010)	−0.020 (0.016)
Previous wage rate	0.0001 (0.0001)	0.0001 (0.0001)	−0.00005 (0.00003)
Constant	−0.597*** (0.105)	−0.608*** (0.104)	−0.689*** (0.140)

Standard errors in parentheses. Column (1) The base model. Column (2) The model without insignificant Job Search Indicators. Column (3) The estimated coefficients are from the exogeneity test models, the instruments and their interactions with the explanatory variables are not presented but available from the authors. The omitted instruments and the interactions are: number of cases handled by each case manager (mgrcasenum), number of clients participating in each CSO (csonum), and the interactions of mgrcasenum and with previous non-employment quarters, previous employment quarters, previous wage rate, the four JSS indicators, and interaction of csonum with employment quarters, previous wage rate, the four JSS indicators. The likelihood ratio test statistic is 15.07, with a p value of 0.58.

^**Significant at 5%.

^***Significant at 1%.

Table 6.

Maximum likelihood estimates for initially employed clients

	(1)	(2)	(3)
First JSS	0.180*** (0.044)	0.165*** (0.042)	0.112 (0.096)
Second JSS	−0.066 (0.069)		−0.119 (0.152)
Third JSS	−0.060 (0.120)		−0.049 (0.259)
Four or More JSS	−0.123 (0.191)		−0.387 (0.413)
Number of kids	−0.008 (0.017)	−0.008 (0.017)	−0.008 (0.018)
Age of the youngest kid	0.009 (0.006)	0.009 (0.006)	0.009 (0.006)
Previous total AS	−0.043 (0.035)	−0.039 (0.035)	−0.041 (0.035)
Previous total PS	0.099** (0.040)	0.101** (0.040)	0.095** (0.041)
Pre-WorkFirst earnings	0.000*** (0.000)	0.000*** (0.000)	0.000*** (0.000)
Previous non-employment quarters	−0.038 (0.042)	−0.056 (0.041)	0.094 (0.089)
Previous employment quarters	0.051** (0.021)	0.034* (0.018)	0.097** (0.043)
Number of adults	−0.164*** (0.060)	−0.165*** (0.060)	−0.162*** (0.060)
Married	0.078 (0.063)	0.078 (0.063)	0.076 (0.064)
Whites	−0.116** (0.048)	−0.111** (0.048)	−0.113** (0.048)
Hispanics	0.092 (0.067)	0.094 (0.067)	0.101 (0.067)
Grade 12	0.086 (0.058)	0.086 (0.058)	0.081 (0.058)
Region 1	0.105 (0.068)	0.105 (0.068)	0.121* (0.069)
Region 2	0.055 (0.077)	0.059 (0.077)	0.056 (0.078)
Region 3	−0.074 (0.082)	−0.076 (0.082)	−0.052 (0.088)
Region 4	0.002 (0.077)	0.001 (0.077)	0.040 (0.081)
Region 5	−0.113* (0.065)	−0.113* (0.065)	−0.119* (0.070)
Unemployment rate	−0.047*** (0.012)	−0.047*** (0.012)	−0.067*** (0.018)
Previous wage rate	0.0004*** (0.00002)	0.0004*** (0.00002)	0.0004*** (0.00004)
Constant	0.766*** (0.132)	0.786*** (0.131)	0.757*** (0.185)

Standard errors in parentheses. Column (1) The base model. Column (2) The model without insignificant Job Search Indicators. Column (3) The estimated coefficients are from the exogeneity test models, the instruments and their interactions with the explanatory variables are not presented but available from the authors. They are available from the authors upon request. The omitted instruments and the interactions are: number of cases handled by each case manager (mgrcasenum), number of clients participating in each CSO (csonum), and the interactions of mgrcasenum and with previous non-employment quarters, previous employment quarters, previous wage rate, the four JSS indicators, and interaction of csonum with employment quarters, previous wage rate, the four JSS indicators. The likelihood ratio test statistic is 20.91, with a p value of 0.23.

^*Significant at 10%.

^**Significant at 5%.

^***Significant at 1%.

For those who are employed Table 6 also indicates that only the first JSS has statistically significant positive impacts on the employment rate. Further, some variables that are significant for the non-employed turn out to be insignificant for the employed. Age of the youngest child is no longer statistically significant for the probability of staying employed once she is employed, neither is marital status, nor is non-employment duration relevant to the probability of staying on the job. On the other hand, the longer an individual is employed, the higher the probability that she will stay employed the next period. This seems to suggest that once a client is employed, what matters is not non-employment history, but her employment history. Furthermore, previous post-employment services appear beneficial for staying employed.

As only the first JSS appears to have significant impacts for the non-employed and the employed, we re-estimate the model by dropping all other JSS dummies for both employed and non-employed and present the results in columns (2) of Tables 4 and 5, respectively. The estimates that drop the insignificant JSS dummies look very close to those with the full set of JSS dummies used as regressors. We therefore focus our discussion of the impacts of one-unit change of a variable based on the model with only 1 JSS used as a conditional variable.

The mean marginal impacts of one-unit change in explanatory variables to the probability of being employed for the non-employed and for the employed are reported in columns (1) and (2) of Table 7, respectively. This table shows that the first JSS increases the probability of being employed by 3.6% for those who were non-employed in the previous period, and increases the probability of staying employed by 3.4%.

Table 7.

Mean group impacts and equilibrium impacts

Parameter	Group impacts		Random individual (3)
	The non-employed group (1)	The employed group (2)
First JSS	0.036	0.034	0.047
Second JSS	0.000	0.000	0.000
Third JSS	0.000	0.000	0.000
Four or More JSS	0.00001	0.000	0.000
Previous total AS	−0.016	−0.008	−0.016
Previous total PS	0.023	0.018	0.028
Pre-WorkFirst earnings	0.00001	0.00001	0.00002
Previous non-employment quarters	−0.029	−0.0007	−0.016
Previous employment quarters	0.023	0.009	0.023
Age of the youngest kid	0.003	0.002	0.0035
Number of adults	−0.033	−0.037	−0.040
Number of kids	−0.018	−0.0016	−0.003
Married	−0.039	0.014	−0.212
Whites	−0.030	−0.018	−0.035
Hispanics	0.076	0.018	0.010
Grade 12	0.035	0.016	0.035
Region 1	0.042	0.027	0.041
Region 2	0.025	0.010	0.029
Region 3	0.034	−0.013	0.031
Region 4	−0.037	0.0003	0.0002
Region 5	−0.0034	−0.0213	−0.013
Unemployment rate	−0.004	−0.008	−0.008
Previous wage rate	0.0001	0.00008	0.00005

Column (3) of Table 7 presents the impacts of the repeated JSS participations as well as other characteristics regardless of an individual’s initial state of employment evaluated at x̄_i. It shows that in the long run, the first job search service increases the probability of being employed by 4.7%.⁵ It also implies that the longer one stays non-employed, the less chance one has for employment. On the other hand, the longer one stays employed, the higher the chance that she stays employed in the future.⁶ This information together with the finding that unconditionally, JSS have positive effects on the probability of employment may shed additional light on the cost and benefit analysis between the education-first or the employment-first strategy.

4. Diagnostic checking

The inference reported in Section 3 requires the validity of the conditional independence assumption (CIA, A3). As pointed out by a referee, CIA implies that there are no unobserved individual-specific effects that could be correlated with w_it, ${\mathbf{w}}_{it}^{\prime }=({\mathbf{x}}_{it}^{\prime },{\mathbf{D}}_{it}^{\prime })$. If CIA assumption does not hold, then our maximum likelihood estimates are biased. In this section we propose specification tests to check whether it is appropriate to impose this assumption. We start by first relaxing the restrictive nature of the conditional independence by assuming a weaker version of conditional independence, namely, conditional on the individual-specific effects and x, ( $y_{it}^{s\ast }\perp {\mathbf{D}}_{it}\mid {\alpha }_i^s$, x_it). We propose a conditional maximum likelihood estimator to allow for the presence of unobserved individual-specific effects conditional on D_it being predetermined, then suggest a Hausman (1978) type specification test to test for the presence of individual-specific effects. We then propose a limited information framework to test for the strong version of the conditional independence assumption, namely, ( $y_{it}^{s\ast }\perp {\mathbf{D}}_{it}\mid {\mathbf{x}}_{it}$) by simultaneously testing for the presence of individual-specific effects and endogeneity of participation in JSS. We propose to test these assumptions sequentially conditional on other assumptions being valid to allow for more efficient use of sample observations or to relax the need for good instruments because consistent estimation methods that simultaneously relax both assumptions impose severe restrictions on the data that can lead to significant loss of sample information. Moreover, the conditional testing procedures have more power relative to particular alternative hypotheses if the conditional event is true. Pedagogically, it is also much simpler to show the validity of proposed procedures before presenting a simultaneous test of endogeneity of D_it and the presence of unobserved individual-specific effects.⁷

4.1. Controlling for individual-specific effects

In this subsection we propose to generalize Honoré and Kyriazidou’s (2000) conditional maximum likelihood estimator to allow for the presence of state-dependent individual-specific effects and slope coefficients assuming D_it is predetermined. We note that under our weaker version of CIA, when (A3) does not hold, both ${\alpha }_i^1$ and ${\alpha }_i^0$(or ${\alpha }_i^0{\delta }_i$ and ${\alpha }_i^0$) appear in (4). If ${\alpha }_i^1$ and ${\alpha }_i^0$ are treated as randomly distributed, one can obtain the MLE provided their conditional distributions given w can be specified. However, the consistency of the estimated parameters depends on whether the conditional distributions of ${\alpha }_i^1$ and ${\alpha }_i^0$ are correctly specified. Moreover, even if the distribution assumptions of ${\alpha }_i^1$ and ${\alpha }_i^0$ are correctly specified, the estimation can be quite involved due to the need for multiple integrations of ${\alpha }_i^1$ and ${\alpha }_i^0$ over the (T_i − t_i) period. On the other hand, if ${\alpha }_i^1$ and ${\alpha }_i^0$ are treated as fixed, there is no need to specify their distributions conditional on w_i, a priori. Therefore, we focus on fixed effect models for (7). However, there are not enough time series observations to obtain consistent estimates of ${\alpha }_i^1$ and ${\alpha }_i^0$, hence β or γ (e.g. see Hsiao (2003)) no matter how large N is.

A consistent estimator of θ = (β′, γ′, b′, g′)′ can be derived if one can transform (7) into a model without incidental parameters ${\alpha }_i^1$ and ${\alpha }_i^0$, i = 1, …, N. The logit form allows such a transformation for observations satisfying certain conditions. For instance, consider the case that T_i − t_i = 3 and consider two events

$$\begin{array}{l}A=\{y_{i0},y_{i1}=0,y_{i2}=1,y_{i3}\},\\ B=\{y_{i0},y_{i1}=1,y_{i2}=0,y_{i3}\},\end{array}$$

where t_i is normalized to be 0. Under the assumptions that w_i2 = w_i3 and y_i0 = y_i3,

$$P(A\mid A\cup B,{\mathbf{w}}_{it},{\alpha }_i^0,{\delta }_i)=\frac{1}{1+exp[({\mathbf{w}}_{i1}-{\mathbf{w}}_{i2}){\mathit{\theta }}_0+({\mathbf{w}}_{i1}{y}_{i0}-{\mathbf{w}}_{i3}{y}_{i3}){\mathit{\theta }}_1]},$$ (11)

where θ₀ = (β′, γ′)′, θ₁ = (b′, g′)′. The conditional probability no longer depends on ${\alpha }_i^0$ and δ_i. Therefore, we propose to estimate θ₀ and θ₁ by maximizing the objective function

$$\begin{array}{l}\sum _{i=1}^N\sum _{t_i\le t\le T_i-3}1(y_{i,t+1}+y_{i,t+2}=1)·1(y_{it}=y_{i,t+3})\\ \times 1({\mathbf{D}}_{i,t+2}={\mathbf{D}}_{i,t+3})·K\left(\frac{{\mathbf{x}}_{i,t+2}-{\mathbf{x}}_{i,t+3}}{{\sigma }_n}\right)\\ \times ln\left(\frac{exp{[({\mathbf{w}}_{i,t+1}-{\mathbf{w}}_{i,t+2}){\stackrel{\sim }{\mathit{\theta }}}_0+({\mathbf{w}}_{i,t+1}{y}_{it}-{\mathbf{w}}_{i,t+3}{y}_{i,t+3}){\stackrel{\sim }{\mathit{\theta }}}_1]}^{y_{i,t+1}}}{1+exp[({\mathbf{w}}_{i,t+1}-{\mathbf{w}}_{i,t+2}){\stackrel{\sim }{\mathit{\theta }}}_0+({\mathbf{w}}_{i,t+1},y_{it}-{\mathbf{w}}_{i,t+3}{y}_{i,t+3}){\stackrel{\sim }{\mathit{\theta }}}_1]}\right)\end{array}$$ (12)

with respect to θ̃₀ and θ̃₁ over the parameter space, where 1 (A) denotes the indicator function, $K(\frac{{\mathbf{x}}_{i,t+2}-{\mathbf{x}}_{i,t+3}}{{\sigma }_n})$ denotes a kernel density function that gives more weight to those observations whose x_i,t+2 are closer to x_i,t+3, and σ_n is a bandwidth that shrinks toward 0 as n increases.

Theorem 1

Let q_it = [w_i,t+1 − w_i,t+2 w_i,t+1y_it − w_i,t+2y_i,t+3], $\mathit{\psi }=({\mathit{\theta }}_0^{\prime },{\mathit{\theta }}_1^{\prime })$,

$$h_{it}(\mathit{\psi })=1(y_{i,t+1}+y_{i,t+2}=1)·1(y_{it}=y_{i,t+3})\times 1({\mathbf{D}}_{i,t+2}={\mathbf{D}}_{i,t+3})\times ln\left(\frac{exp{({\mathbf{q}}_{it}\mathit{\psi })}^{y_{i,t+1}}}{1+exp({\mathbf{q}}_{it}\mathit{\psi })}\right).$$ (13)

Suppose that

• C1The error term ε_it is independently, identically distributed and is independent of D_it, x_it, ${\alpha }_i^s$.
• C2{(y_i, w_i), i = 1, …, N} is a random sample of N observations from a distribution that satisfies (6), where y_i = (y_iti, y_i,ti+1, …, y_i,Ti), w_i = (w_iti, w_i,ti+1, …, w_i,Ti).
• C3The true values of the parameters of interest, ψ₀, are in the parameter space Ψ, which is a compact subset of the Euclidean K-space (R^K), where K = k + q + 1.
• C4(i) The random vector x_i,t+2 − x_i,t+3 conditional on D_i,t+2 = D_i,t+3 is absolutely continuously distributed with density function f (·). f (·) is bounded from above, strictly positive and has support in the neighborhood of zero. (ii)Pr (D_i,t+2 = D_i,t+3) > 0.
• C5E[||w_i,t+2 − w_i,t+3||²|A] and E[||w_i,t+1y_it − w_i,t+3y_i,t+3||²|A] are bounded on their supports, where A = [(x_i,t+2 − x_i,t+3), D_i,t+2 = D_i,t+3] for assumptions (C4)–(C6).
• C6The function E (h (ψ) |A) is continuous in a neighborhood of zero for all ψ ∈ Ψ.
• C7The functions E[(w_i,t+1y_it −w_i,t+2y_i,t+3)(w_t+1y_it −w_i2y_i,t+3)′|A] and E[(w_i,t+1 − w_i,t+2)(w_i,t+1 − w_i,t+2)′|A] and have full column rank in the neighborhood of zero.
• C8K: R^k → R is a function of bounded variation that satisfies: (i) sup_v∈R |K (v) | < ∞, (ii) ∫ |K (v)| dv < ∞, and (iii) ∫ K (v) dv = 1.
• C9σ_n is a sequence of positive numbers that satisfies: σ_n → 0 as n → ∞, but σ_n converges to 0 at a slower rate than n → ∞ (e.g. σ_n is of order n^−1/(2h+1) for a finite positive integer h).

Let ψ̂ be the solution to the problem

$$\underset{\mathit{\psi }\in \mathrm{\Psi }}{max}\sum _{i=1}^N\sum _{t_i\le t\le T_i-3}K\left(\frac{{\mathbf{x}}_{i,t+2}-{\mathbf{x}}_{i,t+3}}{{\sigma }_n}\right)h_{it}(\mathit{\psi }),$$ (14)

then $\widehat{\mathit{\psi }}\stackrel{p}{\to }{\mathit{\psi }}_0$.

Assumptions (C3)–(C6) are the regularity conditions required for the objective function to converge to a non-stochastic limit which is uniquely maximized at ψ₀ by a law of large numbers. Assumption (C7) is required for the identification of θ₀ and θ₁. Assumptions (C8) and (C9) are standard for kernel density estimation. The above assumptions are quite similar to those imposed in Honoré and Kyriazidou (2000), except that we separate the continuous explanatory variables and discrete variables in moment conditions and in kernels. Similarly to that of Honoré and Kyriazidou (2000), the convergence rate is much slower than root-n. It is at rate ${\left(n{\sigma }_n^k\right)}^{1/2}$.

The assumptions for asymptotic normality are similar to those imposed in Honoré and Kyriazidou (2000) except that in addition to conditioning on x_i,t+2 = x_i,t+3, we also need to condition on D_i,t+2 = D_i,t+3. The proof of consistency and asymptotic normality follows straightforwardly those in Honoré and Kyriazidou (2000).

Whether individual-specific effects need to be controlled is critical for the reliability of the MLE. The conditional MLE remains consistent when individual-specific effects are not present. Significant information loss, however, occurs as the conditional MLE greatly restricts data points: less than 10% of the observations used in MLE satisfy the key conditions that y_i,t+1 + y_i,t+2 = 1, y_it = y_i,t+3 and D_i,t+2 = D_i,t+3 for the consistency of the conditional MLE.⁸ Furthermore, very few clients have taken more than two JSS. As the conditional MLE fails to converge with all JSS included, we include only one JSS in the conditional logit estimation as this is our maintained hypothesis anyway.

The conditional MLE yields estimates with larger variances. The Hausman statistic for misspecification is 0.27 for the job-seeker group and 0.98 for the job-holder group, which are not significant at a chi-square distribution with one degree of freedom (3.84 at 5% level). These results appear to suggest that there is no evidence of the presence of no unobserved individual heterogeneity conditioning on observed client characteristics, hence do not appear to contradict using the MLE model to evaluate the effectiveness of repeated JSS.

4.2. A limited information test for conditional independence assumption

One of the critical issues in obtaining accurate estimates of the treatment effects is to control the impact of selection process. Our estimates are obtained under the assumption that the selection is exogenously determined. If this assumption is violated, our inference is biased. Wooldridge (2003) has proposed a likelihood ratio test of conditional independence by specifying a complete model for (y_it, ${\mathbf{D}}_{it}^{\prime }$). However, a complete specification of sequential treatments from an intertemporal optimization framework is very complicated. Moreover, a full information approach could be more sensitive to specification errors. Therefore, in this subsection, we propose to test for CIA from a limited information framework.

Let the latent response function,

$${\mathbf{D}}_{it}^{\ast }=\mathbf{d}({\mathbf{z}}_{it},{\mathbf{x}}_{it})+{\mathbf{v}}_{it}$$ (15)

determine the outcome D_it, where z_it denotes those variables that affect participation, but do not affect the current outcomes of y_it, and v_it is the error term that is orthogonal to z_it and v_it. The z_it variables can be case-manager designation numbers, number of clients attending the same Community Service Office (CSO), etc. because case managers assigned WorkFirst participants to job search services. Participants were assigned to second and higher job search services when the case manager thought it would be helpful for them to go into job search. The way case managers are assigned clients varies from office to office. In some offices, it is a simple rotation as new clients come in, they are assigned to the next case manager who is up in the queue. Other offices assign them by the letter of the clients’ last name — one case manager may get A through G and another H through M. In no office, according to a WorkFirst manager, was there a case where one manager was assigned all the easy cases and another the more difficult cases.

If ε_it and v_it are orthogonal, namely, CIA holds, then the marginal density of ε_it is identical to the conditional density of ε_it conditional on D_it, f (ε_it) = f (ε_it |D_it). It follows from (4) that

$$H_0:E(y_{it}\mid y_{i,t-1},{\mathbf{D}}_{it},{\mathbf{x}}_{it},{\mathbf{z}}_{it},{\mathit{\alpha }}_i)=E(y_{it}\mid y_{i,t-1},{\mathbf{D}}_{it},{\mathbf{x}}_{it},{\mathit{\alpha }}_i),$$ (16)

or if there is no presence of individual-specific effects,

$$H_0^{\ast }:E(y_{it}\mid y_{i,t-1},{\mathbf{D}}_{it},{\mathbf{x}}_{it},{\mathbf{z}}_{it})=E(y_{it}\mid y_{i,t-1},{\mathbf{D}}_{it},{\mathbf{x}}_{it}),$$ (17)

where ${\mathit{\alpha }}_i=\left({\mathit{\alpha }}_i^1,{\mathit{\alpha }}_i^0\right)=\left({\mathit{\alpha }}^1,{\mathit{\alpha }}^0\right)$. If ε_it and v_it are correlated, then CIA does not hold, then

$$E(y_{it}^{\ast }\mid y_{i,t-1},{\mathbf{D}}_{it},{\mathbf{x}}_{it},{\mathit{\alpha }}_i)={\alpha }_i^0(1+{\delta }_i{y}_{i,t-1})+{\mathbf{x}}_{it}^{\prime }({\mathit{\beta }}^0+\mathbf{b}{y}_{i,t-1})+{\mathbf{D}}_{it}^{\prime }({\mathit{\gamma }}^0+\mathbf{g}{y}_{i,t-1})+E({\epsilon }_{it}\mid {\mathbf{D}}_{it},{\mathit{\alpha }}_i),$$ (18)

where the selection factor⁹

$$E({\epsilon }_{it}\mid {\mathbf{D}}_{it},{\mathit{\alpha }}_i)=\lambda ({\mathbf{x}}_{it},{\mathbf{z}}_{it},{\mathit{\alpha }}_i)$$ (19)

is a function of x_it and z_it. In other words,

$$H_1:E(y_{it}\mid y_{i,t-1},{\mathbf{D}}_{it},{\mathbf{x}}_{it},{\mathbf{z}}_{it},{\mathit{\alpha }}_i)\ne E(y_{it}\mid y_{i,t-1},{\mathbf{D}}_{it},{\mathbf{x}}_{it},{\mathit{\alpha }}_i),$$ (20)

or if there is no presence of individual-specific effects,

$$H_1^{\ast }:E(y_{it}\mid y_{i,t-1},{\mathbf{D}}_{it},{\mathbf{x}}_{it},{\mathbf{z}}_{it})\ne E(y_{it}\mid y_{i,t-1},{\mathbf{D}}_{it},{\mathbf{x}}_{it}).$$ (21)

We approximate E (y_it |y_i,t−1, D_it, x_it, z_it, α_i) by¹⁰

$$F[{\mathit{\alpha }}_i^{\ast }(1+{\delta }_i)y_{i,t-1}+{\mathbf{x}}_{it}^{\prime }({\mathit{\beta }}^0+\mathbf{b}{y}_{i,t-1})+{\mathbf{D}}_{it}^{\prime }({\mathit{\gamma }}^0+\mathbf{g}{y}_{i,t-1})+\mathbf{p}({\mathbf{z}}_{it},{\mathbf{x}}_{it},y_{i,t-1})],$$ (22)

where p (z_it, x_it, y_i,t−1) could be a polynomial or series function of (z_it, x_it, y_i,t−1). For instance, if we approximate p (·) by

$$\mathbf{p}({\mathbf{z}}_{it},{\mathbf{x}}_{it},y_{i,t-1})={\mathbf{a}}_1^{\prime }{\mathbf{z}}_{it}+{\mathbf{a}}_2^{\prime }({\mathbf{z}}_{it}{\mathbf{z}}_{it}^{\prime })+{\mathbf{a}}_3^{\prime }({\mathbf{z}}_{it}{\mathbf{x}}_{it}^{\prime })+{\mathbf{a}}_4^{\prime }({\mathbf{z}}_{it}{y}_{i,t-1}),$$ (23)

then a test of H₀ or $H_0^{\ast }$ against H₁ or $H_1^{\ast }$ is equivalent to the test

$${\stackrel{\sim }{H}}_0:{\mathbf{a}}_1=\mathbf{0},{\mathbf{a}}_2=\mathbf{0},{\mathbf{a}}_3=\mathbf{0},\text{and}{\mathbf{a}}_4=\mathbf{0}.$$

As is generally recognized, the choices of management practices and institutional structure can affect program effectiveness, we consider the following variables for the specification of p: (1) The number of cases each case manager handles because case managers were exogenously assigned by the Washington TANF offices. Further, if a case manager handles more cases than the other for the sample period, on average, the clients from those handling more clients would receive less attention, hence their probabilities of participation could be affected. The number of cases occurring in each community service office could also be another variable for p, as more clients getting services from the same community office could affect their probabilities of participations in JSS. We also consider interactions of these two variables with other explanatory variables. Because of severe multicollinearity in the expanded explanatory variables, we only include interaction terms of those variables with those that are individual specific and time varying. This gives 17 additional variables in p. Tables 5 and 6, column 3, report the MLE estimates of θ with the p variables included as additional explanatory variables assuming no individual-specific effects. We note that the coefficients of JSS variables hardly changed with the addition of p. The likelihood ratio test of $H_0^{\ast }$ vs. $H_1^{\ast }$ is 15.07 for the non-employed group and 20.91 for the employed group, which are not significant at a chi-square distribution with 17 degrees of freedom (the critical value is 27.59 at the 5% significance level and it is 24.76 at the 10% significance level).

We also test H₀ vs. H₁ using the conditional MLE approach to allow for the presence of individual-specific effects. Due to multicollinearity there are 7 additional variables in p under the conditional MLE. The likelihood ratio statistic is 8.02 for the non-employed and 10.63 for the employed, both are insignificant at a chi-square distribution with 7 degrees of freedom (the critical value is 15.50 at the 5% significance level and it is 13.36 at the 10% significance level).

5. Conclusion

In this study we have evaluated the effects of repeated job search services and individual characteristics on the employment rates of the prime-age female TANF recipients in Washington State. Using a transition probability framework to deal with the complicated issues of sample attrition, sample refreshment and duration dependence, we estimated conditional and unconditional impacts of the repeated job search services on employment rate. We found that only the first Job Search Services had positive and statistically significant impacts on employment rates. The probabilities of employment are increased by about 3.6% for the non-employed and by 3.4% for the employed. Repeating JSS does not appear to raise the probability of employment.

The TANF has three unique features compared with previous welfare programs. First, there are the universal requirements for recipients to work.¹¹ All states require adults who receive cash welfare assistance to work or to engage in employment-related activities. Second, State governments usually provide cash assistance and other assistance such as child care, transportation assistance to the disadvantaged to make the low-paying work financially attractive to welfare recipients. Third, a 60-month time limit is imposed on each welfare recipient.¹² The principal reasoning behind these appears to be that having a low-paying job now is better than waiting for a high-paying job in the future. We find that each additional quarter of non-employment reduces the probability of employment by 2.9% and each additional quarter of employment raises the probability of employment by 2.3%. The “experience-enhancing” effect together with the finding that the first JSS does raise the probability of employment appears to provide empirical support for requiring TANF recipients to engage in employment-related activities, and a focus on short-term less expensive job search activities can be beneficial. However, repeating JSS does not appear to yield any additional benefit. It appears that for those who have already taken one JSS, perhaps other training programs, such as long-term human capital augment activities could be more beneficial rather than prodding them to repeatedly taking JSS.

Our analysis is based on those who were new entrants to TANF. It will be worthwhile to expand the data to include later years when some recipients approach their 60-month time limits to investigate the effects of 60-month limit on employment outcome. The expanded data may also allow us to investigate more thoroughly the selection issues and the presence of individual effects. The validity of our inference depends on the validity of our assumption, notably the conditional independence assumption and no unobserved individual-specific effects conditional on observables for welfare recipients (or more precisely, TANF recipients). In this paper we have also suggested a conditional maximum likelihood estimator to allow for the presence of individual-specific effects and a limited information procedure to test for the conditional independence assumption. However, given the constraints of our data, the power of these tests could be dubious.

Finally, as pointed out by a referee we are “using State Unemployment Insurance (UI) earnings data matched to program records from Washington State. In many state UI systems there is no way to distinguish between a person who actually earns zero in a quarter and a person who is working in the uncovered sector or in another state”. These systematic and random measurement errors could seriously bias our estimation results and deserve further study.

Appendix. The WorkFirst in the State of Washington: A description of provided services

The data came from the administrative records of the WorkFirst program in Washington state. In order to evaluate the program, the state developed a data file that was an extract from the legacy data systems used to operate the program. These legacy systems, used to manage the WorkFirst program included the Jobs Automated Systems (JAS), and Automated Client Evaluation System (ACES). Additionally, the data for this study came from the Unemployment Insurance data file that includes the covered employment, earnings and hours for all Washington employees covered by the UI system. This earnings data was matched by Social Security Numbers to the clients identified in the WorkFirst data to determine program outcomes.

WorkFirst is a mandatory program under TANF (Temporary Assistance for Needy Families) in the State of Washington. The WorkFirst program serves three groups of people: parents and children aged sixteen or older who receive cash assistance under the temporary assistance for needy families (TANF), general assistance for pregnant women (GA-S) or state family assistance (SFA) programs; parents who no longer receive cash assistance and need some continuing support to remain self-sufficient; and low income parents who support their family without applying for or relying on cash assistance. Its mandatory nature comes from the fact that if one refuses to take activities required by WorkFirst case managers, he/she will be sanctioned. Sanction means reducing a certain percent of cash receipt that one can receive from the TANF program.

When clients first enter WorkFirst, they will work together with case managers to develop the Individual Responsibility Plan (IRP).

The following services can be introduced to clients based on their background:

1. Job Search Services. When clients first enter WorkFirst, the initial focus is to assist them in finding employment. Therefore they will be first introduced to job search. Periods of job search may last up to twelve continuous weeks. The first four weeks are coded as Job Search Initial (JI). They may also be directed to Job Search Workshop (JW). Job Search Services may include one or more of the following forms: (1) classroom instruction that helps in finding job openings, complete applications, practice interviews and apply other skills and abilities with a job search specialist or a group of fellow job seekers; (2) pre-employment training; (3) high-wage/high-demand training.

   By the end of the first four weeks, a job search specialist will determine whether one should continue in job search. Job search will end when (1) clients find a job and work 20 hours or more at an unsubsidized job; or (2) clients become exempt from WorkFirst requirements; or (3) client’s situation changes and he is temporarily deferred from continuing with job search; or (4) Job search specialists have determined that clients need additional skills and/or experience to find a job; or (5) clients have not found a job at the end of the job search period. Participants were assigned to second and higher order job search services when the case manager thought it would be helpful for them to go into job search.

2. Alternative services: For those who are unable to find employment because of problems with substance abuse, domestic violence etc. were referred to alternative services.

3. Post-employment services: Employed clients on the caseload have access to mentors, job-specific education, career planning and other services intended to help them stay employed and find higher-paying jobs. Employed clients who leave the caseload are eligible for post-employment services for up to one year after exiting TANF.