Using random assignment to measure court accessibility for low-income divorce seekers

Significance

The United States faces an access-to-justice crisis: When the vast majority of individuals encounter the court system—when they face eviction or foreclosure, when they need restraining orders against abuse, when they are sued by debt collectors—they do so without a lawyer, frequently because they cannot afford one. This study exposes that the crisis is more pervasive and extreme than previously understood. Using randomization, it documents how a courthouse in one of the most progressive areas of the country remained largely inaccessible to divorce seekers without lawyers, even though the legal procedures for obtaining a dissolution of marriage—which is a constitutional right—were straightforward, ordinarily requiring little more than filing the right paperwork in the right order.

Abstract

We conducted a field experiment in which 311 low-income individuals seeking a divorce were randomly assigned to receive access to a pro bono lawyer (versus minimal help) to assist with filing for divorce. Examining court records, we found that assignment to an attorney made a large difference in whether participants filed for and obtained a divorce. Three years after randomization, 46% of the treated group had terminated their marriages in the proper legal venue, compared to 9% of the control group. Among “compliers”—participants who obtained representation only if assigned to receive it—those with lawyers were far more likely to file for and obtain a divorce than those not assigned lawyers. Because divorce implicates fundamental constitutional interests and can be effectuated only by resort to the courts, the US Constitution requires that dissolution of marriage be made achievable regardless of ability to pay. Yet, we observed few low-income individuals who were able to initiate divorce suits on their own. Through interviews and archival research, we identified barriers that low-income litigants faced in navigating the divorce system, including mandatory wait times, limited hours at important facilities, and burdensome paperwork sometimes requiring access to photocopiers and typewriters. This study therefore documents a salient instance in which a civil legal process was inaccessible to those without lawyers, even though their legal issues were straightforward, involving few if any matters for courts to adjudicate.

Many believe that the United States faces an access-to-justice crisis. The vast majority of individuals in the United States encounter the legal system without a lawyer, frequently because they cannot afford one (1, 2). Unlike criminal defendants facing serious charges, those with civil legal needs do not typically have a right to counsel (2). An influential report from 2017 found that over 70% of low-income households experience at least one civil legal problem annually, including matters related to health care, housing, disability, veterans benefits, and domestic violence (2). Over 85% of these individuals cannot afford an attorney; they must navigate the legal processes on their own (proceed “pro se”) (2).

A movement has mobilized in response to the access-to-justice crisis. The Conference of Chief Justices and Conference of State Court Administrators have pledged to achieve “100 percent access to effective assistance for essential civil legal needs” (3). To make progress toward this goal, however, courts must be able to measure the accessibility of their adjudicatory processes. Counting the number of unrepresented litigants who enter the legal system can be unrevealing because even a courthouse that serves a high volume of pro se litigants may be inaccessible if an even larger number of people with essential civil legal needs never make it to the courthouse.

Here we introduce a quantitative measure of court accessibility. We suggest that if a court system is accessible, then for a legally simple matter—such as a transition from one legal state to another, ordinarily requiring only the filing of the right paperwork in the right order—the presence or absence of traditional attorney–client representation should make little difference to adjudicatory outcomes. We propose using randomized controlled trials (RCTs) to measure the effect of attorney representation (4–6). In a 100% accessible system, being represented by a lawyer, or having the opportunity to be represented by a lawyer, should make no substantial difference to outcomes, at least in legally simple matters.

As an example of how this measure might be deployed, we report the results of an RCT evaluating the effectiveness of a pro bono organization’s oversubscribed divorce lawyer matching practice in Philadelphia County from January 2011 through July 2016. Among this organization’s service population, the majority of clients were seeking uncontested divorces. Most were willing to allocate assets or income streams either according to legal defaults (meaning no additional action on such issues was required in the divorce proceeding) or as part of a separate legal case. Frequently, in the service population, this willingness to rely on default allocations or other proceedings stemmed from the absence of significant assets or income streams to divide. Thus, study participants sought an outcome that, in an accessible legal system, would be achievable without an attorney.

While a handful of RCTs have examined the effect of representation in housing court and in administrative proceedings (4, 5, 7), our research fills an important gap by studying uncontested divorces, which are some of the most straightforward civil cases that courts hear and legal aid providers handle. Eviction cases, for example, can feature comparatively complex legal issues regarding possession, proper termination of tenancy, the tenant’s duty to pay rent, the landlord’s duty to provide repairs, the safety and habitability of the living conditions, duties to accommodate disability, and any counterclaims regarding the landlord’s misconduct, such as discrimination, retaliation, or violations of consumer protection laws. Litigation in housing court might require a review of communications between the landlord and tenant, records of rental payments, and photos of housing conditions. Thus, one might conjecture that access to a lawyer might make a large difference to eviction outcomes (8). By contrast, in uncontested divorce proceedings in which there are no relevant factual disputes and few assets or income streams to divide, there are few matters to litigate or adjudicate. The legal landscape is also less complex: whereas housing matters are governed by overlapping federal, state, and local laws, divorce is primarily regulated by state law (8). The primary role for the volunteer attorney in such cases, then, is helping the client fill out the right paperwork and submit it according to the proper procedures.

In addition, marriage and marriage dissolution have special legal standing as compared to, say, housing and eviction. Because the only way to dissolve a marriage is via the court system, the Supreme Court has identified divorce as a legal process that must, by constitutional mandate, be made accessible to individuals without regard to ability to pay (9–12). Indeed, when the Supreme Court has considered cases brought by litigants whose inability to pay filing fees prevented them from pursuing other pressing legal matters, such as filing for bankruptcy or contesting the termination of government benefits, it has determined that these civil legal needs do not “rise to the same constitutional level” as divorce because these litigants lack a “fundamental interest” in court accessibility (11, 12). In contrast, the Supreme Court has held that when divorce seekers cannot afford to access the courts, it “seriously impair[s] their freedom to pursue other protected associational activities”—namely, remarriage, a right protected by the Constitution (9, 12).

We know of no other RCTs measuring the effect of attorney representation in the context of divorce proceedings or any civil legal proceeding implicating a fundamental constitutional right (9–12). Previous research on divorce has relied on interviews and observational methods. One study, conducted in two Connecticut cities in 1976, found no difference in outcomes between people who filed on their own as opposed to with the assistance of a lawyer (13). The study’s sample, however, was obtained by taking a random draw of court records in these cities and assessing whether, conditional on having a divorce case on file, an attorney made a difference to litigants’ outcomes. It was not an RCT, nor did it examine the pool of individuals who may have sought a divorce but who never made it to the courthouse.

In the present study, we identified a pool of divorce seekers before they entered the court system and randomly assigned them to different levels of legal assistance: an effort to match them with attorney representation versus something less. We worked with Philadelphia Volunteers for the Indigent Program (“Philadelphia VIP”), a pro bono organization with a divorce practice that was persistently oversubscribed. During the study period, Philadelphia VIP’s resources permitted it to find volunteer attorneys for only around 25% of individuals seeking a divorce. The organization was willing to select randomly which individuals were to receive an effort to secure pro bono representation. Thus, we randomly assigned individuals seeking Philadelphia VIP’s assistance in pursuing a divorce to either an effort by the organization to find an attorney to represent them (treated group) or a referral to written instructions on how to obtain a divorce coupled with an offer to answer questions by telephone (control group).

Our study population consisted of 311 Philadelphia County residents. A total of 74 participants (23.8%) were randomized to the treated group, while 237 participants (76.2%) were randomized to the control group

We obtained files for all divorce cases involving study participants in the Family Division of the Court of Common Pleas of the first Judicial District of Pennsylvania (the “Philadelphia Family Court”). We also searched court records at six additional counties: four counties bordering Philadelphia, as well as Cameron County and Potter County. The latter two counties would process divorce cases—even those involving out-of-county litigants—by mail, so long as such cases were uncontested, and were known for imposing cheaper filing fees than those found at the Philadelphia Family Court. Divorce seekers who filed in one of these other counties could achieve a legally valid dissolution of marriage so long as the opposite party consented or did not object, but they ran the risk that, if the opposing party did object, the case would be dismissed for improper venue, which would result in the divorce seekers losing their filing fees and needing to restart the process in the Philadelphia Family Court (Materials and Methods)

A review of these case files provided us with our primary source of information for two key outcome measures: 1) whether the individual had a divorce case file on record within 18 mo of randomization and, if so, 2) whether the individual’s marriage had been terminated within 36 mo of randomization. For both measures, we noted whether the case was filed in Philadelphia County, where venue was proper, or in any of the six other counties we searched.

Results

In total, we found 131 case records that matched our participants’ names within 18 mo of randomization. We observed 74 study participants with divorce cases on file in Philadelphia County. In addition, we observed 56 participants with cases on file in Cameron and Potter counties. Only one participant had a case on file in any of the four other counties. We checked case files 36 mo after randomization to see which had resulted in marriage dissolution and found that 97 of the 131 had done so.

Missing Data.

For 151 of 311 cases, information about whether the study participant obtained a lawyer is missing. This information is missing for participants in the control group who did not succeed in filing for divorce. To perform inference in the presence of this missingness (when necessary), we used the data augmentation algorithm of Tanner and Wong (14).

Balance Checks.

Comparing the treated and control group on a multitude of pretreatment covariates, we find that of 423 comparisons, 19 (4%) show imbalance between groups. Thus, even with a modest sample size of 311 participants, we can conclude randomization was effective in producing treated and control groups that were balanced on observable background variables.

Intention to Treat.

We randomized the presence or absence of an effort by Philadelphia VIP to match the participant to a volunteer attorney. Assuming that few study participants filed divorce cases in counties other than the seven we searched, we can estimate the causal effect of Philadelphia VIP’s efforts.

Considering all seven counties, we find that 61% of the treated group and 36% of the control group had a case file on record after 18 mo, and 50% of the treated group and 25% of the control group had achieved a termination of marriage after 36 mo (Fig. 1). Next, we account for variations in randomization probability and block size by multiplying participants’ outcomes by the inverse probability of their selection to treatment groups and assessing the statistical significance of the differences in means via standard permutation tests. We find that 18 mo after randomization, a treated group participant was 30.93 [95% CI: 20.02, 41.86] percentage points more likely to have a divorce case on file in any of the seven counties we searched (including Philadelphia) than a control group participant, P = 0.000117. A treated group participant was 29.25 [95% CI: 17.56, 40.93] percentage points more likely to have obtained a divorce 36 mo after randomization than a control group participant, P = 0.000020.

Fig. 1.

Comparing Treated versus Control participants on whether individuals (Left) had a divorce case on file within 18 mo and (Right) achieved a termination of marriage within 36 mo in Philadelphia County (where venue is proper) and in six counties outside Philadelphia. For all comparisons, the treated group success rate significantly exceeded the control group success rate.

Limiting our search to just Philadelphia County, where venue is legally proper, we find that 18 mo after randomization, a treated group participant was 43.89 [95% CI: 32.48, 55.30] percentage points more likely to have a divorce case on file, P < 0.000001, and 40.29 [95% CI: 28.70, 51.88] percentage points more likely to have obtained a divorce 36 mo after randomization, P < 0.000001. Most participants (75%) who filed in Philadelphia within 18 mo succeeded in obtaining a divorce within 36 mo. The success rate among control group participants who filed within 18 mo was 64%, while the success rate among treated group participants was 85%.

Thus, Philadelphia VIP’s efforts to match individuals with pro bono lawyers made a large and significant difference to participants’ case outcomes.

Effect of Attorney Representation.

The foregoing analysis estimates the effect of the organization’s effort to provide a lawyer (versus no such effort). However, what is the effect of the presence (versus absence) of a lawyer?

To answer this question, one would ideally randomize which cases had attorney–client relationships and which did not. Ethically, however, we could not prevent control group participants from obtaining lawyers on their own, nor could we force treated group participants to accept attorney representation against their wishes. Therefore, we used several modeling assumptions to estimate the effect of legal representation itself among a subset of participants: those who would receive representation if, and only if, randomized to the treated group (i.e., among the compliers) (Materials and Methods).

In addition, we faced a challenge with missing data. For the treated group, we observed whether participants were represented by counsel regardless of whether they had a case on file at a courthouse. Of 74 participants in the treated group, 62 were successfully matched with a volunteer attorney and two others found lawyers elsewhere. For the control group, by contrast, we observed representation status only if the individual had a case on file at a courthouse. This meant that we could not observe, for example, that a control group participant had secured a lawyer on her own and then decided not to file for divorce.

One of us developed a method to address the statistical challenges posed by our inability to observe whether nonfilers in the control group retained counsel (15). This missing data method deploys Bayesian inference, so we report average figures and 95% credible intervals. Among compliers, participants with lawyers were 55.77 [28.10, 83.81] percentage points more likely to have a case on file in any of the seven counties we searched, including Philadelphia, 18 mo after randomization (Fig. 2). They were 61.30 [20.68, 88.84] percentage points more likely to have achieved a divorce within 36 mo. Limiting the search to Philadelphia, participants with lawyers were 86.33 [69.52, 97.70] percentage points more likely to have a case on file and 87.50 [70.30, 98.55] percentage points more likely to have achieved a divorce within 36 mo.

Fig. 2.

Effect of attorney representation on the probability of filing for divorce within 18 mo and obtaining a divorce within 36 mo in Philadelphia County and in any of seven counties. Estimates appear on the right-hand side of the graphs, indicating that the with-representation group succeeds more often than the without-representation group. Points represent posterior mean estimates and ranges correspond to 95% credible intervals. Each posterior mean is the mean with-representation success rate minus mean without-representation success rate for the complier subset of participants.

Examining case records, we discovered that of the control group participants who managed to file for or obtain a divorce, 80% had obtained attorneys on their own or had cases that were initiated by the opposing spouse. Only 12 of the 237 control group participants (5%) managed to obtain a divorce within 36 mo without having either 1) an attorney of record or 2) the opposing spouse initiate the divorce lawsuit. Of these 12 individuals, only one person (representing 0.4% of the control group) was able to do so in Philadelphia County.

Case Complexity.

Our primary research question asks how great a difference a lawyer made to clients’ outcomes, which we propose serves as a measure of a court system’s accessibility to pro se litigants. The finding that attorney representation made a large difference to litigants’ outcomes is all the more striking given that the legal issues participants faced in this study were ordinarily straightforward. At intake, Philadelphia VIP interviewed each participant for 45 to 60 min regarding marital history, current and past living arrangements, income and assets, children, pets, reasons for divorce, and participants’ goals in seeking out the legal aid provider. These interviews verified that while divorces may sometimes (perhaps frequently) be emotionally and psychologically complicated, participants’ legal needs were straightforward. For example, in over 80% of cases, the spouses had either been separated for years or mutually desired a dissolution of marriage, meaning that the divorces could be uncontested.*

We also reviewed the case files at the Philadelphia Family Court to examine how often the treated group was involved in contested divorces. We observed little litigation activity of any kind, despite the fact that 39 out of the 40 treated group participants who filed in Philadelphia (97.5%) had an attorney who was willing to litigate as necessary. In 80% of these suits, case filings involved no contested motion, hearing, master’s report, property agreement, child custody agreement, or court ruling on a contested motion (other than the court order granting the divorce, which, as noted above, was almost always uncontested). The grounds for the divorce decree in all cases were mutual agreement of the parties or the responding party’s failure to contest the filing party’s case. Only five cases involved contested motions or master’s reports, and the court ruled on none of the motions. We observed no live court hearings. Furthermore, we found that over 80% of treated group participants who terminated their marriages chose not to enter an agreement allocating assets or income streams, despite having been assigned to receive a pro bono attorney who could help them pursue such an agreement. Given random assignment, we can assume that the control group likely had a similarly small fraction of cases involving legal complexity.^†

We can surmise that divorces filed outside of Philadelphia County likewise involved little complexity, even though we were unable to view full case records. Any spouse wishing to contest divorce litigation in an outlying county was legally entitled to prevail by filing a short motion pointing out that venue was improper. Moreover, the outlying county court systems by custom refused to adjudicate contested divorces involving out-of-county litigants.

In summary, our review of the case files leads us to conclude that in the majority of cases in this study, the volunteer attorneys made a difference by completing and filing uncontested paperwork, assuring that service of process was executed properly, and keeping the case moving. Moreover, because of random assignment, the presence of a client’s desire or need to complicate their divorce proceeding through the pursuit of these related matters should be distributed evenly between the groups.

Nondivorce Litigation.

In Pennsylvania, the legal processes for determining child custody, child support, spousal support, and protection from abuse were often handled separately from divorce (16, 17). In Philadelphia, plaintiffs could elect to combine divorce, custody, and support actions or to pursue them separately (16, 17). Thus, Philadelphia VIP lawyers could and did volunteer to help solely with the divorce matter without getting involved in a participant’s other legal needs.

In our study, we were able to observe only divorce cases; we therefore cannot conclude that study participants did not pursue separate custody or support actions. In our review of the case files, however, we observed no contested litigation regarding child custody or support. Thus, we can conclude that no participant chose to combine these processes with the divorce litigation into a single proceeding.

As the primary research question investigated in this study is the difference a lawyer makes, we note that the observed difference between the treated and control groups is unlikely to be explained by complications arising from these nondivorce proceedings. It is certainly possible for a divorce proceeding to be affected by a satisfactory resolution of custody, support, or protection from abuse (“PFA”) litigation. For instance, an individual who achieved a desired custody or child support arrangement in separate litigation might decide that pursuing a divorce would not be worth the trouble or risk of potentially antagonizing the opposing spouse. Given random assignment, however, it is unlikely that the control group entered the study with a stronger desire to seek nondivorce relief than did the treated group. Accordingly, it seems unlikely that the control group, which lacked access to a lawyer, would disproportionately seek nondivorce relief, succeed in obtaining it, and then decide strategically not to pursue a divorce. Furthermore, the hypothesis that success in nondivorce litigation disproportionately deterred the control group from seeking divorce is belied by our finding that many control group participants sought divorces in Cameron and Potter counties, where they could obtain a dissolution of marriage and nothing else. Thus, it is unlikely that nondivorce litigation explains our key findings.^‡

Discussion

The study findings indicate that an effort to secure attorney representation makes a large difference to litigants’ outcomes. We observe that few individuals without lawyers file for divorce in the proper legal venue on their own.

Court Accessibility and Barriers.

A plausible explanation for our findings lies in the procedural barriers would-be divorce filers faced when confronting the prospect of seeking divorce pro se. In a nutshell, using the measurement system we proposed in the introduction, this divorce system was not pro se accessible.

Because the majority of study participants in both conditions who had cases on file within 18 mo succeeded in obtaining a divorce within 36 mo, we sought to understand why pro se litigants might encounter difficulty filing for divorce in the first place. Through interviews and through examining the legal terrain, we identified several procedural hurdles that would-be litigants faced. These included large amounts of paperwork that seemed to require access to photocopiers as well as extensive documentation and record-keeping, not to mention the use of a typewriter to fill out a specific court form (18). They included multiple waiting periods, which previous research has shown will hinder people—especially low-income people—from pursuing their goals (19). They included “hassle factors” such as trips to the post office, courthouse, and law library, each of which was open during limited hours (20, 21). And they included a general reluctance among people with whom would-be filers would come into contact—from court staff to law librarians—to provide guidance that could be construed as legal advice (22).

Prior work has established that even seemingly small roadblocks can have large effects on whether people are able to pursue their goals (19, 23). For instance, long wait times and congested office locations deter people from completing applications for public benefits. One study calculated that the effect of such hassles (or “ordeals”) was so large that low-income individuals were foregoing enrollment despite the fact they stood to gain approximately $100 per hour spent on the application process (24). Field RCTs have documented an inverse relationship between ordeals and participation in assistance programs among eligible individuals (25, 26). This is perhaps one reason why automatic enrollment has been shown to increase program take-up by large margins (19, 27, 28).

What ordeals did an individual seeking divorce without the help of an attorney face? In Philadelphia, a typical process might have unfolded as follows. Pro se litigants who came to the courthouse found no information center providing assistance in divorce matters and no statewide or court-approved forms. Courthouse policy prohibited clerks from providing legal advice, which deterred many clerks from answering basic questions about which forms were appropriate for the type of the divorce the person was seeking (22). Individuals were frequently referred to a law library several blocks away, where they could purchase photocopies of a self-help manual that guided them through the filing process (18). Library staff were also prohibited from providing legal advice, leaving individuals to navigate the 166-page guide on their own.

We examined these self-help materials for further insights into the hurdles litigants faced. The instructions for how to complete the first step—drafting the Complaint in Divorce—spanned ten pages (18). Subsequent pages instructed litigants in how to prepare the Notice to Plead, the Counseling Notice, a photocopy of the marriage certificate, and the Domestic Relations Information Sheet. Those who sought divorce based on a 2 y separation also had to file a Plaintiff’s Affidavit under Section 3301(d) of the Divorce Code, which notified the nonfiling spouse that they had 20 d to challenge the statement that the two had been separated for 2 y (18, 29).

After drafting a complaint, the litigant had to pay a filing fee, which during the study period was over $300 (18). According to the self-help materials, this filing fee could be paid by debit card, credit card, or money order but not by cash or personal check (18). Pro se litigants seeking to apply for a fee waiver had to file an In Forma Pauperis Petition (30), which required photo identification and proof of limited income, meaning additional paperwork (18). After completing these forms, the litigant was advised to find a photocopier to make two copies: one for her own records and one for service of process (27).

During the study period, a litigant could file a divorce case in person with the clerk’s office, which closed at 4:00 PM on weekdays and had no weekend hours (18). A litigant could also file by mail, but the instructions in the self-help materials—to which clerks and librarians referred pro se litigants—did not explain this fact (18). Once the case was filed, a litigant would need to clear additional hurdles, including identifying and filling out forms with abstruse names such as “Praecipe to Transmit the Record to the Prothonotary” (29, 31). A final form, according to the self-help materials, had to be filled out with a typewriter; it could not be handwritten (18).

RCTs as a Measure of Access to Justice.

We began this paper by highlighting the need for a measurement device to quantify the accessibility of an adjudicatory system against the access-to-justice movement’s aspirational goal of 100% accessibility. We proposed that, in a 100% accessible system, the presence or absence of a traditional attorney–client relationship should not make a large difference in the outcomes would-be litigants experience and proposed using field RCTs to measure the size of any such difference.

We anticipate that, if our measuring stick is adopted and implemented, we will observe a range of accessibility measurements in different adjudicatory systems across different legal contexts. We hope that the access-to-justice field will eventually settle on ranges or rules of thumb regarding how large a gap between represented and unrepresented litigants is consistent with the value of equal justice under the law. In the absence of such a benchmark, we submit that the differences observed in this study are too large for the court system to be considered meaningfully accessible, particularly when one considers that a fundamental constitutional right is at stake. The barriers must be lowered so that litigants who cannot afford an attorney can nonetheless vindicate their rights.

Materials and Methods

This study was approved by the Harvard Committee for the Use of Human Subjects in Research (#19632).

Study Partner.

Philadelphia VIP accepts individuals referred by other organizations who have exhausted all other options, and it attempts to match them with a volunteer private attorney working pro bono. Between Philadelphia VIP’s staff and its stable of volunteer attorneys, the organization serves ∼3,500 individuals and families annually on civil legal matters, such as eviction, guardianship, and divorce.

Participant Eligibility.

If an individual consented to take part in the research study, Philadelphia VIP securely transmitted a record of this intake interview to the study team. Philadelphia VIP forwarded a file only if there was a legally sufficient basis to initiate a divorce lawsuit.

During our study period, Pennsylvania law allowed for no-fault divorce, including divorce based on “mutual consent” of the spouses that the marriage was irretrievably broken. It also allowed for divorce based on “2 y separation,” meaning that the parties had lived apart for more than 2 y and did not contest that the marriage was irretrievably broken.

Study Population.

According to intake interviews, the study population was predominantly Black or Hispanic (59% and 22%, respectively). Participants were overwhelmingly female (80%), which is not uncommon for legal aid organizations that handle cases involving domestic relations (32). The average participant was 42 y old and had been married for 12 y. Most participants had been separated for 2 y (56%) with only 8% still living with the opposing spouse. 19% had been divorced before. Nearly half of participants had a minor child (43%).

To be eligible for Philadelphia VIP services, participants could earn at most 200% of the federal poverty guidelines. In practice, study participants earned much less, with a median monthly income of $0 and mean of $410. Two-thirds received some form of public income support. Income was meager even among the 35% of participants holding jobs: half of employed participants earned $12,000 per year or less.

Study participants reported that they were less financially stable than their spouses. Nearly 50% of opposing spouses worked, and of those, half earned more than $28,800 annually. Opposing spouses were twice as likely as participants to anticipate a pension payout in retirement. Disparities, albeit less pronounced, were observed for durable and liquid assets. Legally, the economic imbalances between spouses may have entitled study participants to alimony and a share of the marital estate, but many participants expressed no desire to assert economic claims. The primary desire study participants expressed was to transition from a state of married to one of not married.

Experimental Design.

Two levels of assistance.

Philadelphia VIP offered potential divorce clients two levels of assistance. The first, and preferred, level of assistance was an effort by Philadelphia VIP to find a volunteer attorney who would engage in a traditional attorney–client relationship with the individual. The effort to match individuals with a volunteer attorney typically took Philadelphia VIP staff anywhere from 1 to 6 mo because of the high demand and low supply of volunteer attorneys.

The second level of assistance was to connect participants with alternative resources, such as written instructions on how to obtain a divorce and access to a “counsel and advice” hotline staffed by Philadelphia VIP. A limited number of cases were referred to the Philadelphia Bar Association’s Modest Means Program, through which low-income participants could obtain private attorneys on a reduced-fee basis. Ordinarily, if Philadelphia VIP offered its second level of assistance to a participant, there was no other organization to which that person could turn to obtain free attorney representation because Philadelphia VIP was the legal services provider of last resort.

Randomization.

Study intake lasted from January of 2011 until July of 2013, allowing us to randomize 311 participants. For the first year, Philadelphia VIP had the capacity to offer the first-level assistance to approximately half of participants. In early 2012, budget cuts at another legal aid organization led to a large increase in individuals seeking assistance from Philadelphia VIP. The percentage of participants to whom Philadelphia VIP was able to offer first-level assistance fell from 50 to 15%.

We had anticipated the need to vary the randomization probability during the study. Accordingly, we randomized in batches ranging in size from 10 to 20 cases and stayed in contact with Philadelphia VIP to adjust the randomization probabilities based on the expected volume of cases and Philadelphia VIP’s capacity. We did not inform Philadelphia VIP of the batch sizes.

Data.

Court files.

We obtained files for all divorce cases involving study participants in the Philadelphia Family Court. A review of these case files provided us with our primary source of information for two key outcome measures: 1) whether the individual had a divorce case on file within 18 mo of randomization and, if so, 2) whether the individual’s marriage had been terminated within 36 mo of randomization.

In all cases, we looked for whether a study participant had a divorce case on record, regardless of whether the participant had been the one to initiate the case. The absence of any case record indicated that the individual had not filed for divorce in Philadelphia County. Based on conversations with Philadelphia VIP, we also searched court records at six counties outside of Philadelphia: Bucks, Chester, Delaware, Montgomery, Cameron, and Potter.

Filing in Cameron or Potter Counties carried significant drawbacks for participants. First, any Philadelphia-based study participant who ordinarily would have qualified for a fee waiver due to their low income was required to pay the filing fee, which in 2016 was $86 in Cameron and $84 in Potter, because neither county accepted fee waiver petitions from plaintiffs living outside the county (33–35). Second, the case could be dismissed for improper venue if the opposing spouse objected (35). If that happened, the filing fee would be lost and the participant would need to start over with a new lawsuit. Finally, if the court were to require the litigant to appear, the participant would face a choice between losing the filing fee (and failing to obtain a divorce) or driving 10 h round trip. Record systems in Cameron and Potter counties did not allow us to assess how frequently venue objections or court appearances occurred, but we were able to observe that most filings in these counties resulted in divorces. Thus, we infer that objections and court appearances were infrequent.

Philadelphia VIP records.

In addition to consulting court records, we also obtained additional information directly from Philadelphia VIP. We learned whether participants had abandoned their cases, reconciled with their spouses, continued to seek a divorce, or moved away from Philadelphia. Philadelphia VIP also shared nonconfidential information about telephone calls or other communications with the participant in which the organization conveyed legal information. Finally, Philadelphia VIP staff contacted the volunteer attorneys to ask whether any participants had entered into informal “side agreements” regarding, for example, property or assets that were not filed with the court. None had, indicating that all agreements between spouses were visible to us through the court files.

Analysis Strategy.

To estimate the causal effect of the presence or absence of attorney representation (which we did not directly randomize), we made three key assumptions.

First, we assumed that Philadelphia VIP did not affect client outcomes in the treated group except through the effort to find an attorney. This assumption could have been violated if, for example, Philadelphia VIP staff provided large amounts of telephone advice or other forms of assistance to treated group participants. Because Philadelphia VIP reported few instances of contact with participants in the treated group, we believe this first assumption is plausible.

Second, the statistical modeling would work better when most study participants randomized to treatment ended up with an attorney–client relationship and when most study participants randomized to control did not end up with such a relationship (36). Because 87.8% of participants randomized to a Philadelphia VIP effort to find a lawyer ended up with one, while roughly 36.7% of those randomized to no such effort found one, we believe this second assumption is plausible.

Third, we assumed that no participant would always receive a treatment opposite the one randomly assigned to her (i.e., “no defiers” assumption) (36). In other words, we assumed there was no participant who would experience an attorney–client relationship if randomized to the control group and would experience no attorney–client relationship if randomized to the treated group. Such a situation is hard to imagine for our study participants.

Notably, it is not possible to estimate the effect of attorney representation for all study participants. An unknown subset of participants (“always-takers”) would always be represented by an attorney, even if they were assigned to the control group. Similarly, a different subset of participants (“never-takers”) would never be represented by an attorney, even if assigned to the treated group. For these kinds of participants, it is not possible to estimate the effect of attorney representation because their representation status did not vary as a function of our random assignment (36).

Thus, the effect of attorney representation is measurable only for the subset of participants who would have received representation if randomized to the treated group and would have not received representation if randomized to the control group (compliers) (36). As a result, we report the estimated effect of attorney representation among this subset of participants, known as the “complier average causal effect” or the “local average treatment effect” (36).

Data Availability

All de-identified datasets, analysis code, output, and study materials are available on the Open Science Framework at https://osf.io/yn3rw/.