Person or place-based policies to tackle disadvantage? Not knowing what works
A review of the effectiveness of policies introduced in Great Britain since 1997 to tackle employment, education and income disadvantage, focusing on policies that explicitly take account of people and places.
While the Government has sought to tackle disadvantage across a number of fronts since 1997, person- and place-based policies have mostly developed separately and often in isolation from each other. This separation does not reflect the relationships between places and the poverty and disadvantage of people who live in them.
This study looks at evaluations of the policies targeted at people and places to draw out key messages about what works, comparing and contrasting the effectiveness of person- and place-based interventions.
Summary
This study reviews evaluations of person-based policies (targeted directly at individuals) and place-based policies (designed to tackle neighbourhood deprivation) to find out what kind of interventions are most effective at tackling disadvantage. The review concentrated on policies to improve employment and educational outcomes and to increase incomes.
Key points
- Most policy interventions, whether person or place-targeted, had small, favourable impacts. In the rare cases where information on expenditure was available, costs were generally offset by savings to the Exchequer.
- Both forms of intervention had significant positive impacts on particular aspects of education results and employment. However, it was not possible to determine whether person or place-based policies were better, as they tended to have different objectives that prevented direct comparison.
- Some interventions had negative consequences for the average participant or detrimental effects on some groups of participants.
- It was rarely possible to explain properly how policy interventions worked or why they failed, because the way they were intended to work was not always publicly spelled out in advance.
- Evaluators judged policies to have the greatest impact if they delivered individually tailored support to the most disadvantaged people with minimal complexity. The evaluators considered policies successful if they reflected local needs and priorities and were shaped by active engagement with stakeholders, including end users.
Why evaluate evaluations?
Since 1997, the UK Government has sought to tackle disadvantage across a large number of fronts, stressing the importance of employment and personal responsibility, the scarring effects of childhood poverty and the enabling effects of strong neighbourhoods and social inclusion.
For the most part, policies targeted directly at individuals – person-based policies – have developed separately from policies designed to tackle neighbourhood deprivation – place-based policies. This separation partially reflects the interests and agendas of the different government departments implementing policy, particularly the Department for Work and Pensions (DWP) and Communities and Local Government (CLG).
While many of the policies have been evaluated, little is known about what kinds of policy are most effective or about the cumulative impact of policies targeted at people and places.
The objective of this review was to evaluate the effectiveness of selected policies implemented since 1997 that tackle disadvantage. The policies included were those that addressed material poverty via increased employment, better education and directly through higher incomes. The policies are further distinguished according to the relative importance of people and place in objectives, targeting and delivery.
Limits to knowledge
While many policy evaluations have been conducted since 1997, comparatively few have been assessed against a counterfactual – an estimate of what would have happened had the policy intervention not been introduced. Random controlled trials are very rare and area-based comparisons have frequently not controlled for differences in area characteristics. Many policies have been evaluated shortly after implementation despite evidence that policies need time to ‘bed down’ first. Studies of policies that have been repeatedly evaluated show that impacts vary over time.
Because person and place-based policies often have different objectives, diverse measures of policy outcome were used in evaluations. This meant that it was generally not possible to gauge the relative effectiveness of person and place-based policies. In future, it may be possible to make such comparisons using administrative datasets that link individual-level information over a prolonged period of time. These have recently become available to researchers.
Conclusions from the evaluations about what worked were almost invariably based on retrospective reasoning and are problematic, as the same facts can support a large number of interpretations. There was rarely a clearly stated theory of change that explained how the policy was expected to work by connecting the policy objectives to the programme outcomes, via a clear delivery route and a set of administrative actions and their presumed consequences.
Impact on employment
Most of the interventions assessed had a measurable positive effect in helping people to move off benefit and, in some cases, to find employment. Of the person-focused policies that could be directly compared, Pathways to Work (aimed primarily at Incapacity Benefit recipients) and New Deal for Lone Parents had the strongest impact on the employment rate of participants, relative to controls (see Figure 1). Both of these policies targeted groups that were traditionally considered economically inactive with no obligation to look for work. It is possible, therefore, that both schemes tapped a pool of people eager to return to work who had previously not been encouraged to do so. Pathways to Work added condition management programmes (personal, health-focused support) to existing provisions for disabled people seeking to return to work. In addition, work-focused interviews were made mandatory, which may have reached further groups of more work-ready claimants.
No firm conclusions can be drawn about the relative effectiveness of person and place-based initiatives, other than that one is not clearly superior to the other and that both can have positive impacts on getting people back into work. However, comparisons of Employment Zones with compulsory New Deals and of the Working Neighbourhoods Pilot with the Employment Retention and Advancement intervention show that these particular place-based policies have noticeably better outcomes than person-based ones targeted at similar groups.
Figure 1: Impact of person-centred policies on employment entry rates
The evidence also suggests that the interventions cost the Exchequer comparatively little once account is taken of income from tax and national insurance contributions derived from extra earnings, and of savings in benefit expenditure from people placed in work. While these results are robust, they were achieved against a backdrop of general economic and labour market strength.
It is very difficult to account for differences in the relative effectiveness of interventions, for example, the strong positive impacts of NDLP and Pathways to Work compared to other person-targeted initiatives. A number of the initiatives evaluated struggled to help people furthest from the labour market, who face severe or multiple barriers to employment and need additional and sustained support. However, outreach services and attention to basics, such as accessible premises, can help to support vulnerable clients, even within mainstream provision. Such targeting is best accompanied by flexibility and services that can be tailored to specific needs. This, in turn, requires authority for decisions about provision to be transferred from the centre to those working directly with clients. However, such devolution of responsibility is likely to work well only where staff are adequately trained and properly supervised.
Educational outcomes
A large number of policy interventions have been introduced in the education sphere since 1997, though few have been subject to well-controlled evaluation. The best evaluations show that impacts are generally not large, and can be adverse.
The Sure Start programme, which aims to address disadvantage at the earliest stage of a child’s life by supporting children and their parents (as parents and workers), appears to have had no impact on children’s health or development. However, evidence suggests that improvements were achieved in selected aspects of children’s social functioning and behaviour at the age of three, apparently associated with improved parenting and family functioning. These benefits were, though, limited to children in the least disadvantaged homes. Indeed, there is consistent evidence of adverse outcomes for three-year-olds born to teenage mothers or living in workless households: verbal ability is lower and the children of teenage mothers show more behavioural problems and poorer social functioning than those living in non-Sure Start areas. Less disadvantaged families possibly ‘queue jump’ more disadvantaged ones, who may also respond more negatively to the services on offer.
Of the five programmes aimed at improving education attainment at Level 2 (five GCSEs grades A*-C or equivalent) only two, New Deal for Communities (NDC) and Academies, had a discernable impact on attainment at or above the threshold. Moreover, the NDC result is subject to qualification, the evaluation having been based on a basic ‘matched’ area comparison, i.e. compared to control areas with similar characteristics. These results can be partly explained by the multiple objectives of the various policies. In some cases, greater priority was given to other goals, such as staying on in school or achieving a different level of qualifications. Indeed, policies occasionally had contradictory goals.
The success of Academies was associated with strong, consistent leadership within schools and the availability of additional resources for the buildings, ICT and equipment necessary to create more positive learning environments. Evaluations suggested that person-focused educational interventions work best when flexible programmes target specific needs and creative strategies include the most disadvantaged. Other factors associated with success in the evaluations include: parental support and effective parent-school dialogue; high staff-student ratios; effective leadership and accountability; and the promotion of independent learning, supported by regular feedback.
Income and living standards
Despite the Government’s commitment to the eradication of child poverty and the considerable body of new policy – including tax credits and the National Minimum Wage – that has been implemented to achieve this goal, very few initiatives have been comprehensively evaluated. In part, this may be because policies that involve increasing social security benefits are presumed to impact directly on poverty, although in reality their effect will be mediated by household circumstances and benefit take-up. However, even those reforms that have been extensively evaluated, such as tax credits, have rarely been assessed for their anti-poverty impact. Instead the policy community is reliant on micro-simulation studies (i.e. studies that forecast changes over a relatively short period of time) based on behavioural assumptions that are rarely tested.
The two evaluations reviewed related to the Employment Retention and Advancement (ERA) demonstration and to Work-Based Learning for Adults (WBLA). The former showed that lone parents moving into work through ERA had 12-month earnings that were 29 per cent greater than for the control group. The evaluation of WBLA, however, revealed that movements into work do not necessarily translate into higher incomes because the jobs that people took under the Longer Occupational Training (LOT) scheme were generally low paid.
Insights and possibilities
While person and place-based initiatives can both be effective, their relative effectiveness cannot yet be adequately addressed. However, the development of datasets that link individual-level information over a prolonged period of time may soon make this possible, if only retrospectively. If more is to be learned from the next generation of evaluations, it must always be clear how policies are expected to work, and these ‘theories of change’ must inform the design of impact and process evaluations and the interpretation of the results.
About the study
The review was carried out over six months beginning in November 2006. Policy interventions and their evaluations were identified through a literature search and advice from a panel of academics and policy-makers in central and local government. To be included in the review, evaluations had to have been conducted with a counterfactual, a method of establishing what the outcome would have been if the intervention had not been implemented. They were also required to measure effectiveness, that is, to compare performance against stated policy objectives for at least one relevant outcome variable, and to include a comprehensive and examinable methodology.