Building Advice: The Craft of the Policy Professional - Working paper

Part 1: Introduction: New Ideas for Professionals

Policy analysis and policy advice are applied practices. Whatever the context, or whatever the beliefs about the proper role for the analysis, the hallmark of policy analysis is a systematic comparison of alternative policies in terms of goals that specify the desirable attributes of a good society. Policy analysis improves largely through the steady accretion of experience, reflection, and theories that are the joint products of practitioners and scholars. Occasionally, shifts in orientation or approach may be discerned. This paper focuses on new ideas about, and orientations to, the craft of policy analysis and the production of high quality advice in the policy literature.

The Scope of this Review

This report is a review of recent and seminal work, mostly academic, in the policy field. The review touches on both theoretical and practical issues involved in producing policy advice and carrying out policy analysis. As defined by the State Service Commission's (SSC) Improving Quality Policy Advice project,

High quality policy advice is free and frank, and allows decision makers to confidently and in a timely manner make decisions on how to address a particular policy problem, by presenting them with a range of clearly expressed, logical and practical options which have been developed through the application of analytic methods and information to that issue, and which set out the intended impact of each option in terms of the achievement of desired outcomes, plus mechanisms for implementation, monitoring and evaluation.

This report does not focus directly on high quality policy advice or the craft of policy analysis: this is not a how-to manual, or a guide to best practice. Instead, it approaches its objective obliquely through attention to key ideas in the context and concepts of policy advice, and to some of the main considerations in managing for high quality advice and the production of high quality advice. This approach stems from two premises: first, high quality policy advice is not a phenomenon that can be fully appreciated by examining its contributing features. Second, an ongoing relationship of knowledge and trust between advisors and ministers sustains an ongoing capacity to continue to improve societal well-being. Defined as an output, high quality policy advice is part of this greater whole.

Following this introduction, there are four main sections in the report. First, the paper reviews aspects of the policy environment that affect the work of policy professionals. The section is headed "blurred boundaries" to draw attention not only to the difficulties analysts face in setting up their work and ensuring its adequacy, but also to acknowledge the constant change in the environment. The next section looks at recent theoretical and conceptual developments in policy scholarship. This section highlights the strand of scholarship that has come to be identified as "beyond positivism", as well as the increased attention given to problem definition and evaluation. In the following section, several key choices required to ensure the substantive quality of advice over time are examined. These choices include the way policy units set and manage their work programmes. Finally, the literature on requirements for the process of producing and presenting quality advice is reviewed. A summary section synthesises some key themes from the review. The paper ends with a bibliography that includes a review of the main policy texts of the last ten years.

Within each section, however, it has been necessary to be very selective. First, most attention in this review is given to materials that have been published in the last seven years, or roughly since the publication of the SSC's Policy Advice Initiative, the value of which has not diminished since 1992.

Second, while policy relevant material can be found in a vast array of fields, the focus here is principally on material written explicitly for a policy audience (or, if written initially for another group, interpreted explicitly for a policy audience). This filter cuts out much work in theory and method within disciplines that contribute to policy analysis, where that work is not applied to policy analysis

Third, while many insights for the craft of analysis are revealed by published analyses themselves, this truly vast literature has not been surveyed. Instead, analysts are encouraged to spend some part of every major assignment researching the specific literature bearing on their task. Similarly, it has not been the intention of the paper to provide a review of recent methodological advances, and the reader is referred to the specialised literature.

Finally, the timeframe during which this report was originally completed meant that the materials reviewed were limited to the collections held by SSC and the author. It is well known that published academic accounts, in whatever field, are biased in several ways: they may reflect the inclinations of editors as much as "true merit"; writers choose topics in part to seek peer status, and so on. In the present case, there is a pronounced, and regrettable, absence of practitioner reflections on their practice. Only a modest number of very recent materials were assessed for the 1999 revision.

Three principal conclusions emerge from the review. First, in a climate of significant pressures to do more with less, there is a need to identify quality priorities and to be more proactive in prioritising analytic needs in order that a suitable package of quality enhancement can be instituted. Second, high quality policy advice flows from a community of policy professionals that is managed and led with skill to identify, support, enhance and reward intellectual capital. Third, to be of high quality, advice must be set in a broad context. Depending on where advice fits in the policy process, attention may be needed to links with strategic objectives and other policies, to the range of values affected by the options considered, or to links with prior and subsequent phases in the policy process.

New Ideas in a New Field

Policy analysis emerged as a distinct field in the 1960s. Analysts applied economics, operations research and related approaches to public policy problems. Typically, analysts were academics on short-term assignments, were situated independently near the top of organisational structures and established a mode of exchange with decision-makers built on trust, understanding and reciprocity (Radin, 1997, p.204). Gradually, policy analysis functions increased and analysts became career bureaucrats (Radin, 1997, p.206). Over time, the strengths of the original approach have been built upon, and the weaknesses addressed through a variety of proposed alternative approaches. Reflecting on the course of policy analysis, Peter deLeon argues that it is time for "three steps back to the future" (deLeon, 1994). His three steps revise the policy science paradigm, develop more participatory policy analysis, and focus on problem definition--are echoed in much of the recent work on policy analysis.

This report, by focussing on new ideas in theory, has a distinct bias. None of the new ideas are held up as "ideal". Scholarship advances by pushing at boundaries, and many new ideas fail to maintain importance. At the same time, old ideas that have maintained their value are necessarily under-emphasised. Ideas that work form the core of a wide consensus among policy practitioners. In New Zealand, that consensus appears to encompass several beliefs (SSC, 1992; Wolf, 1998). These include a view that the policy environment is pluralistic, that the role of a government analyst is to help policy-makers look in different ways at the nature of social problems and that there is usually not one best solution to a problem. In addition, most analysts in New Zealand would agree that their role is to provide objective advice. Analysts value their analytic integrity and take seriously their client-oriented responsibility. There is also a great deal of consensus regarding the importance of the foundational tools of policy analysis, including cost benefit analysis and statistics and other quantitative methods. Stokey and Zeckhauser's text (1978) is still in print and still provides a clear, non-mathematical introduction to core methods in analysis (see also, Miller and Whicker, in press).

The Reader

This report is mainly addressed to senior policy analysts, policy unit managers, and others with an interest and engagement with these roles. The reader is therefore assumed to have some practical experience with policy analysis in the policy process. Because it is essentially an academic literature review, the report is pitched to the practitioner who displays or aspires to be a "reflective" practitioner.

According to Schön (1983, viii-ix), the reflective practitioner in everyday dealings displays a tacit "knowing-in-action", but has a capability for "reflection-in-action" through which skilled practitioners often think about what they are doing while they are doing it on-the-spot in situations of uncertainty, uniqueness, and conflict. This person also has the capacity for reflecting on knowing- and reflecting-in-action, through which practitioners can formulate and criticise their action strategies and their ways of framing problems and roles. Using a similar construction Dunn (1994, p.74), implies there is a need to build a bridge between espoused theory and theory-in-use ("reconstructed logic" and "logic-in-use", in his terms) if "academic" ideas are to be of use to the practitioner. The academic ideas are in this report; the practitioner can supply the rest.

Definitions

Since terms are often variously understood, some definitions follow.

• Policy advisor and policy analyst, used interchangeably, refer broadly to non-elected people within government whose work involves the design, implementation, monitoring or evaluation of policy.
• Policy is construed loosely, ranging from large and strategic issues to small operational ones. (Colebatch, 1998, investigates policy as a concept.)
• Policy advice is policy advisors' output in the form of oral briefings, memos, Cabinet papers, speech notes, and the like.
• Policy analysis is taken in a broad sense, consistent with the use of advisor and analyst, and following Weimer and Vining, (1999, p.27): "policy analysis is client-oriented advice relevant to public decisions and informed by social values". (The last phrase, "and informed by social values" was added for the second edition, 1992.) A narrower definition, however, remains prominent in the literature. In these works, policy analysis is the process of applying a defined set of procedures and tools, largely drawn from economics and related disciplines, to public policy problems. The difference extends beyond mere semantics; proponents of the "narrower" definition contend that analysis is a discrete step in the policy process and other steps are more properly in the domain of politics, and hence, not a part of a public servant's role. Proponents of a broader view understand analysis to be far more encompassing, and hold that analysis and advice may occur at all stages of the policy process.

A Profession that "Builds" Advice?

The title of this report summarises two points of strong, but not universal, consensus in the recent literature. First, there is a recognisable "policy professional" whose output is advice. As Hawke (1993) expresses it, policy analysis is a professional activity concerned with the "contribution of research to the development of policy advice" (p.1). Second, advice is constructed or "built" by professionals who practice a craft.

Policy professionals are typically divided into one or another set of categories. One common typology distinguishes policy managers, policy specialists (also called, somewhat ambiguously, policy analysts) and technical specialists (SSC 1991; Scott, 1992). Anderson (1997) refers to policy generalists, specialist personnel, and policy managers. A third scheme identifies policy experts, concerned with information and guidance on substance; policy advocates, who push for implementation of their own plans and goals; and policy troubleshooters, who work with process issues on behalf of a client (Snare, 1996). Yet another writer distinguishes analysts according to policy focus: sectoral (analysts in ministries of environment or women's affairs), specialist (servicing regulatory departments, such as Labour, Conservation, or Customs), and general (working in the central government Treasury, State Services Commission, and Department of the Prime Minister and Cabinet) (Hunn, 1994).

Regardless of specialisation or categorisation, all roles share a focus on policy that distinguishes people in these roles from others with similar training (in, for example, management or statistics). The categories are no longer separate occupations divided by a common output (to paraphrase an old cliché). Just as the profession of architecture may have specialists in drafting, costing and visioning, or specialists in the design of petrol stations and art museums, so the policy professional may be skilled in strategy or in trend analysis, in education policies or women's affairs. Today, it is more accurate to speak in terms of separate functions united in a common policy profession.

The view of the policy field as a profession has doubters. Lawlor summarises:

The jurisdictional boundaries are murky and ill-defined; there is neither consensus about nor hegemony over the core intellectual turf; and there are no protections in the form of credentialling or professional regulation. . . . [a] wide variety of conceptions of the field can be found. . . .The corps of practitioners is highly fractionated among disciplines, substantive interests and ideologies. . . . [there is] an apparent lack of intellectual ferment (1996, p.110-111).

While a serious critique, Lawlor's remarks are grounded in the experience of American analysts who see, at least at the national level, that "no amount of evidence, analysis and reasoning seems able to dislodge the gridlock of special interest capture and institutional inertia" (p.111). In the New Zealand government there is significantly more solidarity in the profession. This is evidenced in a number of ways, including the strong influence of the Treasury in setting boundaries for analysis, the relatively high numbers of lateral interdepartmental job shifts, and the applicability of SSC rules and guidelines across departments.

Professionals 'practice' their professions with a mix of knowledge and applied skill - the key determinants of 'craft'. Whether doctors, actors, or musicians, professionals use judgement to apply a knowledge base to a case at hand. Moreover, as Brunner (1997a) notes, professionals in the "old sense" of professional accept responsibility to use their acquired knowledge and skills in the common interest (p.192). This responsibility is captured in the Public Service Code of Conduct's principle of obligation to Government.

Policy professionals practice their craft in building a piece of policy advice. They collect and use data and information; discuss issues and options with others; and communicate their conclusions and recommendations. Like an architect, the analyst works to a client's requirements, under certain environmental and budgetary constraints. To pursue the analogy even further, quality of advice is not only a function of the piece of advice itself, but of how well it fits into its environment, and how pleasing it is to those who use it or observe it.

Part 2: The Policy Environment: Blurred Boundaries

The analyst today operates in an environment far different from the one in which the early policy scholars wrote and reflected. This context is dynamic and future focused. In any given circumstances, the implications of any description of, or prescription for, analysis may be unclear. While it is true that there was never a 'golden past' in which the landscape had clear demarcations, the landscape today is essentially one of blurred boundaries. Several key issues may be expected to 'make a difference' in the craft of policy analysis and its quality over the next several years.

Globalisation and Internationalisation of Policy Issues

Issues that may once have been uniquely 'domestic' may no longer be so. In some areas of policy, notably environment and trade, many prominent issues are international. The circles in which analysts operate - to share and acquire ideas or to argue and persuade - extend outside the country (OECD, 1995).

A much larger number of policy issues are 'global', in the sense that they concern many countries similarly, even though the details may be more or less unique. Refugee immigration policy is one example. Many issues of a more procedural and functional nature are also international. New Zealand's participation in many of the activities and processes of the OECD is evidence that New Zealand's problems are not unique, and others' solutions may be drawn upon to improve New Zealand's solutions. At the same time, New Zealand's size, location, partnership with Maori, and other unique features mean that imported solutions are unlikely to fit without adjustment.

Domains of Policy Responsibility

The types of issues that analysts work on change over time. Some policy responsibility has been devolved to local authorities. Much operational-level policy work is now the responsibility of delivery agents. In some cases, where responsibility has remained extensive, analytic support may be contracted in rather than produced in house. At the same time, small dedicated policy units face a seemingly endless task of preparing new, fairly high-level analyses.

The separation of policy functions - between ministries and between development and implementation - whatever its benefits, has also put pressure on co-ordination, strategic and long-term policy work and cross-cutting policy work (OECD, 1994a). Within this category, note should also be made of the ebbing and flowing of attention to distributional issues that accompany shifts in dominant government ideology.

Instruments of Policy

Analysts are trained to focus on public action, and to present options that include various forms of public action to decision-makers. In recent years, options that involve individual-based incentive instruments (market mechanisms) are gaining in attractiveness compared with instruments of government control, such as regulation and direct provision. Analysts may find themselves focused at the margins (Radin, 1997, p.214), where the workings of "government interventions" and "private decisions" are intertwined extensively.

The result is that analysts require greater sophistication with understanding and applying economic instruments of public policy, especially in new areas. At the same time, analysts may find themselves caught up in debates to determine the acceptable division between private and public decision making.

The "Information Society"

The information explosion has fostered a sense of fragmentation in the policy context (OECD, 1996b, p.7). Analysts are now exposed to a large pool of information, interpreted multiple ways (OECDb, 1994). This information explosion has also changed the context of the 'expert'. Media and interest groups have greater access to the same information available to analysts and with it the potential for greater influence. In New Zealand, the Official Information Act (OIA) has two implications for policy analysis, which exist in some tension. Knowing that a piece of work may be exposed to public scrutiny engenders care in its preparation. But openness can militate against free and frank advice. Issue fragmentation, greater and broader access to information and the OIA are all reasons why analysts need skill to make more strategic use of information.

Political Preferences for Means, as Well as for Ends

The myth of a politics-administration dichotomy has endured since Woodrow Wilson named it a century ago. According to this idea, politicians decide what is to be done, and bureaucrats decide how. In reality, the art of policy making is the creation of means/ends packages, and is largely developed in an evolutionary interaction sequence between minister and analyst. The analyst serves the minister by exploring and elucidating the contingencies of various combinations. However, the analyst's role may be complicated in any of three situations. First a government (or minister) may strongly prefer some means over ends. Analysts may find it hard to carry out their obligation to present, and have considered, a range options. Second, the minister may lack strong preferences. Here, the analyst is open to criticism of taking over the properly political aspects of the policy process if options are to go forward at all. Third, many situations are highly uncertain or change quickly. The analyst may be subject to a number of pressures: to produce results with severely limited information; to produce results under unreasonable time pressure; and so on. Analysts in all these 'hard' cases are exposed to a greater degree of professional risk should the analysis fail to meet the decision-maker's requirements. Analysts need to be vigilant to changes in the advisory relationship, and to have skill in negotiating the mix of means and ends.

New Zealand has largely avoided the "gulf of mutual comprehension" in C P Snow's infamous phrase between the culture of analysis (intellectual) and the culture of politics (Radin, 1997, p.214). The advisory relationship is direct. The unicameral Westminster political model creates a sharp focus and entails a certain degree of predictability. Nevertheless, MMP has the potential to elevate the importance of political argument in the analyst's task (Boston, et al, 1996; Martin, 1996).

Problems are Big and Hard

Problems may require more than one agency (or more than one specialist focus) to resolve. At the same time, many problems, including those usually addressed within one agency, may become increasingly resistant to change.

As Craswell and Davis point out (1994) most policy areas are interrelated. They are divided into separate boxes for convenience, and co-ordination mechanisms are superimposed. The clearest co-ordination devices in New Zealand are the overarching goals and strategic priority areas. Craswell and Davis also note that informal networks are needed to support the sharing of policy information and expertise. Formal structures may help to maintain links between policy areas. Interestingly (and against theory that policy choices would be settled intra-departmentally and not be put to Cabinet) an improved range of options to put to ministers may come out of co-ordination mechanisms which amalgamate policy areas: a 'proliferation of narrow options' is reduced; options that do go through are broader and more coherent; policy expertise is combined.

As noted by deLeon (1994, p.81), problems are now accepted as "resistant to change, and well beyond the power of [policy analyst's] models to replicate". deLeon suggests that the "way out" requires insights from other social sciences, including social psychology and sociology and political scholarship of institutional design and intergovernmental relations. Others have also warned that analysts should take note that there are many different conceptual perspectives, each different and necessary, none sufficient (Bobrow and Dryzek, 1987; Parsons, 1995, pp.32-54).

Demands for Public Participation

According to Mai Chen (cited in Hayward, 1997, p.411), there are 373 uses of the word "consultation" in new legislation. Similar demands for public participation are faced by governments elsewhere. Very little direct attention, however, has been focused on the analyst's particular needs and required skills in the context of demands for public participation. Participation is a broad notion, and includes consultation as a subset. This review looks at the latter, but the larger pressures of participation also exert effects on analysts. For instance, New Zealand's citizen's referenda law raise the public profile of some issues, and can influence the sorts of options put to decision-makers.

Emphasis on Value for Money

Policy advice is an output category. Ministers and chief executives negotiate its purchase price and quantity. In this context, there is strong interest in ensuring that advice represents value for money. However, it is difficult to specify in advance the precise features of the output. There is pressure to make continual improvements in the value produced at a given price.

Part 3: New Paradigms and Analytic Frameworks

The main new theme to have emerged over the last decade in policy analysis is that mainstream or traditional approaches, characterised by a 'positivist' epistemology, are no longer, except in rare instances, sufficient on their own to generate satisfactory advice. Positivists assume that individuals and societies behave according to 'laws' that can be discovered and used to design corrective interventions. According to this view, policy analysts can use 'objective' methods - drawn from microeconomics and statistics - to predict the outcomes of policy options, and can do so on the basis of 'facts'. Recognising the limitations in the assumptions, some theorists have attempted to 'update' the traditional approaches, devising methods to incorporate non-factual values, for example, in the economics of cost-benefit analysis. Others have responded by questioning the foundational assumptions (Campen, 1986; Zerbe and McCurdy, 1999), and by arguing for an alternative 'paradigm' in policy analysis.

Regardless of paradigm, the recent policy literature has focused attention on the problem definition and evaluation as a policy tool, not just an accountability tool--that is, on the beginning and end of the "policy cycle". Where implementation considerations remain central, analysts are taking up new approaches, notably those based on risk management and other private-sector management ideas.

Beyond Positivism

According to Howlett and Ramesh, a policy paradigm is

an intellectual construct intimately linked to policy subsystems. It is essentially a set of ideas held by relevant policy subsystem members . . . which shapes the broad goals policy-makers pursue, the way they perceive public problems, and the kinds of solutions they consider for adoption (1995, p 190).

In New Zealand, two paradigms have dominated in recent years, welfare economics and public choice. As Schick shows, (1996), these paradigms are blended uniquely in New Zealand. Nevertheless, this dominant blend is within the realm of the positivist policy 'intellectual constructs'.

Positivism in policy analysis "attempts to apply the lessons and procedures of the natural sciences to the social sciences' settings, trying to extract and codify universal laws and their responding behaviors" (deLeon, 1998). Post- positivism is the "contemporary school of social science that attempts to combine the disclosures of social and political theory with the rigor of modern science. It calls for a marriage of scientific knowledge with interpretive and philosophical knowledge about norms and values" (Fischer, 1995, p 243, cited in deLeon, 1998).

There are two principal reasons that the traditional positivist, 'objective' paradigm is said to be inadequate, particularly in complex analyses. Positivist assumptions narrow the analyst's attention to a small number of factors. These factors are said to be related in fixed, context-independent ways. Values do not affect the conduct of analysis (Brunner, 1991, p.82, quoted in Durning, 1996). The second driver is the acknowledgement that analysis exists in a "cacophony of opinions, beliefs, positions, conviction, rules and claims . . . The policy analyst is riding an argumentation carousel" (Hoppe, 1993, p.78, in Durning 1996). The implication of this second factor is that widespread participation is important in the policy process, that there is more than facts to take into account, and so a need for new tools.

The "post-positive" alternatives incorporate some form of subjectivity (Durning, 1996). Post-positive approaches are based on the assumption that all reality is mentally constructed and that all actions and behaviours are contextually determined. Thus there are no universal laws of cause and effect, and the "observer" together with the "observed" create the reality which the observer describes or analyses. The analyst, according to Majone (1989) creates arguments and engages in persuasive discourse involving stakeholders and the public.

Post-positivist alternatives include the nearly traditional "interpretive analysis" of Jennings (1987), the "hermeneutics" of Balfour and Mesaros (1994), and Rogers (1996), the "action theory" of Harmon (1989) and "critical theory" (Forester, 1993; see Bobrow and Dryzek, 1987, for a concise summary). In policy analysis, as in society at large, the word "post-positivism" stirs debate. Proponents of alternative perspectives maintain that practice under positivist assumptions produces poor advice. Reality, say the critics, is a series of decisions because peoples' behaviours mean that it is not possible to get it right the first time or to keep it right indefinitely (Brunner, 1997a, p.205). Rhetorical and advocacy perspectives at the more extreme edge of the post-modern response are dismissed by Lawlor (1996, p.120) as a "swamp of ambiguity, relativism, and self-doubt . . . creating] more problems for the policy analysis business than [they] solve" (Lawlor, 1996).

Lawlor draws attention, however, to the uncontroversial side of these contributions: policy analysis as a form of persuasive argumentation, which can be easily reconciled with traditional applications of tools of analysis (p.12). Policy analysis as argumentation has its roots in Lasswell's policy science tradition (p.112), but is most clearly set out in Majone (1989):

Its crucial argumentative aspect is what distinguishes policy analysis from the academic social science, on the one hand, and from problem-solving methodologies such as operations research on the other (p.7).

Between Research and Advocacy

Like Majone, Weimer and Vining (1999, pp.28-35) situate "policy analysis" on a continuum of professional activities of policy relevance. In their scheme, activities range from academic social science research through investigative journalism that focuses public attention on societal problems. "Analysis" is thus squeezed in two directions. On the one hand, there is pressure to be more objective (like academic research); on the other hand, there is pressure to advance the policy objectives of communities in society or one's own agency, to take values into account, and so on.

Post-positivist approaches may make it hard to distinguish analysis and advocacy. Some writers have declared the distinction redundant. The advocacy perspective, put forward in Sabatier and Jenkins-Smith (1993) holds that "most policy development is organized by belief systems (values, the identification of critical variables, perceived causal relationships) and advocacy coalitions. These coalitions either monitor the status quo, if they are dominant, or engage in policy-oriented learning that changes the conception of policy or the demand for change" (Lawlor, 1996, p 113). The principles of policy-oriented learning are that:

• analysis is usually stimulated by either threats to core values or perceived opportunities to realize core values;
• the crucial role of technical information is to alert people to the extent to which a given situation affects their interests and values;
• once political actors have formed a position on a policy issue, analysis is used primarily in an "advocacy" fashion, that is, to justify and elaborate a position; and
• actors generally find it necessary to engage in an analytical debate, that is, to present substantiation for their positions - if they are to succeed in translating their beliefs into policy (Sabatier and Jenkins-Smith 1993, p 45).

Lawlor (1996), in reviewing the contribution of Duncan MacRae to Fischer and Forester (1993), cites the importance of keeping a line between analytic work and persuasion. Policy analysis and advocacy should not be conflated; at the same time, analytical practice in adversarial settings demands more attention to persuasion, and greater care must be taken to keep the roles separate.

Jennings (1993) illustrates how these various ideas are applied in an approach to policy analysis that finds a middle ground between positivist objectivity and "subjectivism": Concepts used in social inquiry and political argument are, he says, drawn from a common set of norms and understandings. He continues:

The discourse within policy analysis can be understood as having three basic goals: (1) to grasp the meaning of significance of contemporary problems as they are experienced, adapted to, and struggled against by reasonably purposive agents, who are members of the political community; (2) to clarify the meaning of those problems so that strategically located political agents (public officials or policy makers) will be able to devise a set of efficacious and just solutions to them; and (3) to guide the selection of one preferred policy from a range of possible options based on a general vision of the good of the community as a whole (Jennings, 1993, pp 103-014, in Durning, 1996).

A study of US policy analysts and managers (Durning and Osuna, 1994) found little strong evidence that analysts practise post-positivism. However, when that study was replicated in New Zealand, there was evidence that two different "types" of policy professionals have at least some elements of the post-positivist outlook (Wolf, 1998; Durning, 1999). The studies' methodology involved asking respondents to rank a number of statements on the basis of their agreement or disagreement with them. The analysis of the data discerned that there are five broad "types" of analysts in both countries, (although only the first three types are in common; these three types, the objective technician, the client counsellor and the issue advocate are also noted in SSC, 1992, where it is argued that only the objective technician has a clear place in government).

There remains strong resistance to post-positivism. Many analysts believe that their clients demand 'objective' analyses, and their careers depend on successfully satisfying that demand. The organisational context is seen to support a relationship in which a 'technical' analyst serves the 'political' decision-maker. Essentially, the objective technician views the task as a process of analysing information to produce an answer to a given question. However, both the decision-maker's 'need to know' and the 'answer' need to be mediated by advisors/communicators who use skills different from those of traditional analysis. Analysts and policy units are increasingly developing capacity to span the roles.

Most training in policy analysis continues to draw heavily on the positivist assumptions. A survey of fifty-five American graduate programmes in public policy, public management, public affairs, and public administration is reported in the APPAM Guide to Graduate Public Policy Education and Organizations (1997). These programmes have several types of courses in common: microeconomics and macroeconomics, political analysis/policy process, statistics and quantitative methods, organisational theory and/or analysis, public or non-profit management, and ethics.

Alongside the emphasis on paradigms in the literature is an effort to describe alternative frameworks for analysis (Parsons, 1995; Bobrow and Dryzek, 1987). Frames make some topics more central than others, make some instruments more attractive, make some social consequences more legitimate, and are an ethical matter and so political, not just technical (Bobrow and Dryzek, 1987). Bobrow and Dryzek explore five alternative frameworks with attention to those factors:

• Welfare economics - the application of the theories and models of welfare economics to improving the rationality and efficiency of decision making.
• Public choice - the application of economics to political science.
• Social structure - public policy in terms of sociological theory.
• Information processing - diverse approaches that share an interest in how individuals and organisations arrive at judgements, make choices, deal with information, and solve problems.
• Political philosophy - a range of philosophical and social theories ranging from Machiavelli, through American pragmatism, to Rawls's theory of justice and Habermas's communicative rationality; (see also Parsons, 1995).

For illustration, welfare economics holds that values may be treated as prices, humans are instrumentally rational, the role of policy analysts is to identify and ameliorate market failure. Distributive issues are poorly addressed. By way of contrast, the social structure framework pays attention to distribution but with little concern for efficiency; policy makers achieve ends by manipulating social conditions and the analyst's role is to illuminate the consequences in terms of who gets what. Brunner (1997a, pp.200-201) elaborates on this framework: "The social process model directs attention to seven value categories: participants, their perspectives, the situations in which they interact, the base values, strategies they use to shape outcomes, and the effects of those outcomes".

Parsons (1995), while pointing out that frames are not necessarily exclusive or incommensurate, adds three frameworks: political process (approaches to explaining the political context of policy making); comparative politics (methods that use comparisons by socio-economic factors, by political party or by institution, for example); and management (the use in the public sector of techniques once limited to the private sector).

Analysts need not choose a framework (or a mix) in order to do quality work. Frameworks, however may be very useful in assisting to understand the context on the analysis and the pressures facing the decision maker. Thinking is alternative frameworks may help the analyst to understand and clarify the structure of analysis (Parsons, 1995, pp 56-57) by asking four questions: Whose knowledge is being used? What kind of knowledge does it claim to be? When is knowledge produced, used, ignored? and How is knowledge used?

It is unlikely that post-positivist views will come to dominate analysts' work. It is more likely that additional approaches will take their place alongside the traditional approaches. Data may be drawn from new sources and used in different ways. Analysts will employ a range of participatory methodologies more extensively and directly in policy analysis.

The Beginning and the End

The policy process has usually been viewed as a series of analytically distinct "stages", usually described as a cycle. A typical cycle contains the stages, "problem definition, criteria selection, option selection, analysis, decision, implementation, evaluation". Bridgman and Davis (1999) also include consultation and co-ordination stages. Since the 1973 publication of Pressman and Wildavsky's seminal study until quite recently, a major focus of analysis was on implementation. The first policy units in the United States, in the 1960s, featured the term "planning and evaluation" and made no mention of "policy analysis". Now, there is emphasis on problem definition and evaluation (Chapman, 1995; Hawke, 1995; Turner and Washington, forthcoming). To adequately assess this shift, it is important to examine the most recent thinking on implementation.

Hill (1997a) reflects on the debate between "top-down" and "bottom-up" approaches to policy implementation. Top-down approaches have clearly been dominant. As a consequence of starting from a presumption that policy outputs are determined at the "top" as are plans to set them in place, we have a particular kind of picture of the implementation process. When things go wrong (as they inevitably do) terms such as "unintended consequences", "wrong models", or "inappropriate tools" are heard. Hill suggests that rather than thinking in terms of unintended consequences and implementation gaps, analysts come to recognize these features as products of inevitable messiness and incrementalism of the real world (pp.377-8). Against the traditional approach to thinking about implementation, (still useful in cases where the implementation of a policy is clear cut, but runs into difficulties, delays and errors) Hill puts forward an evolutionary model of understanding: "implementation" may be a process of collaboration among government officials and affected people. In the extreme, the collaboration can be co-production (as described in the work of Hanf, 1993, in which pollution policy was shown to be essentially the terms that the regulator was able to reach with the regulatee).

There are several implications for policy analysis. First, analysts must consider implementation issues as part of the policy formulation process. In their consultation efforts, they can recognise more explicitly the importance of the exchange of information with potentially affected parties for the legitimacy and durability of the policy once enacted.

It is useful to supplement older analytical approaches, which emphasise the early identification of implementation 'success factors' with strategies from risk management that can help to manage those factors. These strategies are useful even where the traditional approach to implementation remains valid, but especially where a more integrated policy-implementation concept is more accurate. In these new approaches, experts, ordinary citizens, and policy staff work jointly to clarify policy issues, trade-offs and solutions.

As Anderson (1997, p.479) notes, more and more policy issues, such as global warming and fisheries management, require the integration of scientific information into policy. Too often, it is frustrating for both sides: policy analysts see scientists as rigid and blinkered, while scientists feel the import of their findings is not appreciated. Recent developments in risk management tackle these issues. Comparative risk assessment (Ministry for the Environment, 1996) is a framework for assessing risks (not strictly limited to environmental risks) and highlighting important differences among them that can help to establish priorities and focus limited time and money where they are both needed. The approach is well-suited to bringing scientists, policy analysts and 'ordinary' citizens together to work on issues. Because it involves careful analysis, comparative risk assessment can avoid the allocation of resources to projects before there is good understanding of whether there is risk and whether it is manageable. The US National Academy of Sciences has published an approach to risk characterisation that proposes an iterative "analytic-deliberative" process (Stern and Fineberg, 1996).

In other domains of policy, risk profiling is the favoured term. Risk analysis for policy has some shared features across areas:

• the identification of problems to be assessed and compared;
• a set of criteria for ranking or evaluating problems and risk factors; a ranking process (often multi-stakeholder) resulting in a ranking in order of severity;
• a selection of priorities;
• analysis and selection of strategies to best address the problems or risk factors; and
• a plan to monitor and adjust the strategies. Clemen (1996) contains a good introduction to multi-stakeholder analysis.

Analysts also employ other strategies to assist implementation. Total quality management (TQM) is just one example of a cross-over approach to implementation from private sector management theory and practice. Jameson (1991, p.11) points out, "since we cannot determine the quality of policy advice by primary measures, i.e., end results, we have to consider the process which generates the advice. TQM can help us to understand and improve that process". TQM is based on four principles: continuous process improvement; a culture of employee involvement and empowerment; creation and maintenance of relationships with suppliers and customers; and the collection of data and statistical analysis to assess variations in performance. In policy applications, Jameson recommends disaggregating the process into component aspects and the use of checklists to maintain quality throughout the process. Cohen and Eimecke (1995) support the use of a TQM approach by "public entrepreneurs"- people who are willing to engage in non-linear thinking in the public sector. Entrepreneurs are alert to opportunity, willing to take risks in pursuing opportunity and able to co-ordinate the activities of others.

Problem definition

Increasingly, problem definition is emphasised as a key to successful policy analysis. Writers have come to recognise that several problem definitions may fit a given policy situation, and that problem perceptions matter to analysts and decision-makers. Many problems start off as vague worries that need to be structured before they can be solved. A number of disciplines and activities are drawn upon in problem definition. Problem definition is an ongoing and integral feature of the entire policy process.

In a very recent paper, deLeon (1998) draws attention to the function of the problem in analysis and the need for a complete problem structuring:

First, good policy research must be a function of the problem and resources at hand; otherwise the policy sciences come perilously close to manifesting Abraham Kaplan's famous suggestions that when all one has is a hammer, the whole world looks like a nail. Succinctly, the research problem must dictate the research design, taking it to whatever procedures and philosophies are pertinent. . . . It is difficult to imagine that a policy maker would, for any important policy problem, not be better off with at least an understanding of the conflicting forces than a problematic prediction.

A problem must be solvable to qualify as a policy problem (Wildavsky, 1979). Before techniques of analysis proper may be applied, analysts must "structure" the problem and ensure it is the right one. In Schön's memorable terminology, a practitioner must name the things to attend to, and frame the context: "In order to solve a problem by application of existing theory and technique, a practitioner must be able to map these categories onto features of the practice situation" (1983, pp.40-41).

Dunn's approach features distinguishing problem structuring and problem solving (1994, p.139). First a vague sense of a problem needs to be structured for analysis through a process of abstracting from the problem situation. Problem solving entails the application of "lower-order" methods to assess alternatives (Dunn, 1994, pp 138-140). Dunn details several techniques to assist in problem structuring ranging from boundary analysis through brainstorming to argumentation mapping (pp.161-183). Checkland (1989) has elaborated an approach called soft systems methodology that has found use in exploring the anomalies in legislation and the different perspectives people bring to policy problems.

Perceptions matter in policy, and psychology offers some useful approaches for assessing perceptions and understanding how they change. Irwin (1996) drawing on the work of Tversky and Kahnemann, 1974, among others) suggests that analysts try alternative ways to frame problems, the involvement of others, including those suspected to disagree; and a general vigilance of the psychology of human judgement traps - overconfidence, tailoring current beliefs to past choices, etc.

Various contextual analysis or perceptual/positional "mapping" techniques help analysts to anticipate ways in which recommendations will be seen by different actors and political concerns aroused (Heineman et al, 1997; MacRae and Whittington, 1997). Such approaches have the further advantage in that they may help analysts to properly estimate the decision maker's values, while simultaneously not overestimating the persuasive value of their own analysis (Heineman et al, 1997, p.54)

Evaluation

As with problem definition, the prominence of evaluation in the policy process has recently heightened. First, attention to evaluation is seen as essential to the success of policy. Second, some of the methods of evaluation are extremely useful in policy analysis itself. Finally, evaluation serves as a useful bridge among policy formulation and management, implementation and accountability.

Hinton and Dickie (1996) includes a comprehensive review of the implementation and evaluation literature, with an emphasis on creating a "checklist" of factors to consider in support of more effective policy implementation. Practical guides, such as the NZ Department of Labour Evaluation Guidelines (Labour Market Policy Group, 1995) present an overview of considerations to assist those carrying out evaluation projects, with specific attention to how quality evaluations can assist in achieving more effectively the goals of an organisation. Evaluation may consider questions about how well a given policy or intervention has been implemented and how it may be improved, and how well its objectives have been met. It may also help to answer questions about the appropriateness of the policy approach itself, through questioning its assumptions, and considering whether an alternative approach would be better. Strategic evaluations may focus on a broad strategy or suite of activities rather than on a particular programme.

Patton (1997) discusses what he calls "implementation evaluation", and provides a checklist of typical questions about programme design, feasibility, results, and lessons for future policy making (pp.213-4). An implementation evaluation includes both formative evaluation (assessment of programme activities and processes and how the programme may be improved) and summative evaluation (an assessment of whether a programme achieved its expected results).

Cook notes that over the last quarter century, evaluators have increasingly relied on research syntheses to draw more reliable and credible conclusions about programme implementation, programme effects, and the conditions under which different programmes are more or less effective (Cook, 1997, p.36). A range of studies under different conditions and carried out through different research methods can provide a better understanding of what activities work best for whom than a single study. A substantial and growing body of international research is available that reports evaluation results on a range of government activities.

Patton (1997) and Vedung (1997) note that evaluation techniques may be used to contribute to problem identification and policy formulation, at the beginning of the policy cycle. Patton describes a process of "developmental evaluation" in which evaluators work as a member of a policy team that develops a new policy approach. The evaluator's functions are to help draw out the logic of the proposed policy, to help test that logic, and to help develop an information base to promote better decisions (Patton, 1997 p.105). Vedung describes the process of "evaluability assessment" or "exploratory evaluation" developed originally by Joseph Wholey, in which evaluators work with programme managers to articulate the purpose of a programme and ensure that it is ready to be managed effectively (1997, pp.159-63). Evaluability assessment provides a basis for later, more extensive evaluation of programme effectiveness.

Intervention logic, programme logic or programme theory of action approaches (Funnell, 1997; Patton, 1997) are used primarily in evaluation, but may be applied to policy design and option selection analyses. Programme logic sets out a hierarchy of outcomes in the successful implementation of a policy or programme, together with the assumptions that link the outcomes. Each part of the "logic" may be questioned, extra attention may be provided to ensure that weak links are made, and evaluations may be developed to gauge the extent of key outcome success.

Organisational learning, as developed in the private sector, can be applied in the public sector. Theorists distinguish "single loop" learning which focuses on finding new strategies to achieve existing governing values and "double loop" learning for the adaptation of the governing values themselves (Matheson, 1994). Double loop learning results from questioning assumptions and confronting the traditions in an organisation (Leeuw and Sonnechsen, 1994, p.3).

Analytic Choices and their Management

The responsibility for quality advice lies with all participants in the process. This section focuses on the key management strategies for ensuring the capacity to provide quality advice at the level of a policy team or unit. Three areas are singled out: balancing day-to day and strategic activities, understanding, managing and benefiting from consultation, and enhancing generalist policy leadership and management. In all of these areas, the academic literature is rather thin. The starting point, however, is clear.

Trends suggest that government policy analysts will need to consult with communities of interest more widely than before, and that policies will have to be treated in a broader focus (e.g. Belgrave, 1998). This broader focus has two main dimensions. There is a need for more interagency consultation and long-term planning. Second, there is a need to provide advice that addresses most aspects of the policy 'cycle', including implementation, monitoring, and evaluation. To some observers, this means that analysts need to stay clearly focused on the "government's mandate, philosophical approach, and of political constraints . . . .to ensure that advice will be relevant" (Keating, 1996 p 65). Keating goes on to remark that a key input to relevant advice is to maintain attention to core business in balance with high priority issues. The expectation is that appropriate data will be collected and maintained to support a continued attention to core business.

Finger on the Pulse, Strategic or Both?

Every policy unit is confronted with a choice over the mix of inputs to day-to-day or routine activities and inputs to strategic advice. There is no bright line, however, between the two types of activities, and an imbalance may only be obvious in the extreme. Effective strategies will feature an integration of routine and strategic activities, so as to take advantage of synergies. For instance, if certain information is captured for purposes of routine advice, a body of experience is available for later strategic scrutiny. Increasingly, policy teams are incorporating a 'learning loop' in the quality assurance procedures. Questions are asked specifically to gradually build up a stock of institutional knowledge and to learn from it.

Strategic policy advice encompasses several forms. First, some advice may be considered (in the ideal) to be strategic in the sense that it focuses on the long-term well-being of specific groups in society. The advice, while centring on a group in society is at base advice for the good of all New Zealanders. Such advice is often viewed as essentially political. In the population ministries, this focus acquires a more explicit policy function.

Two other forms of strategic advice may be distinguished. Advice may be explicitly directed toward achieving an overarching goal or strategic priority. Or advice may be strategic within the more narrow confines of a ministry's mandate.

Strategic policy analysis counters some of the tendencies created by consultation pressures. If consultation creates an environment of expectation that a broad range of current issues and interests will be attended to, strategic policy analysis focuses on "broad, inter-sectoral, longer-term" demands (Boston, 1994). Corban (1994) considers strategic policy capacity to be the capacity to effectively co-ordinate policy and to look beyond current priorities and current policy preferences. Corban's investigation revealed that departments see it in their interest to anticipate future demands precipitated by major changes in policy context, so as to be in a position to better respond to them.

Others have placed emphasis on "ongoing" policy development, in which a strong, interdepartmental policy community shares best practices, addresses common problems, and works on issues collaboratively (Canada Privy Council, 1997). In reality, Anderson reports that the "greatest weaknesses in the current system are in addressing longer-term and strategic issues, including major horizontal issues" (1997, p.472). In New Zealand, the co-ordination and strategic demands of the overarching goals/strategic priorities system, while perhaps not a great weakness, nevertheless pose challenges that require constant vigilance (see Schick, 1996, in relation to SRAs).

Consultation

Consultations needed for quality policy advice are required both at interdepartmental and public/stakeholder levels.

In New Zealand, the starting point for defining consultation is the judgement in the 1992 Wellington Airport case (see Hayward, 1997, p.411): consulting involves the statement of a proposal not yet finally decided upon, listening to what others have to say, considering their responses and then deciding what will be done.

The Queensland Government defines consultation with an emphasis on accountability, absent in the standard New Zealand definition, but implied in some practice:

an open and accountable process where individuals and groups have a formal opportunity to influence the outcomes of a policy or decision making process. Through this formal opportunity, governments provide the community with a forum for participation in decision making, thereby promoting co-operative partnerships and more accountable public administration (in Hil and Roughley, 1997, p.24).

Both the New Zealand and Queensland examples emphasise process. Yet, what constitutes effective consultation eludes a definitive answer. Organisations struggle on "a fine line". Policies, Anderson (1997, p.475) notes, "do not emerge fully formed from consultations . . . Good consultations often walk between giving a sense of direction (or at least of major options) and being open to the views of the public". Furthermore, consultation functions as a government to people interaction; the role of the analyst can be ambiguous at a fundamental level. Where consultation is required, governments have a duty, at minimum, to listen to what people have to say. To some extent, this duty is carried out by analysts in the course of policy design: ministers may explicitly require that the advice contain options which are assessed for feasibility in light of consultation. But, in a larger context, it may be unreasonable to expect analysis to carry the full weight of consultative requirements and objectives. This issue has not been adequately addressed in the theoretical literature.

Theoretical work on consultation and policy advice is also thin. There is an extensive literature on public participation in decision making, in which consultation is one approach on a continuum (Arnstein, 1983). Particularly useful contributions are made by Fiorino (1990, 1995) who examines the contribution of participation (in the context of regulatory negotiation) to policy decision making. Moreover, to the extent that good consultation requires good methodology (in survey design or in interview technique, say) then normal research methods textbook contain relevant material. The final sources of insights are contained in reports on consultations and papers prepared as a basis for consultation. In this last category are both official summaries of consultations and scholarly articles (usually on consultation gone awry, such as Game, 1997). As noted in other places in this review, such descriptive work, together with the implied parameters of consultation that can be gleaned in requests for consultative feedback, can be useful to an analyst working in a specific area. For instance, Game (1997) reports on consultations focused on redrawing the boundaries of unitary authorities in shire England; VanNijnatten and Gregoire (1995) and Hil and Roughley (1997) focus on criminal justice policy in Canada and Australia, respectively.

One useful source for emerging best practice is the Canadian Public Consultation Guide (Sterne 1997). This reference details over 50 separate steps, before, during and after consultation. It does not, however, provide much assistance to make choices within these steps, for trouble-shooting, or for consulting under constraints of various sorts.

John Clayton Thomas (1995) has written a useful book, directed to "policy managers" who need to increase public involvement in government decision making. His model, called the "effective decision model of public involvement" helps managers to decide how and when to involve the public. It is structured to be a working guide, while at the same time providing the theory behind the suggestions. Guidelines are provided to help answer whether the public should be involved at all, and if so how intensively. His work addresses questions such as:

• Is there sufficient information to make a high quality decision?
• Is public acceptance of the decision critical to effective implementation? and
• Is there likely to be conflict within the public on the preferred solution?

Some attention to these questions could avoid exacerbating "consultation fatigue".

Consultation is valued for a combination of reasons. Fiorino (1990, 1995) considers that consultation can have instrumental, substantive, and normative rationales. Instrumentally, consultation may assist with the smooth adaptation of policy since participation increases the likelihood that the policy is understood and that stakeholders will have "bought in" to its aims and means. The substantive rationale for consultation reflects the complexity of policy issues and the dispersion of policy-relevant information among stakeholders. Consultation is needed to help ensure that all information needed for the successful design and implementation of policy is brought forward. Such information will touch on factors that will either support or hinder the policy. Finally, the normative rationale captures the thrust of consultation mandates from a democratic perspective: democracy requires the consent of the governed. At least since the wave of student demonstrations in the 1960s, which permanently changed the "democracy, participation and legitimacy" landscape (Pateman, 1970), at least some areas of policy decision making require some evidence of explicit consent. Clearly, which areas these are, and the nature and degree of the consent, remain subject to debate.

Other reasons for consultation have been noted. Consultation has a broadly educative value, irrespective of the content of a given process. People who are consulted, or who are offered an opportunity to be heard, also hear a message that policy trade-offs must be made, that politicians believe they have the rough outline of the desired trade-off, but that ordinary citizens are as well-placed as they are to fine-tune those trade-offs (VanNijnatten and Gregoire, 1995, p.205). Anderson (1997, p.475) remarks that consultations are obviously vital for officials and ministers, for both information and testing reactions, but they are equally part of a citizen's right to know and to participate in the process of government.

There is relatively strong consensus on some features that all consultations should include. Consultation is two-way communication between government and citizens. Thus, surveys and requests for views from the public are, on their own, not consultation. A common complaint about consultation is that the government organisation fails to report back to the respondents on how their information was used in arriving at a decision. Some New Zealand agencies are more effective in feedback than others. The OECD report on consultation (1997) lists "closing the loop" as one of the most important features of all consultations. Less frequently noted is the difficulty from the other side of two-way communication failure. Game refers to a misuse of "consultation" in which the one-way information from the public to government is over-regarded in the decision making process, resulting in "policy making by opinion poll" (Game, 1997, p.68).

The type of policy problem will suggest the appropriate consultation (OECD, 1997b; Thomas, 1995). Relevant dimensions are likely to vary among departments. The OECD emphasises a need to mix extensive and intensive information/opinion gathering approaches. It is essential to have clear transparent objectives and parameters of process of consultation (OECD, 1997b).

The open-endedness of the consultation process sits uneasily in a government culture more comfortable with control through bureaucratic hierarchy (VanNijnatten and Gregoire, 1995, pp.220-221). A key decision in a given instance of consultation, then, is the level within the hierarchy that is directly involved in consulting. Lower level staff may have more intimate knowledge of the details of the proposals and the interests of the various stakeholders, but are less able to move ideas forward when suggested in consultations. Higher level staff have more control but less time for focused attention to specific details and processes.

Consultation requires administrative commitment and organisational capacity (VanNijnatten and Gregoire, 1995, p.205). These authors go on to note that traditional administrative practices emphasise "efficiency" and using "facts as criteria for decision making", a "predilection for secrecy", a "mode of operation characterised by hierarchical structures and a specialisation of labour" (p.206). Under the Official Information Act, New Zealand is arguably no longer characterised by an overt predilection for secrecy (but, see Voyce, 1997), but the other features have bite. As well, New Zealand might suffer relatively less from strains of hierarchy.

Many writers argue that consultation should begin early and be ongoing, even though government retains final decision making responsibility (VanNijnatten and Gregoire, 1995, p.207). Consultation is thus seen as an integral part of policy process, not an add-on. To make it integral requires developing a consultation framework for each newly proposed policy initiative (Hil and Roughley, 1997, p.25). Because each policy is different, and because consultation is part of the process, it is unlikely that any single set of standard operating procedures will suffice. At best, more or less routine consultations may take shape according to departmental guidelines, but there will always be instances that require a fresh approach.

Several authors provide some suggestions for how consultation may be _framed_. These suggestions and others, below, for possible institutional arrangements must be considered with care in the New Zealand context. In particular, it is widely argued that consultation with Maori is an area fraught with difficulties and in need of re-conceptualisation and redesign from the ground up.

Hil and Roughley recommend using a variety of approaches which may be tailored in the framing process: requests for written comments, meetings, conferences, seminars, workshops, tours, phone-ins. A key reminder is to make sure the relevant information is available (in appropriate form) to those being consulted and that the means for soliciting the information is appropriate for the respondents. The Law Commission's consultation strategy for the Women's Access to Justice project was exemplary in this regard.

Writers such as Hil and Roughley warn that the agency should not draw up a "list" of people and organisation to consult. Attractive as this may seem the principle of open response precludes it. Moreover such a measure can appear to be an effort to meet the letter and not the spirit of consultation mandates. Officials who use such a strategy may be accused of tokenism (Midgely, 1986, cited in Hil and Roughley, 1997). Nevertheless, good consultation procedures will include an initial indicative list of desired respondents (or respondent types) and early feedback will be checked against the list, so that the framework may be adapted if needed.

Bond and Camino (1999) suggest one way to institutionalise a consultation mechanism. As part of the reform of New South Wales water allocation policy, and to provide for community involvement, community based management committees are to be set up. Each committee will have broad representation among stakeholders and will be independently chaired. There is a impressive list of functions and expectations of these committees. As a consultation mechanism, these groups can serve as a standing committee, with responsibility to provide specific information and feedback to government at regular, specified intervals. Such a model contrasts with a model in which consultation is one-off and/or ad hoc and focused on a prepared discussion document. The NSW model also contrast with open processes relatively early in the policy development phase, where broad responses on issues, values, and so on are solicited.

A similar strategy is noted in VanNijnatten and Gregoire (1995, p.213): Canadian corrections officials worked with a "prominent community group" to develop a detailed consultation strategy. (Curiously, however, the strategy was explicitly geared to gain acceptance for a community-based correction facility in the locality.)

Attention is needed to the timing of consultations throughout the policy process, and to the most effective focus of consultation at the various stages. Game, (1997, p.71) provides an example in which information solicited in an early in the process focused on community identify, but failed to explore with respondents various possible structural options which might be associated with community identity (and which formed the true purpose of the policy review). Thus, a second phase of consultative research was needed.

This section has assumed a focus on consultation with the public. It must be kept in mind, however that "not the least element of consultation is managing the demands of those within government who think they have a right to know and to be accommodated" (Anderson, 1997 p.476). Interdepartmental consultation is, for some areas of policy, a major element of policy development, but this opens a question of respective responsibility for policy development. Presumably, participants in such discussions are acting in accordance with their respective minister's requirements. According to Bailey (1998), policy debates are raised to a ministerial level when there is no "single powerful group" with the capacity to "gatekeep". Blakeley commented at the 1993 AIC Conference on Efficient and Effective Policy that officials should work constructively to resolve differences as far as possible, but when differences remain, to present options clearly and concisely; even where agreement is reached, the trade-offs implicit in that option should be drawn to minister's attention. By way of contrast, Beryl Radin suggests that much effective policy debate may occur at sub-levels among analysts who identify with certain policy areas or approaches (1997, p.206).

It would be useful to have a better gauge on the extent of interdepartmental consultations; the relative importance of these consultations vis-à-vis consultations with the public and other forms of information gathering and idea testing; and to begin to draw from the experiences of involved officials some general lessons.

Leadership and Management

The final point of this section is a straightforward one, clearly presented by Anderson (1997, p.472): delivering good policy work in government is itself a major managerial task and leadership at the most senior levels of the public service is critical to strengthening policy capacity. Management and leadership encompass functions that both make an organisation and task run well and contribute to substantively high quality advice. Anderson goes on to elaborate seven functions of policy management (p.473):

• theoretical research (even if done in universities);
• analysts need to keep abreast and be able to interpret research for decision-makers);
• applied research; statistics and qualitative modelling;
• environmental scanning;
• trends analysis and forecasting;
• policy analysis; and
• advice.

This last is the core policy function. It is often messy, drawing together all the other elements--research, quantitative and qualitative analysis, program evaluations, consultations, instincts about what is possible and saleable, the wishes of the minister--into options and advice (Anderson, 1997, p.474). Leadership is needed, then not only in the sense of commanding expertise in traditional analysis but in managing consultations, relationships communications among all the contributors to the policy analysis.

Thus, in addition to managerial leadership (human resource and project management, for instance), there is need for policy leadership. Leadership is required in programme design, implementation, monitoring and evaluation. Monitoring and evaluation, for instance may need to be better integrated with policy analysis and advice functions, not cut off as part of audit or corporate services (Anderson, p.478). Part of leadership is knowledge based: a policy leader must be competent in area in which advice is proffered. Another part is person based: leaders have certain qualities of mind, such as creativity, which mark them out as leaders. A third part is functional: leaders are skilled in 'drawing together', at integrating, excelling in complex situations, and so on.

In Anderson's terminology, the ideal leader is a "policy generalist" who synthesizes, interprets, and steers the work of a policy team. Such people need "breadth and curiosity, an ability to deal with uncertainty and a strong sense of what is relevant to decision making" in addition to strong oral and written communication skills and other "normal" competencies. Outstanding generalists can come from any academic background, but are typically not narrow technical specialists. They should also have a depth of knowledge in one or more areas of policy, perhaps learned on the job, and understand the key analytic tools required in their area. Policy generalists are, not surprisingly, scarce entities.

Other references to policy leadership emphasise the need to create "a common language, a community of discourse among those who have different perspectives and interests and who use different specialist languages" (OECD, 1994b, p.25). Leadership (in a full sense of professional leadership) is explored in the work of Schall (1995), who draws on Drath and Palus (1994, p.23) to set out the distinct characteristics needed for "learning from experience":

• the capacity to understand oneself as an individual and as a socially embedded being;
• the capacity to understand systems in general and as mutually related and interacting and continually changing;
• the capacity to take the perspective of another; and
• the capacity to engage in dialogue.

Concrete experience is mediated through observation and reflection (Schall, 1995, p.204). While the more psychological aspects of this list may not appeal to all, the systems and interaction aspects are likely to resonate with senior advisors' experiences of what works and does not in presenting advice to ministers. The profession as a whole requires people with these capacities to serve as mentors for promising successors.

Part 5: Processes for Quality: Toward Better Output

As Hawke (1993, p.4) notes, policy advice encompasses "professional" and "intellectual" work about the role of collective action, including all the processes of policy analysis and policy development, distinguished from political debate. This section examines the professional and intellectual features of processes for high quality policy advice. To do so, the context and choice issues, as set out in the preceding sections, are held constant. That is, for a given advisory requirement, what helps to distinguish better advice from poorer?

Three issues are examined separately in this section, although in reality they are related. First, for a given problem, there is no one-size-fits-all analytical output. Rather, advice needs to be tailored according to its intended purpose. Second, this advice, once tailored needs to be presented appropriately. Finally, the focus turns to the political context of advice.

Producing High Quality Policy Advice

To produce high quality policy advice, the policy analyst must practise a bit of advisory alchemy. Advisory alchemy involves a careful selection and mixing of the products of analysis, in order, one hopes, to transform those base elements into bright gold. Bright gold advice is extremely rare. Analysts far more frequently are aware of the flaws in their gold: its quality gap.

Advice, as noted by Corban (1994, pp.5-6), comes in different flavours: advice about what set of outputs is likely to contribute to desired outcomes; advice to define and clarify outcomes; and advice to assist government to determine priorities between conflicting objectives. Meltsner contributes two additional types of advice: political advice, not unlike Corban's third type, but more sharply centred on political success; and advice to help decision-makers understand the dimensions of a problem and to map actors in a problem situation and its solution (Meltsner, 1982, pp.64-66). Advisors are called upon to help decision-makers with purchase agreements. Finally, advisors may be called upon to provide more detailed analysis on implementation, monitoring and evaluation.

Improved policy advice starts from an accurate identification of factors in need of improvement. Cohen, Eimicke and Ukeles (1995, p.606) present a list in the form of features of the advice situation that can serve to help identify priorities for efforts to improve quality. Does quality suffer because the policy professional must make simplifying assumptions, deal with a dynamic, ever-changing political environment, translate policy design to real-world behaviours, analyse and manage problems in a fishbowl, or experiment with untried solutions to unprecedented as well as routine problems? A list such as this, properly modified to the particular political circumstances, should be able to help analysts and their managers to learn better from their relative successes and failures, and thus to improve advice in the future. Obviously, such an approach can be at best indicative, but analysts may have been slow to pick up on the more reflective practices of other professions and to draw lessons from their experiences as systematically as possible.

It is also instructive to consider what may not lead to a notable quality gap. Brunner (1997a, pp.196-7), for instance, is of the view that when analysts get the analysis wrong it is not due to a lack of satisfactory concepts or theory (which would mean there is a need for more basic research) or due to a lack of adequate funding, access to information, or deference from the public or public officials (which would mean there is a need for more self-promotion). In the view of some, analysts in New Zealand could be better served by basic research (Hawke, 1995). The final set of variables - once consideration is given to the adequacy of theory, findings, access to information, and the vicissitudes of analysing a complex problem in a real world - stem from the analyst's relationship with the minister. These variables concern the clarity of task-setting and quality expectations, the adequacy of feedback from the minister to the analyst, the minister's appreciation of the analytic challenges involved, and so on.

In March 1997, when Minister of State Services, Jenny Shipley set policy analysts a challenge to prepare defined outcomes and solutions for government to consider (see James, 1997). Her diagnosis of the quality gap appears to centre on a certain types of advice - helping ministers to define and clarify outcomes together with helping ministers to assess the possible means (solutions) to attain those outcomes. Further elaboration of this core quality gap is suggested by several of the "tools of coherence" set out in OECD (1996b, p.10). These tools, when translated to the perspective of the analyst suggest first, that decision-makers need advice based on a clear definition and good analysis of issues, with explicit indications of possible inconsistencies. Second, analysts need mechanisms to anticipate, detect and resolve policy conflicts early in the process and to help identify inconsistencies and reduce incoherence. Third, analysts need to understand implementation procedures and monitoring mechanisms that may be established to help ensure that policies can be adjusted in the light of progress, new information, and changing circumstances. Each of these points call on the analyst to think beyond the immediate issue, and to be prepared to present analysis of inconsistencies, conflict and the like as part of option analysis.

While it is true that policy professionals learn their craft by doing, the recent emphasis on problem definition may help professionals improve their advice before it is done. Brunner (1997a, p.196-7) argues that "[m]ost preventable errors of policy analysis stem from the analyst's perspective: Typically, as an analyst simplifies a policy problem, he or she misconstrues some important part of the context or overlooks it altogether. . . . analysis is not sufficiently contextual".

An OECD (1996a, p.16) report recommends advisors draw on varied sources of info, but that this information needs to be filtered, interpreted, and prioritised as a basis for action. This suggestion, together with Brunner's diagnosis, may mean that analysts need to develop better ways of dealing with external influences and alternative perspectives. For instance, some analyses may be improved with a better understanding of the media in policy, or the impact of feminist theory on problem definition (see Parsons, 1995, pp.103-109). The particular role of the population ministries could be explored in this light. Similarly, analysts who work on issues that are frequently under a spotlight could benefit from some focused meta-analysis, that is, analysis of earlier analyses conducted in the spotlight. Some literature pertinent to such an analysis exists in New Zealand, but general lessons remain to be extracted.

Presentation of Policy Advice

Analysis derives form the Greek ana-lusis, which means a thorough splitting apart, or dissolving. This etymology is easily overlooked in the context of presenting policy advice as the output of policy analysis: no minister wants a thoroughly dissolved product. Analysis needs to be followed by synthesis and interpretation.

Most technically trained analysts understand what is required of synthesis: indeed, analysts would be as likely to describe their work as involving synthesis as analysis. The requirement to interpret is less clear. MacRae and Whittington (1997, p.5) note the obvious: advice relates to a particular person, time and context. Interpretation is the process of taking analytic conclusions and re-contextualising them, putting the answers into the language of the listener.

Some have suggested that the presentation of policy advice could benefit if professional communicators had a role to play in the development of policies. Anderson, (1997, p.476), writes:

communicators have a role to play in advising on policy, but they also benefit from being integrated into the development of polices as they hone their messages. While historically the tendency was to underplay the role of communicators, some policy managers increasingly worry about the 'dumbing down' of policy, where attractive policy options lose out to the need to package options in sound bites or to follow opinion polls too slavishly.

Clearly the Canadian context differs from New Zealand's.

Weimer and Vining (1992, pp.236-244) present some accessible advice. They recommend a focus on answering three questions in a report:

• What should the client do?
• Why should the client do it? and
• How should the client do it?

Thus, the advice will contain recommendations that follow from analysis of alternatives; summaries of the advantages and disadvantages of the policy recommended; and a clear set of instructions for action. In the New Zealand context, it is also important to include advice on monitoring the implementation of the policy and when and how to evaluate the impacts of the policy. Multi-framing and perceptual mapping techniques, in addition to their substantive uses, can help the analyst to avoid antagonisms in presentations (Heineman, et al, 1997, p.204).

Dunn (1994) includes appendices on the presentation of policy advice in the form of a policy issue paper, an executive summary, a policy memorandum and an oral briefing. While the material is directed to a fairly junior analyst in the American context, there are some useful summaries of "elements" and checklists, linked to in-depth discussion in the text. For example, following an executive summary, the elements of an issue paper are background (description of situation and outcomes of prior efforts to solve problem); scope and severity of problem (assessment of past policy performance, significance of the problem situation; need for analysis); problem statement (definition of problem, major stakeholders, goals and objectives, measures of effectiveness, possible solutions); policy alternatives (description of alternatives, comparison of future consequences, spillovers and externalities, constraints and political feasibility); and policy recommendations (criteria for recommendation, description of preferred alternative, outline of implementation strategy, provisions for monitoring and evaluation, and limitations and unanticipated consequences) (p.426). A checklist, with illustrations, follows. For example, point 19 (p.429) asks "Are alternative solutions specified?"

The illustration is: "three policy alternatives are analysed in the course of this paper: educational programs to alert citizens to the role they can play in crime control; policy training programs; and advanced crime control technology".

Knowledge of Political Context

Typically, when writers refer to the "art" or the "craft" of policy analysis, they refer in part to the art of matching analytic approaches to questions in order to produce the best quality advice. But, in large measure, the art or craft of policy analysis refers to the ability to make choices of analytic design, and to present all aspects of the advisory process and outcome to the decision-maker. Analysis takes place in a political context. Stone (1997) argues that policy analysis is political argument. It follows that, even where allowance is made for some aspects of analysis to be apolitical, that the analyst requires a good knowledge of the political context.

Unfortunately, this is one of the areas in which practitioners have not recorded their reflections to make lessons available to all analysts. On the evidence of what is taught in American graduate policy programmes, some features of "political analysis" are part of nearly all students' study (APPAM, 1997). The core of political analysis is a study of actors, institutions and processes in decision making. For practitioners who toil daily in this context, explicit study might appear tedious and unnecessary. Yet, practitioners may also be aware that decision-makers want "something more" from them, vaguely articulated as greater sensitivity and awareness of the political context, and the ability to turn this knowledge into sound advice. Analysts perceive this demand as a call for them to be more intuitive in political matters.

Three characteristics (Pollitt and Summa, 1997) of the Westminster system in New Zealand may explain why advisors are reluctant to trade in political analysis:

• the leaders of the party (or coalition) in power can implement radical policies;
• central government dominates local government constitutionally; and
• there is no tradition of powerful autonomous agencies within the state sector.

To break out of the reluctance, analysts need explicitly to take on board what the best of the profession already know: a good understanding of the politics of a policy proposal contextualises an analysis. Such knowledge does not serve as a filter, to present information only selectively to the decision-maker, but serves to provide nuance and texture to what would otherwise appear acontextual.

Political awareness covers several areas. Obviously, opposition views are critical to understand. Views of elected officials at the local level may be important. It is also necessary to know the political history of a proposal. New Zealand policy is arguably less incremental than that of other countries, and the history may be more disjoint. There are, however, few truly new ideas under the sun. If a proposal is a new initiative with no "direct" history, it may be important to look for close analogies and to study that history.

Part 6: Summary of Key Themes

This section summarises some key ideas that might loosely be considered to bear on the 'training needs' of new and less-experienced policy analysts.

There is no reason to question the quality principles elaborated in The Policy Advice Initiative (SSC, 1992). However, the literature suggests that these principles may need to be supplemented in three respects. First, in a climate of significant pressures to do more with less, there is a need to identify quality priorities in order that a suitable package of quality enhancement can be instituted. Second, there is overwhelming evidence that the education, training and development of analysts is crucial. Policy work is intellectual work, involving skill, information, and knowledge (Agor, 1997). Hence, Agor recommends that intellectual capital be identified, that it be managed for enhancement, and that it be managed to enhance performance. This learning capability will not develop solely through the use of quality checks, the use of performance agreements, vision statements, and the like, because achieving the objectives of those instruments is dependent upon the qualities of the people who use them. Third, advice must be broad-based. These three observations together call for far more explicit attention to the development of high quality advisory capacity in a policy advice unit as a whole.

The expanded need to consult (both within and outside government) and greater call for collaboration, participation and collegiality, mean that the operating culture of policy analysts looks far different from the clean, spare traditional culture, in which a rational technician assesses systematically assembled data to arrive at a recommendation. Analysts can expect to interact extensively, and will less frequently 'own' an entire piece of analysis.

The current policy advice output is typically topic-based, but can be about a particular program or location. The inputs may include researching, monitoring, analysing options, writing, discussing and negotiating or issuing instructions about a policy issue (unknown reference on the nature of the work of a policy adviser). The output may take the form of oral advice, cabinet paper, draft speech or briefing notes. From an intra-departmental point of view, at least, the literature suggests that inputs should increasingly come to include value and context analyses. Moreover, it may also be timely to consider identifying dimensions of outputs in purchase agreements that are not tied to a specific piece of topical advice.

Even those who may not be tuned to the newest epistemological jargon understand a point made clearly by Hawke (1993, p.27): choosing a methodology has to be guided by the particular problem under consideration. This review has emphasised the importance of problem structuring. Given an appropriate problem, the analyst with "appropriate thinking skills, an appropriate tertiary eduction, and a desire to perform to a high level when providing public policy advice" (SSC, 1992, p.32), will be able to choose the methodology. But this is only the starting point. An architect with appropriate training, etc, will know how to design a building that will stand up. A doctor will undertake not make anyone worse off.

The remainder of this section summarises some of the skills and capacities identified in the literature which help define the policy professional's role and output quality, and which may carry that person beyond the starting point of appropriate education and will to succeed.

Problem Definition

Problem definition is key. It frames and generates virtually everything that follows in the policy process (deLeon, 1994, p.89). A well-defined problem is more likely to be appropriately analysed and the chosen solution from among the options analysed is more likely to address the problem. Many methods and tools are available to assist the analyst at this stage, but the most important is simple: good problem definition takes time, and may appear a frivolous waste of time while it is underway, but the time will, on average, more than be recovered at subsequent stages.

Paradigm Shift

Recent theorists urge consideration of a range of possible analytic approaches, and signal that the field may be in the process of a paradigm shift. A paradigm shift is not imminent. However, thinking in the alternative contexts may help the analysts to prepare a more complete and appropriate analysis. Further, alternative paradigms may be important constructs for other stakeholders in the policy process, and analysts may benefit from a critical understanding of 'where others are coming from'.

Participation

Policy analysts do not toil alone. At many stages, the work involves others, for instance, to provide information and opinion and to react to draft analyses. Many analytic projects are large and complex, and many hands will take part.

Political

The policy analyst is both in and of a particular kind of political world (Heineman et al, 1997, p.1). Some analysts may wish their jobs were not so 'political'; others thrive in the political environment. The political world may appear differently to analysts of both persuasions depending on the agency in which they work, as different agencies will have a different analytic orientation to the political world.

Client-responsible

The analyst serves, and is responsible to the decision-making client, in a role characterised when successful, by trust and openness. Most analysts appreciate that the "free and frank" convention means that they are responsible to "provide honest, impartial, and comprehensive advice to Ministers, and to alert Ministers to the possible consequences of following particular policies, whether or not such advice accords with the Minister's views" (Public Service Code of Conduct). However, it is also necessary to recognise that the interaction of analyst and minister is dynamic: views evolve, options initially attractive reveal serious flaws on further analysis, or as the political environment changes.

Consultation

Inter-departmental and citizen/stakeholder consultations provide input to both the process and the output of analysis. Consultation, ideally, serves a range of purposes, ranging from the purely expedient to one at the heart of the legitimacy of democratic governance.

Collegiality

Collegial community with other analysts is desired, both across agency lines and across "generations" so that new analysts can "learn the ropes" and become familiar with "prevailing wisdom" (see Considine, 1997, p.20).

Communication

Communication, in consultation but also broadly and generally, is needed to ensure that analysts have a steady store and fresh supply of information, impressions and judgements, which may be tried out on critics, key players and other experienced analysts (Considine, 1997, p.20) and thus help ensure the political context of policy analysis in taken into account.

Coherence, Cross-cutting Issues

Analysts able to produce high quality policy advice will have skills in providing advice on cross-cutting issues, in order to help co-ordinate government policy initiatives, and assist in policy coherence, including the capacity to analyse incoherence so decisions are better informed (OECD, 1996b, pp.9-10). Good policy is less about avoiding contradiction than on managing it (OECD, 1996 policy coherence). Producing quality advice for such management is not risk-free, and calls for advice, which will support the decision-maker in management negotiation.

Consistency, Completeness, Critical Orientation

Quality advice is consistent, complete, and critically oriented. These three key words reach beyond the qualities of a given piece of advice--which also must be timely, evidence adequate consultation and so on as set out in The Policy Advice Initiative (SSC, 1992) - to features of the set of advice and the flow of advice from a policy unit in government. Policy work as a whole is consistent and complete if it successfully builds a strong basis for the achievement of government objectives. Critical orientation is a key competency of analysts and policy units that strive to produce consistent and complete advice.

Continuity

Policy develops iteratively, and thus one dimension of quality policy advice is its continuity, which allows each proposed development to be viewed by the decision-maker in context. This quality is the temporal analog to the extensive thrust of the "consistency, completeness and critical orientation" point above.

Continuity also has a micro application: quality policy advice is a product of the evolution of initiatives, as successive drafts of a piece of advice evolve through joint interactions among policy professionals and the decision maker that culminates in a recommended way to reach an identified outcome.

Creativity

Creativity is an indicative key word, meant to capture the more intuitive, "art" side of the analyst's craft. Analysts can aspire for innovation in context; thinking outside the frame, supported by an accumulated stock of professional knowledge and skill in "double-vision", though which they keep open to a multiplicity of views of a situation (not necessarily to change an immediate action, but to be able to revise thinking) (Schön, 1987; Schön and Rein, 1994).

Quality policy advice is not a product of high technology, issuing forth from a well-oiled machine operated by skilled workers. Neither is it a product of political or rhetorical brilliance. Rather, quality policy advice is painstakingly built by professionals practising a craft for which they have some explicit training and opportunities to reflect and learn from experience. Every analyst should be able to look forward to the remainder of their career as an opportunity to produce advice of increasingly higher calibre. The comparative advantages of every analyst in a policy unit or working on a policy project, for example in trend analysis or in communication, should be acknowledged and supported.

Quality policy advice emerges from a system of professionals, engaged over time by a commitment to excel in their craft.

Part 7: References

The first section is an annotated list of recent policy texts. Following this are references to materials referred to in this review. Not all of these sources will have been explicitly discussed.

The Last Decade of Texts

Policy texts are often about understanding policy, rather that about doing policy work. Nevertheless, a practitioner can profit from selective use of policy texts. In recent years a number of new, or substantially revised texts have appeared. The comments below are directed to practising analysts who want to improve the quality of their output.

Bardach, Eugene. (1996) The Eight-Step Path of Policy Analysis: A Handbook for Practice, Berkeley Academic Press, Berkeley, California.

A slim guide (60 pages) that provides suggestions to help analysts work through a "prototypical" analysis problem. Useful appendix on types of government interventions.

Bridgman, Peter and Davis, Glyn. (1998) Australian Policy Handbook, Allan and Unwin, St Leonards, NSW.

Designed as a practical guide, this short book is organised around stages in a policy cycle. Numerous side-bars aid identification of key points. The tone is factual (even where the topic is usually subject to more debate) and Australian-focused. A good starter text.

Considine, Mark. (1997) Public Policy: A Critical Approach, Macmillan, South Melbourne.

This text looks at the way public policies are made and unmade (p.1). Considine aims at more than informing his reader, however. He outlines "four simple guidelines" of a "critical approach", which if successfully applied will lead to "compelling" explanations that are "part science and part interpretive craft" (p.18). These guidelines are: the activities or phenomenon must be clearly defined; the means for gathering evidence should be explained; the criteria for judgement ought to be transparent and the specific episodes and methods should embrace larger theories

Dunn, William N. (1994) Public Policy Analysis: An Introduction, 2nd edition, Prentice Hall, Engelwood Cliffs, New Jersey.

Dunn features a number of management science techniques and an emphasis on "problem structuring" but does not serve as a "how-to-manual". One of the few sources that has extensive coverage of problem structuring methods. Additional emphasis on policy advice communication and working in political context. Contains an extended section on policy argumentation and the analysis of policy arguments. A number of economic and statistical techniques are illustrated as is the use of computing in analysis.

Heineman, Robert A., et al (1997) The World of the Policy Analyst, 2nd ed, Chatham House Publishers, Chatham, New Jersey.

Heineman, et al (1997) is an original contribution. While it has an explicitly American focus (in contrast to many American authors' implicit American focus), it also has a strong emphasis on the ethical dimensions of policy. The authors' intention is to assist "the analyst to become more sensitive to the salient factors that influence the way he or she conceives and executes task" (p.1). There are chapters on rationality and decision making; cultural setting of policy analysis; ethics and policy analysis; democracy and the fragmentation of consensus; policy analysis and the political arena; policy devolution and policy analysis; and policy analysis and the judicial process.

Hill, Michael. (1997b) The Policy Process in the Modern State, 3rd ed, Prentice-Hall/Harvester Wheatsheaf, Hemel Hempstead.

This book focuses on the stages of the policy process and how to understand the influences affecting the process. Like the study of policy content or outputs, a study of process develops knowledge of policy, not knowledge in policy

Howlett, Michael and Ramesh, M. (1995) Studying Public Policy, Oxford University Press, Oxford.

A good introductory text, which takes a deeper look at the policy cycle, both by focusing on actors, institutions and instruments and through in-depth consideration of the "sub-stages" in agenda-setting, policy formation, decision making, implementation and evaluation. Compact and extensively referenced.

MacRae Jr, Duncan and Whittington, Dale. (1997) Expert Advice for Policy Choice: Analysis and Discourse, Georgetown University Press, Washington, DC.

These authors propose that analysts should downplay a reliance on economic methods, "drawing more heavily on other social sciences and linking advice more closely to public discourse" (p.1). The method they suggest is "to structure policy choice around matrices in which alternatives are assessed in terms of ethical criteria, affected parties, or periods of time" (p.1). These matrices provide a vehicle for exploring trade-offs in policy choices - which clearly is of interest to the decision-maker. For example, the matrices can be used to assess systematically the levels of support and opposition for a policy option held by different stakeholder groups. Focus is on expertise based on specific analytic tasks, and "omits more intuitive skills such as that of "crafting problems" (p.12). Matrices are explored for assessing criteria (ethical considerations concerning what is good for individuals in general or for society or what is morally right); affected parties (usually groups that may be affected differently by a policy) and time periods (from present to remote future).

Parsons, Wayne. (1995) Public Policy: An Introduction to the Theory and Practice of Policy Analysis, Edward Elgar, Aldershot.

This text achieves its importance by bringing to life the shaping influences on public policy theory and practice form the fields of political science, public administration, political theory, sociology, psychology, economics and management. There are over 1400 references in the bibliography. In the text, these are often organised into "key texts" on a given topic, or featured in boxes to illustrate topics. The book is not a "how-to" but will repay the reflective practitioner. Explicitly eschewing a "stagest" organisation of the policy process, Parsons divides the book into four sections, and uses extensive cross referencing, so that the reader can go between sections, following links (the index, unfortunately, is inadequate for such an approach). The sections are meta-analysis (the analysis of analysis), meso-analysis (the analysis of problem definition, agenda-setting and the formation of policy), decision-analysis (analysis of the decision-making process and policy analysis for decision making), and delivery analysis (the analysis of implementation, evaluation, change and impact.

Patton, Carl V. and Sawicki, David S. (1993) Basic Methods if Policy Analysis and Planning, 2nd ed, Prentice-Hall, Englewood Cliffs, New Jersey.

This book presents, in the words of one reviewer "a true smorgasbord of methods under each of the usual steps in conducting a policy analysis. . . [but] an absence of any focus on analytical core concepts. . ." This suggests a potential usefulness to analysts who need to freshen their approaches and who wish to design more finely-tuned analyses.

Weimer, David L. and Vining, Aidan R. (1999) Policy Analysis: Concepts and Practice 3rd ed, Prentice-Hall, Engelwood Cliffs, New Jersey.

Weimer and Vining is the closest to a 'practice hand-book' among the texts reviewed. It is clearly targeted to the graduate student or policy analyst without extensive experience (but who might be working at any level of government). It opens with a very good example of a policy analysis of the Canadian Pacific salmon fishery. It contains a thorough review of market and government failures and reviews generic policy alternatives for "freeing, facilitating, and simulating markets"; using taxes and subsidies to alter incentives; establishing rules; supplying goods through non-market mechanisms; and providing insurance and cushions.

References

Note: Most of the references included below are cited in the review. A few references are included which have not been directly reviewed by the review author, but are judged to be worth including. In some cases, the sources were identified in other materials, and in some cases by contacts of the author.

Agor, Weston H. (1997) "The Measurement, Use and Development of Intellectual Capacity to Increase Public Sector Productivity" in Public Personnel Management, Vol 26, No 2, pp175-186.

Anderson, G. (1997) "The New Focus on the Policy Capacity of the Federal Government" Canadian Public Administration, Vol 39, No 4, pp 469-488.

Arnstein, Sherry R. (1983) "A ladder of citizen participation" American Institute of Planners Journal, July.

Association of Policy Analysis and Management (APPAM) (1997) Guide to Graduate Public Policy Education and Organization, APPAM, Washington DC.

Bailey, Wilson (1998) "Splitting policy development and operations: the update on the law reform model from an operational perspective" Address to AIC Public Policy Conference, 24 March, Wellington.

Balfour, D.L. and W. Mesaros (1994) "Connecting the local narratives: Public administration as a hermeneutic science" Public Administration Review, Vol 54, pp 559-564.

Bardach, Eugene (1996) The Eight-Step Path of Policy Analysis: A Handbook for Practice, Berkeley Academic Press, Berkeley, California.

Belgrave, John (1998) "Policy making in today's environment: Aligning policy to government strategy" Address to AIC Public Policy Conference, 24 March, Wellington.

Blakeley, Roger (1996) "Development of policy aligned to government strategy" Address to AIC Conference on "Strategic Policy Development in an MMP Environment", 17-18 April, Wellington.

Blakeley, Roger (1993) "Managing for quality advice performance" Address to AIC Conference on Efficient and Effective Policy, 29-30 June, Wellington.

Bobrow, Davis and John Dryzek (1987) Policy Analysis by Design, University of Pittsburgh Press, Pittsburgh.

Bond, Margaret and Maria Comino (1999) "Environmental justice and the water market in Australia" in Klaus Bosselmann and Benjamin Richardson, eds. Environmental Justice and Market Mechanisms Dordrect, Kluwer Law International, pp 232-248.

Boston, J, J. Martin, J. Pallot, and P. Walsh (1996) Public Management: The New Zealand Model, Oxford University Press, Auckland.

Brown, Steven R (1980) Political Subjectivity: Applications of Q Methodology in Political Science, Yale University Press, New Haven.

Bridgman, Peter and Glyn Davis (1998) Australian Policy Handbook St Leonards, NSW, Allen and Unwin.

Brunner, Ronald D. (1997) "Introduction to the Policy Sciences" Policy Sciences, Vol 30, No 4, pp 191-215.

Brunner, Ronald D. (1991) "The policy movement as a policy problem" Policy Sciences, Vol 24, pp 65-98.

Campen, James T. (1986) Benefit, Cost, and Beyond: The Political Economy of Benefit-Cost Analysis, Ballinger, Cambridge, Massachusetts.

Canada Privy Council Office (1997) Fourth annual report to the Prime Minister on the Public Service of Canada Canada;

Chapman, R.. (1995) "Improving policy advice processes: Strategic outcome definition, policy evaluation and a competitive policy market" Public Sector, Vol 18, No 2, pp 16-21.

Checkland, Peter (1989) "Soft systems methodology" in Jonathan Rosenhead (ed) Rational Analysis for a Problematic World, John Wiley and Sons, New York.

Clemen, Robert T. (1996) Making Hard Decisions: An Introduction to Decision Analysis, 2nd edition, Duxbury Press, Belmont, Massachusetts.

Cohen, Steven and William Eimecke (1995) The New Effective Public Manager: Achieving Success in a Changing Environment, Jossey-Bass, San Francisco.

Cohen, Steven, William Eimecke and Jacob Ukeles (1995) "Teaching the craft of policy and management analysis: The workshop sequence at Columbia University's graduate program in public policy and administration", Journal of Policy Analysis and Management, Vol 14, pp 606-626.

Colebatch, H.K. (1998) Policy, Allen and Unwin, St Leonards, NSW.

Considine, Mark (1997) Public Policy: A critical Approach, Macmillan, South Melbourne.

Cook, (1997) "Lessons learned in evaluation over the past 25 years", in Eleanor Chelimsky and William R. Shadish, Evaluation for the 21st Century, Sage, Thousand Oaks, California.

Corban, Jeremy (1994) The Provision of Strategic Advice to Government, Thesis submitted for the Master of Public Policy degree, Victoria University, Wellington.

Craswell, Emma and Glyn Davis (1994) "The search for policy coordination: Ministerial and bureaucratic perceptions of agency amalgamations in a federal parliamentary system" Policy Studies Journal, Vol 22, No 1, pp 59-74.

deLeon, Peter (1998) "Models of policy discourse: Insights vs prediction" Policy Sciences, (based on a paper presented at the 1996 Annual Meeting of the American Political Science Association, San Francisco, California, 29 August-1 September).

deLeon, Peter (1997) Democracy and the Policy Sciences, State University of New York, Albany.

deLeon, Peter (1994) "Reinventing the policy sciences: Three steps back to the future" in Policy Sciences, No 27, pp 77-95.

deLeon, Peter (1988) Advice and Consent: The Development of the Policy Sciences, Russell Sage Foundation, New York.

Drath, Wilfred H. and Charles J. Palus (1994) Making Common Sense: Leadership as Meaning-Making in a Community of Practice, Center for Creative Leadership, Greensboro, North Carolina.

Dror, Yehezkel (1988) Notes Towards A Philosophy of Policy Reasoning in M Campanella, (ed) Between Rationality and Cognition: Policy-Making Under Conditions of Uncertainty, Complexity and Turbulence, Albert Meynier, Torino.

Dunn, William N. (1994) Public Policy Analysis: An Introduction, 2nd edition, Prentice Hall, Engelwood Cliffs, New Jersey.

Durning, Dan (1999) "The transition from traditional to postpositivist policy analysis: A role for Q-methodology" Journal of Policy Analysis and Management Vol 18, no 3, pp 389-410.

Durning, Dan (1996) "The transition from traditional to postpositivist policy analysis: A role for Q methodology", Paper presented at the Association of Public Policy Analysis and Management Annual Research Conference, Pittsburgh, Pennsylvania.

Durning, Dan and Will Osuna (1994) "Policy analysts' roles and value orientations: An empirical investigation using Q-methodology" Journal of Policy Analysis and Management, Vol 13, No 4, pp 629-657.

Fiorino, Daniel J. (1995) "Regulatory negotiation as a form of public participation", in O Renn, T Webler, and P Wiedemann (eds) Fairness and Competence in Citizen Participation: Evaluating Models for Environmental Discourse, Kluwer Academic Publishers, Dordrecht.

Fiorino, Daniel J. (1990) "Dimensions of negotiated rule-making: Practical constraints and theoretical implications", in Miriam K Mills (ed), Conflict Resolution and Public Policy, Greenwood Press, New York, pp 141-154.

Fiorino, Daniel J. (1988) "Regulatory negotiation as a policy process", Public Administration Review, Vol 48, No 4, pp 764-772.

Fischer, Frank (1995) Evaluating Public Policy, Nelson-Hall Publishers, Chicago.

Fischer, Frank and John Forester (eds) (1993) The Argumentative Turn in Policy Analysis and Planning, Duke University Press, Durham, North Carolina.

Flyvbjerb, Bent (1998) Rationality and Power: Democracy in Practice, University of Chicago Press, Chicago.

Forester, John (1993) Critical Theory, Public Policy and Planning Practice, State University of New York Press, Albany, New York.

Funnell, S. (1997) "Program logic: An adaptable tool for designing and evaluating programs" Evaluation News and Comment, Vol 6, No 1, pp 5-17.

Game, Chris (1997) "Unprecedented in local government terms: The Local Government Commission's public consultation programme" Public Administration, Vol 75, pp 67-96.

Hammond, K.R. (1996) Human Judgment and Social Policy: Irreducible Uncertainty, Inevitable Error, Unavoidable Injustice, Oxford University Press, New York.

Hanf, K. (1993) "Enforcing environmental laws: The social regulation of co-production", in M Hill (ed) New Agendas in the Study of the Policy Process, Harvester Wheatsheaf, Hemel Hempstead.

Harmon, M. (1989) "Decision and action as contrasting perspectives in organizational theory", Public Administration Review, Vol 49, pp 144-150.

Hawke, Gary (1993) Improving Policy Advice, Institute of Policy Advice, Wellington.

Hawke, Gary (1995) Policy & Social Research, Change Seminars, Wellington.

Hayward, Bronwyn (1997) "Public participation", in Raymond Miller (ed) New Zealand Politics in Transition, Oxford University Press, Auckland, pp 408-417.

Heineman, Robert A., William T. Bluhm, Steven A. Peterson, and Edward N. Kearney (1997) The World of the Policy Analyst, 2nd ed, Chatham House Publishers, Chatham, New Jersey.

Hil, Richard and Leanne Roughley (1997) _Public consultation and juvenile justice reform: A Queensland case study", Australian Journal of Social Issues, Vol 32, No 1, pp 21-36.

Hill, Michael (1997a) "Implementation theory: Yesterday's issue?" Policy and Politics, Vol 25, No 4, pp 375-385.

Hill, Michael (1997b) The Policy Process in the Modern State, 3rd ed, Prentice-Hall/Harvester Wheatsheaf, Hemel Hempstead.

Hinton, S. and Dickie, B. (1996) Policy Implementation and Evaluation: A Review, Waikato Regional Council, Hamilton.

Hoppe, Robert (1996) Book review, Policy Sciences, Vol 29, pp 69-72.

Howlett, Michael & Ramesh, M (1995) Studying Public Policy, Oxford University Press, Oxford.

Hunn, D.K. (1994) "Measuring performance in policy advice: A New Zealand perspective" in OECD, Performance Measures and Target Setting, Paris.

Irwin, T (1996) "The Psychology of Judgment and the Practice of Policy Advice" in Public Sector, Vol 19, No 3, pp 20-22.

James, Colin (1997) "Serving Shipley" in Management, June 1997, p 101.

Jameson, Brian (1991) "Total quality management and the policy advisor" in Public Sector, Vol 14, No 2, pp 11-13.

Jennings, Bruce (1987) "Interpretation and the practice of policy analysis", in F Fischer and J Forester (eds) Confronting Values in Policy Analysis, Sage, Newbury Park, California.

Jennings, Bruce (1993) "Counsel and consensus: Norms of argument in health policy", in Fischer, Frank and John Forester (eds) (1993) The Argumentative Turn in Policy Analysis and Planning, Duke University Press, Durham, North Carolina.

Keating, Michael (1996) "Defining the Policy Advising Function", in J Uhr and K Mackay, Evaluation Policy Advice: Learning from Commonwealth Experience, Federalism Research Centre, Australian National University, Canberra, pp 61-67.

Labour Market Policy Group (1995) Evaluation Guidelines, Department of Labour, Wellington.

Lawlor, Edward F. (1996) "Book review", Journal of Policy Analysis and Management, Vol 15, No 1, pp 110-121.

Leeuw, Frans L., and Richard C. Sonnichsen (1994) "Introduction: Evaluation and organizational learning: International perspectives", in Frans L. Leeuw, Ray C. Rist and Richard C. Sonnichsen (eds) Can Governments Learn? Comparative Perspectives on Evaluation and Organizational Learning, Transaction Publishers, New Brunswick and London.

Legislation Advisory Committee (1995) The Business of Legislation - Discussion Paper, Wellington.

MacRae, Duncan Jr and Dale Whittington (1997) Expert Advice for Policy Choice: Analysis and Discourse, Georgetown University Press, Washington, DC.

Majone, Giandomenico (1989) Evidence, Argument and Persuasion in the Policy Process, Yale University Press, New Haven

Martin, John (1996) "The impact of MMP on policy development and advice", Public Sector, Vol 19, No 2, pp 17-22.

Matheson, Alex (1994) "Policy: The business of relating outputs to outcomes" Address to AIC Conference on "Managing Quality Policy", 7-8 March, Wellington.

Meltsner, Arnold J. (1982) "Political and organizational analysis", The New Educational Programs in Public Policy: The First Decade, Research in Public Policy Analysis and Management Supplement 1, JAI Press, Greenwich, Connecticut.

Midgley, James with Anthony Hall, Margaret Hardiman and Dhanpaul Narine (1986) Community Participation, Social Development and the State, London, Methuen.

Miller, Gerald J and Marcia L Whicker (in press) Handbook of Research Methods in Public Administration, Marcel Dekker, New York.

Ministry for the Environment (1996) Towards Strategic Environmental Priority Setting, Overview and set of nine working papers, MfE, Wellington.

OECD (1997a) Choice of Policy Instruments, PUMA, Paris.

OECD (1997b) Occasional Papers No.17: Consultation and Communications, PUMA, Paris.

OECD (1997c) Various papers from Meeting of Senior Officials From Centres of Government on Improving Policy Capacities for the Long Term PUMA, Paris.

OECD (1996a) Various papers from Meeting of Senior Officials from Centres of Government on Integrating Multiple Interests into Policy, PUMA, Paris.

OECD (1996b) Occasional Papers No.12: Building Policy Coherence: Tools and Tensions, PUMA, Paris.

OECD (1995) Various papers from Meeting of Senior Officials from Centres of Government on The Impact of Globalization on Policy Making, PUMA, Paris.

OECD (1994a) Various papers from Meeting of Senior Officials from Centres of Government on Managing Cross-Cutting Policy Issues, PUMA, Paris.

OECD (1994b) Various papers from Meeting of Senior Officials from Centres of Government on Informing and Advising Government Leaders, PUMA, Paris.

Parsons, W. (1995) Public Policy: An Introduction to the Theory and Practice of Policy Analysis, Edward Elgar, Aldershot.

Pateman, Carole (1970) Participation and Democratic Theory, Cambridge University Press, Cambridge.

Patton, Carl V. and David S. Sawicki (1993) Basic Methods if Policy Analysis and Planning, 2nd ed, Prentice-Hall, Englewood Cliffs, New Jersey.

Patton, Michael Q. (1997) Utilization-Focused Evaluation, 3rd ed, Sage Publications, Thousand Oaks, California.

Peters, B. Guy and Barker, A. (eds) (1993) Advising West European Governments: Inquiries, Expertise and Public Policy, Edinburgh University Press, Edinburgh.

Pollitt, C. and Summa, H. (1997) "Trajectories of reform: Public management change in four countries", Public Money & Management, January-March, pp 7-18.

Pressman, Jeffrey L. and Aaron B. Wildavsky (1973) Implementation University of California Press, Berkeley.

Queensland Office of Cabinet (1993) Consultation: A Resource Document for the Queensland Public Sector, Queensland Government Publishing, Brisbane.

Radin, Beryl A. (1997) "Presidential address: The evolution of the policy analysis field: From conversation to conversations", Journal of Policy Analysis and Management, Vol 16, No 2, pp 204-218.

Rochefort, David A. and Roget W. Cobb (1994) The Politics of Problem Definition: Shaping the Policy Agenda, University Press of Kansas, Lawrence.

Rogers, Katrina S. (1996) Toward a Postpositivist World: Hermeneutics for Understanding International Relations, Environment, and Other Important Issues of the Twenty-First Century, San Francisco State University Series in Philosophy, Vol 3, Peter Land, New York.

Sabatier, Paul and Hank C. Jenkins-Smith (eds) (1993) Policy Change and Learning: An Advocacy Coalition Approach, Westview Press, Boulder, Colorado.

Schall (1995) "Learning to love the swamp: Reshaping education for the public service" Journal of Policy Analysis and Management, Vol 14, No 2, pp 202-220.

Schick, Allen (1996) The Spirit of Reform: Managing the New Zealand State Sector in a Time of Change, State Services Commission and Treasury, Wellington.

Schön, Donald A. (1983) The Reflective Practitioner: How Professionals Think in Action, Basic Books, New York.

Schön, Donald A. and Rein, Martin (1994) Frame Reflection: Towards the Resolution of Intractable Policy Controversies, Basic Books, New York.

Schwartz, Frank (1998) Advice and Consent: The Politics of Consultation in Japan, Cambridge University Press, Cambridge.

Scott, Claudia (1992) "Review of the purchase of policy advice from government departments", Public Sector, Vol 15, No 2, pp 19-24.

Snare, Charles (1996) "Windows of opportunity: When and how can the policy analyst influence the policymaker during the policy process", Policy Studies Review, Autumn/Winter, pp 407-430.

State Services Commission (SSC) (1992) The Policy Advice Initiative: Opportunities for Management, SSC, Wellington.

State Services Commission (SSC) (1991) Review of the Purchase of Policy Advice from Government Departments, SSC, Wellington.

Stern, Paul C. and Harvey V. Fineberg (eds) (1996) Understanding Risk: Informing Decisions in a Democratic Society, National Academy Press, Washington, DC.

Sterne, Peter with Sandra Zagon (1997) Public Consultation Guide: Changing the relationship Between Government and Canadians Canadian Centre for Management Development, Management Practices Paper 19.

Stokey, Edith and Zeckhauser, Richard (1978) A Primer for Policy Analysis, W. Norton and Co, New York.

Stone, Deborah (1997) Policy Paradox: The Art of Political Decision Making, W W Norton, New York.

Thomas, John Clayton (1995) Public Participation in Public Decisions: New Skills and Strategies for Public Managers, Jossey-Bass, San Francisco.

Turner, David and Sally Washington (forthcoming) "Evaluation in the New Zealand Public Management System" in Ray Rist and Rolf Sandahl (eds) Evaluation Atlas Transaction Publishers.

Tversky, A. and D. Kahnemann (1974) "Judgment under uncertainty: Heuristics and biases" Science, Vol 185, pp 1124-1131.

Uhr, James and Keith Mackay (eds) (1996) Evaluating Policy Advice: Learning from Commonwealth Experience, Federalism Research Centre, Australian National University, Canberra.

VanNijnatten, Debora L. and Sheila Wray Gregoire (1995) "Bureaucracy and consultation: The Correctional Service of Canada and the requirements of being democratic", Canadian Public Administration, Vol 38 No 2, pp 204-221.

Vedung, Evert (1997) Public Policy and Program Evaluation, Transaction Publishers, New Brunswick, New Jersey.

Voyce, Evan (1997), "Providing free and frank advice to government: Fact or fiction?", Public Sector, Vol 20, No 1, pp 9-17.

Weimer, David L. and Aidan R. Vining (1992) Policy Analysis: Concepts and Practice 2nd ed, Prentice-Hall, Engelwood Cliffs, New Jersey.

Weiss, Carol H. (1992) Organizations for Policy Analysis: Helping Governments Think, Sage Publications, Newbury Park, California.

Wildavsky, Aaron (1979) Speaking Truth to Power: The Art and Craft of Policy Analysis, Little, Brown and Co, Boston.

Wolf, Amanda (1998) "The roles and value orientations of policy professionals in New Zealand", Victoria University of Wellington..

Zerbe, Richard O. Jr and Howard E. McCurdy (1999) "The failure of market failure" Journal of Policy Analysis and Management Vol 18, no 4, pp 558-578.