Twenty questions for public policy proposals

As I re-read the comments I have received regarding my critique of plans to impose financial penalties for variations in readmission rates, I realized that my correspondents and I were talking past one another.  I was presenting views based on how to design public policy, while they were stressing (understandable) concerns about the quality and cost of medical care.  They were viewing high readmission rates as something that deserved the hammer of a financial penalty, while I was viewing the issue as one of many public policy issues surrounding the health care environment, where unintended consequences of policy intervention are something to be considered.

To help explain how I view these kind of issues, I am going to share a list of twenty questions modified slightly from those Larry Bacow used to share with his students while a professor at MIT (before heading off to be President of Tufts University.)  Even though the list is over twenty years old, and Larry might modify them even more at this point, it is still a remarkably useful framework within which to view a wide range of issues.  Some are not germane to health care, but many are. 

These kinds of questions underlie a lot of the commentary you see from me here on this blog.  Please understand that these questions, although demanding some degree of analytical rigor, are not designed to stymie public policy advances, but to focus public policy interventions in the hope of more effectively solving problems.

1.  In identifying the problem, or proposing the program, what does one hope to change?  Examples:  The overall distribution of income; the income of particular groups; incentives; resources; bargaining power; political power; competitive advantage or opportunities; a condition that afflicts some particular target population; a source of social conflict or friction; some legal, customary, or social arrangement or the legitimacy of certain actions or arrangements; values, tastes and interest; the range of choice available to some group; other.

2.  In identifying that problem, exactly what specific characteristics of the state of the world concern you, and what changes in these specific characteristics is your proposed program intended to achieve?  (Specify characteristics in terms of "final," not intermediate, objectives, e.g., the quality of life of individuals affected by the program.)

"Characteristics" should consist not only of concepts but estimates of the values of these variables.  Identify the data you use in determining these values.

Express the magnitude of the problem in terms of numbers, severity, costs, etc., and indicate the urgency or priority it should have on the agenda of those whose concern it is.

3.  What is there, if anything, about the problem or issue that makes it a matter of public concern rather than something better left to private action or allowed to take care of itself?

4.  What is it that makes the problem hard to analyze or the solution hard to apply?  Examples:  Objectives not clear; instrumental relations not clear; data not available; technological uncertainty; incentives conflict with compensatory objectives; strongly conflicting interest among parties; legal or moral or traditional obstacles; sheer magnitude or cost; jurisdictional confusion; novelty or unfamiliarity of the problem; administrative costs; incentives within administration, including possibly corruption; bad side effects; wasteful side effects; "horizontal" inefficiency; "vertical" inefficiency; political weakness of beneficiaries and advocates; others.

5.  Can you distinguish elements of the problem in terms of:  Market efficiency; administrative efficiency; equity; ethics; justice; information; technology; individual rationality, responsibility or self-control; other.

For example, would the problem disappear if you were satisfied with: The distribution of income (by income size or by particular groups); the working of the market; the equal and effective enforcement of the law; the adequacy of private insurance; the responsiveness of the political system; the elimination of specific handicaps; the goodwill of the parties involved; the state of public information and knowledge; the way existing programs are administered; the conformity of individuals to law or ethical principles; etc.

6.  What are the main changes in behavior that will be induced by the incentives, resources, information, etc., that the proposed program creates?

a.  The intended changes, without which the program serves no purpose;

b.  The undesired changes that nullify the program or elevate its cost;

c.  Changes that are neutral with respect to your purpose but that have to be allowed for or taken care of.

Distinguish the changes in behavior of the intended and unintended beneficiaries from changes in the behavior of administrators and others whose actions matter but who are intermediaries, not beneficiaries.

7.  What is it that defines, bounds or delineates the problem area of the program?  How much leeway is there in defining the problem or in the coverage of the program?  For example:  The "package" of activities involved, e.g., "medicine"; the nature of production, e.g., nuclear energy; the nature of consumption, e.g., alcoholism; the marketing arrangements, e.g., cable television; governmental jurisdiction, e.g., zoning; some connection between revenues and expenditures, e.g., social security or highway programs; the nature of the target population, e.g., the elderly; law, tradition, custom, some moral principles, e.g., adoption; other.

Is there some minimum scope or scale on which the problem has to be tackled to do any good?

8.  a.  Is the problem peculiar to the social and institutional structure of the United States or of the particular area in which it occurs, or is it a universal problem that any society must solve?

b.  How was the problem defined and handled 25 and 50 years ago?

c.  How is the problem defined in other countries or other jurisdictions, and how do they handle it there?

d.  What is new and novel about the problem or its solution?  How do the newness and novelty affect the definition of the problem, the choice of technique for solving it, flexibility or inflexibility in choice of solution, and the precedents that bear on this program or that this program will establish?

9.  Generalization of the problem:  similarities and analogies.

a.  What other problems are like this one, and what are the similarities and differences?

b.  What are the "solutions" of those other problems, and what do they suggest about solutions for this one?

c.  What is the larger problem of which this is only a part?

d.  What are the smaller problems into which this one can be subdivided?

e.  Does your problem fall into some general analytical class of problems, e.g., discrimination, public goods, externalities?

10.  What are the interdependencies between this problem and other problems, this program and other programs?

a.  The solution of which other problems will make this one disappear?

b. What other problems will disappear if this one is solved?

c.  What problems will become solvable by identifiable programs if this one is solved?

d.  What programs should be enlarged or tapered off if this one succeeds?

e.  What small problems does this one subsume?

f.  What big problem is partially ameliorated if this solution works?

g.  What problems get harder if we try this solution here?

h.  What problems are discovered if we solve this one -- i.e., what is it that we ignored that we'd stop ignoring, or begin to treat as a "problem," if this program proves workable?

11.  Is the desired change incremental or systemic?  Is the object: A Constitutional change; a major change in institutions; a shift to a new"equilibrium" that once achieved, will last; a consolidation or standardization of existing programs; an intensification or modification of existing programs; bringing old programs up to date; once-for-all change or change continuous through time; reversible or irreversible change; establishment or disestablishment of precedents?

12.  How do the dimensions of the problem and the effectiveness of the solution vary with the passage of time?  Is the problem getting better or worse over time?  Is the solution getting more effective or more obsolete over time?

13.  What checklist of techniques do you use to determine the optional ways of approaching the problem?  Examples:  Transfer payment in cash; transfer payment in kind; tax or subsidy on the transaction; licenses and prohibitions; contract enforcement; control of prices, rents or wages; insurance; information; publicity; property rights; public provisions; nationalization; provision of legal services; restructuring or a market or profession; provision of some public good or service.

What is, or ought to be, the relation of the technique or instrument to the nature of the problem or the purpose of the program?  (What techniques can be ruled out?)

14.  What assumptions, if any, about the following underlie your analysis:  Cost functions; supply elasticities; consumer preferences; market structure and market response; income elasticities of demand; price elasticities of demand; productivity trends; distribution of income, wealth, and opportunity by income size, by age, by occupation, by residence, by ethnic group; time preference and discount rates; substitutability of inputs and resources; substitution possibilities in consumption; incidence of taxes, benefits, or price changes.

15.  What determines the level of government at which the problem is perceived or the solution is initiated?

a.  Does the problem area coincide with an effective area or jurisdiction?

b.  Is multi-level collaboration or strategy involved?

c.  Is uniformity among states or regions important?

d.  Is it the nature of the problem, the nature of the preferred solution, or what, that determines the level of government or the agency of government?

e.  How much is the current definition of the problem and the current governmental action a product of historical evolution rather than of defensible choice?

16.  Identify the various interested groups that affect and/or are affected by current governmental action on the problem or program.  In what ways does your proposed program (or even your definition of the problem) reflect the need to reconcile conflicts, balance the conflicting interests, combine diverse interests in a "package," avoid drastic shifts of power, observe tradition, and appeal to diverse ends and goals?  Is the problem one that could conceivably be solved with a mix of measures that would leave nobody worse off?  Or does it necessarily detract from the wealth, welfare, power or status of some group?

17.  If you could set research in motion to help you understand this problem and its solution, what research would you initiate?  How long would it take?  Would you wait for it?  Whom would you want to do it?  What would it cost?  Would it involve assembling existing data, generating new data, experiments, regression analysis, case studies, or what?  Could you build the research into the program?  Would it be basic research or applied?  Could you count on getting the results you need?  Exactly how would your proposal be affected by the results?  What process could produce decisive evidence to establish or to falsify the more controversial hypotheses that underlie your analysis or your proposal; and can you identify what results you would take as conclusive?

18.  What mid-course corrections would you envision if your program does not work as planned?  How will you know when and if it is time to make those mid-course corrections?  How will you convince others that the time is ripe and the corrections are the right ones?

19.  a.  Assume that you held the position of one of the actors currently dealing with your problem.  How would you attempt to get your proposed program adopted and implemented?

b.  Identify a specific bet about governmental action in your policy area and the odds at which you would take  a bet that satisfies two further conditions:  (1) you believe that you can win money from your examiners; and (2) you believe your advantage stems essentially from your understanding of causal factors that your examiners might neglect (rather than any specific information you might have.)

20.  If your definition of the problem is correct and if your proposed solution is any good, why hasn't the problem been so defined and a solution applied already?