**Joseph George
Caldwell, PhD (Statistics)**

1432 N Camino Mateo, Tucson, AZ 85745 USA

Tel. (001)(520)222-3446, e-mail jcaldwell9@yahoo.com

25 January 2018

Sample Survey Design for Impact Evaluation

The goal of impact evaluation is to estimate the causal relationship between input (control) variables and output (result) variables, such as the economic impact of a training program on farmer incomes or the health impact of a public-health program. This goal is achieved by means of causal modeling and analysis. There are two basic approaches to causal modeling and analysis. One is to set up an experimental design (randomized controlled trial) in which the treatment intervention is randomly assigned to program-eligible members of a population of interest. The other is to collect observational data on a population of interest, specify a causal model that shows the causal relationship among variables, and estimate the causal effect from the observational data, taking into account the causal model.

Estimation of causal impact involves use of the
methodologies of experimental design and sample survey design. There are very
large literatures on experimental design and descriptive sample survey design.
In many applications, straightforward application of these methodologies is not
practical or useful. For example, it may not be physically or ethically or
politically practical to randomly assign treatment to subjects of interest. *Descriptive
sample survey* design is the “usual” type of sample survey design. It is
used to estimate characteristics of a population, with no consideration of
impact estimation. It is used mainly in program monitoring, not in program
evaluation. This type of survey design is of limited used in support of causal
analysis, since, without randomized assignment of treatment, the observed
treatment effect is usually a biased or inconsistent estimate of the average
treatment effect. For causal analysis, what is required is an *analytical
survey design*, which assures that the survey data will exhibit sufficient
variation in important causal variables to produce unbiased or consistent impact
estimates of useful precision.

In my statistical consulting practice, I specialize in
the design of sample surveys in support of causal analysis. My approach to
this topic is described in the article, *Sample Survey Design for Evaluation
(The Design of Analytical Surveys)*, posted at Internet website http://www.foundationwebsite.org/SampleSurveyDesignForEvaluation.pdf
.

** **