Clifford:
I got to thinking further about the first examples that your were suggesting yesterday, about the relationship of labor market characteristics or private investment to distance from project roads. As I discussed, the partial-treatment-effect model has nothing to do with project location, but only with the national-level relationship of outcomes of interest (e.g., income) to certain travel times (mean travel times from caserio centroids to places of interest, conditioned on a number of variables such as season and day of week). The situations that you initially suggested, where the Transportation project may have been located near the project roads, or where the Transportation project may have consisted of all urban roads, or where another development project might be located right on the roads of the Transportation project, would not bias or confound the results at all. The partial-treatment-effects model was developed from a probability sample of households from the entire nation (excluding certain remote and tourist areas). This model shows the nationwide relationship of outcome (e.g., income) to travel time. For another project to bias or confound the results, it would have to be so large that it changed the national relationship of outcome to travel time at the same time as the Transportation project, i.e., between the two survey rounds. It would not matter at all where it was located, just that it changed the national relationship of outcome to travel time. In such a case, we would not be able to distinguish the effect of the Transportation project from the effect of that project. To avoid this problem was the reason for making the SUTVA assumption (assumption 1 of the list of assumptions).
This issue (about what might bias the results) is almost certain to arise again, and I want to make sure that this point is understood. For example, the FTDA project was located in some of the same areas as the Transportation project, and was concerned with some of the same outcome variables (e.g., income). The fact that it is in the same areas and is concerned with the same outcome variables is irrelevant. The issue is whether it changes the national relationship of income to travel time. In estimating impact, I average the households over the entire nation (via the nationwide probability sample of households). The location of the households to the project roads is irrelevant (e.g., those close to project roads may experience large impact, and those far away may experience small impact). Whether that other project is located near the Transportation project is irrelevant. Whether that other project affects income is irrelevant. (This last point is a key point – it is the main reason why the FTDA project, located in some of the same areas as the Transportation project and concerned with affecting the same outcome variables, does not bias the estimation of impact for the Transportation project.) Whether that other project affects incomes to a greater extent near project roads is irrelevant. All that matters is whether that other project affects the national relationship of income to travel time. In that case, if we did not take it into account (i.e., if it was an unobserved variable), it would be a time-varying unobserved variable that was correlated with travel time, and it would affect our estimate of impact, derived from a national-level partial-treatment-effects model (relating income to travel time).
The key to avoiding all of the problems alluded to in the preceding paragraph was the development of a national-level partial-treatment-effects model relating outcomes of interest to (GIS-model) travel times. While these problems may affect the conventional approach to estimation of impact of road-improvement projects (i.e., a binary-treatment-variable estimate of impact associated with a zone of influence around project roads), they do not pose a threat to validity to the continuous-treatment-variable approach that was used in the present evaluation (i.e., an approach based on a national-level partial-treatment-effects model).
George
FndID(28)
FndTitle(A Note to Clifford Zinnes, 2013-12-21)
FndDescription(The partial-treatment-effect model has nothing to do with project location, but only with the national-level relationship of outcomes of interest (e.g., income) to certain travel times (mean travel times from caserio centroids to places of interest, conditioned on a number of variables such as season and day of week))
FndKeywords(impact evaluation; causal inference; analytical sample survey)