Joseph George Caldwell, Ph.D. (Statistics)
Consultant in Statistics, Economics, Operations Research and Computer Science
1432 N. Camino Mateo, Tucson, AZ 85745-3311 USA
Tel. (001)(520)222-3446, e-mail firstname.lastname@example.org
Ph.D., Statistics, University of North Carolina at Chapel Hill, 1966
B.S., Mathematics, Carnegie‑Mellon University, 1962
to US government agencies, state governments, corporations, and foreign governments
Director/Supervisor of projects in the areas of...
o statistical experimental design and data analysis (Stata, SAS, SPSS)
o sample survey design of major national surveys and statistical reporting systems
o computer models and information systems design (C, MS Access, Xbase, Oracle SQL)
o expert systems / geographic information systems (ArcView)
o systems and software engineering (C, Visual Basic, FORTRAN, DOD-STD-2167A, ISO12207)
o operations research / management science and statistics in industrial and defense applications
o monitoring and evaluation, planning and policy analysis of government programs in health, education, human services, urban problems, rural development, agriculture, tax policy analysis, and public finance
o game theory (zero-sum and non-zero-sum, constrained games, ill-conditioned problems; computer solutions of complex games)
o international development in the Philippines, Haiti, Egypt, Bangladesh, Ghana, Malawi, Botswana, Zambia and Timor-Leste
Manager of contract research firm (seven years); successful bidder on numerous technical contracts, including four Small Business Innovation Research (SBIR) contracts. Director of more than twenty projects.
Adjunct Professor of Statistics at the University of Arizona, Tucson, Arizona
Developer of technical seminars and computer program packages in sample survey design, forecasting, demographic projection, and geographic information systems
Languages: Native in English; working knowledge of Spanish, French; limited Portuguese, German, Arabic
Summary of Experience. Dr. Caldwell's professional career in research and research management has centered on the use of modern analysis techniques to solve practical problems in government, commercial, industrial, and military applications. He has directed major technical projects; developed technical training seminars; accomplished significant research results in statistics; developed statistical, demographic, and geographic-information-system computer program packages; designed statistical reporting and management information systems; and served as professor of statistics, consultant, and manager of a contract research firm.
CAPABILITIES AND EXPERIENCE IN STATISTICS
Education. Dr. Caldwell holds a PhD degree in mathematical statistics from the University of North Carolina at Chapel Hill. In his graduate studies, he specialized in the theory of experimental design and algebraic coding theory. His doctoral dissertation advisor was Prof. R. C. Bose, regarded as the "father" of the mathematical theory of experimental design, and developer of the Bose-Chaudhuri-Hocquenghem (BCH) codes, the best known class of codes for correcting random errors in noisy communication channels. In his doctoral dissertation, Dr. Caldwell developed the best-known class of codes for correcting additive and synchronization errors in noisy communication channels.
Experience. Dr. Caldwell has over thirty years' experience as a consultant and teacher of statistics. He has provided statistical consultation in a wide variety of fields, including sample survey design and analysis; statistical analysis of data; time series analysis and forecasting; simulation and modeling of industrial and military systems; test and evaluation of communications systems; industrial quality control; process control and product improvement; and planning, policy analysis, and program evaluation in health, education, social services, and economic development.
Experience in Monitoring and Evaluation. An area of specialization in which he has applied statistical methodology is monitoring and evaluation. He developed survey designs for a number of monitoring systems and program evaluation studies in the US and foreign countries. In the US, he directed a number of national projects in program monitoring and evaluation, including the Vocational Rehabilitation Evaluation Standards Study for the US Rehabilitation Services Administration; Social Services Effectiveness Evaluation for West Virginia; the Day Care Cost Benefit Study for the US Department of Health and Human Services; Cost-Benefit Analysis of National Institute for Alcohol Abuse and Alcoholism Treatment Centers; Medicaid Standards Impact Assessment. He developed the sampling plans for several national state/federal social and economic programs, including the Sampling Manual for Utilization Review of Medicaid; the Sampling Manual for Social Services (Title XX) Reporting Requirements; and the Sampling Manual for Office of Child Support Enforcement Reporting Requirements. He developed the survey design for the Department of Housing and Urban Development Housing Market Practices Survey; the Research Design for the Urban Arterials Section of the Highway Capacity Manual; and the survey design for the Elementary and Secondary School Civil Rights Survey.
Overseas, he served as Project Director and Chief of Party for the Economic and Social Impact Analysis / Women in Development Project in the Philippines. This project provided consulting in research design (experimental design, quasi-experimental design, survey design, survey instrument design) for a broad range of development projects (health, nutrition, and family planning; education; integrated agricultural production and marketing, aquaculture production, and agro-reforestation; integrated area development; feeder roads; ports; local water systems; electrification; small-scale industries, and tourism). He served as Manager of Monitoring and Evaluation for the Local Development II – Provincial Project in Egypt. This project was the largest USAID-funded local-level rural development project in the world. On this project, which involved the funding of 16,000 local-level projects, a sample survey design was constructed to enable assessment of program impact based on a sample of about 800 projects. The projects included potable water, waste water, roads, buildings, rolling stock, environment, and information systems.
Teaching. Dr. Caldwell served as an adjunct professor of statistics at the University of Arizona. He taught the graduate course, Sampling Theory and Methods, and the undergraduate course, Statistical Methods in Management (for all students of business, public administration, and management information systems).
Technical Training. Dr. Caldwell has developed and presented a number of statistics courses relating to monitoring and evaluation (Statistical Methods for Monitoring and Evaluation: A Comprehensive Survey Course). These courses have been presented on an advertised basis and as in-house courses at client facilities (US Bureau of Labor Statistics; National Opinion Research Center; Bahamas Department of Statistics). Course notes for this course are posted at Internet websites
Research in Statistical Methodology. Dr. Caldwell served as a consultant to the US Department of Education's National Center for Education Statistics, on the Statistical Analysis Group in Education (SAGE) program. In this work, he developed a new approach to the treatment of nonresponse in longitudinal surveys. For the US Office of Naval Research, he directed the project, "Fast Algorithms for Estimation, Prediction and Control." This project was concerned with the development of an estimation methodology that could be used as an alternative to the conventional least-squares procedure, in ill-conditioned estimation problems (singularity, missing values).
Computer Software in Statistics and Demography.
Computer Software for Time Series Analysis, Forecasting and Control. Dr. Caldwell developed the first commercially-available general-purpose Box-Jenkins computer-forecasting package (TIMES, described at http://www.foundationwebsite.org/BoxJenkins.pdf , http://www.foundationwebsite.org/TIMESVol1TechnicalBackground.pdf ). The Box-Jenkins (autoregressive integrated moving average) models are useful in system identification problems, such as forecasting, control, and linear predictive coding of speech.
A computer program for developing the most common Box-Jenkins models is posted at http://www.foundationwebsite.org/BoxJenkinsForecastingProgram.exe.
Computer Software for Demographic Analysis and Synthetic Estimation. Dr. Caldwell developed the DESTINY microcomputer software for making demographic projections (cohort-component, synthetic estimation) (described at http://www.foundationwebsite.org/DestCapINTL.pdf; similar to USAID’s RAPID population-projection program, but extended to handle multiple regions and ethnic groups). The DESTINY system uses the cohort-component method of population projection to produce estimates of population by age, sex, race and region, and applies the method of synthetic estimation to determine forecasts of variables related to population.
For the US Department of Health and Human Services, he directed the project to develop a prototype microsimulation forecasting model and a statistical reporting system to provide the data required by the model. The model -- called MICROSIM -- was developed to forecast caseloads and expenditures for HHS programs under various policy assumptions. He has developed numerous "custom" programs to construct survey designs, conduct sampling, analyze survey data, determine optimal allocations, and conduct cost‑benefit analysis.
Statistical Methodology for Evaluation. An article describing Dr. Caldwell’s approach to the design of analytical surveys (e.g., for impact evaluation of economic and social development programs) is posted at http://www.foundationwebsite.org/SampleSurveyDesignForEvaluation.pdf, and a computer program for determining sample sizes for complex surveys is posted at http://www.foundationwebsite.org/JGCSampleSizeProgramV53_20130917.accde (a Microsoft Access program). An illustrative example of use of this program is presented in the article Determination of Sample Size for Analytical Surveys, Using a Pretest-Posttest-Comparison-Group Design, posted at http://www.foundationwebsite.org/SampleSizeEstimationAnalyticalSurveysGeneric.htm.
Sample Survey Design. Dr. Caldwell developed the design for many important national sample surveys and statistical reporting systems. He specializes in the development of analytical survey designs to collect data for model development. He developed analytical sample survey designs for impact evaluations in the US, Jamaica, Honduras, Ghana, Burkina Faso, Namibia, Benin, Malawi, Zambia, and Côte d’Ivoire, including the following:
o Impact Evaluation of the Programme of Advancement through Health and Education (PATH), Jamaica (a conditional cash transfer program)
o Evaluation of Performance and Impact of Rehabilitation and Intensification of Olive Plantations in Rain-fed Zones, Morocco (Millennium Challenge Corporation)
Agricultural Data Collection in the Sourou Valley and Comoé Valley, Burkina Faso (Millennium Challenge Corporation)
o Community-Based Rangeland and Livestock Management Household Income and Expenditure Surveys, Namibia (Millennium Challenge Corporation)
o Conservancy Support and Indigenous Natural Products Household and Organisational Surveys, Namibia (Millennium Challenge Corporation)
o Impact Evaluation of Water Supply Activity, Ghana (Millennium Challenge Corporation)
o Monitoring and Evaluation of the Competitive African Cashew Value Chains for Pro-Poor Growth Program in Benin, Burkina Faso, Côte d’Ivoire, Ghana and Mozambique (Deutsche Gesellschaft für Zusammenarbeit (GTZ))
o Monitoring and Evaluation of the Competitive Action Cotton for Pro-Poor Growth Program in Benin, Burkina Faso, Côte d’Ivoire, Zambia, Ghana and Malawi (Deutsche Investions und Entwicklungsgesellschaft (DEG))
o Farmer Training and Development Activity, Honduras (Millennium Challenge Corporation)
Transportation Project, Honduras (Millennium Challenge Corporation)o Ghana Trade and Investment Program Survey
o Malawi Annual Primary School Enrollment Survey
o National survey of local development projects in Egypt
o National Center for Health Services Research (NCHSR) Hospital Cost Data Study
o Professional Standards Review Organization (PSRO) Data Base Development Study
o Study of the Impact of National Health Insurance on Bureau of Community Health Service Users
o 1976 Survey of Institutionalized Persons
o Housing and Urban Development (HUD) Housing Market Practices Survey
o Research Design for the Urban Arterials Section of the Highway Capacity Manual
o Elementary and Secondary School Civil Rights Survey
Statistical Program Monitoring Systems. He developed the sampling manuals for the following state-federal reporting systems:
o Sampling Manual for Utilization Review of Medicaid
o Sampling Manual for Social Services Reporting Requirements (Title XX)
o Sampling Manual for Office of Child Support Enforcement Reporting Requirements
Experimental Design and Quality Control. He developed statistical experimental designs for test and evaluation, simulation model run‑sets, chemical and physical experimentation, and industrial quality control applications.
Data Analysis. He has applied statistical software to analyze sample survey data, including for a number of the sample surveys listed earlier. He is an expert in the analysis of time series data, and has analyzed data collected in accordance with statistical experimental designs. He has applied the full range of statistical analysis procedures, including sample survey analysis, multiple regression analysis, multivariate analysis of variance, components-of-variance analysis, factor analysis, and nonparametric analysis.
He is expert in the use of modern commercial statistical analysis software (e.g., Stata, SAS, SPSS) and the use of related microcomputer software (e.g., Microsoft Access database management system).
Dr. Caldwell's recent work has centered mainly in the areas of social and economic development, evaluation and monitoring, institutional development, and management information systems development, in international-development applications. Following are summaries of several projects in these areas.
2016, Statistical Consultant, The Mitchell Group. Expert consultant in statistics; sample weighting specialist. SAREL project (USAID). The Sahel Resilience Learning Project (SAREL) and the Resilience in the Sahel Enhanced (RISE) Initiative Baseline Survey are efforts to increase the resilience of chronically vulnerable populations in the agro-pastoral and marginal areas of Burkina Faso and Niger. The RISE survey was a probabilistic household survey o 2,500 households across villages in the Sahel. The survey was a complex sample survey consisting of a stratified first-stage sample of 100 villages and a second-stage sample of 25 households in each selected village. Advised on specification of proper statistical procedures for analyzing the collected survey data, using Stata (svy module).
2015, Statistical Consultant, National Opinion Research Center of the University of Chicago (NORC). Sample survey design consulting services (statistical power analysis for sample size determination; sample allocation and selection; calculation of survey weights) to proposals and projects in international development (USAID Burundi Village Savings and Loan Association (VSLA) child welfare project; IADB Honduras Bono 10 Mil conditional cash transfer project; USAID Liberia Electoral Access and Participation (LEAP) project).
March 2014 – July 2014. Statistical Consultant, Inter-American Development Bank / Bahamas Department of Statistics. Development and presentation on a training course on small-area estimation, for the Bahamas Department of Statistics. The purpose of the course is to describe statistical methodology for making estimates of unemployment for the Bahamas Labour Force Survey (conducted in May and November of each year), for small islands or island groups for which the sample size for a particular survey round is small or zero.
June 2011 – November 2012. Economist and Statistical Analyst, Impact Evaluation of the Programme of Advancement through Health and Education (PATH), Jamaica. Government of Jamaica / Sanigest, Costa Rica. Responsible for evaluation and sample survey design used to collect household data to evaluate Jamaica’s PATH conditional cash transfer (CCT) program. Adopted the Neyman-Rubin (“potential outcomes,” “counterfactuals”) conceptual framework for the evaluation design, and constructed a sample survey design to support this approach. The sample design was an “analytical” sample design intended to provide data useful for estimating program impact and the relationship of impact to explanatory variables. The sample design was a “matched pairs” design that included matching of eligible households on a number of socio-economic characteristics, prior to selection of probability samples of treated and untreated households. Statistical power analysis was used to determine a sample size sufficient to provide a high level of power for detecting impacts of specified magnitude (“minimum detectable effects”). The precision of impact estimates and the power of statistical tests about those impacts were increased by the use of marginal stratification to assure adequate variability on explanatory variables related to outcomes of interest. The marginal stratification was implemented by setting variable probabilities of selection for each household of the population.
September 2010 – September 2012. Evaluation Expert and Statistician to the project, “Evaluation des performances et de l’impact de l’activité de rehabilitation et d’intensification des plantations d’oliviers au niveau des zones pluviales,” Agence du Partenariat pour le Progrès, Millennium Challenge Account – Maroc, Project Arboriculture Fruitière, National Opinion Research Center of the University of Chicago (NORC). Responsible for sample survey design and selection of samples for an impact evaluation of an olive development project in Morocco.
August 2010 – July 2012. Evaluation Expert and Statistician to the project, “Agriculture Data Collection in the Sourou Valley and Comoé Basin.” Millennium Challenge Account – Burkina Faso (MCA-BF), National Opinion Research Center of the University of Chicago (NORC). Responsible for construction of sample survey design and selection of samples for an impact evaluation of two agricultural development projects in Burkina Faso.
August 2010 – November 2011. Evaluation Expert and Statistician to the project, “Community-Based Rangeland and Livestock Management Household Income and Expenditure Surveys.” Millennium Challenge Account – Namibia (MCA-N), National Opinion Research Center of the University of Chicago (NORC). Responsible for construction of sample survey design and selection of samples for an impact evaluation of a rangeland management project in Namibia.
August 2010 – March 2012. Evaluation Expert and Statistician to the project, “Conservancy
Support and Indigenous Natural Products Household and Organisational Surveys.” Millennium Challenge Account – Namibia (MCA-N), National Opinion Research Center of the University of Chicago (NORC). Responsible for construction of sample survey design and selection of samples for an impact evaluation of an indigenous natural products project in Namibia.
July 2010 – September 2010. Evaluation Expert and Statistician to the project, “Impact Evaluation of Water Supply Activity.” Millennium Development Authority—Ghana (MiDA), National Opinion Research Center of the University of Chicago (NORC). The Water Supply Activity project was undertaken by the Millennium Development Authority – Ghana (MiDA) as part of its Compact with the US Millennium Challenge Corporation (MCC) to improve infrastructure in selected agricultural areas in Ghana. The goal of the water supply activity improvements was to improve the quantity and quality of water in MiDA program areas, and thereby improve the health and economic status of communities in those areas. Of particular interest were effects on household health outcomes, time savings, and income levels. The purpose of the evaluation project was to conduct a rigorous impact evaluation of the program to assess the extent to which it achieved its goals. The evaluation design was a pretest-posttest-comparison-group design, and the basic measure of program impact was a double-difference estimate based on this design. Dr. Caldwell constructed the evaluation and survey design for the evaluation project.
November 2009 – October 2010. Evaluation Expert and Statistician to the project, “Monitoring and Evaluation of the Competitive African Cashew Value Chains for Pro-Poor Growth Program”, Deutsche Gesellschaft für Technische Zusammenarbeit (GTZ) GmbH, National Opinion Research Center of the University of Chicago (NORC). Here follows a brief summary of the project, taken from the project grant proposal: “The project will contribute to sustainably reducing rural poverty in five African countries (Benin, Burkina Faso, Côte d’Ivoire, Ghana and Mozambique). An estimated 2.5 million mainly smallholder farmers grow cashew in Africa. Annually production of almost 750,000MT they supply about 40% of the world’s cashew crop. But only about 12% of cashew nuts are processed into cashew kernels in Africa. The cashew project aims to improve the quality of raw cashew nut cultivation, increase farmer productivity, improve linkages between smallholder farmers and the marketplace, build African processing capacity and promote a sustainable global market for African cashews. The project’s goal is to help 150,000 smallholder cashew farming households in Benin, Burkina Faso, Côte d'Ivoire, Ghana and Mozambique increase their incomes by 50 percent by 2012.” The goal of the program was to increase income and employment for cashew farmers. The purpose of the evaluation project was to conduct an economic impact evaluation of the program to assess the extent to which it is achieving its goals. For the evaluation, surveys were conducted in all five countries of the program. Dr. Caldwell constructed the evaluation and sample survey designs for all program countries except Mozambique. The measures of program impact were double-difference estimates based on pretest-posttest-comparison-group evaluation designs. Sample sizes were determined by statistical power analysis to assure high power for detecting impact effects of specified size. A two-stage sample design was employed, with selection of a first-stage sample of villages and a second-stage sample of farmers within sample villages. The sample design used matching to increase precision of estimates and power of tests of hypotheses. Marginal stratification, implemented through the use of variable probabilities of selection, was used to assure adequate variation in explanatory variables.
March 2009 – March 2013. Evaluation Expert and Statistician to the project, Monitoring and Evaluation of the Competitive African Cotton for Pro-Poor Growth Program ("COMPACI”), Deutsche Investitions und Entwicklungsgesellschaft GmbH (DEG), NORC. The purpose of the project was to conduct an economic impact evaluation of the “Cotton Made in Africa” initiative. For the evaluation, surveys were conducted in six countries: Benin, Burkina Faso, Côte d’Ivoire, Zambia, Ghana and Malaŵi. Under the program, cotton farmers were provided training and services so that their cotton may be certified as having been produced under the “Cotton Made in Africa” (CMiA) program. The goal of the program was to increase income and employment for cotton farmers. Dr. Caldwell constructed the evaluation and sample survey designs for all program countries. The measures of program impact were double-difference estimates based on pretest-posttest-comparison-group evaluation designs. The sample designs for the cotton project were similar to those described above for the cashew program (statistical power analysis, matching, marginal stratification, variable probabilities of selection).
February, 2009 – February 2011. Lead Statistician, Impact Evaluation of Feeder Roads Activity, Millennium Development Authority - Ghana (MiDA), NORC. The purpose of the project is to conduct an impact evaluation of the MiDA Feeder Roads Activity in eight of its 23 program districts. The evaluation will determine the impact of feeder roads improvements on input costs, product prices, and passenger fares and goods’ tariffs that are associated with reduced travel time and vehicle operating cost. The primary data for the impact evaluation will consist of three market surveys, similar in scope to the Consumer Price Index (CPI) survey, examining changes in price over time in localities different distances from the improved road segments. The sample design involved matching of treatment and control localities using a “nearest neighbor” technique with a data set enhanced with GIS methods. The impact of the roads improvements will be determined employing a double-difference estimator applied to changes in prices over the next two years. Dr. Caldwell was responsible for providing advice on the strengths and weaknesses of particular evaluation designs, devising sampling strategies and designs, estimating sample sizes, drawing the sample for data collection activities, preparing weights to apply to the price and fare observations, and assisting with analysis plans to ensure statistical robustness of results.
May 2007 – September 2013. Evaluation Expert and Statistician, Millennium Challenge Account - Honduras Program Impact Evaluation, National Opinion Research Center (NORC), Honduras. Technical advisor to provide evaluation research design and analysis services in support of an economic impact evaluation of roads-improvement and farmer-development projects funded by the Millennium Challenge Corporation in Honduras. Dr. Caldwell developed the evaluation and sample survey designs for the two projects.
For both projects, statistical power analysis was used to determine sample size. Using this approach, the sample size was determined so that the probability (power) of detecting an effect (impact) of a specified size was high. Both projects involved a “panel” sample design in which the survey was administered before and after the program intervention, i.e., the basic design was a “pretest-posttest” design. The conceptual framework for the impact analysis was the “Neyman-Rubin causal model”, or “potential outcomes model,” or “counterfactuals model.”
For the farmer assistance project, eligible villages (“aldeas”) were classified into sets of “matched pairs,” and one member of each pair was randomly selected to receive program services. The matching was done on a number of variables believed to affect outcomes of interest, and available prior to the survey. The matching was done prior to randomized selection for treatment, to increase the precision of impact estimates and the power of tests of hypothesis about them. A probability sample of matched pairs was selected using the technique of “marginal stratification,” to ensure adequate variation (spread, balance) in the design variables. The randomized-assignment-to-treatment sample was supplemented by a sample selected for treatment in the usual fashion (by the program implementer). The data analysis included development of a “two-step” model, in which the first step was a binary “selection” (propensity-score) model and the second step was an “outcome” model that included the selection probability estimated in the first step. The principal impact estimate of interest was the Average Treatment Effect (ATE), or average effect of the program intervention on an eligible farmer. The ATE was a “regression adjusted” or “covariate adjusted” double-difference estimate.
The survey design for the transportation project included selection of a probability sample of caserios (administrative units generally smaller than villages), where marginal stratification was once again used to assure adequate variation in variables believed to affect outcome. In particular, the selection probabilities were set to assure adequate variation in the estimated change in travel time to be caused by the program intervention (road improvements). The estimated change in travel time was calculated from a GIS road-network model that included all official roads in Honduras. The survey data were used to develop an estimate of the Partial Treatment Effect (PTE) (relationship of impact to travel-time variables) and, from the PTE, the Average Treatment Effect.
May – June 2008. Statistical Consultant, Analysis of Poverty and Social Impact of Education Sector Reforms in Mozambique, World Bank / KPMG / Manitou Incorporated. Developed the data-entry program to be used for a national sample survey of households, to assess the economic and social impact of education sector reforms. The US Bureau of the Census CSPro software system was used for this application. The questionnaire and corresponding data-entry forms were in Portuguese.
Dec 2007 – Feb 2008. Systems Integration Consultant, Governance and Economic Management Assistance Program (GEMAP), USAID / Segura Consulting, Liberia. Technical advisor to a project funded by the US Agency for International Development to develop a computer system to automate tax payments. The goal of the project is to establish a “One-Stop Shop” at the National Port, where importers can settle their tax obligations quickly. Developed system requirements specifications and procurement documents. The system includes radio communication links among the National Port, the Ministry of Finance, and the Central Bank.
June 2007. Consultant in Information Technology and Statistics, Guinea Baseline Survey, Indefinite Quantity Contract (IQC) for Democracy and Governance Analytical, Support and Implementation Services, US Agency for International Development / Management Systems International, Guinea. Technical advisor to develop a design for a database to store data required in support of USAID’s Performance Monitoring Plan (PMP) reporting and management needs, and for a statistical sample survey to collect data to be stored in the database. Advised on the database design (e.g., static Word files, static HTML files, standalone Microsoft Access database, networked database, web-based dynamic system (e.g., MS ASP.net, Adobe ColdFusion, Sun Java Server Pages, Linux operating system / Apache web server , MySQL database, PHP web page (LAMP)), selection of sample survey data-entry software (e.g., Epi Info, CSPro, Viking, SPSS), and sample survey design (a two-stage sample survey design using Census enumeration districts as primary sampling units (PSUs) was recommended, to provide data in support of a pretest-posttest comparison-group quasi-experimental design). Statistical power analysis was used to determine the survey sample sizes (number of sample PSUs, number of sample households within PSUs).
Mar 2006 – Sept 2006. Technical Advisor in Personnel Management Information Systems, United Nations Development Program, East Timor. Technical advisor to advise the Government of Timor-Leste on the selection of a software developer to develop a civil-service personnel management information system.
Feb 2002 – April 2005. Technical Advisor in Educational Management Information Systems, Academy for Educational Development, Zambia. Technical advisor to a project funded by the US Agency for International Development, to develop an Educational Management Information System (EMIS) for the Zambia Ministry of Education. The purpose of the EMIS is to collect, store, and retrieve data (produce reports) from the Annual School Census, in support of program planning and analysis by the Ministry and donor agencies. Applications developed in Microsoft Access database development system, the Academy for Educational Development’s EdAssist system, and the ArcView geographic information system (GIS).
Jan 1999 – Jan 2001. Director of Management Systems, Bank of Botswana, Botswana. Responsible for management of all information technology operations for the Bank of Botswana, Botswana’s central (reserve) bank (IT vision, strategy, policy, procedures, operations, acquisition, training, staff development). The Bank’s computer system is comprised of over 300 networked microcomputers running under Windows NT/95/98/2000, Novell 4.1 and UNIX operating systems. Managed a group of 16 information technology specialists to operate and support the Bank’s computer hardware and software applications (network management; Microsoft Office Suite; Internet/intranet; banking operations; accounting; investment portfolio / foreign reserve management; financial data services; economic analysis; human-resources management; and asset management. Introduced modern management and software engineering practices based on standards-based quality management (ISO 9000 Quality Management standard, ISO 12207 Information Technology standard, Carnegie Mellon University Software Engineering Institute Capability Maturity Model (CMM), DOD-STD-498 Software Development and Documentation). Responsible for system development (design, implementation), procurement, training, operations and maintenance (annual budget approximately USD3 million, exclusive of staff salaries, training, and noncomputer facilities and equipment). Responsible for setting Bank’s IT vision, strategy, policy, procedures, security. Supervised approximately 30 IT projects. Directed the Bank’s Year-2000 date-change (“Y2K”) program, in accordance with international standards (Bank for International Settlements and US government) (no date-change problems encountered after the century date change). Directed preparation of the Bank’s first disaster-recovery plan. Directed preparation of the Bank’s first disaster-recovery plan. Supervised the development of the Bank’s first web page, and acquisition of the country’s first “code-line clearing” system (for magnetic-ink character recognition (MICR) of bank checks). Participated in all meetings of the Bank’s Executive Committee and Board of Directors; reported to the Governor and Deputy Governor.
Apr – Oct 1998. IT Specialist, Educational Management Information System Design for Secondary Education Sector Development Project, Asian Development Bank / Academy for Educational Development, Bangladesh. Developed top-level requirements for the Educational Management Information System (EMIS) to be developed under a multi-year development program funded by the Asian Development Bank. Assignment included review of current systems, identification of user information needs, and identification and comparative evaluation of alternative systems.
Sep 1997 – Mar 1998. Consultant in Risk Management, Strategic Sourcing Inc., / Canada Trust Bank, Bank Risk Management, Canada. Consultant in risk management to Canada Trust Bank. Responsible for the development of analytical models for risk management of the Bank's loan products. Developed a model for risk-based variable-rate pricing of loans, using the techniques of Generalized Lagrange Multipliers (GLM) and mathematical simulation. The methodology determines pricing strategies that are optimal with respect to the allocation of capital to the Bank's investment opportunities, taking customer, market, and policy factors into account. The computer simulation approach is used as an efficient framework for exploring alternative pricing strategies; the GLM method is used to determine pricing strategies that maximize stockholder value added (profitability) subject to constraints (on capital reserve requirements, probability of exceeding loss provisions, and other factors). Windows NT, UNIX, SAS, VB5.
May 1996 – Jul 1997. Statistical Consultant to Strategic Sourcing Inc. / Wachovia Bank, Statistical and Optimization Computer Models in Banking, USA. Consultant to First Union National Bank (US sixth largest bank, now Wachovia Bank), conducting statistical analysis to develop customer segmentation models in support of bankcard marketing initiatives. Developed optimization model for identifying profitable locations for automatic teller machines (ATMs). Used SAS statistical analysis software and ArcView 3.0 geographic information system (spatial analyst) to develop logistic regression and discriminant analysis models to identify likely customers for PC banking. Models used a wide range of economic and demographic data at the block group and ZIP-code levels (population, income, employment, sales, shopping centers, crime statistics, traffic counts, ATM locations and characteristics). Windows 95 and UNIX (Sun Solaris SPARCcenter).
Nov 1995 – May 1966. Survey Statistician, Income and Employment Survey for Ghana Trade and Investment Program, Sigma One Corporation / USAID, Ghana. As part of the US Agency for International Development's Trade and Investment Program in Ghana, Dr. Caldwell designed and analyzed the survey to estimate the employment and income associated with every $1,000 of exports in non-traditional areas. The survey was designed to produce national estimates and estimates for selected product sectors (pineapples, pineapple juice, tuna loins / canned tuna, and cashew nuts). The sampling plan involved a probability sample of 300 exporting firms selected with probabilities proportional to a measure of size (export value) without replacement.
May – Jun 1995. Sample Survey Design and Sampling Statistician, Academy for Educational Development / USAID, Malawi. For the Malawi Ministry of Education, Dr. Caldwell developed the sample design for the Annual Primary School Survey. Previously, the annual school survey was a census of all 3,400 schools and three million students; the amount of time and effort required to collect and process all of these data was placing a serious burden on the Planning Unit resources. The sampling plan involves a probability sample of 500 schools selected with probabilities proportional to a measure of size (the previous year's enrollment) using the Rao-Hartley-Cochran method. With the probability sampling approach, all of the information required by the Planning Unit will be available for a fraction of the effort required by the previous approach.
Jun 1993 – Dec 1994. Personnel Management Information System Developer, Civil Servant Personnel Management Information System, Academy for Educational Development / USAID, Malawi. For the Malawi Department of Human Resources Management and Development, Dr. Caldwell designed and implemented the Malawi Civil Service Personnel Management Information System. The system was developed using the dBASE database management information system, for use on microcomputers (standalone or networked) using the MS-DOS operating system. The system includes a variety of demographic and employment-related data for Malawian civil servants, and offers the users (personnel officers) a wide range of easy-to-use data entry and query/report capabilities. Experienced database users may generate queries and reports using SQL (Structured Query Language) commands or any of dBASE's automated query and report-generation features, but the system is designed with a powerful graphical user interface (GUI) so that a nontechnical user may generate all standard queries and reports without the need for any programming or entering of complicated commands, simply by making selections from a suite of menus. Data entry is facilitated by a series of easy-to-use data entry screens, with ample on-line help and validation of all entered data. Employee records may be displayed on the screen or printed.
The system development effort was conducted in full compliance with the DOD-STD-2167A software development standard, and included the production of almost 1,000 pages of detailed system documentation, including a System Design Document, Software Requirements Specification, Software Design Document, Software Programmer's Manual, Software Product Specification, and Software User's Manual. The project included on-the-job training of members of the Department's Management Information Systems Unit (systems analysts, programmers) in systems engineering (requirements analysis, technology assessment, synthesis of alternatives, specification of evaluation criteria, selection of a preferred alternative, top-level design, detailed design (optimization), implementation, and test), the modern software engineering discipline (structured, top-down design), management information system design, dBASE, software development project management, and basic microcomputer upgrading and repair; and classroom instruction for system users (personnel officers) in use of the system for data entry and retrieval (queries and report generation).
Mar 1991 – Oct 1992. Manager of Monitoring and Evaluation, Chemonics International / USAID, Egypt. Served as manager of Monitoring and Evaluation for the USAID-funded Local Development II - Provincial (LDII-P) project, which provided technical assistance in the development and maintenance of USAID-funded infrastructure projects in Egypt (potable water, waste water, roads, buildings, rolling stock, environment, and information systems). The LDII-P project was the largest USAID local development project in the world, having funded the development of over 16,000 local-level projects. In addition to infrastructure development, a major goal of the project was to promote government decentralization and increase the capacity of local governments to plan, finance, implement, and maintain local projects. Principal activities included: (1) the design and implementation of a nationwide project monitoring survey to assess the implementation, operating, and service status of projects; (2) the development of an indicators system to assist local officials in the assessment of need for public services, the availability of services, and the identification and prioritization of local development projects; (3) the design and implementation of a governorate project monitoring system to assist governorate detection and follow-up of implementation and operational problems. On this project, Dr. Caldwell made heavy use of automated management information system tools (dBASE, SPSS) to store, process, and retrieve data on project status and needs assessment (including continuous monitoring of project status indicators), and applied the techniques of sample survey (questionnaire development, stratified random sampling) and rapid appraisal techniques (focus group interviews) to assist end-of-project evaluation, as well as continuous monitoring of indicators. Dr. Caldwell lectured on the use of geographic information systems (GISs) in development planning, and supervised training of development planners in use of the PC-ARC/INFO GIS.
Oct 1979 – Jan 1982. Project Director / Chief of Party, Economic and Social Impact Analysis / Women in Development (ESIA/WID) Project, Vista Research Corporation / USAID / NEDA, Philippines. The purpose of this project, sponsored jointly by the Philippines National Economic and Development Authority (NEDA) and the US Agency for International Development, was to help improve the capability of the Government of the Philippines to monitor and measure economic progress, social change, and the impact of development projects, including the effects on women in their dual role as agents and beneficiaries of development. The contract provided technical services to assist the Philippines Institute of Development Studies (PIDS) to develop and validate analytical frameworks and indicators for analyzing and measuring progress and the impact of development projects on selected areas of concern; to design and field test efficient means for measuring and monitoring project progress and impact indicators; and to determine a better understanding of the mechanisms by which development projects achieve their goals. The development projects included a wide variety of substantive fields ‑‑ health, nutrition, and family planning; education; integrated agricultural production and marketing, aquaculture production, and agro‑reforestation; integrated area development; feeder roads; ports; local water systems; electrification; small‑scale industries, and tourism. The ESIA/WID project identified and evaluated the use of a variety of statistical design and analysis techniques to assist project impact assessment: quasi-experimental designs, sample survey, analysis of variance, multiple regression analysis, questionnaire design, indicator development. For the Philippines Ministry of Health, Dr. Caldwell developed alternative management information system (MIS) designs to support both agency operations and program monitoring. Dr. Caldwell served as chief of party and directed a team of eleven Ph.D. consultants on the ESIA/WID project.
Oct 1975 – Sep 1976. Project Director /Supervisor, Economic Policy Analysis for the Government of Haiti, JWK Intl Corp / USAID, Haiti. Under a contract funded by the US Agency for International Development, this study determined agricultural and tax policy changes that the government of Haiti could employ to increase foreign exchange and increase the income of the small farmer. The study addressed five commodities ‑‑ coffee, cotton, sisal, mangoes, and meat (major emphasis on coffee). The project included the use of sample surveys to collect up‑to‑date data on commodity prices. A major goal of the project was the transfer of policy analysis capabilities to members of the Haitian Ministry of Agriculture. Dr. Caldwell supervised a team of four Ph.D. consultants (economists) on this project.
SUMMARY OF RELATED APPLICATIONS
As mentioned, Dr. Caldwell has experience in a wide variety of applications areas. In his professional career, he has served both as a consultant specialist and as a research manager. He has directed many research projects, and supervised project teams of up to a dozen technical experts. The paragraphs that follow provide additional information on some of these applications.
Standards-Based Quality Management. For larger projects, Dr. Caldwell employs a “standards-based quality management” approach to project management. This approach makes full use of internationally recognized management and technical standards that are applicable to the effort. Examples of projects that he directed that employed this approach are the following:
· Manager of Research and Development and Principal Scientist, US Army Electronic Proving Ground’s Electromagnetic Environmental Test Facility. In this role, all of the engineering and software development efforts directed by Dr. Caldwell were conducted in conformance with applicable US military standards (software development, systems engineering, test and evaluation).
· Personnel Management Information System (PMIS) for the Government of Malawi. This project, which developed the personnel management information system for the Malawi civil service, was conducted in strict compliance with the leading software development standard at the time, the US Department of Defense’s Defense Systems Software Development, 2167A, which was the predecessor to today’s international information-technology standard, ISO 12207, Software Life Cycle Processes
· Research in Artificial Intelligence for Noncommunications Electronic Warfare Systems. The purpose of this project was to develop an automated system for generating military scenarios for use in testing of military electronic-warfare systems. This project was developed in full compliance with the DOD-STD2167A Defense Systems Software Development Standard.
· Director of Management Systems for the central bank of Botswana. As Director of Management System for the Bank of Botswana, Dr. Caldwell introduced a number of quality-management initiatives, including:
o Direction of the Bank’s Year-2000 program using guidelines published by the US General Accounting Office (“Year 2000 Computing Crisis: Business Continuity and Contingency Planning”) and the Bank for International Settlements. As a result of this program, the Bank did not experience a single “Year 2000 date change” problem.
o Use of the ISO 12207 Information Technology Standard to guide all major software development and acquisition efforts (such as the effort to acquire a national code-line clearing system based on magnetic-ink character recognition of checks, and the project to acquire a computer network management system for the Bank).
o Initiation of an effort to have the Bank’s Management Systems Department operate in compliance with the ISO 9000 Quality Management Standard.
o Assessment of the software development capability of the Bank’s staff and its software suppliers using the Carnegie Mellon University Software Engineering Institute’s Capability Maturity Model (CMM) (predecessor of the ISO 15504 Standard, Software Process Improvement and Capability Determination (“SPICE”)).
o Direction of the project to develop the Bank’s Business Continuity Plan / Disaster Recovery Plan, using the Business Continuity Planning Guidelines issued by the Texas Department of Information Resources.
o Direction of the project to develop an information technology security plan, using the US General Accounting Office’s Information Security Risk Assessment guidelines.
In addition to providing assurance that work conducted in compliance with international professional standards will be of high quality, one of the other distinct benefits of using standards-based quality management is that staff members benefit greatly from being provided the opportunity and experience of working in compliance with quality management and technical standards.
Statistical Reporting Systems, Program Monitoring Systems, Management Information Systems. Dr. Caldwell developed the design for many important national sample surveys and statistical reporting systems. He specializes in the development of analytical survey designs to collect data for model development, and has developed new techniques for handling nonresponse in longitudinal surveys. Surveys were listed earlier. Reporting systems and management information systems include:
o Zambia Education Management Information System (EMIS)
o Malawi Personnel Management Information System (PMIS)
o Sampling Manual for Utilization Review of Medicaid
o Sampling Manual for Social Services (Title XX) Reporting Requirements
o Sampling Manual for Office of Child Support Enforcement Reporting Requirements
Evaluation Research. Projects for which Dr. Caldwell developed the evaluation design or sample survey design were listed earlier. Dr. Caldwell has conducted a number of evaluation research studies, including the following:
o Evaluation Survey of USAID Local Development Projects in Egypt
o Social Services Effectiveness Evaluation for West Virginia
o Day Care Cost‑Benefit Study
o Vocational Rehabilitation Evaluation Standards Study
o Cost‑Benefit Analysis of National Institute for Alcohol Abuse and Alcoholism (NIAAA) Alcoholism Treatment Centers
o Medicaid Standards Impact Assessment
Systems and Software Engineering; Computer Models, Systems and Applications; Management Information Systems; Database Systems. Dr. Caldwell has directed numerous software engineering projects, applying the modern principles of systems and software engineering. This approach includes requirements specification and analysis, technology review, synthesis of system alternatives, cost-effectiveness analysis of alternatives and selection of a preferred alternative, detailed design, implementation and test. For the software subsystem he utilizes top-down, structured design, and has experience using international standards, including the ISO 12207 Information Technology Standard and its predecessors (the US Department of Defense's Software Development Standard (DOD-STD-2167A and MIL-STD-498)).
He has extensive hands-on microcomputer systems development experience. He designed and implemented a 50,000-line C-language microcomputer program (an integrated geographic information system / expert system), and personally conducted all of the software and database design and most of the programming for the information systems work in the Egypt, Malawi and Zambia applications mentioned above (dBASE, MS Access, SQL).
In a banking application, he developed a geographic information system application (ArcView 3.0 GIS, SAS) to identify good locations for bank automated teller machines (ATMs). He developed simulation/optimization system for a bank to determine optimal loan pricing strategies (Windows NT, Microsoft Visual Basic 5.0).
His computer experience includes mainframe, mini- and microcomputer applications. Most recent work has been on MS-Windows-based microcomputer operating systems. His system design work includes both hardware and software system design. Much experience with MS-Windows application development systems (MS Access, Visual Basic, C/Visual C++, Visual Fortran, Visual FoxPro, FrontPage web page development.
Automated Receiver Operating Characteristic System; Diagnostic Imaging Systems. Conducted requirements analysis and specification for the statistical system of an automated receiver operating characteristic (ROC) system. The goal of the development effort was to develop an easy-to-use, microcomputer-based system for facilitating the design, implementation and analysis of receiver operating characteristic experiments. (A ROC experiment is an experiment designed to determine and describe the accuracy of a diagnostic system, such as a computer imaging system. The system is to make a decision about what alternative state of nature is true, based on an (image) observation. The ROC methodology lends itself well to graphical presentations on microcomputer screens, e.g., in medical diagnostic imaging systems or military multisensor fusion applications.)
Monitoring and Evaluation. He served as Manager of Monitoring and Evaluation for the Local Development II - Provincial (LDII-P) project, the largest local development project funded by the US Agency for International Development (16,000 projects). In this role, he directed the design and analysis of sample surveys to assist monitoring and evaluation of the implementation and operational status of local infrastructure-development projects; the development of an indicators system to assist local planners in (1) assessing the need for and availability of public services (water, roads, schools, health facilities) and (2) identifying, designing, and selecting local development projects (e.g., roads, water systems, buildings); and the development of a governorate-level system to assist monitoring and follow-up of development projects.
Public Finance. In addition to his work in tax policy analysis, Dr. Caldwell directed studies to develop alternative allocation/matching formulas for major state/federal programs:
o Vocational Rehabilitation State Allocation Formula
o Medicaid and Aid to Families with Dependent Children (AFDC) Matching Percentage Formula
Privatization, Decentralization, and Democratization. Broad experience in monitoring, evaluation, and policy analysis related to privatization, decentralization, and democratization, with special emphasis on the development of "harmonious" tax systems that support these objectives. In the Haiti agricultural policy analysis project mentioned above, emphasis was on the identification of changes in tax policy that would increase small-farmer incomes. In the Egypt LDII-P project, a major thrust of the project was to implement the infrastructure development projects at the village level, using local contractors. Training was provided in project planning, design, selection, contracting procedures, monitoring, and financing; Dr. Caldwell directed the development of systems to facilitate decentralized (local-level) development, and to monitor progress in local capacity to design, implement, and finance local-level projects. In the Philippines ESIA/WID project, heavy emphasis was placed on assessment of the role of women in development and on estimation of income changes associated with development projects. In his book on tax policy (http://www.foundationswebsite.org/VAT.pdf), Dr. Caldwell presents a systematic methodology for tax system development ("tax engineering") which takes into account social, economic, and political constraints and objectives.
Management Consulting / Business Experience. Dr. Caldwell has substantial experience in management consulting to industry, including consulting, training, and system development in forecasting, quality control, product improvement, process control, and economic analysis of production alternatives. He founded and managed his own contract research firm (Vista Research Corporation, operated full-time for seven years), and a ladies' fashions importing/retailing firm (Sonora Marketing Corporation). In these efforts, Dr. Caldwell designed, implemented and managed all major functional components of the operations (marketing, production, and finance).
Operations Research and Statistics in Industrial and Defense Applications. Dr. Caldwell has applied a wide variety of operations research and statistical techniques to solve practical problems in industrial and defense applications. Industrial applications include the use of simulation and modeling, experimental design, and statistical forecasting techniques to solve problems in process control, statistical quality control, demand forecasting, and economic analysis of alternative modes of production in the textile and pharmaceutical industries. Defense applications include mathematical analysis of alternative defense strategies using the theory of nonzero‑sum games and Lagrange multipliers; test and evaluation of electronic communications systems and equipment using experimental design and simulation and modeling; and correlation / tracking of targets. He pioneered the use of statistical experimental design to specify runs for large‑scale simulation models.
Test and Evaluation in Communications‑Electronics. He served as Manager of Research and Development and Principal Scientist of the US Army Electronic Proving Ground's Electromagnetic Environmental Test Facility, which is responsible for test and evaluation of US Army communications-electronics equipment. He supervised the design and analysis of development tests of military communications‑electronics (C‑E) systems, including queuing analysis of message flow, radar system evaluation, Bayesian reliability analysis, and components-of-variance analysis of voice scoring data in noisy communication channels. He directed the specification of software and hardware architectures for large‑scale dynamic event‑driven and real‑time simulation systems.
Artificial Intelligence / Expert Systems / Geographic Information Systems. For the US Army Communications-Electronics Command (CECOM), he directed the project, "Research in Artificial Intelligence for Noncommunications Electronic Warfare Systems." This project developed an expert (rule-based) system for positioning military units and equipment, taking into account tactical doctrine, the mission of the friendly forces, the nature of the enemy threat, the location of friendly and enemy forces, and geographic features (digital terrain data). The system incorporated the NASA-developed C-Language Integrated Production System ("CLIPS") expert system and used digital mapping data extracted from the US Army's Geographic Resources and Services System (GRASS) geographic information system (GIS). The system (programmed in C) was developed for MS-DOS-based 80x86 microcomputers, made heavy use of graphics, and incorporated mouse and windows features.
Information Theory / Intelligence Analysis. Dr. Caldwell developed the best‑known class of error‑correcting codes for correcting both additive and synchronization errors in digital transmission. He developed statistical procedures for correlation/tracking of ocean traffic using satellite intelligence sensor reports.
Computer Languages, Packages, and Systems. Heavy experience in applications programming in C, Visual C, Visual Basic, dBASE/FoxPro, Microsoft Access, SQL and Fortran on mainframe computers, minicomputers and microcomputers under a variety of operating systems (MS-DOS, Microsoft Windows, UNIX, IBM, CDC, UNIVAC, Sun Solaris SPARCcenter and others); experienced in application of statistical program packages, such as SAS and SPSS. Strong microcomputer experience, including the development of graphics-based microcomputer software for geographic information systems applications. Familiar with a variety of commercial microcomputer software (e.g., word processing, electronic spreadsheet, data base, desktop publishing, accounting). Experience working in a Microsoft Windows / UNIX network environment (VB, SAS, Oracle). Familiar with Microsoft Office suite of products (Word, Access, Excel, PowerPoint, FrontPage) on Windows 95/NT/XP or UNIX client/server system.
Languages. English (native); working knowledge of French and Spanish; limited German, Portuguese and Arabic (for transportation, household use).
Geographic Experience. USA, Canada, Haiti, Philippines, Egypt, Malawi, Ghana, Bangladesh, Botswana, Zambia, Timor-Leste, Guinea, Liberia, Namibia, Burkina Faso, Honduras, Jamaica
Publications. Over fifty publications in the areas described above, and books on tax reform and global population (list available on request). Many articles on diverse topics (energy, environment, politics, tax reform, music, guitar, defense, religion / spirituality / philosophy, science fiction).
Honors / Awards. Tau Beta Pi National Engineering Honorary Society,,eneral Motors Scholarship (Carnegie‑Mellon University, Pittsburgh), NASA Fellowship (University of North Carolina at Chapel Hill)
Professional Affiliations. Institute for Management Sciences and Operations Research (INFORMS), American Statistical Association, Institute of Mathematical Statistics
Consultant in Statistics, Optimization and Information Technology, 1974-present (various organizations, including National Opinion Research Center at the University of Chicago (NORC), The Mitchell Group, Sanigest, Academy for Educational Development (AED), Chemonics, Wells Fargo Bank, Bank of Botswana, United Nations Development Program (UNDP), contractors of United States Agency for International Development (USAID), Millennium Challenge Corporation (MCC), Inter-American Development Bank (IDB),African Development Bank (AfDB), Asian Development Bank (ADB))
Software Engineer (Developer of Education Management Information System (EMIS)), Academy for Educational Development, Lusaka, Zambia, 2002-05
Director, Management Systems Department, Bank of Botswana, 1999-2001
Software Engineer (Developer of national civil service Personnel Management Information System (PMIS)), Academy for Educational Development, Lilongwe, Malawi, 1993-94
President and Manager, Vista Research Corporation, Tucson and Sierra Vista, AZ, 1988-91
Adjunct Professor of Statistics, University of Arizona, Tucson, AZ, 1982-86 (taught the graduate course in sample survey design and analysis and the core statistics course for students in business, public administration and management information systems)
Director of Research and Development and Principal Scientist of US Army Electronic Proving Ground's Electromagnetic Environmental Test Facility, Bell Technical Operations and Combustion Engineering, Tucson and Sierra Vista, AZ, 1982-86, 1986-88
Principal Engineer, SINGER Systems and Software Engineering, Tucson, AZ, 1986
President and Manager, Vista Research Corporation, Alexandria, VA, and Tucson, AZ, 1977-81
Vice President, JWK International Corporation, Annandale, VA, 1974-76
Principal, Planning Research Corporation, McLean, VA, 1972-74
Member of the Technical Staff, Lambda Corporation / General Research Corporation, McLean, VA, 1967-72
Senior Operations Research Analyst, Deering Milliken Research Corporation, Spartanburg, SC, 1966-67
Operations Research Analyst, Research Triangle Institute, Research Triangle Park, NC, 1964-66