Providing effective evaluation of economic, social and public health programs has become an increasingly important requirement for both public and private institutions. Program evaluation allows us to assess the effectiveness of a specific program or intervention, and to use the subsequent results to identify how we can improve the future implementation of the program. As such program evaluation can be an extremely valuable tool for managers and evaluators looking to improve the quality of their programs and improve participation outcomes for the individuals involved. TStat’s residential summer school offers participants a unique opportunity to acquire the requisite toolset, both theoretical and applied, for the correct implementation of effective modern micro-econometric methods for implementing evidence-based program evaluation using Stata. As such, the program has been developed to encompass both: standard statistical methods of program evaluation: regression-adjustment, matching, selection-models and difference-in-differences methodologies; and the more advanced econometric techniques: for example, instrumental variables, synthetic control method, and regression discontinuity design.
The School opens with an optional introductory one data introduction to the statistical software Stata module to enable participants unfamiliar with the statistical software Stata to acquire the necessary introductory toolset to enable them to fully participant in the applied sessions during the course of the week.
In contrast to previous editions of the school, the 2020 edition includes a supplementary evening Case Study Group session during which participants have the opportunity to present and discuss their own research agendas related to applied program evaluation. Course leaders will be available to discuss with participants the appropriates of the methodologies adopted in their case study, the interpretation of the results obtained and also to indicate potential problems to be aware of given the characteristics of the underlying data, as well as providing feedback and guidance on possible future developments of individual research agendas. The school closes with a Q & A style session entirely dedicated to real world program evaluation case-studies, where the participants will have the opportunity to actively engage with the course tutor in a current program evaluation case study.
In common with TStat’s training philosophy, throughout the course of the week theoretical sessions are reinforced by both case study examples, in which the course tutors discuss current research issues, highlight the potential pitfalls and advantages of individual techniques, and an applied (hands-on) segment during which participants have the opportunity to implement these newly acquired techniques using real data under the watchful eye of the course tutor. Specific attention is given to both the intuition behind the choice and implementation of a specific technique, given the analysis in hand is of the utmost importance. In this manner, the course leader is able to bridge the “often difficult” gap between abstract theoretical methodologies, and the practical issues one encounters when trying to implement such techniques on real data.
At the end of the school participants are expected to be able to master complex evaluation design by: identifying the type of data required in their specific policy framework; evaluating which specific econometric method is more appropriate for the analysis in hand; and finally extracting policy recommendations from the obtained results. Participants should leave the course being in a position to autonomously implement, with the aid of the Stata routines utilized during the sessions, the techniques discussed during the course of the school.
Managers, researchers and professionals working in public and private institutions needing to undertake econometric program evaluation analysis using micro data. Although these methodologies are commonly used to evaluate economic policy interventions in, for example, the labour market, investment activities of enterprises, education policy, regional development, etc., they can in fact be used across a variety of studies, such as Public Health sector or interventions in the Bank sector or Financial markets, which aim to estimate the ex-post impact of a given intervention or project on specific targets.
Participants are expected to have a working knowledge of Stata and an intermediate knowledge of econometrics, and in particular knowledge of Ordinary Least Squares, Logit and Probit regression models and instrumental variable analysis.
MODULE A | DAY 1 – STATA IN JUST ONE DAY!
SESSION I: INTRODUCTION – GETTING STARTED
File types in Stata
Working interactively in Stata
Saving output: the log file
Loading Stata databases
The Log Output File
Saving databases in Stata
Exiting the software
SESSION II: PRELIMINARY DATA ANALYSIS
A preliminary look at the data: describe, summarize commands
Abbreviations in Stata
Statistical Tables: table, tabstat and tabulate commands
SESSION III: DATA MANAGEMENT
Selecting or eliminating variables
The count command
Creating sub-groups: the prefix by
Creating new variables: generate
Operators in Stata
The command assert
program evaluation in action
Missing values in Stata
Modifying variables: replace, recode
Creating Labels: variable labels and value labels
Creating dummy variables
SESSION IV: IMPORTING DATA FROM SPREADSHEETS
Import Excel and Export Excel commands
The insheet and outsheet commands
Reading in Text Data Files
Issues to watch out for when importing data
Redefining missing values
dealing wih “messy” strings
SESSION V: GRAPHICS – A BRIEF INTRODUCTION
Stata’s syntax for two way graphs
Saving and exporting graphs
Useful graph commands
Personalizing a graph
Stata’s Graph Editor
Merging data bases
APPENDIX B: MORE ADVANCES ISSUE (time permitting)
Merging data bases
e-class and r-class variables
MODULE B | DAY 2 – THE BASICS OF EVIDENCE-BASED PROGRAM EVALUATION
SESSION I: AN INTRODUCTION TO PROGRAM EVALUATION
Mastering evidence-based program evaluation: an overview
The concept of counterfactual causality
Experimental and quasi-experimental settings
The selection bias problem
Selection on observables and selection on unobservables
Definition of treatment effects: types of effects and potential outcome
Notation and working assumptions
Available econometric methods: limits and advantages
Stata for effective program evaluation: user-written commands and the teffects package
SESSION II: WORKING UNDER OBSERVABLE SELECTION USING REGRESSION ADJUSTMENT
Program evaluation under observable selection
The control-function regression approach
Linear and non-linear Regression-Adjustment (RA)
Stata implementation with the commands teffects ra and ivtreatreg
DAY 3 – GETTING A BIT MORE MODEL-FREE VIA MATCHING AND REWEIGHTING
SESSION I: MASTERING THE MATCHING APPROACH
Why the Matching Approach can be superior to the Regression Adjustment?
The rationale and usefulness of the Matching method
The propensity score: definition, properties, usefulness
Matching identification of treatment effects
Matching on the covariates versus Matching on the propensity score
Matching in practice: how to run tests and robustness checks
Implementation in Stata
SESSION II: EMPOWERING YOUR SKILLS BY REWEIGHTING
From Matching to Reweighting: a short but instructive journey
Reweighting as a general approach to program effect’s evaluation
Reweighting on the propensity score
Advantages and limits of using a Reweighting procedure
Implementation in Stata
DAY 4 – RELAXING THE ASSUMPTION OF OBSERVABLE SELECTION USING INSTRUMENTAL-VARIABLES (IV) AND SELECTION MODELS
SESSION I: THE “WHY” AND THE “HOW” OF INSTRUMENTAL-VARIABLES
The causal logic of instrumental variables: definition and simple examples
Treatment endogeneity and consistent estimation of program effects
Types of IV methods
Implementation in Stata
SESSION II: DIGGING INTO THE IV APPROACH WITH THE LOCAL AVERAGE TREATMENT EFFECT
The Local Average Treatment Effect (LATE) setting: an illustrative example
The relation between LATE and IV
A primer into Regression Discontinuity Design (RD)
SESSION III: ALTERNATIVES TO IV AND THE SELECTION MODELS
Dealing with unobservable selection when an IV is not available
The Heckman selection model and its generalization
Implementation in Stata
DAY 5 – DIFFERENCE-IN-DIFFERENCE
SESSION I: UNDERSTANDING AND MASTERING THE DIFFERENCE-IN-DIFFERENCES
Difference-in-differences (DID) program evaluation setting
DID with longitudinal data
DID with repeated cross-section
The analysis of pre- and post-treatment effects
Implementation in Stata
SESSION II: SYNTHETIC CONTROL METHOD
The Synthetic Control Method (SCM): setting
Parametric and nonparametric SCM with Stata implementation
The placebo test and its graphing
Stata applications on real datasets
WINE MEETS PROGRAM EVALUATION: EVENING SUPPLEMENTARY WORKGROUP
During this informal early evening gathering, participants are given the opportunity to come together to discuss, via a poster or short presentation, their own research and/or program evaluation projects, whilst enjoying an offering of Tuscan Tapas and a glass of Tuscan wine.
DAY 6 – PROGRAM EVALUATION IN PRACTICE
SESSION I: A TEMPLATE FOR AN EFFECTIVE EVIDENCE-BASED PROGRAM EVALUATION
Designing an ex-post program evaluation: an overview
Description of the main steps of analysis from data collection to reporting
The use of the results for policy making
Reporting and communicating program evaluation results
SESSION II: PROGRAM EVALUATION CASE-STUDY
Implementing evidence-based program evaluation techniques: a case-study
The potential and limitation of program evaluation: a discussion
Summing-up and concluding remarks
PARTICIPANTS’ Q & A SLOT
In this concluding session participants are offered the opportunity to delve deeper into specific arguments of interest, when we open up the floor to questions and discussions of specific aspects of program evaluation relevant to their research work.
A Gentle Introduction to Stata, 6th Ed., Alan Acock (2018) Stata Press
Data Analysis Using Stata, 3rd Ed., Ulrich Kohler, Frauke Kreuter (2012) Stata Press
Data Management Using Stata: A Practical Handbook, Michael N. Mitchell, (2010) Stata Press
The Workflow of Data Analysis Using Stata, J. Scott Long (2009) Stata Press
Mostly Harmless Econometrics: An Empiricist’s Companion, Joshua D. Angrist e Jorn-Steffen Pischke (2008) Princeton University Press
Microeconometrics Using Stata, Colin Cameron and Pravin K. Trivedi (2010) Stata Press
Econometric evaluation of socio-economic programs: theory and applications, Giovanni Cerulli (2015) Springer Verlag
Dr. Una-Louise BELL, TStat Training | TStat S.r.l.
Dr. Giovanni CERULLI, National Research Council of Italy
Dr. Roberto GABRIELE, University of Trento
ENTIRE WEEK (MODULES A plus B, 6 days)
Full-Time Students*: € 1620.00
Academic: € 2640.00
Commercial: € 3900.00
MODULE B (5 days)
Full-Time Students*: € 1350.00
Academic: € 2200.00
Commercial: € 3250.00
*To be eligible for student prices, participants must provide proof of their full-time student status for the current academic year. Residential costs for full time students are completely sponsored by TStat Training through our Investing in Young Researchers Programme. Participation is however restricted to a maximum of 3 students.
Fees are subject to VAT (applied at the current Italian rate of 22%). Under current EU fiscal regulations, VAT will not however applied to companies, Institutions or Universities providing a valid tax registration number.
Please note that a non-refundable deposit of €100.00 for full-time students and €250.00 for Academic and Commercial participants, is required to secure a place and is payable upon registration. The number of participants is limited to 15. Places will be allocated on a first come, first serve basis.
Course fees cover: i) teaching materials (copies of lecture slides, databases and Stata routines used during the summer school; ii) a temporary licence of Stata valid for 30 days from the day before the beginning of the school; iii) half board accommodation (breakfast, lunch and coffee breaks) in a single room at the CISL Studium Centre or equivalent (6 nights for entire school, 5 nights for Modules B). Participants requiring accommodation the night of the final day of the school, are requested to contact us as soon as possible.
To maximize the usefulness of this summer school, we strongly recommend that participants bring their own laptops with them, to enable them to actively participate in the empirical sessions.
Individuals interested in attending this summer school must return their completed registration forms by email (email@example.com) to TStat by the 1st August 2020.
TStat’s residential summer school offers participants a unique opportunity to acquire the requisite toolset, both theoretical and applied, for the correct implementation of effective modern micro-econometric methods for implementing evidence-based program evaluation using Stata. As such, the program has been developed to encompass both: standard statistical methods of program evaluation: regression-adjustment, matching, selection-models and difference-in-differences methodologies; and the more advanced econometric techniques: for example, instrumental variables, synthetic control method, and regression discontinuity design.