It's been a long time since we arrived in Ghana, and all of our good intentions about posting regularly have dissolved! While accessing the internet is fairly inconvenient, I think the real reason for my lack of posts is that I keep feeling that I’m about to get really into the work I’m doing, but for one reason or another continue to face delays and interruptions.
A couple of weeks ago, following the NYU policy evaluation course, I returned to work on the interest rate sensitivity study that IPA (the same organization that Anna is working for this summer) is conducting in and around Accra. IPA is working with an international savings and loans organization, Opportunity International, to do a special loan promotion which has been designed as a randomized trial of various interest rates and marketing techniques. Needless to say, all our international economic development classes are coming in handy. The rationale is to discover what rates appeal to which types of business owner, thereby helping us to better understand the choices and constraints of small to medium sized businesses.
It was really helpful to take the policy evaluation course after having a couple of weeks of hands-on survey management, and I definitely felt like I could connect some of the best practices and theory that we had heard about in the class to the realities on the ground here in Accra. My role, which I share with another graduate student intern (from Duke), is to conduct monitoring and auditing of the survey / promotion as it continues. This involves going into the field and conducting some of the same interviews that the individual loan officers (ILOs) completed, as well as checking their data entry and map-making. It’s become very clear to us that this auditing is very much necessary, as we have already uncovered a range of mild to serious problems. In some ways, the process of discovering this has been very complementary to the class content; we’re learning all the ways in which a very well designed survey can run into problems in implementation.
This week we are back on track, after spending last week training the ILOs, both new and experienced, on how to conduct the study in the field and the branch. The challenge now is to set up some systems for ongoing monitoring and evaluation as the study continues for the next 6 months or so. Fortunately we have a large pool of excel-hungry public administration students here to help us do it!
One of the most unexpected “learning outcomes” for me so far has been around the role that I see for myself in evaluations like these. While I am fascinated by the theory and by the survey design, it’s become clear to me that I’m most effective in managing the organizational support and relationships necessary for these studies to be successfully conducted. In other words, I’ve come back to the practice of public administration itself via evaluation. Now, I’m wondering how many studies have failed because of organizational or logistical issues, or simply because economists sometimes have difficulty conveying the real-world importance of these types of evaluations. “Survey management” means, or should mean, a lot more than just survey design and hiring the right people. Seeing this first hand, I’m even more encouraged by the efforts of Banerjee and others to argue that evaluations should be considered as stand-alone projects, rather than as an afterthought to the “real work” of an organization.
Hi Martha!
I laughed out loud when I read your blog. I too have realized that I like the project managment part of evaluation. Seeing how vulnerable surveys are to error and mis-managment I have spent quite a lot of time thinking about the validity of results. Also I see limitations in the popular desire to gather as much quantitative data as possible. So much is missed if the right questions are not asked. (Lastly, I realize that I am a do-er and like the people part of the project) Cant wait to chat about EVERYTHING! Sounds like things good. -Jess
Posted by: Jessica | July 18, 2007 at 05:33 AM