About this page
GiveWell aims to find the best giving opportunities we can and recommend them to donors. We tend to put a lot of investigation into the organizations we find most promising, and de-prioritize others based on limited information. When we decide not to prioritize an organization, we try to create a brief writeup of our thoughts on that charity because we want to be as transparent as possible about our reasoning. The following write-up should be viewed in this context: it explains why we determined that (for the time being), we won't be prioritizing the organization in question as potential top charity. This write-up should not be taken as a "negative rating" of the charities. Rather, it is our attempt to be as clear as possible about the process by which we came to our top recommendations.A note on this page's publication date
The last time we examined the charities working primarily in the U.S. was in 2010. As of 2011, we have de-prioritized further work on this cause. The content we created in 2010 appears below. This content is likely to be no longer fully accurate, both with respect to what it says about the organization and with respect to what it implies about our own views and positions.- Analyzing and reporting on the child-focused programs included in the Monroe County and City of Rochester budgets.3
- Aiming to increase local government (e.g., Monroe County and the City of Rochester) funding for evidence-based social programs.4
- Publishing transparent criteria for identifying and rating the evidence base of US social programs and applying these criteria to the programs they advocate for.5 The Children's Agenda is explicit about the evidence-ranking it gives to each program mentioned in its action plan6 including that the Coping Power Program,7 one of the main programs The Children's Agenda advocates for, only qualifies as having "2nd tier" evidence. Given that TCA needs to work with local government officials, we understand that TCA must consider factors in addition to the level of evidence, such as the local political context.8
- The Children's Agenda played a role in bringing the Nurse-Family Partnership program to Rochester. According to Carolyn Lee-Davis, Policy Analyst at TCA, Jeff Kaczorowski, Executive Director of TCA, evaluated several programs, chose NFP, advocated for it, and found funding for it.9 We have some reason to believe that TCA's role was more in securing (private) funding than in advocating for the program, which was locally well-known at that time.10
- According to Ms. Lee-Davis, The Children's Agenda faces a funding gap of approximately $60,000 for the coming year. The Children's Agenda is a small organization with only two full-time and two part-time employees.11 At this staffing level, The Children's Agenda reports finding itself unable to adequately prepare and participate in meetings at which it could offer insight into evidence-based programs for the problems the city is trying to address. To that end, The Children's Agenda is aiming to hire one additional full-time policy person in the coming year.12
- The Coping Power Program does not meet the evidence standards we have settled on for this cause. The Children's Agenda concedes this to some degree (as mentioned above, it does not give the program its highest evidence rating), and there is certainly an argument that supporting this program is worthwhile within the local political context. However, with the work we've done to date, we are not ready to assess this sort of argument, and thus not ready to support advocacy for a program whose evidence base does not meet our standards.
- The Children's Agenda also takes part in advocacy for other programs whose evidence base does not meet our standards, such as greater subsidies for child care. Again, while there is an argument that expanding funding for these programs would be net beneficial (and is more politically feasible than funding highly evidence-based programs), we are not at this point ready to assess such an argument.
Sources
- Children's Agenda. Audited financial statements (2009) (PDF).
- Children's Agenda. Children's Agenda table participation (DOC).
- Children's Agenda. Community action plan: Update to the Board of Directors (DOC).
- Children's Agenda. Executive summary of the Children’s Agenda’s analysis of the proposed 2010 Monroe County budget (PDF).
- Children's Agenda. Letter to Attorney General Andrew Cuomo (PDF).
- Children's Agenda. Letter to County Executive Brooks (PDF).
- Children's Agenda. Recent accomplishments (April 2010) (DOC).
- Kaczorowski, Jeff, and Carolyn Lee-Davis. 2009. The 2010 community action plan for Greater Rochester's children: Featuring the 2009 community status report on children and youth (PDF). Rochester: The Children's Agenda.
- Lee-Davis, Carolyn. Policy Analyst. Phone conversation with GiveWell, September 2, 2010.
- Lee-Davis, Carolyn. Policy Analyst. Phone conversation with GiveWell, October 26, 2010.
- Ross, R. Danforth. Member of the Board of Directors of The Children's Agenda; former Director, Child and Family Services, Monroe County Department of Human Services. Phone conversation with GiveWell, September 22, 2010.
- 1. "We have two full-time employees: Michelle Yale (an administrator) and myself. Nancy, who you spoke to, is part-time, and focuses on grantwriting and marketing/communication. Jeff is the part-time executive director." Carolyn Lee-Davis, phone conversation with GiveWell, September 2, 2010. "The Children‘s Agenda is an independent and non-partisan organization that promotes evidence-based programs and policies for the health and well-being of children in Monroe County." Kaczorowski and Lee-Davis. 2009, Pg 3.
- 2. "We’ve been around for 7 years and were originally started by a community foundation." Carolyn Lee-Davis, phone conversation with GiveWell, September 2, 2010.
- 3. "Regarding public dollars, we do an analysis of the county budget and see how dollars are being spent for kids. We started doing that before we were formally incorporated. Three years ago, we did the same thing for the Rochester City budget to see how dollars are spent around kids in the city." Carolyn Lee-Davis, phone conversation with GiveWell, September 2, 2010.
- 4. "The other thing they wanted us to do was watch public dollars around kids and when public money was being spent on kids that they’re being spent as effectively as possible. Our goal was to help encourage policy makers spend dollars wisely.... We look to see if a program has had a randomized-controlled trial (RCT), ideally replicated in another community (in particular, for Rochester, has it been replicated in an urban area like Rochester), if there are outcomes for kids that extend beyond 18 months after the program end, and if the effects are sizeable. Those are the main things we look at." Carolyn Lee-Davis, phone conversation with GiveWell, September 2, 2010.
- 5.
"How is “evidence-based” determined? Please find below one paradigm for considering levels of evidence that indicate that a program or policy is evidence-based.
- Step 1. Review the Science and Research Literature to see if the literature shows positive outcome for this type of program/intervention; check if evaluation to date of this specific program shows outcomes that are positive. Learn from existing evaluations and research; don‘t do things already shown not to work and don‘t reinvent the wheel.
- Step 2. Is there a meaningful outcome? Ideally, a meaningful outcome is an understandable, objective measure of behavior. For example, an important, practical outcome may be decreases school absences, versus scoring 50 points on a new rating scale invented specifically for the evaluation.
- Step 3. Is there ―Peer-reviewed‖ (reviewed by independent experts in the field) research on this particular program/intervention that is published in scientific journals? This is important because an independent review eliminates the developer‘s likely bias towards a positive effect. **A third tier program has research that has been assessed in peer-reviewed journal.
- Step 4. Did the evaluation include a Comparison Group that did not participate in the program/intervention? The Comparison Group should be as similar as possible to the group that does not receive the program. This helps control for trends that can be naturally occurring: some situations improve (or worsen) by themselves. **A second tier program has completed an evaluation with a comparison group, ideally in an randomized controlled trial (RCT).
- Step 5. Evaluation should avoid potential biases in selection nand differences beween the groups that are difficult to see or measure. The ideal is to run a RCT. These are expensive, but show true evidence of effectiveness by comparing those who receive the intervention with those who in the comparison group that are selected by chance. This is the best way to assure the 2 groups being compared are truly equal, and this must be checked at beginning and end.
- Step 6. Is there more than one randomized controlled trial, or one that is conducted at more than one location/site? This indicates the programs ability to be replicated and further strengthens the findings of the evaluation. It minimizes the likelihood of chance in producing the positive outcome(s) and shows that the program works other than under unique scientific circumstances.
- Step 7. Was there a Long Term Follow-up, conducted at least 18 months after the program was completed? Was there a meaningful difference (even if statistically different, it should have practical, ―real-life‖ meaning) in terms of the positive effect(s) demonstrated? This is a real measure of the program/intervention‘s power: it makes a lasting and meaningful difference. An example of the latter is a difference of being absent 1 more day per school year vs. 20 more days per school year even if both were statistically truly significant. **A top tier program has conducted a RCT with at least two sites and has meaningful outcomes, ideally reinforced by a long-term follow-up evaluation. The Continuum Continues: Putting the Paradigm to Use to Find Solutions.... With the above paradigm in mind, we have selected programs which are all either 1st or 2nd tier.... Evidence-based programs for Children and Youth 1st Tier." Kaczorowski and Lee-Davis 2009, Pg 26-27.
- 6. Kaczorowski and Lee-Davis 2009, Pg 27.
- 7. Kaczorowski and Lee-Davis (2009), Pg 21 specifies Coping Power as the solution The Children's Agenda recommends for quality after-school programs.
- 8. "GiveWell: What about the programs on the Social Programs That Work web site (http://evidencebasedprograms.org/) by the Coalition for Evidence-Based Policy, would you advocate for those? The Children’s Agenda: It would be great to have the programs on that site available to the children in a community, but it’s complicated, for example, you can't just pick the Carrera adolescent pregnancy prevention program and bring it here. With the NFP program, no one wants to see kids abused, so it is easy to talk about. But there are differing moral views on the Carrera program." Carolyn Lee-Davis, phone conversation with GiveWell, October 26, 2010.
- 9. "The board who put us together said, 'We want you to keep analyzing the public budget but also do something about child abuse and neglect.' And, as Jeff [Kaczorowski] looked into what he could do, he went and read tons of research to see all the things you can do to attack this problem. He did all the legwork and said, here’s a program that without a doubt is going to reduce it. He did the selling of the idea by using the evidence of effectiveness. He did the original private fundraising for it. He got local business leaders and foundations to put in money for it and did work to work with county to find a funding stream. He worked with county to write an application to secure state money." Carolyn Lee-Davis, phone conversation with GiveWell, September 2, 2010.
- 10. "When the Children’s Agenda first approached the department of social services, I was the Planning Coordinator for the department. I met with the DSS Service Director and representatives from the Children’s Agenda about the possibility of funding Nurse-Family Partnership. There was funding available from the state for what they call Community Optional Services, which required coordination with the DSS which had to sponsor the program and request approval of it from the New York State Office of Children and Family Services. This fundng stream requires a 35% match from local funding (i.e., the county Department of Social Services had to provide 35% of the funds, and the state would match). For preventive services the match can be through private local donations, as opposed to tax dollars, but those funds have to come to the county without restrictions on what specific program they're going to be used for, although it's perfectly okay for there to be an understanding of what the funds are likely to be used for. I checked with a number of state people and found out that we could in fact do that, and I'd known about the NFP for quite a while (it's hard not to if you're a human service person in Rochester, because NFP was originally a University of Rochester Medical School project). Technically the funds from private donations could only account for half of the local contribution. What took people a while to realize was that you could effectively fund the program with no local tax dollars as long as you did your homework because while half of the local share of the total preventive services program had to be tax levy that applied to the total program not to particular projects, the county already had a large preventive services program with the local share funded through tax levy dollars so we could expand and develop new programs by using donated funds as the local match. We were probably the first group in the state to read the regulations and figure out what they really were and figure out that we could do that. NFP had never been implemented locally, so then it was a matter of doing the application to New York State, and I walked the NFP folks through the application process because what they needed to do was to request a number of fairly specific regulatory waivers that required specific language and I happen to be good at that kind of language. I worked with our county budget folks to make sure they knew that there would be no local cost other than some minor administrative costs. Then we had to do a legislative referral to the county legislator to appropriate the funds. Once the program was approved by the State and the County Legislature, implementation was the responsibility of NFP and we were fairly “hands off.” What made this all possible was when Jeff said, essentially, 'I can raise the local funds if you can walk us through the rest of the process.' I was skeptical, but he came back with a commitment from a local businessman for 1/3 the cost of the program and from there it was mostly a matter of a number of technical hurdles that we had to overcome." R. Danforth Ross, phone conversation with GiveWell, September 22, 2010.
- 11. "We have two full-time employees: Michelle Yale (an administrator) and myself. Nancy, who you spoke to, is part-time, and focuses on grantwriting and marketing/communication. Jeff is the part-time executive director." Carolyn Lee-Davis, phone conversation with GiveWell, September 2, 2010.
- 12. "GiveWell: Do you need additional money to maintain your current operations? What would you do with additional money? Could you effectively utilize it? The Children’s Agenda: We would like to grow a little bit. We hoped to hire additional staff and we’re not at a point where we can do that yet. The areas that we put in that action plan are the things we’re going to work on for the next few years: expand enrollment in NFP and find more funding streams to do it; expand enrollment in quality early childhood programs, and doing a lot of policy work around trying to improve where the kids are at right now. I think regarding NFP, we’ll invest a lot of time but given our track record on it, I think we’ll be successful in continuing to push the community forward on it. When we wanted to grow this year, we wanted to add one more policy person. We have high levels of youth violence, and that’s an area where an additional voice is needed in the community because we have a long history of not doing things around youth violence. We need a voice to bring together everyone: judges, cops – but we don’t have the capacity to take it on. Because Jeff and I keep going to meetings and we try to be involved but we don’t really have the time, and we were hoping to add someone to do that. GiveWell: So meetings are happening and you can’t attend them? The Children’s Agenda: To be honest, we do OK at showing up. But, you need to be effective at meetings, and you need to be prepared with how am I going to bring this idea up that half the people in the room won’t like. We have to think about how to convince people of the idea and bringing the idea to the table so that they’ll accept it. We want to be at those tables and want to be effective and we don’t want to keep showing up without being effective. And, we need another staff person to help with that." Carolyn Lee-Davis, phone conversation with GiveWell, September, 2010. "GiveWell: What is your room for more funding and what would you do at different levels of more funding? The Children’s Agenda: We want to add an additional full-time policy analyst and a part-time development person. Our board approved that plan in January 2010, but we held off because we didn't have the funding. We just found out that we’ve received a grant that is specifically to be used for a part-time development person, so we're going to hire such a person. But we don't have the funding to help with additional policy work, so the next thing we would spend additional funding on is a full-time policy analyst. The main goals of having an additional policy analyst are be to work in additional areas where we only can have limited capacity now, for example, we'd like to participate a lot more in areas such as youth violence. An additional person would also help fortify what we're doing in different areas. We're more of a local organization in the city and county, but for the last two years, we've been working more with statewide groups, and an additional policy analyst would help us do more work at the state level as well. It would be a junior person and we've budgeted $50,000–$60,000 per year for such an employee (a $42,000 per year salary plus benefits) GiveWell: What would you do if you got less, say $30,000? The Children’s Agenda: We would check if it were viable to hire a policy analyst on a less than full-time basis. There's a good chance it'd be viable, and even a part-time person would be very beneficial. GiveWell: And if you got more, say $140,000? The Children’s Agenda: This would probably be a good question to ask Jeff [Executive Director, The Children’s Agenda]. I think we wouldn't increase our staffing beyond what we've suggested at this time. We've been small, staying at 1.5 staff for about 3 years and then 2.5 staff for another 4 years. We're growing slowly, and that suits us. We try to exist with about 6 months cash reserves, but now we’re working on slightly less than that. A part of the decision what to do with that amount of additional funding would be to look at what made sense in terms of securing reserves and growth." Carolyn Lee-Davis, phone conversation with GiveWell, October 26, 2010.