Evidence on Effectiveness of Quality Rating Systems

NO EVIDENCE THAT QRSs IMPROVE PROGRAM OR CHILD OUTCOMES

“The design does not permit us to determine if Parent Aware causes outcomes for programs, parents, or children. We can look at patterns of associations, but causation cannot be determined.” (Parent Aware Third Year Review, MELF, November 2010, PowerPoint, p. 9)

“Program ratings are not evenly distributed across the 4 star levels, and sample sizes are small. This limits the conclusions that can be drawn about the Rating Tool.” (Ibid)

“No definitive patterns of linkages between rating categories, program characteristics or proxy scores developed from director and teacher surveys, and child outcomes were identified.” – (MELF Yr. 3 Report, 11/10, p.132)

“Despite their growing popularity, there is little information available about how well QRISs work. A logic model presented in this report posits a clear path to improved provider quality and better child outcomes, but it is largely untested. We do not know how well QRISs measure what they purport to measure, whether parents pay attention to ratings in selecting care, whether providers that participate in QRISs actually improve the quality of the care they provide, or whether children benefit from the improved care they are receiving as their provider receives quality-improvement support.” (Zellman, et al, Assessing the Validity of the Qualistar Early Learning Quality Rating and Improvement System as a Tool for Improving Child-Care Quality, Rand Corporation, 2008)

We found few relationships between individual Q-QRIS components and child outcomes and virtually none between star ratings and child outcomes. As with the process/quality correlations, the results were not replicated across waves. (ibid, emphasis in original)

Bryant concluded her presentation by reiterating that we should not overstate the strength of association between our measures of quality and child outcomes. The relationships are found consistently but are often small. (Roundtable on Measuring Quality in Early Childhood and School-Age Settings: At the Junction of Research, Policy and Practice, Meeting Summary, Washington, D.C., December 4-6, 2006, Child Trends, emphasis added)

“Small but meaningful differences in the structure and design of QRSs and the lack of research on the implications of these differences make it difficult to synthesize lessons learned across programs.” (Tout, et al, Issues for the Next Decade of Quality Rating and Improvement Systems, Child Trends, May, 2009, emphasis added)

“At least three QRSs are including an examination of child outcomes in their current evaluations. Indiana, Minnesota, and Missouri each include an examination of fall-to-spring change in levels on particular child outcomes participating in QRS programs…These designs allow the evaluators to determine if higher rating levels are associated with more positive child development, not to assess the impact of QRSs on children.” (Ibid, p. 6, emphasis added)

IMPLEMENTATION PROBLEMS

“The overall participation rate of eligible programs is 14%, with higher participation rates found (between 30 and 45%, depending on the pilot area) for center-based programs.” (MELF, Year 3 Evaluation, p. 130, emphasis added) [The program does not seem to be very popular, especially with family childcare homes.]

“Two-thirds of programs received automatic 4-star ratings.” (Ibid, p.130) [2/3 of the programs are getting a free pass].

• 147 accredited child care centers and family child care programs
• 23  Head Start programs [[Head Start has been shown yet again to not work with fade out of beneficial effects by the first grade and harm to the math skills of 3 year old children.]
• 53 School Readiness programs

“Of the programs that received full Parent Aware ratings, two-thirds received 3- or 4-stars, and one-third received 2-stars (29%) or 1-star (4%).” (Ibid, emphasis added) [This begs the question of what a rating really means]

“During the pilot, it was noted by the Parent Aware Implementation Team and by some providers that the rules for rating shifted over time.”  (Ibid, p. 137, emphasis added)

RESTRICTING ACCESS AND IMPOSING POLITICALLY CORRECT STANDARDS, SUBJECTIVE ASSESSMENTS

The Parent Aware quality rating system uses only state approved curriculum and assessments.  This includes the Work Sampling System, which is also called the Kindergarten Readiness Assessment (p.17) used by proponents to falsely say that 50% of children in Minnesota are not ready for kindergarten.  The assessments contain very subjective items, such as:

• “Shows beginning understanding of numbers and quantity”
• “Shows an appreciation for books and reading
• “Shows empathy and caring for others”
• “Responds to artistic creations or events”

Quotes from Improper Special Interest Influence in Key Contracts: An Analysis with Preliminary Observations on the Politicized Agenda in Child Day Care, Mark Kindt, 1996:

• Issuance of complex regulations for day care licensure “under the disingenuous guise of protecting the health and safety of children but which in reality are calculated to restrict entry, limit competition, reduce access, limit parental choice, and increase cost”
• “Development of a politicized core curriculum for day care providers (a so-called national standard) to be used to form the minds of children with a radical ideology before they enter public schools;”
• “Further restricting the free choice of parents who receive government subsidies by limiting the access of their children to affordable quality day care by promoting various requirements that discriminate against religious providers of day care.”
• “Development of a college curriculum designed to turn out ‘politically correct’ day care professionals
• “Accreditation of day care centers based on their adoption of the approved curriculum and membership in affiliated organizations”
• The national-to-local model for the child care services presents significant public policy issues relating to questionable political content in day care curriculum and to substantial questions of public accountability for so-called public-private partnerships.
• Most citizens would recognize the anti-bias curriculum as a highly politicized curriculum which seeks to impose a particular ideological world-view upon children. Most taxpayers would simply be astounded that tax dollars are routinely being spent toward the state-by-state implementation of these apparently politicized standards.