When starting my graduate school search, the National Research Council rankings (which come out every decade) were a bit outdated. That said, they provided a nice starting point, listing the top twenty-five programs in my field. I then turned to U.S. News and World Report, which also ranks graduate programs (make sure to check when they were last rated, as many fields are not reassessed every year). In that publication, my field had been ranked the previous year and the magazine broke down some detailed rankings by sub-field in my discipline. As I've later learned, the rankings were far from perfect. Any ranking system which heavily weighs peer-perception of other programs can move slowly. Harvard may have been a great program in your field twenty years ago, and it may be that older faculty asked to rate other programs haven't noticed major changes in the department. Upstart programs bringing in exciting young faculty and graduate students may be forced to wait several years before their position is accurately reflected in the rankings.
While program rankings may be slow to catch up to reality - perception of how strong your program is matters in terms of winning extramural funding and applying for jobs once you've completed the program. Brand names of elite institutions may not matter as much as they do for undergraduate programs, but they do matter when a job search committee is trying to justify a hire to a dean. Graduates of Harvard may have an advantage here, even if the strongest faculty in your field are at Michigan State. Deep in the recesses of my soul, I hate this sort of elitism, but it is to your benefit to recognize that it exists across many fields before choosing a graduate program.
In sum, perception matters in terms of program ranking, but it isn't the end-all-be-all for graduate education. Ask the graduate programs to which you are admitted for information about placement rates. Where are people getting jobs? Are people languishing for years on the job market before leaving the profession or the academy? Or are they getting nice jobs right away? Rankings can help you choose where to apply and help you choose where to attend graduate school, but remember they are but one of many factors in choosing a graduate program.
