Evaluating Counseling Programs

By: Patrick O'Connor Ph.D

The last twenty years of education reform indicates our gut feelings about our work is no longer enough to satisfy bosses, policymakers, and funding sources.  In addition to knowing we make a difference, we have to show we make a difference—and that requires data.

The good news is that we come to this challenge well prepared.  Almost every school counselor has had a course in statistics, and while t tests and z scores may be a blur, three key ideas from Statistics 101 are crystal clear: There is such a thing as bad data; not all data involves numbers, and statistics can be skewed to tell just one side of the story. Keeping these three things in mind, we can pursue client-centered data knowing what questions we want to answer, and how to go about answering them.

There is such a thing as bad data, and many school counseling programs use poor data simply to keep their bosses happy.  A classic example is a counseling office that gives the principal the total number of student visits to the counseling office in a year.  This is helpful at a basic level—10 total visits in a year clearly suggests something’s wrong—but a richer approach can be had asking two additional questions: Which services was the student seeking in that particular visit, and had the student been in the counseling office before?  This information can strengthen the numbers the administration wants (“450 of our 525 students visited the counseling center this year”) and gives counselors some idea of popular and underused programs (“Only 40 of those visits were related to career counseling, and those were made by the same 14 students.”) Suddenly, this is data that has a richer purpose.

Not all data involves numbers  Another source of bad data is used to evaluate the strengths of their college counseling services.  Known as “the list”, counselors devote all kinds of time tracking students down to find out where they were accepted to go to college, and where they plan to attend.

These numbers may soothe the principal and the public (“See how many students are going to State U!”), but they don’t really tell us if the students feel the counseling office helped them make a good college decision. A quick look at a college counseling survey at http://www.surveymonkey.com/s.aspx?sm=ONl3pPcBpZcC7OpTFFUH_2fQ_3d_3d shows a different approach to evaluation.  Not only will the answers provide ideas to improve counseling services; it gives respondents the opportunity to provide data in writing.  It’s easy to take these comments and create a sense of the majority, while the responses show greater depth than a scale of 1 to 10. (And if you’re on a budget—you can create your own 10 question survey for free at www.surveymonkey.com)

Data can be skewed  Stats class showed us examples of surveys that didn’t tell the whole story, and we can’t let that happen with us. If you haven’t surveyed students and parents before, your first attempt should be in the fall, when you can measure current attitudes towards counseling services.  This data can offer ways to tweak counseling services during the school year; this way, the data for the end of the year can indicate progress in meeting students’ needs, even if there is room for growth in meeting all needs.

Counselors effectively soothe student fears about testing all the time—isn’t it time we overcame our profession’s anxieties over data driven evaluation?  It’s one of the best student-centered ways to keep count of the services that count most. 

The Three Big Picture Items Counselors Need to Focus On

By:  Patrick O'Connor Ph.D

Spring is a busy time for school counselors.  Between state mandated testing, scheduling for next year, awards presentations, college decisions, and professional conferences, we might as well bring a sleeping bag to the office and request a meal voucher from the budget we don’t have.

If you can’t get to a conference this year, here’s a summary of the three big ideas that seem to be featured in counseling workshops across the country:

No one knows what counselors do, but they think they do. Everything from student satisfaction surveys to media portrayals of counselors suggest there is a huge misunderstanding of what counselors do, what they’re supposed to do, and how well they do it. It would be discouraging enough if this was limited to the students and families we serve, but this lack of awareness impacts our relationships with classroom teachers (who often see us as “the schedule changers who have their own office”) and our principals (who often see us as an extra administrator).

This confusion may not be new news, but the communication gap is taking on new importance.  As budgets get tighter and demand for higher test scores increases, it’s easier for administrators to reduce or eliminate counseling positions if there is no clear understanding or evidence of the difference counselors make in schools or in the lives of students.  This may not be a fair perception, but it exists.

It’s up to us to tell them.  Counselors I speak to eagerly agree that our work is misunderstood—but when I suggest it’s really up to us to fill those void, the agreement turns to frustration.  This is completely understandable; counselors come to work early, stay late, and rarely stop for lunch.  Add on evening programs and summer committee work, and it’s easy to relate to the resistance to the idea that, for as hard as we work, we have to do more.

But we do.  An education analyst I spoke with said it wouldn’t surprise him if school administrators aren’t already talking about distributing counseling services to social workers (for personal issues) and paraprofessionals (for scheduling and college/career issues).  In the absence of any protest or clarification from counselors, it’s easy to see why policymakers might view this as an economical alternative.  It’s also easy for counselors to see why this could lead to a loss of student support- but that’s the part others can’t see. 

We need to tell them in a way that has meaning to them.  Every counselor knows effective communication occurs when the speaker shares ideas in ways that make sense to the listener. Proving the case for school counselors is no different: if administrators understand data, we express our worth in facts and figures; if parents don’t know what we do all day, we share the results of a time-on-task analysis; if a school board wonders if we make a difference, we present all of this, along with videotapes of students willing to share their success stories.

If a client came to us and said they were suffering because someone else misunderstood them, we’d gently inform the client that they have to change.  As we consider the future of school counseling, it’s time for us to follow our own advice—reach out, seek to understand, and speak in a language others relate to.  We know we make a difference in the lives of our students, but the time has come for others to not just know that, but understand that.  In the interest of service to students, counselors need to reach out and offer that understanding, now.

How to Make the Most out of a College Fair

By: Patrick O'Connor Ph.D

More and more high schools are offering spring college fairs to help juniors and their families focus on the qualities they’re looking for in the right college match.  There’s nothing to replace a campus visit, but college visits cost time and money, and you’ll need to make the most of both junior year.  College fairs help you do that— held in fall and spring, a fair can have representatives from up to 400 colleges, all eager to talk to you about their college and your life.  Many fairs feature information on choosing and applying to college and financial aid, and most fairs are free.

With so many colleges at a fair, it’s easy to get intimidated—so plan ahead.  Take a pen, a highlighter, an unofficial copy of your transcript, and 5 questions committed to memory that will help you learn more about a college.  What you ask is up to you—majors, food, chances for research, cost, social life-- just make sure the answers will help you decide if this place is worth a closer look.

At the fair, get a map of where the booths of the colleges are located.  BEFORE you go onto the floor, highlight the colleges you’re interested in (this same list might be on a Web site—even better, since you can research colleges ahead of time.)  Once you’re at a booth, you might have to wait to ask questions—this is good!  Use this time to listen to what the representative is saying to other students-- since they will most likely be discussing general questions, you can use your time to ask more detailed stuff.

Once it’s your turn, get busy.  “Hi, my name is (NO student does this, but you should; it shows confidence, and gives the rep the chance to remember you) and I go to Captain Jack High School.”  From here, you want to ask your questions; make eye contact as they answer, and don’t rush them. 

If you feel you’re hitting a good vibe, pull out your transcript and say “Just one more question.  I’m putting my senior schedule together.  Here’s what I’ve taken so far; what other courses would your college like to see me take?”  ABSOLUTELY, POSITIVELY NOBODY does this at a college fair, which is why you should.  Most of the time, you’ll actually get some great advice (or even a scholarship offer), but don’t be surprised if they don’t know what to say—either way, you’ll be remembered by reps in a very positive way.  Thank them for their time, fill out a registration card (that’s important), tell them you hope they come by your school to visit, and move on. 

Make quick notes on this college *before* you visit the next booth.  You can use your “waiting time” at the next booth to do this, but write at least something down—you don’t want to confuse your colleges.

If you can do about 7-10 colleges and spend time at an information session of interest to you, call it a victory with an after-fair pizza (this is why you bring your parents along—to pay!)  You now have some solid information on which colleges are road trip worthy, and some solid information about yourself as well—truly a dynamic duo.

One of the many college fair options is a series of national college fairs operated by the National Association for College Admission Counseling.  To find a list of NACAC fairs, visit http://www.nacacnet.org/college-fairs/students-parents/Pages/default.aspx

A College Access Study to Pay Attention To

By Patrick O'Connor Ph.D

Have you ever wondered if you’ve made a difference in the college plans of a student?  If so, there’s some good news in this week’s New York Times—but you have to dig a little for it.

David Leonhardt reported on a study where highly able students from low-income areas were given information on some highly selective colleges.  The report states 54 percent of the students who received this information were admitted to one of the highly selective colleges described in the information  This compares to a 30 percent admission rate for another group of highly able, low-income students who received no such information.

The results suggest that the work counselors do in individual meetings, group presentations, bulletin boards, and web sites gives students and parents the information they need to broaden their view of what’s possible after high school.  When you can double the number of students admitted to top colleges, the work you do for all students is affirmed and confirmed, and that is welcome news.

At the same time, a key part of the study’s design gives counselors something to consider.  The college information the students received didn’t come from a school counselor; it was mailed to their homes by College Board.  Since the students were randomly selected, it’s safe to conclude the only significant difference between the students in the two groups were the mailings they received, not the use or availability of a counselor (for more information, see http://www.nytimes.com/2013/03/31/opinion/sunday/a-simple-way-to-send-poor-kids-to-top-colleges.html?partner=rss&emc=rss&_r=2&).

It’s easy to see how budget-minded school boards might use this information to reduce the number of school counselors or the scope of our work.  The structure of the study makes it far too easy for a casual reader to conclude “Well, if poor kids can get into Harvard just by reading a brochure, why can’t our students apply to college themselves?”

Given the impressive results of this study, counselors would be wise to read the study closely and have a response ready for counseling critics:

·         --Could the 54 percent admission rate increase even more if students had access to a trained school counselor?

·        -- The study doesn’t address how students felt about the application process.  Did the information make applying easy, or would it have helped if a counselor was able to offer advice on which extracurriculars to list, how the essays should be structured, and which teachers to approach for letters of recommendation?

·         --Was it easy for the student to determine which of the highly selective schools would best meet their needs and interests?  While the graduation rates of all highly selective schools are high, there are ample differences between, say, MIT and Brown—differences a student should consider to avoid becoming one of the few students who doesn’t complete a degree.

·       --  Highly able students have long had a reputation for self-selecting their college with success.  What modifications to this study would increase the admission rates of all students to all colleges, and what role would a counselor play in those modifications?

This study clearly rebukes the idea that low-income students don’t want to attend highly selective colleges, and the results of the study offer counselors tremendous ideas on how to help all low-income students improve college access and completion.  At the same time, not every student will earn an A in Algebra 2 just by reading the book; given the high number of students wanting to attend colleges that admit a low percentage of applicants, trained counselors are needed more than ever before to help students discover college options and make strong, successful college choices.