Enrollment Analytics: Using Data Modeling to Improve Student Recruitment, Retention, Budgeting, and Planning:

Changing Higher Ed podcast 276 with host Dr. Drumm McNaughton and guest Dr. Emily Chase Coleman

Transcript: Changing Higher Ed podcast 276 with host Dr. Drumm McNaughton and guest Dr. Emily Chase Coleman | Enrollment Analytics: Using Data Modeling to Improve Student Recruitment, Retention, Budgeting, and Planning

Introduction to Changing Higher Ed^®

Welcome to Changing Higher Ed^®, a podcast dedicated to helping higher education leaders improve their institutions. With your host, Dr. Drumm McNaughton, CEO of The Change Leader, a consultancy that helps higher ed leaders holistically transform their institutions. Learn more at changinghighered.com. And now, here’s your host, Drumm McNaughton.

[00:00:20] Podcast Focus and Guest Background

Drumm McNaughton: Thank you, David. My guest today is Emily Chase Coleman, co-founder and CEO of HAI Analytics, Inc. Emily and HAI empower colleges and universities to make data-informed decisions around enrollment, financial aid, and student success. She has over 20 years of experience and a PhD in social psychology and statistics from Cornell, and she bridges the gap between data science and institutional strategy.

For decades, enrollment strategies have been driven by institutional traditions, gut instinct, and anecdotal evidence. Those approaches no longer work. Emily argues that database decision-making is now a competitive necessity, and she joins me today to talk about how embracing real-time analytics and predictive modeling enrollment leaders can make evidence-based choices that enhance student recruitment, improve retention, and drive institutional growth.

Emily, welcome to the program.

Emily Coleman: Thanks so much for having me, Drumm.

Drumm McNaughton: It’s my pleasure. You are the third program that we’ve had in a row talking about enrollment.

Emily Coleman: That’s great.

Drumm McNaughton: We’ve talked about enrollment, all the issues, and what colleges need to do. We’ve talked about international enrollment, and now we’re going to approach it from a different perspective because you’re, pardon the expression, you’re a data wonk.

Emily Coleman: Yes, I am. I take that as a compliment.

Drumm McNaughton: And you should because you’re really, really good at it.

[00:01:54] Emily’s Journey in Higher Education

Drumm McNaughton: Give us a little background, if you would.

Emily Coleman: I started out in higher ed 23 years ago. I had come out of a PhD program with a degree in psychology and statistics, and I wanted to continue the statistics part of that. So I got a job at Syracuse University working in the enrollment management office. I didn’t even know what that was at the time, but they wanted me to build predictive models of yield, of retention, for both undergrad and grad. And I ended up loving it, and I stayed there for 12 years. Most recently, I was the assistant VP of enrollment management, and then I went to a higher ed consulting firm, where I stayed for five years. I oversaw the predictive modeling team there. And then seven years ago, my business partner and I decided to go out on our own and form HAI, and we provide all kinds of predictive modeling and financial aid optimization services to higher ed institutions.

Drumm McNaughton: It’s interesting because when you first got started at this over 20 years ago, there weren’t a lot of people doing this, were there?

Emily Coleman: No. It was much less common. And I was the first person at Syracuse to have my role. The person I worked for there, who at the time was the associate VP, had come from Cornell, and he had a researcher who worked for him building predictive models. So he was familiar with it, and he wanted to bring that to Syracuse, so he created the position, and that’s how I ended up there.

Drumm McNaughton: So, for those who don’t understand what predictive modeling is, give us a little bit of an understanding, please.

Emily Coleman: Sure. So predictive modeling is taking historical data and basically creating an algorithm that will predict future outcomes. So the statistical model will determine what the factors are that have a significant influence on whatever it is you’re measuring. How much influence does each factor have? With that, you can predict future outcomes based on prior behavior. So that’s a very simple, high-level overview.

Drumm McNaughton: Thank you. That went over my head. I’m a qual guy.

[00:04:03] Challenges in Predictive Modeling

Drumm McNaughton: I’m not a quant guy I would have to assume that given all of the turmoil in higher ed right now, predictive modeling, I would say it’s probably a little challenging with all the different things that are going on. Is that a fair statement?

Emily Coleman: That is a very fair statement. Yes, it has been a challenging few years, just from a modeling perspective, because if we can’t rely on historical data, it’s more difficult to predict future outcomes. So when COVID started, that turned everything upside down, and we couldn’t really use the same models we had to predict enrollment. As soon as that kind of straightened out a bit or became more normal, we had the new FAFSA that the government released, and that was a mess and was delayed, and there were all sorts of problems that went along with that. So it has given us several years that aren’t related to the prior year in the way that things typically are, which makes it harder to build a predictive model.

Drumm McNaughton: And now we have the new era. Higher ed is going in what many people feel is an unfortunate direction. We have the demographic or the enrollment cliff. With all the visa issues, many students are pushing back to spring because they couldn’t get through the visa process, and we’re going to lose, I’ve read, up to a third of the international students. Costs have grown, although they are actually lower than inflation, but the costs have grown over the past 20 years. I could be really flippant about it and just say, It’s a bloody mess, but it’s a mess.

Emily Coleman: Yeah. Yeah, it’s a tough time for higher ed for sure.

Drumm McNaughton: It is, and what I just heard you tell me was that with predictive modeling, it’s nice to have some decent numbers. First off, how do you define decent numbers?

Emily Coleman: we try to look for one thing, with any model, you have to have a big enough sample size. If an admit pool is too small, then we’re not going to be able to build a predictive model. If we have an adequate sample size, then we need to make sure that the data are accurate. So one of the things that we’ve seen many times, not so much anymore, is that campus visits weren’t always recorded or they weren’t recorded in the same way, and so then a variable like that, which can be very predictive of enrollment, we’re not able to put in a model. So the first thing that we do when we start a project is we create what we call an actuals report, where we’re basically creating descriptives of the admit pool for the past three years for an institution. And then we show that to them and have them verify, “is this in line with what you know your numbers to be”? And that’s a really important kind of quality control piece of it, so that we don’t start building models on misinformation.

Drumm McNaughton: So we’re going to get into the process in just a little bit. I think you’ve given us a great start to that. There’s a lot of things going on.

[00:07:21] Financial Factors in Enrollment

Drumm McNaughton: What do you see as the number one factor that causes a student to come or not come to an institution? To enroll or not?

Emily Coleman: It does vary by institution. I think probably the most common would be financial factors. Can the student afford to go to the university? Are they willing to pay what the university is charging? Certainly, for your more elite schools, you see things like high school GPA have a very strong impact, and then it depends. Different kinds of smaller schools or niche schools will have different factors, but typically, it’s rare that we don’t see financial aid or net cost come in as a factor in the model.

Drumm McNaughton: Which many institutions look at financial aid as that silver bullet to be able to enroll or attract students. There’s a different side to that coin, though, isn’t there?

Emily Coleman: There is, and that can be a tough conversation to have. There are so many small liberal arts colleges in the US that are struggling right now, and the easiest thing to do to boost enrollment is to offer more financial aid, but that doesn’t always get them where they need to be.

So all of the schools that we work with right now are great in terms of the way they set their budgets, but we sometimes have to have conversations about, you can’t let the tail wag the dog, if that’s the right expression. You can say, this is what it costs us to run this institution, so we need this many students in order to do that.

They have to be realistic and look at how many students can we get and at what price point, and then build the budget around that. And for some of these schools, they’re really grasping for a lifeline, but, for the most case, we see schools shifting in that direction to being much more conservative in budgeting, preparing for things like high discount rates because they keep going up every year. But yes, financial aid is the easiest thing to point to. Sometimes it is the whole factor, sometimes it’s part of the factor, and sometimes it’s not a factor at all in terms of what’s going on at an institution.

Drumm McNaughton: Sure. And it also makes a lot of sense to me that, if you use financial aid, and I see this all the time with institutions that we work with, that they’ll go, “oh gosh, well, we’re going to set the enrollment number at last year plus 10% plus 12%” without a good, first off, they didn’t make the number last year. They don’t have a good plan, they just set the number and they base a budget on that, versus, as you said previously, facing reality. What can they really attract?

Emily Coleman: Yes

Drumm McNaughton: Do you see that a lot?

Emily Coleman: We do see that a lot, and we see it overall at institutions, and then we also see it within programs at an institution. Higher ed institutions are not known to be nimble when it comes to adapting what they’re teaching, what they’re offering.

Drumm McNaughton: Really?

Emily Coleman: Yes, they have a whole infrastructure built around a certain number of students and a certain program and faculty and everything else. And it is hard to shift that if you have some architecture studio that is very specifically designed for architecture students. And then you can’t hit the enrollment in that program, it’s hard to just fill in with another program. So that’s where I think these, kind of, unrealistic goals, they come in and they just keep being put in place year after year because it’s much harder to change kind of the structure of the offerings.

Drumm McNaughton: Yeah, you just brought up an interesting point. I’ve never heard anybody say this before, so kudos to you. This is really, it’s insightful. You can’t make up enrollment if you’ve got one program that is not enrolling the number of students that it needs, you can’t necessarily rely on another program to bring in those extra students because there may not be the infrastructure, there may not be the labs, et cetera, et cetera.

Emily Coleman: Yep. Yep, it’s a big problem and it’s a question that keeps me up at night. How do we help institutions become more nimble? Adapt to changing circumstances? Everyone got a lesson in that with COVID because suddenly every institution had to figure out how to teach their students online. The ones who did that the best were the ones who were already doing it. But, it did force everyone to come up with this alternate offering in terms of the way students are learning, which I think strengthened them over time if they can keep that up. But, we talk a lot about diversification, of teaching offerings, teaching methods online, in-person, hybrid, the more an institution can meet students where they are, the easier time they will have hitting their enrollment goals.

Drumm McNaughton: Yeah, so true.

[00:13:05] Adapting to Changing Enrollment Trends

Drumm McNaughton: Let’s talk a little bit about the enrollment process. It’s changed over the last 20 years, hasn’t it?

Emily Coleman: Yes, it definitely has. I think that there are a lot of reasons why it has changed. It used to be the case that a college degree was seen as something valuable in and of itself. People went to college to have that experience to learn, and there’s much more of a consumer mindset now, and I think that has a lot to do with the cost of college now. Families, parents, they really are scrutinizing ROI. Is it going to be worth it for me to send my kid here? They want to be hear about job outcomes, they want to be hear about internships, things that not a lot of people were talking about 25 years ago. It was a different reason why kids went to college. And now people are considering alternatives or they’re going to community college, it definitely has shifted the way schools need to respond to that and communicate with families.

Drumm McNaughton: It’s so critical that they do that, and there are quite a number of institutions out there who haven’t changed for a myriad of reasons and we don’t need to go into those. But you just talked about a number of the important things, the consumer mindset. Parents and students, but especially parents, they want to know what’s the return on investment. What are the job prospects? If my Mary or my Johnny go to your program, what can they expect for a job coming out? That webpage, the “X, Y, Z University at a glance” is so critical to give people an idea, not only from the university perspective, but also from the college and the program perspective.

Emily Coleman: Yeah, even the most prestigious schools need to be able to speak to where their students are going. It used to be the case that a strong alumni network was enough to recruit parents to get on board with an institution, but people really want more concrete, real outcomes now, or examples of those.

Drumm McNaughton: It’s interesting you bring up the parents and the alumni. Are you finding that legacy enrollments are dropping or are they making a big difference?

Emily Coleman: That’s a good question. A lot of schools have moved away from any sort of preferential treatment for legacy students. There still are a lot of schools that have that in place, but certainly if we look 20 years back, that was a bigger factor than it is now. I haven’t looked systematically at it, but we definitely do see dropping levels of legacy students or, it used to be the case that if you were a legacy, your probability of enrollment was much higher than if you weren’t, and that difference is narrowing. So we don’t see that being such a big factor.

Drumm McNaughton: Also with that, the institutions have to adapt to this new normal, the customer mindset. And I know I have a conversation with faculty and faculty in many respects, some of them get it, but many of them don’t get it. They say, “well, the customer’s always right”. No, the customer isn’t always right but there’s more students out there who are coming in and saying, “I’m the customer.” And every business should have the rules by which they operate, and if the customer wants something outside of those rules, you have to say no.

Emily Coleman: Yes. Yeah, there is a disparity in what is driving faculty and what’s driving students, and it’s probably at its most disparate than it’s ever been because we still have faculty members who have gone into academia with kind of the old mindset of this is knowledge for the sake of knowledge and education for the greater good. And they understandably so don’t want to be be seen as people running a business or selling a business. But the families have shifted away from, again, not that they don’t appreciate sort of knowledge for the sake of knowledge, but it isn’t as much of a driver as it used to be.

Drumm McNaughton: You’re absolutely right, and there are so many different avenues now that young people, let’s just put it young people, can take to get into the workforce. College is certainly one of those. For graduate work, that’s a little bit different story, and we’ll get into that in just a moment. For undergraduate, you’ve got internships that give you exposure, but not everybody should go to college, and some people should delay going to college so that they have a little bit more, I’ll just call it EI, emotional intelligence, to be able to be self-starters, to be able to drive your own process. Are you seeing those kinds of things as well?

Emily Coleman: Definitely. The gap year has become such a common thing and everyone knows what that means. I think about 30 years ago when I went to college, no one I knew took a year off between high school and college. They either went straight to college or they went to work. And I think younger people now are, they have a better grasp on their emotional intelligence as you’re saying, and their capacity to just take on certain challenges. So I think they are more willing to make their own path and do what feels best for them, and parents are more willing to accept that. So we definitely do see a big shift there.

Drumm McNaughton: And how does that factor into, I knew I’d bring it back to predictive modeling at some point, how does those types of things factor into your modeling? Let’s get into the process that you use to do this, ’cause I personally, I think it’s fascinating.

Emily Coleman: Yes. Yes. So with something like that, some of the factors that we would include in a model are the age of the student, have they previously enrolled at another institution, and we’re looking at the fact that students who have done those things probably have different drivers of enrollment than students who are coming straight out of high school, we’re cognizant of that.

There often are not enough of those students in a pool for us to include that as a factor in a model. So what we have to do in that case is we have to just be very aware of the descriptives of the pool and how we may need to adjust our assumptions about the model or our interpretation of the outcomes. We talk a lot about how statistical models are great, artificial intelligence is amazing, but we need to have that human element as well. And we need to be, yes, we named our company after it. HAI stands for Human and Artificial Intelligence.

Drumm McNaughton: Oh, I was going to ask you what that stood for.

Emily Coleman: Yeah. We’ve seen a rise in, with these AI models that are so powerful and sophisticated that, students come out of college knowing how to build these amazing models. But also sometimes with the idea that you don’t have to think about it, you just put all the data in and the model will run and then it will tell you the answers. And that’s dangerous for a lot of reasons, we can see a lot of bias. So we’re always talking about making sure that we are paying attention to the data, not just taking everything straight from a model and interpreting that way, but looking at shifts that have happened in the pool outside of the model.

We usually start our work at the top of the funnel. So even if we’re developing a yield model. We’ll start with an inquiry to applicant conversion model. And the reason we do that is sometimes you can see those shifts at the top of the funnel in factors like who’s coming straight from high school and who isn’t, that then are not obvious when you get to the admit pool, but they will affect yield. So it helps us to build models that are more accurate, basically, when we get to the yield stage.

Drumm McNaughton: What do you tell an institution whose data is for lack of a better word, wonky.

[00:21:37] Data Requirements for Predictive Modeling

Drumm McNaughton: Before we get to, “what do you tell ’em?” what kind of data do you really need to be able to run one of these models?

Emily Coleman: So we need, essentially, everything that would come in on an application and everything that would come in on a financial aid application. Where they’re from, where they went to school, their grades, the major they’re interested in, how did they first get in touch with this school? Did they reach out or did the school find them? Those are all factors that we would put in a model.

And then on the financial side, for students who apply for federal financial aid, we’re looking at parent income, levels of need, aid offered. So those are the essential elements. We also tend to include some geodemographic data that we get from the census or from Experian. That can help, especially with those students who don’t apply for aid, it can help fill in details about them. Did they not apply for aid because they truly don’t need it? Or did they not apply for aid because they’ve already decided they’re not coming to your institution? It’s important to distinguish between those pools if we’re going to make accurate yield predictions. So that, that’s yeah, that’s an overview of what we would put in a model.

We asked for three. Recently, it’s been harder to build a model on three years because of just all of the changes that have happened between 2020 and now. So for larger schools right now, we’re often building just on the prior year. We hope to get back to that place where we can include more years of data into a model because obviously the more you have, the more accurate they’ll be.

Drumm McNaughton: So what do you do if you don’t have all the data that you need? We have a term called “WAG”, which is Wild blank, blank, blank, Guess. But that doesn’t help an institution.

Emily Coleman: Yes. So there are different things we can do. One of the things that we do is we talk to an institution about the fact that before we can help them in terms of modeling, they need to be collecting certain data and they need to be putting that into a system. It needs to be reliable and consistent. So sometimes the project stops there because they haven’t recorded certain data that they’re going to need. That’s rare. Usually there is something we can do, even if it’s just with descriptives. A lot of graduate programs that we work with, they want to be know how to optimize aid, but the pools are very small, so we can’t build a yield model the way we would for an undergrad population.

Instead, we need to really look closely at the historical data and make some educated guesses about what the impact aid has been, if we control for the other factors. And while often for those schools we will help them with an aid strategy that minimizes risk. So, if there are a group of 10 students that have a zero likelihood of enrolling and you offer them a 90% discount and just one of them comes, that’s a net positive. And that’s something that can be difficult to grasp because you feel like you’re not charging them anything. But if it’s a student that had no possibility of enrolling before that then, anyway, I won’t go too far down that rabbit hole. But we are looking at different ways that we can make assumptions, make educated guesses, minimize risks for those programs that don’t have all the data that are needed to build a true robust model.

Drumm McNaughton: So you’re able to build financial aid modeling, predictive modeling of yield, how you can stimulate changes to aid and ultimately enrollment. Is that correct?

Emily Coleman: Yes.

Drumm McNaughton: And with that enrollment, that is critical because if you can predict the enrollment, you can predict how much financial aid you’re going to need to give, et cetera. You’ve got good predictive numbers for budgeting.

Emily Coleman: Yes. Yes.

Drumm McNaughton: So what you’re actually doing is giving institutions data driven numbers to base their strategic plan on, their manning, et cetera. Can you take these enrollment numbers down to programs as well?

Emily Coleman: Depending on the size of the program, we can. If it’s a program of 10 students, then no, but we’ll work with a lot of schools that have an education college that has a hundred students in it, or a hundred incoming students. So a lot of times we are looking program by program and just controlling for that in the model that we’re building.

Drumm McNaughton: Because I, what I’m seeing is, it’s perfectly fine to go out and take a look at, these are the students, within our mission, et cetera, that’s critical. But unless you can break it down to say, how many students can you admit? Will you admit, and of course your marketing comes into this as well, where’s your reach, and all of those other things. It’s really looking at it enrollment management has got to be holistic. It is combining all of the predictive models that you’ve got. It has to look at your marketing, has to look at your programs, what the job needs are. It’s not a simple one plus one equals two.

Emily Coleman: It definitely is not. And we don’t get into the marketing side or the value proposition, but we are often talking to schools about the fact that if that doesn’t change nothing else will change. And the other thing that we do when we build a yield model and we look at financial aid strategies, is we build a first to second year retention model. Because we don’t want to be be shortsighted.

We want to be make sure that whatever strategy we’re developing will not just bring students in the door, but we’ll allow them to stay and succeed. And it used to be that admissions and retention were two separate offices on campus that never spoke to each other and that’s more rare now.

Now the VP of enrollment management has an eye on retention. People realize that it’s much cheaper to keep a student than it is to recruit a new one. So everyone’s looking at that and from the first stage of inquiry, schools more and more are looking at that and saying, “which of these prospective students will become happy alums”, they really have a long-term look on it as opposed to just let’s get to the next stage of the funnel. Which is great, I think it serves institutions better and students better as well.

Drumm McNaughton: I had just recently had a conversation with someone from the City University of New York, CUNY, and they’ve got a very small school there. But they have combined enrollment, student support, career placement, and alumni relations all together. To me, you’ve got all the key pieces in the room to make sure you doing these things right to boost your enrollment. Does that make sense to you from a data perspective?

Emily Coleman: Absolutely. Yeah. I think you need to have that more comprehensive approach. Everyone needs to be sitting at the table working toward the same goal of enrolling students at this institution. If a school is operating in silos, they just aren’t as effective. And so if you have folks from all of the places that you mentioned at the table, then you can really come up with an enrollment strategy that is realistic, let’s say, that you know there’s a good chance that you will be able to accomplish your goals.

Drumm McNaughton: And with so many institutions in distress nowadays, they need to be doing this yesterday, not sometime in the future.

Emily Coleman: Yes, absolutely.

[00:29:43] Three Takeaways for Higher Education Presidents and Boards

Drumm McNaughton: So Emily, this has been a fascinating conversation for me. I thank you so much for being on the show as we always do our two wrap up questions. Three takeaways for higher ed presidents and boards. What do they need to be thinking about from a modeling perspective to make sure that their enrollment stays where it needs to be?

Emily Coleman: They need to make sure they have the right data. So we talked about schools that don’t have the right data. One of the most common issues we see with data is that the students who don’t enroll the school purges their financial aid offer data, and if that’s the case, then we can’t build a model. And if you aren’t data-driven in the way you’re allocating aid then you’re really just taking guesses at what it takes, and you’re probably giving too much money to some students and not enough to others. So that focus on data and making that a priority, I would say that that is one of the things that they need to look at.

The diversity in teaching methods, like we talked about earlier. Do you have online offerings? Are you appealing to non-traditional students? Do you have part-time offerings? That’s an obstacle, and a lot of times board members I think, don’t want to hear that. If their alums at an institution, they want the institution to look the way it did when they were there. And that isn’t the world anymore, so we really need to be open-minded and thinking about strategy and how do we diversify the revenue that’s coming in? Because most schools are heavily dependent on tuition, revenue as their main source. And then the last thing I would say is setting realistic goals. So, that’s another thing that we see with presidents and with boards is just setting goals that are, that it’s just a backwards relationship that, this is what we need in revenue so this is what our goals are. And they need to really be realistic about how many students they can enroll, what’s the price point in order to really survive.

[00:31:52] Bonus Takeaway from Dr. McNaughton

Drumm McNaughton: Yeah, and that is so critical. We see so many institutions not doing that, not setting realistic goals, and then all of a sudden it’s like, “oh my gosh, we’re in trouble”. Instead of being able to look down the line and say, “Hey, in five years we’re potentially going to be in trouble. Now is the time to either turn things around and at the same time, start looking to see if maybe we can partner with somebody else to strengthen, whether it’s backend partnership or whatever”. That old “M word” that no higher ed institution likes to talk about, “merger”.

Emily Coleman: Yes, yes.

[00:32:32] What’s in the Works for HAI

Drumm McNaughton: So what’s next for you?

Emily Coleman: Well, we are continuing to build HAI to diversify our offerings. Most of our, partners now we’re doing full yield modeling and financial aid optimization, but we’ve developed some data visualization tools that can help schools that don’t necessarily want that intense level. We have some announcements coming up about partnerships with other companies that round out our services. So looking at the recruitment and the messaging and the marketing and all that, and how do we bring that together with what we’re doing. So it’s an exciting time for us. We continue to grow and get a lot of interest from people and think about ways that we can do what we’re doing a little bit better.

Drumm McNaughton: That sounds fabulous, and again, I want to thank you for being on the program. I have really enjoyed our conversation. I look forward to the next time our paths cross.

Emily Coleman: I do too. Thank you for having me.

Drumm McNaughton: Thanks for listening today and a special thank you to my guest, Emily Chase Coleman from HAI Analytics. Emily, thank you so much for being on the program. Some great information for the listeners, and I look forward to the next time our paths cross.

To my listeners, thanks again for listening, see you next week.