Rachael Maves: Hi everyone. You are listening to the Rachael Maves podcast and I'm Rachael Maves, your host. Today we are talking to Amy Chen Kulesa, senior associate partner at Bellwether. Amy is a senior associate partner at Bellwether, a national nonprofit that collaborates with education leaders and organizations, from school districts and Charter networks to foundations and nonprofits, to expand their impact on systematically marginalized young people. At Bellwether, Amy has led numerous strategic planning, growth and Improvement projects with schools, networks and nonprofits. Recently, she has co-led Bellwether's work in AI and served as the lead author for the Learning Systems shaping the role of artificial intelligence and education report series, and I have to say it's fantastic.

If you haven't read it, go check it out. Amy's career began in the private sector, Consulting, after which she directed data and strategy for a Boston Public Charter School. Amy, thank you for joining me today.

Amy Chen Kulesa: Thanks, it's an honor to be here.

Rachael Maves: So, as I start with everybody on the show, because we really want to highlights and Elevate stories and people's experiences, I'd love to hear how you ended up in education.

Amy Chen Kulesa: Yeah, I think my path is both informed by my family's history and path as well as my own professional Journey. So I was born in the US to immigrant parents who really saw education as their ticket opportunity and education really runs in my family. My mom, as well as all four of my grandparents, were at some point in their life teachers and they really shaped. The transformational power of Education was really instilled in upon me early in my life. I moved back to the. I lived in Shanghai for a little bit when I was young, moved back to the US when I was in fourth grade and was in English Second Language learner at a local public school and that re experience really taught me firsthand how important education was in opening doors in academics and socially and the importance of the basic writing, reading, math, critical thinking skills really transformed how I interact with the world.

My first unofficial job was actually a teacher when in high school I taught painting and drawing to my younger brother and the neighborhood group of kids and I loved being around kids and helping them and kind of creating the light bulb moment. Despite my kind of early interest in education, I took a different path initially for my first real job I'll call it and I studied Engineering in college and then worked at private sector Consulting. I chose that job because I wanted to like learn as much as possible, wanted to kind of see, quote, unquote best practices in the private sector, and I did learn a lot. I learned Financial modeling, how to build PowerPoint decks, how to speak and earn the respect of, you know, Chief, like Executives, as a young five for five feet, 11, female. But after a while I felt like I wanted more social impact. So that's where I transitioned and did the Ed Pioneers Fellowship.

I landed at a charter school called Boston Prep in the B grader Boston area and there was where I learned about some of the systemic inequalities and the systems and structures that surround our education system. I witnessed some of the traumas that influence and experience the student experiences for the school P I was at and I also saw like really stellar and amazing and mission-aligned teachers who are just masters of their craft. Yeah, and that's where I knew like education was definitely the place I wanted to spend the rest of my. And I've since Boston Prep. I wanted to either go deeper into teaching or look back at a more National level and see if there are places to connect the dots and learn from different patterns, and I've been at Bellwether. That's how I got to Bellwether for now seven years and I think this is like my sweet spot right now, combining the two prior jobs of the Consulting and the school experience where I'm able to really address some of the systemic inequalities that exist, especially for marginalized students, and I'm able to do so with some of the nerdy data and research and, you know, financial analysis and stakehold engagement stuff that I also like to do, and all while working along other smart Mission aligned people like yourself.

So, yeah, that's where, that's where I am.

Rachael Maves: I love that's fantastic. So I'm curious like it sounds like really you had this like draw towards the more systems perspective in lens. What are some of the projects that you've had the opportunity to work at Bellwether that have made that intersection between being able to look at those broader patterns and Trends but then also connecting to your classroom background.

Amy Chen Kulesa: Yeah, that's a great question. So I about whether we work across different School type work, nonprofit work and system type work. The most recent project that I think we've talked about before is around artificial intelligence, and this is a space that's still very much nascent. The technology is there, it's, it's developing and how it lives at school is still early. So we have the pleasure of kind of having that systemwide view to think about. Not just how can individual schools potentially dip their toes in it or an attech company do that, but what do we need in the ecosystem to really support a vibrant e ecosystem and Community to make sure that things are quality and Equitable and that may be Beyond individual organizations and more at the like.

When you look across the map, what is missing? We've also done some of this work around like whole child, for example, where we know academics are important and we also know that you know social, emotional Health, mental health, ENT, entity those are also really essential. So we've kind of looked across different exemplars to try to highlight different case studies and that was work that we did a few years ago with the Chan Zuckerberg initiative and highlighting some of the systemwide exemplars, and I'm grateful to do some of that work because it's it's fun and, yeah, well, it's such a needed space, too in education. I think that's one of our barriers is finding those places like Bellwether and the work that you're doing, that really do Elevate the bright spots and best practices and what we can be thinking through for the practitioners that then can take that information and say, okay, how do we make this live within our systems, knowing that this is evidence-based, research based and we can get good results for our kids.

Rachael Maves: So it's just fantastic, Amy, I'd love to hear because the AI is on everyone's mind, I'd love to hear like just a couple of points from the research that you found that really stood out to you.

Amy Chen Kulesa: Yeah, definitely. So one lens that we've taken was what can be safe and effective for students and what are some of the enabling conditions for that. And so, through many conversations and we've talked to 50 plus experts, ranging from school leaders to researchers and academics, to, you know, Ed Tech leaders who are coding and developing the work, to also Skeptics who Shar, you know, in real concerns about some of the cautionary tals that AI could do and some of our key takeaways there. One is that, rather than leading with tools, we do recommend leading with capacity building. There needs to be the knowledge, the skills, the critical thinking to decipher what the purpose of the tools should be used for, how it can be implemented with Fidelity, how it can be safe and effective as opposed to just here's a shiny thing, let's let's test it out. The second key theme that we really tried to Elevate and found was the importance of data and the underlying data that trains and informs and fine-tunes the model.

So any tool is as good as its inputs. So some people say, like garbage and garbage out, and that's that's very much true with AI too, and what information you feed it, how it's able to process it, matters, and whether or not the output of the data is quality. And then the last thing that we try to really Elevate is around the importance of inclusive design, so being grounded in research and being grounded in the human relationships. So leading with human, leading with what we know and leading with the history and the wealth of research that we've had. We can still very much learn from that, and that is, if not even, more, relevant and important to pay attention to in the world of AI.

Rachael Maves: I love that you came back to that and you hit that piece, and that was one that really struck me and what I appreciated about the way in which you were presenting this information is leading with human. I we hear a lot about AI and this almost fear that it's going to replace Educators or we're not going to have that human connection anymore in the classroom, and I see that as being, you know, one extreme. And you know and we've talked about this also, Amy I think the pandemic gave us a really good picture of what that would look like potentially, of not having those face-to-face interactions, and for some kids that worked great, but for the majority of students, we saw an increase in depression rates. We saw an increase in suicide ideation. We saw an massive impact in mental health. We saw kids who left and never came back and were not re-engaged in our system.

We saw a massive decline in performance with standards. So, like I for me, I'm I'm looking at and excited about where are the ways that we can leverage AI that actually allow more time to be able to focus on the things that we need to focus on. So, as a teacher in the classroom, connecting with my kids, having a deep understanding of where they are with their learning and where they need to go with next steps, what it is that really excites them and how can I engage them in a relevant way based on what they're looking forward to. I mean, those are the places where I see AI really having a massive impact, and I'd love like just your thoughts as you're thinking through it, because it's there's a there's a wide spectrum of where people are landing with AI right now.

Amy Chen Kulesa: Yeah, and I'm jumping ahead a little bit, but, like I know, one of your question is typically like what keeps you at night? And I do think the polarity of how think yeah is what sometimes keeps me up at night, and I think you're speaking to that a little bit too, where you know, as we're recording, this is the week of it's election week, where our company or not company, our country, it is quite divided and this, I think, applies within and Beyond AI and how we're. We're just thinking about things. So I think in Ed reform or in education, we often do like you know, school choice is wonderful, or you know Charters and vs are awful, or we need greater accountability, or we need to remove assessments, and I think that AI people have coined kind of the Boomers or doomers, where the boomers are really excited and the doomers are, you know the world is going to end because of AI, and I think that the conversations need to be a lot more nuanced. So, exactly to what you were saying, like it can be true that some students really thrive in front of a computer and that's able to really personalize their learning, they're able to catch up on work or be ahead on work. It can also be that, like, sitting in front of a screen like is not engaging for some people, right, and it can be true that AI can help strengthen relationships.

To what you were saying, by, you know, taking time, taking some of the more bureaucratic paperwork, say, off of teachers plates. It can also be that it, you know, ruins relationships and kind of gets in the way of those authentic conversations if you have AI replacing that. So I think the environmental context, the individual preferences and strengths, how the tools are developed, how the tools are used, like all of those factors and more influence whether or not AI is beneficial or harmful and I think it's. We need to get under the good versus bad conversation to like unpack the specific use cases, the specific conditions and the enabler conditions, to kind of see what, like how can we make AI positive? Beneficial.

Rachael Maves: Yeah, I love what you said about the polarity right now of our nation and in some of these topics is exactly right. I feel like in, especially in education has almost become this hot button for all of these different arguments and ideologies to be, like, played out and discussed, which is unfortunate, because I feel like and I actually just wrote a LinkedIn post about that this morning I feel like it's gotten us away from some of the like core things we should actually be talking about in education, like when I look at the nape scores and where we are after the pandemic and the impact and ripple effect that will have for this generation throughout adulthood and into their careers, and we're we're not talking about that like. Those are the places where I feel like we would have a lot of momentum in that middle ground space to say how do we leverage the best thinking of everyone to move this work forward and it unfortunately, topics come up and I feel like it's immediately we're going to take aside and it's going to be a extreme side and we lose some of that conversation in the middle. That's a it's a very apropo one to keep you up at night, Amy.

Amy Chen Kulesa: [speaker uncertain] I think a lot of people probably share that and to like double click on that a little bit, right. So take the Boomer and Doomer extreme per perspectives. I think if we don't do anything about AI, I think there's a risk in exacerbating some of the inequalities that exist in our society. Right like we need to develop and cultivate the critical reading skills to make sure students are, you know, learning and training for the jobs of the future and not the jobs today. So that, I think, is important to be aware of. Also, think that the Boomers, who might be really excited and optimistic, runs the risk of doing something too fast, too quickly, without the critical thinking and the thoughtfulness to implement.

And you know, several months ago, the Los Angeles Unified School District, the news broke out where you know they invested in a vendor that supported them to develop an AI tool. The vendor end up being bankrupt, the tools now not non-existent, and I think that runs a risk where some people can look at that and say I will not touch AI or some people you know might, and I think there's there's caution to that and there's they learnings to learn there, where, rather than you know, putting all of the students information systems and learning management systems and hooking it up to an AI chatbot and doing so in less than 12 months, maybe we can do slow steps to, you know, get stolder engagement to see what are the families actually want. Sequence that out. Maybe test with some of the lower hanging fruits. Instead of putting student information in, maybe just do your student family handbook, which is already public information. Maybe just do the student calendar so that there's some ease into it and you learn and you refine before some of the potentially high impact but also riskier mov that people can make.

Rachael Maves: Yeah, that makes a lot of sense and I feel like that's kind of segueing a little bit into those like ideas for moving forward. As you think about just where we're at on that spectrum and the polarity that we're facing in our country right now, and especially in education and on topics like AI. What are your ideas or thoughts for ways in which we can move forward and where have you seen it done really well? I'd love to hear that too, if you have ideas.

Amy Chen Kulesa: Yeah, that's a great question. I think the I do go back to kind of our report and some of the key takeaways there where, first and foremost, the capacity building feels really critical and important and this is also expensive and timeconsuming to do. So I do want to kind of be very cognizant of that. I think that there are efforts out there right now where there's a an organization called AI for Equity who has convened a set of Charter leaders. I know organizations such as Chiefs for Change and the ILO group are doing capacity building supports for a range of districts around the country too, but starting to really have those authentic conversations with people to unpack both policies and how do we write those, as well as identifying use cases and pilots and doing so thoughtfully and engaging stakeholders, that is. Slow but meticulous and needed process, and we I think that without doing that, it runs the risk of potentially doing something too fast and putting some of the student information at risk.

Yeah, so I think capacity building is definitely one, and it's one that there's. There's the a significant need where the supply of the folks doing it today is definitely significantly under the demand of what's needed in the field. Yeah, I think the second piece of what we need is around quality. So. Deciphering. Okay, so we have AI, but how do we ensure that it's quality?

Yes, yeah, this is a place where we can look into the tech world and, I think, learn where, yes, AI models hallucinate, it has imperfections, and in the tech world, there are methods such as rag, which kind of retrieves information, or fine-tuning, which uses a data set to improve upon the AI model. So, basically, AI models can be trained. To not halling, not hallucinate as much if you prioritize a certain set of data. Yeah, in education we run into the challenge of there not that many highquality, robust public data sets out there. The data sets that we do have are either not clean or behind, you know, barriers. So there's opportunity to do that and there's also opportunity, I think, to establish a set of benchmarks so that we ensure the tools and the outputs are quality.

So benchmarks are basically a long set of tasks that we can kind of run AI models through. So it can, we can set a bunch of benchmarks to: is this AI model supporting multilingual Learners? Is this AI model good for students with dyslexia? Is this model aligned to a set of Standards? You can have a range of those and have AI models kind of be put to the test, so to speak, and ensure that there's incentives for the tools to be quality and make it easier for the school leaders and District leaders to decipher, because right now there's there's a lot on the market and you know things look shiny and it's likely not the best use of time for folks to dig into all the details, yet it's important to do so for the sake of students.

Rachael Maves: Yeah, I, one of the things that I want to highlight for our listeners, especially those who aren't in education and I think you touched on something earlier, Amy and as you're talking about these like next steps, of ways that we can move forward, I think it's like an important thing to highlight. Back to this: really, can I? If there isn't an embracing and this is just my thinking and opinion if there isn't an embracing of AI, and how do we figure this out? I fully agree that this is going to exacerbate opportunity gaps, and the reason I say that is I feel like we learned this lesson in education and Technology when I look at prepandemic. The number of students who had access to computers and were using them on a reg basis, which now it has become part of everyone's work every day. So I don't think there is a career where you would say that you don't have to have some sort of technological capacity and understanding the gap between students who live in wealthy areas and Wealthy school districts that were committed to technology compared to students that did not have that same access is massive.

And so there that we had this already Gap in Tech technology that I don't think anything short of a pandemic would have forced technology into the classroom for all kids, which I think is like one of the very few Silver Linings that I think came out of that time. But I see the same thing now with AI. Like I don't see AI going away. I don't see it ever becoming something that just phases out. I see it. We will learn as a society how to use this and it will become part of the way in which we engage, and if education doesn't align and figure out how to do this effectively, then we're going to create those gaps all over again.

Amy Chen Kulesa: Yeah, what you said really resonates and it makes me think two things. So I do think in conversation, many folks compare AI to personalized learning and say personalized learning didn't work. AI is not going to work. I do think I'm like personalized learning or a tech trends of the past. AI, to me, feels different in that it's way more ubiquitous. Whether you like it or not, AI is being pushed on you and Google results now have AI.

You know, probably in the very soon, near future, like shopping experiences will be AI integrated. So whether or not you have AI in the classroom is not necessarily an option anymore and it furthermore, it impacts the again this: the skills that the students need, because jobs will incorporate AI, comp corporations will do so. For us to be nimble and to be responsive to the changing environment and set our students up for Su success, I think we need to consider at least the impact of AI. I also think in our world, where we most care about the marginalized students, we often talk about Gap closing and. We sometimes talk about it with a fallacy, assuming that the target is static, and say: you know, the less resourced students are trying to catch up to the current well-resourced students, but the world is more Dynamic than that. Right the well resource students are also continuing to improve.

Those schools are continuing to innovate. So we don't just need to catch up to today, but we need to catch up to where they will be three years from now and five years from now, and I think that's another reason why we can't just ignore AI.

Rachael Maves: Yeah, and I have to do just like a practical application, because I don't. I was actually talking to my sister about this. I don't. I don't think we've really grasped yet the way in which it can help kids and even families. So I am not, Amy. I'm not a math person.

I can't remember if I shared this with you or not. My background: I was a English teacher and you know, when my kids come home and they're all in advanced math classes and they're stuck on something, I'm like sweet, I have no clue like where to even start with you. But I use ChatGPT often and was able to put the question in for her and then prompt it similar to what we've seen in kigo to say: here's the question I'm trying to answer, walk through step by step, but do not solve the problem for her. And so then it went through and it was a really quick, easy prompt to be able to write. It went through with her sitting next to me. She's looking at the steps and then catches her mistake of where she was not correctly solving the problem and as soon as saw it she was like, oh, this is what I did wrong.

And then she was able to solve it. And then, because I want to test her understanding, I asked chat GPT to create another problem like this one, but not to provide any steps be because I wanted to see: did she really understand and can she apply that learning now in another setting? And she was able to do it. But like there's little things like that, I think we can leverage in a whole different way. That also assists families that are still like: what do we do and how do we help our kids understand when there's we don't have the understanding ourselves to be able to support them?

Amy Chen Kulesa: Yeah, I would say I think this highlights that you are a good parent and a good teacher and that you did two things really well there. One, you asked it not to give the answer, because it can give you the answer and in that case it deterior, deteriorates the learning. So like that is a possibility and very you know, present thing that can happen and will happen. So that is key number one. And then two, you repeated it and kind of gave it more, gave your daughter and the tool asked the tool for more practice. Again, I think those type of skills and way of thinking is exactly what I might mean by like the capacity building, so that educators are aware of the possibility of what's good and also know to not just stop short of that one question, to not just stop short of that you know that answer, but to use it to engage in the skill development and the critical thinking.

Yes, so I think your example is helpful in that it can also go the bad way and kind of, but it didn't, it didn't. But I think it's important for us as a sector to be aware of both and to shape the field so that we're doing more of the good.

Rachael Maves: Yeah, so final question, Amy, as we wrap up this episode: what question? And education? Are we not asking that we really should be?

Amy Chen Kulesa: When I think about AI, I think there's a scary question that we're maybe not yet ready to ask but we'll eventually need to consider. Right now we're thinking about how can AI make our work right now easier, with the assumption that the work itself and how it's done doesn't necessarily significantly change. But I think the more transformational, scary, exciting question perhaps is where might AI actually dramatically change the underlying way we do work? Because either a AI might do certain things better or in a different way. So, as an example, in the medical world, Google's Med gyni has been fine-tuned to be trained on a large set of medical question U medical images so think CT scans, retina scans and from that it's able to diagnose certain illnesses with decent accuracy. And also in the medical world, within drug Discovery AI, combined with large data sets of what drugs work well for certain things and combined with Technologies like crisper Gene editing, it's able to identify potential drugs, candidates and combinations that would have taken us years, maybe forever, to find.

So applying to that to Ed, like all these are hypothetical, but it makes me wonder, like: are there potential ways for AI to maybe better or more rapidly diagnose students with different learning differences? Are there ways it can kind of put the early warning indicator type of work on steroids and do it much better? Are there ways to like look at learning environmental conditions, like take into account the temperature of the room, the time, the noise level, and like identify conditions with which certain students learn better? And like adjust the classrooms and the schedules, like there all these things that I think can AI could be used, that could be really powerful, and I sh started this with. I don't know if we're ready yet, because all those use cases could be harmful too if they're not done with thoughtful care, and the tools do hallucinate and do have imperfections. So I think we need to careful to not run to that right now, but we should proactively think through some of these so that we're not retroactively responding to a crisis, but we're proactively planning for something that can benefit students.

Rachael Maves: Yeah, that's fantastic, Amy. Thank you for your time today. I really appreciate it and it's such a an interesting conversation, one that we're going to continue to Grapple with, and I know that you're deep into the weeds of AI, so I'd love to continue having these conversations.

Amy Chen Kulesa: It's just very prevalent for the time that we're living in right now, and especially for education, so thank you for the opportunity. It's great chatting.

---

This transcript was created from the episode's captions and lightly edited for readability. It may contain small errors; please refer to the audio for exact wording.