Resources

• The Data Equity Framework

• Identity Sorting Dials

• Talking Data Equity 

• Data in Community Webinar: How communities support more equitable data design, collection, and interpretation.

• Continuous Improvement Mindsets

​

Table of Contents

• [00:00 - 02:39] (2:39) Introductions and a celebrating a 25th anniversary

• [02:40 - 08:24] (5:44) Alyssa’s education background and joining the WREN team

• [08:25 - 12:30] (4:05) Data and measurement accessibility

• [12:31 - 16:05] (3:35) The role of community in data and measurement 

• [16:06 - 20:33] (4:27) The types of data the WREN collects

• [20:34 - 26:43] (6:09) The role and importance of qualitative data

• [26:44 - 28:27] (1:43) Conclusion

​

Transcript

Cameron Yee 00:11

Welcome to Bright Spots Highlights from the Western Regional Educator Network. I'm Cameron Yee, Communications Coordinator for the WREN. Today I'm with Alyssa Leraas, the WREN's Coordinator of Data and Measurement. In today's episode, we'll get to know a little about Alyssa, her education and career background, and how that led to her joining the WREN staff. We'll also learn about the different ways the WREN utilizes data and measurement and the principles that guide these efforts. Thanks for taking the time to be here, Alyssa. 

 

Alyssa Leraas 00:38

Yeah, sure. Happy to be here. 

 

Cameron Yee 00:43

So we're in, we're recording on January 31st, 2024. [Gosh] Yeah, it's been kind of a month. [Indeed.] Yeah. Ice storm, among other things. And we had to postpone or reschedule our original recording date because of that, partly because of that. But that also puts us closer to sort of an interesting discovery we made as we were listening to previous podcasts and trying to figure out the theme song, what it seemed to be inspired from. And there's a lot of ideas and then the spouse of one of our staff members had the epiphany and it's like, it's No Scrubs! [Yeah.] And then all of us said, That's it! And then I did a little digging and it turns out that No Scrubs debuted on February 2nd, 1999, which is two days from now, is the 25th anniversary of No Scrubs. And it's sort of a dangerous question, but where were you on February 2nd, 1999? 

 

Alyssa Leraas 01:50

I mean, I wrote down in my notes that I was three, but now that I'm thinking about it, I was still two, two-and-a-half, which I think this conversation kind of came about as a way to illustrate how I am the youngest member of the WREN team. But yeah, two. So I don't know, I was probably at daycare. I certainly wasn't listening to No Scrubs. [Your parents were playing it for you.] Yeah, no, we were still in our Disney show tune phase, I'm certain. 

 

Cameron Yee 02:21

Yeah. Well, I was working here at Lane ESD in my, approximately my second year here. Yeah, it's kind of amazing that it's been 25 years since that song, because it's still on the radio. It's in constant rotation in karaoke, I think, also. [Yeah, for sure.] 

 

Cameron Yee 02:40

So there's no great segue into the next one, necessarily. But kind of touching on sort of your position on the team in terms of age, I think there's maybe a segue around, like, you and I, how we are sort of the non-educators on the team [quote unquote]. Whenever there's yeah. I just came into the education field because I started working at Lane ESD and my background was in communications and kind of technology type stuff. So it never was in the classroom. So usually when there's introductions of the team, there's like the lineup, like how many years in education have you been? We're always looking at each other like, I don't know where to stand. [Or what does it mean to be in education?] Yeah. And then like I try to qualify it and say like, I've only worked here for 25 plus years. I don't really – yeah. So it's sort of embracing that role of like, are you an educator? No matter your position. But how did you kind of come into your role here?

 

Alyssa Leraas 03:53

Yeah, I think I was thinking a little bit more about that question too of like how many years have you been in education and I would add the layer for me that I did work with students in a school building for a year serving with AmeriCorps after I graduated undergrad but I knew that it was temporary. I knew I didn't want to be a teacher or in a school all of the time so it's like I have this one weird one-year stint where I was like more or less in the classroom, but I wasn't really a teacher. And so it's like, well, I kind of was in education for a year, sorta, and then transitioned, obviously. 

 

But my background, yeah, I mean, I did my undergrad since we kind of talked about that in political science and a little begrudgingly, I suppose, in economics. So really kind of started thinking about data a lot more, kind of applied research. What does data tell us about the world kind of through an economics policy lens? I always thought that I was going to go to law school and then realized I didn't actually want to practice law and kind of found my way veering into the education policy world towards the end of undergrad, which is what led me to working in a school for a year. And then after I kind of decided no on the law school thing, I started looking at public administration and public policy programs for grad school and really wanted to dig more into kind of the policy research and analysis piece, specifically in education. And so ended up moving out to Eugene and going to the University of Oregon and got my Master's of Public Administration, really focused again kind of in that education policy world. And when this job opened up, well, I like heard about it through the line of people, some colleagues and stuff that I had had classes with at UO and kind of mentioned like, yeah, I really kind of want to do data evaluation work in an education program that's really focused around equity. And so when this job opened up, Sue Wilson quickly sent me the email and was like, you should definitely apply for this. They're doing really interesting work. And so that was kind of my connection and how I ended up here.

 

Cameron Yee 06:17

I guess for more information is that you're not the first data and measurement coordinator that we had on the team. Had someone very kind of briefly. [Yeah, I think she was here about six months.] So how do you see that? Did anything change, I guess, between the first person and then your taking on the position? 

 

Alyssa Leraas 06:39

Yes, partly just because the programming expanded so quickly. Because we were hiring more people, like Aly and Erin had both just been hired about six months before I was hired. And so programming was really starting to take off. We were getting ready to kind of transition back into like actual being in person in schools all the time coming into the Fall of 2021. And so programming was just changing really quickly and really, really expanding. And so I think just that alone kind of changes the direction of what we are measuring and what data we have access to and can collect. And I think by nature of the WREN, like we have just kind of transitioned and flowed, I would say, with the needs. And so I think there was some really helpful things that the previous data coordinator handed off to me, just some initial kind of structures that were being built out at the really kind of broad level. But then I was really kind of filling in those gaps of like, as we are developing programming, what are priorities and goals when we're thinking about measurement and impact from the WREN? And kind of more some of the like practical things, like, okay, so we have this, we have kind of this big picture vision, but what does that actually mean when we are doing it in practice? So I think some of those things kind of fell to me and I had a starting point at least, like there was information that they could share with me about things that had been done previously, but I did really get to build it out, I would say from there, like my own role kind of building it out from there. 

 

Cameron Yee 08:25

Yeah, so when we were planning this podcast, I asked a question of you that I haven't asked the other participants, which sort of indicated like sort of my own like sense of I guess discomfort in a way. Like it feels like talking about data and measurement is a little outside my realm of experience. And so as we were talking, I asked you, do I need to do anything to prepare to talk to you? And it sort of revealed like it's sort of like the math question. Or like talking about math is like intimidating for people. But that introduced another topic around like just accessibility of data. Would you want to share your philosophy on that and your perspective on…

 

Alyssa Leraas 09:08

Yeah, yeah. Yeah, I mean, I think I really did say to you like no, because everything that I share should be accessible to anybody regardless of kind of their experience or understanding of or education in kind of “data” and math. I don't like to think that I do a lot of math in my job, but I guess it comes up here and there. Everybody does math all the time. But yeah, and I think my background in economics specifically and then kind of going into grad school was still focused a lot on the policy analysis piece, which is really like an applied economics like data analysis work with models and all the fancy things. But really coming from that realm in a lot of ways, and I would say like the discipline of economics is starting to move away from this a little bit. There is definitely a show for how complicated can we make this? How inaccessible can we make it? Like how can we, because there's a lot of layers and they're trying to prove causality, which like maybe we can do, maybe we can't. We can argue about that another day. And if the information is not accessible to the people who are making decisions based off of the research or the analysis or whatever, what's the point? And how do we kind of bridge that kind of academic research field to like practitioners. And I think that was kind of where I saw my role. Like I have this background in training in economics and I can follow the stupid models that are way too complicated. For the most part, I'm not, I'm no Doctor of Economics, but for the most part, I can follow and take what I need to from that. I know how to discern that information and then being able to bring it to folks in a way that's really here's what the research is telling us, here's what we're learning from this information, what do we do with that as practitioners, as people who are trying to make impacts in like our local spheres, how do we kind of condense and synthesize that information in a way that is accessible for people to use. I think to your point, like I think that is kind of one of my roles in this work is like how how to make that accessible and how to bring folks into that so that they feel like they have as much kind of ownership over the data, the evaluation, the measurement pieces of the WREN as me who like comes with this “expertise” as they say. So really trying to bridge that and bring it as an offering kind of to our community, to our network, so that they can be active participants in kind of the data and measurement work. And it doesn't have to be this kind of expertise, like pretentious kind of data world that some of my colleagues like to live in outside of, outside of the WREN. Colleagues is in broad in broad terms. [Your peers.] Yes, my peers. Yeah. 

 

Cameron Yee 12:31

And I think on our kind of agenda, this is a little bit later or listed a little bit later but it also seems like a nice segue into just the human-centeredness of your approach, our approach, and also the community aspect of looking at data, analyzing it, and making sure that there are multiple perspectives on the information. 

 

Alyssa Leraas 12:59

Yeah, I have kind of been reflecting on and have kind of I guess I would say tag-lined, but I definitely got this from somebody else. I should find my reference. But the idea that data is by community, for community, and with community. Data is representing people, regardless of how kind of aggregated it is or how high level it is, it is still representing people, and we are telling a story about individuals in that information, and it is like a gift that they are sharing that with us in all regards. I think, again, the higher kind of like the Census for example, we don't usually talk about the Census in that way and yet like we don't have any of that information if people aren't willing to share that with us. And so then what is our responsibility to those individuals? And if we are asking for that information from people we shouldn't be asking for it if it isn't something that is also bringing benefit to them or isn't something that they are also seeking answers to. So that's kind of the for and with community like it should always be returned to the people who are offering that in a way that is helpful to them. So I think I kind of like you were saying with the perspectives and stuff like kind of bring that in from a lens of like we need to be getting feedback on survey questions, like what kinds of questions are we answering? Arguably, you can back up even further, like what are we even trying to understand? Like what are the questions we're asking? Which I think our team, the WREN does that a lot, like our questions, our problems of practice that we've been working on, like those are all developed in partnership with Coordinating Body members and other network members. And so then when you get to kind of, how are we measuring that? It's like, what kinds of questions are we asking? What survey questions are we putting forward? Whose eyes do we have on those? How are we sharing the information? Is it being disseminated only via email? Are we sharing it through other networks? Are we sharing it in other mediums and then again kind of coming back to that analysis piece like it should always be returned to the people who offered it and so I am not going to kind of recruit the entire network to help do the data analysis. But I do have representatives from our network and from our community that are on the Data and Measurement Task Force that can help me with some of that work. So it's other folks outside of this world of data and measurement that bring their own expertise and their own identity and experiences to the work and offer that lens as we are looking and analyzing that data. So I have actually invited them to do some of the data analysis work with me in a way that really folds them into that process. Like I was saying, it gives them ownership over the work that we are doing in a way that, again, just like isn't necessarily traditional for data and evaluation or program evaluation, but is definitely like where we are moving more towards, especially when we think about doing it from a place of equity and human-centeredness. 

 

Cameron Yee 16:07

So what are the kinds of data that you're collecting?

 

Alyssa Leraas 16:12

So kind of the two big surveys that I hold is, is this true? This is true. The two big surveys that I hold are our Network Health Survey and then kind of our Professional Development Implementation Survey. So the Network Health Survey is really looking at how are we, how are we fostering our network? How are we fostering the community of the WREN to bring folks together, to make it a collaborative space, to make it a meaningful space, and to foster connection among network members. So our Coordinating Body identified goals for our network that again, were focused around that connection, collaboration, and meaningful engagement of network members. And so that's kind of what the Network Health Survey is trying to capture is how we are doing towards those goals, more or less. And then the other big one is the Professional Development kind of implementation one, where we're looking at really big picture, how are folks using the learnings that they've had in the professional development they've participated in with the WREN. This is in contrast to the feedback survey that Michelle sends out following each professional development survey, which is, Michelle's survey is really focused more on kind of that quick snapshot like what are your takeaways and would you recommend this? Like, should we continue offering this more or less? And what adjustments need to be made to this offering? Whereas mine, again, is really thinking big picture, like what are people doing with this? And so we send it out twice a year with the hopes that giving people a little bit more space after their participation in a given offering gives them a chance to actually do the implementation. And so we can capture some of that information. And I do a lot of Likert scale questions and a lot of open response questions to kind of mix like, yes, it's a survey and also we are still trying to capture the stories and experiences of folks participating in our network and our professional development. And so just offering people an opportunity to kind of share that in their own words and mixing kind of that more quantitative numbers focused responses in the Likert scale with that qualitative narrative feedback that we get through the open response. So those are kind of the two big surveys and then we complement, I would say, complement all of that work with interviews that we do throughout the year, typically with design team folks, either the team leads or the team members, which is really focused more on the coaching programming. And it always kind of feeds into a lot of those other things just because people are participating with the WREN in so many different facets. That they talk about professional development a lot they talk about kind of their experience with the Coordinating Body, if they are, if they have overlapped in any regard that way. 

 

Cameron Yee 19:08

Does the Network Health survey go out twice a year also or more frequently. I’m just trying to remember.

 

Alyssa Leraas 19:17

Me too. I think that we have moved it to twice a year. The school year goes by really quickly, really quickly, and there's only like we only have so many convenings and so if you're kind of thinking about timeline it's like we can't expect these things to change if we aren't doing anything to address them. So our Network Health Survey, it's like we have a convening with our full network in the fall and with our full network in the spring. And then there's different kind of iterations throughout the year where we're maybe engaging with design teams or we're engaging with Coordinating Body members, but there isn't necessarily overlap there explicitly. There might be, but not as explicitly. And so we were doing it more frequently, I think, my first year. And then it just kind of became clear, like, the information isn't varying enough to need to do it more frequently. And again, thinking about like, what are we doing to kind of impact those things and where those occurrences falling in our calendar year, and they fall in the fall and in the spring. And so it just kind of makes sense that that is when we are offering that survey. Yeah.

 

Cameron Yee 20:34

Yeah, and going back to sort of the type of data, what struck me when I joined the team was sort of the focus on what has historically been called qualitative data. And I think whenever I've heard that in the past, it's kind of been dismissive of, oh, that's qualitative data, or that's anecdotal. I think it's more the term that I hear. And it's just sort of dismissed as like, OK, that's great. They had this experience. But let's look at the hard data or the numbers. And that seems to have at least shifted. I mean, it's a focus within our program. And from an external viewpoint, it seems like there’s been a shift a little bit even outside of our program. What do you feel is like the importance of qualitative data compared to the traditional quantitative? 

 

Alyssa Leraas 21:28

It brings me back to my economics roots a little bit because the discipline of economics is like comically into the quantitative data like I think I have just started some of the really applied economists bringing in some more of that qualitative data into their work. Every other social scientist is like mixed methods, which is like the combination of the quantitative and the qualitative. So it's interesting to like think back to economics and have it be like so ingrained in kind of that quantitative realm. I think that what I typically will say about the qualitative data is that it offers like the context and the humanity to whatever is showing up in the quantitative data. So whatever is kind of showing up in the numbers, like we get so much more understanding of that through kind of the experiences and the narratives that people share. So we can look at the numbers kind of, but then what story is actually underneath all of that. And that's what we get in the qualitative data. And I would say that I agree that that is definitely kind of a shift that is happening in research generally around moving towards more at least combining the qualitative data with the numbers. And I think that there's more and more argument that the numbers are just not sufficient. We don't capture enough information. We don't know what like what to do next if we aren't getting kind of the stories and experiences of people beyond just kind of yeah it worked or no it didn't work or here are the grades or here's the attendance data like we don't have a good understanding of what what that means like what the problems are if we other than like low attendance. Okay there's not a lot of places to go from there so then i think that's a very good cue and i think that shows up a lot in our work because we aren't just trying to say like, oh yeah, we've improved our numbers, but why? Why does it work there? How do we bring it somewhere else? Like that's part of the scaling work that we're doing in the continuous improvement process is like, what about it worked? And how do we bring that somewhere else? And we don't have that understanding if we aren't collecting more of that narrative experience data. 

 

Cameron Yee 23:56

I think my own experience just working in IT and sort of hearing people's experiences with, say, changes that we made. And I think at one time my thought was always like, is this an outlier? Is this experience an outlier? And then there's also this tendency to like sort of dismiss it if you deem that it is an outlier, because you're trying to meet the needs of a whole versus an individual. But I think what has been sort of refreshing for me is, in joining this team, is like, nothing is deemed an outlier. It's all considered in the mix of things. Does that seem accurate to you? 

 

Alyssa Leraas 24:43

Yeah, I think that's true. I think that can also be one of the challenges in this work is we are trying to capture so much information and so many stories impact in very different ways and it can be hard to kind of find, I mean as the communications coordinator, it can be hard to find like the through line or like what is the story that we are trying to share about our work because it can, because we are responding to individuals on such an individual basis most of the time with the information that we're collecting. And I think that that also lends itself to like, we don't talk about it as outliers and we are always trying to gather multiple perspectives. So even if we hear it from one person, like who else do we need to talk to? Like maybe that's the starting point to signal, like we need to do some more investigation on this. And we have the other side too, where it's like we get feedback all the time, that scheduling, like we'd rather have it during the day and have subs, we'd rather have it in the evening so that we don't have to worry about subs and get stipends like, and anybody that you talk to has a different opinion on that. And so there is a little bit of like, at the end of the day, a decision for the collective needs to be made. And how, how do we add variety? How do we do some of each that we are responding to kind of both of those needs? What are spaces where we can be more individualistic in our responses, which I think is really where our coaching comes in, like our coaches can be really responsive to individual teams and their needs. And so I think it's also a little bit of the balance, like cue for further investigation, maybe when we hear an anecdote, and ensuring that we're getting those multiple perspectives, and then what is the balance and kind of how that's used in decision making, I think is how I see it show up a lot in our work. 

 

Cameron Yee 26:44

Great. We are running out of time. Time flew, I don't know what happened. 

 

Alyssa Leraas 26:52

Shocker, I do love to talk.

 

Cameron Yee 26:55

Well, thank you for making the time to be here. I'm sure we could talk at length. There are things we didn't really cover, like y’know continuous improvement and what your experience was with that before joining the team, and also just continued conversation around data and accessibility of data, I think is an intriguing topic and how we make that both accessible and human centered. So I hope maybe there's a part two in the future, like continued data conversation. But thank you for being here and participating in this. Looking forward, we're gonna talk to one of our Coordinating Body members, Kanoe Bunney, who works for Linn Benton Community College. And so that'll be in the next segment. 

 

Alysaa Leraas 27:39

Yeah, thank you Cameron. 

 

Cameron Yee 27:40

You're welcome.

 

Cameron Yee 27:42

If you enjoyed this episode, please like, subscribe, or follow us on whatever podcast platform you're hearing this on. For additional information and related resources, follow the link in the episode description or browse our website at westernren.org. Thanks again for listening, and we hope you can join us for the next episode of the Bright Spots Podcast, Highlights from the Western Regional Educator Network.

 

Transcribed by https://riverside.fm