Episode 4 - Women & Tech Series: The Networking Dynamics of Women in Tech.

00:00:17:21 - 00:00:39:07
Jie Ren
Hi everyone. Welcome to my podcast, When Tech Meets Ed. This is Professor Jie Ren. Today I'm very happy to have Dr. Sofia Bapna from the University of Minnesota to join us to discuss a very important topic which is Women in Tech from the perspective of networking. So, Sofia, thank you so much for joining us today.

00:00:39:09 - 00:00:45:10
Sofia Bapna
Thank you Jie for inviting me. I’m so excited to be here and looking forward to our conversation.

00:00:45:10 - 00:00:51:06
Jie Ren
Thank you so much. So why don't you tell us about you to our audience?

00:00:51:08 - 00:01:22:02
Sofia Bapna
I'm a faculty and a professor at the University of Minnesota Carlson School of Management, and one of the areas of focus is my research around increasing women's participation in the workforce, especially in STEM fields. And this really has two branches. So one of them examines the effect of legislations. So we may have new maternity leave legislation. And how does that affect you know is it the intent is for it to be effective.

00:01:22:02 - 00:01:40:10
Sofia Bapna
but is it really effective? And the other is in areas where we know there are frictions, what interventions can be introduced to reduce those frictions and improve the outcomes for women. So that's the type of work I do.

00:01:40:12 - 00:01:58:08
Jie Ren
Okay. So I know that you have done, many research projects along these lines. Right. And then, first of all, I want to know what is the motivation that drew you into this domain? Let's start with that and then yeah, elaborate on the research project. Yeah.

00:01:58:08 - 00:02:20:19
Sofia Bapna
Yeah. So that's a really interesting question. And I've been thinking about it for a bit. Right. So it's not it's not one factor. I wish I could just say it was this one and that's what, you know got me started. So it starts a little bit with my background. I am, my bachelor's degrees is in engineering. So computer engineering and we were 2% women in my engineering school.

00:02:20:21 - 00:02:23:16
Sofia Bapna
2%. Yes.

00:02:23:18 - 00:02:47:18
Jie Ren
Okay. Just want to share some statistics to you. I, I went to a, University of tech related, was in China and then it's not 2%, but definitely is like low, I would say maybe 20 or 30% or even lower than that. That would be all the girls like women. Yeah. Compared to a counterpart in terms of the, the gender you know 2%.

00:02:47:18 - 00:03:09:11
Sofia Bapna
Yeah. And this is overall in the engineering school right? There are certain domains say computer engineering which had slightly more women, but still it was never more than 25% in any area. [Jie Ren
 Wow.] So that was one factor, which obviously shaped my understanding of the situation of women in tech. I’ve worked in tech for over a decade. I worked in tech consulting for over a decade.

00:03:09:13 - 00:03:31:05
Sofia Bapna
I've been an entrepreneur, and, also a manager. So in all of these different fields, women are underrepresented. And so to me, you know, I think I had a good understanding of some of the issues, concerns. I had a lot of, a lot of colleagues who have talked about, things that, you know, matter to them, differences that they saw.

00:03:31:07 - 00:03:53:15
Sofia Bapna
So there was that grounding. And then, close to when I was graduating from my big program, I saw a grant application, from the Kauffman Foundation and was a call for proposals. And this particular proposal talked about increasing women's participation in entrepreneurship. And I was like, wow, that's really interesting. Is so interesting and important question.

00:03:53:17 - 00:04:14:21
Sofia Bapna
And I also felt it was something that, you know, I, I felt I could contribute to that. I had, you know, I had some research ideas that, you know, potentially, you know, could fit the call. And so it was almost as if, you know, I had the background and there was this little trigger, which sort of, you know, nudge, you. Nudge me towards.

00:04:14:21 - 00:04:25:06
Sofia Bapna
[Jie Ren: Yes, yes.] And, and so then I put in the application and what I found, and got the grand, and what I found is that really

00:04:25:08 - 00:04:26:03
Jie Ren
Congratulations.

00:04:26:03 - 00:04:49:20
Sofia Bapna
Thank you, thank you. It was a long time ago, but thank you. Yes. You know, there's sometimes some things, you know, that there there are some things that shape the way you move forward. Right. And so that I think was one of the moments. And I found that people really resonated to the research. Right. Like women and even men like I have never had, you know, men tell me that this is not important, that this doesn't make sense.

00:04:50:00 - 00:05:12:04
Sofia Bapna
So I just found that there was a lot of, you know, people responding well to it. And then there was another really pivotal moment I feel. So this was when, so my coauthor, Russ Russell Funk and I, we have several papers together, and we were interested in exploring, you know, incubators and how they function.

00:05:12:06 - 00:05:27:20
Sofia Bapna
And I am a firm believer in engaged scholarship, which is about really asking questions around what's happening in the real world versus just sitting back, you know, and, looking at what the literature has to say and moving forward from that, you know, that that.

00:05:27:20 - 00:05:30:08
Jie Ren
On business context behind. 100%.

00:05:30:10 - 00:05:57:20
Sofia Bapna
Yeah, and not that the other approach is bad is just my style that, you know, maybe because I, you know, I've been a practitioner for over a decade. As I said, I worked in tech. So the first thing we did was to go and talk to a bunch of companies that, you know, incubator companies in Minnesota. And one of the places we went to, they said to us that, you know, we spend a lot of money on networking events, and we're not really sure whether there's any payback to that.

00:05:58:01 - 00:06:18:19
Sofia Bapna
You know, are there any useful connections being made here? And is this, you know, really worthwhile or just people just connecting with people they already know? And, you know, we are spending this money and time and there's no feedback. And so we went back and I looked into the literature like, what does research tell us about how networking evolves?

00:06:18:21 - 00:06:29:19
Sofia Bapna
And and then from there I realized, especially in fields where women are a minority, there can be very different outcomes for men and women at networking events.

00:06:29:21 - 00:06:31:15
Jie Ren
And I noticed that as well.

00:06:31:17 - 00:06:37:04
Sofia Bapna
You did. So tell me more. Tell me more Jie about what, where where have you. What context? Have you seen this?

00:06:37:04 - 00:07:08:18
Jie Ren
So umm, okay, so I, I, I like to attend networking events even before I graduated from the PhD program. Right. So I know that you know this as well. There is a website called Meet Up. [Sofia Bapna: Okay.] Right. So they could be like events, for example, in New York City surrounding certain topics, right? For example, technology topics, etc.. And then sometimes I would go there just to see like what is happening about the industry, etc. and also just to, you know, network with people. Strangers.

00:07:08:20 - 00:07:38:14
Jie Ren
And I notice that especially for the tech topic related events, in terms of the participation rates, across genders, and you definitely will see like more men than women, they are. That's number one. And then another thing is, like I see usually not not every time though usually you will see people are in clusters and then could be, you know, and then some men like dominating the conversations etc.

00:07:38:16 - 00:08:03:14
Jie Ren
And also you also see men are speaking up right. Ummm confidently and everything. But women are more shy in terms of listening to the conversation. So I noticed that. Right. I think part of that could be, okay, so since we are talking about this, right, I also want to hear your opinion in terms of, you said that 2%, right?

00:08:03:14 - 00:08:27:21
Jie Ren
That that is, so I was quite you know, I was like, oh, I understand that the percentage could be low, but 2%. So how do you feel to be the only or the few girls in that environment? Like, how did you feel? Do you feel like you are proving yourself, like you made it? Was I sometimes you feel like, I shouldn't be underestimated because of my gender.

00:08:28:01 - 00:08:59:22
Sofia Bapna
Yeah. So a lot of interesting things you said here. [Jie Ren: Yes.] So I want to actually start out with what you said about your experience at the conference. Right. So I'm so glad you said that, because that is exactly what I hear over and over again here. And the research that we have done around networking, which I'd love to talk about a little bit more in a bit, but that when when women hear about that research and the potential interventions, firstly they they feel validated because of the experiences you shared.

00:09:00:00 - 00:09:21:09
Sofia Bapna
And then they're like, oh yeah, you know, those interventions actually make sense. And and also be sure, you know, in a very rigorous way that they do make sense. So, so I wanted to respond to your first part about your experience, which I thank you for sharing that. And then, you know, it's really interesting.

00:09:21:11 - 00:09:46:03
Sofia Bapna
I when I was in undergrad, I don't know, maybe I was naive or maybe you just take life for granted. I really didn't notice that this was, you know, this disparity was a thing that it can impact outcomes, that there are systematic differences. You and I just went about life, right? I was in college. I was doing my thing.

00:09:46:03 - 00:10:07:01
Sofia Bapna
I was having a fabulous time. I had great friends. I was going about my team right at that time. But then over time, you begin to realize that you know that there are differences. There are, you know, as you said, you know, you go to events. There's differences in the way women FSE versus men, there's differences in career progression.

00:10:07:01 - 00:10:13:15
Sofia Bapna
But at that time specifically, I don't think I was conscious about it.

00:10:13:17 - 00:10:38:15
Jie Ren
Okay. So, I know that we are going to talk about the networks, uhh networks. The networking perspective. And then especially in that very context. Right. You have way less women back again in undergraduate studies, way less women than men. Do you think that will also help you in a way, in terms of you are forced to connect with like the male’s networks?

00:10:38:17 - 00:10:40:02
Jie Ren
Do you think that way?

00:10:40:04 - 00:10:40:13
Sofia Bapna
Yeah.

00:10:40:14 - 00:10:52:12
Jie Ren
Meaning like all folks, I’m all female school versus a mixed school, I think, like, except here we have the the ratio is not like a perfect balance in terms of 50/50. But you know,

00:10:52:14 - 00:11:03:13
Sofia Bapna
Okay. So you brought up something really interesting. You said all female school versus you know, weight loss is a coed, right? I have so many, so many things that I want to say yes to this.

00:11:03:13 - 00:11:06:13
Jie Ren
[Inaudible]

00:11:06:15 - 00:11:41:04
Sofia Bapna
But since you mentioned that all female. So this is a really interesting study where, uhh this is, Belliveau . And what she studied, studied was exactly what you said, which is, you know, all, you know, graduates from uhh, female colleges versus, coed colleges. And, of course, she controls for a whole bunch of factors, like qualifications, majors, you know, jobs, types of jobs they applied for, you know, salary, expectations, everything.

00:11:41:10 - 00:12:14:17
Sofia Bapna
And what she finds is that women coming from all female colleges, actually, received lower job offers, like, as in lower salaries, the salaries were significantly lower than women coming from coed colleges that in terms of their job offers and, what the study suggests is this was because of the perception of your network, not as not your real network, because she actually measures the real networks.

00:12:14:18 - 00:12:43:06
Sofia Bapna
The fewer advised networks, the, the networks, you know, that they had in terms of people already in the profession, both male and female, all of that. Right? So she control it was not your real network, but the perception of your real network, which made employers, feel that women didn't have, you know, accurate information about going salary rates and hence they were offered significantly lower salaries when you came from an all women's college. [Jie Ren: Very interesting.]

00:12:43:06 - 00:13:03:08
Sofia Bapna
Yeah. So that was, a really I thought, it's a small study, in terms of sample size, but I think a really well-done study, especially because they consider all of these different factors. Right? The actual networks and, you know, the jobs, the, umm the qualifications, all of that. And yet they find this difference.

00:13:03:12 - 00:13:32:03
Jie Ren
That's really interesting. This truly is not really about the actual network and how much effort that it can put there to build your actual network is also people's perception, the employer's perception of you. [Sofia Bapna: Yeah, yeah.] Okay. So, before we move to the networking part, right. Maybe let's spend a little time to talk about qualification. Right. For a person's success, for example.

00:13:32:05 - 00:14:02:11
Jie Ren
Right. And then we talk about like women being shy. May maybe in some, some women could be shy and then maybe there could be like socially shaped, etc., etc.. Do you think that, you know, in addition to the networking parts, what advice that you would give to women in terms of be more active and then in terms of maybe forming the networks and then believing in yourself, don't feel like you are underestimated, you can do anything, you know, that kind of like to cheer them up.

00:14:02:12 - 00:14:24:13
Sofia Bapna
Yeah, yeah, yeah, I know, I love that comment. And, the thing is that what research tells us is that women tend to be a little bit more hesitant in terms of reaching out. And, they believe that, you know, networking for a certain outcome feels a little insincere. I'm not talking about a specific woman, but women in general.

00:14:24:14 - 00:14:26:02
Sofia Bapna

00:14:26:02 - 00:14:30:07
Jie Ren
Across societies, not necessarily in one culture. It could be very much across societies.

00:14:30:09 - 00:14:37:15
Sofia Bapna
Yeah, yeah. This is, I mean, maybe there are certain cultures where, you know, this is not true, but in general, like.

00:14:37:16 - 00:14:38:14
Jie Ren
Yeah, yeah.

00:14:38:16 - 00:14:46:20
Sofia Bapna
And, so, so they feel a little bit hesitant in terms if it's insincere. And then the other aspect.

00:14:46:20 - 00:14:50:07
Jie Ren
Insincere in what way? Meaning? So, I'm doing sectional like now, like?

00:14:50:11 - 00:15:20:18
Sofia Bapna
I have an ulterior motive, you know, like, I'm not doing this, you know, or there's a motive, and hence it feels insincere. And the other thing is that, again, this is what research shows is that women feel that they need to exhibit reciprocity in their, in their networking exchanges. So when they approach very senior people, they feel like they have less to offer, and hence they feel reluctant to do that right,

00:15:20:18 - 00:15:40:10
Sofia Bapna
to do this type of networking because they feel what they have to offer is less. And I'm not saying, you know, this is right or wrong. This is just what research tells us, right? This is what this is how you know, whether they're I don't think they're doing it consciously, but this is how it, umm this how they feel.

00:15:40:10 - 00:16:00:13
Sofia Bapna
And hence it has certain effects on the outcomes. Right. If you're less if you're more reluctant than men to reach out to senior people, then the outcomes are also going to be different. Right? And so now, you know, a lot of conventional advice, many, many people would say that women should lean in, they should do more, they should put themselves out there.

00:16:00:15 - 00:16:12:06
Sofia Bapna
And I'm not disagreeing with that. Right. That's one approach. But is that approach right for everybody? Right? You know, some women, even if you tell them it's hard.

00:16:12:06 - 00:16:13:05
Jie Ren
Is it is hard.

00:16:13:05 - 00:16:14:03
Sofia Bapna
Yeah, it is hard.

00:16:14:03 - 00:16:20:13
Jie Ren
People pleaser. Right. Etc., etc. very much relevant to like a women's community.

00:16:20:13 - 00:16:43:17
Sofia Bapna
Yeah, yeah. And maybe, just maybe those hesitations are still there. Right? And then they have to overcome them and, you know, push forward. So yes, I'm not I'm not saying that that advice is not good, but it's it's been around. But I think what we need to keep in mind is that organizations can do something right. Companies and organizations can do things to make things easier.

00:16:43:17 - 00:17:08:19
Sofia Bapna
So, but before I go into the such as, I really want to talk about why? Why is it that we want organizations or companies or people organizing these events to do something different? Because I feel, not I feel like factually, I believe and the research shows that there are actually structural barriers that affects women's networking outcomes.

00:17:08:21 - 00:17:31:19
Sofia Bapna
And I want to talk about them, because then that sets us up to understand why it's not enough to tell women, go and lean in, go and do more. Put yourself out there. Right? Because once we understand these structural barriers, then we can say, okay, given this situation, these are the types of, you know, such as, you know, interventions that they can introduce.

00:17:31:21 - 00:17:56:13
Sofia Bapna
Yeah. So what I want to talk about two things. Right? So the first one is what would happen when we go to a event, where we barely know anybody? Okay. We don't know anybody. There's uncertainty. There's ambiguity. Like who should I talk to? What should I say? And some people are extroverts. I mean, talking about people in general. You, you come to an event and there's, you know, okay.

00:17:56:18 - 00:18:09:20
Sofia Bapna
Now what? Right. And so what we do naturally, right? We not thinking about we just doing it naturally is that we gravitate towards people like ourselves.

00:18:09:22 - 00:18:10:18
Jie Ren
Similarity.

00:18:10:22 - 00:18:33:12
Sofia Bapna
Similarity. Right? And we're not even thinking. But we do that. And so what's going to happen is that women are more likely to talk to more women, they'll talk to men to, but in the conference, you're likely to talk to more women. Men are likely to talk to more men. So now, what? Why? So, you know, that's fine.

00:18:33:16 - 00:18:53:12
Sofia Bapna
But why should it matter? Right. So I want to talk about the two structural issues that come into play here. Okay. The first structural issue is that women out of mind are the minority in many fields, such as tech. Right. So we said we talked about this 25% number. So if you're talking to more women you're talking to a subset

00:18:53:12 - 00:19:26:19
Sofia Bapna
and a 75% there that you are talking less too, right? So you're talking in this little cluster. And then, the other thing is that, in fields such as tech and many other fields, men tend to occupy the more senior positions. [Jie Ren: Yes.] So now when they're more senior and they are the majority, and if you are speaking to the smaller, are more likely to speak to the smaller group of women who are less senior then your outcomes from that event are also going to be different, right?

00:19:26:21 - 00:19:43:05
Sofia Bapna
Because if you're more senior, then you tend to be more connected, more influential, have more information about opportunities. And, and so the outcomes for men and women are likely to be different from these events.

00:19:43:07 - 00:19:54:15
Jie Ren
Just because of the similarity, because of similarity along, for example, this lens of gender. [Sofia Bapna: Yeah.]

00:19:54:15 - 00:20:18:16
Sofia Bapna
[Jie Ren: Massively different.] Yeah. And I want to talk about one more pathway and then also tell you how different these outcomes can be through our research, which we have examined. Right? So the other pathway is so now we're talking about situation where you almost know nobody at the conference. Then think about another situation where you know, say a few people say, you know, 25% or one third, right?

00:20:18:18 - 00:20:37:06
Sofia Bapna
And what would what would happen then? How would that unfold? Is that people once again try and, they don't do it consciously, but they gravitate towards people they already know. And again, given what I said in the first place, how do networks form, women's.

00:20:37:09 - 00:20:41:17
Jie Ren
Similarity, similarity first and then the second one will be familiarity.

00:20:41:19 - 00:21:09:10
Sofia Bapna
Yeah. So yeah. So so then so this is a familiarity. Right. But because you have formed your networks earlier which have more women. So with the familiarity you're again going to connect back with the women that you know because your network has more women. Right. So from both pathways now, you're connecting to, to more women, which means, again, that your outcomes are likely to be different because men are better connected, more senior.

00:21:09:12 - 00:21:35:23
Sofia Bapna
Right. And this, an interesting paper by Ingram and Morris which examined. So, they took a bunch of, you know, executives, these were managers, bankers, consultants, entrepreneurs, 100 of them. And they asked them before a mixer or post-work mixer, you know, what is your goal for this event, right? Is it meeting your contacts? Meeting existing contacts?

00:21:35:23 - 00:21:58:19
Sofia Bapna
You know what what what are you thinking about? And they all overwhelmingly said it's about making new connections. And they had them all where an electronic tag name tag, and do whatever they did in the mix, and then examined who did they actually connect with? [Jie Ren: Okay.] They found that overwhelmingly they actually connected with people they already know.

00:21:58:21 - 00:21:59:22
Jie Ren

00:22:00:00 - 00:22:13:10
Sofia Bapna
So again, it's not just women. Everybody does this naturally. So why? Because it just takes so much more work. To, you know, it is work right? To now get to know somebody. It's cognitive work.

00:22:13:12 - 00:22:13:18
Jie Ren
Yeah.

00:22:13:18 - 00:22:17:18
Sofia Bapna
And also, you don't know how you're going to be received.

00:22:17:20 - 00:22:20:09
Jie Ren
By unknown, right? Sometimes can be terrifying.

00:22:20:11 - 00:22:48:18
Sofia Bapna
Yeah, yeah. So now those are just, you know, those are the structural pieces at play, right? The seniority, of men and the, you know, the fact that they are the larger proportion in that in that field, in most fields, such as tech, right? So now, if you take that forward, we wanted to say, okay, I’m going to connect it back to my incubator example where you said that, you know, is this paying off or not?

00:22:48:20 - 00:23:21:09
Sofia Bapna
And then we, you know, we have these structural issues. So, Russ, Michael Russell Funk and I, we wanted to examine firstly, so we want to confirm that are women's networking outcomes different or not. And then men's when they go to a majority men conference. And then for us that was not enough. Right. We wanted to examine what would be the effect of interventions that we designed to change these barriers.

00:23:21:09 - 00:23:22:16
Sofia Bapna
And these interventions.

00:23:22:18 - 00:23:39:11
Jie Ren
These are the equivalent, like the intervention can translate to the equivalent of organizational support or environmental support. [Sofia Bagna: Yeah.] Like one organization could in implement in order to foster the networking the better networking for women.

00:23:39:11 - 00:24:01:18
Sofia Bapna
Exactly, exactly. So what can organizations do is exactly start. [Jie Ren: Exactly right.] That was your earlier question. So that led to, you know, okay, what are these interventions or what can organizations do and what can they do in an event to make these outcomes different for women? Right. So, so what we did is, we obviously incubators are really small.

00:24:01:20 - 00:24:34:09
Sofia Bapna
And the events, at least the ones that we connected with, tended to be small. Not really a good place to examine this, uhh question. So we partnered with an organization that runs conferences, and they run conferences in big data and analytics, AI type of conferences, really large conferences, close to a thousand people each. And, and what we did is we said to them that, you know, firstly we connected because they also want to foster connections.

00:24:34:09 - 00:24:40:09
Sofia Bapna
Right? They want their events to be successful in terms of people meeting each other.

00:24:40:09 - 00:24:42:02
Jie Ren
Mutually beneficial.

00:24:42:04 - 00:25:12:10
Sofia Bapna
Yeah, exactly, exactly. And so, essentially work with them. And what we said was that, before the conference, the conference organizer would, send out messages to the attendees. Okay. And, we took the attendee pool this close to a thousand people. We randomly assign them to three buckets. Okay. So one bucket we call the control. And then we had two others where we had one where one of them was a search intervention.

00:25:12:12 - 00:25:35:12
Sofia Bapna
And I'll explain that in a minute. And the other was a social intervention. And the idea here was, and of course, that both men and women are in these buckets. Right. And just like any, any conference in tech and, you know, even the proportion of women in tech which is reflected in conferences in tech, they were about 25% women,

00:25:35:13 - 00:25:39:02
Sofia Bapna
right? So now we had both men, men and women assigned into these three buckets.

00:25:39:02 - 00:25:39:15
Jie Ren
Randomly assigned.

00:25:39:15 - 00:26:04:14
Sofia Bapna
Randomly assigned to be super important. Yes? [Jie Ren: Yes.] Little detail there, but super important because that ensures that we don't have any selection into one of these buckets. And they were not aware that we were studying them. Right. As part of this, you know, their, their networking as in this conference. So, so there was no active observer bias in terms of, okay, if you know that you're being observed, you may behave in a certain way, right?

00:26:04:14 - 00:26:16:17
Sofia Bapna
So the random randomly assigned to these three buckets, one bucket was a control. And the conference organizer to the control sends an email saying networking is important. Please remember to network.

00:26:16:19 - 00:26:17:14
Jie Ren

00:26:17:16 - 00:26:42:14
Sofia Bapna
And here really what we wanted to examine is organically what is a differences in outcomes for men and women in terms of networking to confirm that, you know, these structural problems that we talked about and also the other barriers such as, you know, your hesitation to network? [Jie Ren: Yes.] Things like that, you know, do they have do they have effects on the outcomes for men and women?

00:26:42:16 - 00:27:08:03
Sofia Bapna
And, given this prompt, we examined several outcomes. So firstly, we examined post outcomes set of post conference self-reported outcomes of networking. So the organizers sent and you know, always sends a survey after the event. And in their survey, the embedded a few questions related to networking. So one was how many new contacts you need, how many minutes on average would you spend with each new contact? How many, how many existing contacts did you meet?

00:27:08:05 - 00:27:31:16
Sofia Bapna
How many minutes on average did you spend with existing contacts? Then we also, apart from self-reported, we tracked these people on LinkedIn and we saw, and their connections on LinkedIn. We observed their connections on LinkedIn,

00:27:31:18 - 00:27:42:08
Sofia Bapna
and we did another thing where we actually tracked them, for a year after the event [Jie Ren: Mmm okay.] And job change for a year. 

00:27:42:10 - 00:27:45:13
Jie Ren
To see the real effect of the networking?

00:27:45:13 - 00:27:52:05
Sofia Bapna
Yeah and especially of our interventions, because we wanted to see how our interventions affect, you know, all of these outcomes. Right?

00:27:52:05 - 00:27:53:02
Jie Ren
Mmmhmm, especially long term.

00:27:53:04 - 00:28:01:01
Sofia Bapna
Yeah. Yeah. And I'm going to get back to long term. Remind me, because I have something interesting you say about long term later.

00:28:01:02 - 00:28:11:02
Jie Ren
Because we want to know the cumulative advantage as the starting point to, you know, how much cumulative advantage you can get if you just adjust how your network at the beginning. [Sofia Bapna: Yeah.] Yes.

00:28:11:04 - 00:28:35:12
Sofia Bapna
So, so what we are so basically the control we find on all of these outcomes, self-reported as well as the ones on LinkedIn, the men did significantly worse than women. So this confirmed what we were trying to say. Exactly. So now that really sets us up to say, let us talk about interventions. I mean, there is there is a difference that we show.

00:28:35:14 - 00:29:14:05
Sofia Bapna
And now and the literature suggests it, but we are actually showing it empirically that this is happening. So now let's start thinking about interventions. Right? And so let me just start with a research intervention. Now this intervention is motivated by the idea that, you know, if you are networking with, you know, just women who are less senior, you know, you may have, you may be less likely to get information about important opportunities or resources or diverse opinions on something that you're working on, which could be a new solution.

00:29:14:07 - 00:29:43:05
Sofia Bapna
So, we call this a search party or the search for useful or helpful information. And what we did here is so, (inaudible) you were at the conference and, you know, you were given to this group. The second bucket search intervention was given, sent an email with 16 recommendations of people they could connect with. So you might have received me, a bunch of other people. Right, to connect with.

00:29:43:07 - 00:30:12:00
Sofia Bapna
So you got 16. But what was really interesting about those 16 is that each of us were connected to 16 others. So that meant you were one degree away from 16, but two degrees away from 256. [Jie Ren: Yes.] So now we've suddenly shrunk the network. So if you came up to me and said, you know, I'm really interested in such and such, I'd say, wow, I just met so-and-so. Who's doing exactly that?

00:30:12:02 - 00:30:39:02
Sofia Bapna
And can make that connection between YouTube. So we essentially shrunk the search network, and, wanted to see whether it's, it's, you know, is it because women are having difficulty finding useful connections? The other one we did was, to try and reduce social barriers. This was called the social intervention. So here we the the email again.

00:30:39:02 - 00:30:47:13
Sofia Bapna
It sent them 16 connections but these 16 connections only received each other. So.

00:30:47:13 - 00:30:48:15
Jie Ren
It was close network?

00:30:48:15 - 00:31:03:18
Sofia Bapna
It was close network of 16. And so everybody in that 16 received the same 16. And we said, you’re part of a group. Okay. So what this meant was, without thinking, that group identity would kick as.

00:31:03:21 - 00:31:05:14
Jie Ren
Belonging to the community.

00:31:05:18 - 00:31:39:14
Sofia Bapna
Yeah. Because even, just assume you assigned people to a blue team and a red team and suddenly you're like, okay, you know, I'm going to help my blue team, right? Like, because I'm in this team even though I don't know this. Right? So this is my group. And so what would happen is say it would, you know, help remove some of the frictions related to, say, post MeToo, where men and women, you know, feel that they may not want to spend too much time with each other for fears of how it might be perceived or how it might look.

00:31:39:16 - 00:32:03:00
Sofia Bapna
Right? So now there's a legitimate reason for me to interact. Or, you know, somebody may feel very comfortable going in approaching a senior. That hesitation goes away because it might feel like, okay, you know. [Jie Ren: We are a team now.] We assigned to this group and you know, I'm here, you know, to chat with you. Right? This is legitimate reason I wanted to reduce some of the social frictions.

00:32:03:02 - 00:32:34:09
Sofia Bapna
And what we found very interestingly, was that the search intervention age, basically brought women's outcomes, networking outcomes for all the outcomes that I discussed, self-reported number of new contacts, met, time spent with new contacts, then the LinkedIn, you know, contacts, on LinkedIn, job change for you, all of them to the level of men's and obviously significantly higher than the control group.

00:32:34:11 - 00:32:42:10
Sofia Bapna
Women in the search intervention, a significantly higher than women in control. So so that tells us something, right?

00:32:42:12 - 00:32:43:14
Jie Ren
Yes.

00:32:43:16 - 00:33:03:17
Sofia Bapna
That really, you know, if we reduce a search intervention, then search the search barriers through an intervention such as, ours, that it can significantly improve outcomes for women. And so obviously, the other interesting question is what happened with your social intervention?

00:33:03:19 - 00:33:06:11
Jie Ren
I want to hear the rest of the story. So, please.

00:33:06:12 - 00:33:28:20
Sofia Bapna
So the social intervention here, what we found is that, women spend significantly more time, that is, if you were in the social intervention, you spend significantly more time with new connections than women in control. So you were part of a group. So, you know, you spent more time with people in your with people with those connections. Right.

00:33:28:20 - 00:33:29:18
Sofia Bapna
Which makes logical sense.

00:33:29:18 - 00:33:34:07
Jie Ren
Like mentoring in this new relationship [Sofia Bapna: Yeah so.] with each member of this group. Yeah.

00:33:34:07 - 00:33:58:02
Sofia Bapna
With members of the group. Right. So you, you spend more time with them because you’re part of the group. But because of the way that those connections with structure. But you had 16 people connected to each other versus having that second degree of 256 where you met more, they could potentially meet more useful connections. So that pathway led to a different outcome.

00:33:58:04 - 00:34:04:21
Sofia Bapna
And again, it depends on what you're trying to do. But these are useful. Both of them are useful interventions.

00:34:04:23 - 00:34:33:11
Jie Ren
Can I ask a question in terms of these two treatment groups. Right. So you have the third group and the other one’s social group, but they generate different, as you said, different outcomes. One probably is about how many people you get to connect to through your, your connection to start with or through your connections. Connections. The other so the first one is to reach the second one probably will be the strength of the tie.

00:34:33:13 - 00:34:52:05
Jie Ren
Right. So like if you want to get closer to each member of this team, maybe that would be the second one. Even though the in terms of the performance indicators, the third one is oh, so the second treatment group, the second, the social is not behaving so well compared to the.

00:34:52:07 - 00:35:14:06
Sofia Bapna
Yeah. Such. Yeah, yeah. It's possible. It's it's possible right. Like that that these perhaps stronger ties could lead to different outcomes over time. So we don't want to discount them. It's just a different, different result from that intervention. Right. And so interestingly, when you mentioned long term long term.

00:35:14:07 - 00:35:15:15
Jie Ren
Long term, yes.

00:35:15:17 - 00:35:21:07
Sofia Bapna
So this year is a ten year since we ran that that study.

00:35:21:09 - 00:35:25:07
Jie Ren
That was the longitude longitudinal, in fact?

00:35:25:08 - 00:35:50:12
Sofia Bapna
So, I’m interested in going back and seeing what is a longitudinal effect. Right. And also, you know, to your point, maybe the social intervention had different long term effects because we know what companies those connections were at right. Each connection. So we can also look at people's movement across, you know both interventions your connections first degree but second degree,

00:35:50:14 - 00:36:06:07
Sofia Bapna
your connections in this, in that little flow, this group. So I'm really excited about the potential for this, you know, to to go back and look at that data, not in terms of long term effects, like what's what's happened, you know, after ten years.

00:36:06:10 - 00:36:12:16
Jie Ren
Is a quality of the relationship or is it the quantity of the connections?

00:36:12:19 - 00:36:18:17
Sofia Bapna
Yeah. And indeed we do know that the search intervention did it did help immediately. Right? [Jie Ren: Mmm yes, yes.] Like we did

00:36:18:17 - 00:36:19:05
Jie Ren

00:36:19:10 - 00:36:27:03
Sofia Bapna
see those immediate impact. Right. So now it's about like what what is it long term for each of them. So super excited.

00:36:27:05 - 00:36:52:19
Jie Ren
Super, is super exciting. I'm so looking forward to next time. Maybe we can, like, chat again on this on this topic. So, given this fantastic research, can you also translate that into, you know, the individual level effort in terms of what each woman could do to learn from it? The second one, because, the intervention that you said is very much about the organizational support.

00:36:52:20 - 00:37:00:18
Jie Ren
Right. So how can an organization, to turn that into their own intervention?

00:37:00:20 - 00:37:27:21
Sofia Bapna
Yeah. So I have actually so much that I want to say. [Jie Ren: Yes.] Okay. I'm going to start with, super interesting study by, Blue and Cane where what they do is, they examine the effect of a search. It's for a consulting company where they introduced a tool through which people expertise search tool, so people could search for expertise within the company

00:37:27:23 - 00:37:56:18
Sofia Bapna
and what they found is that, again, remember the connection, the word search, expertise search and the search. You know, the search intervention that I had. So when you when people could find, you know, experts they women benefited significantly more. So again, it was a matter of the way their networks lay. Right. They didn't know how to find the expertise within the organization.

00:37:56:20 - 00:38:28:10
Sofia Bapna
And this is a consulting company. And talking about 2008, $2010. And what they found is after the introduction of this tool, because they were able to connect with not only more people, but make more diverse connections in terms of finding diverse knowledge within the organization. But they measured the, actually their billing increase. So in 2008 and $10 women's billing per month increased by $1,000 a month.

00:38:28:12 - 00:38:37:21
Jie Ren
Just to increase the visibility. Yeah. With expertise from the like the diverse group. Yeah. Then you would see, like, more billable hours.

00:38:37:21 - 00:39:00:14
Sofia Bapna
Yeah, yeah. Because they could find the knowledge within the organization. Yes. So you know that that I thought was amazing. [Jie Ren: Yeah, it is amazing.] To do have, you know, a tool that can have this type of an effect. So that's one, the other one, I would say, is very related to the, the conference, interventions that we designed.

00:39:00:14 - 00:39:23:01
Sofia Bapna
Right. So you could do exactly the same thing. And, you can find that, you know, the details of that design in my, missed quarterly publication. And, you know, you could do that where, you know, you have 16 connections or whatever may be the right number, depending on the size of the conference. And then they connected each each of those people are connected to, you know, a different set of 16.

00:39:23:03 - 00:39:26:20
Sofia Bapna
Or you could have those groups, but I'm going to suggest two more.

00:39:26:22 - 00:39:27:23
Jie Ren
Yes.

00:39:28:01 - 00:39:59:14
Sofia Bapna
Okay. These are far simpler because that takes a little bit of design. Right. You know, the 16 plus a 16. And, it requires a bit of time, but 2 super simple ones, which is not going to consume too much time. So one of them I've been running at, one of my annual conferences for the last three years, workshop and information systems economics. Uhh, I received really good feedback, by both men, women, senior people, junior people.

00:39:59:16 - 00:40:24:15
Sofia Bapna
So the intervention is simple. What happens is within the conference agenda, we, I talked to the organizers and managed to get anything between an hour and an hour and a half, whatever they able to give me, I take it. But the one thing that I ask is that there's no parallel sessions. So it's during the day, not the end of the day, because otherwise it feels like, you know, I'm tired.

00:40:24:15 - 00:40:40:04
Sofia Bapna
Maybe I should, you know, go home or change or go back to my conference room or whatever, maybe, you know, so during during the conference event, during the day, and no parallel sessions, because then you're not conflicted about, should I do this or this? Right?

00:40:40:04 - 00:40:43:09
Jie Ren
We're not distracted by other, other events.

00:40:43:09 - 00:41:04:07
Sofia Bapna
Yeah, yeah. Because now, you know, you're you're trying to figure out what what is better or worse, right? So those are the two things I ask for. And so it's and it is in the conference agenda, you know, marked as whatever you want, you want to call it, structured networking or whatever name, but networking event.

00:41:04:09 - 00:41:06:00
Sofia Bapna
And then what happens is.

00:41:06:04 - 00:41:08:18
Jie Ren
And that is open to all people, not just women?

00:41:08:20 - 00:41:12:12
Sofia Bapna
Yeah. All people, because that's really important. Yeah. Like, you know, we talked about the need for.

00:41:12:12 - 00:41:13:01
Jie Ren
Like for.

00:41:13:01 - 00:41:37:14
Sofia Bapna
These diverse ties. Right. And so this is useful not just for women though as you’ll hear. Yeah. Like it's useful for both senior junior you know women and you know it's it's it's I think it has broad applicability I agree. And so so basically what I do is, then I have an advance set up at the venue tables with numbers.

00:41:37:16 - 00:41:57:21
Sofia Bapna
And as people enter, each one is given a card and the card has two numbers on it. This is a table numbers. And these table numbers are randomly assigned. So when you walk in you don't know what table number you're going to receive. So it's not as if in advance. You know somebody can say I really want to meet such and such.

00:41:57:21 - 00:42:02:16
Sofia Bapna
Can you assign me to this table because I don't know who's going to until when, in which table they'll be at. Right. So it's truly.

00:42:02:16 - 00:42:03:23
Jie Ren
Random.

00:42:04:04 - 00:42:25:17
Sofia Bapna
There's some sort of serendipity here, right. And so and also I, I, you know, design it so that you have about six or so people, you know, in each table, of course, how to predict exactly how many people are going to come to the event, but around that many people per table. So I, I, you know, the numbers are such that you have about that many people per table,

00:42:25:19 - 00:42:44:22
Sofia Bapna
the assignment. They come in, they get the card, they go and sit on the table and, you know, they meet these other people who randomly show up at the table. The tables are all numbered. And then the back of the card has a few prompts. So the first prompt is an icebreaker. Something like if it's in a different city.

00:42:44:22 - 00:43:14:20
Sofia Bapna
We had it once in Thailand, this conference. So we said, you know what is something interesting you did in the city your planned to do? Then there’s two questions that we have. One is tell us about a this is a research conference. But you can change it to your context a research project that you're really excited about. And the next one is tell us about some research or teaching related that you want advice or help on.

00:43:14:22 - 00:43:26:13
Sofia Bapna
And so, you know, then, you know, after a point in time, depending on how much time is allocated, I announce, and then they move on to the next, they move on to the next table number in that card.

00:43:27:07 - 00:43:52:21
Sofia Bapna
And so the feedback that I receive overwhelmingly is for juniors. It legitimizes talking about their research. Right. So I can go up to somebody and say, you know, because I can't just go to a conference, find somebody and just start talking about my research. Right? Yeah. But now you're at this table. It's an opportunity for you to talk about your research or talk about something that you could do.

00:43:52:23 - 00:44:13:21
Sofia Bapna
You know, that that way you could. You need some help or some ideas. You know, something that, you're trying to solve? Right. So it legitimizes being to, you know, being able to talk about this to all these people whom you couldn't just approach randomly and, you know, start, start this conversation.

00:44:13:22 - 00:44:17:15
Jie Ren
It does help your participants, especially the junior people.

00:44:17:19 - 00:44:39:20
Sofia Bapna
Exactly. And, you know, they could be senior people in the table who’s who, who might say, or even not senior people who might say, you know what? If somebody in my department is studying something really close, let me make that connection for you. Right? And then senior people. So I’ve heard things like, you know, I really connected with people who I wouldn't usually talk to right.

00:44:39:22 - 00:44:52:18
Sofia Bapna
Why people, senior people, have been going to these conferences for a long time. They tend to have their networks and they may not be doing it on purpose, but they may just end up talking to people they already know. Right?

00:44:52:23 - 00:44:53:19
Jie Ren
Very much so.

00:44:54:00 - 00:45:17:14
Sofia Bapna
Yeah. So we talk about that familiarity. Yes. And so they are in their little circles right. Yeah. But now this allows them to make, you know, those connections that are you that you know that they enjoy and hear about, you know, what people are doing. Right? And so they've said to me that, you know, it was really great because I got to, you know, talk to these people whom I wouldn't have otherwise spoken to.

00:45:17:16 - 00:45:42:14
Sofia Bapna
And the other thing that was interesting is one of the seniors told me that, you know, I actually got to meet somebody. I was planning to meet for a long time, but somehow, you know, in, in my, in, inbox, the, you know, that person's messages got buried, buried. I forgot about it. And now suddenly, that person was at my table. And so people are busy, right?

00:45:42:14 - 00:45:57:20
Sofia Bapna
Especially seniors. And so this. It's not bad intentions or neglect or whatever. So this provides those types of opportunities. So I think super easy intervention. Right. To implement.

00:45:57:22 - 00:46:12:08
Jie Ren
Just to have that awareness right for companies, organizations as well as individuals right to know that the power of networks and and purposely do the intervention, as you said, or purposely from the individuals level that could reach out.

00:46:12:10 - 00:46:12:19
Sofia Bapna
Yeah.

00:46:12:22 - 00:46:14:11
Jie Ren
To especially new people.

00:46:14:11 - 00:46:31:05
Sofia Bapna
Yeah. And then I think, you know, if an event such as this is being organized, I would say the advice is make sure you go to it. Right. And you know, so and some of the feedback I received was what was also like I didn't expect it, but it was actually fun, you know. So you have fun. Yeah.

00:46:31:09 - 00:46:33:11
Jie Ren
Serendipity. Right. So that's very fun.

00:46:33:12 - 00:46:50:02
Sofia Bapna
Yeah. Yeah. So you know it can be fun to do. Right. So the other, other really interesting, intervention. I don't want to keep using that word, but I'm trying to think of a better word for it.

00:46:50:04 - 00:46:51:13
Jie Ren
Organizational support, how about that?

00:46:51:15 - 00:47:24:22
Sofia Bapna
Yeah, sure, sure. That's a nice one. Yeah. Don’t you know, conference support that. Conference support that you can provide during events? Yes. Is, this is in this article in nature, which is written by, I think the 6, 5 or 6 women who, from different research fields, all experienced networking barriers at conferences and events. And they, instituted something and that they wrote about, which they called speed geeking.

00:47:25:00 - 00:47:26:23
Jie Ren
Kind of like speed dating.

00:47:26:23 - 00:47:27:21
Sofia Bapna
Exactly.

00:47:27:23 - 00:47:28:07
Jie Ren
Okay.

00:47:28:08 - 00:47:30:10
Sofia Bapna
Exactly. They say it's inspired by it.

00:47:30:10 - 00:47:31:15
Jie Ren
Tell me about it.

00:47:31:17 - 00:47:52:17
Sofia Bapna
It was inspired by speed dating. They they say that in the article and here the idea is that they assign. So basically they have tables and say about 8 to 10 people per table. And then there are geeks who are experts equivalent of your conference. You know, the people who speak at conferences, maybe they're sharing their research. Right.

00:47:52:17 - 00:48:15:11
Sofia Bapna
So your speakers, either the geeks and to one geek is assigned to each table, and then the geek speaks about their area for five minutes. In their case. But you can decide what makes sense. And then the, the group, the 8 to 10 people, there's a Q&A session for five minutes. Okay. And then so after that ten minutes.

00:48:15:12 - 00:48:42:10
Sofia Bapna
Yeah. The geek moves to the next table. So these geeks move around to all the tables until the until all the geeks have circulated through all the tables in the room. So essentially what this allows is it allows access to that expert within that table and a really conversational dynamic. So it changes the way, you know, firstly changes access it.

00:48:42:10 - 00:49:03:11
Sofia Bapna
It also provides an opening point. Right. So now you are there. You you know, you spoke about it. You and this little table. Then you know, there's another chance later you can pick up on the conversation. So I thought that was really neat. What did you call it? You gave it a good name. [Jie Ren: Organizational support.]

00:49:03:12 - 00:49:04:17
Sofia Bapna
Yes, yes.

00:49:04:19 - 00:49:28:02
Jie Ren
I think that that one, the the geek, what's that called? It's now like, the speed dating speed speed geeking. Sorry. I was like, trying to. [Sofia Bagna: Yeah.] That make that metaphor? It's it reminds me very much of the, the, a search condition that you had, right. In a way that is increasing the quantity of your contacts.

00:49:28:02 - 00:49:34:04
Jie Ren
You are forced to talk to that person, but also it has a little flavor of the social condition that you had.

00:49:34:04 - 00:49:46:07
Sofia Bapna
Yeah. Yeah, I like I like the connection that you made to the past. Right. Because now. Yeah. Essentially within each table you're connecting to a whole bunch of people, and then you’re moving to the next table and connecting to the next person there.

00:49:46:07 - 00:50:08:06
Jie Ren
Yeah, quantity there. How do you afford to have that conversation that is somehow nurturing the relationship. Right. Yeah. So that's really nice. So which leads us to the final question. I hate to end this conversation. Hopefully you can join me another time to talk about the long term effect of this organization of support. Right. All right.

00:50:08:06 - 00:50:34:07
Jie Ren
So, we are both educators, right. And then we are not only talking, about folks like, oh, sorry, talking to women that are already in career. Right. Like, folks are bosses and professors or researchers at conferences. And then, there are many young women, right? They could be undergraduates, graduate students or high schoolers or even younger.

00:50:34:13 - 00:50:46:16
Jie Ren
So, umm and then if they aspire to be a tech person, right, to pursue a career along that line. So what advice would you give to them?

00:50:46:18 - 00:51:03:07
Sofia Bapna
Okay. So that's a really good question. I would say, you know, I think being aware of these structural barriers is one thing, because if you're not aware of the structural issues, you might

00:51:03:09 - 00:51:27:17
Sofia Bapna
you might be trying to network but not seeing the outcomes coming. Right. So that helps you understand. It may help you validate some of what you're experiencing, just like you mentioned. Right. Like you've seen at conferences such and such happen. Right? So but you if you don't understand the structural issues underlying it, then it may become hard to understand why.

00:51:27:23 - 00:51:48:06
Sofia Bapna
Why is it that, you know, my efforts are not having the same payoff, right. So that's one piece. And the other piece is okay to advocate for change, right? So I you know, I go to these conference organizers every year and, and but they give me a slot for an hour, an hour, an hour and an hour and a half.

00:51:48:06 - 00:52:17:19
Sofia Bapna
Why is it that, you know, my efforts are not having the same payoff, right. So that’s one piece. And the other piece is okay to advocate for change, right? So I you know, I go to these conference organizers every year and, and ask them give me a slide for an hour, an hour, an hour and an hour and a half. Right. And I do this to create better outcomes. So so bringing structured does and always you know I'm going bring it's making structure into networking right. These these these changes into the not asking people to just to network organically which is fine. But providing structure so that they can network better. It doesn't always have to come from the organizer.

00:52:17:20 - 00:52:31:23
Sofia Bapna
Right. So you could say, because these are such simple, things to implement, right? You could say, hey, you know, can we do such and such at this event? And, you know, that might help outcomes for them as well, right?

00:52:32:02 - 00:52:56:15
Jie Ren
Yeah. I have one more comment to make based upon your research and our conversation, which is, let’s expect the long term effect, right? Especially for them who who tend to be young, right? The high schoolers or undergraduates, like if they pay more attention to the structure in the network, as you mentioned, they probably will see the outcome of the cumulative advantage.

00:52:56:20 - 00:53:02:11
Jie Ren
[Sofia Bapna: Yeah, yeah.] More than people who are in the later stage in their career.

00:53:02:11 - 00:53:25:09
Sofia Bapna
100% right. Because if if women have better outcomes from these events, from when they're younger. Yes. Then they're going to be more likely to be successful. They're going to stay in tech. So, you know, it's really interesting that in tech the attrition rate for women is significantly higher than men. So women are leaving tech at higher rates. Right. So why.

00:53:25:09 - 00:53:41:19
Sofia Bapna
Because they don't feel the careers progressing. And so if we have successful events where we feel like they're making the career changes, the progression of the career that they, they desire, then it's going to have a cumulative effect for sure.

00:53:41:21 - 00:53:52:03
Jie Ren
Yes. Wonderful. So do you have any other thoughts? Last piece of advice that you want to give to the audience?

00:53:52:05 - 00:54:16:06
Sofia Bapna
One thing that one can't resist thinking about is this world of AI right? Today. And how it's disrupting everything. And I don't know, we're all figuring it out right now, but some of the early research says things like women are less likely to use AI than men. Women are more likely to be penalized for using AI than men. Right?

00:54:16:08 - 00:54:28:11
Sofia Bapna
Which is interesting. I think things are likely to change. These are early days, but my advise is that we need to embrace. We need to embrace AI. We need to stay on the top of it.

00:54:28:11 - 00:54:30:03
Jie Ren
Cannot agree more. Yes.

00:54:30:05 - 00:55:10:08
Sofia Bapna
And we need to see how professions are changing. They are going to change right?[Jie Ren: For sure.] And we can't sit back until both men and women, everybody we can’t sit back and let, you know, let and just watch. Right. So I think really educating yourself, being aware and embracing using the AI technologies to do your job is better. Or if you feel really your job is going to become obsolete because of it, remember that there are so many new jobs that are emerging because of AI.

00:55:10:08 - 00:55:23:01
Sofia Bapna
[Jie Ren: Yes.] So those are opportunities. And, I think we need to be a little bit dynamic now, and be aware of the changes coming our way.

00:55:23:03 - 00:55:33:11
Jie Ren
Thank you so, so much. Sophia. I really enjoyed our conversation. I wish you could join me again on a different episode, especially talking about the long term effect of networking. Thank you so much.

00:55:33:13 - 00:55:36:13
Sofia Bapna
Thank you. I so enjoyed our conversation as well.