Boris: So what are some of the learnings from that from you know like experimenting and figuring out what are the pockets of adoption, what works to accelerate development, what doesn't work like what's like one surprising thing that works and one thing that doesn't?
Andy: One thing for sure is smaller teams, is a great accelerant. Each person can kind of own a thing or a couple different domains, and you don't have to, like, coordinate as much with each other or like, hey, I accidentally like stepping on your toes or whatever. Another good learning was actually like, it actually pays off to do some upfront investment to setting up your code base to be agent friendly. And one of our successful examples, the tech lead he jotted down on it like a couple dozen, maybe 50 plus like principles of architectural principles that he cared about for a particular code base. And he jotted it down in these markdown files within the GitHub repository and, basically so that when the agent was coding that he could reference these architectural principles. And it actually was super helpful. Another thing that was really helpful was standardizing skills across the team. So for example, for our mobile development, the team put together skills around basically making it really easy to spin up the simulator and test particular workflows, things that would be super manual and annoying for an engineer to instrument themselves is now just like a skill that you can call and kind of run in the background.
So I'd say those are some positive use cases that we saw in terms of AI adoption. The challenge that we ran into was particularly for more user facing features where you're not just coding a platform change or code migration. It requires cross-functional buy-in. You need the designs to be aligned. You need to make sure that from a product perspective, you're solving the right problems. And we have a lot of these processes that we've put in place for good reason, like product review or ship review or design review, things that historically were very important for us to get alignment with each other. But now, given how fast things are moving, it's just like it's a huge friction point. And so we've had to really build alignment with cross-functional leaders to make sure that if we want to really take advantage of AI velocity, it's not just an engineering problem, it's a whole company problem. Cross-functional leaders need to be brought in to change the way they work in order to unlock this type of velocity. Like we need to be okay with not necessarily having product review work in the same way. Or do we even need to have product review?
If we want designers to be coding directly, making changes. You know, historically, an engineer would have to spend a lot of time getting the design to be pixel perfect. But if the engineers can get it in a workable state, and then we can pass it off to the designer who can get to pixel perfect self sufficiently, that's really powerful. I think something that we've generally found to be true is the more self-sufficient a team is or an individual is, the faster they can move. And so that's kind of the philosophy that we're trying to really push when we are encouraging people to be more AI forward.
Boris: So less process, less design docs, less reviews.
Andy: And just like working directly in prototypes, if you're a designer, like working directly in the code. Yeah. Basically streamlining a lot of the process or reimagining it maybe because like some of this process is there for important reasons. Maybe there's compliance related things that need to go on, but then it's just asking the team like, hey, if we want to shrink the timeline by 3x, like, what's a way we can leverage agents to automate a lot of it, or make it faster? And I think the mindset in terms of how teams should change is like, hey, you don't necessarily need everybody to be a domain expert, and in fact, you should encourage people to be as fluid across the code base as possible.
A lot of our mobile engineers, we actually encourage them now to try to be more full stack, which is a lot more accessible now than it was even a year ago. We still need domain experts in, for example, iOS to make sure that there's particular parts of the architecture or kind of how we manage memory or latency and performance. There's certainly domain expertise is important, but you don't need your whole team to be an iOS expert. You can have much fewer of them. Be gatekeepers and figure out how to build agents to basically check for the things or write in MD files. Their architectural principles are things they care about. And you have a lot more people who can be generalists and moving fluidly across the codebase. And I think that's like a mindset change. Also that we need to get people more comfortable with.
Boris: Yes. So when you have one of these small teams working on something, but you want to give them the space to experiment and to kind of figure out the process, how do you think about staffing this team? What's the ideal kind of profile for it?
Andy: We try to make sure that for these teams that we're giving these remits to, we want to make sure that they feel safe and experimenting and pushing here. So I will actually directly lean in or I'll ask, like executives from like the engineering or, various adjacent functions to actually lean in with these teams and basically have like executive VP level sponsorship on that particular workstream to figure out, hey, what do you whatever you need to do to get unblocked, like you have the sponsorship from high up leadership to do so? Because one thing that I've seen not necessarily work as well in the beginning is if you ask people to do this, but they're working within the same confines of, kind of their normal job, where, okay, like I need to go through all these processes or all this stuff, then they're going to give up pretty quickly. Unless you empower them to be like, hey, you actually, not only are you going to get a token budget to kind of reimagine the things of AI, but I want you to raise to us your cross-functional leadership team, what is hindering you outside of just generating code?
And so I think that's worked really well to make people feel safer. We're still trying to figure out how to scale that. But ideally what would happen is as we see these success examples crop up, we share those written artifacts with various leaders. So that people can kind of organically try to adopt these practices.
Boris: Like the limiting factor used to be the speed of coding and that limited kind of the speed of delivery that dictated the team set up, that dictated having to have process in these org reviews and kind of like the cadence of exact check ins and all this stuff. Absolutely. And now, because engineering is so fast, it kind of forces you to rethink all stuff.
Andy: There's actually so much more that a team of 3 to 5 can do. It's almost like you kind of want to invest in making that unit move as fast as possible. Because like if you can get that unit to work 3 to 5 times faster, then that's totally game changing for the org.
Boris: Okay. I wanted to touch on maybe a few, a few final things. So, how were you thinking about ROI? So you talked a little bit before about the first when you're kind of early in the journey, you just want to give people as many tokens as possible with the experiment. Now you're thinking a little bit more about ROI. How do you think about it?
Andy: I think for engineering it's a little bit more straightforward because you can actually, I know, like code throughput, it's not a perfect metric and it's a gameable metric, but it's directionally kind of correct in a lot of ways. And also the fact that, like, you can get more, you can get the same project done faster with fewer people means that the people you have today, you want them using it as much as possible. I think in knowledge work, I think we're so early and I think to a point you made, I think there's a lot of things outside of just like the triaging my email type use cases that I think are really powerful with Cowork that we're just scratching the surface on, like, I think for sales teams, having to do outreach with like tens of thousands of merchants. Right? Like that's a lot of manual work. And if we can really streamline that and make it more accurate and vetted, like with consistency via AI, that's super powerful.
There's two ways that we are thinking about it right now. One is just in terms of raising the baseline, like how much time are we saving the average person in terms of tasks that everybody needs to do? And then there's where I think actually might actually end up being more impactful is on a per department or domain basis. Can we identify particular workflows to automate that would make it significantly more efficient to run that department. So that's kind of how we're thinking about it right now, but in the knowledge workspace, to be honest, we're so early that we're in an exploration phase right now.
Boris: Yeah. So for coding, it's about like the volume of code increases. So like the code throughput you can measure and then the speed of delivery, you can measure pretty easily. And just like freed up people to do stuff.
Andy: Yeah. And I think ultimately for engineering well it's not just engineering because it involves like design involves cross-functional support. But at the end of the day, we want customer value to be delivered faster. So if we can actually prove that out, not just in terms of code getting merged, but also like customers seeing the product live faster, I think for us that's like the ultimate success metric.
Boris: Yeah, yeah. And then yeah, and like for knowledge work, we're still figuring it out. It's still domain dependent.
Andy: I think there is a lot of value to be had if we can get everybody to be even 10% more productive with this stuff, that's huge, at a company of our scale. And so I do think there is value in terms of raising the floor. But I do think with knowledge work, there's still a lot of explorations to be had around domain specific workflows.
Boris: And I guess it's like it's just back to the way that innovation actually happens. Like if you're a top down, like, hey, you go automate this workflow, you might actually just pick the wrong workflow or the wrong person to automate it. Totally. But if you give everyone the freedom to experiment, you're going to be surprised by, you know, some person over here automates this thing. Some person over here automates this thing, and you might not even have known it's a bottleneck.
Andy: Totally. And that's why, like, we're trying to call them AI champions internally right now. It's like we're trying to identify champions within each domain, like sales or marketing or operations or, you know, support. And so we're trying to identify who organically is cropping up, who's doing this in their spare time. We're just naturally gravitating and being prolific with these tools to really help figure out, okay, like if you're the champion for this thing, one, I'm going to call you the champion for your domain. And then two is like, what can you what's what are the big pain points within your domain? Because I don't know. And then, can you figure out a way to either make that workflow a lot more streamlined using agents, or can you figure out a way to make it much easier for your coworkers to be streamlined using a shared set of skills
Boris: So like famously you started using AI in coding interviews pretty early, I think, before even we did. How do you think about it? Like, how do you find these people that succeed in this kind of environment that are happy, you know, experimenting and that enjoy this kind of work?
Andy: I think our mindset is give people the tools and just see who ends up naturally picking it up. One thing I've also found to be true is the people who are the most AI-pilled, they tend to find it fun. You know, you want people who are having fun with it, and those people will naturally, you will naturally find out about those people, whether it's through Slack channels, or…
Boris: They're going to tell.
Andy: They're going to tell you. Yeah. Because they're having so much fun and enthusiasm around that they can't help but share the enthusiasm with their peers. So I think it hasn't been that hard for us to find a champion, especially on the non-technical side. And then on the technical side, I think we've just found people who've naturally become power users and taken off of that.
Boris: Okay, I want to close with one last question. What is your advice for engineering leaders? So like other CTOs and leaders at other companies? And then on the flip side, what's your advice for new grads? Sure. In college, right now you're graduating. What would you tell each of them?
Andy: So I would say for the engineering leaders, make sure you use the AI so that, like, you can encourage your teams to use AI. But I also think it also gives you a sense of empathy in terms of the types of anxiety or stress that engineers might go through to change how they work. And it's kind of an identity crisis in some ways for engineers who are so historically, a lot of people are tied to the craft of the code that they're writing. But you kind of gotta let go of that a little bit. And I think as a leader, I think you really need to understand that. And the best way to understand this is play around with it yourself and understand how powerful these tools are, but also how drastically your workflow needs to change.
And then also encouraging people to share written artifacts. I'll say it again. It's like that's, in my opinion, the most scalable way to distribute learnings at scale. It's also a byproduct of it if you actually end up having artifacts that agents can read. So that's what I would say to engineering leaders. New grads are picking this up so quickly in a lot of ways. My encouragement is like, like stay curious, play around with the tools. I think a lot of companies like DoorDash, what we are really excited about with the new grad talent is people who've grown up with this paradigm of working, who can surprise us in terms of ways that we can leverage technology that we didn't think was previously possible.
Boris: Andy, thank you so much.
Andy: Yeah. Boris, it's a pleasure to be on here. Thanks for having me.