AI and High-Performing Teams: Navigating the New Development Landscape

A few months back, I had the opportunity to catch up with Chad Beninati, and we had a great conversation about how he uses AI as part of his development work. I had been playing around with AI and was seeing some limited results. It made me believe that a lot of the hype around AI replacing developers soon was hype. However, after my conversation with Chad, I realized I was missing a critical aspect of this equation: vibe coding. Outside of the terrible name, using development IDEs with AI agents built in, changes the results substantially.

We discussed the limitations of these AI Agents. AI Agents are the equivalent of a Jr. Developer with a photographic memory. By that, I mean they can build any scaffolding you ask for in just about any language. They can also help configure local development environments, update documentation, write tests, etc. But they're terrible at the actual architectural design of code. In addition, due to their limited context sizes (which are pretty significant), after a while, they lose the context of the code already created and the architecture of the project, and they will re-create the code in multiple places, creating a dependency nightmare.

Because of these main limitations, while these AI Agents can be massive productivity boosts, they need to be used by developers with a solid grasp of code architecture. They can review, edit, and reject changes on the fly as the AI generates code. I see these being used primarily by Sr. Engineers effectively, and those with less real-world experience risk creating an absolute mess of the codebase, potentially destroying any productivity gain initially achieved.

We also discussed the limitations of using AI to solve complex problems. When I talk about complex problems, I think of complexity in the context of Dave Snowden's Cynefin Framework.

The distinction I want to make between a complex problem and a complicated problem is the understanding of the problem and the existence of potential solutions. Complicated problems, for example, have "good practices," and by this I mean this problem may have multiple ways to solve it, and each option has a set of known side effects or implications for the rest of the problem sets you're working on. In this case, you need a developer to identify the "best" practice for their context out of the existing good practices.

Complex problems, on the other hand, are genuinely new problem sets, and there is no defined set of good practices. In these cases, the practices are emergent; in other words, they require experimentation and learning to understand what "good" might look like in this context.

I also want to distinguish between complicated and complex problems, which aren't based on your knowledge but on global knowledge. If someone has solved this problem before and there's a case study of how that problem was solved, you have a complicated problem with a potential "good practice." Complexity doesn't have that. It truly requires invention and innovation to solve.

Because of the way large language models work, they cannot invent new ideas. Instead, they use existing knowledge and patterns to predict and ultimately generate things. This works great for complicated problem sets because, again, the large language model knows about and can leverage existing knowledge of good practices.

This continues to reinforce the idea that humans need to create hypotheses about potential experiments that we can run to validate ideas.

We also discussed the breakdown of complex and complicated work in development environments. When we think about the percentage of code that we write for any product, 90-95% is scaffolding; it's complicated work, and it's an already understood problem to solve. That last 5-10%, however, is complex.

With those constraints as we understand them, we see the need for Sr. Engineers to be part of the development process moving forward with AI Agents is still required.

So what does this mean for your business?

In my current role, I spend a lot of time thinking about team and organization structures and creating environments conducive to high-performing teams. However, the introduction of vibe coding immediately raised several questions about how this changes the way these environments look.

For instance, what does pair coding look like with AI Agents? What do code reviews look like with AI Agents? What about deployment practices? AI-generated tests often don't give you the information you're looking for. Frequently, AI will rewrite the code to make bad tests pass, breaking functionality.

These challenges will change how we imagine teams working. This makes me think of the definition of a team and a group from The Flow System. When thinking of the 90% of work that is complicated, a group probably provides the right structure. In this case, a group may be a handful of developers working on complicated workloads using AI Agents as their "co-pilots" to craft the scaffolding. In this context, leveraging Kanban and continuous flow workflows makes sense.

However, the last 10% of complex work requires a true team to collaboratively identify experiments, learn from experiments, and continue to do so until they have identified an emergent practice that delivers value. At this point, they can move it to the complicated workflow. Here, SCRUM with short feedback cycles or XP makes a lot of sense with the focus on an iterative development approach and fast learning.

While thinking through this, I kept thinking about Toyota's core value of Respect for People. When looking at adding robotic automation into manufacturing, the focus is really on alleviating repetitive work and reducing workplace injury. However, the goal isn't to replace people entirely with robotic automation. It's to focus on improving the quality of life and the overall product quality we're building.

One thing Chad said to me towards the end of our conversation around product development in the age of AI that has stuck with me is that now, with the speed at which we can build products, it will be more the case that the actual products are disposable. The data, however, is going to be even more critical.