Checklist for healthy multi-discipline pods
The organizational doc that's one of the best things to survive the implosion of a unicorn startup
Background
What’s a Convoy?
Some time ago I worked at a company called Convoy. No, not named after the Kris Kristofferson/Ali MacGraw film of the same name. It was attempting to radically reshape how freight systems work in the US. With a focus on maximizing availability with zero waste in trucking. An amazing bunch of people worked there, and had an appetite to attack huge inefficiencies in a big part of the US economy - full truck load (FTL) trucking. Much has been written about what worked and what didn’t. But ultimately the once unicorn level company fell apart. The outcome was painful to watch on many levels. The platform and name somewhat continue with Flexport who acquired the tech stack and small number of the players for what sounds like a cut-rate deal.
When I joined Convoy the lead, original investor said something to me along the lines of “I think this has the potential to be a trillion dollar company, but even if it were to fail this effort will drive forward the entire market in a way to make things more efficient and better for carriers (aka truckers).” I thought that second part make sense - and from what I can it has come to fruition. With legacy brokerages as well as newer upstarts continuing the tech approaches that Convoy brought. Though likely with a more fully formed depth of understanding of what makes the whole system tick and what risks to avoid.
I’m not here to re-hash the failures - but to share some of the culture work that I’d hate to be lost to time.
The Convoy pod model
I’d arrived when Convoy was a pretty good sized company - probably with around 300 engineers and data scientists. Groups were structured according to something referred to internally as a “pod model” which was clearly derived from the oft cited Spotify “squad model.”1 Even though “everyone understood it,” not much had been formally captured on what was supposed to be true about each Pod. If one asked about it you generally was told the following was at the heart of the design;
Pods would be organized against business problems that were mostly separable from other areas. meaning that the pod could operate autonomously the majority of the time.
Pod members would be cross functional. While reporting would be functional - the “first-team” identity of everyone in the Pod - would be the pod. Whether they were a practicing engineer, data scientist, designer, product manager, etc. Typically, but not always there would be an engineering manager (EM) and/or Data science manager (DSM) within the Pod, along with a Product lead.
Pods would take OKR’s which were equally shared across the group - meaning there were Pod goals, not engineering goals vs. business goals.
This generally worked pretty well. Though I’ve had a view for some time that when one overly insists on groups being hyper-independent of others it’s very easy for them to trap themselves in goals that are local minima. Meaning the team being extremely successful at whatever goal they took might feel great to them, but possibly would not move the needle on company level (global) outputs - such as highly loyal customers, making more money now and in the future, etc. This type of problem was definitely showing up more and more as additional Pods were created and goals were set mostly bottoms up within them. A common example would be a team insisting on prioritizing work that from their own estimates was worth say +20bps of revenue at the expense of another Pod’s work they estimated at +200bps of revenue.
There were nudges to address these sorts of downsides of course. The large-ish number of Pods were grouped into a smaller number of coherent areas. The intent being that if the pods had some overlap, then collecting them together with other pods with cross dependencies then “super pod” leaders could keep things moving towards globally optimal outcomes.
The problem statement that led to the rest of this article
As growth continued the culture between pods began to diverge. In hindsight this seems completely natural as the oral history can only scale so far without documents to support it. One senior engineer really highlighted this problem - pointing to teams where ways of working between product managers, engineers and scientists was beginning to break down in their view - but maybe working as intended from someone else’s perspective. There’s a common, though in my mind unhelpful trope that “Product Managers (PM’s) are the CEO of their area.” Which in my view likely contributed as part of the problem. PM’s could easily feel (or be seen to that) they were leading the pods - rather than the pods working together in a state of balanced anarchy for Convoy’s greater good. TBH - I just made that phrase up now. But I think it sounds better than “_____ is toxic.” which was one of the more extreme phrases thrown around at the time around roles and responsibilities.
Documenting a future state
In order to address this question of “how should Pods operate in practice” a working group was formed to propose shared expectations to the head of engineering and the CPO. Who would then use it across the tech org to align expectations, and provide an ongoing reference. Somehow much like Opus getting the VP candidate slot in Bloom County - I must have missed an important meeting and was placed on this working team. This group included Alex Turek, Casey Olives, Marcus Hanson, and Karthik Vasudevan. Edits on top of the working group content (particularly in the second section) were then done by Tim Prouty and Ziad Ismail - our head of engineering and CPO respectively. My memory isn’t what it used to be - so it’s entirely possible I forgot someone. If so please let me know and I’ll both apologize and update this ASAP.
We divided the work into two core sections: (a) a checklist of processes one should expect to see around them, and (b) a breakdown of expectations between the different technical disciplines making up the pod. With the concept that Pods/teams could then self assess if they saw these behaviors within their local group - and speak up if they didn’t. Eventually we included questions on if these things were present in ongoing tech pulse surveys.
A small caveat
I was always a lot more attached to the first “checklist” section of content as I believed that was the most important piece of the framework. Maybe it’s my whole Gen X aversion to being labeled, but what an engineer does vs. a product person vs. a data scientist bored me. My biased recollection is I contributed a LOT to the first section, and somewhat less to the second. I believe deeply in the framework of the “pod checklist.” In that if you don’t have similar characteristics you should consider adding them into your group’s ways of working.
For reasons I don’t fully understand people really seem to like to argue about the roles and who does what. I offer that section below for historical context - but I don’t agree with everything there 100% (and I authored less of it directly). Again - maybe it’s just the anti-establishment streak in me. I’ve heard others call it out as also valuable. I do feel strongly about the “everyone is responsible for …” part as it strongly aligns with my belief that understanding the big problem helps everyone deliver true value.
One last piece of background
I know this document escaped the local gravity of Convoy, even before the company went kaput. People who left used it in their companies - in some case with only minor modifications. I’ve left out some of the amusing stories as to how I know that to be true. But hire me for some consulting or just grab a coffee with me and I’ll dish. ;-)
Without further delay I offer our old working group’s writing for you to use, dissect, and modify as needed with the hope that it helps your team
.
.
.
The High Performing Pod document
The purpose of this document is to drive clarity in terms of how we expect high-performing pods to function. This specifically focuses on the interactions between Design, Engineering, Product Management, and Science roles.
POD CHECKLIST
This section outlines the characteristics that we expect each pod member to be able to observe. If you do not observe these, you should have a conversation with your manager and pod about how to fix this.
The definition of success exists: There is a long-term vision and the pod has OKRs to measure progress toward that vision. They do not need to all be numeric or precise. The important thing is that it should be clear that we are getting closer to the vision as they are achieved.
Simpler goals tend to be better than complex goals. Fewer goals tend to be better than more goals.
Good goals are generally easy to measure and are controllable by the pod.
Alignment with the company is clear: The members of the pod understand how achieving these goals aligns with the company strategy.
A written backlog exists: The backlog contains the work of the pod including design, engineering, product, and science.
Ideas come from everyone. All members of the pod have a clear and recurring forum to contribute to the backlog. This can mean suggesting ideas and improving existing ones.
A prioritization framework exists: Members of the pod can point to the backlog and broadly explain why it was prioritized that way. This doesn’t mean they agree 100%, but they understand the logic.
Estimates are done to improve execution and prioritization.
We use estimates in many parts of our work. This includes how long something takes to build, how useful the feature will be, etc. We use these estimates to make better decisions even though we know they will not be precise.
Effort spent estimating should be proportional to the complexity of the work being estimated. Time spent on estimates should be dramatically less than doing the work itself.
Estimates of how long a task takes should be driven within the discipline doing the work.
Other members of the pod should ask about the reasoning behind estimates. Estimates are pressure tested to find different approaches. (“What trade-offs can we make?”, “Is there a simpler solution?”, “What other solutions can we consider?”)
Continuous improvement: The pod holds regular retrospectives and takes action to improve the performance of the pod.
The pod tracks and inspects what the ongoing costs are of running the existing systems and works to minimize these costs. Any work that is expensive to do and occurs frequently should be reviewed to see if there is a way to invest now to make it faster in the future.
Retros on estimates (e.g. work time, impact) aren’t about punishing ourselves for being wrong. They’re for improving our future estimates and finding problems that can be solved.
Meetings are valuable uses of time. The pod audits meeting burden and looks to eliminate meetings that are no longer valuable.
Encourage debate and make fast decisions.
We have specific roles to identify who the decision-maker is for different decisions. If the decision is primarily about software, engineering typically owns the decision. If the decision is about models, experiment design, or data architecture, data science typically owns the decision. If the decision is about feature prioritization, product management typically owns the decision. If the decision is about how interface objects behave and look, design typically owns that decision.
We debate ideas in the room. You are here because we believe that you will have an impact on the company through your judgment and execution. There is no value in having a dissenting opinion that you don’t voice. Decision-makers seek out disagreement and dissenting points of view.
We don’t need consensus on decisions. Decision-makers should balance the cost of further discussions with the cost of making the wrong decision.
Disagreements that aren’t quickly converging and where you can’t CIRD (Challenge Ideas, Respect Decisions) should be escalated to your manager. Issues that stem from conflicting objectives should be escalated immediately.
Celebrate wins - Pods recognize great work and reinforce behavior that they want to amplify.
Psychological safety: the pod actively works to develop a culture of psychological safety, defined as “a shared belief held by members of a team that the team is safe for interpersonal risk-taking.’’
We are radically open-minded. Sincerely believe that you might not know the best solution, and always seek to learn rather than to be right. Appreciate thoughtful disagreement. See feedback as a gift. Focus more on understanding others than on being understood.
We admit mistakes and learn from them without blame. Everyone is fallible. Admitting and normalizing mistakes helps build a culture where everyone feels comfortable taking risks
We show vulnerability to build trust. Make time to connect about life outside of work. Care about the other people on your team as people.
POD DISCIPLINES
This section outlines the general expectations of each discipline. Since you are hired with a certain job title, it is reasonable to expect that you will use your specialized skills to deliver impact. Based on level, different amounts and different types of impact are expected as defined in our level rubrics (PM, Eng, DS, Design).
There are a couple of caveats that are worth calling out:
This is how a standard pod where there is a mix of disciplines should operate. In more specialized pods (e.g. Infra) some of the disciplines may be missing and the responsibilities may be different.
If the circumstance needs you to wear a different hat, you should do whatever you can to help the company succeed, even if it means taking on responsibilities that don't normally fall within the scope of your role.
We also recognize that people will not perfectly fit into the discipline taxonomy and healthy pods make room for individuals. For example, we may have engineers who also have strengths as product managers and will be more active in setting the vision and OKRs.
EVERYONE
Understand the customer - Everyone is responsible for understanding our customers, the business, and the broader industry. We believe people do better work when given the context and a problem to solve, rather than the specifications of a solution. Design and Product Management are responsible for bringing the customer to the pod.
High standards - Everyone is responsible for having high standards and finding ways that the pod can continuously improve.
Achieving OKRs - Everyone is responsible for achieving the OKRs that have been established by the pod. If we are falling behind, everyone needs to work on thinking through solutions to get back on track.
Team health - Everyone is responsible for team health, improving their peers, and making the team a more fun and inclusive environment where we can do our best work.
DESIGN
The role of Design is to contribute to the long-term vision while supporting the near-term execution of product development.
Vision & Planning
Understand who the customers are, what their needs are, and what their issues are. Design is grounded in a deep understanding of the customer. Design must understand who the customer is (and who they are not), their needs, desires, and issues. They may need to dig into many different customer types and roles if many “kinds” of customers will use a product, service or system.
Create a design vision that meets customer and business needs based on technological constraints. Through an iterative process design translates the needs and requirements of the customer, business, and technology into a vision for what the customer would experience.
Rally stakeholders around a design vision. Like the PM, designers rally the team and stakeholders around their designed solution.
Mind the org gaps. We all know they exist, no matter what org model you come up with there will always be potential for seams in the experience where organizations meet. As a horizontal discipline, it is part of a designer's job to flag and solve for these gaps.
Execution
Create interaction flows, wireframes and visual design assets to communicate the design in enough detail such that it can be built. Design turns the product concepts into pictures and words that communicate the solution. This comes in a number of forms: flow diagrams, wireframes, and red-line documents. Additionally, design provides visual assets needed by engineering.
Assure that the shipped design meets customer needs, resolves issues, and complies with known design tenets. Design ensures that the design actually meets customer needs and solves customer problems. Design engages in activities such as usability testing and surveys that validate that the design meets customer needs.
ENGINEERING
The role of engineering is to:
Develop innovative solutions: Engineering is responsible for developing technically viable solutions that solve the customer’s problems. Engineering incorporates the strategic business context and collaborates with other disciplines in the pod to propose and build creative, robust solutions.
Decide on engineering (how): Engineering is responsible for deciding on the engineering trade-offs for how systems, services, and products are built. The following engineering tenets guide these trade-offs:
Strive for simplicity: Trading off speed vs. quality is a false dichotomy. Engineering works to find the simple solution, builds it well, and ships it quickly. We celebrate when we bring simplicity to parts of our system that are more complex than they need to be.
Build as an owner: Engineering takes pride in building products that bring joy to our end customers and building services that bring joy to the engineers consuming our APIs, using our libraries, and working in our codebases. We think long term to build in a way that increases our future Innovation velocity vs. just our near term velocity.
Operate with excellence: Our customers run their business around the clock, and they depend on us to do the same. Engineering builds our systems to last with reliability, scalability, and operability from the beginning, not as an afterthought.
Own the execution: Engineering is accountable for estimating, planning, and executing what needs to be built. Engineering works with PM to adjust scope, resourcing, and timelines as new learnings emerge throughout execution, identifying opportunities to deliver the majority of the value sooner.
PRODUCT MANAGEMENT
The role of product management is to:
Establish the strategic direction: Product Management is responsible for establishing the vision for a P&E pod and successfully making progress against it. An effective product manager establishes clear alignment to the overall company strategy, defines OKRs and ensures that everyone in the pod understands the direction. This is done with input from all the other disciplines to both get the best ideas and increase buy-in.
Understand the customer’s problems, the business, and the industry: Product Management is close to the customer, has a deep understanding of the business, and the broader industry. Product Management brings the problems and opportunities to the pod and provides the context necessary for the other disciplines to come up with the best possible solutions.
Decide on product priorities (What): Product Management is responsible for establishing the product priorities that best make progress against the strategic direction for the pod. Product Management considers trade-offs across external stakeholders and internal pod member priorities (e.g. OE, IV, etc.). An effective product manager creates an environment in which all disciplines are contributing their best ideas and are bought into the plan.
Align stakeholders and unblock team: Product Management is responsible for alignment with stakeholders. This can be business team stakeholders or other pods. Product managers may involve or delegate to other disciplines as needed but should aim to get alignment with the fewest number of people and in the shortest time. Effective PMs foresee blockers and are proactive on making sure that the team is not blocked by other stakeholders.
Ensure successful product launch: Product Managers are responsible for the successful launch of the feature to customers and gathering feedback from customers.
SCIENCE
Role of Data Science is to:
Identify high impact opportunities: Data Science works in partnership with product and business stakeholders to identify the highest leverage opportunities for the business through deep analytics and strategic research.
Develop innovative solutions: Data Science is responsible for developing scientifically sound, robust, and performant models and/or optimization frameworks that incorporate the strategic business context and solve the customer’s problems.
Own the execution: Data Science is accountable for estimating, planning, and executing what needs to be built, in collaboration with Engineering partners. Data Science works with Eng and PM to adjust scope, resourcing, and timelines as new learnings emerge throughout execution, identifying opportunities to deliver the majority of the value sooner.
Accelerate learning through measurement: Data Science is accountable for best practices in measurement and experimentation, enabling reliable, rapid, and iterative product development and innovation.
Data quality and excellence: Data Scientists are the stewards of Convoy’s data, acting as the primary owners of business logic and metrics that our customers and stakeholders use to make timely decisions to operate the business.
POD LEVEL LEADERSHIP
Pod leadership typically consists of a manager or lead from Design, Engineering, Product Management and Science. In addition to discipline specific responsibilities, the role of pod leadership is to:
Resolve the majority of disagreements. If there is still disagreement, quickly escalate.
Operate a healthy, inclusive pod where all cross-discipline members of the pod are able to do their best work.
Pressure test estimates, plans, and designs to insist on high standard of quality and speed
Set pod OKRs that hit the balance of aggressive and achievable.
Hire and develop the people necessary to achieve the pod’s OKRs.
Identify and manage any inter-pod dependencies.
Yes - I realize there’s a lot of other stuff written about the Squad model. When I first started researching it after joining Convoy I was amused to no end that Google’s auto-suggestion for “Spotify Squad model” added “failed” as a common search term.