Frank Tisellano

I’ve reviewed hundreds of resumes and interviewed many dozens of product managers, both as a PM helping to vet potential teammates and as a leader designing a hiring process, writing interview questions, and ultimately making the hire/don’t-hire call.

Because PMs come from diverse backgrounds and because every company does Product Management differently, there’s no ideal interview process that works for everyone. But most great PMs share key attributes and behaviors that make them great.

As a PM conducting interviews early in my career, I had a great boss who gave fantastic guidance for a beginner interviewer. When I stepped into a leadership role, I searched for resources to help me design our hiring process but struggled to find deep, evidence-based, PM-focused content. Most of it was fluff more focused on SEO keywords than on doing any real guidance.

So I learned the hard way — by doing.

Over time, I’ve honed a number of interview questions (scenarios, really) I believe get to the heart of these attributes and behaviors. Responses to scenarios like these are difficult to rehearse, since they’re unpredictable and since good interviewers can ramp the difficulty of the scenario when candidates respond well.

Now that I’ve moved on to a new company and no longer use these questions, I thought I might share them.

This particular scenario tests a PM’s ability to work cross-functionally, to prioritize, and to achieve buy-in for a proposal, in addition to gauging the “softer” skills needed to be a great product manager.

One could of course test these abilities through behavioral questions of the “tell me about a time when…” variety, and I often do as a supplement to the scenario below. In my experience, however, questions about a candidate’s history are easier to practice for, and they make the interview more confrontational and less collaborative because they require the interviewer to question the candidate’s real-life decisions.

The Scenario

You’re the PM for a digital marketing analytics product that aggregates data from across a customer’s web properties and displays it in one place.

Your VP of Sales comes to you, excited to share news that he’s working a deal with what would be your second-largest customer. The prospect is ready to do the deal, but they want you to commit to building a custom integration before they sign. How do you handle it?

Note: I explicitly tell candidates to use me as a role play partner and that I can play any role. Rather than having them tell me what they would do (“I’d talk with my engineering manager”), I ask them to actually play out that interaction with me.

TL;DR What Great Candidates Do

Good candidates ask probing questions, seek additional business and customer context, establish and articulate a reasonable decision-making framework, demonstrate an awareness of the various needs and priorities of each stakeholder group, and demonstrate skill and craft and experience in achieving consensus and buy-in across those groups.

The Worst Possible Answer

There’s no “best” approach here. There’s not even really a “right” approach. But there’s definitely a wrong approach.

The last thing I want to hear is a knee-jerk “answer” to the scenario, delivered before the candidate asks even a single question. Some candidates think I’m looking for something like, “I’d never compromise the integrity of the product for a single customer’s request.” Others say, “Second biggest customer? Let’s do it.”

No. Not good. Don’t do this.

One Valid Approach

Sales/Business Context

Key conversation: VP of Sales

First, good PMs seek to contextualize the opportunity:

• What does “second biggest customer” even mean?
• How big are they relative to our existing revenue?
• What does the distribution of our customer sizes look like? Is it more of a head/long tail distribution or is it more balanced?
• Is the prospect a well-known company? Would they be a good reference/logo?

Really great PMs demonstrate their experience by asking more subtle questions, attempting to gauge the impact of this discovery work on their time:

• Is the VP of Sales the kind of person who regularly cries wolf over deals like this? Are they showing their hand too early?
• How far along is the deal in the sales process? Are there competitors in the deal?

Finally, they work with the VP of Sales to set up a conversation with the prospect to validate their request and understand their problem first-hand.

Requirements Gathering and Discovery

Key conversation: the prospect

In the prospect conversation, good PMs seek to understand what the prospect’s real problem is and how urgently it needs to be solved.

They’re poised and warm, and they create a collaborative environment that elicits positivity and attention from the prospect.

They gather information about what the prospect says they’re looking for — in this case, the CMO wants a custom integration with a 3rd-party who tracks estimates of their physical (billboard) advertising traffic.

Good PMs dig more deeply to understand why — in this case, it turns out the hypothetical prospect is Uber, whose marketing team wants to correlate web traffic/conversion rates for new drivers by city with billboard volume and location.

They ask how the prospect is solving this problem today and whether they have any mockups or ‘Excel Specials” that exemplify what they’re looking for in the product.

They want to know the timeline — no, the real one. They know how hard to push back on the prospect, how to do a bit of pre-negotiation before bringing the requirements back to their engineering team for discussion. They offer multiple solutions to gauge where the MVP for this customer is — they know that this work will interrupt strategic work already in flight and seek to minimize the disruption as much as possible, even (especially) in this early discovery stage.

On that note, good PMs are technical enough and understand their own product’s architecture well enough to have a rough idea of how complex the integration will be, but they’re reluctant to share a timeline before consulting with their engineering partners.

Great PMs probe to understand the relationship between the buyer and her requirements — is this something that’s been handed down to her by the CEO? Does her team have an urgent operational need, or does she need this data to demonstrate her team’s success internally? This is tricky information to acquire without being crass.

Finally, good PMs confirm The Problem with the prospect by summarizing and articulating it briefly and coherently.

Getting Estimates and Understanding Tradeoffs

Key conversations: engineering lead, customer success, sales

OK, now we understand The Problem, its nuances, and the timeline the prospect expects. Once we’ve understood the requirements, we need to get to talk with engineering to estimate the cost of our potential solutions. Usually, candidates push the conversation in this direction themselves. Sometimes, they need a bit of a nudge.

Since this situation is hypothetical, candidates naturally don’t have much context as to where work for this prospect fits into the existing roadmap. But the situation isn’t totally unlike what PMs run into when starting a new job. New PMs on real product teams need to be proactive in acquiring context about existing priorities (and the rationale for those priorities), so I expect them to ask for the same context in the interview as they scope the new work.

These are the basic questions I’m hoping to hear regarding estimation:

• Can we work through an estimate for each of these (list them) options?
• What work do we have in flight already? How strategic is it?

Typically, I describe the hypothetical work in flight as ultra-strategic. It’s an ML-based feature that surfaces correlations between various marketing channels automatically. Buzzwordy but also insanely valuable for busy CMOs and their teams.

More savvy candidates will ask:

• How much context switching tax will we pay to put the current work on hold and come back to it later?
• Is there any way for us to split the team to keep pushing the existing work forward?
• Could we hire contractors to do this for us?
• Have we made any promises to existing customers regarding the work in flight? Have we announced a ship date?

In most cases, I explain that our top customers (including our #1 customer) are aware of the ML-based work in flight and have been kept up to date on the timeline for it. It will not be easy to explain a delay to them. I ratchet the difficulty of the scenario up or down depending on how well candidates plan to manage existing customers’ expectations.

I expect the candidate to ask to speak with folks from customer success and sales to assist in understanding the tradeoffs, to plan for communications, and to run through the what-ifs of a delay in the existing work.

In addition to confirming that candidates can participate in basic estimation and negotiation exercises, I’m also hoping to understand the extent to which they’re entrepreneurial. Are they willing to challenge me, to challenge the prospect, to challenge even the scenario itself to do what’s right for the business and for our customers?

Making a Decision and Getting Buy-In

Key conversations: sales, support, tech, business leadership (everyone, really)

We’ve spoken with the prospect. We’ve spoken with sales and customer success. We have our estimates. We understand the broader strategic context. We need to make a decision.

But any candidate with even a little experience knows that this isn’t her decision to make unilaterally. No CEO or GM (not to mention hungry Sales VPs) is going to give an IC PM this level of autonomy. This decision ultimately lies with the folks running the business, and it’s our job as PMs to enable and support that decision making process (and in turn to influence it) so that we do what’s right for the business.

Here, I ask bluntly, “What’s your recommendation?”

I mostly don’t care what the recommendation is, but it’s important that candidates use information from the scenario (as much ‘data’ as they have) to support their position. Mealy mouthed platitudes will not fly here.

Once they’ve expressed their recommendation, I want to know how they’ll get buy-in. Yes, getting buy in can be an art, but Product Management is a craft, and good PMs know how to articulate their process, even if some of it comes down to good relationships and other hard-to-articulate behaviors.

Again, there’s no right answer here, but here’s how I’d do it: first, I’d present my findings and the options (plus my recommendation) one-on-one to the heads of every department in the business. I’d get their feedback, ensuring that my articulation of their position and that my recommendation stands up to their scrutiny. I want to come out of each of these conversations with a “Yes, that sounds good to me.”

Once I have individual buy-in from each department head, I’d hold a group meeting with every stakeholder in the room to present, now with allies (i.e., all the folks I just got buy-in from), my plan to the ultimate decision maker (e.g., CEO, GM, etc.). We get group buy-in with everyone in the room, we make the call, and we start executing¹.

Boom. Product managed. Interview nailed. Job got-ed(?).

If you have any questions or feedback or you think this scenario is stupid and should never be used in an interview again, I could and would love to talk about it. Feel free to shoot me an email or find me on twitter.