Product Iteration: Using Customer Feedback & Data

The Evolving Landscape of Product Iteration for Startups

Product iteration has consistently ranked among the primary challenges faced by startups. Previously, decisions were largely guided by customer feedback and intuitive judgment.

However, the current environment offers a significant advantage: access to substantial amounts of data. This wealth of information presents both possibilities and the need for a refined approach to decision-making.

Insights from TechCrunch Disrupt 2021

This evolving context was the focus of a panel discussion at TechCrunch Disrupt 2021. The conversation featured Jean-Denis Grèze from Plaid, Stephanie Mencarelli of InVision, and Pete Thompson representing eBay.

The panel explored diverse viewpoints concerning data-driven strategies, effective user base segmentation, and optimizing iteration speed.

Key areas of discussion included how to effectively leverage data alongside traditional feedback methods. Participants shared insights on identifying and prioritizing impactful changes.

• Understanding the nuances of different user segments was highlighted as crucial.
• The importance of rapid iteration cycles to quickly validate hypotheses was also emphasized.

The conversation underscored that successful product development now requires a balance between qualitative insights and quantitative analysis.

Ultimately, the panel suggested that a new form of strategic advantage – an arbitrage of data and intuition – is essential for startups navigating this dynamic landscape.

The Significance of Data in Business

The discussion began with a challenging inquiry: Is it possible for a new business or technology firm to rely too heavily on data?

Thompson asserted a firm “no” to this proposition, while Mencarelli expressed a less definitive viewpoint.

Thompson’s Perspective:

Mencarelli’s Response:

Despite his initial strong stance, Thompson previously acknowledged the importance of considering factors beyond data analysis. He stated the necessity of contemplating unexplored avenues and potential developments that data alone cannot reveal.

He illustrated this point with an example from his experience at eBay:

The Critical Role of Effective Segmentation

Building upon Mencarelli’s earlier observation regarding the value of analyzing focused groups, the discussion shifted to the crucial practice of segmentation.

Grèze detailed Plaid’s approach to this:

Plaid successfully identified Copilot, a personal finance management application previously featured by TechCrunch, as a valuable partner. This collaboration proved beneficial for several reasons. Copilot submitted highly detailed support requests, a substantial portion of Plaid’s users shared comparable applications and requirements, and their team actively collaborated with Plaid to refine the product development plan based on their insights.

The necessity of segmentation is particularly evident at eBay, as Thompson highlighted. The platform supports “approximately 20 million sellers worldwide, operating in nearly 190 markets.”

These sellers range from individuals pursuing supplemental income to multinational corporations like Nike, presenting a wide spectrum of requirements. eBay, as a broad-based marketplace, must effectively cater to this diversity.

Thompson suggested that eBay’s methodology could offer valuable lessons for other e-commerce businesses:

Mencarelli further explained that InVision also considers this extensibility within its freemium business model:

Relative Speed in API Development

A compelling point was raised regarding the "move fast and break things" philosophy, and whether it applied to Plaid's operational approach.

Grèze offered a nuanced perspective.

He explained that the appearance of differing speeds stems from the nature of API solutions, where development timelines are considerably longer.

The development process isn't solely internal; it encompasses the customer's development cycle as well.

This necessitates a greater degree of initial confidence in decisions, as iterative A/B testing isn't a viable path to success.

Furthermore, once an API is integrated by customers, its removal or significant alteration becomes problematic.

Considering these factors, Grèze emphasized the importance of “relative speed.”

Plaid doesn’t aim to match the pace of companies like Facebook; it acknowledges being slower in comparison.

However, it strives to be faster than other companies operating within the API space.

This is achieved through strong product conviction, collaboration with design partners, and a focus on iterative development with a specific customer base.

Specifically, Plaid leverages self-service customers to facilitate smoother transitions to new experiences, minimizing disruption for both parties.

Key Factors Enabling Relative Speed

• Product Conviction: A strong belief in the direction of the product.
• Design Partners: Collaborative relationships for valuable feedback.
• Self-Service Customers: Facilitating easier iteration and adoption of new features.

These elements allow Plaid to maintain agility and responsiveness within the constraints of the API ecosystem.

Roadmap Development Strategies

According to Mencarelli, InVision prioritizes velocity, alongside more enjoyable elements like pizza. This appreciation led to the development of their “slices” concept for project planning.

The importance of moving past simply adding small improvements was also emphasized by Thompson. However, he cautioned against pursuing radical, disruptive changes all at once.

The Staircase Analogy

Thompson illustrated his point with a helpful analogy: a staircase. This represents a balanced approach to innovation.

Rather than attempting massive leaps, progress is made through a series of manageable steps. Each step builds upon the last, leading to significant overall advancement.

• Incremental Innovation: Small, continuous improvements.
• Step-Function Innovation: Larger, more impactful changes.

The goal is to avoid the pitfalls of both stagnation and overwhelming disruption. A steady, deliberate pace is key.

“Slices” as a Planning Tool

InVision’s “slices” methodology reflects this philosophy. They represent focused units of work.

These slices allow for rapid iteration and delivery of value. They also facilitate a more manageable approach to complex projects.

Expanding Perspectives in User Feedback

The discussion serendipitously turned to Spotify, a platform Thompson had experience with during her time at Sonos, and where Mencarelli had previously been employed. This prompted her to recount a story concerning the music streaming service: they encountered difficulty obtaining constructive criticism from Swedish users participating in in-office testing, as these individuals expressed overwhelming satisfaction with the service.

Although this specific challenge may be relatively rare, the resourceful solution implemented by Spotify is likely to be of interest to entrepreneurs and startup leaders.

Mencarelli shared her insights:

Earlier in the panel discussion, Mencarelli also addressed the crucial subject of inclusivity. She explained that a positive outcome of the pandemic for InVision was the increased opportunity for a wider range of individuals to contribute feedback.

This experience fostered an optimistic outlook regarding the evolution of work practices and the future of product development.

A complete, automated transcript of the discussion is available for review here.