Responsible AI for Startups | Building Better Businesses

The Simplicity of Responsible AI for Startups

Many founders believe implementing responsible AI practices is a complex undertaking, potentially hindering their company’s advancement. A common misconception is that a large, dedicated team – similar to Salesforce’s Office of Ethical and Humane Use – is essential to prevent the creation of detrimental products.

However, the reality is considerably more straightforward.

Founders Already Practicing Responsible AI

To understand how founders approach responsible AI in practice, I engaged with several successful early-stage founders. My research revealed that many were already employing these practices.

They simply framed it as “sound business practice.”

Business Sense Leads to Responsible Outcomes

It became clear that implementing simple, commercially viable practices that enhance product quality significantly mitigates the risk of unintended societal consequences. These approaches are rooted in the understanding that successful AI deployment centers on people, not just data.

Acknowledging the constant presence of human oversight allows for the development of a more ethical and effective business.

AI as a Bureaucracy

Consider AI as analogous to a bureaucratic system. Like any bureaucracy, AI relies on established guidelines – the model – to make logical decisions in most situations.

However, these guidelines cannot encompass every conceivable scenario, mirroring an AI model’s inability to anticipate all potential inputs.

Disproportionate Impact of AI Failures

When these general policies or models falter, marginalized groups are often affected most severely. A well-known example involves Somali immigrants being incorrectly flagged for fraud due to their unique shopping patterns.

The Role of Human Judgment

Bureaucracies address this issue through “street-level bureaucrats” – individuals like judges, DMV agents, and teachers – who can handle exceptional cases or choose not to enforce policies rigidly.

Teachers, for instance, can waive prerequisites under specific circumstances, while judges can exercise discretion in sentencing.

Humans in the Loop are Essential

Given that all AI systems will inevitably encounter failures, maintaining human involvement – much like in a bureaucracy – is crucial. As one founder articulated, “If I were an extraterrestrial observing Earth, I’d conclude: Humans are information processors – I should utilize them.”

Whether humans are operators providing augmentation when the AI is uncertain, or users deciding to accept, reject, or modify model outputs, their actions ultimately determine the real-world effectiveness of any AI-driven solution.

Five Practical Suggestions for Responsible AI

Here are five actionable recommendations shared by founders of AI companies for integrating and leveraging human input to build more responsible AI, while simultaneously benefiting business outcomes:

• Prioritize Human Oversight: Design systems where humans can readily intervene when the AI encounters ambiguity.
• Empower User Control: Allow users to modify or reject AI-generated results.
• Focus on Augmentation: Utilize AI to enhance human capabilities, rather than replace them entirely.
• Embrace Continuous Feedback: Actively solicit and incorporate user feedback to refine the AI model.
• Consider Diverse Perspectives: Ensure diverse teams are involved in the development and deployment of AI systems.

The Strategic Implementation of Artificial Intelligence

Currently, a significant number of organizations are formulating plans to deploy services utilizing fully AI-driven workflows. However, when these automated systems encounter difficulties functioning across diverse scenarios, it is frequently those who are already disadvantaged who experience the most negative consequences.

When attempting to pinpoint the causes of system failures, entrepreneurs often systematically remove individual components, maintaining a strong desire for maximum automation. A more effective strategy might involve the incremental introduction of AI elements, one at a time.

Even with the advancements in AI technology, numerous processes remain more cost-effective and dependable when incorporating human oversight. Deploying a complete, end-to-end system with multiple components activated simultaneously can complicate the process of determining which areas are most appropriate for AI integration.

A Phased Approach to AI Adoption

Many founders with whom we have consulted perceive AI as a means of offloading repetitive, low-priority tasks from human employees. They typically begin with entirely human-operated systems to identify the specific tasks that would benefit most from automation.

This “AI second” methodology also allows entrepreneurs to enter markets where initial data sets are limited. Individuals involved in operating system components simultaneously generate the data necessary for future automation of those same tasks.

One founder shared that, had they not received guidance to implement AI incrementally, and only after demonstrating superior accuracy compared to human operators, their venture would likely have failed to launch.

Benefits of Gradual AI Integration

• Improved system reliability.
• Reduced risk for marginalized groups.
• Easier identification of optimal AI applications.
• Facilitates entry into data-scarce markets.
• Data creation through human operation.

By prioritizing a measured approach, companies can maximize the benefits of AI while mitigating potential drawbacks.

Introducing Intentional Resistance

A common belief among entrepreneurs is that a product’s success hinges on its immediate usability, requiring minimal effort from the user.

However, when Artificial Intelligence is implemented to streamline existing processes – often inheriting pre-existing levels of trust in the resulting output – an overly smooth integration can prove detrimental.

Consider the case of Amazon’s facial recognition technology, which, as revealed by an ACLU audit, incorrectly flagged 28 members of Congress (a significantly high percentage of whom were African American) as potential criminals. The core issue stemmed from lenient default settings.

The initial accuracy threshold was set at just 80%, an unsuitable value given the risk of users accepting positive identifications without scrutiny.

Encouraging users to actively explore a product’s capabilities and limitations prior to full deployment can mitigate the risk of misaligned expectations and potentially harmful outcomes.

Furthermore, this approach can lead to increased customer satisfaction with the product’s ultimate performance.

One founder we interviewed discovered that customers achieved greater efficacy with their product when required to personalize it before initial use.

He considers this a key element of a “design-first” philosophy, enabling users to leverage the product’s strengths in a manner tailored to their specific needs.

Although this method demands a greater initial time investment, it ultimately resulted in revenue increases for the customer base.

The Importance of Context in AI Systems

A significant number of AI-driven systems are designed to generate output suggestions. However, these suggestions require human implementation to be effective.

The absence of sufficient context can lead to suboptimal recommendations being accepted without scrutiny, potentially resulting in negative consequences. Conversely, even highly accurate suggestions may be disregarded if users lack trust in the system and a clear understanding of its reasoning.

Empowering Users with Decision-Making Tools

Instead of automating decisions entirely, a more effective strategy involves equipping users with the resources they need to make informed choices. This method leverages human judgment to identify and correct potential errors in model outputs.

Furthermore, it fosters user confidence and acceptance, which are crucial for the successful deployment of any AI product. A human-in-the-loop approach is vital.

A Case Study: From Recommendations to Augmentation

One entrepreneur discovered that direct recommendations from their AI were often overlooked by users. Despite the model’s demonstrated accuracy, individuals tended to disregard its suggestions.

The solution involved removing the recommendation feature and instead utilizing the AI to provide supporting information for user decisions. For example, the system now highlights similarities between the current situation and five previous cases, detailing the outcomes of each.

This shift resulted in greater user engagement and a noticeable increase in revenue. Augmenting human capabilities proved more effective than simply providing answers.

Prioritizing All Stakeholders, Not Just Purchasers

A common challenge within the enterprise technology sector involves products being designed primarily for executive-level users, often to the detriment of those directly interacting with them. This issue is particularly pronounced in the realm of Artificial Intelligence, where solutions frequently integrate into complex systems impacting numerous indirect users alongside a smaller group of direct ones.

The situation at Starbucks, stemming from the implementation of automated scheduling software, serves as a pertinent example. The software’s focus on efficiency overlooked crucial aspects of employee well-being. Following employee advocacy and extensive media coverage, including a feature in the New York Times, employee feedback was incorporated, leading to improved morale and increased productivity.

Rather than solely addressing stated customer requests, it’s vital to comprehensively map all stakeholders and thoroughly understand their requirements before defining the optimization goals for your AI. This proactive approach can prevent the creation of unintentionally detrimental products and potentially reveal more lucrative business avenues.

A founder we consulted exemplified this strategy by directly observing users in their work environment to gain a deep understanding of their needs. This was then complemented by discussions with both customers and union officials to develop a solution beneficial to all parties.

Initial customer feedback centered around a desire for a tool enabling increased individual workloads. However, these conversations uncovered an opportunity to generate cost reductions for customers by optimizing the current workload distribution.

This realization enabled the founder to create a product that not only supported human workers but also delivered greater financial benefits to management than the originally proposed solution would have.

Understanding the Concept of “AI Theater”

Limiting exaggerated claims about your AI’s capabilities can simultaneously prevent potential negative repercussions and enhance product sales. A measured approach is key.

While promotional hype surrounding AI undeniably boosts product sales, mastering the art of preventing buzzwords from obscuring accuracy is paramount. Overstating the autonomous functions of a product, though potentially beneficial for initial sales, carries the risk of adverse consequences if applied without discernment.

For instance, a founder we interviewed discovered that emphasizing the power of their AI also heightened customer anxieties regarding data privacy. This apprehension remained, even after clarifying that the specific product features in question relied on human evaluation rather than data processing.

Strategic language selection can effectively manage user expectations and foster product trust. Instead of employing terminology centered on full autonomy, several founders found that terms such as “augment” and “assist” were more conducive to user acceptance. This “AI as a tool” perspective also minimized the potential for unwarranted reliance, which could lead to unfavorable outcomes. Clarity serves a dual purpose: it discourages overconfidence in AI and supports sales efforts.

These are actionable insights gleaned from the experiences of actual founders, aimed at mitigating the risks of unintended harm from AI and fostering the development of enduringly successful products. We also foresee a promising avenue for new ventures to create services that simplify the process of building ethical AI solutions that also drive business growth. Therefore, we present the following requests for aspiring startups:

• Prioritize Human Oversight: Startups are needed to address the challenge of maintaining human attention within “human in the loop” systems. Effective delegation to human reviewers necessitates ensuring they recognize instances where the AI exhibits uncertainty, enabling timely and meaningful intervention. Research indicates that when an AI demonstrates 95% accuracy, individuals tend to become complacent and overlook the remaining 5% of errors. The solution extends beyond mere technology; akin to social media’s roots in psychological innovation, we anticipate that startups in this domain will emerge from a deeper understanding of human behavior.

• Facilitate Responsible AI Compliance: There is a clear opportunity for startups to consolidate existing responsible AI standards and provide compliance measurement tools. The proliferation of AI standards in recent years reflects growing public demand for AI regulation. A recent poll revealed that 84% of Americans believe AI requires careful management and consider it a high priority. Companies are eager to demonstrate their commitment to responsible AI practices, and tools that showcase adherence to standards established by organizations like IEEE and CSET would be highly valuable. Furthermore, the proposed EU AI Act (AIA) places significant emphasis on industry standards, suggesting that compliance will become mandatory if the AIA is enacted. Given the market that developed around GDPR compliance, this area warrants close attention.

Whether implementing these strategies or launching a new company, adopting simple, responsible AI practices can unlock substantial business opportunities. Thoughtful consideration is essential when deploying AI to avoid creating potentially harmful products.

Fortunately, this careful approach will yield positive results in the long-term success of your business.