Research-Based Workflow for Startups - Is Your Business Ready?

The Growing Investment in AI Research and Development

Forecasts from the President’s Council of Advisors on Science and Technology indicate that U.S. businesses are projected to invest over $100 billion annually in AI R&D by the year 2025.

Currently, a significant portion of this expenditure is concentrated among six major technology corporations: Microsoft, Google, Amazon, IBM, Facebook, and Apple, as highlighted in a recent study conducted by CSET at Georgetown University.

However, a crucial question arises for startups whose products are fundamentally built upon artificial intelligence.

The Challenge for Early-Stage AI Companies

Is it feasible for companies in their early stages to sustain a research-driven approach? Startups and scaleups typically prioritize tangible product development over extensive research.

This emphasis stems from the need to create solutions that resonate with customers, attract investors, and satisfy stakeholders.

The optimal scenario involves a balance between both aspects – innovation through research and delivering practical value.

Assessing Readiness for AI Research

Prior to allocating resources to establish an AI research lab, careful consideration is essential to evaluate your company’s preparedness.

A thorough assessment will help determine if the timing and resources are aligned with the strategic goals of the organization.

Understanding these factors is key to successfully integrating research into the core operations of a growing AI-focused business.

Consider these points before making a decision.

Assembling a Dedicated AI Research Team

If your organization is focused on pioneering AI research, the initial crucial step involves recruiting one or two dedicated researchers. Unbabel successfully implemented this strategy early on by onboarding Ph.D.-level experts and initiating research even before the corresponding product was fully developed.

Some researchers will focus on developing solutions from the ground up, while others will analyze your existing data to identify and adapt pre-trained models to your specific requirements.

Following this, the recruitment of research engineers or machine learning operations (MLOps) professionals becomes essential. It's important to recognize that research constitutes only a fraction of the effort required to deploy AI into a production environment.

Research engineers are responsible for transitioning research findings into practical applications, continuously monitoring model performance, and refining models when their predictive accuracy declines or deviates from expected behavior. Automation is frequently employed by these professionals to streamline both monitoring and deployment processes, reducing the need for manual intervention.

These tasks typically fall outside the expertise of a research scientist, whose primary focus lies on working with datasets and models during the training phase. However, a collaborative and iterative feedback loop between researchers and engineers is vital for refining and retraining models based on real-world inference performance.

Key Roles in an AI Research Team

• Researchers: Focus on model development and adaptation.
• Research Engineers/MLOps: Responsible for deployment, monitoring, and refinement.

Effective collaboration between these roles is paramount for successful AI implementation.

Identifying Key Areas for Improvement

Research from CSET indicates that a significant majority – 85% – of AI laboratories located in North America and Europe prioritize basic AI research. Conversely, less than 15% concentrate on practical development. Global trends differ, with most labs in nations like India and Israel centering their efforts on development initiatives.

The disparity stems from resource allocation. While entities like Google’s X division possess the financial resources to pursue ambitious, long-term projects, the majority of startups must concentrate on innovation that yields a demonstrable competitive edge or enhances existing products.

Applied research is frequently employed to address a defined challenge or facilitate product enhancement. This approach often leverages pre-existing, publicly available machine learning models in conjunction with proprietary data sets.

This methodology can be implemented to reduce operational costs, optimize processes, evaluate product performance, or introduce new functionalities. The accessibility of APIs simplifies applied research, enabling the integration of academic advancements into products and subsequent measurement of success against key business indicators.

A further development-oriented research type is fundamental research. This is undertaken when novel solutions are required for unresolved problems or when necessary datasets are unavailable.

Our team, for example, addressed existing challenges in quality estimation by creating a completely new metric for evaluating machine translation quality. This development was released as open-source to benefit the wider community and avoid redundant effort.

Although not directly linked to specific product features, this work has provided valuable insights for both our internal team and our clientele regarding product functionality and potential areas for improvement. A well-defined business rationale is crucial for securing alignment at both executive and board levels.

Leveraging Research for Talent Acquisition

Recent data indicates a resurgence in the demand for Artificial Intelligence, following a brief dip during the COVID-19 pandemic. Simultaneously, a significant talent shortage exists – almost 40% of organizations identify a deficiency in specialized skills as a primary obstacle to AI implementation.

When establishing an AI research division, a strategic approach involves hiring individuals with a combined affiliation: a position within an AI-centric academic institution alongside their role at the company. Proximity to a strong academic hub is also advantageous, facilitating access to a skilled workforce.

For instance, locating near a prominent university allows for engagement with master’s and doctoral students. Offering summer internships can serve as a pathway to full-time employment, broadening access to potential candidates and exposing students to career paths they might not have previously explored.

This strategy is frequently employed by leading corporate AI research facilities globally. A notable example is Facebook’s recruitment of Yann LeCun, a leading AI researcher, eight years ago.

LeCun continues to serve as both VP and Chief AI Scientist at Facebook, while also maintaining his professorship at New York University. Such arrangements foster collaboration between the company and the university, as demonstrated by their recent work to accelerate MRI scan processing.

Cultivating these relationships can strengthen connections and mitigate concerns about a “brain drain” from universities. It’s crucial to ensure research efforts benefit the company, the academic institution, and its students.

Joint research initiatives, like the Facebook example, present substantial opportunities. Currently, we are collaborating with Carnegie Mellon University in Portugal, alongside other academic institutions, on a project focused on a multilingual conversational agent. This collaborative model yields benefits for all participating parties.

Benefits of Academic-Industry Partnerships

• Expanded Talent Pool: Access to students and researchers.
• Reduced Recruitment Costs: Internships as a pre-hire pathway.
• Innovation & Collaboration: Joint research projects drive advancements.
• Positive Reputation: Demonstrates commitment to academic development.

Ultimately, a symbiotic relationship between industry and academia is a powerful tool for navigating the challenges of the AI talent shortage and fostering continued innovation.

Establishing Realistic Expectations for AI Investment and Return

Many startups and rapidly growing companies operate with a rapid pace of resource allocation and research and development, specifically tailored for product creation. However, academic-style research cannot be expedited. Achieving success in this field demands a deliberate and systematic approach that requires a significant time commitment.

Consider whether your organizational culture embraces the concept of long-term research as a pathway to achieving cutting-edge results. Alternatively, assess if your objectives are overly ambitious and potentially unrealistic.

Presenting a clear rationale for establishing a research function to the board of directors early on is beneficial. Should the leadership team express reservations regarding its value, a phased approach can be adopted. This involves initially hiring a single researcher and supplementing internal efforts by outsourcing research tasks to relevant communities.

Existing platforms like Kaggle, or specialized communities – such as the natural language processing community that collaborates with universities – can provide valuable support for projects.

Collaborating with universities offers opportunities to apply for joint research grants, sponsor master's or doctoral students, or engage in year-long projects that integrate student contributions with company personnel.

Financially, an AI lab should be categorized alongside other R&D initiatives. Securing investment from leadership or the board is more achievable when you can demonstrate the potential impact and justify the value of successful projects.

Some organizations are naturally inclined towards research due to their inherent structure, while others may encounter greater resistance in articulating the benefits and aligning expectations with practical realities.

Begin by thoroughly understanding your product and identifying the key performance indicators (KPIs) that would be affected by successful research outcomes. This understanding will enable you to prioritize research efforts effectively.

To validate your concepts, it’s crucial to develop datasets that accurately reflect the product KPIs. This ensures you have the necessary documentation to demonstrate the link between research and tangible product improvements when questioned.

Prioritizing Research Initiatives

• Clearly define product understanding.
• Identify relevant KPIs.
• Develop proxy datasets.
• Document the value proposition.

Establishing these foundations will allow for a more transparent and justifiable approach to AI research investment.

Establishing and Maintaining a Successful AI Laboratory

The creation and continued operation of an AI lab represents a significant financial investment and carries the potential for diverting focus from core business objectives. A primary expense typically centers around computational resources, whether procured through on-site infrastructure or cloud-based GPU solutions.

Proactive cost forecasting, coupled with consistent synchronization between research endeavors and product development, is paramount to success.

Effective research necessitates a clearly defined plan and methodology for integrating findings directly into the product lifecycle. Disconnect between the product team’s goals and the lab’s objectives will inevitably lead to research stagnation.

However, with appropriate alignment and dedicated support, a company can attract highly skilled researchers.

These researchers, feeling professionally satisfied, can drive substantial innovations that would likely remain unattainable through product development alone.

Consider these key elements when building your AI lab:

• Cost Management: Carefully plan for computing expenses.
• Strategic Alignment: Ensure research directly informs product development.
• Talent Acquisition: Attract and retain top-tier researchers.

By prioritizing these factors, organizations can maximize the return on investment in their AI initiatives and unlock transformative potential.