From Microsoft’s 485,000 Nvidia Chips to Databricks’ $43B Valuation

The AI Compass

December 18, 2024

Your competitors are keeping an eye on AI — are you? We want to make it easy for you. Each week, we select and demystify the top five AI news items for business, product, and technology leaders.

Cut through the noise! Spend just five minutes per week with The AI Compass to stay ahead and make informed decisions.

Subscribe now!

The AI industry is undergoing significant shifts that are shaping the future of business and technology. Microsoft's major investment in Nvidia's advanced AI chips positions the company ahead in the new AI infrastructure competition. AI firms are receiving unprecedented venture capital funding, capturing 42% of US investments in 2023. However, challenges such as AI scaling limitations suggest the possible end of the rapid growth era, and a recent report highlights safety shortcomings among leading AI labs. Additionally, Grammarly's acquisition of Coda, with a new CEO at the helm, aims to redefine AI productivity tools. Navigating these developments is essential for leaders to capitalize on opportunities while mitigating risks in the evolving AI landscape.

AI Companies Ranked on Safety Measures: Key Findings and Areas for Improvement

Unsplash

As companies race to develop more powerful AI systems, safety measures are often falling behind. A recent report by the Future of Life Institute, a nonprofit dedicated to reducing global catastrophic risks, shines a light on this critical issue. The report evaluates several leading AI companies—including OpenAI, Google DeepMind, Meta, and others—on their approaches to AI safety, revealing significant gaps and areas for improvement.

The institute assembled a panel of seven independent experts, including Turing Award winner Yoshua Bengio and Sneha Revanur from Encode Justice. They assessed the companies across six key areas: risk assessment, current harms, safety frameworks, existential safety strategy, governance and accountability, and transparency and communication. These criteria were chosen to provide a comprehensive view of how well each company is addressing both immediate and long-term risks associated with AI technologies.

One of the critical findings of the report is that all the flagship AI models evaluated are vulnerable to "jailbreaks"—techniques that can override the system's safety guardrails. This vulnerability poses significant risks, as it could allow users to manipulate AI systems into providing harmful or inappropriate outputs.

Furthermore, the panel noted that current strategies employed by these companies are inadequate for ensuring that future AI systems, which may rival human intelligence, remain safe and under human control. Stuart Russell, a professor of computer science at the University of California, Berkeley, stated, "Although there is a lot of activity at AI companies that goes under the heading of 'safety,' it is not yet very effective."

The report highlights that some companies are neglecting basic safety measures—referred to as "low-hanging fruit." For example, adopting existing guidelines for risk assessments is a relatively straightforward step that companies like Zhipu AI, x.AI, and Meta could implement to improve their safety posture. By adhering to established frameworks, these companies could better identify potential risks and develop strategies to mitigate them.

Addressing the fundamental risks inherent in current AI model production will require significant technical breakthroughs. Traditional AI models are often "black boxes"—complex systems whose internal workings are not easily understood. This opacity makes it challenging to guarantee safety, especially as AI systems become more powerful. Researchers are actively exploring techniques to make AI models more interpretable and transparent, which could help in monitoring and controlling their behavior.

Among the companies evaluated, Anthropic, the developer of the chatbot Claude, ranked the highest in terms of safety measures, receiving a C grade. However, even this highlights that there is substantial room for improvement across the industry. Meta, the parent company of Facebook and developer of the Llama series of AI models, scored the lowest with an F-grade overall. OpenAI and Google DeepMind each received a D+.

The report underscores the need for independent oversight in AI development. Relying solely on companies to self-regulate may not be sufficient to ensure safety. Tegan Maharaj, an assistant professor at HEC Montréal and one of the panelists, emphasized the importance of accountability and external evaluation.

Yoshua Bengio noted the value of initiatives like the AI Safety Index in holding firms accountable and encouraging the adoption of more responsible approaches. "They are an essential step in holding firms accountable for their safety commitments and can help highlight emerging best practices," he said.

Trend Micro Introduces Advanced 'AI Brain' to Enhance Automated Cybersecurity Defenses

Unsplash

Trend Micro, a leading cybersecurity vendor, has introduced a new and advanced "AI brain" within its security tools to automate and enhance threat defenses. For years, companies have anticipated that AI-enabled tools would transform cybersecurity by predicting attacks and automatically addressing new vulnerabilities. That future is now becoming a reality.

Cyberattacks often exploit human errors, such as delayed patching of security flaws or failing to recognize hackers impersonating legitimate users. Security teams are overwhelmed with numerous alerts each day about potential threats to their systems. Trend Micro's AI brain powers autonomous agents that can assess these alerts and respond to them automatically, reducing the burden on human teams.

What sets Trend Micro's AI brain apart from other AI-powered cybersecurity tools is its extensive training and capabilities. The AI brain has absorbed every published book in the cybersecurity industry, processed over 35 years of Trend Micro's blogs and internal documents, and been trained on the company's global threat research. This rich and unique dataset enables the AI to act not just as an assistant but as an adviser or even a "commander" that can predict attacks and evaluate the risks of emerging threats.

Furthermore, the AI brain continuously updates itself with the latest threat intelligence. By integrating real-time data from global cyber threat landscapes, it adapts to new attack vectors and evolving tactics used by malicious actors. This ensures that the AI remains current and effective in countering emerging cybersecurity challenges.

Trend Micro allows customers to control what proprietary data the AI can access, addressing privacy concerns that businesses often have with AI tools. This flexibility enables companies to leverage the AI's capabilities while maintaining control over sensitive information.

The new autonomous cybersecurity agent, powered by the AI brain, completes tasks with minimal prompting. It can retain institutional knowledge about vulnerabilities and attack histories, which might otherwise be lost when employees leave. Additionally, it can proactively adjust data storage protocols in response to new privacy laws and protect against known ransomware threats by taking preventive actions.

While many customers currently use the AI tools to prioritize workflows and assist with decision-making, Trend Micro anticipates that trust in the AI will grow. Over time, more businesses may allow the AI to perform basic patches and other routine tasks autonomously. "It's evolving," said Rachel Jin, Trend Micro's Chief Enterprise Platform Officer. "Not every company will move to full automation, but we expect them to get closer to an autonomous model."

As one of the first companies to deploy a cybersecurity AI agent to customers, Trend Micro sets itself apart through its depth of knowledge and threat intelligence. In an industry where many vendors are introducing AI tools, the differentiators will be the quality of training data and the ability to predict hackers' next moves. "The key competition is knowledge, skill sets, and threat intelligence," Jin emphasized.

Trend Micro is considering offering an AI-focused product package with advanced workflows, which could help offset increased cloud computing costs. For now, however, the company is providing these AI enhancements to all customers, viewing them as a competitive advantage that should be widely accessible.

Desktop AI Assistants: Unlocking Productivity While Addressing Security Risks

Unsplash

Artificial intelligence has arrived on the desktop. Microsoft 365 Copilot, introduced last year, is now widely available. Apple Intelligence has reached general beta availability for users of recent Macs, iPhones, and iPads. Google Gemini is reportedly on the horizon, with an in-development agent feature dubbed Project Jarvis that may soon allow AI to take actions through the Chrome browser.

These AI assistants hold immense promise for knowledge workers, offering capabilities like sifting through business information and automating tasks—so-called "agentic" abilities. However, they also raise significant concerns for business leaders and chief information security officers (CISOs). According to a Gartner survey, 40% of firms delayed their rollout of Microsoft 365 Copilot by three months or more due to security worries, stemming from issues like oversharing of information and inadequate access controls.

"The combination of these powerful AI capabilities with existing security vulnerabilities in many businesses poses a significant risk," says Jim Alkove, CEO of Oleria, an identity and access management platform for cloud services. "When you combine large language model technology with the runtime security risks we've been dealing with—information access and auditability risks—it ends up having a multiplicative effect on risk."

Despite these concerns, desktop AI is set to take off in 2025. Companies are eager to adopt technologies like Microsoft 365 Copilot and other desktop AI systems. According to Gartner's "The State of Microsoft 365 Copilot: Survey Results," while only 16% have rolled out the technology to all workers, 60% are evaluating it in pilot projects, and a further 20% are in the planning stages.

Workers are enthusiastic about AI assistants. Gartner found that 90% of respondents believe their users would fight to retain access to their AI assistant, and 89% agree that the technology has improved productivity.

However, a key challenge is that these technologies are often "black boxes" in terms of their architecture and protections, leading to a lack of trust. With a human personal assistant, companies can conduct background checks, limit access to certain systems, and audit their work—measures that are currently difficult to replicate with AI assistants.

"AI assistants—whether on the desktop, mobile device, or in the cloud—often have far more access to information than they need," Alkove explains. "You can grant your assistant access to email and your calendar, but you cannot restrict them from seeing certain emails or calendar events. They can see everything."

This highlights the need for organizations to establish granular control over AI assistants to ensure they only access and perform necessary tasks without overstepping boundaries. Implementing such controls requires a security framework that allows for delegated tasks with specific permissions and time limits.

"How do you grant access to portions of your information and portions of the actions you would normally take as an individual, to that agent, and also only for a period of time?" Alkove asks. "You might only want the agent to take an action once, or only for 24 hours. Making sure you have those kinds of controls is critical."

Without such security designs and controls, attacks are likely to follow. Earlier this year, a prompt injection attack scenario highlighted the risks to businesses. Security researcher Johann Rehberger demonstrated that an indirect prompt injection attack through email, a Word document, or a website could trick Microsoft 365 Copilot into acting as a scammer, extracting personal information, and leaking it to an attacker. While Rehberger notified Microsoft of the issue, it's unknown whether a comprehensive fix has been implemented.

The ability of AI assistants to access operating system capabilities or device functions makes them attractive targets for fraudsters. "An AI assistant gives them the ability to do things on your behalf without specific consent or control," says Ben Kliger, CEO of Zenity, an AI agent security firm. "Many of these prompt injection attacks are trying to social engineer the system—going around other controls in your network without having to socially engineer a human."

To mitigate these risks, organizations need visibility into and control over the security of AI technology. This includes examining what the AI system is doing, how employees are interacting with it, and what actions are being delegated to the AI.

"These are all things that the organization needs to control, not the agentic platform," Kliger says. "You need to break it down and look deeper into how those platforms are being utilized, and how people build and interact with them."

There is a growing recognition of the need for industry best practices or frameworks to guide companies in securely integrating desktop AI systems with agentic capabilities. While such frameworks are still emerging, companies can start by implementing strong identity and access management systems, conducting regular audits, and applying the principle of least privilege to AI assistants.

Microsoft acknowledges the data-governance challenges posed by AI assistants but argues that these issues are not new—just made more apparent due to AI's arrival.

"AI is simply the latest call to action for enterprises to take proactive management of controls—their unique, respective policies, industry compliance regulations, and risk tolerance should inform—such as determining which employee identities should have access to different types of files, workspaces, and other resources," a company spokesperson said in a statement.

Microsoft points to its Microsoft Purview portal as a way for organizations to continuously manage identities, permissions, and other controls. Using the portal, IT administrators can help secure data for AI apps and proactively monitor AI use through a single management location.

As for Google and Apple, specific security measures they plan to implement remain less publicly detailed. Google declined to comment on its forthcoming AI agent. However, companies should anticipate that similar security considerations will apply and prepare accordingly.

In conclusion, desktop AI assistants offer significant productivity benefits but come with security risks that organizations must address proactively. By establishing granular control over AI assistants, implementing robust security measures, and staying informed about emerging best practices, companies can harness the power of AI while protecting their sensitive data.

UK Proposes Allowing AI Training on Copyrighted Works with Opt-Out and Considers 'Right of Personality' Protections

Unsplash

The UK government has introduced a proposal that would allow technology companies, such as Google and OpenAI, to train their AI models using copyrighted content. This initiative aims to clarify legal uncertainties and encourage investment in AI development within the UK. A key part of the proposal is an opt-out scheme for creators who do not want their work used for AI training.

The opt-out mechanism is intended to be straightforward and effective, giving creators control over their copyrighted material. Creators would be able to notify AI companies of their decision to opt out, preventing unauthorized use of their work. The government emphasizes the need for transparency from AI developers about the materials used in training, ensuring that creators can exercise their rights easily.

Data Protection Minister Chris Bryant described the proposal as a "win-win," offering creators control and potential new revenue through licensing agreements. However, the proposal has faced criticism from those advocating for creatives' rights. Book publishers have called it "untested and unevidenced," worried about its impact on smaller creators who may not have the resources to enforce their rights effectively.

Questions have been raised about how the proposed copyright exemption fits with existing intellectual property laws. Critics argue that it could undermine established protections for creators, and they are calling for more evidence and legal support for the change. News media organizations fear it may allow AI companies to avoid responsibilities, affecting fair payment for content use.

In addition to the copyright exemption, the government is considering introducing a "right of personality," similar to laws in the US. This would protect individuals from unauthorized AI use of their likeness or voice. The move comes after incidents where AI technologies replicated celebrities' voices without consent, such as in the case of Scarlett Johansson. Introducing this right could have significant implications for AI developers, requiring them to navigate new legal obligations when creating models that simulate human features.

Tech UK, representing technology companies, supports the proposal. They highlight that it allows creators to reserve their rights while promoting innovation in AI. The government is seeking feedback on both the copyright exemption and the "right of personality," aiming to address concerns and refine the proposals before they are implemented.

Grammarly to Acquire Coda, Appoint New CEO, Aiming to Redefine AI Productivity

Grammarly, the well-known AI assistant, has announced its intent to acquire the productivity platform Coda. As part of the acquisition, Coda’s CEO and co-founder, Shishir Mehrotra, will become the new CEO of Grammarly.

This strategic move aims to transform Grammarly from a writing assistant into a comprehensive AI productivity platform. By integrating Coda's flexible and powerful AI tools and surfaces, Grammarly plans to offer customers access to company knowledge, generative AI chat, a productivity suite, and trusted AI agents. This aligns with Grammarly's long-term vision of a world where humans and AI work together seamlessly across all work environments.

Alex Shevchenko, Grammarly co-founder, stated, “For the past 15 years, Grammarly has been building an AI superhighway with the capacity to do much more than writing assistance. The acquisition of Coda is a big step toward achieving our vision to redefine productivity for the AI era.”

Shishir Mehrotra expressed enthusiasm about the combined potential of the two companies. “We have a massive opportunity to reinvent productivity as we know it. Combining Grammarly’s trusted AI assistant with the intelligence and flexibility of Coda can provide our customers with the productivity gains they need across their entire organization,” he said.

The integration of Coda's AI tools is expected to enhance Grammarly's existing products and services for current customers. Grammarly plans to make its AI assistant smarter and more contextually aware by leveraging company knowledge from Coda Brain. This will enable permission-aware connections with other systems and applications—from email and documents to CRM and project trackers—allowing users to work smarter and faster.

Grammarly and Coda products will continue to function as customers know and love. In the future, Grammarly intends to unify its assistant with Coda Docs to provide a dedicated surface where customers can collaborate with AI from start to finish.

Shishir Mehrotra's appointment as the new CEO signifies a significant step in Grammarly's organizational direction. A tech veteran with over 25 years of experience, Mehrotra has a track record of building platforms that redefine and lead categories. Prior to founding Coda, he served as YouTube’s Chief Product Officer and Chief Technology Officer, helping grow it into the world’s largest video destination. His leadership is expected to bring fresh perspectives and drive Grammarly's mission to redefine productivity with AI.

Hemant Taneja, CEO and Managing Director of General Catalyst, commented on the acquisition: “I believe this will create a market leader in applied AI for enterprises, combining two companies with complementary strengths into a platform that empowers the modern workforce. Grammarly is now building products that leverage AI to boost productivity, reduce costs, and make workers' lives easier in the platforms they’re already using today.”

With this acquisition, Grammarly is increasingly focused on how AI agents can not only improve applications but also benefit entire enterprises by increasing productivity and delivering better business outcomes. The combined strengths of Grammarly and Coda aim to empower over 50,000 teams at companies like Atlassian, Ford, Chevron, Figma, DoorDash, Square, and The New York Times.

The goal is clear: to bring Grammarly and Coda together as an AI productivity platform equipped with the skills, tools, and knowledge customers need to work smarter and faster in the AI era.

OpenAI Debuts o1 Model, Real-Time Enhancements, and Preference Fine-Tuning in December 2024 Updates

Unsplash

In a bid to fortify its presence in the competitive AI landscape, OpenAI has introduced a fresh lineup of model upgrades, developer tools, and cost reductions aimed at driving commercial adoption and increasing the flexibility of its platform. Chief among these announcements is OpenAI o1, a next-generation reasoning model designed to handle intricate, multi-step tasks with heightened accuracy and efficiency. Complemented by enhancements to the Realtime API—most notably WebRTC integration—and a new Preference Fine-Tuning method, the latest suite of tools aims to empower developers to build richer, more responsive, and more tailored AI applications.

1. Next-Generation Model: OpenAI o1

OpenAI’s new flagship model, o1, represents a meaningful leap beyond its predecessors—such as GPT-4—particularly for businesses and developers looking to implement complex, multi-step reasoning capabilities. While GPT-4 established a high watermark for general-purpose language processing, o1 intensifies the model’s problem-solving acumen with a series of targeted upgrades.

Notably, o1 introduces robust support for function calling, a feature that enables the model to interface seamlessly with external data sources and APIs. This integration allows developers to orchestrate real-time responses and dynamically retrieve fresh information—key for customer support bots, supply chain analytics, and financial forecasting tools, where timely accuracy is paramount. The model’s newly introduced “developer messages” feature further streamlines the customization process. Developers can define a model’s stylistic and behavioral parameters—such as tone, formatting, or adherence to brand guidelines—right within the request, reducing the need for post-processing.

Structured Outputs, another highlight, ensure that responses conform rigorously to predefined JSON Schemas. This eliminates guesswork in parsing model outputs and enhances reliability for downstream tasks. Just as crucial, o1 incorporates vision capabilities, allowing it to reason over images. This opens doors for use cases in manufacturing (e.g., visual quality checks), scientific research (e.g., image-based data extraction), and even advanced coding tasks that require the interpretation of diagrams or screenshots.

Compared to its o1-preview predecessor, the finalized o1 model consumes fewer “reasoning tokens,” thereby reducing latency. Developers can dial in the complexity of the model’s thought process using the new reasoning_effort parameter, balancing speed against depth of analysis. Across multiple benchmarks—ranging from general question answering and coding evaluations to intricate math and agent tasks—o1-2024-12-17 outperforms both o1-preview and the established gpt-4o variant, underscoring that these enhancements are not merely incremental but transformative.

2. Realtime API Enhancements: WebRTC Integration and New Audio Models

OpenAI’s Realtime API receives a significant overhaul, aimed at improving the responsiveness and adaptability of real-time conversational experiences. Central to this upgrade is WebRTC integration, an open standard widely used for peer-to-peer, browser-based communications. By embracing WebRTC, OpenAI removes a substantial engineering burden from developers who aim to deploy voice applications. This built-in support manages audio encoding, noise reduction, and congestion control, enabling fluid interactions over variable network conditions. As a result, developers can stand up real-time voice assistants, translation tools, or interactive tutors with minimal code overhead.

Further advancements come in the form of GPT-4o audio and GPT-4o mini models, now available through the Realtime API at substantially reduced costs. The introduction of GPT-4o mini makes high-quality voice interfaces accessible at a fraction of the price, broadening the use cases for voice-driven applications—particularly in domains like e-commerce customer service, virtual “concierge” experiences, or scaled-down IoT devices that require natural language interfaces without hefty compute expenses.

Developers gain additional knobs for fine-tuning their real-time sessions. Features such as concurrent out-of-band responses allow background tasks—like content moderation or classification—to proceed without interrupting the main user interaction. The capability to define custom input contexts and precisely control response timing provides developers with a granular handle on when and how models respond. These refinements collectively reduce latency and improve the overall quality and reliability of voice-based AI services.

3. Preference Fine-Tuning: A More Nuanced Path to Customization

Beyond model performance and connectivity improvements, OpenAI is also reshaping how developers refine a model’s behavior. The newly introduced Preference Fine-Tuning method eschews the traditional supervised approach of training on fixed input-output pairs. Instead, it employs a comparative strategy: developers supply pairs of model-generated outputs, indicating which one they prefer. Over time, the model internalizes these preferences, increasing the likelihood that it will produce the “better” response style while minimizing the risk of undesirable outputs.

The key advantage of Preference Fine-Tuning is its adaptability to subjective or nuanced criteria. While traditional supervised fine-tuning excels at tasks with clear “correct” answers, Preference Fine-Tuning is well-suited for scenarios where excellence is a matter of style, tone, or subtle qualitative differences. For example, a financial analysis assistant might need to break down complex metrics with a certain level of detail or a specific tone that resonates with professional users. Through iterative comparisons, developers can steer the model’s voice to align more closely with user expectations, increasing overall satisfaction and utility.

Enterprises and developers that have tested Preference Fine-Tuning report tangible improvements. In one instance, a financial analytics assistant not only met strict accuracy benchmarks but also handled out-of-distribution queries—such as requests for niche metrics or custom KPIs—more gracefully after undergoing Preference Fine-Tuning.

Navigating the AI Data Drought: How Developers Are Addressing the Looming Shortage

Unsplash

AI developers are rapidly exhausting the Internet's reservoir of data to train large language models (LLMs) like those behind ChatGPT. As we approach a potential data scarcity, the AI community is seeking innovative solutions to sustain the growth and advancement of AI technologies.

The Internet, though vast, is not infinite. The remarkable progress in AI over the past decade has been largely driven by scaling up neural networks and feeding them enormous amounts of data. This approach has been surprisingly effective in enhancing LLMs' abilities to replicate conversational language and develop emergent properties like reasoning. However, experts warn that we are nearing the limits of this scaling strategy.

One prominent study by researchers at Epoch AI, a virtual institute, projects that by around 2028, the size of datasets used to train AI models will reach the same size as the total estimated stock of public online text. In other words, AI could run out of readily available training data in about four years. Simultaneously, data owners—such as newspaper publishers—are increasingly restricting access to their content, further tightening data availability.

This impending data bottleneck is already causing concern. "I strongly suspect that's already happening," says Shayne Longpre, an AI researcher at the Massachusetts Institute of Technology. The scarcity of high-quality training data could slow down the rapid improvement of AI systems.

Alternative Data Sources and Methods

To address the shortage of high-quality training data, AI developers are exploring several alternative approaches:

Synthetic Data Generation: Companies like OpenAI are generating new data using existing models. By creating synthetic data that mimics natural language, developers can augment their datasets without relying solely on external sources.

Unconventional Data Sources: Developers are tapping into less traditional sources of text, such as specialized industry publications, academic papers, and multilingual content. This diversification can provide fresh training material and improve models' versatility.

Data Partnerships: Forming partnerships with content providers allows AI companies to access proprietary data legally and ethically. Collaborative agreements can benefit both parties through compensation and controlled data usage.

Impact of Legal Restrictions

Legal actions and tighter content control by data owners are significantly impacting the AI landscape. Lawsuits filed by major publishers against AI companies for copyright infringement highlight the growing tensions. If courts rule in favor of content providers, requiring financial compensation for data use, it could make it more challenging for AI developers, especially those with limited resources, to access necessary data.

These restrictions could slow the pace of AI advancements, as developers may face higher costs and legal barriers in obtaining quality training data. "Academics will be most hit by these deals," notes Longpre. The potential decline in open-access data could hinder innovation and democratization in AI research.

Shift to Specialized Models

The data scarcity might catalyze a shift from large, general-purpose LLMs to smaller, specialized models. Focusing on domain-specific applications allows developers to train models on smaller, more manageable datasets tailored to particular industries or tasks.

A move toward specialized models could:

Enhance Efficiency: Smaller models require less computational power and data, reducing costs and environmental impact.

Improve Performance: Specialized models can outperform general models in specific domains by leveraging expert knowledge and targeted data.

Promote Innovation: Fostering the development of niche AI applications can lead to breakthroughs in areas like healthcare, finance, and engineering.

Looking Ahead

While the data drought poses challenges, the AI community remains proactive. Developers are innovating to overcome data limitations, whether through synthetic data, new data sources, or strategic partnerships. The industry may evolve toward a more sustainable model that balances scaling with efficiency.

The potential shift toward specialized AI models could redefine the landscape, emphasizing quality over quantity. This evolution offers opportunities for companies to leverage AI in more targeted, effective ways, aligning with specific business needs and sectors.

Leading AI Labs Receive Low Marks for Safety in New Report

Unsplash

A recent report by the Future of Life Institute (FLI) has revealed significant shortcomings in the safety testing practices among top AI labs. The institute commissioned a team of external experts, including renowned AI researcher Yoshua Bengio, to evaluate the safety measures and testing protocols of leading AI companies such as Meta, OpenAI, Google DeepMind, and Anthropic.

The assessment was based on specific criteria, including the robustness of safety protocols, transparency in model development, the ability to prevent misuse, and compliance with proposed safety standards. The evaluation metrics focused on how well these companies understand and control the behavior of their AI systems, particularly in preventing harmful or unintended outcomes.

The findings were concerning: Meta received a failing grade largely due to its open model system, which allows users to bypass safety measures easily. OpenAI and Google DeepMind both scored D+, with issues related to insufficient transparency and inadequate safety testing. Anthropic performed slightly better with a C grade but still fell short of meeting key safety standards.

Key shortcomings identified in the report include a lack of rigorous safety testing before deploying new AI models, insufficient mechanisms to prevent misuse, and inadequate transparency about how models operate and are improved over time. For instance, OpenAI faced criticism for the limited detail in its "system card" for its latest model, o1, which does not fully reflect ongoing modifications that are not publicly documented.

Max Tegmark, president of the FLI, highlighted the problematic incentive structures within AI companies that allow them to set and enforce their own safety standards without external oversight. He emphasized the absence of legally mandated safety standards in the AI industry, contrasting it with industries like pharmaceuticals, where strict regulatory frameworks ensure safety and efficacy before products reach the market.

In response to these challenges, there are growing calls for the establishment of legally mandated safety standards in the AI sector. Such regulations could involve compulsory safety assessments, third-party audits, and compliance with standardized testing protocols before AI systems are deployed. The FLI plans to conduct these evaluations biannually, aiming to encourage companies to enhance their safety practices through competition for higher rankings.

The report underscores the urgent need for more robust safety measures and regulatory oversight as AI systems become increasingly powerful and integrated into critical aspects of society. Without significant improvements, there is a risk that AI technologies could produce unintended and potentially harmful consequences.

UAE's Falcon 3 Challenges Open-Source Leaders Amid Surging Demand for Efficient Small AI Models

Technology Innovation Institute

The Technology Innovation Institute (TII), backed by the UAE government, has announced the launch of Falcon 3, a family of open-source small language models (SLMs) designed for efficient operation on lightweight, single GPU-based infrastructures. Falcon 3 aims to democratize access to advanced AI capabilities for developers, researchers, and businesses by offering models that are both powerful and accessible.

Key Features and Performance

Falcon 3 features four model sizes—1B, 3B, 7B, and 10B parameters—with both base and instruct variants. These models have been trained on 14 trillion tokens, more than double that of their predecessor, Falcon 2. The models employ a decoder-only architecture with grouped query attention (GQA), a mechanism that shares parameters to minimize memory usage for the key-value (KV) cache during inference. This leads to faster and more efficient processing, especially when handling diverse text-based tasks.

The grouped query attention mechanism is a refinement in transformer architectures that reduces the memory footprint during inference. By grouping queries in the attention mechanism, GQA reduces the number of unique queries that need to be processed, effectively sharing computations and decreasing the amount of memory required to store intermediate results. This optimization is particularly beneficial for deployment on devices with limited resources, as it enables the models to run efficiently without sacrificing performance.

Falcon 3 supports four primary languages—English, French, Spanish, and Portuguese—and features a 32K context window. This extended context window allows the models to process long inputs, such as detailed documents, making them suitable for applications that require understanding extended text.

According to the leaderboard on Hugging Face, Falcon 3's 10B and 7B models outperform or closely match popular open-source counterparts in their size class, including Meta’s Llama and Alibaba’s Qwen-2.5. In benchmarks testing reasoning, language understanding, instruction following, code, and mathematics tasks, Falcon 3's models achieve state-of-the-art results.

However, in the MMLU (Massive Multitask Language Understanding) benchmark, Falcon 3 models underperform compared to some competitors. MMLU evaluates a model's ability to understand and process human language across a wide range of academic subjects and professional topics, from mathematics and law to medicine and history. The underperformance in MMLU suggests that while Falcon 3 excels in focused tasks, there is room for improvement in broader, more diverse language understanding tasks that require extensive world knowledge and comprehension across multiple disciplines.

Licensing and Compliance

All Falcon 3 models are released under the TII Falcon License 2.0, a permissive license based on Apache 2.0 with an added acceptable use policy. This license encourages responsible AI development and deployment while offering flexibility for both commercial and non-commercial use. Compared to the standard Apache 2.0 license, the TII Falcon License 2.0 includes specific provisions that prohibit the use of the models in applications related to illegal activities, disinformation, and other harmful use cases. This ensures that users can leverage the models freely while adhering to ethical guidelines.

Implications and Future Developments

The launch of Falcon 3 comes at a time when the demand for small language models is rapidly growing. SLMs, with fewer parameters and simpler designs than large language models (LLMs), offer efficiency and affordability, making them suitable for deployment on devices with limited computational resources. They are ideal for various applications across industries such as customer service, healthcare, mobile apps, and IoT, where typical LLMs might be too resource-intensive.

TII aims to serve a broad range of users by enabling cost-effective AI deployments without computational bottlenecks. The Falcon 3 models are versatile; the base models are suitable for generative tasks, while the instruct variants excel in conversational tasks like customer service or virtual assistants.

To help users get started, TII has launched the Falcon Playground, a testing environment where researchers and developers can try out Falcon 3 models before integrating them into their applications.

Looking ahead, TII plans to expand the Falcon family by introducing models with multimodal capabilities, expected to launch in January 2025. These future models will likely integrate processing of different data types, such as text and images, further broadening the potential applications.

NVIDIA Unveils Jetson Orin Nano Super Developer Kit: Affordable AI Supercomputer with Enhanced Capabilities

NVIDIA

NVIDIA has unveiled its most affordable generative AI supercomputer, the Jetson Orin Nano Super Developer Kit, priced at $249. This compact device delivers up to a 1.7x gain in generative AI performance, making advanced AI technology accessible for commercial AI developers, hobbyists, and students. With a 70% increase in performance, boasting 67 INT8 TOPS (Tera Operations Per Second), and a 50% increase in memory bandwidth to 102GB/s compared to its predecessor, the Jetson Orin Nano Super is designed to support a broad range of AI applications.

The developer kit includes a Jetson Orin Nano 8GB system-on-module (SoM) equipped with an NVIDIA Ampere architecture GPU and a 6-core Arm CPU, allowing it to support multiple concurrent AI applications and high-performance inference tasks. It can handle up to four cameras with higher resolution and frame rates, enhancing its capabilities in computer vision tasks.

The Jetson Orin Nano Super is capable of running various AI models, including those for large language model (LLM) chatbots, visual AI agents, and AI-based robots. While it may not have the computational resources to run very large-scale models like GPT-3 natively, it is well-suited for small to medium-sized models and can serve as an edge device interfacing with larger models running on cloud services. This makes it ideal for developers looking to explore and implement generative AI, robotics, or computer vision applications at the edge.

The developer kit supports mixed-precision computation, including FP16 and FP32, enhancing computational efficiency while balancing accuracy and performance. The reliance on INT8 precision enables high throughput with lower power consumption, although it may impact the accuracy of certain generative AI applications that require higher precision. Developers can choose the appropriate precision level based on their application's requirements.

Regarding memory capacity and computational power, the Jetson Orin Nano Super offers 8GB of memory. While this is sufficient for running multiple concurrent AI applications, developers should be mindful of limitations when deploying high-demand tasks like large language models or advanced computer vision algorithms. Techniques such as model optimization, quantization, and pruning can help efficiently utilize available resources.

The Jetson AI lab provides support for cutting-edge models and tutorials, and the broader Jetson community offers extensive resources and inspiration. Software updates are also available to boost performance for existing Jetson Orin NX and Orin Nano systems, ensuring that developers can stay at the forefront of AI innovation.

Microsoft's Bold Investment in Nvidia AI Chips Signals Shift in AI Infrastructure Competition

Shutterstock

Microsoft has made a significant strategic move by acquiring approximately 485,000 of Nvidia’s advanced "Hopper" AI chips this year, which is more than twice the amount purchased by its closest U.S. competitors. This substantial investment positions Microsoft at the forefront of the rapidly evolving artificial intelligence (AI) infrastructure landscape.

As the primary supporter of OpenAI, Microsoft is investing heavily to enhance its AI capabilities under CEO Satya Nadella's leadership. By securing nearly half a million Graphics Processing Units (GPUs), Microsoft is accelerating its AI infrastructure development, giving it a competitive edge over rivals like Meta, Amazon, and Google. Meta, the next largest U.S. purchaser, bought an estimated 224,000 Hopper chips, while Amazon and Google acquired 196,000 and 169,000 respectively, according to analysts at Omdia.

The high demand for Nvidia’s GPUs has exceeded supply for the past two years, largely due to the surge in interest following the launch of OpenAI's ChatGPT. Microsoft's Azure cloud infrastructure plays a critical role in training OpenAI’s latest AI models. The company's aggressive expansion in AI infrastructure is part of its strategy to compete with other tech giants and startups, including Google, Anthropic, and Elon Musk’s xAI, as well as Chinese rivals like ByteDance and Tencent.

In China, companies like ByteDance and Tencent have each ordered about 230,000 of Nvidia's chips this year, including the H20 model, which is a version of the Hopper chip adjusted for U.S. export controls. These export controls and potential changes in U.S. administration policies could significantly impact the global AI chip supply chain. Stricter regulations might limit Chinese access to advanced chips, thereby affecting the competitive dynamics between U.S. and Chinese tech firms.

While Microsoft relies heavily on Nvidia's GPUs, it is also developing its own custom AI chips. Amazon and Google are following a similar path, investing in their proprietary AI accelerators. The development and adoption of these custom chips could alter Nvidia's market position in the long term, as tech giants seek to reduce dependency on third-party suppliers and tailor chips to their specific needs.

Despite rising competition from custom AI chips and from competitors like AMD, Nvidia remains a key player in the AI hardware market. The company's value has soared this year to over $1 trillion due to the tech industry's rush to build large clusters of GPUs. However, Nvidia's stock growth has recently slowed amid concerns about slower growth and increased competition.

The global tech industry is projected to spend $229 billion on servers in 2024, with Microsoft and Amazon leading in capital expenditures. Nvidia GPUs are expected to account for a significant portion of this server spending. Microsoft's strategic investment in Nvidia's GPUs underscores the importance of advanced hardware in developing next-generation AI systems and services.

Microsoft acknowledges the complexity of building AI infrastructure and the necessity of integrating various technological components to provide unique services to its customers. The company's substantial investment in AI hardware, combined with its development of custom chips, reflects a comprehensive approach to securing its position in the future of AI.

Databricks Raises $500 Million at a $43 Billion Valuation, Strengthening Its Position in the AI Landscape

Wikimedia

Databricks, a leading AI and data analytics company, announced in August 2023 that it has raised $500 million in a new funding round, valuing the company at $43 billion. This significant investment underscores the sustained investor confidence in the AI sector and highlights Databricks' pivotal role within it.

Founded in 2013 and headquartered in San Francisco, Databricks provides an integrated platform that unifies data engineering, machine learning, and analytics. Its Lakehouse Platform enables organizations to consolidate their data warehouses and data lakes into a single, unified system. This simplifies data management and accelerates the development of AI applications, such as chatbots and predictive analytics tools.

The recent funding round was led by T. Rowe Price Associates, with participation from existing investors including Andreessen Horowitz, BlackRock, and Tiger Global Management. This influx of capital is set to fuel Databricks' continued innovation, support strategic acquisitions, and drive international expansion. The company also plans to offer employees the opportunity to sell a portion of their shares, providing liquidity while reinforcing commitment to its long-term vision.

Databricks has demonstrated robust financial performance, with an annualized revenue run rate exceeding $1.5 billion as of 2023. Serving over 10,000 customers globally—including industry leaders like Shell and Comcast—the company plays a critical role in helping organizations leverage big data and AI to drive innovation and operational efficiency. Over 500 of these customers are expected to contribute more than $1 million each in annual recurring revenue.

The substantial funding reflects a broader trend of significant venture capital investment in AI and data analytics firms. Organizations across various sectors are increasingly seeking to harness AI capabilities to gain competitive advantages. Databricks' ability to simplify complex data processes and empower businesses with AI tools positions it favorably in a competitive market landscape.

It's noteworthy that Thrive Capital, a prominent investor in AI-focused companies, has made substantial investments in the sector, including leading OpenAI's funding round in earlier years. While Thrive Capital is not reported to have led this particular funding round for Databricks, its involvement in AI investments signals a continued strong interest in the sector. Regarding legal concerns, The New York Times has considered legal action against AI companies like OpenAI over copyright issues. However, there are currently no direct legal implications for Databricks. The company specializes in providing AI infrastructure and tools rather than generating content, which may mitigate potential risks related to content copyright infringement.

AI Scaling Challenges Signal End of Gold Rush Era, Shifting Competitive Landscape

Shutterstock

Amid the many uncertainties in the AI field, one thing has always seemed clear: bigger and more expensive systems produce better results. This belief drove relentless fundraising by model developers like OpenAI, valued at $157 billion, and massive capital expenditures by big tech companies. However, this certainty is now fading. Researchers have run out of new data to train their software, and simply throwing more resources at the problem is no longer yielding smarter outputs. This shift may mark the end of the gold rush phase, opening the field for more agile competitors.

Evidence suggests that AI scaling laws are unraveling. Cutting-edge systems have already absorbed most of the world's useful and available training data. Multiple AI labs are struggling to improve the next generation of models. Sundar Pichai, CEO of Alphabet, recently noted that leading models have reached similar performance levels, and making further improvements has become more challenging. Similarly, Sam Altman, CEO of OpenAI, acknowledged that the easy gains from scaling have diminished, even though he believes there is "no wall" to progress.

The impact is significant: larger models are no longer guaranteeing better performance. This development undermines the previous arms race for chips and data centers. Analysts had expected companies like Microsoft to spend $64 billion on capital expenditures in 2025, six times as much as General Motors. Investors supported this surge; the combined market capitalization of Alphabet, Amazon, Meta Platforms, Microsoft, and Nvidia has increased by $8 trillion since OpenAI released ChatGPT in November 2022.

With the effectiveness of brute-force scaling in question, researchers are turning to algorithmic improvements. Techniques like "test-time compute" aim to enhance the inference process—the moment when a customer uses the AI system. By allowing models extra time to spot patterns or use new data, they can potentially deliver better results. This shift could alter the competitive landscape. If creating competitive AI products no longer requires enormous computing power, it lowers barriers to entry. Startups can develop specialized AI solutions at minimal cost, possibly based on open-source models from companies like Meta. This opens opportunities for new players to serve specific industries such as legal services or software development.

Established tech giants may also feel the effects. On one hand, companies like Microsoft and Alphabet can reduce their massive investments, pleasing shareholders wary of overenthusiasm in new technologies. Past booms in railroads and telecommunications highlight the dangers of excessive spending without guaranteed returns. On the other hand, they may face increased competition from nimble startups.

For companies like Nvidia, the implications are profound. Nvidia has benefited enormously from the demand for its graphics processing units (GPUs), essential for training large AI models. Elon Musk's xAI, for example, recently planned to build a supercomputer with 1 million GPUs. If the industry shifts away from brute-force scaling, demand for such large-scale computing power could decline. Companies might opt for more specialized, cost-effective chips, putting Nvidia's $3.3 trillion equity value at risk.

Regardless of which companies prevail, a slowdown in AI training costs is likely good news for investors. The cost of processing data has plummeted—from $60 to process a million tokens three years ago to just 6 cents today—as per venture firm Andreessen Horowitz. This cost deflation should aid adoption, allowing progress to spread more widely. At Meta, for example, quarterly ad revenue has increased by 46% since the pre-ChatGPT period, possibly due to better advertising targeting, while operating costs have risen only 5%.

After the gold rush comes the hard task of proving a return on investment and justifying investors’ high expectations. The industry faces a turning point: as the scaling laws that drove explosive growth lose their potency, success will depend on innovation and efficiency rather than sheer size. This new phase could lead to a more balanced and competitive market, benefiting both businesses and consumers.

AI Companies Receive 42% of U.S. Venture Capital Investment in 2023

Unsplash

Venture capital investment in the United States has shifted dramatically towards AI companies at an unprecedented rate, according to HSBC Innovation Banking's latest quarterly outlook for the U.S. technology sector, "Innovation Horizons," released on Monday (Oct. 16).

In 2023, 42% of U.S. venture capital was invested in AI companies, up from 36% in 2022 and 22% in 2021, the report states. This significant increase reflects the growing emphasis investors are placing on artificial intelligence as a transformative technology poised to redefine multiple industries.

The 42% figure represents actual data collected from venture capital investments made in 2023, illustrating a clear trend rather than a projection. HSBC Innovation Banking classified a company as an "AI company" if artificial intelligence technologies are a core component of its products, services, or operations. This includes firms developing AI algorithms, machine learning platforms, natural language processing applications, computer vision technologies, and other related innovations.

The report also highlights that as of 2023, 20 AI companies have each raised $2 billion or more in venture funding. "Venture capital has always gravitated toward transformative industries, but the level of consolidation we're seeing within one category is unprecedented," said Dave Sabow, Head of HSBC U.S. Innovation Banking. "The radical change this investment will fuel places us in the dawn of 'The Agentic Age,' an era where autonomous artificial intelligence capabilities fundamentally redefine how we communicate, work, and interface with digital and physical worlds."

This significant concentration of venture capital in AI could impact investment trends and growth opportunities in other technology sectors. As more funding flows into AI, other sectors may experience increased competition for investment dollars, potentially slowing growth in areas like biotechnology, fintech, and traditional software. However, it could also spur innovation across industries as companies integrate AI technologies into their products and services to stay competitive.

Investment giant BlackRock noted that it expects the coming years to be significant for infrastructure and cybersecurity investments, with the AI boom playing a major role. "It's still very early in the AI adoption cycle," said Jay Jacobs, BlackRock's U.S. head of thematic and active ETFs. Jacobs added that AI firms need to build out their data centers and that protecting that data will likely be a wise investment.

The HSBC report also pointed out that R&D spending from the so-called "Magnificent 7" companies—Tesla, Nvidia, Microsoft, Meta, Apple, Amazon, and Alphabet—totaled more than all the dollars invested in U.S. startups in 2023. This underscores the substantial resources major tech companies are dedicating to AI and related technologies.

HSBC Innovation Banking expects the U.S. tech sector to see new waves of growth and tailwinds for returns resulting from expected changes in the acquisition market, deregulation, and fiscal policies that stimulate economic activity. The focus on AI could lead to increased mergers and acquisitions as larger firms acquire smaller AI startups to enhance their capabilities.

In summary, the unprecedented allocation of venture capital to AI companies is reshaping the technology investment landscape. Understanding how AI is classified and its impact on other sectors is crucial for navigating the evolving competitive environment.

OpenAI's Rising Costs and For-Profit Shift: Understanding the $37.5 Billion Annual Spend by 2029

Unsplash

Building advanced AI models like ChatGPT requires enormous financial resources. That's the driving force behind OpenAI's plans to change how it's managed.

Early last year, OpenAI raised $10 billion. Just 18 months later, the company had spent most of that money. So it raised $6.6 billion more and arranged to borrow an additional $4 billion. But in another 18 months, OpenAI will need another cash infusion because the San Francisco start-up is spending more than $5.4 billion a year. By 2029, OpenAI expects to spend $37.5 billion annually.

Several factors are driving this projected increase in spending. Developing and training advanced AI models require vast computing power, which is expensive. OpenAI invests heavily in specialized hardware and cloud infrastructure to support its AI systems. The costs for energy consumption and data storage also add up significantly. Additionally, the company is focusing on research and development to stay ahead in a competitive market. Hiring top AI talent and expanding its team contribute substantially to its expenses. OpenAI is also looking to scale its operations globally, which involves additional costs in marketing, sales, and customer support.

To sustain such high expenditures and attract investor confidence as a for-profit entity, OpenAI plans to generate revenue through various channels. It offers AI services and licensing agreements to businesses and developers. By providing access to its models and technologies, OpenAI aims to create sustainable income streams. The company is developing new AI-powered products and solutions that can be monetized. Demonstrating strong revenue growth potential is key to attracting investors who are confident in the company's ability to provide returns.

Changing from a nonprofit research lab to a for-profit company could have implications for OpenAI's mission and stakeholder relationships. While the shift may bring in more investment capital, there are concerns about balancing profit motives with the original mission of ensuring that AI benefits all of humanity. Stakeholders who supported OpenAI's nonprofit beginnings might be wary of changes in transparency, priorities, and commitment to ethical AI development. Navigating these challenges will be crucial for OpenAI as it moves forward.

OpenAI's accelerating expenses are the main reason the corporate structure of the company, which began as a nonprofit research lab, could soon change. OpenAI must raise billions of additional dollars in the years to come, and its executives believe it will be more attractive to investors as a for-profit company.

Shelpuk

AI Technology Consulting

Shelpuk

AI Technology Consulting

The AI Compass

Need to talk to AI experts? Contact us!