Armilla seeks to offer AI warranties to companies

News
February 15, 2024

Armilla aims to provide companies with AI warranties

Armilla AI offers warranties for AI models, providing companies with the confidence they need to embrace AI technology. With the ability to verify the quality, fairness, and security of AI models through comprehensive assessments, Armilla AI is revolutionizing the way organizations approach AI adoption. In this blog, we’ll explore how Armilla AI’s unique approach sets them apart, the benefits of their warranties, and the growing success of their client base.

GenAI, especially when sourced from third-party providers, presents numerous potential challenges. From generating inaccurate information to perpetuating biases and toxicity, navigating the landscape of AI is rife with pitfalls. Additionally, the risk of infringing upon copyright regulations adds another layer of complexity. According to a recent study by MIT Sloan Management Review and Boston Consulting Group, over 55% of AI-related failures within organizations can be attributed to third-party AI tools.

Given these concerns, it’s unsurprising that many companies approach the adoption of this technology with caution. However, envision a scenario where there exists a solution to mitigate these risks.

This was the entrepreneurial spark that ignited in Karthik Ramakrishnan, an experienced entrepreneur and electrical engineer during his tenure at Deloitte. Having co-founded two ‘AI-first’ companies, Gallop Labs and Blu Trumpet, Ramakrishnan recognized that the lack of trust and inability to quantify risks were major impediments to widespread AI adoption.

“Presently, numerous enterprises seek to harness AI to enhance efficiency and stay competitive in the market,” Ramakrishnan explained in an email interview with TechCrunch. “However, in doing so, many resort to third-party vendors, deploying AI models without fully comprehending the quality of the products… The rapid advancement of AI necessitates a dynamic approach to addressing evolving risks and potential harms.”

In response to this challenge, Ramakrishnan partnered with Dan Adamson, an esteemed expert in search algorithms and seasoned startup founder, to establish Armilla AI. Their mission: to provide warranties on AI models to corporate clients.

But how is this possible given the often opaque nature of AI models and the barriers imposed by licenses, subscriptions, and APIs? The answer lies in benchmarking, Ramakrishnan’s insight revealed, coupled with a meticulous strategy for customer acquisition.

Armilla adopts a comprehensive approach to evaluating models, regardless of whether they are open source or proprietary. Drawing insights from the global AI regulatory framework, the company conducts meticulous assessments to ensure the quality of each model. These assessments encompass a wide range of criteria, including detection of hallucinations, evaluation of racial and gender biases, assessment of fairness, examination of overall robustness and security, and consideration of various theoretical applications and use cases. Armilla leverages its proprietary assessment technology to conduct these evaluations, ensuring a thorough and rigorous analysis process.

If the model meets the required standards, Armilla provides assurance by offering a warranty. This warranty covers any fees paid by the buyer to utilize the model.

“Our primary offering to enterprises is to instill confidence in the technology they’re acquiring from third-party AI vendors,” explained Ramakrishnan. “Enterprises can engage us to conduct assessments on the vendors they’re considering. Similar to the penetration testing conducted for new technology, we perform rigorous evaluations for AI, ensuring its reliability and security.”

In a conversation, the topic arose regarding Armilla’s policy on testing certain models for ethical considerations, such as a facial recognition algorithm from a vendor associated with questionable practices. Ramakrishnan, in response, stated:

“It would not only contradict our ethical principles but also undermine our business model, which is built on trust, to provide assessments and reports that offer false assurance in AI models that present concerns for clients and society. From a legal perspective, we refrain from engaging with clients whose models are prohibited by the EU or have faced bans, as is the case with certain facial recognition and biometric categorization systems. However, we are open to evaluating applications deemed ‘higher risk’ according to the EU AI Act.”

The landscape of warranties and policy coverage for AI is not entirely novel—a revelation that caught this writer off guard, to be honest. Last year, Munich Re introduced an insurance product called aiSure, aimed at safeguarding against losses stemming from potentially unreliable AI models by subjecting them to benchmarks akin to Armilla’s. Beyond warranties, an increasing number of vendors, including OpenAI, Microsoft, and AWS, offer safeguards concerning copyright infringements that may arise from the deployment of their AI tools.

However, Ramakrishnan asserts that Armilla’s approach stands apart.

“Our evaluation encompasses a broad spectrum of areas, including key performance indicators (KPIs), processes, performance metrics, data quality, and both qualitative and quantitative criteria, all at a fraction of the cost and time,” he elaborated. “We assess AI models in accordance with the stipulations outlined in legislation such as the EU AI Act or the AI hiring bias law in NYC—NYC Local Law 144—and other state regulations, such as Colorado’s proposed AI quantitative testing regulation or New York’s insurance circular on the use of AI in underwriting or pricing. Moreover, we are equipped to conduct evaluations mandated by other emerging regulations as they unfold, such as Canada’s AI and Data Act.”

Armilla, which commenced its coverage in late 2023 with backing from carriers Swiss Re, Greenlight Re, and Chaucer, reports having secured around ten customers to date. Among them is a healthcare company employing GenAI to streamline the processing of medical records. According to Ramakrishnan, Armilla’s client base has been expanding at a rate of 2x month over month since the fourth quarter of 2023.

“We cater to two primary audiences: enterprises and third-party AI vendors,” Ramakrishnan explained. “Enterprises utilize our warranty to establish protection for the third-party AI vendors they engage with. Third-party vendors, on the other hand, leverage our warranty as a seal of approval for the trustworthiness of their product, thereby expediting their sales cycles.”

Warranties for AI seem like a logical step forward. However, I can’t help but ponder whether Armilla will manage to stay abreast of the swiftly evolving AI policy landscape (e.g., New York City’s hiring algorithm bias law, the EU AI Act, etc.), which could potentially expose it to significant liabilities if its assessments—and contracts—aren’t airtight.

Ramakrishnan swiftly dismissed this apprehension.

“The regulatory landscape is evolving rapidly across various jurisdictions,” he remarked, “and it’s imperative to grasp the intricacies of legislation worldwide. There’s no universal approach that can serve as a global standard, so we must piece together the puzzle. While challenging, this process fortifies our position, creating a competitive advantage for us.”

Armilla, headquartered in Toronto with a team of 13 employees, recently secured $4.5 million in a seed funding round. The investment was led by Mistral (not to be confused with the similarly named AI startup) and included participation from Greycroft, Differential Venture Capital, Mozilla Ventures, Betaworks Ventures, MS&AD Ventures, 630 Ventures, Morgan Creek Digital, Y Combinator, Greenlight Re, and Chaucer. This funding round brings Armilla’s total raised capital to $7 million. Ramakrishnan outlined that these funds will be allocated towards enhancing Armilla’s existing warranty offerings and unveiling new products.

“Insurance is poised to play a pivotal role in mitigating AI-related risks, and Armilla is leading the charge in developing insurance products that enable companies to deploy AI solutions securely,” commented Ramakrishnan.

Conclusion

Armilla AI is revolutionizing the adoption of AI technology by offering warranties on AI models to corporate customers. By conducting thorough assessments and benchmarking, Armilla ensures the quality, fairness, and security of AI models, providing confidence to enterprises in the technology they procure. With the backing of carriers like Swiss Re, Greenlight Re, and Chaucer, Armilla is gaining traction in the market and attracting a growing client base.

By addressing the risks and concerns associated with third-party AI tools, Armilla is paving the way for widespread AI implementation and driving efficiency in organizations. With Armilla’s unique approach to warranties and comprehensive assessments, companies can now embrace AI with peace of mind, knowing that they are protected and supported in their AI endeavors.

Have you ever considered the idea of warranties for AI models to mitigate the uncertainties associated with their adoption? How important is it for your company to have confidence in the technology procured from third-party AI vendors? What are your thoughts on the concept of warranties and policy coverage for AI models? Do you believe they could provide a sense of security and reassurance? Share your insights below.