AI has reshaped how many industries do business, and there’s currently a mad dash to leverage tools for greater efficiency. This trend has introduced a new challenge–businesses have to evaluate often complex AI software and technologies before they sign a contract with a vendor. Without a full grasp of everything under the hood, businesses can open themselves up to risk, data vulnerabilities and more.
The good news is that asking the right questions can help you gain a better understanding of AI vendors and uncover any potential risk before signing a contract. Let’s take a closer look at ten essential questions your business should ask AI vendors.
1. What Exactly is the Underlying AI Tool?
It’s essential to have a functional understanding of an AI tool and how it is going to be used in your business. Too often these types of solutions are clouded with buzzwords that leave the personnel evaluating the technology with more questions than answers. You need to have a clear idea of what’s being purchased, what the vendor’s technology truly is and how it’s recommended to be integrated into your current systems.
A good vendor provides a transparent explanation of the technology being used, with clear documentation on how it’s built so that you understand exactly how it can be used in your business and the potential risks associated with that use.
Red Flag to Watch Out For:
- No User Guidance: Be wary of vendors who don’t have guidance or risk mitigation suggestions for end users.
2. How is the AI Governed and Assessed for Risk?
Any AI tool or technology is only as safe as the accountability framework supporting its operation. Look for vendors that have an established set of guidelines that are in line with AI best practices and security standards for your industry. Vendors that are aligned with standardized frameworks like the NIST AI Risk Management Framework (RMF) or have achieved an ISO/IEC 42001 certification are a good place to start.
Red Flag to Watch Out For:
- Regulatory Avoidance: For example, if an EU-based vendor dismisses compliance questions with “we aren’t located in the US so certain frameworks don’t matter”.
3. What Data is Being Used, and How is it Handled?
Data privacy is crucial when you’re deciding to partner with a new AI vendor. It’s important to have absolute transparency into what data is going into their system, how that data is processed, if your data is being used for training purposes and more. A reliable vendor will be able to offer a full breakdown of how your data will be segregated, used and isolated to maintain privacy.
Red Flag to Watch Out For:
- Unrestricted Ingestion: This is when a vendor takes data from your business and uses it to optimize the performance of everyone using their technology or even other partners without proper anonymization.
4. Does the Data Stay in Your Geographic Region?
Where data resides can dictate whether your business stays compliant with local legislation like GDPR and other local data residency laws or requirements. If an AI vendor routes data on a global level, it can accidentally push protected information through servers across international borders. AI vendors should be able to provide a clear data localization framework to ensure your data stays in the geographic region it needs to.
Red Flag to Watch Out For:
- Lost Without a Map: The vendor can’t offer a definitive map of data centers it uses in your geographic region and/or relies on international dynamic routing.
5. Can They Describe the AI Model Architecture?
By understanding the vendor’s technical foundation, you’re better informed of how the vendor addresses the unique modern security risks. Get a clear map of the hosting structure and know for example if the vendor is operating as an additional layer on top of a base large language model (LLM) like ChatGPT. AI vendors should also provide documentation about the testing and built-in security controls such as vulnerability identification or red teaming.
Red Flag to Watch Out For:
- Supply Chain Stability: The vendor’s core technology is a wrapper around a larger technology and doesn’t offer protection against vulnerabilities in that core system.
6. How is the AI Maintained, Explained and Monitored?
AI tools aren’t “set-it-and-forget-it” systems. Over time, AI technology can degrade and experience shifts in reasoning to create unreliable outputs. That’s why it’s important to know not only what features or variables drive the decisioning but also how an AI tool from a vendor is maintained over time. You’ll want to look for vendors that have transparent tracking into the work that goes into keeping the technology running as expected.
Red Flag to Watch Out For:
- Untraceable Logic: The AI vendor can’t provide a reason why the system arrived at a particular decision or output.
7. How Do You Manage Output Integrity and IP?
In modern corporate law, it’s key to have a firm handle on the intellectual property risk, validation guardrails and other hallucination controls in place to manage output integrity. For AI vendors, this means having clarity about who owns what the technology creates and how incorrect outputs are caught. The AI vendor should have guardrails in place to prevent toxic, biased or otherwise dangerous outputs that infringe on someone else’s IP.
Red Flag to Watch Out For:
- Ambiguous Ownership: The contract lacks clear language on intellectual property rights for AI-generated content, creating potential liability for the user.
8. How Does the AI System Keep Up with Compliance?
The regulatory landscape for AI tools changes at a rapid pace. With new state-level laws, an AI vendor needs to have a process in place to keep up and ensure their technology follows compliance guidelines. You should look for vendors that have the ability to create new compliance guardrails in their system.
Red Flag to Watch Out For:
- Stagnant Guardrails: The vendor relies on a static compliance framework and can’t provide a clear, ongoing process for adapting to new state or local laws.
9. How Does the Tool Directly Benefit Our Business Objectives?
Beyond the legal and security checks on an AI tool, it also has to provide a tangible impact for your business. Ask the vendor for case studies or data that shows the value the AI tool offers for your industry and outlines what makes it unique from other options out there.
Red Flag to Watch Out For:
- Vague Statements: The AI vendor makes generalized statements without having data or proof points to back it up.
10. Can I Get a Technical Software Bill of Materials?
It’s common for AI tools to contain a complex system of interdependent software libraries. Ask the vendor if they provide a structured Software Bill of Materials (SBOM) that outlines:
- Base foundation models and specific versions being used
- Embedded open-source software libraries
- Every integrated third-party API pathway
- Specialized AI-specific code dependencies and frameworks
Red Flag to Watch Out For:
- Hidden Model Versions: The vendor doesn’t list the exact foundational models or version numbers that are working behind the scenes.
Have Questions About Digital Debt Collection? Retain Has Answers
Retain white-label debt collection software optimizes engagement with every account by automating outreach with the right time, message and channel. With patented machine learning, Retain gives your recovery strategy compliant digital communications that boosts repayments while lowering operational costs. Contact us today to learn more.
