How we help with the AI Diagnostics Fund

Aival can help Imaging Networks in the AI Diagnostics Fund (AIDF) run their 'mini-competitions' to systematically choose between vendors. Additionally, we can help with in-service evaluations for evidence generation and monitoring of a deployed product.

  • Our comparison software allows clinical users to easily assess and compare product outputs on sample datasets. We display face-to-face results of different products, allowing users to accept and reject outputs and vote for their preference. These votes can be stored and accumulated for many users.
  • Our evaluation software can help evidence the Technical Merit and Cultural Fit dimensions of the assessment, enabling systematic and objective comparison across products.
  • Once a product has been chosen and deployed, our monitoring software can help ensure sustained performance over time. Aival can help with 'in-service evaluations' to generate evidence of performance across ranges of populations and image variations.

Guidelines for procurement can be found in Crown Commercial Service's Artificial Intelligence Dynamic Purchasing System Customer Guidance Document (RM6200). In particular, our software can provide evidence to assess the Data Ethics Requirements for RM6200 Artificial Intelligence Suppliers (Annex 5):

Explainability

  • Provide methods of explainability of black-box classification products, understandable to a non-expert audience
  • Provide third-party audit of the AI solution

Ethical considerations relating to data limitations, fairness and bias

  • Help identify data limitations
  • Evidence that the AI system fits the requirements of the Data Ethics Framework
  • Evidence an approach to eliminate bias, ethical issues, or other safety risks
  • Provide software to allow the solution to be checked to detect bias
  • Evidence audit and assurance with respect to intrinsic attributes such as considerations of fairness and mitigation of bias

Concept drift

  • Provide software to help ensure solution does not drift from its intended outcome

Documentation

  • Evidence of assurance processes

Lifecycle management

  • Third party solutions to assess how accuracy and integrity is sustained over time
  • Testing services for monitoring the efficacy of the model once deployed

These form part of the procurement criteria of Annex 7 of the AI DPS Customer Guide above, summarised below:

Quality (20-60%):

  • Technical Merit
  • Added Value/Innovation
  • Approach to delivery of services

Cultural Fit (10-20%):

  • Approach to deliver social value
  • Approach to delivering ethical dimensions
  • How the Supplier works with other people
  • Software licence / subscription
  • How the Supplier shares knowledge and experience

Price (20-60%):

  • Time and materials
  • Milestone payment
  • Fixed cost
  • Combination of pricing methods

To learn more about how we can help with systematic and objective assessment of imaging AI products, please get in touch through our contact form or email info [@] aival.io.

Download our brochure with more details below:

Download Brochure

Learn more

Solutions for AI evaluation

>

Solutions for AI monitoring

>

Work with us

>

Get in touch

>
We value your privacy!
By clicking "Accept cookies", you agree to store cookies on your device and disclose information in accordance with our Cookie Policy.
Reject Cookies
Accept Cookies