8.2 How does Taiwan AI Labs Federated Validation System work?

Taiwan AI Labs provides a Federated Validation (abbreviation FV) system, which could implement AI Model validation in an easiest way.

By default, all institutes will have one dedicated Taiwan AI Labs Federated Validation machine - which is called as FV Edge Machine, licensed and issued by Taiwan AI Labs. All FV project participants will have the FV Edge Machines, so they will join the same FV project, to complete a FV plan. 

The FV Aggregator could be running on one of the FV Edge Machine, or using the service provided by Taiwan AI Labs. FV Aggregator needs to have a public IP address, so all FV Edge Machines could communicate with it.

A FV Edge Machine is installed inside institute's intranet (no public IP required), and with the firewall open for accessing the Aggregator destination (FQDN) through 443 HTTPS port.

The AI model is imported to the FV Aggregator (see Chapter 5 for technical details),  and ground truth datasets are uploaded locally on each FV Edge Machine. 

While the validation environment is set, PI could kick off the validation progress. The AI model will be downloaded directly
from Aggregator to all FV Edge Machines automatically.

After the model is downloaded to the FV edge machine, the FV system will validate the AI model performance using the local Ground Truth Datasets automatically, and generate the Validation Result accordingly.

The FV system will then return the Validation Result from FV Edge Machine back to the FV Aggregator, and remove the temporarily downloaded AI model from the FV Edge Machine.

All FV Edge Machines do the same progress. All Validation Results are sent to FV Aggregator automatically after validation is complete.

FV Aggregator will summarize all the received Validation Results, and generate the final Validation Report accordingly.

The Federated Validation of this dedicated AI model performance is then complete.