ILC Structure

LBNL requires data submitters to participate in an inter-laboratory comparison (ILC, also know as round-robin and inter-laboratory study (ILS)) as a way to verify that they understand the procedures and their equipment works.There are multiple ASTM standards describing how to run such an activity, but they are designed to validate the method rather than the labs, so while the procedure described here is inspired by them it is not perfectly described by them.

All submitters shall have successfully participated in the most recent inter-laboratory comparison

The ILC might be defined to test several different procedures (e.g. solar optical and thermal IR measurements) and should be set up so that it is possible to qualify to perform a subset of all measurements that are tested during the activity.

The outline of the process is as follows

1.    Samples selection committee: Form a committee that decides what samples are required to verify the procedures under scrutiny.

2.    Identify participants: Compile a list of participants to get a lower limit of the number of participants.

3.    Acquire samples: Acquire samples that meet the specification of the sample selection committee. Each participant needs their own set in a parallel ILC, extra sets are needed to accommodate labs that want to qualify before the next ILC.

4.    Organizer writes instructions: Prepare instructions on what to measure, and how to report the results.

5.    Organizer characterizes samples for consistency: Perform limited characterization of all samples to quantify consistency.

6.    Participants measure: Samples are received, characterized, and then the data is formatted and sent back to the organizer.

7.    Organizer writes initial report: When a critical mass of participants has responded, an initial report is prepared by the organizer to determine what constitutes accurate results.

8.    Measurement iteration: Participants that did not qualify get advice on how to improve their measurements.

9.    Final report: Final report, including lessons learned and suggested improvements on the procedures, is published.

10.  New participants: Allow new lab to qualify so that they do not have to wait until the next ILC. The organizer sends samples, analyzes results, and makes sure the new participant qualifies. Before the organizer is running out of sample sets, it is advised to require the new submitters to send the box back after they have qualified, effectively turning the final boxes into a serial ILC.

The main benefit of running a parallel ILC is that it takes less time for a large number of participants, but it also allow the participants to have samples at hand to verify maintained accuracy if they update equipment or modify their measurement procedures. The drawback is the sample variation, but handling of optical materials might induce changes to the surfaces, which would make it difficult to analyze the final result.