EP15A3verificationofprecisionandestimationofbias[共3页]

1、CLSIEP15-A3:verificationofprecisionandestimationofbiasWearepleasedtohaveaguestessayexplainingthelatestinMethodVerification,specificallythenewestversionoftheCLSIguidelineEP15onMethodVerification.Forlabsseekingaquickchecktoinsuretheirmethodsaremeetingmanufacturerspecifications,EP15maybetherightchoiceW

2、hatsNewinCLSIEP15-A3:UserVerificationofPrecisionandEstimationofBias;ApprovedGuideline-ThirdEditionR.NeillCarey,Ph.D.July2015IntroductionCLSIEP15wasreleasedasanA3documentinSeptember2014.Thisisitsfourthiteration,andalthoughitretainsmuchofitsoriginalapproach,thereweresomesignificantchangesintheA3versio

3、n.Themostsignificantchangeisthecreationofarelativelysimpleexperimentthatgivesreliableestimatesofameasurementproceduresimprecisionanditsbias.TheessentialstoaccomplishthiswerepresentinEP15throughallofitspreviousversions,buttheyarerefinedandcombinedinEP15-A3tomakeasingleexperiment.Heresabriefdescriptio

4、noftheprotocol.SpecificationofAcceptablePerformanceBeforedoinganythingelse,theusershouldspecifytotalallowableerror,andderivefromittheallowablestandarddeviation(or%CV),andtheallowablebias.Theusershouldascertainthattheimprecisionofthecandidatemeasurementproceduremeetsthecriterionforallowableimprecisio

5、nbeforebeginningtheevaluation.Ifthemeasurementproceduresimprecisionreportedinpublications,suchasthemanufacturersstatedimprecision,doesnotmeetthecriterion,theprecisionverificationproceduredescribedinEP15-A3isnotappropriate.Beingalimitedexperiment,itislessrigorousthantheexperimentdescribedinCLSIEP5-A3

6、,whichwouldbemoreappropriate.VerificationofPrecisionEP15firstdescribesaprecisionverificationexperiment.Iftheuserisevaluatingaprocedureforwhichtherearemanufacturersprecisionclaims,orpublishedprecisionresults,thatweredevelopedusingCLSIEP5,theusercanverifythepublishedprecisioninanexperimentlastingasfew

7、asfivedays.Patientsamples,referencematerials,proficiencytestingsamples,orcontrolmaterialsmaybeusedasthetestsamples,providedthereissufficientsamplematerialfortestingeachsamplefivetimesperrunforfivetosevenruns.Precisionshouldbetestedwithtwoormoresamplematerialsatdifferentmedicaldecisionpointconcentrat

8、ions.Theexperimentproducesatleast25replicatescollectedoveratleast5daysforeachsamplematerial.Therepeatability(previouslytermedwithin-run)andthewithin-laboratory(previouslytermedtotal)standarddeviationsarecalculatedbyananalysisofvariancetechnique(ANOVA)thatproperlyaccountsforthewithin-runandbetween-ru

9、ncontributionstotheoverallimprecisionofthemeasurementprocedure.TheuserneedsaccesstosoftwaretodotheANOVAcalculations,buttheyareavailableinExcel,Minitab,Analyze-it,andothersoftwarepackagesthatdostatisticalcalculations.Therepeatabilityandwithin-laboratorystandarddeviationsarethencomparedtotheclaimedorp

10、ublishedstandarddeviations.Ifthecalculatedstandarddeviationsarelessthanthepublishedvalues,theuserhasverifiedtheclaim.Sometimesthecalculatedstandarddeviationsmayexceedthepublishedvalues,andyetthetruestandarddeviationsarelessthanthepublishedvalues.Forexample,ifthetruestandarddeviationswereactuallyexac

11、tlyequaltotheirclaimedcounterparts,thecalculatedstandarddeviationswouldexceedtheirpublishedcounterpartsfiftypercentofthetimeinverificationexperiments.Toallowforthispossibility,theusercalculatesaverificationlimitbasedonthepublishedstandarddeviationandthesizeoftheusersexperiment.Ifthecalculatedstandar

12、ddeviationislessthantheverificationlimit,itisnotstatisticallysignificantlylargerthanthepublishedstandarddeviation,andtheuserhasverifiedthepublishedprecision.Ifthecalculatedprecisionexceedstheverificationlimit,thecalculatedstandarddeviationisstatisticallysignificantlylargerthanthepublishedstandarddev

13、iation,andtheuserhasfailedtoverifythepublishedimprecision.Thedocumentincludestablestosimplifythecalculationoftheverificationlimit.EstimationofBiasBecausetheprecisionexperimenthassomanyreplicatemeasurements,collectedoverseveraldays,resultsfromtheprecisionexperimentmaybeusedtomakeareliableestimateofth

14、ebiasofthemeasurementprocedurerelativetotheassigned(target)valuesofthesamplematerialsusedintheexperiment.Theonlyrequirementisthattheassignedvaluemustbeavailable.Thechoiceofmaterialdependsonthepurposeoftheuserinestimatingthebias.Twoormoreappropriatematerialsshouldbetestedintheprecisionexperiment.Ifth

15、euserisinterestedinestimatingbiasrelativetothepeergroupforproficiencytesting,andwantstoestimatehowthemeasurementprocedurewillperformwellonproficiencytesting,proficiencytestingmaterialswithpeergroupvaluesforthemeasurementprocedurebeingevaluatedareappropriate.Forbiasrelativetothequalitycontrolpeergrou

16、p,qualitycontrolmaterialswithpeergroupvaluesforthemeasurementprocedureareappropriate.Assayedqualitycontrolmaterialsarenotappropriate,unlesspeergroupinformationisavailable.Typically,thereisnowaytoestimatetheuncertaintyoftheassayedvalues,whichisneededtodetermineifthecalculatedbiasisstatisticallysignif

17、icant.Internationallyrecognizedhighorderreferencematerials,suchasamaterialfromtheU.S.NationalInstituteofStandardsandTechnology,orfromtheJointCommitteeforTraceabilityinLaboratoryMedicine,orfromsimilarorganizationsmaybeappropriateiftheuserwishestoestimatethebiasrelativetotheassignedconcentrationsofsuc

18、hmaterials.Useofthesematerialsisimportantinestablishingthetraceabilityofmeasurementprocedures.Patientsamplesorcontrolmaterialswhichhavebeenrepeatedlyassayedwithameasurementprocedurefelttobesubstantiallyequivalenttothemeasurementprocedurebeingevaluatedmaybeappropriateiftheuserisinterestedinestimating

19、biasrelativetothatmeasurementprocedure.Thiscouldbeuseful,forexample,iftheintentoftheexperimentwastoestimatethebiasofonelaboratoryinasystemrelativetoanother,ortothemeanofthelaboratoriesinasystem.Ifthesamplematerialsareappropriate,andtargetconcentrationsareavailable,theusercanestimatethebiasbetweenthemeanconcentrationcalculatedintheprecisionexperimentrelativetothetargetconcentrationofeachofthematerials.Todeterminewhetherthereisanystatisticallysignificantbiasbetweenthemean