WorldDigitalTechnologyAcademy(WDTA) LargeLanguageModelSecurity TestingMethod WorldDigitalTechnologyAcademyStandard WDTAAI-STR-02 Edition:2024-04©WDTA2024–Allrightsreserved. TheWorldDigitalTechnologyStandardWDTAAI-STR-02isdesignatedasaWDTA norm.ThisdocumentisthepropertyoftheWorldDigitalTechnologyAcademy(WDTA)andis protectedbyinternationalcopyrightlaws.Anyuseofthisdocument,includingreproduction, modification,distribution,orrepublication,withoutthepriorwrittenpermissionofWDTA,is prohibited.WDTAisnotliableforanyerrorsoromissionsinthisdocument. DiscovermoreWDTAstandardandrelatedpublicationsathttps://wdtacademy.org/. VersionHistory* StandardID Version Date Changes WDTAAI-STR-02 1.0 2024-04 InitialReleaseForeword The"LargeLanguageModelSecurityTestingMethod,"developedandissuedbytheWorldDigital TechnologyAcademy(WDTA),representsacrucialadvancementinourongoingcommitmentto ensuringtheresponsibleandsecureuseofartificialintelligencetechnologies.AsAIsystems, particularlylargelanguagemodels,continuetobecomeincreasinglyintegraltovariousaspectsof society,theneedforacomprehensivestandardtoaddresstheirsecuritychallengesbecomes paramount.Thisstandard,anintegralpartofWDTA'sAISTR(Safety,Trust,Responsibility)program, isspecificallydesignedtotacklethecomplexitiesinherentinlargelanguagemodelsandprovide rigorousevaluationmetricsandprocedurestotesttheirresilienceagainstadversarialattacks. Thisstandarddocumentprovidesaframeworkforevaluatingtheresilienceoflargelanguagemodels (LLMs)againstadversarialattacks.TheframeworkappliestothetestingandvalidationofLLMs acrossvariousattackclassifications,includingL1Random,L2Blind-Box,L3Black-Box,andL4 White-Box.KeymetricsusedtoassesstheeffectivenessoftheseattacksincludetheAttackSuccess Rate(R)andDeclineRate(D).Thedocumentoutlinesadiverserangeofattackmethodologies,such asinstructionhijackingandpromptmasking,tocomprehensivelytesttheLLMs'resistanceto differenttypesofadversarialtechniques.Thetestingproceduredetailedinthisstandarddocument aimstoestablishastructuredapproachforevaluatingtherobustnessofLLMsagainstadversarial attacks,enablingdevelopersandorganizationstoidentifyandmitigatepotentialvulnerabilities,and ultimatelyimprovethesecurityandreliabilityofAIsystemsbuiltusingLLMs. Byestablishingthe"LargeLanguageModelSecurityTestingMethod,"WDTAseekstoleadtheway increatingadigitalecosystemwhereAIsystemsarenotonlyadvancedbutalsosecureandethically aligned.Itsymbolizesourdedicationtoafuturewheredigitaltechnologiesaredevelopedwithakeen senseoftheirsocietalimplicationsandareleveragedforthegreaterbenefitofall. ExecutiveChairmanofWDTA