lawoflargenumbers(LLN)
Thelawoflargenumbers(LLN)isatheoreminprobabilitythatdescribesthelong-termstabilityofthemeanofarandomvariable.Givenarandomvariablewithafiniteexpectedvalue,ifitsvaluesarerepeatedlysampled,asthenumberoftheseobservationsincreases,theirmeanwilltendtoapproachandstayclosetotheexpectedvalue.
TheLLNcaneasilybeillustratedusingtherollsofadie.Thatis,outcomesofamultinomialdistributioninwhichthenumbers1,2,3,4,5,and6areequallylikelytobechosen.Thepopulationmean(or"expectedvalue")oftheoutcomesis:
(1+2+3+4+5+6)/6=3.5.
Thegraphtotherightplotstheresultsofanexperimentofrollsofadie.Inthisexperimentweseethattheaverageofdierollsdeviateswildlyatfirst.AspredictedbyLLNtheaveragestabilizesaroundtheexpectedvalueof3.5asthenumberofobservationsbecomeslarge.
Anotherexampleistheflipofacoin.Givenrepeatedflipsofafaircoin,thefrequencyofheads(ortails)willincreasinglyapproach50%overalargenumberoftrials.Howeveritispossiblethattheabsolutedifferenceinthenumberofheadsandtailswilltendtogetlargerandlargerasthenumberofflipsincreases.[1]Forexample,wemaysee520headsafter1000flipsand5096headsafter10000flips.Whiletheaveragehasmovedfrom0.52to0.5096,closertotheexpected50%,thetotaldifferencefromtheexpectedmeanhasincreasedfrom20to96.
TheLLNisimportantbecauseit"guarantees"stablelong-termresultsforrandomevents.Forexample,whileacasinomaylosemoneyinasinglespinoftheAmericanroulettewheel,itwillalmostcertainlygainverycloseto5.3%ofallgambledmoneyoverthousandsofspins.Anywinningstreakbyaplayerwilleventuallybeovercomebytheparametersofthegame.ItisimportanttorememberthattheLLNonlyapplies(asthenameindicates)whenalargenumberofobservationsareconsidered.Thereisnoprinciplethatasmallnumberofobservationswillconvergetotheexpectedvalueorthatastreakofonevaluewillimmediatelybe"balanced"bytheothers.SeetheGambler'sfallacy.
History
TheLLNwasfirstdescribedbyJacobBernoulli.[2]Ittookhimover20yearstodevelopasufficientlyrigorousmathematicalproofwhichwaspublishedinhisArsConjectandi(TheArtofConjecturing)in1713.Henamedthishis"GoldenTheorem"butitbecamegenerallyknownas"Bernoulli'sTheorem"(nottobeconfusedwiththeLawinPhysicswiththesamename.)In1835,S.D.Poissonfurtherdescribeditunderthename"Laloidesgrandsnombres"("Thelawoflargenumbers").[3]thereafter,itwasknownunderbothnames,butthe"Lawoflargenumbers"ismostfrequentlyused.
AfterBernoulliandPoissonpublishedtheirefforts,othermathematiciansalsocontributedtorefinementofthelaw,includingChebyshev,Markov,Borel,CantelliandKolmogorov.ThesefurtherstudieshavegivenrisetotwoprominentformsoftheLLN.Oneiscalledthe"weak"lawandtheotherthe"strong"law.Theseformsdonotdescribedifferentlawsbutinsteadrefertodifferentwaysofdescribingthemodeofconvergenceofthecumulativesamplemeanstotheexpectedvalue,andthestrongformimpliestheweak.
Forms
Bothversionsofthelawstatethatthesampleaverage
convergestotheexpectedvalue
whereX1,X2,...isaninfinitesequenceofi.i.d.randomvariableswithfiniteexpectedvalueE(X1)=E(X2)=...=µ<∞.
AnassumptionoffinitevarianceVar(X1)=Var(X2)=...=σ2<∞isnotnecessary.Largeorinfinitevariancewillmaketheconvergenceslower,buttheLLNholdsanyway.Thisassumptionisoftenusedbecauseitmakestheproofseasierandshorter.
Thedifferencebetweenthestrongandtheweakversionisconcernedwiththemodeofconvergencebeingasserted.
Theweaklaw
Theweaklawoflargenumbersstatesthatthesampleaverageconvergesinprobabilitytowardstheexpectedvalue
Thatistosaythatforanypositivenumberε,
(Proof)
Interpretingthisresult,theweaklawessentiallystatesthatforanynonzeromarginspecified,nomatterhowsmall,withasufficientlylargesampletherewillbeaveryhighprobabilitythattheaverageoftheobservationswillbeclosetotheexpectedvalue,thatis,withinthemargin.
Convergenceinprobabilityisalsocalledweakconvergenceofrandomvariables.Thisversioniscalledtheweaklawbecauserandomvariablesmayconvergeweakly(inprobability)asabovewithoutconvergingstrongly(almostsurely)asbelow.
AconsequenceoftheweakLLNistheasymptoticequipartitionproperty.
Thestronglaw
Thestronglawoflargenumbersstatesthatthesampleaverageconvergesalmostsurelytotheexpectedvalue
Thatis,
Theproofismorecomplexthanthatoftheweaklaw.Thislawjustifiestheintuitiveinterpretationoftheexpectedvalueofarandomvariableasthe"long-termaveragewhensamplingrepeatedly."
Almostsureconvergenceisalsocalledstrongconvergenceofrandomvariables.Thisversioniscalledthestronglawbecauserandomvariableswhichconvergestrongly(almostsurely)areguaranteedtoconvergeweakly(inprobability).Thestronglawimpliestheweaklaw.
Thestronglawoflargenumberscanitselfbeseenasaspecialcaseoftheergodictheorem.
Differencesbetweentheweaklawandthestronglaw
TheWeakLawstatesthat,foraspecifiedlargen,(X1+...+Xn)/nislikelytobenearμ.Thus,itleavesopenthepossibilitythat(X1+...+Xn)/n−μhappensaninfinitenumberoftimes,althoughithappensatinfrequentintervals.
Thestronglawshowsthatthiscannotoccur.Inparticular,itimpliesthatwithprobability1,foranypositivevalueε,(X1+...+Xn)/n−μwillbegreaterthanεonlyafinitenumberoftimes.[4]
Activitiesanddemonstrations
Therearevarietiesofwaystoillustratethetheoryandapplicationsofthelawsoflargenumbersusinginteractiveaids.TheSOCRresourceprovidesahands-onlearningactivitypairedwithaJavaapplet(selecttheCoinTossLLNExperiment)thatdemonstratethepowerandusabilityofthelawoflargenumbers.
[edit]Seealso
Centrallimittheorem
Gambler'sfallacy
Lawofaverages
References
^Tijms,Henk(2007).UnderstandingProbability:ChanceRulesinEverydayLife,CambridgeUniversityPress.pp.17.ISBN978-0-521-70172-3,http://books.google.com/books?id=Ua-_5Ga4QF8C&printsec=frontcover#PRA2-PA17,M1.
^JakobBernoulli,ArsConjectandi:Usum&ApplicationemPraecedentisDoctrinaeinCivilibus,Moralibus&Oeconomicis,1713,Chapter4,(TranslatedintoEnglishbyOscarSheynin)
^hacking,Ian.(1983)"19th-centuryCracksintheConceptofDeterminism"
^SheldonRoss,AFirstCourseinProbability,Fifthedition,PrenticeHallpress
Grimmett,G.R.andStirzaker,D.R.(1992).ProbabilityandRandomProcesses,2ndEdition,ClarendonPress,Oxford.ISBN0-19-853665-8.
RichardDurrett(1995).Probability:TheoryandExamples,2ndEdition,DuxburyPress.
MartinJacobsen(1992).VideregåendeSandsynlighedsregning(AdvancedProbabilityTheory)3rdEdition'',HCØ-tryk,Copenhagen.ISBN87-91180-71-6.
