In particular they are useful for compactly representing and discussing the linear programming problem Maximize subject to i j This appendix reviews several properties of vectors and matrices that are especially relevant to this problem We shoul ID: 23411
Download Pdf The PPT/PDF document "Vectors and Matrices Appendix Vectors an..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
488VectorsandMatricesA.2Equalityandorderingofvectorsaredenedbycomparingthevectorsindividualcomponents.Formally,letyD.y1;y2;:::;yk/andzD.z1;z2;:::;zk/betwok-dimensionalvectors.Wewrite:yDzwhenyjDzj.jD1;2;:::;k/;yzorzywhenyjzj.jD1;2;:::;k/;yzorzywhenyj]TJ/;ག ;.9;Y T; 12;.294; 0 T; [00;zj.jD1;2;:::;k/;andsay,respectively,thatyequalsz,yisgreaterthanorequaltozandthatyisgreaterthanz.Inthelasttwocases,wealsosaythatzislessthanorequaltoyandlessthany.Itshouldbeemphasizedthatnotallvectorsareordered.Forexample,ifyD.3;1;2/andxD.1;1;1/,thenthersttwocomponentsofyaregreaterthanorequaltothersttwocomponentsofxbutthethirdcomponentofyislessthanthecorrespondingcomponentofx.Analnote:0isusedtodenotethenullvector(0,0, ,0),wherethedimensionofthevectorisunderstoodfromcontext.Thus,ifxisak-dimensionalvector,x0meansthateachcomponentxjofthevectorxisnonnegative.Wealsodenescalarmultiplicationandadditionintermsofthecomponentsofthevectors.Denition.ScalarmultiplicationofavectoryD.y1;y2;:::;yk/andascalarisdenedtobeanewvectorzD.z1;z2;:::;zk/,writtenzDyorzDy;whosecomponentsaregivenbyzjDyj.Denition.Vectoradditionoftwok-dimensionalvectorsxD.x1;x2;:::;xk/andyD.y1;y2;:::;yk/isdenedasanewvectorzD.z1;z2;:::;zk/,denotedzDxCy,withcomponentsgivenbyzjDxjCyj.Asanexampleofscalarmultiplication,consider4.3;0;1;8/D.12;0;4;32/;andforvectoraddition,.3;4;1;3/C.1;3;2;5/D.4;7;1;2/:Usingbothoperations,wecanmakethefollowingtypeofcalculation:.1;0/x1C.0;1/x2C.3;8/x3D.x1;0/C.0;x2/C.3x3;8x3/D.x13x3;x28x3/:Itisimportanttonotethatyandzmusthavethesamedimensionsforvectoradditionandvectorcomparisons.Thus.6;2;1/C.4;0/isnotdened,and.4;0;1/D.4;0/makesnosenseatall.A.2MATRICESWecannowextendtheseideastoanyrectangulararrayofnumbers,whichwecallamatrix.Denition.AmatrixisdenedtobearectangulararrayofnumbersAD26664a11a12a1na21a22a2n::::::am1am2amn37775;whosedimensionismbyn.AiscalledsquareifmDn.ThenumbersaijarereferredtoastheelementsofA.Thetableauofalinearprogrammingproblemisanexampleofamatrix.Wedeneequalityoftwomatricesintermsoftheirelementsjustasinthecaseofvectors. 490VectorsandMatricesA.2ofmatrixmultiplicationissometimesreferredtoasaninnerproduct.Itcanbevisualizedbyplacingtheelementsofnexttothoseofqandadding,asfollows:1q1D1q1;2q2D2q2;::::::mqmDmqm: qDmXiD1iqi:Intheseterms,theelementscijofmatrixCDABarefoundbytakingtheinnerproductofAi(theithrowofA)withBj(thejthcolumnofB);thatis,cijDAiBj.Thefollowingpropertiesofmatricescanbeseeneasilybywritingouttheappropriateexpressionsineachinstanceandrearrangingtheterms:ACBDBCA(Commutativelaw)AC.BCC/D.ACB/CC(Associativelaw)A.BC/D.AB/C(Associativelaw)A.BCC/DABCAC(Distributivelaw)Asaresult,ACBCCorABCiswelldened,sincetheevaluationscanbeperformedinanyorder.Thereareafewspecialmatricesthatwillbeusefulinourdiscussion,sowedenethemhere.Denition.Theidentitymatrixoforderm,writtenIm(orsimplyI,whennoconfusionarises)isasquarem-by-mmatrixwithonesalongthediagonalandzeroselsewhere.Forexample,I3D2410001000135:Itisimportanttonotethatforanym-by-mmatrixB,BImDImBDB.Inparticular,ImImDImorIIDI.Denition.ThetransposeofamatrixA,denotedAt,isformedbyinterchangingtherowsandcolumnsofA;thatis,atijDaji.IfAD241304;thenthetransposeofAisgivenby:AtD2423401435:Wecanshowthat.AB/tDBtAtsincetheijthelementofbothsidesoftheequalityisPkajkbki.Denition.Anelementarymatrixisasquarematrixwithonearbitrarycolumn,butotherwiseonesalongthediagonalandzeroselsewhere(i.e.,anidentifymatrixwiththeexceptionofonecolumn). 492VectorsandMatricesA.4LettingytD26664y1y2:::yn37775beacolumnvector,sincethedualvariablesareassociatedwiththeconstraintsoftheprimalproblem,wecanwritetheduallinearprogramincompactformasfollows:Minimizebtyt;subjectto:Atytct;yt0:Wecanalsowritethedualintermsoftheuntransposedvectorsasfollows:Minimizeyb,subjectto:yAc;y0:InthisformitiseasytowritetheproblemintermsoftherowvectorsAiofthematrixA,as:Minimizey1b1Cy2b2CCymbm,subjectto:y1A1Cy2A2CCymAmc;yi0.iD1;2;:::;m/:Finally,wecanwritetheprimalanddualproblemsinequalityform.Intheprimal,wemerelydeneanm-dimensionalcolumnvectorsmeasuringtheamountofslackineachconstraint,andwrite:Maximizecx,subjectto:AxCIsDb;x0;s0:Inthedual,wedeneann-dimensionalrowvectorumeasuringtheamountofsurplusineachdualconstraintandwrite:Minimizeyb,subjectto:yAuIDc;y0;u0:A.4THEINVERSEOFAMATRIXDenition.Givenasquarem-by-mmatrixB,ifthereisanm-by-mmatrixDsuchthatDBDBDDI;thenDiscalledtheinverseofBandisdenotedB1:NotethatB1doesnotmean1=BorI=B,sincedivisionisnotdenedformatrices.ThesymbolB1isjustaconvenientwaytoemphasizetherelationshipbetweentheinversematrixDandtheoriginalmatrixB.Thereareanumberofsimplepropertiesofinversesthataresometimeshelpfultoknow. 494VectorsandMatricesA.4Ifb11D0,wemerelychoosesomeothervariabletoisolateintherstequation.Inmatrixform,thenewmatricesofthexandycoefcientsaregivenrespectivelybyE1BandE1I,whereE1isanelementarymatrixoftheform:E1D2666664k1000k2100k3010::::::km0013777775;k1D1 b11;:::kiDbi1 b11.iD2;3;:::;m/:Further,sinceb11ischosentobenonzero,E1hasaninversegivenby:E11D26666641=k1000k2100k3010::::::km0013777775:Thusbyproperty(iii)above,ifBhasaninverse,thenE1Bhasaninverseandtheproceduremayberepeated.Somexjcoefcientinthesecondrowoftheupdatedsystemmustbenonzero,ornovariablecanbeisolatedinthesecondrow,implyingthattheinversedoesnotexist.Theproceduremayberepeatedbyeliminatingthisxjfromtheotherequations.Thus,anewelementarymatrixE2isdened,andthenewsystem.E2E1B/xD.E2E1/yhasx1isolatedinequation1andx2inequation2.Repeatingtheprocedurenallygives:.EmEm1E2E1B/xD.EmEm1E2E1/ywithonevariableisolatedineachequation.Ifvariablexjisisolatedinequationj,thenalsystemreads:x1D11y1C12y2CC1mym;x2D21y1C22y2CC2mym;::::::xmDm1y1Cm2y2CCmmym;andB1D2666411121m21222m::::::m1m2mm37775:Equivalently,B1DEmEm1E2E1isexpressedinproductformasthematrixproductofelementarymatrices.If,atanystageintheprocedure,itisnotpossibletoisolateavariableintherowunderconsideration,thentheinverseoftheoriginalmatrixdoesnotexist.Ifxjhasnotbeenisolatedinthejthequation,theequationsmayhavetobepermutedtodetermineB1.Thispointisillustratedbythefollowingexample: 496VectorsandMatricesA.5multiplicationoftwopartitionedmatricesAD24A11A12A21A22A31A3235;andBDB11B12B21B22resultsinABD24A11B11CA12B21A11B12CA12B22A21B11CA22B21A21B12CA22B22A31B11CA32B21A31B12CA32B2235;assumingtheindicatedproductsaredened;i.e.,thematricesAijandBjkhavetheappropriatedimensions.Toillustratethatpartitionedmatricesmaybehelpfulincomputinginverses,considerthefollowingexample.LetMDIQ0R;where0denotesamatrixwithallzeroentries.ThenM1DABCDsatisesMM1DIorIQ0RABCDDI00I;whichimpliesthefollowingmatrixequations:ACQCDI;BCQDD0;RCD0;RDDI:SolvingthesesimultaneousequationsgivesCD0;ADI;DDR1;andBDQR1Ior,equivalently,M1DIQR10R1:NotethatweneedonlycomputeR1inordertodetermineM1easily.Thistypeofuseofpartitionedmatricesistheessenceofmanyschemesforhandlinglarge-scalelinearprogramswithspecialstructures.A.5BASESANDREPRESENTATIONSInChapters2,3,and4,theconceptofabasisplaysanimportantroleindevelopingthecomputationalproceduresandfundamentalpropertiesoflinearprogramming.Inthissection,wepresentthealgebraicfoundationsofthisconcept.Denition.m-dimensionalrealspaceRmisdenedasthecollectionofallm-dimensionalvectorsyD.y1;y2;:::;ym/.Denition.Asetofm-dimensionalvectorsA1;A2;:::;Akislinearlydependentifthereexistrealnumbers1;2;:::;k,notallzero,suchthat1A1C2A2CCkAkD0:(1)Iftheonlysetofjsforwhich(1)holdsis1D2DDkD0,thenthem-vectorsA1;A2;:::;Akaresaidtobelinearlyindependent. 498VectorsandMatricesA.5sothatArisdependentuponA1;A2;:::;Ar1.Then,settingrD1,wehaver1XjD1jAjrArD0;whichshowsthatA1;A2;:::;Ararelinearlydependent.Next,ifthesetofvectorsisdependent,thenrXjD1jAjD0;withatleastonej6D0;sayr6D0.Then,ArDr1XjD1jAj;wherejDj r;andArdependsuponA1;A2;:::;Ar1:Property2.TherepresentationofanyvectorQintermsofbasisvectorsA1;A2;:::;Amisunique.Proof.SupposethatQisrepresentedasbothQDmXjD1jAjandQDmXjD10jAj:EliminatingQgives0DPmjD1.j0j/Aj.SinceA1;A2;:::;Amconstituteabasis,theyarelinearlyindependentandeach.j0j/D0:Thatis,jD0j,sothattherepresentationmustbeunique.ThispropositionactuallyshowsthatifQcanberepresentedintermsofthelinearlyindependentvectorsA1;A2;:::;Am,whetherabasisornot,thentherepresentationisunique.IfA1;A2;:::;Amisabasis,thentherepresentationisalwayspossiblebecauseofthedenitionofabasis.Severalmathematical-programmingalgorithms,includingthesimplexmethodforlinearprogramming,movefromonebasistoanotherbyintroducingavectorintothebasisinplaceofonealreadythere.Property3.LetA1;A2;:::;AmbeabasisforRm;letQ6D0beanym-dimensionalvector;andlet.1;2;:::;m/betherepresentationofQintermsofthisbasis;thatis,QDmXjD1jAj:(2)Then,ifQreplacesanyvectorArinthebasiswithr6D0;thenewsetofvectorsisabasisforRm. 500VectorsandMatricesA.5Proof.IfQ1;Q2;:::;QkandA1;A2;:::;Araretwobases,thenProperty4impliesthatkr.ByreversingtherolesoftheQjandAi,wealsohaverkandthuskDr,andeverytwobasescontainthesamenumberofvectors.Buttheunitm-dimensionalvectorsu1;u2;:::;umconstituteabasiswithm-dimensionalvectors,andconsequently,everybasisofRmmustcontainmvectors.Property6.EverycollectionQ1;Q2;:::;Qkoflinearlyindependentm-dimensionalvectorsiscon-tainedinabasis.Proof.ApplyProperty4withA1;A2;:::;Amtheunitm-dimensionalvectors.Property7.Everymlinearly-independentvectorsofRmformabasis.Everycollectionof.mC1/ormorevectorsinRmarelinearlydependent.Proof.Immediate,fromProperties5and6.IfamatrixBisconstructedwithmlinearly-independentcolumnvectorsB1;B2;:::;Bm;thepropertiesjustdevelopedforvectorsaredirectlyrelatedtotheconceptofabasisinverseintroducedpreviously.Wewillshowtherelationshipsbydeningtheconceptofanonsingularmatrixintermsoftheindependenceofitsvectors.Theusualdenitionofanonsingularmatrixisthatthedeterminantofthematrixisnonzero.However,thisdenitionstemshistoricallyfromcalculatinginversesbythemethodofcofactors,whichisoflittlecomputationalinterestforourpurposesandwillnotbepursued.Denition.Anm-by-mmatrixBissaidtobenonsingularifbothitscolumnvectorsB1;B2;:::;BmandrowsvectorsB1;B2;:::;Bmarelinearlyindependent.Althoughwewillnotestablishthepropertyhere,deningnonsingularityofBmerelyintermsoflinearindependenceofeitheritscolumnvectorsorrowvectorsisequivalenttothisdenition.Thatis,linearindependenceofeitheritscolumnorrowvectorsautomaticallyimplieslinearindependenceoftheothervectors.Property8.Anm-by-mmatrixBhasaninverseifandonlyifitisnonsingular.Proof.First,supposethatBhasaninverseandthatB11CB22CCBmmD0:LettingDh1;2;:::;mi,inmatrixform,thisexpressionsaysthatBD0:Thus.B1/.B/DB1.0/D0or.B1B/DIDD0.Thatis,1D2DDmD0,sothatvectorsB1;B2;:::;Bmarelinearlyindependent.Similarly,BD0impliesthatD.BB1/D.B/B1D0B1D0;sothattherowsB1;B2;:::;Bmarelinearlyindependent. 502VectorsandMatricesA.6 FigureA.1Proof.Byreindexingifnecessary,wemayassumethatonlytherstrcomponentsofyarepositive;thatis,y10;y20;:::;yr0;yrC1DyrC2DDynD0:WemustshowthatanyvectorysolvingAyDb;y0;isanextremepointifandonlyiftherstrcolumnA1;A2;:::;ArofAarelinearlyindependent.First,supposethatthesecolumnsarenotlinearlyindependent,sothatA11CA22CCArrD0(5)forsomerealnumbers1;2;:::;rnotallzero.IfweletxdenotethevectorxD.1;2;:::;r;0;:::;0/,thenexpression(5)canbewrittenasAxD0.NowletwDyCxandNwDyx.Then,aslongasischosensmallenoughtosatisfyjjjyjforeachcomponentjD1;2;:::;r;bothw0andNw0.Butthen,bothwandNwarecontainedinS,sinceA.yCx/DAyCAxDAyC.0/Db;and,similarly,A.yx/Db.However,sinceyD1 2.wCNw/,weseethatyisnotanextremepointofSinthiscase.Consequently,everyextremepointofSsatisesthelinearindependencerequirement.Conversely,supposethatA1;A2;:::Ararelinearlyindependent.IfyDwC.1/xforsomepointswandxofSandsome01;thenyjDwjC.1/xj.SinceyjD0forjrC1andwj0;xj0;thennecessarilywjDxjD0forjrC1:Therefore,A1y1CA2y2CCAryrDA1w1CA2w2CCArwrDA1x1CA2x2CCArxrDb:Since,byProperty2inSectionA.5,therepresentationofthevectorbintermsofthelinearlyindependentvectorsA1;A2;:::;Arisunique,thenyjDzjDxj:ThusthetwopointswandxcannotbedistinctandthereforeyisanextremepointofS.IfAcontainsabasis(i.e.,thetowsofAarelinearlyindependent),then,byProperty6,anycollectionA1;A2;:::;AroflinearlyindependentvectorscanbeextendedtoabasisA1;A2;:::;Am.Theextreme-pointtheoremshows,inthiscase,thateveryextremepointycanbeassociatedwithabasicfeasiblesolution,i.e.,withasolutionsatisfyingyjD0fornonbasicvariablesyj,forjDmC1;mC2;:::;n.Chapter2showsthatoptimalsolutionstolinearprogramscanbefoundatbasicfeasiblesolutionsorequivalently,now,atextremepointsofthefeasibleregion.Atthispoint,letususethelinear-algebratools A.6ExtremePointsofLinearPrograms503ofthisappendixtodrivethisresultindependently.Thiswillmotivatethesimplexmethodforsolvinglinearprogramsalgebraically.SupposethatyisafeasiblesolutiontothelinearprogramMaximizecx,subjectto:AxDb;x0;(6)and,byreindexingvariablesifnecessary,thaty10;y20;:::;yrC10andyrC2DyrC3DDynD0:IfthecolumnArC1islinearlydependentuponcolumnsA1;A2;:::;Ar,thenArC1DA11CA22CCArr;(7)withatleastoneoftheconstantsjnonzeroforjD1;2;:::;r.MultiplyingbothsidesofthisexpressionbygivesArC1DA1.1/CA2.2/CCAr.r/;(8)whichstatesthatwemaysimulatetheeffectofsettingxrC1Din(6)bysettingx1;x2;:::;xr,respectively,to.1/;.2/;:::;.r/.TakingD1gives:QcrC1D1c1C2c2CCrcrastheper-unitprotfromthesimulatedactivityofusing1unitsofx1;2unitsofx2,throughrunitsofxr,inplaceof1unitofxrC1.LettingNxD.1;2;:::;r;C1;0;:::;0/;Eq.(8)isrewrittenasA.x/DANxD0.HereNxisinterpretedassettingxrC1to1anddecreasingthesimulatedactivitytocompensate.Thus,A.yCNx/DAyCANxDAyC0Db;sothatyCNxisfeasibleaslongasyCNx0(thisconditionissatisedifischosensothatjjjyjforeverycomponentjD1;2;:::;r).ThereturnfromyCNxisgivenby:c.yCNx/DcyCcNxDcyC.crC1QcrC1/:Consequently,ifQcrC1crC1,thesimulatedactivityislessprotablethanthe.rC1/stactivityitself,andreturnimprovesbyincreasing.IfQcrC1]TJ/;ག ;.9;Y T; 10;.816; 0 T; [00;crC1,returnincreasesbydecreasing(i.e.,decreasingyrC1andincreasingthesimulatedactivity).IfQcrC1DcrC1,returnisunaffectedby.Theseobservationimplythat,iftheobjectivefunctionisboundedfromaboveoverthefeasibleregion,thenbyincreasingthesimulatedactivityanddecreasingactivityyrC1,orviceversa,wecanndanewfeasiblesolutionwhoseobjectivevalueisatleastaslargeascybutwhichcontainsatleastonemorezerocomponentthany.For,supposethatQcrC1crC1.ThenbydecreasingfromD0;c.yCNx/cyIeventuallyyjCNxjD0forsomecomponentjD1;2;:::;rC1(possiblyyrC1CNxrC1DyrC1CD0/.Ontheotherhand,ifQcrC1crC1;thenc.yCNx/]TJ/;ག ;.9;Y T; 11;.648; 0 T; [00;cyasincreasesfromD0Iifsomecomponentofjfrom(7)ispositive,theneventuallyyjCNxjDyjjreaches0asincreases.(Ifeveryj0,thenwemayincreaseindenitely,c.yCNx/!C1;andtheobjectivevalueisunboundedovertheconstraints,contrarytoourassumption.)Therefore,ifeitherQcrC1crC1orQcrC1crC1;wecanndavalueforsuchthatatleastonecomponentofyjCNxjbecomeszeroforjD1;2;:::;rC1.SinceyjD0andNxjD0forj]TJ/;ག ;.9;Y T; 11;.592; 0 T; [00;rC1;yjCNxjremainsat0forj]TJ/;ག ;.9;Y T; 11;.592; 0 T; [00;rC1.Thus,theentirevectoryCNxcontainsatleastonemorepositivecomponentthanyandc.yCNx/cy.Withalittlemoreargument,wecanusethisresulttoshowthattheremustbeanoptimalextreme-pointsolutiontoalinearprogram.