In particular they are useful for compactly representing and discussing the linear programming problem Maximize subject to i j This appendix reviews several properties of vectors and matrices that are especially relevant to this problem We shoul ID: 23411
Download Pdf The PPT/PDF document "Vectors and Matrices Appendix Vectors an..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
488VectorsandMatricesA.2Equalityandorderingofvectorsaredenedbycomparingthevectors’individualcomponents.Formally,letyD.y1;y2;:::;yk/andzD.z1;z2;:::;zk/betwok-dimensionalvectors.Wewrite:yDzwhenyjDzj.jD1;2;:::;k/;yzorzywhenyjzj.jD1;2;:::;k/;y�zorzywhenyj&#x]TJ/;ག ;.9;Y T; 12;&#x.294;&#x 0 T; [00;zj.jD1;2;:::;k/;andsay,respectively,thatyequalsz,yisgreaterthanorequaltozandthatyisgreaterthanz.Inthelasttwocases,wealsosaythatzislessthanorequaltoyandlessthany.Itshouldbeemphasizedthatnotallvectorsareordered.Forexample,ifyD.3;1;�2/andxD.1;1;1/,thenthersttwocomponentsofyaregreaterthanorequaltothersttwocomponentsofxbutthethirdcomponentofyislessthanthecorrespondingcomponentofx.Analnote:0isusedtodenotethenullvector(0,0,…,0),wherethedimensionofthevectorisunderstoodfromcontext.Thus,ifxisak-dimensionalvector,x0meansthateachcomponentxjofthevectorxisnonnegative.Wealsodenescalarmultiplicationandadditionintermsofthecomponentsofthevectors.Denition.ScalarmultiplicationofavectoryD.y1;y2;:::;yk/andascalar isdenedtobeanewvectorzD.z1;z2;:::;zk/,writtenzD yorzDy ;whosecomponentsaregivenbyzjD yj.Denition.Vectoradditionoftwok-dimensionalvectorsxD.x1;x2;:::;xk/andyD.y1;y2;:::;yk/isdenedasanewvectorzD.z1;z2;:::;zk/,denotedzDxCy,withcomponentsgivenbyzjDxjCyj.Asanexampleofscalarmultiplication,consider4.3;0;�1;8/D.12;0;�4;32/;andforvectoraddition,.3;4;1;�3/C.1;3;�2;5/D.4;7;�1;2/:Usingbothoperations,wecanmakethefollowingtypeofcalculation:.1;0/x1C.0;1/x2C.�3;�8/x3D.x1;0/C.0;x2/C.�3x3;�8x3/D.x1�3x3;x2�8x3/:Itisimportanttonotethatyandzmusthavethesamedimensionsforvectoradditionandvectorcomparisons.Thus.6;2;�1/C.4;0/isnotdened,and.4;0;�1/D.4;0/makesnosenseatall.A.2MATRICESWecannowextendtheseideastoanyrectangulararrayofnumbers,whichwecallamatrix.Denition.AmatrixisdenedtobearectangulararrayofnumbersAD26664a11a12a1na21a22a2n::::::am1am2amn37775;whosedimensionismbyn.AiscalledsquareifmDn.ThenumbersaijarereferredtoastheelementsofA.Thetableauofalinearprogrammingproblemisanexampleofamatrix.Wedeneequalityoftwomatricesintermsoftheirelementsjustasinthecaseofvectors. 490VectorsandMatricesA.2ofmatrixmultiplicationissometimesreferredtoasaninnerproduct.Itcanbevisualizedbyplacingtheelementsofnexttothoseofqandadding,asfollows:1q1D1q1;2q2D2q2;::::::mqmDmqm: qDmXiD1iqi:Intheseterms,theelementscijofmatrixCDABarefoundbytakingtheinnerproductofAi(theithrowofA)withBj(thejthcolumnofB);thatis,cijDAiBj.Thefollowingpropertiesofmatricescanbeseeneasilybywritingouttheappropriateexpressionsineachinstanceandrearrangingtheterms:ACBDBCA(Commutativelaw)AC.BCC/D.ACB/CC(Associativelaw)A.BC/D.AB/C(Associativelaw)A.BCC/DABCAC(Distributivelaw)Asaresult,ACBCCorABCiswelldened,sincetheevaluationscanbeperformedinanyorder.Thereareafewspecialmatricesthatwillbeusefulinourdiscussion,sowedenethemhere.Denition.Theidentitymatrixoforderm,writtenIm(orsimplyI,whennoconfusionarises)isasquarem-by-mmatrixwithonesalongthediagonalandzeroselsewhere.Forexample,I3D2410001000135:Itisimportanttonotethatforanym-by-mmatrixB,BImDImBDB.Inparticular,ImImDImorIIDI.Denition.ThetransposeofamatrixA,denotedAt,isformedbyinterchangingtherowsandcolumnsofA;thatis,atijDaji.IfAD24�1�304;thenthetransposeofAisgivenby:AtD242�340�1435:Wecanshowthat.AB/tDBtAtsincetheijthelementofbothsidesoftheequalityisPkajkbki.Denition.Anelementarymatrixisasquarematrixwithonearbitrarycolumn,butotherwiseonesalongthediagonalandzeroselsewhere(i.e.,anidentifymatrixwiththeexceptionofonecolumn). 492VectorsandMatricesA.4LettingytD26664y1y2:::yn37775beacolumnvector,sincethedualvariablesareassociatedwiththeconstraintsoftheprimalproblem,wecanwritetheduallinearprogramincompactformasfollows:Minimizebtyt;subjectto:Atytct;yt0:Wecanalsowritethedualintermsoftheuntransposedvectorsasfollows:Minimizeyb,subjectto:yAc;y0:InthisformitiseasytowritetheproblemintermsoftherowvectorsAiofthematrixA,as:Minimizey1b1Cy2b2CCymbm,subjectto:y1A1Cy2A2CCymAmc;yi0.iD1;2;:::;m/:Finally,wecanwritetheprimalanddualproblemsinequalityform.Intheprimal,wemerelydeneanm-dimensionalcolumnvectorsmeasuringtheamountofslackineachconstraint,andwrite:Maximizecx,subjectto:AxCIsDb;x0;s0:Inthedual,wedeneann-dimensionalrowvectorumeasuringtheamountofsurplusineachdualconstraintandwrite:Minimizeyb,subjectto:yA�uIDc;y0;u0:A.4THEINVERSEOFAMATRIXDenition.Givenasquarem-by-mmatrixB,ifthereisanm-by-mmatrixDsuchthatDBDBDDI;thenDiscalledtheinverseofBandisdenotedB�1:NotethatB�1doesnotmean1=BorI=B,sincedivisionisnotdenedformatrices.ThesymbolB�1isjustaconvenientwaytoemphasizetherelationshipbetweentheinversematrixDandtheoriginalmatrixB.Thereareanumberofsimplepropertiesofinversesthataresometimeshelpfultoknow. 494VectorsandMatricesA.4Ifb11D0,wemerelychoosesomeothervariabletoisolateintherstequation.Inmatrixform,thenewmatricesofthexandycoefcientsaregivenrespectivelybyE1BandE1I,whereE1isanelementarymatrixoftheform:E1D2666664k1000k2100k3010::::::km0013777775;k1D1 b11;:::kiD�bi1 b11.iD2;3;:::;m/:Further,sinceb11ischosentobenonzero,E1hasaninversegivenby:E�11D26666641=k1000�k2100�k3010::::::�km0013777775:Thusbyproperty(iii)above,ifBhasaninverse,thenE1Bhasaninverseandtheproceduremayberepeated.Somexjcoefcientinthesecondrowoftheupdatedsystemmustbenonzero,ornovariablecanbeisolatedinthesecondrow,implyingthattheinversedoesnotexist.Theproceduremayberepeatedbyeliminatingthisxjfromtheotherequations.Thus,anewelementarymatrixE2isdened,andthenewsystem.E2E1B/xD.E2E1/yhasx1isolatedinequation1andx2inequation2.Repeatingtheprocedurenallygives:.EmEm�1E2E1B/xD.EmEm�1E2E1/ywithonevariableisolatedineachequation.Ifvariablexjisisolatedinequationj,thenalsystemreads:x1D11y1C12y2CC1mym;x2D21y1C22y2CC2mym;::::::xmDm1y1Cm2y2CCmmym;andB�1D2666411121m21222m::::::m1m2mm37775:Equivalently,B�1DEmEm�1E2E1isexpressedinproductformasthematrixproductofelementarymatrices.If,atanystageintheprocedure,itisnotpossibletoisolateavariableintherowunderconsideration,thentheinverseoftheoriginalmatrixdoesnotexist.Ifxjhasnotbeenisolatedinthejthequation,theequationsmayhavetobepermutedtodetermineB�1.Thispointisillustratedbythefollowingexample: 496VectorsandMatricesA.5multiplicationoftwopartitionedmatricesAD24A11A12A21A22A31A3235;andBDB11B12B21B22resultsinABD24A11B11CA12B21A11B12CA12B22A21B11CA22B21A21B12CA22B22A31B11CA32B21A31B12CA32B2235;assumingtheindicatedproductsaredened;i.e.,thematricesAijandBjkhavetheappropriatedimensions.Toillustratethatpartitionedmatricesmaybehelpfulincomputinginverses,considerthefollowingexample.LetMDIQ0R;where0denotesamatrixwithallzeroentries.ThenM�1DABCDsatisesMM�1DIorIQ0RABCDDI00I;whichimpliesthefollowingmatrixequations:ACQCDI;BCQDD0;RCD0;RDDI:SolvingthesesimultaneousequationsgivesCD0;ADI;DDR�1;andBD�QR�1Ior,equivalently,M�1DI�QR�10R�1:NotethatweneedonlycomputeR�1inordertodetermineM�1easily.Thistypeofuseofpartitionedmatricesistheessenceofmanyschemesforhandlinglarge-scalelinearprogramswithspecialstructures.A.5BASESANDREPRESENTATIONSInChapters2,3,and4,theconceptofabasisplaysanimportantroleindevelopingthecomputationalproceduresandfundamentalpropertiesoflinearprogramming.Inthissection,wepresentthealgebraicfoundationsofthisconcept.Denition.m-dimensionalrealspaceRmisdenedasthecollectionofallm-dimensionalvectorsyD.y1;y2;:::;ym/.Denition.Asetofm-dimensionalvectorsA1;A2;:::;Akislinearlydependentifthereexistrealnumbers 1; 2;:::; k,notallzero,suchthat 1A1C 2A2CC kAkD0:(1)Iftheonlysetof j’sforwhich(1)holdsis 1D 2DD kD0,thenthem-vectorsA1;A2;:::;Akaresaidtobelinearlyindependent. 498VectorsandMatricesA.5sothatArisdependentuponA1;A2;:::;Ar�1.Then,settingrD�1,wehaver�1XjD1jAj�rArD0;whichshowsthatA1;A2;:::;Ararelinearlydependent.Next,ifthesetofvectorsisdependent,thenrXjD1 jAjD0;withatleastone j6D0;say r6D0.Then,ArDr�1XjD1jAj;wherejD� j r;andArdependsuponA1;A2;:::;Ar�1:Property2.TherepresentationofanyvectorQintermsofbasisvectorsA1;A2;:::;Amisunique.Proof.SupposethatQisrepresentedasbothQDmXjD1jAjandQDmXjD10jAj:EliminatingQgives0DPmjD1.j�0j/Aj.SinceA1;A2;:::;Amconstituteabasis,theyarelinearlyindependentandeach.j�0j/D0:Thatis,jD0j,sothattherepresentationmustbeunique.ThispropositionactuallyshowsthatifQcanberepresentedintermsofthelinearlyindependentvectorsA1;A2;:::;Am,whetherabasisornot,thentherepresentationisunique.IfA1;A2;:::;Amisabasis,thentherepresentationisalwayspossiblebecauseofthedenitionofabasis.Severalmathematical-programmingalgorithms,includingthesimplexmethodforlinearprogramming,movefromonebasistoanotherbyintroducingavectorintothebasisinplaceofonealreadythere.Property3.LetA1;A2;:::;AmbeabasisforRm;letQ6D0beanym-dimensionalvector;andlet.1;2;:::;m/betherepresentationofQintermsofthisbasis;thatis,QDmXjD1jAj:(2)Then,ifQreplacesanyvectorArinthebasiswithr6D0;thenewsetofvectorsisabasisforRm. 500VectorsandMatricesA.5Proof.IfQ1;Q2;:::;QkandA1;A2;:::;Araretwobases,thenProperty4impliesthatkr.ByreversingtherolesoftheQjandAi,wealsohaverkandthuskDr,andeverytwobasescontainthesamenumberofvectors.Buttheunitm-dimensionalvectorsu1;u2;:::;umconstituteabasiswithm-dimensionalvectors,andconsequently,everybasisofRmmustcontainmvectors.Property6.EverycollectionQ1;Q2;:::;Qkoflinearlyindependentm-dimensionalvectorsiscon-tainedinabasis.Proof.ApplyProperty4withA1;A2;:::;Amtheunitm-dimensionalvectors.Property7.Everymlinearly-independentvectorsofRmformabasis.Everycollectionof.mC1/ormorevectorsinRmarelinearlydependent.Proof.Immediate,fromProperties5and6.IfamatrixBisconstructedwithmlinearly-independentcolumnvectorsB1;B2;:::;Bm;thepropertiesjustdevelopedforvectorsaredirectlyrelatedtotheconceptofabasisinverseintroducedpreviously.Wewillshowtherelationshipsbydeningtheconceptofanonsingularmatrixintermsoftheindependenceofitsvectors.Theusualdenitionofanonsingularmatrixisthatthedeterminantofthematrixisnonzero.However,thisdenitionstemshistoricallyfromcalculatinginversesbythemethodofcofactors,whichisoflittlecomputationalinterestforourpurposesandwillnotbepursued.Denition.Anm-by-mmatrixBissaidtobenonsingularifbothitscolumnvectorsB1;B2;:::;BmandrowsvectorsB1;B2;:::;Bmarelinearlyindependent.Althoughwewillnotestablishthepropertyhere,deningnonsingularityofBmerelyintermsoflinearindependenceofeitheritscolumnvectorsorrowvectorsisequivalenttothisdenition.Thatis,linearindependenceofeitheritscolumnorrowvectorsautomaticallyimplieslinearindependenceoftheothervectors.Property8.Anm-by-mmatrixBhasaninverseifandonlyifitisnonsingular.Proof.First,supposethatBhasaninverseandthatB1 1CB2 2CCBm mD0:Letting Dh 1; 2;:::; mi,inmatrixform,thisexpressionsaysthatB D0:Thus.B�1/.B /DB�1.0/D0or.B�1B/ DI D D0.Thatis, 1D 2DD mD0,sothatvectorsB1;B2;:::;Bmarelinearlyindependent.Similarly, BD0impliesthat D .BB�1/D. B/B�1D0B�1D0;sothattherowsB1;B2;:::;Bmarelinearlyindependent. 502VectorsandMatricesA.6 FigureA.1Proof.Byreindexingifnecessary,wemayassumethatonlytherstrcomponentsofyarepositive;thatis,y1�0;y2�0;:::;yr�0;yrC1DyrC2DDynD0:WemustshowthatanyvectorysolvingAyDb;y0;isanextremepointifandonlyiftherstrcolumnA1;A2;:::;ArofAarelinearlyindependent.First,supposethatthesecolumnsarenotlinearlyindependent,sothatA1 1CA2 2CCAr rD0(5)forsomerealnumbers 1; 2;:::; rnotallzero.IfweletxdenotethevectorxD. 1; 2;:::; r;0;:::;0/,thenexpression(5)canbewrittenasAxD0.NowletwDyCxandNwDy�x.Then,aslongasischosensmallenoughtosatisfyj jjyjforeachcomponentjD1;2;:::;r;bothw0andNw0.Butthen,bothwandNwarecontainedinS,sinceA.yCx/DAyCAxDAyC.0/Db;and,similarly,A.y�x/Db.However,sinceyD1 2.wCNw/,weseethatyisnotanextremepointofSinthiscase.Consequently,everyextremepointofSsatisesthelinearindependencerequirement.Conversely,supposethatA1;A2;:::Ararelinearlyindependent.IfyDwC.1�/xforsomepointswandxofSandsome01;thenyjDwjC.1�/xj.SinceyjD0forjrC1andwj0;xj0;thennecessarilywjDxjD0forjrC1:Therefore,A1y1CA2y2CCAryrDA1w1CA2w2CCArwrDA1x1CA2x2CCArxrDb:Since,byProperty2inSectionA.5,therepresentationofthevectorbintermsofthelinearlyindependentvectorsA1;A2;:::;Arisunique,thenyjDzjDxj:ThusthetwopointswandxcannotbedistinctandthereforeyisanextremepointofS.IfAcontainsabasis(i.e.,thetowsofAarelinearlyindependent),then,byProperty6,anycollectionA1;A2;:::;AroflinearlyindependentvectorscanbeextendedtoabasisA1;A2;:::;Am.Theextreme-pointtheoremshows,inthiscase,thateveryextremepointycanbeassociatedwithabasicfeasiblesolution,i.e.,withasolutionsatisfyingyjD0fornonbasicvariablesyj,forjDmC1;mC2;:::;n.Chapter2showsthatoptimalsolutionstolinearprogramscanbefoundatbasicfeasiblesolutionsorequivalently,now,atextremepointsofthefeasibleregion.Atthispoint,letususethelinear-algebratools A.6ExtremePointsofLinearPrograms503ofthisappendixtodrivethisresultindependently.Thiswillmotivatethesimplexmethodforsolvinglinearprogramsalgebraically.SupposethatyisafeasiblesolutiontothelinearprogramMaximizecx,subjectto:AxDb;x0;(6)and,byreindexingvariablesifnecessary,thaty1�0;y2�0;:::;yrC1�0andyrC2DyrC3DDynD0:IfthecolumnArC1islinearlydependentuponcolumnsA1;A2;:::;Ar,thenArC1DA1 1CA2 2CCAr r;(7)withatleastoneoftheconstants jnonzeroforjD1;2;:::;r.MultiplyingbothsidesofthisexpressionbygivesArC1DA1. 1/CA2. 2/CCAr. r/;(8)whichstatesthatwemaysimulatetheeffectofsettingxrC1Din(6)bysettingx1;x2;:::;xr,respectively,to. 1/;. 2/;:::;. r/.TakingD1gives:QcrC1D 1c1C 2c2CC rcrastheper-unitprotfromthesimulatedactivityofusing 1unitsofx1; 2unitsofx2,through runitsofxr,inplaceof1unitofxrC1.LettingNxD.� 1;� 2;:::;� r;C1;0;:::;0/;Eq.(8)isrewrittenasA.x/DANxD0.HereNxisinterpretedassettingxrC1to1anddecreasingthesimulatedactivitytocompensate.Thus,A.yCNx/DAyCANxDAyC0Db;sothatyCNxisfeasibleaslongasyCNx0(thisconditionissatisedifischosensothatj jjyjforeverycomponentjD1;2;:::;r).ThereturnfromyCNxisgivenby:c.yCNx/DcyCcNxDcyC.crC1�QcrC1/:Consequently,ifQcrC1crC1,thesimulatedactivityislessprotablethanthe.rC1/stactivityitself,andreturnimprovesbyincreasing.IfQcrC1&#x]TJ/;ག ;.9;Y T; 10;&#x.816;&#x 0 T; [00;crC1,returnincreasesbydecreasing(i.e.,decreasingyrC1andincreasingthesimulatedactivity).IfQcrC1DcrC1,returnisunaffectedby.Theseobservationimplythat,iftheobjectivefunctionisboundedfromaboveoverthefeasibleregion,thenbyincreasingthesimulatedactivityanddecreasingactivityyrC1,orviceversa,wecanndanewfeasiblesolutionwhoseobjectivevalueisatleastaslargeascybutwhichcontainsatleastonemorezerocomponentthany.For,supposethatQcrC1crC1.ThenbydecreasingfromD0;c.yCNx/cyIeventuallyyjCNxjD0forsomecomponentjD1;2;:::;rC1(possiblyyrC1CNxrC1DyrC1CD0/.Ontheotherhand,ifQcrC1crC1;thenc.yCNx/&#x]TJ/;ག ;.9;Y T; 11;&#x.648;&#x 0 T; [00;cyasincreasesfromD0Iifsomecomponentof jfrom(7)ispositive,theneventuallyyjCNxjDyj� jreaches0asincreases.(Ifevery j0,thenwemayincreaseindenitely,c.yCNx/!C1;andtheobjectivevalueisunboundedovertheconstraints,contrarytoourassumption.)Therefore,ifeitherQcrC1crC1orQcrC1crC1;wecanndavalueforsuchthatatleastonecomponentofyjCNxjbecomeszeroforjD1;2;:::;rC1.SinceyjD0andNxjD0forj&#x]TJ/;ག ;.9;Y T; 11;&#x.592;&#x 0 T; [00;rC1;yjCNxjremainsat0forj&#x]TJ/;ག ;.9;Y T; 11;&#x.592;&#x 0 T; [00;rC1.Thus,theentirevectoryCNxcontainsatleastonemorepositivecomponentthanyandc.yCNx/cy.Withalittlemoreargument,wecanusethisresulttoshowthattheremustbeanoptimalextreme-pointsolutiontoalinearprogram.