On optimal quantization rules for sequential decision problems XuanLong Nguyen Martin J
141K - views

On optimal quantization rules for sequential decision problems XuanLong Nguyen Martin J

ainwright and Michael I Jordan Department of Electrical Engineering and Computer Science Department of Statistics Uni ersity of California Berk ele xuanlongwainwrigjordan eecsberkeleyedu Abstract consider the pr oblem of sequential decen tralized de

Download Pdf

On optimal quantization rules for sequential decision problems XuanLong Nguyen Martin J

Download Pdf - The PPT/PDF document "On optimal quantization rules for sequen..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Presentation on theme: "On optimal quantization rules for sequential decision problems XuanLong Nguyen Martin J"— Presentation transcript:

Page 1
On optimal quantization rules for sequential decision problems XuanLong Nguyen Martin J. ainwright and Michael I. Jordan Department of Electrical Engineering and Computer Science Department of Statistics Uni ersity of California, Berk ele xuanlong,wainwrig,jordan @eecs.berkeley.edu Abstract consider the pr oblem of sequential decen- tralized detection, pr oblem that entails the choice of stopping rule (specifying the sample size), global decision function (a choice between tw competing ypotheses), and set of quantization rules (the local decisions on the basis of which the

global decision is made). The main esult of this paper is to esolv an open pr oblem posed by eera alli et al. [12] concer ning whether optimal local decision functions or the Bay esian ormulation of sequential decentralized detection can be ound within the class of stationary rules. pr vide negati answer to this question by exploiting an asymptotic appr oximation to the optimal cost of stationary quantization rules, and the asymmetry of the ullback-Leibler di er gences. In addition, we sho that asymptotically optimal quantizers, when estricted to the space of blockwise stationary quantizers,

ar lik elihood- based thr eshold rules. In this paper we consider the problem of sequen- tial decentralized detection (see, e.g., [10], [11], [5]). Detection is classical discrimination or ypothesis- testing problem, in which observ ations are assumed to be dra wn i.i.d. from the conditional distrib ution where is the unkno wn state of the en vironment and in which the goal is to infer Placing this problem in communication- theoretic conte xt, decentr alized detection pr oblem is ypothesis-testing problem in which the decision-mak er is not gi en access to the ra data points ut instead must

infer based only on set of quantization rules or local decision functions where is the time step. Finally placing the problem in real-time conte xt, the sequential decentr alized detection pr oblem considers stream of data and corre- sponding stream of summary statistics and ould lik to ackno wledge support from NSF grant 0412995, and Sloan Fello wship (MJW). asks the decision-mak er to mak decision re arding in manner that optimally trades of accurac and delay That is, the sequential decentralized detection problem xtends the classical formulation of sequential centralized decision-making

problems (see, e.g., [8], [4]) to the decentralized setting. In setting up general frame ork of sequential decen- tralized problems, eera alli et al. [12 defined prob- lems (case through case E), distinguished from one another by the amount of information ailable to the local sensors. In particular in case E, the local sensors are pro vided with memory and with feedback from the global decision-mak er (also kno wn as the fusion center ), so that each sensor has ailable to it the current data as well as all of the summary statistics from all of the other local sensors. In other ords,

each local sensor has the same snapshot of past state as the fusion center; this is an instance of so-called quasi-classical information structure [3] for which dynamic programming (DP) characterizations of the optimal decision functions are ailable. eera alli et al. [12 xploit this act to sho that the decentralized case has much in common with the centralized case, in particular that lik elihood ratio tests are optimal local decision functions at the sensors and that ariant of sequential probability ratio test is optimal for the decision-mak er Unfortunately ho we er part of the spirit of

the decen- tralized detection is ar guably lost in case E, which re- quires full feedback. In particular in applications such as po wer -constrained sensor netw orks, we generally do not wish to assume that there are high-bandwidth feedback channels from the decision-mak er to the sensors, nor do we wish to assume that the sensors ha unbounded memory Most suited to this perspecti eand the focus of this paper is case A, in which the local decisions are of the simplified form i.e., neither local memory nor feedback are assumed to be ailable.
Page 2
Noting that case is not

amenable to dynamic pro- gramming and is presumably intractable, eera alli et al. [12] suggest restricting the analysis to the class of stationary local decision functions; i.e., local decision functions that are independent of The conjecture that stationary decision functions may actually be opti- mal in the setting of case (gi en the intuiti symmetry and high de gree of independence of the problem in this case), en though it is not possible to erify this optimality via DP ar guments. The truth or alsity of this conjecture has remained open since it as first posed by eera alli et al.

[12 ], [11]. In this paper we resolv this question by sho wing that stationary decision functions are not in act optimal for decentralized problems of type A. do so by pro viding an asymptotic characterization of stationary decision functions, one which allo ws us to pro vide countere xamples to the stationarity conjecture, both in an xact and an asymptotic setting. In the asymptotic setting (i.e., when the cost per sample goes to 0), we pro vide proof that in general there is al ays range of prior probabilities for which stationary strate gies are suboptimal. (W note in passing that the

reason for the suboptimality is easily statedit arises from the asymmetry of the ullback-Leibler di er gence.) It is well kno wn [10 that optimal quantizers when unr estricted can be xpressed as threshold rules based on the log lik elihood ratio (LLR). Our countere xamples in the stationary conjecture imply that the thresholds need not be stationary (i.e., the threshold may dif fer from sample to sample). In the remainder of the paper we ad- dresses partial con erse to this issue: specifically if we restrict ourselv es to stationary (or blockwise stationary) quantizer designs, then

there xists an optimal design consisting of LLR-based threshold rules. This section pro vides necessary background on the Bayesian formulation of sequential detection problems, and ald approximation of the optimal cost. Sequential detection pr oblems: Let and represent the conditional distrib utions of conditioning on and respecti ely focus on the Bayesian formulation of the sequential detection problem [8], [11], and let 1) and 0) denote the prior probabilities on the tw ypotheses. sequential decision rule consists of stopping time (defined with respect to the sigma field ), and

decision function (measurable with respect to ). The cost function is weighted sum of the sample size and the probability of incorrect decision := cN (1) where is the incremental cost of each sample. The erall goal is to choose the pair so as to minimize the xpected loss (1) It is well kno wn that stopping rule based on the lik elihood ratio is optimal [13 ], [1]. In particular gi en the partial sum := =1 log the optimal stopping rule tak es the form inf a; (2) for real numbers Gi en this stopping rule, the optimal decision function has the form if b; if (3) no de elop an xact xpression for

the optimal cost of the decision rule (3) Consider the tw types of error: No define [log jj and [log jj ith this notation, the cost a; of the decision rule based on en elopes and can be written as a; cN c c (4) where the second line follo ws from ald equation. ald appr oximation: When the data are i.i.d. condi- tioned on the ypothesis, the optimal cost can be charac- terized using dynamic programming approach [1], [8]. Let us consider an approximate xpression for the cost, due to ald (cf. [9]). be gin, the errors and are related to and by the classical inequalties (1 =e and (1 In

general, these inequalities need not hold with equality because the lik elihood ratio at at the optimal stopping time might ershoot either or (instead of attaining precisely the alue or at the stopping time). The approximation is based on ignoring this ershoot, and replacing the inequalities by equalities to solv for corresponding alues of and and (5)
Page 3
Alternati ely we can deri and in terms of and as: := log and := log (6) The approximate mapping (5) and (6) between a; and yields the follo wing approximation (1 (7a) (1 (7b) Combining this approximation with the approxima- tion

(6) for and we obtain ald appr oximation for the cost of sequential test with error and := c jj c (1 jj (8) where we define (1 jj (1 log log In the follo wing section, we will xploit this approximation in our analysis of quantizer design. urning no to the decentralized setting, the primary challenge lies in the design of the quantization rules An fix ed set of quantization rules yields sequence of compressed data to which the classical theory can be applied. In the decentralized setting, we write := for the distrib utions of the compressed data, conditioned on the ypothesis. say

that quantizer design is stationary if the rule is independent of in this case, we simplify the notation to and In addition, we define the KL di er gences := jj and := jj Moreo er let denote the analogue of the function in equation (8), defined using In this section, we describe ho w by xploiting ald approximation for sequential problemsit is possible to pro vide an asymptotic characterization of the optimal cost of an stationary quantization rule. ppr oximate quantizer design: Gi en fix ed station- ary quantizer ald approximation (8) suggests the follo wing strate gy for

solving approximating the cost of sequential detection strate gy or gi en set of errors and first assign the alues of thresholds and using approximation (6). Then use the quantity as an approximation to the true cost a; This approximation essentially ignores the ershoot of the lik elihood ratio Analyzing this ershoot to obtain finer approximation has been major theme in sequential analysis (cf. [9], [4], [7]). or the purpose of quantizer design, ho we er as we shall see, the approximation error incurred from ignoring the ershoot is only at most whereas the choice of quantizer

generally results in change of the order ( log Asymptotic analysis: The quantization problem in- olv es an output space that consists of finite set of dis- crete elements (e.g., for 1-bit quantization). Consequently we assume without loss of generality that sup 2U log =f The follo wing lemma guarantees that the approximation is asymptotically xact up to de viation and pro vides basis for characterizing the optimal cost: Lemma 1. (a) The err or in the appr oximation (8) is bounded as a; (9) (b) Define the optimal cost inf a;b a; Then as we have log (10) Pr oof: (a) be gin by

bounding the error in the approximation (6). By definition of the stopping time we ha either (i) or (ii) Consider all realizations for which condition (i) holds; for an such sequence, we ha aking sum er all such realizations, using the defi- nition of and and performing some algebra yields the inequality or equi alently log Similar reasoning for case (ii) yields log No note that (1 Conditioned on the ent b; we ha Similarly conditioned on the ent we ha This yields jj )) Similar reasoning yields (1 jj
Page 4
(b) By part (a), it suf fices to establish the asymp- totic

beha vior (10) for the quantity a; inf ˛; where the infimum is tak en among pairs of realizable error probabilities Moreo er we only need to consider the asymptotic re gime since the error probabilities and anish as It is simple to see that (1 jj log (1 = (1) and (1 jj log (1 = (1) Hence, inf ˛; can be xpressed as inf ˛; c log (1 = c log (1 = (11) This infimum is achie ed at c and c these error probabilities can be realized (for small) by using suf ficiently lar ge threshold and small threshold Plugging these quantities into equa- tion (11) yields inf ˛;

log as claimed. no consider the structure of optimal quantizers. It as sho wn by Tsitsiklis [10] that optimal quantizers tak the form of threshold rules based on the lik elihood ratio =P eera alli et al [12], [11 ask ed whether these rules can be tak en to be stationary; problem that has remained open. In this section, we resolv this question with ne ati answer First, we pro vide countere xample in which the optimal quantizer is not stationary Ne xt, we sho that using stationary quantizer can be suboptimal en in an asymptotic sense (i.e., as ). Illustrati counter example: be gin with simple ut

concrete demonstration of the suboptimality of sta- tionary designs. Consider problem in which and the conditional distrib utions tak the form 10 1999 10000 10000 and Suppose that the prior probabilities are 100 and 92 100 and that the cost for each sample is 100 If we restrict to binary quantizers (i.e., ), then there are only three possible quantizers: 1) Design A: As result, the corresponding distrib ution for is specified by and 2) Design B: The corresponding distrib ution for is gi en by 9999 10000 10000 and Method (0 08) (0 08) (0 08) (0 08) Cost 0567 0532 0800 0528 ABLE I.

Numerically computed costs for the three stationary designs and for the mix ed design 3) Design C: The corresponding distrib ution for is specified by 8001 10000 1999 10000 and No consider the three stationary strate gies, each of which uses only one fix ed design A, or C. or an gi en stationary quantization rule we ha classical centralized sequential problem, for which the optimal cost (achie ed by SPR T) can be computed using dynamic-programming procedure [13], [1]. Accordingly for each stationary strate gy we compute the optimal cost function for 10 points on the -axis by

performing 300 updates of Bellman equation [2]. In all cases, the dif ference in cost between the 299th and 300th updates is less than 10 Let and denote the optimal cost function for sequential tests using all s, all B s, and all C s, respecti ely When aluated at 08 these computations yield the numerical alues sho wn in able I. Finally we consider non-stationary rule obtained by applying design for only the first sample, and applying design for the remaining samples. Ag ain using Bellman equation, we find that the cost for this design is 0528 which is better than an of the

stationary strate gies. Asymptotic suboptimality of stationary designs: no pro that there is al ays range of prior proba- bilities for which stationary quantizer designs are sub- optimal. Our result stems from the follo wing observ a- tion: the form of the approximation (8) dictates that in order to achie small cost we need to choose quantizer for which the KL di er gences jj and jj are both as lar ge as possible. Ho we er due to the asymmetry of the KL di er gence, these maxima are not necessarily achie ed by single quantizer In such settings, it can be possible to construct non-stationary

quantizer with better cost than stationary design. Recall the notation, for an fix ed quantizer := jj and := jj Pr oposition 2. Let and be any two quantizer s. If the following inequalities hold and (12)
Page 5
then ther xists non-empty interval A; (0 suc that as ; if ; min ( log if A; ; if wher denotes the optimal cost of the stationary design based on the quantizer and ; denotes the optimal cost of sequential test that alternates between using and on odd and ven samples espectively Pr oof: According to Lemma 1, we ha log (13) No consider the sequential test that applies

quantizers and alternately to odd and en samples. Further more, this test considers tw samples at time. Let and denote the induced conditional probability distrib utions, jointly on the odd-e en pairs of quantized ariables. From the additi vity of the KL di er gence and assumption (12), there holds: jj (14a) jj (14b) Clearly the cost of the proposed sequential test is an upper bound for ; Furthermore, the ap between this upper bound and the true optimal cost is no more than Hence, as in the proof of Lemma 1, as the optimal cost ; can be written as log (15) From equations (13) and (15), simple

calculations yield the claim with )( )( )( )( Remarks: Let us return to the xample pro vided in the pre vious subsection. Note that the tw quantizers and satisfy assumption (12), since jj 4045 jj 45 and jj 4337 jj 5108 As result, there xist priors for which sequential test using stationary quantizer design (either or for all samples) is not optimal. Blockwise stationary designs: Restricting to the class of stationary designs, despite its possible suboptimality is computationally desirable. In this section, we consider designs that are bloc kwise stationary in the sense that there xists some

some natural number such that +1 +2 and so on. or each let denote the class of all blockwise stationary designs with period Thus, as increases, we ha hierarch of increasingly rich quantizer classes that will yield progessi ely better approximations to the optimal solution. It is well kno wn [10 that optimal quantizers when unr estricted are necessarily lik elihood-based threshold rules. Our preceding results imply that the thresholds need not be stationary (i.e., the threshold may dif fer from sample to sample). The follo wing theorem addresses partial con erse to this issue: Theor em 3.

Restricting to the class of bloc kwise sta- tionary and deterministic decision rules, ther xists an asymptotically optimal quantizer suc that ar lik elihood atio rules. The insight underlying this theorem is quasicon- ca vity result of (modulo the term) with respect to the ector of probabilities 1; induced by the quantizer proof and further detail can be found in technical report [6]. [1] K. J. Arro D. Blackwell, and M. A. Girshick. Bayes and min- imax solutions of sequential decision problems. Econometrica 17(3/4):213244, 1949. [2] D.P Bertsekas. Dynamic pr gr amming and stoc hastic contr ol

olume 1. Athena Scienti˛c, Belmont, MA, 1995. [3] C. Ho. eam decision problems and information structures. Pr oceedings of the IEEE 68:644654, 1980. [4] L. Lai. Sequential analysis: Some classical problems and ne challenges (with discussion). Statist. Sinica 11:303408, 2001. [5] Mei. Asymptotically optimal methods for sequential hang e- point detection PhD thesis, California Institute of echnology 2003. [6] X. Nguyen, M. J. ainwright, and M. I. Jordan. On optimal quantization rules in some sequential decision problems. ech- nical report, Dept of Statistics, UC Berk ele May 2006. [7] H.

Poor An Intr oduction to Signal Detection and Estimation Springer -V erlag, Ne ork, NY 1994. [8] A. N. Shiryaye Optimal stopping rules Springer -V erlag, 1978. [9] D. Sie gmund. Sequential analysis Springer -V erlag, 1985. [10] J. N. Tsitsiklis. On threshold rules in decentralized detection. In Pr oc. 25th IEEE Conf Decision Contr ol pages 232236, 1986. [11] eera alli. Sequential decision fusion: theory and applica- tions. ournal of the anklin Institute 336:301322, 1999. [12] eera alli, Basar and H. Poor Decentralized sequen- tial detection with fusion center performing the sequential test.

IEEE ans. Info. Theory 39(2):433442, 1993. [13] A. ald and J. olfo witz. Optimum character of the sequential probability ratio test. Annals of Statistics 19:326339, 1948.