/
Communication Complexity, Information Complexity and Applications to Privacy Communication Complexity, Information Complexity and Applications to Privacy

Communication Complexity, Information Complexity and Applications to Privacy - PowerPoint Presentation

calandra-battersby
calandra-battersby . @calandra-battersby
Follow
370 views
Uploaded On 2018-10-30

Communication Complexity, Information Complexity and Applications to Privacy - PPT Presentation

Toniann Pitassi University of Toronto 2Party Communication Complexity Yao 2party communication each party has a dataset Goal is to compute a function fD A D B m 1 m 2 ID: 703795

complexity protocol privacy information protocol complexity information privacy bounds bound communication private error protocols party par exp theorem partition

Share:

Link:

Embed:

Download Presentation from below link

Download Presentation The PPT/PDF document "Communication Complexity, Information Co..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.


Presentation Transcript

Slide1

Communication Complexity, Information Complexity and Applications to Privacy

Toniann

Pitassi

University of TorontoSlide2

2-Party Communication Complexity

[Yao]

2-party communication: each party has a dataset. Goal is to compute a function f(DA,DB)

m

1

m

2

m

3

m

k-1

m

k

D

A

x

1

x

2

x

n

D

B

y

1

y

2ym

f(DA,DB)

f(D

A

,DB)

Communication complexity

of a protocol for f is the number of bits exchanged between A and B.

In this talk, all protocols are assumed to be randomized.Slide3

Deterministic Protocols

A deterministic protocol

Π specifies:Function of board contents:if the protocol is overif YES, the outputif NO, which player writes nextFunction of board contents and input available to player P:what P writesCost of

Π = max number of bits written on the board over all inputsSlide4

Randomized Protocols

In a randomized protocol

Π, what player P writes is also a function of the (private and/or public) random string available to PProtocol allowed to err with probability ε over choice of random stringsThe cost of

Π

= max number of bits written on the board, over inputs and random stringsSlide5

Communication Complexity

Focus on randomized communication complexity:

CC(F,ε) = the communication cost of computing F with error ε.A distributional flavor of randomized communication complexity: CC(F,

μ

,ε)

= the communication cost of computing F with error

ε

with respect to μ

.Yao’s minimax: CC(F,ε)=max

μ

CC(F,μ

,ε).

5Slide6

Stunning variety of applications of CC Lower Bounds

Lower Bounds for Streaming Algorithms

Data Structure Lower BoundsProof Complexity Lower BoundsGame TheoryCircuit Complexity Lower BoundsQuantum Computation

Differential Privacy

……Slide7

2-Party Information Complexity

2-party communication:

each party has a dataset. Goal is to compute a function f(DA,DB)

m

1

m

2

m

3

m

k-1

m

k

D

A

x

1

x

2

x

n

D

B

y

1

y

2ymf(D

A,DB)

f(D

A

,DB)

Information complexity

of a protocol for f is the amount of information the players reveal to each other / or to an eavesdropper (Eve)Slide8

Information Complexity

[Chakrabarti,Shi,Wirth,Yao

‘01], [Bar-Yossef,Jayram,Kumar,Sivakumar ‘04]Entropy:

H(X) = Σ

x p(x) log (1/p(x)

Conditional entropy: H(X|Y) = Σy H(X|Y=y) p(Y=y)

Mutual Information:

I(X;Y) = H(X) - H(X|Y)External IC:

information about XY revealed to Eve ICext

(

π,

μ) = I(XY;π)

IC

ext (f,μ,

ε) = max

π IC

ext(π,μ

)Internal IC:

information revealed to Alice and Bob

ICint

(π,μ) = I(X;π|Y) + I(Y;π|X) ICint

(f,μ,ε) = maxπ ICint (π,

μ)Slide9

Why study information complexity?

Intrinsically interesting quantity

Related to longstanding questions in complexity theory (direct sum conjecture)Very useful when studying privacy, and quantum computationSlide10

Simple Facts about Information Complexity

External information cost is greater than internal:

ICext(π,μ) ≥ IC

int (

π,

μ)

ICext

(π) = I(XY;

π) = I(X;π

) + I(Y;π

| X) ≥ I(X;π

|Y) + I(Y; π | X) =

ICint (

π)Information complexity lower bounds imply Communication Complexity lower bounds: CC(f,

μ,

ε) ≥

ICext(f,μ

) ≥

ICint (f,

μ,ε) Slide11

Do CC Lower Bounds imply IC Lower Bounds? (i.e., CC=IC?)

For constant-round protocols, IC and CC are basically equal [CSWY, JRS]

Open for general protocols. Significant step for general case by [BBCR]Slide12

Compressing Interactive Communication

[Barak,Braverman,Chen,Rao

]Theorem 1For any distribution μ, any C-bit protocol of internal IC I can be simulated by a new protocol using O(√(CI) logC

) bits.

Theorem 2For any product distribution

μ, any C-bit protocol of internal IC I can be simulated by a protocol using O(I logC) bits.Slide13

Connection to the Direct Sum Problem

Does it take m times the amount of resources to solve m instances?

Direct Sum Question for CC: CC(fm) ≥ m CC(f) for every f and every distribution?

- Each copy should have error ε

For search problems, the direct sum problem is equivalent to separating NC1 from P !Slide14

Connection to the Direct Sum Problem, 2

The direct sum property holds for information complexity: Lemma [Direct Sum for IC]: IC(fm) ≥ m IC(f)Best general direct sum theorem known for cc:

Theorem [Barak,Braverman,Chen,Rao

]: CC(fm)

√m CC(f) ignoring polylog

factorsThe direct sum property for cc is equivalent to IC=CC! Theorem [

Braverman,Rao

]: IC(f,

μ,ε) = limn 

∞ CC(Fn

, μn,ε

)/nSlide15

Methods for Proving CC and IC Lower Bounds

Jain and

Klauck initiated the formal study of CC lower bound methods: all formalizable as solutions to (different) LPs Discrepancy Method, Smooth Discrepancy MethodRectangle Bound, Smooth Rectangle BoundPartition BoundSlide16

The Partition Bound

[Jain, Klauck

]Min Σz,R wz,R

∀ (

x,y) Σ R, (

x,y) ϵ R

wf

(x,y),R

≥ 1-ε

∀ (

x,y) ΣR, (

x,y) in R Σz

wz,R = 1

∀ z,R w z,R

≥ 0Slide17

Relationships

The Partition bound is greater than or equal to all known CC lower bounds methods, including:

DiscrepancyGeneralized DiscrepancyRectangleSmooth RectangleSlide18

[KLLR] define the relaxed partition bound. The relaxed partition bound is greater than or equal to all known CC lower bound methods (except the partition bound).

They show that the relaxed Partition bound is equivalent to designing a zero-communication protocol with error exp(-I)

Given a protocol for f with ICint = I, they construct a zero-communication protocol st

(i

) non-abort probability is exp(-I), and (ii) if it does not abort, it computes f correctly whp

All known CC Lower Bound Methods Imply IC Lower Bounds!

[

Kerenidis,

Laplante, Lerays,

Roland Xiao ‘12] Slide19

Applications of Information Complexity

Differential Privacy

PARSlide20

Applications of Information Complexity

Differential Privacy

PARSlide21

Differential Privacy: The Basic Scenario

[Dwork

, McSherry, Nissim, Smith 06] Database with rows x1..

xn

Each row corresponds to an individual in the databaseColumns correspond to fields, such as “name”, “zip code”; some fields contain sensitive information.

Goal: Compute and release information about a sensitive database without revealing information about any individual

Sanitizer

Output

DataSlide22

Differential Privacy

[

Dwork,McSherry,Nissim,Smith 2006]

Y

Pr [response]

ratio bounded

Q = space of queries; Y = output space; X = row space

Mechanism M:

X

n

x Q

 Y

is

-differentially private if:

for all q in Q, for all

adjacent x, x’ in

X

n

, the distributions

M(x,q), M(x’,q) are similar: ∀ y in Y, q in Q: e -𝜀 ≤ Pr[M(x,q) =y] ≤

eε Pr[M(x’,q)=y]Note: Randomness is crucialSlide23

23

Achieving DP: Add

Laplacian Noise f = maxD,D’ |f(D) – f(D’)|

0

b

2b

3b

4b

5b

-b

-

2b

-

3b

-

4b

Theorem:

To achieve

-differential privacy, add symmetric noise [Lap(b)] with b =

f/

.P(y)

∽ exp(-|y - q(x)|/b)=exp( - | y – q(x’)| / f )Pr [M(x, q) = y]

Pr [(M(x’, q) = y]exp( - | y – q(x)| / f )≤exp().Slide24

Differentially Private Communication Complexity: A Distributed View

Andrews,Mironov,P,Reingold,Talwar,Vadhan

Goal:

compute a joint function while maintaining privacy for any individual, with respect to both the outside world and the other database owners.

Multiple databases, each with private data.

D1

D2

D3

D4

D5

F(D1,D2,..,D5)Slide25

2-Party Differentially Private CC

2-party (& multiparty) DP privacy

: each party has a dataset; want to compute a joint function f(DA,DB)

m

1

m

2

m

3

m

k-1

m

k

D

A

x

1

x

2

x

n

D

B

y

1

y

2ymZA  f(D

A,DB)

Z

B

 f(DA,DB)

A’s view should

be a

differentially private

function of

D

B

(even if A deviates from protocol), and vice-versaSlide26

Two-Party Differential Privacy

Let P(

x,y) be a 2-party protocol. P is ε-DP if: (1) for all y, for every pair x, x’ that are neighbors

, and for every transcript

π,

Pr[P(x,y) = π ]

exp(ε

) Pr[P(x’,y) = π

](2) symmetrically, for all x, for every pair of

neighbors y,y

’ and for every transcript π

Pr[P(x,y)=

π ] ≤ exp(ε) Pr[P(

x,y’) = π

]

Privacy and accuracy are the important parametersSlide27

Examples

Ones(

x,y) = the number of ones in xy Ones(00001111,10101010) = 8. CC(Ones) =

logn.

There is a low error DP protocol.

2. Hamming Distance HD(x,y)

= the number of positions

i where xi

≠ yi.

HD(00001111, 10101010) = 4

CC(HD)=n. No low error DP protocol

Is this a coincidence? Is there a connection between low cc and low-error DP protocols?Slide28

Information Cost and DP Protocols

[McGregor, Mironov

, P,Reingold,Talwar,Vadhan] Lemma. If π has

ε-DP, then for every distribution

μ on XY,

IC(π,μ,ε

)

≤ 3εn

Proof sketch: For every z,z’, by ε

-DP,

exp(-2εn) ≤ Pr[

π(z) = π]/Pr[π

(z’)=π] ≤ exp(2

εn) I(π(Z); X) = H(

π(Z)) – H(π

(Z) | Z) = Exp{z,

π} log[ Pr[π(Z)=

π | Z=z] / Pr[π

(Z)=

π] ] ≤ 2 (log ε

) εnDP Partition Theorem. Let P be an ε-DP protocol for a partial function with error at most γ. Then log prtγ(f) ≤ 3

ε n Slide29

Lower

Bound:Hamming Distance

[McGregor, Mironov, P,Reingold,Talwar,Vadhan] Gap Hamming:

GHD(x,y

) = 1 if HD(x,y) > n/2 +

√n 0 if HD(x,y) < n/2 – √

n

Theorem. Any

ε-DP protocol for Hamming distance must incur an additive error Ω(√n).

Note: This lower bound is tight.

Proof sketch:

[Chakrabarti-Regev 2012] prove: CC(GHD,μ

,1/3) = Ω (n).

Proof shows GHD has a smooth rectangle bound of 2Ω(n). By Jain-Klauck

, this implies that the partition bound for GHD is at least as large. Thus proof follows by DP Partition Theorem.

Slide30

Implications of Lower bound for Hamming Distance

1.

Separation between ε-DP protocols and computational ε-DP protocols [MPRV]: Hamming distance has an O(1) error computational

ε-DP protocol, but any

ε-DP protocol has error √

n. We also exhibit another function with linear separation. (Any ε

-DP protocol has error

Ωn)

2. Pan Privacy: Our lower bound for Hamming Distance implies lower bounds for pan-private streaming algorithmsSlide31

Pan-Private Streaming Model

[Dwork,P,Rothblum

, Naor,Yekhanin]Data is a stream of items; each item belongs to a user. Sanitizer sees each item and updates internal state. Generates output at end of the stream (single pass

).

state

Pan-Privacy:

For every two

adjacent streams

, at any

single point in time

, the

internal state

(and final output) are differentially private. Slide32

What statistics have pan-private algorithms?

We give pan-private streaming algorithms for:

Stream density / number of distinct elementst-cropped mean: mean, over users, of min(t, #appearances)Fraction of users appearing exactly k times Fraction of users appearing exactly 0 times modulo k Fraction of heavy-hitters, users appearing at least k timesSlide33

Pan Privacy lower bounds via

ε-DP lower bounds

Lower Bounds for ε-DP communication protocols imply pan privacy lower bounds for density estimation (via Hamming distance lower bound).Lower bounds also hold for multi-pass pan-private modelsAnalogy: 2-party communication complexity lower bounds imply lower bounds in streaming model.Slide34

DP Protocols and Compression

So back to Ones(

x,y) and HD(x,y)...is DP the same as compressible?Theorem. [BBCR] (Low

Icost

implies compression) For every product distribution

μ, and protocol P, there exists a protocol Q (β-approximating P) with comm. complexity ∼

Icost

μ(P) x polylog(CC(P))/

βCorollary.

(DP protocols can be compressed)

Let P be an ε-DP protocol P. Then there exists a protocol Q of cost 3

εn polylog(CC(P))/β and error

β.

DP almost implies low cc, except for this annoying polylog(CC(P)) factorMoreover, the low cc protocol can often be made DP (if the number of rounds is bounded.)Slide35

Differential Privacy and

Compression

We have seen that DP protocols have low information costBy BBCR this implies they can be compressed (and thus have low comm complexity)What about the other direction? Can functions with low cc be made DP?

Yes! (with some caveats..the error is proportional not only to the cc, but also the number of rounds.)

Proof uses the exponential mechanism [MT]

Slide36

Applications of Information Complexity

Differential Privacy

PARSlide37

37

Approximate Privacy in Mechanism Design

Traditional goal of mechanism design: Incent agents to reveal private information that is needed to compute optimal results.Complementary, newly important goal: Enable agents not to reveal private information that is not

needed to compute optimal results.

Example (Naor

-Pinkas-Sumner, EC ’99): It’s undesirable for the auctioneer to learn the winning bid in a 2

nd–price

Vickrey auction.Slide38

38

Perfect Privacy

[Kushilevitz ’92]Protocol P for f is perfectly private iff for all x,x’,y,y’ f(

x,y)=f(

x’,y’)  R(

x,y)=R(x’,y’)

f is perfectly privately computable

iff M(f) has no forbidden

submatrix

f(x

1

, x

2

) = f

(

x’

1, x2) = f(x’1

, x’2) = a, but f

(x1, x’

2) ≠ a

x1 x’1X2

X’2Slide39

39

Example 1: Millionaires’

Problem(not perfectly privately computable)0123

0 1 2 3

millionaire 1

millionaire 2

A(f)

f(x

1

, x

2

) = 1 if x

1

≥ x

2

; else f(x

1

, x

2

) = 2Slide40

40

Example 2:

Vickrey Auction[Brandt, Sandholm]

2, 1

winner

price

2, 0

1, 0

1, 1

1, 2

2, 2

1, 3

0

1

2

3

bidder 1

bidder 2

0 1 2 3

R

I

(2, 0)

The ascending-price, English auction protocol is the unique perfectly private protocolHowever the communication cost is exponential !!Slide41

41

Worst-case PAR

[Feigenbaum, Jaggard,Schapira ‘10] Worst-case privacy approximation ratio of a protocol π

for f:

PAR(f,π

) = max x,y | P(x,y)|/ |R(

x,y)|,

P(

x,y): set of all pairs (x’,y’) st f(

x,y

)=f’(x’,y’)

R(x,y): rectangle containing (x,y) induced by π

Worst-case PAR of f:

PAR(f) = min π PAR(f,π

) Slide42

42

Average-case PAR

[Feigenbaum, Jaggard, Schapira ‘10]

(1) Average-case PAR of

π:

AvgPAR1(f,π) = log E

(

x,y)

|P(x,y)|/|R(x,y

)|

AvgPAR1(f) = minπ

AvgPAR(f,π)

(2) Alternative definition:

AvgPAR2(f,π) = I(XY;

π | f)

= E(

x,y) log |P(x,y)/|R(

x,y)|

AvgPAR2(f) = min

π AvgPAR2(f,π

)1 is log of Expectation, 2 is Expectation of log.For boolean functions, AvgPAR2(f) is basically the same as Icost(f) (differs by at most 1).

Slide43

43

New Results

[Ada,Chattopadhyay,Cook,Fontes,P ‘12] Using the fact that AvgPAR1 ≥ AvgPAR2, together with known IC lower bounds:

Theorem

AvgPAR2 of set intersection is Ω

(n)(2) We prove strong tradeoffs for both worst-case PAR and

avgPAR

for Vickrey auctions.

(3) Using compression [BBCR], it follows that any deterministic, low AvgPAR1 protocols can be compressed. Thus binary search protocol for millionaires implies a polylogn randomized protocol.

Slide44

Important Open Questions

IC=CC?

IC in the multiparty NOF settingIC lower bounds for search problems Very important for proof complexity and circuit complexityOther applications of IC Data structures? Game Theory?Slide45

Thanks!