Toniann Pitassi University of Toronto 2Party Communication Complexity Yao 2party communication each party has a dataset Goal is to compute a function fD A D B m 1 m 2 ID: 703795
Download Presentation The PPT/PDF document "Communication Complexity, Information Co..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
Communication Complexity, Information Complexity and Applications to Privacy
Toniann
Pitassi
University of TorontoSlide2
2-Party Communication Complexity
[Yao]
2-party communication: each party has a dataset. Goal is to compute a function f(DA,DB)
m
1
m
2
m
3
m
k-1
m
k
D
A
x
1
x
2
x
n
D
B
y
1
y
2ym
f(DA,DB)
f(D
A
,DB)
Communication complexity
of a protocol for f is the number of bits exchanged between A and B.
In this talk, all protocols are assumed to be randomized.Slide3
Deterministic Protocols
A deterministic protocol
Π specifies:Function of board contents:if the protocol is overif YES, the outputif NO, which player writes nextFunction of board contents and input available to player P:what P writesCost of
Π = max number of bits written on the board over all inputsSlide4
Randomized Protocols
In a randomized protocol
Π, what player P writes is also a function of the (private and/or public) random string available to PProtocol allowed to err with probability ε over choice of random stringsThe cost of
Π
= max number of bits written on the board, over inputs and random stringsSlide5
Communication Complexity
Focus on randomized communication complexity:
CC(F,ε) = the communication cost of computing F with error ε.A distributional flavor of randomized communication complexity: CC(F,
μ
,ε)
= the communication cost of computing F with error
ε
with respect to μ
.Yao’s minimax: CC(F,ε)=max
μ
CC(F,μ
,ε).
5Slide6
Stunning variety of applications of CC Lower Bounds
Lower Bounds for Streaming Algorithms
Data Structure Lower BoundsProof Complexity Lower BoundsGame TheoryCircuit Complexity Lower BoundsQuantum Computation
Differential Privacy
……Slide7
2-Party Information Complexity
2-party communication:
each party has a dataset. Goal is to compute a function f(DA,DB)
m
1
m
2
m
3
m
k-1
m
k
D
A
x
1
x
2
x
n
D
B
y
1
y
2ymf(D
A,DB)
f(D
A
,DB)
Information complexity
of a protocol for f is the amount of information the players reveal to each other / or to an eavesdropper (Eve)Slide8
Information Complexity
[Chakrabarti,Shi,Wirth,Yao
‘01], [Bar-Yossef,Jayram,Kumar,Sivakumar ‘04]Entropy:
H(X) = Σ
x p(x) log (1/p(x)
Conditional entropy: H(X|Y) = Σy H(X|Y=y) p(Y=y)
Mutual Information:
I(X;Y) = H(X) - H(X|Y)External IC:
information about XY revealed to Eve ICext
(
π,
μ) = I(XY;π)
IC
ext (f,μ,
ε) = max
π IC
ext(π,μ
)Internal IC:
information revealed to Alice and Bob
ICint
(π,μ) = I(X;π|Y) + I(Y;π|X) ICint
(f,μ,ε) = maxπ ICint (π,
μ)Slide9
Why study information complexity?
Intrinsically interesting quantity
Related to longstanding questions in complexity theory (direct sum conjecture)Very useful when studying privacy, and quantum computationSlide10
Simple Facts about Information Complexity
External information cost is greater than internal:
ICext(π,μ) ≥ IC
int (
π,
μ)
ICext
(π) = I(XY;
π) = I(X;π
) + I(Y;π
| X) ≥ I(X;π
|Y) + I(Y; π | X) =
ICint (
π)Information complexity lower bounds imply Communication Complexity lower bounds: CC(f,
μ,
ε) ≥
ICext(f,μ
,ε
) ≥
ICint (f,
μ,ε) Slide11
Do CC Lower Bounds imply IC Lower Bounds? (i.e., CC=IC?)
For constant-round protocols, IC and CC are basically equal [CSWY, JRS]
Open for general protocols. Significant step for general case by [BBCR]Slide12
Compressing Interactive Communication
[Barak,Braverman,Chen,Rao
]Theorem 1For any distribution μ, any C-bit protocol of internal IC I can be simulated by a new protocol using O(√(CI) logC
) bits.
Theorem 2For any product distribution
μ, any C-bit protocol of internal IC I can be simulated by a protocol using O(I logC) bits.Slide13
Connection to the Direct Sum Problem
Does it take m times the amount of resources to solve m instances?
Direct Sum Question for CC: CC(fm) ≥ m CC(f) for every f and every distribution?
- Each copy should have error ε
For search problems, the direct sum problem is equivalent to separating NC1 from P !Slide14
Connection to the Direct Sum Problem, 2
The direct sum property holds for information complexity: Lemma [Direct Sum for IC]: IC(fm) ≥ m IC(f)Best general direct sum theorem known for cc:
Theorem [Barak,Braverman,Chen,Rao
]: CC(fm)
≥
√m CC(f) ignoring polylog
factorsThe direct sum property for cc is equivalent to IC=CC! Theorem [
Braverman,Rao
]: IC(f,
μ,ε) = limn
∞ CC(Fn
, μn,ε
)/nSlide15
Methods for Proving CC and IC Lower Bounds
Jain and
Klauck initiated the formal study of CC lower bound methods: all formalizable as solutions to (different) LPs Discrepancy Method, Smooth Discrepancy MethodRectangle Bound, Smooth Rectangle BoundPartition BoundSlide16
The Partition Bound
[Jain, Klauck
]Min Σz,R wz,R
∀ (
x,y) Σ R, (
x,y) ϵ R
wf
(x,y),R
≥ 1-ε
∀ (
x,y) ΣR, (
x,y) in R Σz
wz,R = 1
∀ z,R w z,R
≥ 0Slide17
Relationships
The Partition bound is greater than or equal to all known CC lower bounds methods, including:
DiscrepancyGeneralized DiscrepancyRectangleSmooth RectangleSlide18
[KLLR] define the relaxed partition bound. The relaxed partition bound is greater than or equal to all known CC lower bound methods (except the partition bound).
They show that the relaxed Partition bound is equivalent to designing a zero-communication protocol with error exp(-I)
Given a protocol for f with ICint = I, they construct a zero-communication protocol st
(i
) non-abort probability is exp(-I), and (ii) if it does not abort, it computes f correctly whp
All known CC Lower Bound Methods Imply IC Lower Bounds!
[
Kerenidis,
Laplante, Lerays,
Roland Xiao ‘12] Slide19
Applications of Information Complexity
Differential Privacy
PARSlide20
Applications of Information Complexity
Differential Privacy
PARSlide21
Differential Privacy: The Basic Scenario
[Dwork
, McSherry, Nissim, Smith 06] Database with rows x1..
xn
Each row corresponds to an individual in the databaseColumns correspond to fields, such as “name”, “zip code”; some fields contain sensitive information.
Goal: Compute and release information about a sensitive database without revealing information about any individual
Sanitizer
Output
DataSlide22
Differential Privacy
[
Dwork,McSherry,Nissim,Smith 2006]
Y
Pr [response]
ratio bounded
Q = space of queries; Y = output space; X = row space
Mechanism M:
X
n
x Q
Y
is
-differentially private if:
for all q in Q, for all
adjacent x, x’ in
X
n
, the distributions
M(x,q), M(x’,q) are similar: ∀ y in Y, q in Q: e -𝜀 ≤ Pr[M(x,q) =y] ≤
eε Pr[M(x’,q)=y]Note: Randomness is crucialSlide23
23
Achieving DP: Add
Laplacian Noise f = maxD,D’ |f(D) – f(D’)|
0
b
2b
3b
4b
5b
-b
-
2b
-
3b
-
4b
Theorem:
To achieve
-differential privacy, add symmetric noise [Lap(b)] with b =
f/
.P(y)
∽ exp(-|y - q(x)|/b)=exp( - | y – q(x’)| / f )Pr [M(x, q) = y]
Pr [(M(x’, q) = y]exp( - | y – q(x)| / f )≤exp().Slide24
Differentially Private Communication Complexity: A Distributed View
Andrews,Mironov,P,Reingold,Talwar,Vadhan
Goal:
compute a joint function while maintaining privacy for any individual, with respect to both the outside world and the other database owners.
Multiple databases, each with private data.
D1
D2
D3
D4
D5
F(D1,D2,..,D5)Slide25
2-Party Differentially Private CC
2-party (& multiparty) DP privacy
: each party has a dataset; want to compute a joint function f(DA,DB)
m
1
m
2
m
3
m
k-1
m
k
D
A
x
1
x
2
x
n
D
B
y
1
y
2ymZA f(D
A,DB)
Z
B
f(DA,DB)
A’s view should
be a
differentially private
function of
D
B
(even if A deviates from protocol), and vice-versaSlide26
Two-Party Differential Privacy
Let P(
x,y) be a 2-party protocol. P is ε-DP if: (1) for all y, for every pair x, x’ that are neighbors
, and for every transcript
π,
Pr[P(x,y) = π ]
≤
exp(ε
) Pr[P(x’,y) = π
](2) symmetrically, for all x, for every pair of
neighbors y,y
’ and for every transcript π
Pr[P(x,y)=
π ] ≤ exp(ε) Pr[P(
x,y’) = π
]
Privacy and accuracy are the important parametersSlide27
Examples
Ones(
x,y) = the number of ones in xy Ones(00001111,10101010) = 8. CC(Ones) =
logn.
There is a low error DP protocol.
2. Hamming Distance HD(x,y)
= the number of positions
i where xi
≠ yi.
HD(00001111, 10101010) = 4
CC(HD)=n. No low error DP protocol
Is this a coincidence? Is there a connection between low cc and low-error DP protocols?Slide28
Information Cost and DP Protocols
[McGregor, Mironov
, P,Reingold,Talwar,Vadhan] Lemma. If π has
ε-DP, then for every distribution
μ on XY,
IC(π,μ,ε
)
≤ 3εn
Proof sketch: For every z,z’, by ε
-DP,
exp(-2εn) ≤ Pr[
π(z) = π]/Pr[π
(z’)=π] ≤ exp(2
εn) I(π(Z); X) = H(
π(Z)) – H(π
(Z) | Z) = Exp{z,
π} log[ Pr[π(Z)=
π | Z=z] / Pr[π
(Z)=
π] ] ≤ 2 (log ε
) εnDP Partition Theorem. Let P be an ε-DP protocol for a partial function with error at most γ. Then log prtγ(f) ≤ 3
ε n Slide29
Lower
Bound:Hamming Distance
[McGregor, Mironov, P,Reingold,Talwar,Vadhan] Gap Hamming:
GHD(x,y
) = 1 if HD(x,y) > n/2 +
√n 0 if HD(x,y) < n/2 – √
n
Theorem. Any
ε-DP protocol for Hamming distance must incur an additive error Ω(√n).
Note: This lower bound is tight.
Proof sketch:
[Chakrabarti-Regev 2012] prove: CC(GHD,μ
,1/3) = Ω (n).
Proof shows GHD has a smooth rectangle bound of 2Ω(n). By Jain-Klauck
, this implies that the partition bound for GHD is at least as large. Thus proof follows by DP Partition Theorem.
Slide30
Implications of Lower bound for Hamming Distance
1.
Separation between ε-DP protocols and computational ε-DP protocols [MPRV]: Hamming distance has an O(1) error computational
ε-DP protocol, but any
ε-DP protocol has error √
n. We also exhibit another function with linear separation. (Any ε
-DP protocol has error
Ωn)
2. Pan Privacy: Our lower bound for Hamming Distance implies lower bounds for pan-private streaming algorithmsSlide31
Pan-Private Streaming Model
[Dwork,P,Rothblum
, Naor,Yekhanin]Data is a stream of items; each item belongs to a user. Sanitizer sees each item and updates internal state. Generates output at end of the stream (single pass
).
state
Pan-Privacy:
For every two
adjacent streams
, at any
single point in time
, the
internal state
(and final output) are differentially private. Slide32
What statistics have pan-private algorithms?
We give pan-private streaming algorithms for:
Stream density / number of distinct elementst-cropped mean: mean, over users, of min(t, #appearances)Fraction of users appearing exactly k times Fraction of users appearing exactly 0 times modulo k Fraction of heavy-hitters, users appearing at least k timesSlide33
Pan Privacy lower bounds via
ε-DP lower bounds
Lower Bounds for ε-DP communication protocols imply pan privacy lower bounds for density estimation (via Hamming distance lower bound).Lower bounds also hold for multi-pass pan-private modelsAnalogy: 2-party communication complexity lower bounds imply lower bounds in streaming model.Slide34
DP Protocols and Compression
So back to Ones(
x,y) and HD(x,y)...is DP the same as compressible?Theorem. [BBCR] (Low
Icost
implies compression) For every product distribution
μ, and protocol P, there exists a protocol Q (β-approximating P) with comm. complexity ∼
Icost
μ(P) x polylog(CC(P))/
βCorollary.
(DP protocols can be compressed)
Let P be an ε-DP protocol P. Then there exists a protocol Q of cost 3
εn polylog(CC(P))/β and error
β.
DP almost implies low cc, except for this annoying polylog(CC(P)) factorMoreover, the low cc protocol can often be made DP (if the number of rounds is bounded.)Slide35
Differential Privacy and
Compression
We have seen that DP protocols have low information costBy BBCR this implies they can be compressed (and thus have low comm complexity)What about the other direction? Can functions with low cc be made DP?
Yes! (with some caveats..the error is proportional not only to the cc, but also the number of rounds.)
Proof uses the exponential mechanism [MT]
Slide36
Applications of Information Complexity
Differential Privacy
PARSlide37
37
Approximate Privacy in Mechanism Design
Traditional goal of mechanism design: Incent agents to reveal private information that is needed to compute optimal results.Complementary, newly important goal: Enable agents not to reveal private information that is not
needed to compute optimal results.
Example (Naor
-Pinkas-Sumner, EC ’99): It’s undesirable for the auctioneer to learn the winning bid in a 2
nd–price
Vickrey auction.Slide38
38
Perfect Privacy
[Kushilevitz ’92]Protocol P for f is perfectly private iff for all x,x’,y,y’ f(
x,y)=f(
x’,y’) R(
x,y)=R(x’,y’)
f is perfectly privately computable
iff M(f) has no forbidden
submatrix
f(x
1
, x
2
) = f
(
x’
1, x2) = f(x’1
, x’2) = a, but f
(x1, x’
2) ≠ a
x1 x’1X2
X’2Slide39
39
Example 1: Millionaires’
Problem(not perfectly privately computable)0123
0 1 2 3
millionaire 1
millionaire 2
A(f)
f(x
1
, x
2
) = 1 if x
1
≥ x
2
; else f(x
1
, x
2
) = 2Slide40
40
Example 2:
Vickrey Auction[Brandt, Sandholm]
2, 1
winner
price
2, 0
1, 0
1, 1
1, 2
2, 2
1, 3
0
1
2
3
bidder 1
bidder 2
0 1 2 3
R
I
(2, 0)
The ascending-price, English auction protocol is the unique perfectly private protocolHowever the communication cost is exponential !!Slide41
41
Worst-case PAR
[Feigenbaum, Jaggard,Schapira ‘10] Worst-case privacy approximation ratio of a protocol π
for f:
PAR(f,π
) = max x,y | P(x,y)|/ |R(
x,y)|,
P(
x,y): set of all pairs (x’,y’) st f(
x,y
)=f’(x’,y’)
R(x,y): rectangle containing (x,y) induced by π
Worst-case PAR of f:
PAR(f) = min π PAR(f,π
) Slide42
42
Average-case PAR
[Feigenbaum, Jaggard, Schapira ‘10]
(1) Average-case PAR of
π:
AvgPAR1(f,π) = log E
(
x,y)
|P(x,y)|/|R(x,y
)|
AvgPAR1(f) = minπ
AvgPAR(f,π)
(2) Alternative definition:
AvgPAR2(f,π) = I(XY;
π | f)
= E(
x,y) log |P(x,y)/|R(
x,y)|
AvgPAR2(f) = min
π AvgPAR2(f,π
)1 is log of Expectation, 2 is Expectation of log.For boolean functions, AvgPAR2(f) is basically the same as Icost(f) (differs by at most 1).
Slide43
43
New Results
[Ada,Chattopadhyay,Cook,Fontes,P ‘12] Using the fact that AvgPAR1 ≥ AvgPAR2, together with known IC lower bounds:
Theorem
AvgPAR2 of set intersection is Ω
(n)(2) We prove strong tradeoffs for both worst-case PAR and
avgPAR
for Vickrey auctions.
(3) Using compression [BBCR], it follows that any deterministic, low AvgPAR1 protocols can be compressed. Thus binary search protocol for millionaires implies a polylogn randomized protocol.
Slide44
Important Open Questions
IC=CC?
IC in the multiparty NOF settingIC lower bounds for search problems Very important for proof complexity and circuit complexityOther applications of IC Data structures? Game Theory?Slide45
Thanks!