Lijie Chen Ryan Williams Context The Algorithmic Method for Proving Circuit Lower Bounds Proving limitations on nonuniform circuits is extremely hard Prior approaches restrictions ID: 1014813
Download Presentation The PPT/PDF document "Stronger Connections Between Circuit Ana..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
1. Stronger Connections Between Circuit Analysis and Circuit Lower Bounds, via PCPs of ProximityLijie ChenRyan Williams
2. Context: The Algorithmic Method for Proving Circuit Lower BoundsProving limitations on non-uniform circuits is extremely hard.Prior approaches (restrictions, polynomial approximations, etc.) face barriers (Relativization, Algebrization, Natural Proofs).Algorithmic MethodNon-trivial circuit-analysis algorithm Circuit Lower Bounds.Breakthroughs where previous approaches failed (NEXP ACC0).Believed to be possible for strong circuits (even ).
3. Context: A Frontier of Circuit Complexity, Depth-2 Threshold CircuitsTHR gates : , . MAJ gates : when ’s and are bounded by poly(n). THRTHR THRTHRTHRTHRWe can also define
4. Context: A Frontier of Circuit Complexity, Depth-2 Threshold CircuitsExponential Lower Bounds are known for [Hajnal-Maass-Pudlák-Szegedy-Turán’93] [Nisan’94] [Forster-Krause-Lokam-Mubarakzjanov-Schmitt-Simon’01] Frontier Open Question: Is NEXP ?Potential Approaches in this talk. NEXP Non-deterministic Exponential Time.
5. Motivation: Apply the Algorithmic Method to THR of THR?-SAT ? x s.t. -CAPP Estimate quantity,with additive error What Circuit-Analysis Tasks?Non-trivial Circuit-Analysis AlgorithmsCircuit Lower Bounds Derandomization!!: constant or inverse polynomial time?
6. Most previous work on the algorithmic method exploits SAT algorithms.ProblemSAT of THR of THR is probably very hard.A special case is MAX--SAT, for which no non-trivial ( time) algorithm is known for and clauses.Considered to be a barrier for the Algorithmic Approach. THRTHR THRTHRTHRTHRMAX--SAT MAJ Motivation: Apply the Algorithmic Method to THR of THR?
7. SAT of THR of THR : probably very hardBut derandomization is widely believed to be possible.From Derandomization (CAPP) Circuit Lower BoundsFor a circuit class , -time CAPP for () [Williams’13/14, Santhanam Williams’14, Ben-Sasson Viola’14]-time CAPP for () can’t be -approximated by [R. Chen Oliveira Santhanam’18] -time CAPP for () [Murray Williams’18] -time CAPP for () can’t be -approximated by [L. Chen’19] NQPNon-deterministic Quasi-Polynomial Time. Motivation: Apply the Algorithmic Method to THR of THR?
8. Back to THR of THRSAT of THR of THR : probably very hardTo show , we need to derandomize , which could be harder. Our result 1 It suffices to derandomize . Our result 2Surprisingly, it indeed only suffices to derandomize or !
9. General Result: A Stronger Connection Between Circuit-Analysis Algorithms and Circuit Lower BoundsFor a circuit class :-time CAPP for , , or .-time CAPP for , , or . Why the constant “2”?Short answer: A PCP system needs to make at least queries. Long answer: See the paper
10. Tighter Connections for Algorithms/Lower Bounds for THR of THRLuckily, the “2” doesn’t matter for -time CAPP algorithm for . -time CAPP algorithm for .: depth-d, poly-size, linear threshold circuits
11. Let Us Make Our Life Even EasierTHRTHRTHRTHRMAJMAJMAJMAJPoly-size and are equivalent for Non-Trivial ( time) CAPP Algorithms when Proved by new structure lemmas for
12. Let Us Make Our Life Even EasierTHRTHRTHRTHRTHRMAJMAJMAJPoly-size and are equivalent for Non-Trivial ( time) CAPP Algorithms for any constant ! Proved by new structure lemmas for
13. CorollaryIf there are-time CAPP for with , or a -time CAPP for with constant , then .
14. Another Application: Inapproximability by Depth-2 Neural NetworksDepth-2 Neural Network THRTHRTHR ReLUReLUReLU ThmFor every and constant , there is a function such that cannot be approximated by Depth-2 Neural Networks of size Improved [Wil’18], which proved that there is such an which cannot be exactly computed by Depth-2 Neural Networks of size .
15. PhilosophyUsing PCP Algorithmically to Prove Circuit Lower Bounds (Remember: PCPs are algorithms!)If you want to prove , then PCPs should make your life much easier (now you only need an algorithm for -approximation to 3-SAT!) [Håstad’97] (Well, I don’t really believe in .) We only want to derandomize circuits. But PCPs still make our life easier (though in a more indirect way)
16. Starting Point: Non-deterministic Derandomization Suffices for Circuit Lower Bounds-GAP-TAUT (tautology) Distinguish between (Yes Case).(No Case) [Wil’13] time non-deterministic algorithm for GAP-TAUT on poly-size general circuits with . Non-deterministic Algorithm for GAP-TAUTGiven a general circuit , we want a time non-deterministic algo , such that:If is a tautology, then accepts on some guesses.If , rejects on all guesses.
17. Proof Overview: OutlineAssume Non-trivial CAPP on with constant non-deterministic GAP-TAUT for Contradiction! Starting Point [Wil’13] time non-deterministic algorithm for GAP-TAUT on poly-size general circuits with . Key point: make use of this assumption as much as possible!Think of as
18. Goal: Designing the Algorithm under AssumptionGoalGiven an circuit , under the two assumptions, design a time non-deterministic algo , such that:If is a tautology, then accepts on some guesses.If , rejects on all guesses. It is universal Assume Non-trivial CAPP on with constant non-deterministic GAP-TAUT on Think of as
19. Review: Approach of [Wil’14]Guess-and-Verify-Equivalence implies collapses to .That is, under assumption, the given general circuit has an equivalent circuit .If we can find , then we can derandomize instead, where we have algorithms! Problem: How to find ? Allowed to use non-determinism so one can guess . But still have to verify is equivalent to , which seems HARD. Solution Well, just guess more circuits!
20. Review: Approach of [Wil’14]Guess-and-Verify-EquivalenceSuppose has gates, let be the corresponding sub-circuits. is the output gate. are inputs. implies collapses to .We guess circuits , hoping that . We wish to check . To do this, for each , suppose gate- has inputs from gate- and gate-. We verify . Then run CAPP on . ProblemChecking for all requires solving SAT for .
21. A Local-checkable Proof System ViewProblem: the previous approach requires solving SAT for . Let This is a Claimed Proof for by giving values at all gates.Intuitively, it is supposed to be the computation history of on input . Local checks on For each , . What is so good about this proof ?
22. A Local-checkable Proof System ViewLet A Claimed Proof for by giving values at all gates. One can get functions on , such thatEach is an of 3 bits (or their negations) from .If on the correct guesses , all ’s are satisfied by . (Completeness)If , for all possible at least one is not satisfied by . (Soundness)
23. Guess circuits , let Estimate . (.) (:number of ’s)If is a tautology. Then on the correct guess, If then on all guesses, . An AttemptTo distinguish the above two cases, we need a CAPP algo with error . But we only assume a CAPP algo with constant error!
24. What Went Wrong?One can get functions on , such thatEach is an of 3 bits (or their negations) from .If on the correct guess , all ’s are satisfied by . (Completeness is 1)If , for all possible at least one is not satisfied by . (Soundness is ) This is an extremely ``bad’’ PCP! Why not just use the PCP theorem?If there is a verifier who picks a random and checks whether . She detects an error only with probability when . Proof System View : a claimed proof of : local check of the verifier
25. Issues When Applying PCPs DirectlyTherefore, we want a proof system for verifying , such that given the random bits, verifier queries both input and proof .If , exists , such that always accept.If , for all ,rejects w.h.p. Recall that in the end we want to estimate Key properties being used in previous attempt: These local checks (verifier’s queries positions) do not depend on the input ! Use PCPs of Proximity! Like PCPs but both input and proof are given as oracles.PCPsV (input) (proof) Unlimited access3 queriesNow, can depend on many bits of . PCPs of ProximityV (input) (proof) 3 queries in total
26. Issues When Applying PCP DirectlyTherefore, we want a proof system for verifying , such that given the random bits, verifier queries both input and proof .If , , such that always accept.If , , rejects w.h.p. Counter-example? Suppose computes the parity.Parity changes if we flip a random bit of .The verifier can’t distinguish unless she queried that bit. Solution Give access to an error correcting code of !
27. Combing PCP of Proximity and ECCsPCP of Proximity Verifier is given both the input () and the proof as oracles and makes queries. accepts w.p. 1, when ; accepts w.p. , when makes robustly output ( is zero in a small hamming ball around ). (like property testing) How it avoids the parity counter example?No inputs can make parity robustly output ! V (input) (proof) 3 queries in total
28. PCP of Proximity with ECCsVerifier is given both the encoded input () and the proof as oracles and makes queries. accepts w.p. 1, when ; accepts w.p. , when . Use of Proximity for verifying , makes robustly output when !DEC(corrupted ) is still
29. Final AlgorithmEstimate . (.).If is a tautology. Then on the correct guesses, If then on all guesses, . Guess circuits , let Fix to be -linear. That is, is a parity on a subset of bits in . Suppose there is uniform parity circuit in for now (this assumption can be avoided) Now constant error CAPP algo for suffices!
30. Future WorkNEW Building on the PCPP based approach, [Alman Chen’19] give a construction of Razborov-rigid matrices in . Can we find non-trivial CAPP algorithms for or to prove circuit lower bounds for ?Recall: we know exponential lower bounds for these two models! Can we ``mine’’ some algorithms from these proofs?
31. Thank You
32. Applying to the Previous AlgorithmGuess circuits , let
33. PCP of ProximityFix a circuit C. There is a PCP of Proximity system with proximity parameter completeness , and soundness (, both are constants), number of random bits , satisfying the following properties: The verifier tosses random coins, queries 2 positions of . ( is the input, is the proof).If , accept w.p. . (Completeness)If is -far from , accepts w.p. . (Soundness when x is far from making accept).Moreover, the correct proof can be computed from and inpolynomial time.
34. Applying PCP of ProximityFix an -linear error correcting code , with decoder , which can recover fraction of errors. Given circuit , construct, as . Fix the PCP of Proximity system for circuit E, guess circuits , such that is supposed to be the proof that
35. Applying PCP of ProximityFix the PCP of Proximity system for circuit E, guess circuits , such that is supposed to be the proof that (Recall )Let be the -local check corresponding to random bits . For input , proof , is the probability that verifier accepts . When (), s.t. .(Completeness) When (), is -far from .(If is -close to , ). .(Soundness)
36. Applying PCP of ProximityWhen (), s.t. .(Completeness) When (), . If is a tautology, on certain guesses , If , on all guesses , To distinguish these two cases, it suffices to estimate each within .
37. Applying PCP of ProximityTo distinguish these two cases, it suffices to estimate each within . WLOG, can assume is an OR on two bits. Also assume can (uniformly) compute parity. Now it suffices to solve CAPP for for a constant error.