ABY 3 A Mixed Protocol Framework for Machine Learning Peter Rindal Payman Mohassel Machine Learning and Privacy Cognitive tasks voice facial recognition Medical genetic testing disease prediction ID: 773243
Download Presentation The PPT/PDF document "ABY 3 : A Mixed Protocol Framework for ..." is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
ABY3: A Mixed Protocol Framework for Machine Learning Peter RindalPayman Mohassel
Machine Learning and Privacy Cognitive tasks: voice, facial recognitionMedical: genetic testing, disease prediction Financial: fraud detection, credit ratingInference: User does not want to share their dataModel owner does not want to share modelTraining: Requires large data sets, often from different sources
Training A better Model ML Algorithm Model More data better model! Richer data better model! Each party can have similar or different types of data
ML Algorithm Model Security Concerns Data sharing often prohibited for completive or regulatory reasons Use encryption? Encrypt at rest Encrypt in transit Decrypted in use!!
Our Results New three party MPC Protocols:Efficient support for fixed-point arithmeticsImproved matrix multiplication Efficient piece-wise polynomial evaluation Conversions between Arithmetic, Boolean, and Yao secret sharesAlways encrypted machine learning training and inference:Linear RegressionLogistic RegressionNeural NetworksExtendable to other models
Protocols and Building block
Always Encrypted = + + = + + Each party “Encrypts” their data such that = + +
Always Encrypted = + + = + + Each party “Encrypts” their data such that = + +
Always Encrypted = + + = + + Each party “Encrypts” their data such that Share Types: A rithmetic: B oolean: Y ao G.C.: = + +
Always Encrypted = + + = + + Each party “Encrypts” their data such that = + +
Each party “Encrypts” their data such that Possible to compute any function / circuit in this format Always Encrypted Multi-party Computation
How to Compute on Shared Data Say we have shared Addition :
How to Compute on Shared Data Multiplication : Use “replicated secret sharing”
How to Compute on Shared Data Multiplication : Use “replicated secret sharing” Observe: Let
How to Compute on Shared Data Multiplication : Use “replicated secret sharing” Observe: Let
Treat bottom bits as factional bits Multiplication drops top/bottom bits Decimal Multiplications in Integer Group . . × . 32 bits . 16 bits 16 bits
Treat bottom bits as factional bits Multiplication drops top/bottom bits Mohassel Zhang ‘17 Introduced secret shared version Adds small rounding errors Only works on 2-out-of-2 secret sharing We extend to any honest majority Briefly switch to 2-out-of-2 sharing Decimal Multiplications in Integer Group .16 bits .×.32 bits.16 bits Reveal() Preprocess:
Treat bottom bits as factional bits Multiplication drops top/bottom bits Mohassel Zhang ‘17 Introduced secret shared version Adds small rounding errors Only works on 2-out-of-2 secret sharing We extend to any honest majority Briefly switch to 2-out-of-2 sharing Round preservingMalicious secure Decimal Multiplications in Integer Group .16 bits .×.32 bits.16 bits Reveal() Preprocess: Share # Share #
Matrix Multiplication on Shared Data Given, Compute Classic “rows columns” algorithm. E.g. Each multiplication requires communication Communication = numbers! Recall “half” a multiplication is done locally. Do half multiplies for Only send final result times less communication
Piece-wise Polynomial Let first threshold … Strategy: Polynomial easy to compute How to compute ? = Convert-to-binary = Most-significant-bit = Converty-to-arithmeticTotal cost: rounds and bits of communication // range test
Conversion Need to convert secret sharing of :Arithmetic to binary: Binary to Arithmetic: Strategy: Parties provide as input to the binary MPC protocol Evaluate the circuit: = Optimizations: Implement using low depth circuit, depth Have party locally compute and input Also implement conversion to/from garbled circuitsGarbled circuits allow low latency for high depth circuit ArithmeticShare BinaryShare Yao GarbledCircuit
Malicious Security We extend our semi-honest protocol to the malicious settingIf any one party tries to cheat, the protocol can detect it.Leverage replicated secret sharingMost operations roughly times more expensive
Application: Machine Learning
Linear Regression on Shared Data Given, Find a linear function s.t. x y
Linear Regression on Shared Data Given, Find a linear function s.t. Model: Update function: x y Cost function: (a.k.a. L2 norm)
Linear Regression on Shared Data Given, Find a linear function s.t. Model: Update function: x y Cost function: (a.k.a. L2 norm) Learning Rate Error Magnitude Error Direction
Batch Linear Regression on Shared Data Given, Find a linear function s.t. Update in batches: For Must do many matrix multiplications x y Current Batch
Logistic Regression Instead of computing We compute is very expensive to compute with just operations We build a piece-wise polynomial protocol for
Logistic Regression Instead of computing We compute is very expensive to compute with just operations We build a piece-wise polynomial protocol for or higher degree approximation
Neural Network Generalization of logistic regression Each node is a regression problemReplace logistic function with ReLU.Easy to implement with piecewise polynomialOutput nodes use soft-max: For inference/prediction, replace with arg -max. For training, Approximate Approximate or use one garbled circuit ReLU: ReLUReLUReLUReLU SoftmaxSoftmax
Performance – Inference The models are for the MNIST dataset with D = 784 features. 93% 98% * 97% 99%
Performance – Logistic Regression Training Measures iterations / second, larger = betterDimension = # of featuresBatch Size B = # examples used at each iterationTotal running time: Up to times faster Communication: Up to 600 times less
Summary – ABY3 New Protocols:Efficient support for fixed-point arithmeticsImproved matrix multiplication Efficient piece-wise polynomial evaluationConversions between Arithmetic, Boolean, and Yao secret sharesPrototype machine learning implementation:Linear Regression – 12000 iterations / secondLogistic Regression – 2000 iterations / second Neural Networks – 10 millisecond inference
The End, Questions? Peter RindalPayman Mohassel
Truncation on 2-out-of-2 Secret Shares . bits . . bits . bits .. bits Assumption: no overflowing..Operation: truncate top/bottom bits. . - 1 error if + overflows Pr[large error] = . . = Probability of -1 error removing a mod wrap around Party 1 Party 2 [MohasselZhang17]
Truncation on 2-out-of-3 Secret Shares . bits . . bits . . . = Probability of -1 error removing a mod wrap around Party 1 . . . . . . Party 2 . . . . . . Party 3 Assumption: no overflowing Operation: truncate top/bottom bits Pr[ large error ] = - 1 error if + overflows
Truncation/Multiplication on 2-out-of-3 Secret Shares Multiplication : Preprocess Let (integer multiplication) Compute
Truncation/Multiplication on 2-out-of-3 Secret Shares Multiplication : Preprocess Let (integer multiplication) Compute
Performance – Linear Regression Training Measures iterations / second, larger = betterDimension = # of featuresBatch Size B = # examples used at each iteration