Protobufs Demystifying Googles hottest binary protocol Prasanna Kanagasabai Jovin Lobo About us Prasanna Kanagasabai Security Engineer T houghtWorks Member of ID: 580260
Download Presentation The PPT/PDF document "The Curious case of" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
The Curious case of Protobufs…
De-mystifying Google’s hottest binary protocol
Prasanna
Kanagasabai
Jovin LoboSlide2
About us :
Prasanna Kanagasabai :Security Engineer @
T
houghtWorks
Member of null- The Open Security Community .Author of IronSAP a module over IronWASP. Speaker @ nullcon-Delhi, Clubhack, IIT Guwahati and various null meetups.Jovin Lobo :Associate Consultant @ Aujas NetworksMember of null- The Open Security Community.Author of GameOver – Linux distro for learning web security.Spoken at nullCon, GNUnify before. Slide3
Agenda
Introduction.Anatomy of ProtobufsDefining Message formats in .Proto files.Protobuf
compiler
Python API to read write messages.
Encoding SchemeProblem Statement. Decoding like-a-pro with IronWasp ‘Protobuf Decoder’.Slide4
Introduction:Protocol Buffers
a.k.a Protobufs :Protobufs
are Google's own way of serializing structured data .
Extensible, language-neutral and platform-neutral .
Smaller, faster and simpler to implement. Java, C++ and Python Slide5
Anatomy:Over view :Slide6
Defining a .Proto file.
#> less Example.protomessage Conference {
required string
conf_name
= 1 ; required int32 no_of_days = 2 ; optional string email = 3 ; }// * 1,2,3 are unique tags. These are used by the fields in binary encoding.* For optimization use tags from 1-15 as higher nos. will use one more byte to encode. Slide7
Compiling Syntax:
protoc –I=$_input_Dir --python_out=
$_
out_Dir
$_Path_ProtoFileEg:protoc –I=. --python_out=. Example.protoThis will generate a Example_pb2.py file in the specified destination directory.Slide8
$ProtoFile_pb2.pyThe
Protobuf compiler generates special descriptors for all your messages, enums, and fields.It also generates empty classes, one for each message type:Eg:Slide9
Reading and writing messages using the Protobuf binary format :
SerializeToString()serializes the message and returns it as a string.
ParseFromString
(data)
parses a message from the given string.Slide10
Demo: Protobuf… how it wrks
Slide11
Encoding.
example2.protomessage Ex1 { required int32 num = 1; // field tag
}
Code snippet:
obj = example2_pb2.Ex1();obj.num = 290; // field valueobj.SerializeToString();Output : 08 A2 02 #hex000010001010001000000010 #binarySlide12
Problem statement.Slide13
This is what freaked him out
08 A2 02
000010001010001000000010Slide14
Lets Decode it .. Step 1 :
Find the wire type . Step 2:Find the field number.Step 3:Find the field tag.Slide15
Step1: finding wire type.
0000 1000 1010 0010 0000 0010To find wire type take the first byte: 0000 1000 1010 0010 0000 0010
[0]
000 1000
Drop MSB from First byte. 0001 000 The last 3 bits give wire type. Wire type is 000 type = 0 is Varint.Slide16
Wire typesSlide17
Step 2: Field tag.What we already have is
0001000 Now we right shift value by 3 bits and the remaining bits will give us the field tag.00010000001
000
‘0001 ‘ i.e. ‘ 1’
So we get the field tag = 1Slide18
Step 3: Find the field value
0000 1000 1010 0010 0000 0010We drop the 1st byte
1010 0010
0000 0010
Drop the MSB’s from each of these bytes1010 0010 0000 0010010 0010 000 0010 Reverse these bytes to obtain the field value.000 0010 010 0010 000 0010 010 0010 i.e 256 + 32 + 2 = 290So we finally get the value of the field = 290. Slide19
So we successfully decoded
example2.protomessage Ex1 { required int32 num = 1; }
Code snippet:
obj
= example2_pb2.Ex1();obj.num = 290;obj.SerializeToString();Output : 08 A2 02 #hex000010001010001000000010 #binaryWe successfully Decoded Value : “290” Slide20
Demo : Lets do this live Slide21
Automating all this with IronWasp Protobuf Decoder:
About IronWasp
:
IronWasp is an open-source web security scanner.It is designed to be customizable to the extent where users can create their own custom security scanners using it.Author – Lavakumar Kuppan (@lavakumark)Website : www.ironwasp.orgSlide22
ProtoBuf DecoderSlide23
Road Map for Protobuf DecoderSlide24
01101000001111010000010110111001111001001000000101000101110101011001010111001101110100011010010110111101101110011100110010000000111111Slide25
01101000001111010000010110111001111001001000000101000101110101011001010111001101110100011010010110111101101110011100110010000000111111Slide26
01101000001111010000010110111001111001001000000101000101110101011001010111001101110100011010010110111101101110011100110010000000111111
Hmmm … Decoding ……Slide27
Any Questions ?
Done … It says ……Slide28
Any Questions ?
Done … It says ……Slide29
Thank You