What constitutes a useful experimental result?
Description: What constitutes a useful experimental result? Ratul Mahajan Microsoft Research On experimental vs. theoretical research It is the hand that shapes the head, not the other way around. -- translation from a Marathi haiku (thanks to Jitu
Related Topics
Download Presentation
"What constitutes a useful experimental result?" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
slide1. What constitutes a useful experimental result? Ratul Mahajan
Microsoft Research<br>
slide2. On experimental vs. theoretical research “It is the hand that shapes the head, not the other way around.”
-- translation from a Marathi haiku
(thanks to Jitu Padhye) ratul | usc-tpwn | 08 2<br>
slide3. Neither hardware nor market realities limit experimental results severely Yes, both are constraining
Yes, both represent moving targets
But, no reason to be paralyzed
Often, there are ways to skirt these limitations
E.g., leveraging broadcast capability, software radios ratul | usc-tpwn | 08 3<br>
slide4. Instead, it is poorly specified applicability context Does a given result apply in a different setting?
Can be hard for even good experimental results
It is a problem common to other fields
E.g., network measurement, cancer research ratul | usc-tpwn | 08 4<br>
slide5. Context should be captured at all layers PHY
BPSK vs. QAM-64 for ANC and SIC
Binary vs. gray loss patterns for routing
MAC
Transmit rate adaptation for opportunistic routing
The extent of collisions for rate adaptation
Network (topology)
Mesh vs. infrastructure for hidden terminals ratul | usc-tpwn | 08 5<br>
slide6. Transport
TCP vs. UDP for performance
Application
Bursty vs. always-on for network coding
Environment
Interference-free or interference-ridden
Mobility Context should be captured at all layers (2) ratul | usc-tpwn | 08 6<br>
slide7. More on capturing context Exactly what should be captured?
More detail is better but that has its limits
Should reproducibility be the goal?
Should practical approximations be OK?
May usefully expand applicability
“Applicability considerations” in papers? ratul | usc-tpwn | 08 7<br>
Microsoft Research<br>
slide2. On experimental vs. theoretical research “It is the hand that shapes the head, not the other way around.”
-- translation from a Marathi haiku
(thanks to Jitu Padhye) ratul | usc-tpwn | 08 2<br>
slide3. Neither hardware nor market realities limit experimental results severely Yes, both are constraining
Yes, both represent moving targets
But, no reason to be paralyzed
Often, there are ways to skirt these limitations
E.g., leveraging broadcast capability, software radios ratul | usc-tpwn | 08 3<br>
slide4. Instead, it is poorly specified applicability context Does a given result apply in a different setting?
Can be hard for even good experimental results
It is a problem common to other fields
E.g., network measurement, cancer research ratul | usc-tpwn | 08 4<br>
slide5. Context should be captured at all layers PHY
BPSK vs. QAM-64 for ANC and SIC
Binary vs. gray loss patterns for routing
MAC
Transmit rate adaptation for opportunistic routing
The extent of collisions for rate adaptation
Network (topology)
Mesh vs. infrastructure for hidden terminals ratul | usc-tpwn | 08 5<br>
slide6. Transport
TCP vs. UDP for performance
Application
Bursty vs. always-on for network coding
Environment
Interference-free or interference-ridden
Mobility Context should be captured at all layers (2) ratul | usc-tpwn | 08 6<br>
slide7. More on capturing context Exactly what should be captured?
More detail is better but that has its limits
Should reproducibility be the goal?
Should practical approximations be OK?
May usefully expand applicability
“Applicability considerations” in papers? ratul | usc-tpwn | 08 7<br>