Ad-hoc routing protocol showdown

This CMU paper from mobicom '98 uses simulation to compare the performance of ad-hoc routing protocols DSDV, TORA, DSR, and AODV.

They had to modify the ns network simulation software to account for node mobility, a more realistic physical layer, wireless NICS (not just wired), 802.11 MAC using DCF.

They also implemented each of these protocols and discovered, along the way

Not surprisingly, they found that node mobility has a negative impact on performance. Beyond that, the protocols break into two classes. This is obvious when looking at the shapes of the curves in figure 3. As the paper puts it, "TORA, DSR, and AODV-LL are all on-demand protocols, and their overhead drops as the mobility rate drops. As DSDV-SQ is a largely periodic routing protocol, its overhead is nearly constant with respect to mobility rate."

In a recent conversation with Ion Stoica during a meeting about experimentally reproducing a transport layer congestion event coined incast, I inquired as to the value of simulating incast vs. reproducing it using a real switch and nodes. Ion was less than enthusiastic about simulation, so we are moving ahead with a real experiment. This made the think about how there seems to be two camps in network research, those who simulate and those who run the real experiments. We have a number of paper from both camps. Here are just a few of the papers we have read which fall into one or the other.

Simulate

• XCP
  
• RED
  
• CSFQ
  

Real Experiments

• Macaw
  
• Mechanisms for improving TCP over wireless (Snoop)
  
• Roofnet
  
• ETX
  

It seems to me that simulating wireless networks is more difficult than simulating wired networks because there is more variability in their performance, and with wireless networking technology costing so little, what are the reasons that any research group would continue to simulate? One argument that comes to mind is that a good model allows and encourages us to more thoroughly understand what is causing the effects we observe in real life. This sort of reasoning leads to an argument that we should probably both simulate and then verify our simulations on real hardware for all networking research projects (and perhaps all systems projects in general). Of course, this takes extra time and effort which we might not have to spend. Also, I should go back and review the "Modeling Wireless links for Transport Protocols" paper to remind myself of the arguments they present for modeling/simulating.