It is relevant to the SST Community as our project helps to map out certain areas in the school with stronger Wi-Fi signals, as well as those areas in school with weaker Wi-Fi signals. As such, a map can be drawn up to help students know where to go if they require a strong Wi-Fi signal to submit an urgent task.
It can also be used to help people when designing buildings, as since nowadays most buildings have Wi-Fi systems in operation, our research can help designers understand how to make sure the necessary areas in the building are effectively covered by the Wi-Fi signal.
Areas for further study
We initially wanted to test how certain material blocks WiFi waves. As this is a school project, we have limited resources and time to conduct such a massive scale project as to make the results more reliable, we would have to use special RF/WiFi wave isolation materials. In the future, should this be attempted, the experimenter must consider the nature of WiFi and RF waves and how to isolate them to get the best and most accurate readings
An Experimental Performance Comparison of 3G and Wi-Fi Richard Gass1 and Christophe Diot2 1 Intel Labs 2 Thomson Abstract. Mobile Internet users have two options for connectivity: pay premium fees to utilize 3G or wander around looking for open Wi-Fi access points. We perform an experimental evaluation of the amount of data that can be pushed to and pulled from the Internet on 3G and open Wi-Fi access points while on the move. This side-by-side comparison is carried out at both driving and walking speeds in an urban area using standard devices. We show that significant amounts of data can be transferred opportunistically without the need of always being connected to the network. We also show that Wi-Fi mostly suffers from not being able to exploit short contacts with access points but performs comparably well against 3G when downloading and even significantly better while uploading data. 1 Introduction Wireless communication is an important part of everyday life. It allows people to stay connected with their jobs, family, and friends from anywhere there is connectivity. The two dominant wireless technologies are Wi-Fi and third generation cellular (3G) networks. IEEE 802.11, commonly known as Wi-Fi, refers to a set of standards which operate in the unregulated ISM band. They are very well known for providing wireless connectivity in homes, offices, and hot-spots. They provide throughput of up to 600 Mbits/s with a coverage area in the hundreds of meters. Wi-Fi is easy and inexpensive to deploy, and is ubiquitous in urban areas. Despite access controls being deployed and newer access points (APs) being configured with security enabled by default, many Wi-Fi APs remain open. In addition, the growing popularity of community networks such as FON 3 and the growing list of large cities providing free wireless makes opportunistic communication a realistic scenario in urban areas. Due to the sparse and non-coordinated deployment of APs, Wi-Fi is not an “always connected” technology. It is designed primarily for the mobile user that accesses the network while relatively stationary. It provides high data rates between locally connected clients but is limited by the capacity of the link between the AP and the Internet. 3 www.fon.com 3G is based on technology that has evolved to fill the growing need for data in wireless voice networks. 3G provides seamless connectivity across large coverage areas with advertised data rates of 2 to 14 Mbits/s, shared among all users connected to any given base station. 3G network operators charge either based on consumption or have flat rate monthly plans. These networks are expensive to deploy and the performance experienced by users is sensitive to the number of users in a cell due to the large coverage areas. For data applications, one could argue that persistent connectivity may not be necessary. Instead, being connected “frequently enough” should be acceptable if applications and communications protocols could take advantage of short, but high bandwidth contact opportunities. We present results of a side by side, Wi-Fi vs 3G face-off. We show that with default access point selection (greatest signal strength), unmodified network setup methods (scan, associate, request an IP address with DHCP), and off the shelf equipment with no modifications or external antennae, opportunistic Wi-Fi performance is comparable to 3G. Despite only connecting to open or community access points in a typical urban residential area, Wi-Fi throughput surpasses 3G at walking and driving speed while uploading data and is nearly equivalent to 3G while downloading. The remainder of this paper is organized as follows: We first explain how the experiments were conducted and describe the equipment and software setup in Section 2. Next, in Section 3, we show the results of the experimental runs with the comparisons of 3G vs Wi-Fi under driving and walking conditions as well as look at the effects related to the uploading or downloading of data. Finally, we discuss related work in Section 4 and conclude the paper in Section 5. 2 Experiment description The experiments consist of two mobile clients and a server that is always connected to the Internet. One mobile client uses its Wi-Fi interface to transmit and receive data to/from the server and the other uses 3G. Experiments are performed both on foot and in a car following the same route. Wi-Fi and 3G tests are run simultaneously for a true side-by-side comparison. While downloading, the data originates at the servers and is streamed down to the mobile clients. Conversely, when uploading, the data originates on the mobile clients and is streamed to the servers. We investigated the potential of using the 3G device for collecting both 3G and Wi-Fi data but discovered that stationary Wi-Fi transfers in the uplink direction were capped around 6 Mbits/s, well below the advertised rates of an 802.11G enabled interface. We also saw variations in the Wi-Fi throughput while running simultaneous 3G and Wi-Fi experiments on the same mobile device. Due to these limitations, we chose to use a separate platform for each technology. 2.1 Server setup The servers run the Ubuntu distribution of Linux (version 8.04.1 with a 2.6.24- 19-server kernel) and are publicly accessible machines on the Internet that are the source or sink for the clients. The servers are virtual machines running on the Open Cirrus cluster hosted at Intel Labs Pittsburgh (ILP). The dedicated Internet connection to ILP is a 45Mbit/s fractional T3 and did not pose any restrictions in these experiments. We ran extensive tests of the code on the virtual machines and saw no performance related issues with the system or the network. The 3G server runs the apache web server and hosts large, randomly generated data files that can be downloaded by the client. The Wi-Fi server runs a simple socket program that generates data with /dev/random and streams it down to the Wi-Fi client. When data is being uploaded from the client, both the 3G and Wi-Fi server run our socket program that receives the data and sends it to /dev/null. The network interfaces for both servers are monitored with tcpdump and the resulting data traces are stored for off-line analysis. 2.2 Wi-Fi client The Wi-Fi client setup consists of an IBM T30 laptop with a default install of the Ubuntu distribution of Linux (version 8.04 with a 2.6.24-21-server kernel). The internal wireless device is the Intel 2915ABG network card using the unmodified Intel open source Pro/Wireless 2200/2915 Network Driver (version 1.2.2kmprq with 3.0 firmware). No external antenna is connected to the laptop for the experiments. The laptop attempts to connect to the Internet by first scanning the area for available open or community APs (excluding those with encryption enabled and those we have marked as unusable4 ) and chooses the one with the strongest signal strength. Once the AP is selected, it begins the association process followed by IP acquisition via DHCP. If the AP allocates an IP address to the client, it attempts to ping a known server to confirm connection to the Internet. Once Internet connectivity is verified, the Wi-Fi client begins either downloading or uploading data from/to the server via our simple socket program. After the client travels out of range of the AP, it detects the severed connection by monitoring the amount of data traversing the network interface. Once the client stops seeing packets for more than a configured time threshold, the current AP is abandoned and the search for another available AP begins. We choose 5 seconds in our experiments to allow ample time to make sure we do not attempt to reconnect to an AP that is at the trailing edge of the wireless range. All experimental runs utilize a USB global positioning system (GPS) receiver that is plugged into the laptop capturing speed, location, and time once per second. The GPS device is also used to synchronize the time on the laptop. The 4 An example entry is CMU’s public Wi-Fi that is open but only allows registered MAC addresses to use the network. laptop captures all data that is transmitted or received over the wireless interface with tcpdump. 2.3 3G client The 3G experiments employ an out of the box Apple iPhone 3G with no modifications to the hardware. The iPhone connects via the AT&T 3G network, uses a jail-broken version of the firmware (2.2, 5G77), and its modem baseband firmware is at version 02.11.07. The 3G client begins by first synchronizing its clock with NTP. Once the clock has been synchronized, it launches tcpdump to monitor the 3G wireless interface. After the monitoring has started, the client begins either downloading or uploading data. To download data, we use an open source command line tool for transferring files called curl. The curl program downloads a large file from the server and writes the output to /dev/null to avoid unnecessary CPU and battery consumption on the mobile device. This also allows us to isolate only network related effects. If the client is uploading data, the dd command continuously reads data out of /dev/zero. The output is piped into netcat and the data is streamed to the server. (a) (b) Fig. 1. Maps of an area in Pittsburgh showing (a) all available open access points and (b) the route followed for the experiments 2.4 The experiment route The experiments are performed in a residential area of Pittsburgh, Pennsylvania near the campus of Carnegie Mellon University (CMU). Figure 1(a) is a map of the area where we focused our measurement collection. This area lies between the CMU campus and a nearby business district where many students frequently travel. Each red tag in the figure represents an open Wi-Fi AP found from our wireless scans5 . The area is also covered by 3G service allowing us to compare 5 Our scan logs reveal 511 APs in the area with 82 that appear open. Table 1. 3G vs Opportunistic Wi-Fi Radio Speed Data-flow Usable contact time Throughput Total transfer 3G driving download 760 seconds 579.4 kbits/s 55 MB Wi-Fi driving download 223 seconds 1220 kbits/s 34 MB 3G walking download 3385 seconds 673 kbits/s 285 MB Wi-Fi walking download 1353 seconds 1243 kbits/s 210 MB 3G driving upload 866 seconds 130 kbits/s 14 MB Wi-Fi driving upload 118 seconds 1345 kbits/s 20 MB 3G walking upload 3164 seconds 129 kbits/s 51 MB Wi-Fi walking upload 860 seconds 1523 kbits/s 164 MB the two access technologies. We believe this area to be representative of typical Wi-Fi densities found in most European or US urban areas6 . Figure 1(b) shows the route selected in this area for our experiments. The experiment starts at the leftmost tag at the bottom right hand corner of the figure and follows the indicated route until the destination (same as start position) is reached. The total distance of the route is about 3.7 miles. For the walking experiments, we maintain a constant speed (2.4 MPH) throughout the course of the route. While driving, we obeyed all traffic laws and signs and remained as close to the speed limit (25 MPH) as possible. 3 Results Table 1 summarizes the results of the experiments which are based on 16 runs from different days performed in the afternoon and late evening. 3.1 3G vs Wi-Fi downloads Figure 2 shows the instantaneous throughput achieved for a single, representative experiment for 3G and Wi-Fi at driving speeds of up to 30 MPH. The 3G device is able to transfer around 55 MB of data for the 760 seconds of the experiment duration. During this time, the Wi-Fi client connects opportunistically to APs along the route and manages to spend 223 seconds connected, transferring 34 MB. These “in the wild” results clearly show the potential of this untapped resource of open Wi-Fi connectivity and have a similar behavior to the isolated and controlled experiments in [5, 6, 14]. Deeper investigation of our logs shows that the majority of contacts were initiated while the client was either stopped, slowing down, or accelerating after a stop. This meant that the client stayed within the range of a single AP for longer durations and allowed more time to perform the steps needed to setup a connection and begin a data transfer. Since our AP selection algorithm was to 6 http://wigle.net 0 1 2 3 4 5 6 7 8 9 0 100 200 300 400 500 600 700 0 10 20 30 40 50 60 70 MBits/s Total data transferred (MB) Time (sec) 3G 3G total Wi-Fi Wi-Fi total Fig. 2. Instantaneous throughput (Mbits/s) for 3G vs Wi-Fi downloads at driving speeds and total data transferred (MB) always select the AP with the strongest signal, while moving, this was generally not the optimum choice. When the client approaches a potential AP, it would be best to select the AP that would be just coming into range to maximize the usable connection duration. We found that many connection attempts succeeded but when the data transfer was about to begin, the connection was severed. This does not mean that opportunistic contacts cannot happen at speeds, but instead brings to light the need for faster AP association and setup techniques similar to QuickWiFi and better AP selection algorithms for mobile clients. Both of these would allow better exploitation of opportunistic transactions for in-motion scenarios. Also plotted in Figure 2 is the total amount of data transferred for each access technology. The 3G connection is always connected throughout the entire experiment and shows a linear increase in the total bytes received. Wi-Fi, is represented by a step function which highlights how each connection opportunity benefits the overall amount of data received. Each point on the “Wi-Fi total” line represents a successful contact with an AP. Even though Wi-Fi contacts show large variability due to the intermittent nature of the contact opportunities, there is still a significant amount of data transferred because of the higher data rates of the technology. This is more apparent in the walking experiments where the speeds are much slower and the use of the sidewalks brings the client physically closer to the APs, allowing the client to remain connected for longer durations. Figure 3 shows throughput results of a single, representative experiment for 3G and Wi-Fi at walking speeds. The walking experiments last around 3385 sec- onds and 3G is able to transfer around 285MB of data. Wi-Fi, on the other hand, is only connected for 1353 seconds of the experiment and downloads 210MB. Again, each Wi-Fi contact is able to exploit the opportunity and take advantage of very short, high throughput contacts, transferring significant amounts of data. 0 2 4 6 8 10 12 14 0 500 1000 1500 2000 2500 3000 0 50 100 150 200 250 300 MBits/s Total data transferred (MB) Time (sec) 3G 3G total Wi-Fi Wi-Fi total Fig. 3. Instantaneous throughput (Mbits/s) for 3G vs Wi-Fi downloads at walking speeds and total data transferred (MB) 3.2 3G vs Wi-Fi uploads Figure 4 shows the instantaneous throughput of 3G and Wi-Fi uploads at walking speeds. It has a similar behavior to that of the downloads described previously but this time, the total data transferred for Wi-Fi exceeds 3G by 2.6 times. This is due to poor upload performance of 3G on the mobile device. The instantaneous 3G traffic pattern shows transitioning between idle states and periods of data transfers that result in throughput much less than that of the downloads (averaging at 130 kbits/s). In order to understand this phenomenon, we performed additional experiments with a stationary laptop (Lenovo T500 using the iPhone SIM card) and the iPhone with updated software and baseband firmware, 3.0 (7A341) and 04.26.08 respectively. We found that this periodic pattern is no longer evident. The new traces exhibit more consistent, albeit lower, throughput throughout the entire duration of an upload. The total amount of data transferred for a similar experiment did not change. We conjecture that these are due to improvements in the iPhone baseband software which allow more efficient buffering of data, eliminating the burstiness of the traffic egressing 0 1 2 3 4 5 6 7 8 9 0 500 1000 1500 2000 2500 3000 0 20 40 60 80 100 120 140 160 180 MBits/s Total data transferred (MB) Time (sec) 3G 3G total Wi-Fi Wi-Fi total Fig. 4. Instantaneous throughput (Mbits/s) for 3G vs Wi-Fi uploads at walking speeds and total data transferred (MB) the device. Further upload experiments with the laptop show that it is able to transfer data at twice the rate of the iPhone. These results suggest hardware limitations on the iPhone and/or an artificial software limitation placed on the device7 . One of the side observations from our experiments that impact the mobile client throughput is that residential Internet service rates are much higher than shown in . Upon further investigation, we discovered that Verizon FIOS8 has recently become available in this area and our experiments show that some homes have upgraded to this higher level of service. This is hopeful for utilizing opportunistic communications since more data can be transferred during these very short contact opportunities. It is also important to note that during these experiments, the full potential of the Wi-Fi AP was not reached and instead was limited to the rate of the back-haul link the AP was connected to. Even though the cost of higher throughput links are dropping in price for residential service plans, affordable service provider rates are still well below the available wireless rates of 802.11. This will always place the bottleneck for this type of communication at the back-haul link to the Internet9 . 7 http://www.networkperformancedaily.com/2008/06/ 3g iphone shows bandwidth limi.html 8 http://www22.verizon.com 9 http://www.dslreports.com/shownews/Average-Global-Download-Speed-15Mbps- 101594 4 Related work This work compares two dominant access technologies, namely 3G and Wi-Fi, in the wild. Despite many works related to the performance of 3G and Wi-Fi networks, this is the first work to publish a side-by-side comparison while in motion. This work highlights the potential of Wi-Fi as a contender for high throughput in-motion communication. The performance of communicating with stationary access points has been studied in a variety of different scenarios. There have been experiments on a high speed Autobahn, in the Californian desert, and on an infrequently travelled road in Canada where the environment and test parameters were carefully controlled. These works showed that a significant amount of data can be transferred while moving by access points along the road. The authors of  took this idea into the wild and reported on 290 drivehours in urban environments and found the median connection duration to be 13 seconds. This finding is very promising for in-motion communications. This could potentially allow large amounts of data to be transferred over currently under-utilized links without the use of expensive 3G connections. Previous work investigating performance of HSDPA (High Speed Data Packet Access), and CDMA 1x EV-DO (Code Division, Multiple Access, EvolutionData Optimized) networks show similar findings with variability in these data networks[10, 12, 8]. We also see this behavior in our experiments run on a HSDPA network. 5 Conclusion In this paper, we perform a comparison of two popular wireless access technologies, namely 3G and Wi-Fi. 3G provides continuous connectivity with low data rates and relatively high cost while Wi-Fi is intermittent with high bursts of data and comes for free when they are open. We experimentally show that with default AP selection techniques, off-the-shelf equipment, and no external antennae, we are able to opportunistically connect to open or community Wi-Fi APs (incurring no cost to the user) in an urban area and transfer significant amounts of data at walking and driving speeds. Intermittent Wi-Fi connectivity in an urban area can yield equivalent or greater throughput than what can be achieved using an “always-connected” 3G network. Wi-Fi could be easily modified to increase the number of successful opportunities. (1) Reduce connection setup time with APs, especially with community networks like FON that have a lengthy authentication process. (2) Clients could take advantage of Wi-Fi maps and real time location updates in order to choose which APs will provide the most benefit to the in-motion user. Finally, WiFi is bottlenecked by the ISP link and (3) caching data on the AP (both for upload and download) would eliminate the Internet back-haul link bottleneck. We are currently testing an improved in-motion Wi-Fi architecture that exhibits significantly higher transfer rates than 3G at all speeds. References 1. IEEE Standard 802.11: 1999(E), Wireless LAN Medium Access Control (MAC) and Physical Layer Specifications, August 1999. 2. IEEE 802.11n-2009, Wireless LAN Medium Access Control (MAC) and Physical Layer Specifications Enhancements for Higher Throughput, June 2009. 3. Bychkovsky, V., Hull, B., Miu, A., Balakrishnan, H., and Madden, S. A measurement study of vehicular internet access using in situ wi-fi networks. In MobiCom ’06: Proceedings of the 12th annual international conference on Mobile computing and networking (Los Angeles, CA, USA, 2006), ACM, pp. 50–61. 4. Eriksson, J., Balakrishnan, H., and Madden, S. Cabernet: Vehicular content delivery using wifi. MobiCom ’08: Proceedings of the 14th ACM international conference on Mobile computing and networking (2008), 199–210. 5. Gass, R., Scott, J., and Diot, C. Measurements of in-motion 802.11 networking. In WMCSA ’06: (HotMobile) Proceedings of the Seventh IEEE Workshop on Mobile Computing Systems & Applications (Semiahmoo Resort, Washington, USA, 2006), IEEE Computer Society, pp. 69–74. 6. Hadaller, D., Keshav, S., Brecht, T., and Agarwal, S. Vehicular opportunistic communication under the microscope. In MobiSys ’07: Proceedings of the 5th international conference on Mobile systems, applications and services (San Juan, Puerto Rico, 2007), ACM, pp. 206–219. 7. Han, D., Agarwala, A., Andersen, D. G., Kaminsky, M., Papagiannaki, K., and Seshan, S. Mark-and-sweep: Getting the “inside” scoop on neighborhood networks. In IMC ’08: Proceedings of the 8th ACM SIGCOMM conference on Internet measurement (Vouliagmeni, Greece, 2008), ACM. 8. Jang, K., Han, M., Cho, S., Ryu, H.-K., Lee, J., Lee, Y., and Moon, S. 3G and 3.5G wireless network performance measured from moving cars and highspeed trains. In ACM Workshop on Mobile Internet through Cellular Networks: Operations, Challenges, and Solutions (MICNET) (Beijing, China, October 2009). 9. Jones, K., and Liu, L. What where wi: An analysis of millions of wi-fi access points. In in Proceedings of 2007 IEEE Portable: International Conference on Portable Information Devices (May 2007), pp. 25–29. 10. Jurvansuu, M., Prokkola, J., Hanski, M., and Peral¨ a, P. H. J. ¨ HSDPA performance in live networks. In ICC (2007), pp. 467–471. 11. Kozuch, M., Ryan, M., Gass, R., Scholsser, S., O’Hallaron, D., Cipar, J., Stroucken, M., Lopez, J., and Ganger, G. Tashi: Location-Aware Cluster Management. In First Workshop on Automated Control for Datacenters and Clouds (ACDC09) (Barcelona, Spain, June 2009). 12. Liu, X., Sridharan, A., Machiraju, S., Seshadri, M., and Zang, H. Experiences in a 3G network: Interplay between the wireless channel and applications. In ACM MOBICOM (San Francisco, CA, September 2008). 13. Nicholson, A. J., Chawathe, Y., Chen, M. Y., Noble, B. D., and Wetherall, D. Improved access point selection. In MobiSys ’06: Proceedings of the 4th international conference on Mobile systems, applications and services (New York, NY, USA, 2006), ACM, pp. 233–245. 14. Ott, J., and Kutscher, D. Drive-thru internet: IEEE 802.11b for automobile users. In INFOCOM 2004. Twenty-third AnnualJoint Conference of the IEEE Computer and Communications Societies (Hong Kong, March 2004), vol. 1, IEEE, p. 373. (www.pam2010.ethz.ch/papers/full-length/8.pdf (R Gass))