Timeframe: October 2017 – January 2021
Status: Completed
Funding: National Science Foundation, CNS-1718400 (see below for attribution)

The focus of this project was to explore how we could leverage the intelligent network edge to improve network performance.  In short, this work explored the extent to which we could leverage pro-active pushes, be those pushes being done locally (D2D) or remotely (trusted third party) to a mobile device or the network edge (ex. Mobile Edge Computing) to reduce network consumption. Specifically, this work aimed to explore several key approaches that include:

• Provider Accessible Storage Subsystem (PASS): What happens if we take the spare storage on a device and make it writable (accessible) to a trusted third party?  What if a content provider or middleware could push content to our device or a nearby MEC node to pre-stage content when the network is less busy?  Rather than reacting to content swings, could we trigger rules or choose when to push to the device.  Could we allow the usage to be highly malleable or to operate in a less than best effort manner whereby client usage always takes precedence and the allowed storage is as available?  What clever constructs could we make depending on the extent to which the mobile client / user is aware of such an approach? Are there interesting economic models that could emerge from such approaches?
• Whirlwind (D2D Exchanges): On the other extreme, what if we allow devices to pro-actively exchange previously seen content?  Could we exchange content via approaches such as Bluetooth, WiFi Direct, or LTE Direct whereby clients agree to share popular content when in proximity?  Leveraging our past work on opportunistic networking that shows reasonable availability, prevalence, and reciprocity, what should be exchanged in terms of content?  Should it simply be hashes which then has rules to trigger when to exchange or should the client pro-actively grab / fetch the objects or streaming blocks?  Should there be a manager or external entity to help coordinate these actions?
• Short Video Services: We began also exploring with this work various behaviors of short video services, particularly TikTok with regards to pre-staging due to significant prominence and how time-shifting is being leveraged in such platforms including TikTok and Facebook Watch.  In particular, we were also interested to study if adaptation occurs as part of the service and what type of staging criterion might be employed.  Notably, each of these services conduct staging during the first streaming video rather than pro-actively staging as we had envisioned with PASS.
• Adchestrator: One particular emerging consumer of network bandwidth is that of mobile ads.  With an increasing emphasis on engaging ads as well as rich, video-based ads, ads now constitute a non-trivial part of the underlying network bandwidth consumption.  For challenged networks such as those in dense venues (e.g. WiFi in stadiums, auditoriums) or with limited resources (cellular), this work has explored the extent to which we can leverage our work on PASS to pre-stage mobile ads.  In particular, our work has been interested in the extent to which time shifting ads both alleviates bandwidth bottlenecks as well as providing intriguing new opportunities with respect to network pricing (e.g. should an ad that is pre-staged fetch a different price).  This work is on-going and continues with our work on Fast Mobile Network Characterization whereby the longitudinal sampling becomes the driver in tandem with push-based operations via PASS.
• Hybrid Approach: To what extent could we blend such strategies between the extremes of the push from external parties to the localized D2D exchange?  What might effective strategies be with respect to information exchanges that yield a net positive in terms of network performance versus mobile device energy consumption?

People

The following individuals are supported / have been supported in efforts related to this effort.

• Prof. Aaron Striegel, Principal Investigator, Professor, University of Notre Dame
• Shangyue Zhu, Graduate Student – Data Collection, Software Construction, Simulation [passed proposal defense]
• Alamin Mohammed, Graduate Student – Data Collection, Software Construction [passed proposal defense]
• Poorna Talkad Sukumar – Graduate Student – Data Collection and Visualization [Union College]
• Lixing Song, Graduated with PhD – Data Collection [Rose-Hulman]
• Xueheng Hu, Graduated with PhD – Data Collection and Analysis [Amazon Lab 126]
• Zhongying Qiao, Graduate with Masters – Simulation + Data Analysis [Bay Area]

Publications / Datasets

• We have three datasets (Fall 17, Fall 18, Fall 19) available from captures taken from the WiFi that we provide that the University Development Tent.  The tent served up WiFi from 4+ 802.11ac access points to several hundred users over the course of several hours.  Taps and instrumentation were set up at various points and the following datasets can be requested for analysis by researchers following the signing of a DUA (Data Usage Agreement).
  • Packet timing / metadata (Layers 2-4) as captured from the wireless controller (inbound / outbound – Internet) which serves as the gateway for the WiFi service via NAT.
  • Accompanying SNMP data (all MAC addresses hashed) for many of the runs taken periodically via snmpwalk across the service period including details with regards to transmit / receive and varying phone models
  • The ability to run redundancy elimination algorithms on payload data (send us a Docker container and we can give you back the output / results)
  • During our data collection efforts, several weekends also had data gathered as part of the DARPA Spectrum Forensics effort in collaboration with Crane Naval Base.  Data as part of that effort was gathered both at the development tent (2x weekends) as well as at the stadium (2x weekends).
• Publications from this effort
  • A. Mohammed, “Network Instrumentation and Measurements: Shortcomings and Opportunities for
    Better User QoE,” Ph. D proposal, July 2021. [supported in part by this effort] [Send e-mail to striegel@nd.edu for a copy]
  • S. Zhu, “Exploring and Managing QoE in Short Video Platforms,” Ph. D proposal, May 2021. [supported in part by this effort] [Send e-mail to striegel@nd.edu for a copy]
  • Work under preparation on Adchestrator by A. Mohammed – to be submitted in late 2021 [supported by this effort]
  • Work under preparation on short video streaming services led by S. Zhu – to be submitted in late 2021 [supported by this effort]
  • Z. Li, Q. Liao, A. D. Striegel, “A Game-theoretic analysis on the economic viability of mobile content pre-staging,” Wireless Networking 26, 667–683 (2020). DOI: https://doi.org/10.1007/s11276-019-02176-3 [supported in part by this effort]
• Related effort by our group
  • X. Hu, A. Striegel, “PASS: Content Pre-staging through Provider Accessible Storage Service,” in of ICCCN, Vancouver, Canada, August 2017.
  • X. Hu, A. Striegel, “Redundancy Elimination Might Be Overrated: A Quantitative Study on Wireless Traffic,” in of INFOCOM IECCO Workshop (Integrating Edge Computing, Caching, and Offloading in Next Generation Networks), Atlanta, GA, May 2017.
  • Liao, Z. Li, A. Striegel, “On the Economics of Mobile Content Pre-Staging,” in Proc. of 5th Workshop on Smart Data Pricing (at IEEE INFOCOM), San Francisco, April 2016.
  • X. Hu, L. Meng, A. Striegel, “Evaluating the Raw Potential for Device-to-Device Caching via Co-Location,” in Proc. of MobiSPC, August 2014

Major Findings / Contributions

• Over the course of three years, we conducted redundancy elimination studies on data going to / from the wireless controller gateway at the University Relations (Development) tent.  This tent hosts several hundred individuals to which we offer courtesy high-speed WiFi (802.11ac) during a multi-hour tailgate for the purpose of data analysis as related to redundancy elimination and other research topics.  In earlier work analyzing redundancy as observed both in prior tailgates as well as a commuter train to / from Chicago, we have observed relatively lower levels than expected in terms of redundancy. Most importantly, we have observed further decreases as both video as well as HTTPS prevalence have dramatically redundancy.  Our general conclusion is that passive extraction of redundancy is unlikely to be effective by either in-line elimination or for any gains from D2D-based exchanges.
• Generally, D2D-based exchanges are unlikely to be significantly advantageous in practice for the purposes of significant reductions in bandwidth.  The sparsity of content makes it difficult to discern what to share (from a D2D perspective) and further complications from Apple / Google in terms of simple longitudinal app execution make it unlikely to this to be feasibly deployable / realizable.  As such, we conclude that push-based architectures with an eye towards pre-staging (MEC-driven or cloud-driven) for content likely makes more sense than on-demand content.  The sole exception remains synchronous content best done by multicast but experiences from our stadium venue point to even that synchrony being push-able (replays versus live watching).
• The notion of PASS is still a difficult one to realize in practice in large part due to fundamental security issues on mobile devices.  In short, the net win that one receives from providing flexible storage is likely overshadowed by potential security issues that could emerge in such an architecture.  We believe that an ad-based scheme offers a clearer economic incentive though that work is still more theoretical than practical at this time due to a fairly substantive paradigm-shift required to embrace such an approach.
• Short video streaming services conduct fairly extensive pre-staging as part of the normal video viewing process.  While the first video is being watched and has been successfully downloaded (or even in parallel), subsequent pieces of content for future videos are also downloaded.  Different services apply different approaches ranging from time-based buffer approaches (download X seconds of up to Y videos) to size-based buffer approaches (download X MB for up to Y videos).  Variations on services also include the depth of Y (from 1 to 3 or even 5 in past iterations).  While highly effective at improving the perceived QoE for a given user, there are potential larger implications for other users on the network.  In short, pre-staging hides the wait time (startup latency) for browsing the next video.  Moreover, such behavior also encourages fast browsing (watch a bit, swipe after a short viewing) as the user is not penalized with a large startup latency in order to download the start of a video.
• While we had intended to re-architect our ScaleBox code in Go to do redundancy elimination to a new / more flexible platform that will offer PASS services in a portable / easy to stand up context (Docker container for servers / MEC deployment), that effort unfortunately has not entirely panned out.  Work on our full framework does continue and we do still plan to release the PASS components next year (2022) via GitHub with the code released via the MS-PL (Microsoft Public License).

News

Support

This work has been supported in part by the National Science Foundation through grant CNS-1718400.