Streaming Algorithms

Top PDF Streaming Algorithms:

Coresets and streaming algorithms for the k-means problem and related clustering objectives

Coresets and streaming algorithms for the k-means problem and related clustering objectives

In the last section, we saw the Merge-and-Reduce technique, which provides a rather general way to design streaming algorithms. We have already heard about several coreset constructions for the k-means problem in Section 4.2.3. Most of these were used or could at least be used to design streaming algorithms with the Merge-and-Reduce technique using Theorem 5.1.2 or a similar result. There are three exceptions. Feldman, Monemizadeh and Sohler [FMS07] construct a weak coreset. Thus, they cannot use the principle directly but have to adjust it accordingly. The same holds for the streaming algorithm due to Feldman and Langberg [FL11a]. Even though their work contains the construction of a strong coreset, the stated streaming result relies on a weak coreset, and the authors do not state a result for the strong coreset. Frahling and Sohler [FS05] do not use the Merge-and-Reduce technique at all. They design a strong coreset that can be maintained in the Dynamic geometric data stream model, so deletions of points are also allowed. Their coreset construction uses statistics about the point set that can be updated under insertions and deletions of points.
Show more

269 Read more

Streaming Algorithms for k-Means Clustering with Fast Queries

Streaming Algorithms for k-Means Clustering with Fast Queries

The earliest streaming clustering method, Sequential k-means (due to [7]), maintains the current cluster centers and applies one iteration of Lloyd’s algorithm for every new point received. Because it is fast and easy to implement, Sequential k-means is commonly used in practice (e.g., Apache Spark mllib [10]). However, it cannot provide any approximation guarantees [11] on the cost of clustering. BIRCH [12] is a streaming clustering method based on a data structure called the “CF Tree”, and returns cluster centers through agglomerative hierarchical clustering on the leaf nodes of the tree. CluStream[13] constructs “microclusters” that summarize subsets of the stream, and further applies a weighted k-means algorithm on the microclusters. STREAMLS [3] is a divide-and-conquer method based on repeated appli- cation of a bicriteria approximation algorithm for clustering. A similar divide-and-conquer algorithm based on k-means++ is presented in [2]. However, these methods have a high cost of query processing, and are not suitable for continuous maintenance of clusters, or for frequent queries. In particular, at the time of query, these require merging of multiple data structures, followed by an extraction of cluster centers, which is expensive.
Show more

29 Read more

Streaming algorithms for bin packing and vector scheduling

Streaming algorithms for bin packing and vector scheduling

While these algorithms provide satisfying theoretical guarantees, simple heuristics are often adopted in practice to provide a “good-enough” performance. F IRST F IT [32], which puts each incoming item into the first bin where it fits and opens a new bin only when the item does not fit anywhere else achieves 1.7- approximation [16]. For the high-multiplicity variant, using an LP-based Gilmore-Gomory cutting stock heuristic [22, 23] gives a good running time in practice [2] and produces a solution with at most OPT + σ bins. However, neither of these algorithms adapts well to the streaming setting with possibly distinct item sizes. For example, F IRST F IT has to remember the remaining capacity of each open bin, which in general can require space proportional to OPT.
Show more

21 Read more

Towards a theory of parameterized streaming algorithms

Towards a theory of parameterized streaming algorithms

Figure 1 Pictorial representation of classification of some graph problems into complexity classes: our results are in black and previous work is referenced in blue. All results are for 1-pass deterministic algorithms on insertion-only streams unless otherwise specified. It was already known that k-VC ∈ FPS [13, 11] using only 1-pass, but here we design an algorithm with optimal space storage at the expense of multiple passes.

34 Read more

Mining and Learning in Sequential Data Streams: Interesting Correlations and Classification in Noisy Settings

Mining and Learning in Sequential Data Streams: Interesting Correlations and Classification in Noisy Settings

frequency of the parent. The reason for referring to such heavy hitters as “conditional” is by analogy to conditional probabilities: essentially, we seek children whose probability is high, conditioned on the parent. These should be distinct from the parent-child pairs which are overall most frequent, since these can be found by using existing heavy hitter algorithms. While this is a natural goal, it turns out that there are several ways to formalize this, which we discuss in more detail in Chapter 3. Thus, the first challenge is to formalize the notion of Conditional Heavy Hitters. As we consider streaming data that consists of symbols from possibly large alphabets, the second challenge is to accurately extract Conditional Heavy Hitters in real time with limited space budget. To this end, we developed several streaming algorithms that use pruning based on the value of the conditional probability. The core structure of the algorithms, which keeps estimated statistics about the frequencies of paren-child pairs, depends on the simple characteristics of data. The characteristics correspond to the expected size of possible parents that have at least one heavy hitter child. If the expected size of parents is small we call such a dataset “sparse”, otherwise, we refer to it as to a “dense” dataset.
Show more

159 Read more

Interactive Algorithms: Pool, Stream and Precognitive Stream

Interactive Algorithms: Pool, Stream and Precognitive Stream

The following lower bounds show that the number of iterations required by any stream emulator for utility-based algorithms must be at least quasi-linear in q. We provide two results. The first result considers a stream algorithm that selects exactly q elements, and it allows q to be large with respect to m. The second result considers stream algorithms that are allowed to select more than q elements. In this case, the lower bound is smaller, but it is reduced only linearly in the number of selections. It follows that a constant factor increase in the number of selections cannot overcome the quasi-linear dependence on q in the number of iterations. The proofs of the lower bounds are based on constructing a utility function which in effect allows only one set of selected elements for a given distribution, and forces the stream algorithm to select them in the same order as the pool algorithm. The first lower bound considers stream emulation with exactly q selections. The bound holds for both standard streaming algorithms and precognitive streaming algorithms.
Show more

39 Read more

B490 Mining the Big Data. 0 Introduction

B490 Mining the Big Data. 0 Introduction

– Finding Frequent Items in Data Stream : Implement streaming algorithms taught in class, and run them on large data sets to find frequent items. Compare the results with the true freque[r]

75 Read more

Towards HPC and Big Data Convergence in a Component Based Approach

Towards HPC and Big Data Convergence in a Component Based Approach

Contributions • Performance analysis of Big Data Tools • Time series data visualization • Parallel streaming algorithms • HPC integration to big data • Twister2 Big data toolkit.. • Unif[r]

78 Read more

Design of An Global Multicast Demonstrator for Live Video Streaming on Adobe s Flash Platform

Design of An Global Multicast Demonstrator for Live Video Streaming on Adobe s Flash Platform

Multicast has been proposed to implement at the application layer[7][18][23] and named Application Layer Multicast (ALM) in its early stage. In ALM, each client takes over the tasks of routers in IP multicast, such as replicating and forwarding of packets. Conviva 4 , a pioneer to implement ALM as the solu- tion to video streaming, has developed and deployed the live streaming system based on overlay multicast streaming protocols[17][38][40]. ALM provides low latency and in-order delivery of packets. However, the traffic depends on mul- ticast tree consisting of clients. It is very sensitive to node failure in its tree. ALM fits into the definition of Peer-to-Peer (P2P) system, such as delivering packets among all clients across Internet. Some P2P applications such as BitTorrent for file-sharing have gained tremendous popularity and business success. P2P has been proposed and implemented as the solutions for video streaming[24][46][16]. The early ALM based on multicast tree is also known as tree-based P2P multicast. Most of current P2P video streaming systems have utilized a mesh topology and are known as mesh-based P2P multicast.
Show more

100 Read more

White Paper OTT Streaming 2 nd edition, September 2011 Lionel Bringuier, CTO, Anevia

White Paper OTT Streaming 2 nd edition, September 2011 Lionel Bringuier, CTO, Anevia

Nowadays, HLS streaming is without any doubt the most widespread protocol used for OTT, as it is available on all Apple devices (iPhone, iPad, iPod...) as well as on some software players and a number of set top boxes. The keynote delivered by Steve Jobs on September 1st, 2010 was one of the first major events broadcasted live over HLS. It was also the day Jobs announced the second version of Apple TV, a set-top-box geared towards HLS streaming. The success of the iPad is largely based on users looking to use it for video application. A study by MeFeedia has shown that iPad owners watch three times more online video than traditional web users. Netflix and Hulu, who both offer Flash-based web sites, have launched their own iPad applications.. In July 2011, BBC launched the international version of its iPlayer iPad application allowing overseas subscribers to watch their favorite BBC shows over the top.
Show more

17 Read more

Streaming Locality: Streaming Media and the Production of Space and Subjectivity.

Streaming Locality: Streaming Media and the Production of Space and Subjectivity.

152 streaming providers without the data counting toward one’s monthly limit. T-Mobile’s “Binge On” plan, for example, offers users “optimized video streaming”, as well as unlimited streaming for services such as YouTube, Netflix, Hulu, and HBO (T-Mobile, n.d.). In addition to these major providers, the T-Mobile also partners with news, sports, religion, children’s content, and adult content with this plan (T-Mobile, n.d.). This interest in mobile streaming as strategy for both mobile service providers and streaming content platforms is unsurprising—the massive explosion in audio-visual streaming providers is part of an unsettled landscape in which viewers have begun to transition from the more traditional broadcast and cable models of television. Even for a service as popular as Netflix, there remain questions of just how many users the provider can continue to add, and this essential need to consistently expand is compounded by the company’s 20.54 billion in debts and obligations (Darville, 2017). However, Netflix has continually exceeded new subscriber expectations to the point that the streaming provider surpassed Disney as the most valuable media company in the world (Winkler, 2018; Sun, 2018). For Netflix, substantial credit for this achievement is due to mobile streaming practices. These streaming partnerships often function to create molar hierarchies, which in turn have impacts for how the subject streams. Those with access to more data, more areas of connectivity, and better partnerships have the ability to become a more effective mobile streaming subject.
Show more

238 Read more

The usage of streaming portals and copyright : a prevention of copyright infringement through legal streaming

The usage of streaming portals and copyright : a prevention of copyright infringement through legal streaming

Die bestehende technische Konvergenz ermöglicht dem Nutzer ein Ort- und Zeitunab- hängiges streamen von Medieninhalten. Legale Streaming Portale können auf allen In- ternetfähigen Endgeräten (Smartphone, Computer oder Smart TV) genutzt werden. Das Nutzungsverhalten der Internetnutzer zeigt, dass die Hälfte der deutschen Bevölkerung legale Video-Portale, sei es werbefinanziert oder beitragsfinanziert, zum Streamen von Medieninhalten benutzen. 78 Prozent der Personen nutzen kostenlose Portale, 17 Pro- zent dagegen nehmen den Dienst der kostenpflichtigen Video-on-Demand-Anbieter in Anspruch, wobei 41 Prozent illegale Plattformen bevorzugen. 223 Anhand dieser Nutzer-
Show more

75 Read more

Publisher & Contact Information

Publisher & Contact Information

Subsequent   Max  Polite  File   Load  Size   Subsequent   Max  User-­‐ Initiated  File   Load  Size Subsequent  Max   User-­‐Initiated   Additional   Streaming  File  Size Max  Video  &   Animation   Frame  rate Maximum   Animation   Length                       (i.e.  Flash™)

9 Read more

Citrix iForum 2005 Wrap-Up

Citrix iForum 2005 Wrap-Up

At iForum 2005, Citrix announced Project Tarpon as a desktop streaming solution. But is it streaming[r]

44 Read more

A Live Online Lecture System Using Adaptive Streaming Over HTTP

A Live Online Lecture System Using Adaptive Streaming Over HTTP

Due to above reasons, media streaming is nowadays more and more provided over-the-top (OTT) using HTTP streaming technologies. in order to further improve the performance and efficiency of media applications, Adaptive HTTP Streaming was proposed(Web-5), The basic idea is to chop the media file into segments which can be encoded at different bitrates or resolutions. The segments are provided on a Web server and can be downloaded through standard HTTP GET requests. The adaptation to the bitrate, resolution, etc. is done on the client side for each segment, e.g., the client can switch to a higher bitrates – if bandwidth permits – on a per segment basis. This makes the download behaviour of the client adaptive and dynamic to fit best for its given bandwidth. There are also other proprietary solutions from different companies like Microsoft’s Smooth Streaming [Web-6], Adobe’s Dynamic HTTP Streaming(Web-7) and Apple’s HTTP Live Streaming (Web-8) which more or less adopt the same approach.
Show more

8 Read more

QuickTime Streaming Server Darwin Streaming Server

QuickTime Streaming Server Darwin Streaming Server

In this example setup, it will be necessary to create a broadcaster user account, because the encoding software and QTSS are on separate computers. This allows a Session Description Protocol (SDP) file to be created on the server by the encoding software, which QuickTime Broadcaster does automatically if the Automatic Unicast (Announce) transport mode is selected. The SDP file provides information about the format, timing, and authorship of a live streaming broadcast. Once specified, the user name and password will be entered through QuickTime Broadcaster.

86 Read more

Survey of Streaming Data AlgorithmsSurvey of Streaming Data Algorithms

Survey of Streaming Data AlgorithmsSurvey of Streaming Data Algorithms

Each  node  of  a  decision  tree  contains  a  test  on  an  attribute  of  the  data  set.  This  test  determines  which  path  a   data  item  should  take.  A  decision  tree  classifier  sends  a  data  item  from  its  root  to  leaves  based  on  the  tests  at   each  node  along  the  path.  The  leaves  are  the  classes  of  the  classifier.  When  constructing  decision  trees  usually   the  algorithms  starts  from  an  empty  node  set  and  construct  nodes  based  on  the  attributes  that  do  the  best   split.    There  are  heuristics  like  information  gain,  Gini  index  for  choosing  the  attribute  that  does  the  best  split.   In  a  batch  algorithm,  because  all  the  training  data  is  available  the  algorithm  can  calculate  information  gain  or   Gini  index  for  each  of  the  attribute  to  choose  the  best.  For  a  stream  setting  because  it  cannot  access  all  the   data,   the   problem   is   to   decide   how   many   data   points   has   to   be   seen   before   deciding   to   split   based   on   an   attribute.   The   Hoeffding   tree   gives   an   innovative   method   for   making   this   decision   in   a   stream   setting.   The   Hoeffding  bound  [6]  is  a  statistical  result  used  by  the  Hoeffding  trees  to  achieve  this.    Let’s  take  a  real  valued   random  variable  r  whose  range  is  R.  Assume  n  observations  of  this  variable  are  made  and  computed  the  mean   𝑟.  The  Hoeffdin  bound  states  that,  with  probability  1 − 𝛿,  the  true  mean  of  the  variable  is  at  least  𝑟 − 𝜖  where  
Show more

10 Read more

Streaming Data Analysis using Apache Cassandra and Zeppelin

Streaming Data Analysis using Apache Cassandra and Zeppelin

Big data is a popular term used to describe the large volume of data which includes structured, semi-structured and unstructured data. Now-a-days, unstructured data is growing in an explosive speed with the development of Internet and social networks like Twitter,Facebook & Yahoo etc., In order to process such colossal of data a software is required that does this efficiently and this is where Hadoop steps in. Hadoop has become one of the most used frameworks when dealing with big data. It is used to analyze and process big data. In this paper, Apache Flume is configured and integrated with spark streaming for streaming the data from twitter application. The streamed data is stored into Apache Cassandra. After retrieving the data, the data is going to be analyzed by using the concept of Apache Zeppelin. The result will be displayed on Dashboard and the dashboard result is also going to be analyzed and validating using JSON.
Show more

8 Read more

On Erasure Coding for Distributed Storage and Streaming Communications

On Erasure Coding for Distributed Storage and Streaming Communications

The systems literature on real-time streaming deals mainly with the transmission of media con- tent (i.e., video and audio) over the Internet, with the user-perceived quality of the received stream as the performance metric. In practice, the encoding of the raw media content, packetization of the coded data (possibly with interleaving) for transmission, and application of forward error correction (FEC) codes are usually performed by different components of the system separately (e.g., [56,57]). FEC codes (e.g., exclusive-or parity [58], Reed-Solomon [59]), if used, are typically applied to blocks of packets to generate separate parity or repair packets (e.g., [60, 61]). Furthermore, the decoding delay requirement is not explicitly considered during the coding process. The patent of Rasmussen et al. [62] describes a system in which a live stream of data is divided into segments, each of which is encoded into one or more transmission blocks using an FEC code (e.g., LT [63], Reed-Solomon); these blocks are optionally subdivided and interleaved in a variety of ways be- fore being transmitted over one or more channels. A similar streaming system is also considered in the patent of Luby et al. [64], which describes computationally efficient methods for decoding FEC-encoded blocks to achieve low latency.
Show more

196 Read more

Traffic Pattern-Based Content outpouring Detection for trusty Content Delivery Networks

Traffic Pattern-Based Content outpouring Detection for trusty Content Delivery Networks

equipped egress nodes is an easy option to avoid leakage of streaming contents to outside networks. on this solution, the packet header records (e.g., destination and source net protocol addresses, protocol kind, and port number of outgoing visitors) of each streamed packet is inspected. In case the inspected packets do not verify the predefined filtering coverage, they are blocked and dropped. however, it's miles tough to completely save you streaming content material leakage by using packet filtering by myself due to the fact the packet header information of malicious users is unspecified in advance and may be without problems spoofed. on this paper, we recognition at the unlawful redistribution of streaming content material via an authorized person to outside networks. The existing proposals and display records acquired at exceptional nodes inside the middle of the streaming route. The retrieved facts are used to generate site visitors patterns which appear as unique waveform percontent,just like a fingerprint. The technology of site visitors sample does no longer require any data on the packet header, and therefore preserves the user’s privateness. Leakage detection is then completed by means of comparing the generated site visitor’s patterns. However, the existence of movies of different length in the community environment causes a full-size degradation within the leakage detection overall performance. Accordingly, developing an revolutionary leakage detection method robust to the variant of video lengths is, indeed required. in this paper, by using comparing special length films, we decide a dating between the duration of films to be compared and their similarity. Primarily based in this relationship, we determine decision threshold allowing accurate leakage detection even in an environment with one-of-a-kind length videos. The rest of the paper is organized as follows: a typical video leakage scenario, detection device and techniques are described. First we depict the disadvantage of the existing scheme due to the variant of video period in practical surroundings, then we defined the proposed leakage detection scheme, and we evaluate its calculation price in assessment to that of the existing scheme. Moreover, we evaluate the effectiveness and the accuracy of the proposed scheme with respect to one-of-a-kind duration films, and its robustness to network surroundings adjustments.
Show more

6 Read more

Show all 10000 documents...