real-time volume processing

Top PDF real-time volume processing:

Real-Time Big Data Analytics using Hadoop

Real-Time Big Data Analytics using Hadoop

Abstract-- Recent Technology have led to a large volume of data from different areas(ex. medical, aircraft, internet and banking transactions) from last few years. Big Data is the collection of this field’s information. Big Data contains high volume, velocity and high variety of data. For example Data in GB or PB which are in different form i.e. structured, unstructured and semi-structured data and its requires more fast processing or real-time processing. Such Real time data processing is not easy task to do, Because Big Data is large dataset of various kind of data and Hadoop system can only handle the volume and variety of data but for real-time analysis we need to handle volume, variety and the velocity of data. To solve or to achieve the high velocity of data we are using two popular technologies i.e. Apache Kafka, which is message broker system and Apache Storm, which is stream processing engine.
Show more

5 Read more

Hybridization of Genetic Algorithm and Linear Programming for Solving Cell Formation Problem with Alternative Process Routings

Hybridization of Genetic Algorithm and Linear Programming for Solving Cell Formation Problem with Alternative Process Routings

In this paper, in order to increase the flexibility of manufacturing cells a mathematical programming model has been developed for solving a CF problem considering APR. This model can select more than one process route for each part. Also, several real-life production factors, namely operation sequence, production volume, processing time, workload balancing, machine capacity, and cell size, are considered. Due to the complexity and combinatorial nature of the proposed model, an algorithm comprised of a genetic algorithm (GA) and a linear programming (LP) is proposed for solving the model.
Show more

6 Read more

Real-Time Data Processing With Lambda Architecture

Real-Time Data Processing With Lambda Architecture

Data has evolved immensely in recent years, in type, volume and velocity. There are several frameworks to handle the big data applications. The project focuses on the Lambda Architecture proposed by Marz and its application to obtain real-time data processing. The architecture is a solution that unites the benefits of the batch and stream processing techniques. Data can be historically processed with high precision and involved algorithms without loss of short-term information, alerts and insights. Lambda Architecture has an ability to serve a wide range of use cases and workloads that withstands hardware and human mistakes. The layered architecture enhances loose coupling and flexibility in the system. This a huge benefit that allows understanding the trade-offs and application of various tools and technologies across the layers. There has been an advancement in the approach of building the LA due to improvements in the underlying tools. The project demonstrates a simplified architecture for the LA that is maintainable.
Show more

45 Read more

The 8 Requirements of Real-Time Stream Processing

The 8 Requirements of Real-Time Stream Processing

On Wall Street and other global exchanges, electronic trading volumes are growing exponentially. Market data feeds can generate tens of thousands of messages per second. The Options Price Reporting Authority (OPRA), which aggregates all the quotes and trades from the options exchanges, estimates peak rates of 122,000 messages per second in 2005, with rates doubling every year [13]. This dramatic escalation in feed volumes is stressing or breaking traditional feed processing systems. Furthermore, in electronic trading, a latency of even one second is unacceptable, and the trading operation whose engine produces the most current results will maximize arbitrage profits. This fact is causing financial services companies to require very high- volume processing of feed data with very low latency.
Show more

6 Read more

REAL-TIME MULTIMEDIA TRANSMISSION AND PROCESSING

REAL-TIME MULTIMEDIA TRANSMISSION AND PROCESSING

Abstract: The rapid increase in computing power and communication speed, coupled with computer storage facilities availability, has led to a new age of multimedia applications. Multimedia is practically everywhere and all around us we can feel its presence in almost all applications ranging from online video databases, IPTV, interactive multimedia and more recently in multimedia based social interaction. Key challenges to be addressed include specification of stream and resource characteristics, high demands on processing and Real- time delivery of multimedia streams, wireless communication between devices and transmission of streams, and architectures for the integration of numbers of devices from various manufactures with diverse demands and capabilities. In this paper, we are introducing where todays real-time multimedia applications are advancing. Then, we briefly introduces real-time, their classification, problems and techniques for their transmission. Then we studied the techniques used for Real-time multimedia signal processing, as a growing research field.
Show more

9 Read more

Thread-based analysis of embedded applications with real-time and non real-time processing on a single-processor platform

Thread-based analysis of embedded applications with real-time and non real-time processing on a single-processor platform

There is a large demand for real-time (RT) and non real-time (NRT) combinations in the market [DED]. In a typical automation system (AS) the RT part is responsible for controlling, data acquisition, signal conditioning and monitoring. The NRT part is responsible for data post processing, visualization, data persistence, system parameter settings and so on. In general, the NRT part is mostly used for the Graphical User Interface (GUI). Consequently, most NRT systems are based on up-to-date general-purpose operating system such as Microsoft Windows. In this context an important parameter is the interoperability which is defined as the ability to run NRT applications along with RT applications.
Show more

10 Read more

Real Time Video Processing For Drone-Based Lightning Sensor

Real Time Video Processing For Drone-Based Lightning Sensor

Lightning can strike in a variety of colours. The most common colour of lightning is white, but lightning can actually appear red, yellow, green, even blue or purple. Commonly people use flash sensor to do photograph lightning by using DSLR camera [2]. However, this method required expensive equipment and too heavy to carry on drone. Besides, the camera alone do not has any processor to do real time video processing to get satisfy output. The Raspberry Pi is a tiny and affordable computer that can use OpenCV (Open Source Computer Vision) to run video processing script. OpenCV is a library of programming functions mainly aimed at real-time computer vision.
Show more

24 Read more

Optimizing Resource Allocation for Approximate Real-Time Query Processing

Optimizing Resource Allocation for Approximate Real-Time Query Processing

The approximate query evaluation techniques were considered in the context of very large data warehouses and mobile networks [1, 4, 3, 7]. The approximation is typically based on sampling, wavelets, or synopsis. However, in both cases the query optimiza- tion was not extensively studied. For data warehouses, the dominating cost is produced by accesses to a single huge table, hence the performance depends mostly on the perfor- mance of a single operation, namely, data extraction from this table. For mobile networks, the queries are typically very simple and the optimization is not an issue. Note that the critical resource might depend on the context: in contrast with large databases and data warehouses, where time is the most important, the energy might be more valuable for mobile or sensor devices. Our approach does not depend on the nature of the resources to be allocated and might be applicable in both contexts.
Show more

20 Read more

A REVIEW: MAPREDUCE AND SPARK FOR BIG DATA ANALYTICS

A REVIEW: MAPREDUCE AND SPARK FOR BIG DATA ANALYTICS

45 | P a g e technology have been studied for decades. Representative open source systems include Spark, Storm .[9] The streaming processing paradigm is used for online applications, commonly at the second, or even millisecond, level.

9 Read more

Real Time Processing of Web Pages for Consumer Analytics

Real Time Processing of Web Pages for Consumer Analytics

A Software Engine is a self sustaining automated process that enables software in well defined way for its various components wherein it collaborates to work intuitively such that manual intervention in processing is minimal. It encompasses following features: Domain independence, Longevity, Reusability, Scalability, adaptability and incorporation of enhanced functionality. Other salient features include dynamic analysis, usage of minimal resources and applicability in diversified domains.

15 Read more

Data Structure for Real-Time Processing in 3-D

Data Structure for Real-Time Processing in 3-D

Dynamic processing of large 3-D point cloud data from ladar.k. Local computation on 3-D point sets.[r]

34 Read more

Embedded Real-Time Architecture for Level-Set-Based Active Contours

Embedded Real-Time Architecture for Level-Set-Based Active Contours

Methods described by partial differential equations have gained a considerable interest because of undoubtful advantages such as an easy mathematical description of the underlying physics phenomena, subpixel precision, isotropy, or direct extension to higher dimensions. Though their implementation within the level set framework offers other interesting advantages, their vast industrial deployment on embedded systems is slowed down by their considerable computational effort. This paper exploits the high parallelization potential of the operators from the level set framework and proposes a scalable, asynchronous, multiprocessor platform suitable for system-on-chip solutions. We concentrate on obtaining real-time execution capabilities. The performance is evaluated on a continuous watershed and an object-tracking application based on a simple gradient-based attraction force driving the active countour. The proposed architecture can be realized on commercially available FPGAs. It is built around general- purpose processor cores, and can run code developed with usual tools.
Show more

16 Read more

Java Real-Time Distributed Processing over Chorus/OS

Java Real-Time Distributed Processing over Chorus/OS

• byte arrays may not be allocated in the Java heap, but using native memory.. Real-time extensions (cont.)[r]

23 Read more

Weather radar information processing and real time flood forecasting

Weather radar information processing and real time flood forecasting

This thesis describes research into remotely sensed weather radar information systems and specifically addressesthree problems; 1 Weather radar data processing; 2 Real-time flood forecas[r]

300 Read more

Efficient Data Streams Processing in the Real Time Data Warehouse

Efficient Data Streams Processing in the Real Time Data Warehouse

Architecture for continuous queries over data streams is presented in [4] which provide a way to contain the maximum data streams in short memory. It divides the storage in four containers named stream, store, scratch and throw. Stream holds the continuous processing elements, store saves those streams which are to be required after short period. Scratch contains streams for use in future analysis. The data no more beneficial is disposed through throw container. A data streams solution extract them using queue networks [8] in which streams are stored in queues before processing. ETL performance is then evaluated using queue theory.
Show more

5 Read more

Real-time High Volume Data Replication White Paper

Real-time High Volume Data Replication White Paper

Several options exist to optimize HVR’s performance at the target location. The default mode of HVR is “trickle integrate” in which every transaction is immediately propagated but transactions are grouped into larger transactions for optimum performance (so long as there is enough transaction volume). On a busy system end- to-end latency is often seconds at most, and less than a second in many cases. Trickle integrate works well on an OLTP database that is optimized for fast inserts, updates and deletes.

22 Read more

Java Based Real Time Programming - Free Computer, Programming, Mathematics, Technical Books, Lecture Notes and Tutorials

Java Based Real Time Programming - Free Computer, Programming, Mathematics, Technical Books, Lecture Notes and Tutorials

Perhaps the most well-known usage of semaphores is for mutual exclusion. Any of the semaphore types used for signaling can be used for mutual exclusion, but there are reasons for providing a special mutual exclusion semaphore . Such a semaphore is sometimes referred to as a ’mutex’. We will call it a mutex semaphore . As de- picted in Figure 3.3, the very same thread that has called mutex.take(), thereafter having exclusive access to the shared data, should call mutex.give() when the crit- ical section is left. Any one activity not following this protocol, for instance by calling give one extra time before take, would damage the whole application. Therefore, it is a good idea to have a dedicated mutex semaphore class, in which the take saves a reference to the currently executing thread (entering the critical section) and give checks that the caller is the same thread (leaving the critical section). Of course, one could imagine situations when the unlocking should be taken care of by another thread, just like a person locking a room could hand over the key o someone else that takes care of unlocking, but that is simply not permitted.
Show more

128 Read more

Enhancing real time focus and context direct volume rendering

Enhancing real time focus and context direct volume rendering

This constant sampling rate is a reasonable assumption for slice based rendering but makes pre-integrated rendering unachievable in real-time for adaptive sampling techniques such as tho[r]

212 Read more

Imagery non-photorealistic art processing in modern product design relying on an inline block optimization hierarchy tree algorithm

Imagery non-photorealistic art processing in modern product design relying on an inline block optimization hierarchy tree algorithm

that is closest to the real world and has a high degree of truth [2]. Realistic rendering technology has become more and more mature with wide application. With the continu- ous advancement of computer software and hardware tech- nology, we are faced with images and display with higher and higher resolution. The computer-simulated image be- comes increasingly clear and close to the real world [3, 4]. However, when applying NRAP to image processing in modern product design, two major problems are faced: the computational complexity of image processing is too large and the storage of measurement matrices is huge [5]. The computational complexity of problematic image processing is too large. Image processing performance in exchange for computational complexity is reduced, but it is not ideal for image processing in modern product design [6]. As the measurement matrix storage capacity is huge, a structural random matrix is proposed to reduce the storage capacity of the measurement matrix. The whole image can be measured quickly. But the structure random matrix adopts the popular orthogonal transformation matrix (Fourier transform, discrete cosine transform (DCT), Hadamard transform). Ac- cording to NRAP non-correlation measurement theory, the matrix does not have good generality [7]. There is poor cor- relation between and independent Gaussian measurement matrix and any fixed matrix, so it can be used to design a common image coding strategy in modern product design. More recently, it was proposed that an inline block meas- urement and image processing phase combination, while using an independent and identically distributed Gaussian measurement matrix, avoids the huge amount of storage de- fects, making this method particularly suitable for real-time images [8]. In the framework of the combination of embed- ded block measurement and image processing, the use of directionality transformation to increase the image sparsity further improves the image processing
Show more

11 Read more

Amplify Serviceability and Productivity by integrating machine /sensor data with Data Science

Amplify Serviceability and Productivity by integrating machine /sensor data with Data Science

satisfaction. Frequent failures without any alarms were impacting the end customers and hence customers dissatisfaction was eroding market share. Many times its field engineers were struggling with prioritization and resource allocation . The OEM’s Competitors were leading with new IOT enabled solutions. To turn the tide, the strategy and operations team came up with an ambitious plan to enable digitization of elevator system. The concerned business team wanted to do the pilot with certain locations elevator connected to centralised monitoring system almost getting synchronized with each elevators in near real time.
Show more

13 Read more

Show all 10000 documents...