Aug 8, 2018
The blog touches on the essential aspects of Structure Streaming in Spark in a very basic form. It covers Structured Streaming, Spark Session, Schema, Console Sink and some other topics crucial to understanding Structure Streaming in Spark. Structured Streaming is a scalable and fault-tolerant stream processing engine built on the Spark SQL engine. The streaming computation is same as the batch computation as we will apply it on static data. It runs incrementally and continuously by Spark SQL engine which also takes care of updating the final result as streaming data continues to arrive.