Mar 11, 2018
Big Data examined
They also developed a workflow manager, Luigi, which they open sourced. Luigi is a Python framework for data flow definition and execution. Luigi is used to crunch a lot of data. Most of the data is user-centric data, such as billions of log messages that allows Spotify to provide music recommendations or select for example the next song heard on the radio. The data however is also used in decision-making, providing forecasting information and business analytics. According to the Jason Palmer from Spotify Labs, data is part of their culture.
via Datafloq