#apache spark

[cecillian-hobbies]

FRAMEWORKS Y DEEP LEARNING

Un artículo que habla sobre las frameworks y deep learning, originado de mi investigación por la red para redes neurales artificiales

INTRODUCCIÓN Deep Learning es una rama de la inteligencia artificial que se basa en la utilización de redes neuronales profundas para el análisis de datos. Estas redes están compuestas por varias capas de nodos o neuronas, las cuales se encargan de procesar y analizar los datos de entrada. Una de las principales ventajas de las redes neuronales profundas es su capacidad para aprender y mejorar…

View On WordPress

[mongodbgui]

Data Science vs Data Analytics vs Big Data ☞ http://go.codetrick.net/372001407c #bigdata #hadoop

[nosql-master]

Taming Big Data with Apache Spark and Python - Hands On! ☞ http://bit.ly/2QS9WMk #bigdata #hadoop

[bigdatadataa]

Build a Custom Open Images Dataset for Object Detection ☞ https://bit.ly/3h2LcfV #bigdata #Data

[ejaz-talentscout]

Are you passionate about data science but not sure where to start?

Here are some of the key capabilities that are important to get into this field.

Mathematics: Strong foundation of Statistics, Algebra.

Learn to code: To start with learn Python and R and later learn Spark, Scala as per requirement.

SQL: You must be very strong in structure query language and analytical functions using SQL

Exploratory Data Analysis: Learn Data Wrangling / Data Crunching/ Data Aggregation.This you can do with the help of R, Python, and SQL.

Data Visualization: Learn how to Visualize data- using any visualization tool.

Learn the Art of Storytelling.

If you are good with all the above skills then get in machine learning algorithms, never directly jump into deep learning as you may get distracted.

Hope this helps.

[datascience-from-the-trenches]

Optimizing skewed operations in Apache spark

Suppose we want to compute an aggregate function f(x) over data grouped by city and state. This might results into a skewed workload for those executors that will end up processing keys (city, state) with high frequency in our dataset.

To rebalance the computation more fairly, we can add an additional column whose values are distributed uniformly to the data and use it in the group by. This simple but effective trick is known as ‘salting’.

Source: https://youtu.be/_ArCesElWp8

[unpredictablewoman1]

[datascienceunicorn]

Scalable Log Analytics with Apache Spark — A comprehensive case-study

Data Analysis & Visualization at Scale on Semi-structured Data