gopherdex @gopherdex - Tumblr Blog

I'm explaining here how to speed up your web scraper in order to create a high-performance scraper in Go (Golang) by using concurrency.

#golang #concurrency #webscraping #parallelism #goroutines

#golang #channels #goroutines #concurrency

Interfaces specify what methods a type should have and the type decides how to implement these methods. Go interfaces are implemented implicitly.

#golang #interfaces #method signatures

We can get a list of files inside a folder on the file system using various golang standard library functions.

#golang #filepath #ioutil #os

There’s always a bit of friction when it comes to making JSON work with statically types programming language. One one hand, JSON data can be anything from a simple number, to a complex array of embedded objects. Working with a language like Go means you have to make the variable structure of JSON fit into structured variables.

There are two types of data you will encounter when working with JSON:

Structured data

Unstructured data

#golang #json #structured data #unstructured data #unmarshal

#golang #interfaces #gif #bytes

I'm developing an API client where I need to encode a JSON payload on request and decode a JSON body from the response. I've read the source code from several libraries and from what I have seen, ...

“It really depends on what your input is. If you look at the implementation of the Decode method of json.Decoder, it buffers the entire JSON value in memory before unmarshalling it into a Go value. So in most cases it won't be any more memory efficient (although this could easily change in a future version of the language).So a better rule of thumb is this:Use json.Decoder if your data is coming from an io.Reader stream, or you need to decode multiple values from a stream of data.Use json.Unmarshal if you already have the JSON data in memory.For the case of reading from an HTTP request, I'd pick json.Decoder since you're obviously reading from a stream.“ - James Henstridge

#stack overflow #golang #unmarshal #newdecoder #json #http

When Go was first released, there was a way to do some things in concurrency. As time has gone on, various things have changed. The Context package for one thing. :) This article doesn’t go into all of the ways of doing concurrency but will focus on one problem and take you through a few different solutions so you can see how things have evolved.

#golang #channels #goroutines #concurrency #cancelling