#memoryview

[simoluan]

(possibly) significant speedup coming up

So, I was trying to find a way to optimize the slowest parts of sims3py. And serendipity found me this:

http://eli.thegreenplace.net/2011/11/28/less-copies-in-python-with-the-buffer-protocol-and-memoryviews

This is the first time ever I heard about "buffer protocol" and "memoryview".

Reading the article and digesting the information, I suddenly realize that these pair of Python features could very likely be the key to a performant Python program!

You see, sims3py has to do a HUGE amount of slicing and dicing. Every invocation of slicing apparently creates a copy of the buffer. And if the copy doesn't neatly fall into the size of the memory page, performance will suffer.

After finding out about these two Python features, I see how I can speedup the operations by preventing memory copy. The article mentions a 300x speedup; but honestly, I'll take it even if it promises only 2x speedup.

I'm giddy with excitement! Stay tuned on the repo for some speed boostin' commits!

[inercia]

"buffer and memoryview are functions that allow direct access to an object's byte-oriented data without needing to copy it first."

[inercia]

A remainder on how to use Python's memoryviews for reading and sending data: For sending data, we can use [buffers](http://docs.python.org/2/library/functions.html#buffer): buf = buffer(message,0) while len(buf): buf = buffer(message,len(buf) + sock.send(buf)) and for receiving, we use [memoryviews](http://docs.python.org/2/library/stdtypes.html#memoryview-type) view = memoryview(bytearray(bufsize)) while len(view): view = view[sock.recv_into(view,1024):]

[kurtmckee]

Eli Bendersky explains how to speed up large memory operations by using buffers and memoryview objects.