styofa doing anything
šŖ¼

⣠Chile in a Photography ā£
Keni
trying on a metaphor
Show & Tell
2025 on Tumblr: Trends That Defined the Year

pixel skylines
Jules of Nature

JVL

blake kathryn

Janaina Medeiros

Origami Around
Peter Solarz
Lint Roller? I Barely Know Her

if i look back, i am lost
let's talk about Bridgerton tea, my ask is open
One Nice Bug Per Day
AnasAbdin
$LAYYYTER

seen from United States
seen from Türkiye
seen from United States
seen from Spain
seen from United States

seen from United Kingdom

seen from Malaysia
seen from Malaysia
seen from United States
seen from Canada
seen from Netherlands
seen from Singapore
seen from Poland

seen from Kyrgyzstan
seen from Bangladesh
seen from United States

seen from Germany

seen from United States
seen from Argentina

seen from United States
@wrongbat
Making water Kefir
A friend of mine started talking to me about Kefir, and how healthy it is. Apparently it's supposed to be the ultimate pro-biotic. That is, it stimulates the growth of the right kind of bacteria in your gut. I thought it'd be fun to try to make some, as I like making fermented foods, but I didn't know where to buy it in Tokyo (where I live). After googling around a bit, I found this Facebook group and asked where to buy it. A very kind woman offered to send me some grains just minutes after I posted in the group. So already the day after first hearing about Kefir, I had some in my postbox.
There're several kinds of Kefir. The bacteria live in little chunks called "grains". You can add the grains sugar water. The bacteria like to eat the sugar which they convert into other things, such as lactic acid and carbon dioxide. According to this site they even produce alcohol, yay! As they eat the sugar they also grow larger, so you will end up with more grains than when you started the fermentation process.
I started with with dissolving two spoons of brown sugar into 500mL water, and then I added two tablespoons of Kefir grains to that. I also added some sterilized egg shells as I was told they grains like calcium.
After six days of fermenting this stuff at room temperature bubbles had formed in the liquid, and I filtered the it through a colander to separate the grains. This is the finished product that Iām currently drinking.
Two two spoons of grains were now four spoons of grains, so Iām currently making 1L of drink. I also tried adding a bit more sugar this time to get more carbon dioxide, as I like fizzy drinks. Below is the new brew to the left, and the fizzy fermented drink to the right.
If you live in Japan and want to try out making your own water Kefir
zeromq Context destroy method
The pyzqm tutorial is very good for learning the basics of how to use zmq. I've learned a lot from reading it. Now I'm reading the PyZMQ API documentation because I feel that investing some time in learning the finer apsects of ZMQ is worth the effort. I'm discovering some handy tricks that I thought I'd share on my blog.
You know how you use a Context instance to create sockets? Well, the Context class has a very nice method for closing all sockets that were created by that context. Here's how you do it.
Here's a program that creates two servers and two clients that sends a message every second. After three seconds a timer kicks in and closes all
import zmq from time import sleep from multiprocessing import Process from threading import Timer, Thread context = zmq.Context() def server(name, port): server = context.socket(zmq.ROUTER) server.bind('tcp://*:{}'.format(port)) while True: print('{}: {}'.format(name, server.recv_multipart())) def client(name, port): client = context.socket(zmq.DEALER) client.connect('tcp://localhost:{}'.format(port)) while True: client.send_multipart(["", "From {} with love".format(name)]) sleep(1) def destroy(): context.destroy() if __name__ == "__main__": Process(target = server, args = ('server1', 5563)).start() Process(target = server, args = ('server2', 5564)).start() Process(target = client, args = ('client1', 5563)).start() Process(target = client, args = ('client2', 5564)).start() Timer(3, destroy).start()
However, this program won't work as intended. The clients will keep sending and the servers will keep printing what they receie. Why is this? Because we created the servers and clients as processes! When you create a process a copy is made of all the objects in scope. So the context object that the timer closed was never used to create any sockets! Aha, basic lesson in processes. Actually I made exactly that mistake :).
So let's create the servers and clients as threads instead:
if __name__ == "__main__": Thread(target = server, args = ('server1', 5563)).start() Thread(target = server, args = ('server2', 5564)).start() Thread(target = client, args = ('client1', 5563)).start() Thread(target = client, args = ('client2', 5564)).start() Timer(3, destroy).start()
This time the message flow stops, but not in a very nice way. The server threads throw an ContextTerminated exception, the clients throw ZMQError: Socket operation on non-socket exception, and then the program hangs. It's actually written in the docs:
destroy involves calling zmq_close(), which is NOT threadsafe. If there are active sockets in other threads, this must not be called.
So finally my conclusion is that context.destroy() is mainly useful when you have a single thread using many sockets. There are plenty of use cases for that. Here's a simple example where a single thread has two threads talking to each other (pretty useless, but just for the purpose of demonstation :)
import zmq from time import sleep from multiprocessing import Process from threading import Timer, Thread from itertools import count context = zmq.Context() def main(): server = context.socket(zmq.ROUTER) server.bind('tcp://*:5563{}') client = context.socket(zmq.DEALER) client.connect('tcp://localhost:5563') for i in count(): if i == 3: context.destroy() if not client.closed: client.send_multipart(["", "From client with love"]) sleep(1) print(server.recv_multipart()) else: break if __name__ == "__main__": main()
Why did I start learning zeromq?
I started out learning zeromq because I wanted to learn how to send messages between many programs running of different computers. I had never done anything like this before. I was aware that there was something called sockets (Iād heard of TCP sockets and so on), but learning how to use them properly seemed like a huge effort to me, so I never really got started. I kept thinking something like āsockets are for network engineers, which Iām notā.Ā
When I was at university I was interested in multi-agent simulation. Well, the idea of simulating complex systems using many simple agents cinhabiting an environment and communicating with each other seemed interesting anyway. For one reason or another I decided to try to simulate a financial market and figure out why flash crashed were happening. I think it was because Iād read āThe Black Swanā by Nicholas Taleb. His writing style felt inspiring to me at the time. So anyway, I started writing a simulation engine in Java in spite of being mainly a Python programmer. Something with strong opinions about the speed of Python vs. within the lab where I was studying. Anyway, I wrote the whole thing as a giant Java program to run in a single thread, and it became a monster, but one that worked for the purposes of doing my simple research. (check it out here). I graduated, and I thoughtĀ āthatās pretty much it for me and multi-agent systemsā.
I went and got a job in an IT company where I started learning about web-programming and having a fun time doing it. I didnāt think much about multi-agent systems until almost a year later. I started to think that I could maybe combine what now knew about web stuff and what I knew about multi-agent stuff, mix it and make something out of it. My first idea was to create a trading platform where people could play little trading games against each other or program algorithms to trade for them in the same games. Iām still working on that, and itās a much larger project than I first thought (usually this is what happens for me). But even though Iām just getting started Iāve already learned so much!Ā
My first attempt was to make a web interface that connected the user to a program running an auction. The web app was based on flask and used web sockets to connect the user to the auction. The proof-of-concept prototype was more or less working (for a single user trading with himself...), but I had no idea on how to expand it into a larger system with many auctions running at the same time. And then I started writing integration tests to make sure that what I had already made would continue to work in the future. And honestly all that extra effort was very demotivating to me, and I gradually stopped working on the app.Ā
Then again a few months later, one of my friends told me about this thing called zeromq. I looked at the front page of zeromq and didnāt really understand much of what was written there. Something about sockets on steroids and so on. To someone like me for who the concept of a socket was still pretty vague and scary it didnāt really make much sense. But I liked the friendly way that the documentation was written in, so I decided to give it a shot an bought the zeromq book. I have to say itās one of the only books on programming that Iāve actually enjoyed reading. Not that other programming books are not good, but theyāre usually pretty dry and devoid of humor. This is not the case with the zeromq book, and youāll often find yourself smiling when youāre reading it. Apart form the joy of reading a well written book, I also started to understand how incredibly useful a tool zeromq is.Ā
You know that feeling when you learn something, and that new knowledge makes you feel powerful? Thatās what I felt like when I was reading about zeromq. Basically zeromq is the first tools that has enabled me to feel some confidence about writing distributed systems. Of course Iām still facing many big issues, and writing a market app is still a lot of work, but thanks to zeromq I now have an idea of how to make it.Ā
Agent class using PyZqm and tornado ioloop
I created a simple class that will be the parent class for all the various types of agents in my multi-agent simulation. The class inherits from Process, so it needs to define a run() method. This run method will be called when the process is started.Ā
Agent has an abstract method, setup(), the implementation of which is done in the sub-class. In setup() the sockets are created, and the agent sends initial requests/init messages.
Iāll figure out how to deal with heart-beating stuff and reliability later. For now this class will do for prototyping applications.
Hereās the code:
import zmq from zmq.eventloop import ioloop, zmqstream from multiprocessing import Process import abc from datetime import datetime class Agent(Process): __metaclass__ = abc.ABCMeta def __init__(self, name): Process.__init__(self) self.context = zmq.Context() self.loop = ioloop.IOLoop.instance() def run(self): self.say('Setting up agent...') self.setup() self.say('Starting ioloop...') self.loop.start() @abc.abstractmethod def setup(self): return def say(self, msg): print('{} - {}: {}'.format(datetime.now().strftime('%H:%M:%S'), self.name, msg)) class Server(Agent): def handle_socket(self, msg): address, m = msg[0], msg[2] self.say(m) self.socket.send_multipart([address, '', 'REQUEST OK']) def setup(self): self.socket = self.context.socket(zmq.ROUTER) self.socket.bind('tcp://*:6000') stream_pull = zmqstream.ZMQStream(self.socket) stream_pull.on_recv(self.handle_socket) class Client(Agent): def handle_socket(self, msg): self.say(msg) self.socket.send_multipart(["", 'NEW REQUEST']) def setup(self): self.socket = self.context.socket(zmq.DEALER) self.socket.connect('tcp://localhost:6000') stream_pull = zmqstream.ZMQStream(self.socket) stream_pull.on_recv(self.handle_socket) self.socket.send_multipart(["", 'NEW REQUEST']) if __name__ == '__main__': Server(name = 'server').start() Client(name = 'client').start()
request/reply (DEALER/ROUTER) with zeromq IOLoop
In my previous post I tried out the tornado-based IOLoop with PUSH/PUSH zmq sockets. Hereās an example for for to use the asynchronous DEALER/ROUTER sockets:Ā
import zmq from zmq.eventloop import ioloop, zmqstream from time import sleep from multiprocessing import Process ioloop.install() context = zmq.Context() def server(): def handle_socket(msg): address, m = msg[0], msg[2] print('In server: {}'.format(m)) socket.send_multipart([address, '', 'REQUEST OK']) socket = context.socket(zmq.ROUTER) socket.bind('tcp://*:6000') stream_pull = zmqstream.ZMQStream(socket) stream_pull.on_recv(handle_socket) ioloop.IOLoop.instance().start() def client(): def handle_socket(msg): print('In client: {}'.format(msg)) socket.send_multipart(["", 'NEW REQUEST']) socket = context.socket(zmq.DEALER) socket.connect('tcp://localhost:6000') stream_pull = zmqstream.ZMQStream(socket) stream_pull.on_recv(handle_socket) socket.send_multipart(["", 'NEW REQUEST']) ioloop.IOLoop.instance().start() if __name__ == '__main__': Process(target = server).start() Process(target = client).start()
The main point is here that both client and server make an instance of the ioloop, so everything is event driven. Hence, in order to get messages flowing, the client has to send out a message before starting the ioloop. I'd like to try to code a PING/PONG heartbeating for reliability using this ioloop class.. (http://zeromq.org/deleted:topics:heartbeating)
IO loop in zeromq
I've been writing code for implementing a reactor loop with handler functions for sockets. The idea was that it should be easy to create agents with an arbitrary number of sockets, and which then have a central loop which polls the sockets and call the appropriate handlers. Of course this has already been done, and since my own efforts were not going so well I started trying out the tornado IOLoop. It's pretty awesome. Check out the example below.
import zmq from zmq.eventloop import ioloop, zmqstream from time import sleep from multiprocessing import Process ioloop.install() context = zmq.Context() def server(): socket = context.socket(zmq.PUSH) socket.bind('tcp://*:6000') while True: socket.send('Yo') sleep(1) def worker(): def handle_socket(msg): print('In worker 1: {}'.format(msg)) socket = context.socket(zmq.PULL) socket.connect('tcp://localhost:6000') stream_pull = zmqstream.ZMQStream(socket) stream_pull.on_recv(handle_socket) ioloop.IOLoop.instance().start() def worker2(): def handle_socket(msg): print('In worker 2: {}'.format(msg)) socket = context.socket(zmq.PULL) socket.connect('tcp://localhost:6000') stream_pull = zmqstream.ZMQStream(socket) stream_pull.on_recv(handle_socket) ioloop.IOLoop.instance().start() if __name__ == '__main__': Process(target = server).start() Process(target = worker).start() Process(target = worker2).start()
The example is based on code in the pyzmq documentation: http://learning-0mq-with-pyzmq.readthedocs.org/en/latest/pyzmq/multisocket/tornadoeventloop.html
Python multi key dictionary
Normal Python dictionaries have a simple one-to-one mapping, where a single key maps to a single value. However, sometimes itās useful to be able to access a value via several different keys. If you want to do that, you can use multi_key_dict. Just do
$ pip install multi_key_dict
Hereās an example of how this data structure works:
So basically you can specify multiple keys and get the value using either of them. Note that this is not the same as using a tuple in a normal dictionary, since in that case you will have to use the entire tuple to get the value.Ā
If you try to add a value with a key which overlaps with an already existing key, an exception is thrown, which means is significant since it means you can be sure that each key maps to a single value.Ā
One place where you need to be careful when using this data structure is that values are overwritten when placing a value referenced by a key which is part of a key already in the dictionary:
This is a bit confusing since 1 will suddenly map to ābā instead of āaā, even though this mapping was never explicitly entered in the dictionary. Fortunately the class contains aĀ method which you can use to know when this happens.
Connecting a zmq ROUTER socket to multiple peers
Using zeromq itās very easy to connect the asynchronous DEALER socket to more than oner peer. When you send a message on the DEALER socket, zmq will choose the least recently used peer to send it to. Iāve made a little example below to illustrate.
import zmq from multiprocessing import Process from time import sleep addr1 = 'ipc://broker1' addr2 = 'ipc://broker2' context = zmq.Context() def worker(): socket = context.socket(zmq.DEALER) socket.connect(addr1) socket.connect(addr2) while True: socket.send_multipart(["", "Hello"]) print(socket.recv_multipart()) sleep(1) def broker_1(): socket = context.socket(zmq.ROUTER) socket.bind(addr1) while True: addr, e, msg = socket.recv_multipart() print("In broker 1: %s"%addr) socket.send_multipart([addr, "", "hello from broker 1"]) def broker_2(): socket = context.socket(zmq.ROUTER) socket.bind(addr2) while True: addr, e, msg = socket.recv_multipart() print("In broker 2: %s"%addr) socket.send_multipart([addr, "", "hello from broker 2"]) if __name__ == '__main__': Process(target = worker).start() Process(target = broker_1).start() Process(target = broker_2).start()
Simple arithmetics with opamps
Iāve read that you can use opamps to do arithmetic operations like adding, subtracting, etc, but I was likeĀ āwtf, how?ā until I just stumbled into it by accident by tinkering with variations of the circuit below:
The opamp doesnāt really do anything here except inverting the signal. This is because of two sets of facts.Ā
First, Vin is a sine wave with a 5VDC offset and V1 is 5VDC. That means that when Vin = 5V, no current will flow through R1 and R2, which means that thereās no voltage drop over neither resistor. Then because the signal is connected to the inverting input of the amplifier, Vout = -Vin.
Second, since R1 = R2 and since the current that flow between two two resistors is the same, the voltage drops are also the same. So when Vin is at its peak value of 6V, this pulls a current of (6V - 5V)/10kĪ© = 100µA through R1. This same current flows through R2 and creates a voltage drop 1V, making Vout = 5V - 1V = 4V. Similarly when Vin = 4V, Vout = 6V.Ā
Multiplication and division
Itās easy to multiply or divide Vin. Simply change the ratio R1/R2. When R2 < R1, more current will flow through R2 than R1, and thus the voltage drop over R2 will be smaller than over R1, and you will have scaled down (divided) the signal.
When R2 > R1 the signal is amplified, that is, multiplied.Ā
Addition and subtraction
Now letās see what happens when we change the level of the voltage supply going into the positive input on the opamp.Ā
When Vin is 6V, a current of 100µA will flow, causing a voltage drop of 1V over R2. Then Vout = 7V + 100µA * 10kĪ© = 8V. Hmm. When Vin = 4V a current of 300µA will flow through in the reverse direction through R2, causing a voltage *gain* of 3V, making Vout = 10V. When Vin is 5V Vout is 9V.Ā
So weāve managed to add 4V to Vin! Thatās pretty awesome. You can subtract by setting V+ less than 5V (the mean of Vin).Ā
Simple way to use Python decorators
Probably the simplest way to use a decorator in Python is to do something before the decorated object (classes and functions) is defined. In the example code below the class A is never instantiated, but the code in the decorator is still executed. Also it's important to keep in mind that the code in decorator is executed exactly once each time it's used to decorate an object. If you want to make a wrapper function that does something before and/or after the
def decorator(obj): print "In the decorator now, where we can do something before obj is defined" return obj @decorator class A: pass
which generates the output
In the decorator now, where we can do something before obj is defined
This is useful if you want to register an object in a global data structure, like this:
animals = [] def register_animal(obj): print 'Registering animal: ', obj animals.append(obj) return obj @register_animal class Cat: pass @register_animal class Dog: pass class Teapot: pass print 'Registered animals: ', animals
which results in the output
Registering animal: __main__.Cat Registering animal: __main__.Dog Registered animals: [<class __main__.Cat at 0x10e538f58 >, <class __main__.Dog at 0x10e565120>]
So whatās the deal with the at-mark? Itās a neat syntax. What happens is this. When the interpreter reaches the @, it calls decorator(func) where func is the function f. It then replaces the definition of f with whatās returned by the decorator. In this case the original function. We can check this:
def decorator(func): print "In decorator: ", func return func @decorator def f(): print 'In f()' if __name__ == '__main__': print f
Output:
In decorator: <function f at 0x104432de8> <function f at 0x104432de8>
You can see that the address of func and f are the same. So really, thatās the simplest way to use decorators. In later blog posts I will show a few other examples of more advanced ways of using decorators. Mainly how to pass arguments to the decorator, how to create decorators that wraps the decorated function, and how to use classes as decorators (my personal favorite :)
A few LTSpice resources
This is mostly for my own reference.Ā
LTSpice keyboard shortcuts for OSX can be found here:
http://cds.linear.com/docs/en/software-and-simulation/LTspiceShortcutsForMacOSX.pdf
And this page has some good tutorials for learning how to use LTSpice. Iāve basically been following his tutorials until now:Ā http://www.simonbramble.co.uk/lt_spice/ltspice_lt_spice.htm
First time simulating opamp in LTSpice
Iām trying to learn how to use opamps, so Iām Ā starting with one of the simplest circuits, using a single-supply amplifier because I then only have to use one voltage source to power the amplifier. The circuit below uses negative feedback to amplify Vin. I wanted the input voltage to vary during simulation, so I changed it to a piece-wise linear function.Ā
To do this go to advanced properties of the voltage source (on mac you simply right click on the voltage source and pressĀ āAdvancedā). Then you choose the style PWL (piece-wise linear) and fill in the points as follows:
This makes the voltage source jump from 0V to 1V at 0.001s, stay 1V for another millisecond and then jump back to 0V. The figure below shows the simulation output, and it shows that the amplifier has a gain of 3.
one-to-one map in Python
Thereās a module called bidict which implements a one to one mapping. In a normal dict keys are unique whereas values can be duplicate. In a bidict both keys and values are unique. A key points to a single values, which points back to the same key. So just as putting a key-value pair in a dictionary where the key already exists will cause the old value to be replaced with the new value, so will placing a key-value pair in a bidict where the value already exists cause the old value to be replaced by the new value:
In [78]: b = bidict({1:2}) In [79]: b Out[79]: bidict({1: 2}) In [80]: b.put(3, 2) In [81]: b Out[81]: bidict({3: 2})
As for speed, bidict somewhat slower than a standard dictionary, but still pretty fast:
In [53]: d = {1:2} In [54]: b = bidict({1:2}) In [55]: %timeit d.get(1) 10000000 loops, best of 3: 134 ns per loop In [56]: %timeit b.get(1) 1000000 loops, best of 3: 363 ns per loop In [57]: %timeit b.inv.get(2) 1000000 loops, best of 3: 620 ns per loop
Writing a wrapper function WITHOUT using decorators
When I first started learning about Python decorators I found them pretty confusing an unintuitive, and it took me a while before I started using them. What I often want to do is modify a function with some extra functionality without having to add the extra code in my original function. Decorators are excellent for doing that, but if you find them confusing (which I did) then you can write a wrapper like this:
def wrap(agent): assert hasattr(agent, 'loop') lfunc = agent.loop def before_loop(): print('Before loop') def after_loop(): print('Doing stuff after loop') def wrapped(*args, **kwargs): before_loop() try: lfunc(*args, **kwargs) except Exception, e: print(e) after_loop() agent.loop = wrapped class TestAgent: def loop(self, n): for i in range(n): print('In loop') print(x) if __name__ == '__main__': a = TestAgent() wrap(a) a.loop(5)
if you run the script you will see that the before_loop and after_loop are also executed:
Before loop In loop In loop In loop In loop In loop Doing stuff after loop
The try/except block in wrapper() makes sure that after_loop() is called even if loop() throws an exception.
You should not forget to use a copy of the loop function (third line in the example above) as it will cause a RuntimeError due to infinite recursion. So this will crash:
def wrap_broken(agent): assert hasattr(agent, 'loop') def before_loop(): print('Before loop') def after_loop(): print('Doing stuff after loop') def wrapped(*args, **kwargs): before_loop() agent.loop(*args, **kwargs) after_loop() agent.loop = wrapped class TestAgent: def loop(self, n): for i in range(n): print('In loop') if __name__ == '__main__': a = TestAgent() wrap_broken(a) a.loop(5)
Python atexit module in multi-process program
Iām trying to figure out an easy way of knowing when . The python module atexit letās you specify a callback which is executed after the program finishes. To be precise the documentation states that
"The atexit module defines a single function to register cleanup functions. Functions thus registered are automatically executed upon normal interpreter termination.ā
So hereās a little example that I wrote to exactly understand what that means.
from __future__ import print_function from multiprocessing import Process, current_process import atexit def exit_handler(): print('In exit handler') def work(): print('(in {}) done: {}'.format(current_process().name, sum(range(10000000)))) if __name__ == '__main__': print('Before starting') Process(target = work, name = 'worker-1').start() Process(target = work, name = 'worker-2').start() atexit.register(exit_handler) print('Back in main')
Running the program will generate the following output:
Back in main In exit handler (in worker-1) done: 49999995000000 (in worker-2) done: 49999995000000
So you see that the exit handler is called when the interpreter exits the main script, but not after each of the processes are terminated.
Kill script without having to manually find the PID
For instance, find process running python script named market_v4.py and kill it:
pgrep python | xargs ps | grep market_v4.py | sed 's/s.*//' | xargs kill
The command works as follows.
pgrep python
shows the PID of all python processes
xargs ps
shows the pid and COMMAND used to start the process
grep market_v4.py
matches the line where the command is python market_v4.py
sedĀ ās/s.*//ā
subsitutes everything except the PID with nothing (leaving only the PID). And finally:
xargs kill
kills the process :).
Thereās probably a shorter way to do it, but I think this is pretty straightforward.