Snorting Code

random tech stuff

[snortingcode]

I’ve moved!

I’ve moved to: https://prasoonj.github.io/snortingcode/

[snortingcode]

Upset About Facebook’s Data Scandal? Move on!

The point we’re all missing with the Facebook data leaks and no one really wants to talk about is something that invalidates Facebook’s source of income and your purpose on this “social networking” website!

If Facebook is a platform for “friends” why are the political parties, big corporations, businesses even on it?

A platform for friends is bound to be personal. One would expect it to have access to some very intimate details that can be used in any form of manipulation if one intends to do that. A Trump election campaign is not my “friend”. I might really be expecting Narendra Modi to change India into a country that has gold plated roads and convert each citizen into a swatch Bharat candidate whose fart smells of raspberries but, Modi is not my “friend”.

These organisations have one agenda when they join Facebook - mass manipulation! If we express shock at the ways in which they’ve abused Facebook’s loosely defined data protection policies, we are merely engaging in self-deception.

This platform is doing exactly what it is meant to do - provide access to your personal information to agencies that intend to use it to manufacture consent. Each new feature that’s added to the platform has just one function - increase user engagement to the point that you are so addicted that you take stupid decisions.

I thought it was quite clear that this “free” platform was a trap. Honestly, I’m just surprised that people thought otherwise. If you “Like” a page titled “Trump for President”, you are engaging in active propaganda. Compare any of your friends’ Facebook posts with that of your favourite politician - does it ever look like you are looking at the daily activities of another human being? Do they even ever poop?! Or have sex, get drunk, have a crapy hangover, sound stupid and express gratitude when you correct them?

These are NOT YOUR FRIENDS! When you sign up to follow these personalities and corporations, you are signing a consent form to be ready to be manipulated! So, take responsibility for your stupidity and move on!

[snortingcode]

Let’s Settle The Technology Debate?!

Let me begin with a disclaimer: I’ve bet my career on Technology so, what I have to say about it is certainly influenced by a confirmation bias.

I came across a Facebook post by a friend, that talked about a Robot Chef - one can download recipes over the Internet and, have this giant pair of robotic arms prepare food for them. Seeing it in motion was a delight for me, others did not share the same feeling. 

There were essentially 2 common themes in the concerns expressed in the comment section:

This is going to kill the employment that my “maid” currently has - cooking meals for me and my family (Low skilled labor in India is very cheap so the richer people often have maids. Yes, this is India’s post-colonial irony!).

This would suck the joy out of a creative activity.

You could apply similar arguments to any domain where automation is taking baby steps. Let’s talk about them one at a time.

Employment in the age of automation

Yes, irrespective of what your friendly, neighbourhood entrepreneur tells you, automation IS eating into the employment scene. There are going to be fewer jobs around and currently held skill sets would become redundantly undesirable, irrelevant. This is already happening in areas where automation has gone beyond the “baby steps”, although, this is nothing new!

There are 2 solutions that come to my mind (killing new technology is not a solution and, it would require nothing short of a world war followed by a frozen nuclear winter to do that - I don’t recommend doing that although your favourite world leader might urge you to support that idea):

Explore and create new skill sets: Remember that “maid” that you are concerned about? Start investing in their future, urge them to take up evening classes to learn whatever they are capable of learning - soft skills, a new language (English in India is a sought after skill), elementary coding skills if they have some basic education.Make sure that you are paying them enough so that they can create “post-retirement” means of living for themselves. Help them with investments and future planning.

Urge your governments to increase spending on things that matter -> health and education: If you live in a country where education and healthcare is not a fundamental right of every citizen, there’s something broken about your country! Vote to repair that scheme of things. A minimum basic income scheme is also something that governments should start to explore actively!

Technology = End of creativity

Ask any techie you know and they would tell you otherwise! But, I blame the tech world for this misconception. People think that technology is not a creative endeavour, possibly because tech sits on top of some ivory tower which reduces their customers to a UUID.randomUUID(). The tech world has largely failed to educate their users about the technology they use on a regular basis. We don’t have to be in this ignorant state! The technology we create doesn’t need to be “dumbed down” but, we need to stop thinking of our users as “dumb users” (I’ve heard that phrase a number of times in UX design discussions).

For the non-techies reading this article, well, just because we use tools that most of you don’t have much experience with, does not really make us less creative. Imagine telling a violinist that they are not in a creative profession because true creativity is just vocalisation of ragas! In fact, the artists producing electronic sounds are facing similar questions around their work. (Don’t) watch this video that trivialises Alt-j’s music and you’ll understand my point better (they have over 8 million views!) I’m certain they won’t say the same thing about a vocalist!.

This is insane! If you think that technology is not a creative activity/profession, try surrounding yourself with all the failed tech products that you can think of, for a day, here’s a list you can begin with: Windows 7, Orkut, Windows Vista, Zune (not many companies have the money to market failed products hence that list is heavy with Microsoft). 

In fact, creativity is the litmus test for a tech product! When we hire employees to build the next generation technology, we prefer people who can exhibit creative thinking ahead of any other skill that they might posses. The reason for that is simple, we don’t want to copy existing products, we want to build things that don’t exist yet. Imagining such products is impossible without creativity! Yes, the robotic companies are “just copying” basic human capabilities but, with that reductionist argument, Da Vinci was just copying some woman sitting on a chair!

Final word

If there’s one thing I want to convince you with this article, it is this: Whichever side of the fence you are on in this technology debate, you need to know what’s going on on the other side! 

If you are creating technology, you need to be able to effectively address these fears that your future customers might have.

If you are a user (and you are! There’s no escape, the matrix has you :D ), engage with technology the way you engage with your local politician. If you think that things are broken, educate yourself enough to be able to dig out email IDs of people who make these tech products and write to them! They owe you an explanation!

[snortingcode]

Cooking yummy Functional code - (partial, curry, compose)!

[This is part of a series of posts on Scala and Functional Programming: Part 1: Scala Tricks - tupled/untupled

Part 2: Higher Order Functions

Part 3: Cooking yummy Functional code - (partial, curry, compose)!

For a basic introduction to Scala, syntax, etc. you can check out Scala Fast Lane ]

I don't intend to turn this post into a click-bait article but, I find these 3 Functional Programming principles to be fundamental to understanding FP it well. Without them, you would write imperative code in your favorite functional programming language!

You would notice that they weave into each other and produce some rather beautiful imagery about FP.

Partial Functions

Let's begin with Partial functions. The idea is neat, if you have a function that takes several arguments, you can convert it into a function that that takes fewer arguments. You produce this magic by creating a function that fills-in the rest of the argument values and returning the same function.

In the example below, we'll take a function f: (A, B) => C (takes 2 arguments of types A and B and returns a type C) and convert it into a partial function that takes the value for the first argument and returns a function that just needs the second argument to produce the C that you needed!

scala> def partial[A,B,C](a: A, f: (A,B) => C): B => C = (b:B) => f(a, b) partial: [A, B, C](a: A, f: (A, B) => C)B => C

[If the parameterised type signature looks scary to you, you should read more about it and come back here. It is there for a reason. It emphasises the need for functions like these!]

Currying

Next, let's look at currying. A (rather silly) analogy to understand currying is to think of yourself as an indian housewife making a great indian curry for your family in the evening! The number of ingredients that you would need can be intimidating so there are 2 things you do -

Combine few ingredients together so that they are easier to handle (the typical off-the-shelf sambar poweder!).

Add them one after the other till you get the curry your family loves!

Currying converts a function that takes N arguments into a function that takes one argument. This new function returns another function. If we go back to our analogy, each 'step' in this currying process is a function application that takes you closer to your goal - the yummy result of a function that needs a ton of arguments.

scala> def curry[A,B,C](f: (A, B) => C): A => (B => C) = (a: A) => (b: B) => f(a, b) curry: [A, B, C](f: (A, B) => C)A => (B => C)

There are reasons to confuse currying and partial functions. Here's some insight that might help:

Look at the type signatures of the two:

partial: [A, B, C](a: A, f: (A, B) => C)B => C vs curry: [A, B, C](f: (A, B) => C)A => (B => C)

In partial, you are "applying" one of the arguments so the returned function is free of A. Its job is done! But, in curry, you get a function that needs just one of the arguments and passes the buck to the function that it then returns!

Think of a partial application as "forgetful" and currying as, well, like I said, the curry recipe will always be able to tell you how you reached there!

Function Composition

The last piece of the puzzle here is function composition. Well, TBH, I'm rather certain that you are already quite familiar with function composition so, I'll include it here just for completeness (in some shallow sense :D )

The only analogy you need here is the idea of pipes from your terminal/shell! You "pipe" the output of one function as input to another function and you have function composition.

scala> def compose[A, B, C](f: B => C, g: A => B): A => C = (a: A) => f(g(a)) compose: [A, B, C](f: B => C, g: A => B)A => C

This is the same as g compose f or f andThen g, where compose and andThen are utility functions provided by scala, which might make you notice that we did not use any magic function to explain what partial and curry were all about! Well, the magic of FP is probably hidden in the fact that there is no magic. We could use the elementary idea of higher order functions and build everything from there!

[snortingcode]

Are you interested?

Salespersons ruin everything! Adding to their long list of spoils is the phrase “Are you interested?”. If you’ve ever had a person call you about a bank loan, credit card or a new insurance policy, you know what I’m talking about. I remember when the phrase “Are you interested?” was used to express deep, emotional relationships. “I’m in love with you. Are you interested in me?”, “We are going for a trek next Sunday, are you interested?”

There was a purpose about that phrase. Being interested in something was just that - being genuinely interested in it. Now, thanks to the cold calling of salespersons (and that “Interested” button on Facebook events), it is just an euphemism for “Have I got your attention?”. Now, those two are very different things. A dog wagging his tail gets my attention almost always! That does not mean that I’m ready for a commitment, a relationship with him no matter how cute I find him (and dogs, truly are awesome!) “Have I got your attention”, to most people, feels too superficial for some reason.

I recently connected 2 of my contacts over a phone call for a business venture. My motive to get this relationship going was to have the pleasure of seeing a business flourish, observe how small businesses interact with their customers and, use that learning for my own startup. The response after this introduction was - “Yes, I’m interested!” (possibly without the “!”). I was quite happy with my efforts, hoping to make a difference in someone’s life when I heard that. If they are interested, they would make things work! However, this turned out to be one of those “You’ve got my attention” cases. Fizzled out in a few days!

Well, here’s what I do: If I’m interested, I make an effort to get things rolling. I call, message, reach out, talk, break walls, build bridges. If I’m not, I say, “Sorry, that does not interest me” and I stop right there. If an idea is not interesting enough to be worth a dream, I don’t keep it dangling in my brain. There are always much better things to look forward to. I’m not a great judge of ideas TBH. I’ve missed out on a lot of great opportunities that would have been great ventures I’m sure. But, I don’t think I remember many that I chased with all my energy and was disappointed about my effort. That for me is something I would live with - an honest effort or an honest denial.

The most crucial thing to learn for me right now is to identify people who mean it when they say, “I’m interested!”

[snortingcode]

Competition, Insecurity and Scientific Method

I’ve noticed a pattern lately. At the risk of over-fitting, let me share what I think, so I can learn from your experiences as well.

Competition has been showcased as the cornerstone of human progress. “We are getting better every day because we compete with each other”, has become the golden mantra that guides policy making, relationships at work, even family ties at times! I don’t want to go into how this idea has been used to justify unfair business practices, justify neoliberal economic policies (with a completely circular argument!), ignore the privileges that govern so much of our lives and ultimately gives rise to a false notion of “free will”!

I wish to talk about something more insidious. Possibly, even try and make sense of why the world seems broken, why relationships seem fake and why it is so difficult to find true love! _ _

I believe that competition is a facade. The real evil is a deep sense of insecurity that goes unnoticed. Hiding behind this facade are all sorts of unsorted issues that, over the years, shape into a complex entangled mess of falsehoods which can be impossible to get rid of without a focused effort.

Let’s start with some personality traits. A typical insecure person described by your psychology textbook would be someone with a “lack of confidence”, “lack of protection” which is what it most certainly is. However, people have deception as a survival skill. We won that fight with the gang of wolves because we could convince them that we were stronger! We practice deception from an early age. The way animals engage in a “fight-play” routine, we hone our deception skills. Over the years, we get really good at it. We acquire such excellent deception skills that we are able to deceive ourselves! Insecurity is often not dealt with inspection/introspection, it is given a lethal dose of deception. Such insecure people exhibit high confidence, a fake sense of superiority and an attitude that gives them “that edge” that’s so palpable in company boardrooms. They are the ones that “get things done” (but, they never do these things! They don’t know how to!)

Such insecure people who have learned to deceive themselves come with a myriads of unhealthy traits. This fake sense of superiority’s first victim is the willingness to try out new things. Kids don’t care what they look like, how rich or poor they are, how tall they are, the color of their skin - and they know how to participate even when they are sure to fail. That’s how all of us took our first steps! We failed again and again (with a lot of people watching!) We are talking about people who don’t try new things because they might fail. They would demean, insult and try self-deception to get out of knowing about something new because that’s easier than facing failure.

There’s a thing about science - it doesn’t care about our societal status! Science can tell even the most influential person that they are wrong. The people we are talking about then, do not have a scientific method. They might be scientists, scholars, doctors and engineers but, they are scared of science! These are people who are most gullible when it comes to “ancient wisdom”. These are people who would tell you that “science does not have all the answers” to justify a scientifically incorrect view that they hold close to heart. Since these are people who are all about authority, they find it difficult to defy authority. Deliver an authoritative judgement about an issue and they would take it to their graves. These are personality worshipers, they follow ideologies over methods. For them science comes in 2 flavors - “my science” and “their science”. One is wrong and the other is right. These are people who have a “right way to drink alcohol”, “authentic recipes”, “correct grammar”, “the best workout”, “traditional is better” approach to life!

Another classic identification is the victim game. Such people find victims for their wits, sarcasm, wisdom and knowledge (Think about it! Victim of knowledge!). They thrive in groups that they can control and they always have a victim who is miserable. When the tables turn, things become ugly! The are always on the lookout for issues with others around them and they try their best to expose them. While they try their best to hide their own insecurities, they use all their social intellect to identify others’ insecurities and use them against their victims. They are usually very good at the insecurities game because they swim in it more often than others. A good way to know this is to compare their behavior in a group versus when they are alone with you (alone is generally a lot nicer and very likeable).

Such people often have very magnetic personalities. Abusive relationships have this character as well. It often becomes impossible to leave the tormentor because they are so good to you at times! So good that you love them! When these people are not around, we terribly miss them. Remember, they are very good at reading insecurities!

These people are easy to spot online as well - #onlyWayToLive #beLikeMe #alwaysTheBest and sometimes, #worstPlaceOnEarth #cantHearThisCrappyMusic #whyAreAllPeopleSoStupid (notice the “all or nothing” feeling there? You should read more about it!)

Well, that said, what’s the solution? Should there be a large scale LSD-in-water-supply project to repair the world? I can offer my insights about my own battle with insecurities, anxiety and depression. The solution depends on which side of the facade you are (I’ve been on both and its not fun at all!).

Let’s start with when you are a victim.

Victim cheat code:

Be aware of yourself

Look for signs like limited social interaction, limited participation in social activities

Follow the scientific method religiously (see, what I did there :D )

Don’t take yourself too seriously (check out Dudeism!)

If you feel things are going out of hand, avoid or confront the people in your life who you think might need help in dealing with their insecurities and anxiety (let experts handle it though).

A good question to ask is - “Can I be completely honest around this person?” If the answer is “No”, there’s a problem that needs to be addressed.

Things are a lot more complicated when the facade has caged you inside a growing mass of insecurities.

Be honest! Avoid even white lies (no they are not required to live in the world!)

Be sincere in your judgement of yourself. Be open about failure, about being wrong.

Turn that competition inward. Compete with your own self and start by attempting to fail without feeling miserable about it.

Talk about mental health with your friends and family. Depression and anxiety affects us all on some level.

Putting yourself in others’ shoes is a great technique to still “get things done” while being nice to others. Try it as often as you can.

Talk about your insecurities (even the most private ones!) Talking about things engages your prefrontal cortex (the region of your brain that makes you human) and eases the fears and anxiety that your amygdala (the reptilian brain) is constructing.

Try to be more mindful. Meditation is great (try to learn it the right way and avoid mixing it with religion/spirituality - see the next point)

Try to be more flexible and open to new ideas and changes in life. “Ancient wisdom” belongs in the ancient times.

[snortingcode]

How to gain > 4 extra hours everyday!

[You can expect me to write a lot more about productivity in the following months as I prepare fellow teammates to have long streaks of productive runs!]

Everyone can use some extra time each day! Let’s jump right in:

Time your shower. Also, limit the time you take to decide what to wear for the day. On an average, people might spend between 30 minutes to 1 hour on this daily activity. Reduce it to under 10 minutes. Time saved: 15 Minutes!

Replace music with earplugs. We listen to music, while working, to drown out the noise. This might not seem like a distraction since we are so used to it, but in the end you’ll end up taking more time to do every task. Buy some good quality earplugs and create a 25/5 minutes rule - work with earplugs for 25 minutes, then take a music and water break for 5 minutes (also, walk around to get those steps!). You will find your productivity increase (you’ll enjoy music better too!) Calibrate your time saved but, my experience is that your productivity would go up by at least 20% -> for an 8-10 hour work day, you should be able to save about 2 hours! Time saved: 2 hours!

Eat with a (very) small set of people. Eating time is meant for eating. Eating alone can be difficult for a few people but, ensure that you avoid big groups while going for lunch/dinner. Bigger groups at the lunch table result in less time spent on eating and more spent on banter. This is REALLY BAD for your health as well! When we chit-chat while eating, we are more likely to over/under eat and in general, not enjoy our food a lot. If a discussion is happening over lunch, ensure that the discussion starts and ends before everyone begins to eat (remember to not start the discussion before the order is placed!). If you eat 3 meals a day, you can save an average of 15 minutes per meal. Time saved: 45 minutes!

Shut-down at least an hour before your sleep time. This might sound counter-intuitive but, we waste a lot of time sleeping because we don’t have regular sleeping times. When you shut-down an hour before sleeping and engage in things like meditation or take a cold shower and prepare for bed, you can easily reduce the twisting-turning time that spills over to the low productivity levels early in the morning. This alone can add several minutes (even hours!) to your day so I won’t count this in. Time saved: A LOT!

Have distraction-free family time. We do the things we do for our loved ones! Quality family time is not just about the time spent together. If we are not really with them, when we are together, we would be forced to sit through a crappy movie, forced to respond to WhatsApp texts while we are at work and forced to do several other things that never actually feel like we’ve spent time together at all. A good, engaging discussion after finishing dinner is far better than time spent together watching NEWS! Do this and you’d see not just having more time on your hands but a much deeper connect with people you love! I won’t time this one either because family deserves all the time after all! Remember, no phones, no TV, no work calls, no other distractions when you are with other people - they deserve your time! Time saved: A LOT!

Switch off the Television; throw it out of your house. After Vietnam’s disasters, one general of the US army commented - “We should’ve just given them free Television sets!” If there’s one mode of misery that connects all of humanity today, it is the tele! Television doesn’t just make you dumb every minute you spend time on it, it actually sets you up for all sorts of nasty social disasters as well. Let things of importance come to you! A neurotic tweet by a deranged person is not worth your time! A prime-time discussion about that tweet might sound like “intellectual orgasm” but, it is absolute garbage! Switch off the TV and you easily save several hours a day! Time saved: 1 hour (or 5 if you like to keep yourself really “informed”)!

There you go! These are not difficult things to do and they certainly don’t turn you into a robot! If anything, these would improve the quality of your personal and social life!

[snortingcode]

Hacking My Biology

Have you heard about bio-hacking? Depending on who you heard it from (and how much they love their alcohol), you might think that it is a story leaping out of a Black Mirror episode or that it is the best thing to ever come to your life/lifestyle.

Let’s get done with the commonly perceived ideas first. The essentials of bio-hacking include things that you can do - invasive/non-invasive/supplements/drugs/lifestyle changes  - to make everlasting improvements to how you feel everyday (both mind and body). You would hear about Nootropics like Modafinil, MDMA, etc. You would also hear about crazy lasers looking into your scalp, funny looking eye-wear, thermal jackets that let you adjust the temperature of your body from an app so you can sleep better!

Well, to be honest, those are the holy grail of bio-hacking. If you can pop 50mg of Modafinil everyday without it affecting your health (kidney, liver, etc.), you can get a lot more done in a regular day than most people around you. But, to get to those super-human mental abilities, you need to make some dramatic changes to your lifestyle so that your body is in a supreme state of health to be able to support a brain that’s working more efficiently than ever before! Implants, neural-link, and every crazy thing comes later.

I like to think of bio-hacking as a mechanism to optimize every aspect of my life. Every part of my body needs to be working at an optimal level. Let me break it down:

Sleep

I don’t need to convince you how important a good sleep is. But, let me use this topic to introduce compound interest to you. Just like the money on CI in a bank, every night of disturbed sleep feeds into a continuous cycle of doom! Each night you don’t sleep well, you add poison to your body. This CI principle applies to every aspect of bio-hacking. Every good thing you do to your mind/body adds up, every bad thing adds up as well!

Here’s my strategy for sleep which is really something your grandmother would agree with - sleep for at least 8 hours a day, sleep everyday at the same time, don’t sleep on a full stomach. That is the simple part. Now, let’s add an OCD type finesse to it:

I wear blue-light blocking glasses several hours before sleep (blue light reduces the production of melatonin in your body which is how your circadian rhythm regulates sleeping and waking hours.

I take ice-cold water showers just before sleeping - your body temperature needs to be lowered to prepare you for sleep, I induce it artificially.

I don’t eat anything after 19:00 hours. That gives me enough time to enter a ketosis state right when I go to bed - the brain needs the best quality nutrition during sleep.

I also use ear-plugs several minutes before I hit the bed. I sleep wearing them (you’ll get used to it in 2-3 days!)

I stop indulging in my phone/laptop at least a few hours before I sleep.

At least an hour before I sleep, I switch off even the minutest of LEDs (that uber-cool sleep light of your MacBook that resembles your breathing rhythm could wake you up in the middle of the night without you knowing that you woke up!)

I use an app, SleepCycle, that monitors my sleep. (It also measures my resting heart rate every morning!)

I don’t drink coffee after 13:00. With my high metabolism, that’s the best I can do. Anything later than that would affect my sleep!

Several bio-hackers take sleep inducing melatonin sprays, etc. to get a good night’s sleep.

Eat

Another obvious topic but, when you add the bio-hacking optimization zeal to it, it turns into a very powerful tool. Things you eat power your brain’s health. Eat healthy! But, with so many different cultures promoting their own “healthy diet”, it is difficult to zero in on just one. To be honest, that’s not true! We know what’s healthy and what’s not. Our body knows as well. If you feel bloated after a meal, you’ve either eaten too much or something that you should not have! I combine a (mostly) Keto diet with Intermittent Fasting (IF). Here are the things that I’ve been doing for several months/years. But, now I do them with zero cheat days:

I fast. A lot. There are days when I eat just a single meal. There have been weeks in the past when I relied only on water! It started off as a personal challenge to gain control over my mind - “A person who can fast, think and meditate can do anything in life.” - Herman Hesse in Siddhartha (slightly modified to make it inclusive.) Here’s the great thing about IF, if you want to lose weight, you don’t need to eat something special, you just need to stop eating! Start small, there are several great resources to equip yourself with the knowledge you need to start IF so that you understand what’s happening to your body.

I don’t consume processed sugar in any form! I’ve conditioned my brain in such a way now, that I feel disgusted when I see a piece of rich cake!

Don’t eat right before you hit the bed - that’s the worst thing that you can do to your sleep! (This was mentioned in the sleep section as well. That’s not mindless repetition, it’s part of a theme that I’ll spell out for you towards the end! Hang on!)

Eat natural. I get my proteins from animal sources (mostly) but, I cook a ton a vegetables at home. It is healthy to cook your own meal - therapeutic and you know what’s going into your body!

I eat very limited amounts of processed food. A hearty burger with a thin slice of bread after 20 hours of fasting is my “cheat day”!

I follow a 8/16 fasting regimen - eat for 8 hours, fast for 16! Out of those 16, I sleep 8 so, it’s not as difficult as it sounds. People do a “Warrior’s regimen” as well - 4/20. See what works best for you.

The Intermittent Fasting is possibly the hardest to start with for most people. I say this because I’ve been trying to promote IF for the past several years when I discovered the joys of fasting and I don’t know anyone that I was able to convince yet! That said, if you plan well, it is very easy to do. Always remember, less than 10,000 years ago, we did not have farming, we went for hunting and at times returned with no food for the family!

Drugs

We talked about poison entering your body! There’s nothing worse than alcohol and tobacco for you! I mean, yes, you’ll die faster if you gulped down some mercury but, seriously, we should all question our governments about the legality of these things! Alcohol - in any amount - kills! All those “health wine” “studies” are total crap. Alcohol is very bad for you, even in limited amounts!

The same goes with tobacco. Tobacco is a toxin that the tobe-plant produces to protect itself from predators. It is supposed to be lethal for anyone with muscles and a relatively smaller body-mass! If we consumed it in large quantities, proportional to our body mass, we would die just like that poor caterpillar that attacks the plant. Nicotine does help you to relax your mind but, the net loss is just too big! When you smoke (anything!), you inhale some extremely poisonous gases, tar and a host of terrible things!

I don’t have a drop of alcohol anymore!

I hang out with people who are cool with that. Most reasonable, adults are!

I don’t smoke! (The pollution in Bangalore is enough worry for me already - please suggest a good pollution mask?)

When I go out with friends who smoke, I insist on sitting in the non-smoking area or something well ventilated. Sometimes I get lucky!

Exercise

Do you see a theme here already? These are not things we didn’t know about. All regular stuff, done with a brutal honesty to yourself!

Find the exercise that works best for you! Although, any exercise is potentially good, pumping iron alone would not give you a supreme health! It might give your body a super-human strength but, your mind and other important organs might suffer if you are not careful. I would recommend that you mix days of heavy and light workout. Here’s what I do:

Yoga and Pilates everyday - I start my day with these and do them several times a day since it does not need a special arrangement or a lot of time (or a shower afterwards!). Just stretch when you like you’ve been sitting for too long! I use the EyeSaver App on my laptop to remind me of that. My FitBit helps as well.

I’ve been trying to work on my hand-stand for the past year (I started because I saw some gorgeous pictures of my ex-boyfriend doing hand-stands in Borneo Island :P ). I’ve made considerable progress after I started Pilates for strength!

I try to run every evening (having a friend to nudge you works really well here!). I do some 10-15 minutes of slow pace running followed by a very intense sprint to death. I’ve been able to bring my resting hearth rate to around 60 after I started preparing my hearth for some serious pumping action - the SMART Goal is to bring it between 55-60!

Brutal Honesty

This one’s the most important thing in my opinion! Be brutally honest about yourself. Lying takes a lot of energy! Your brain has to work extra hard on things that are mostly not even important. In my experience, people lie because they want to create deception (an innately human quality) to their advantage!

While honesty would not help you to win over the liars, a brutal honesty would be impossible to defeat! Try it out for yourself. Not having to lie in any situation requires practice but, once you start doing it, there’s no turning back and things become easy!

Here are a few things people often lie about which I avoid at any cost:

Sexual experiences, sexuality, sexual expressions. It happens to be a very important aspect of being human (more to follow in the next section). There’s no need to lie about sex, (almost) everyone does it! Duh!

Simple things like - “How do I look?” - Don’t lie there, people will stop trusting you soon!

Lying to escape authority - I think authority needs to be called out. I am as equal as anyone else on this planet, I don’t need your authority. I tell people when they are being authoritative. Some of them get pissed, some understand and I get improved versions of these individuals to hang out with.

Sex

Sex is important for a healthy adult. Human beings (just like other animals) start engaging in sexual experiences right after/during puberty. It is an activity that gives extreme pleasure which is (as of now, things hopefully, would change soon) not possible by any other means. If you’ve tried having dopamine, serotonin explosions in your head by other means, you would know what I mean. Things are not so simple with sex. Somehow, it is not just about the chemicals (although you can enhance the experience exponentially if your brain get artificial supplies of these chems!)

With all its advantages, sex can be pretty harmful. Too less or too much of it can make you go insane! This is an area of your life that you need to optimize as best as possible because we have so less control over it. Here are some things that might help, although these things are ultra-personalized:

Find steady sex partners who are comfortable with you. Saves a lot of Tinder time if you have a list of numbers on your phone already! I know who to call when!

Human beings are serial monogamists. Follow that pattern and you’ll not have love-sex crisis in your life ever!

When I have sex with a stranger/non-lover, I try to have a role-play scenario in my head. I pretend that we are lovers. Works wonders for me. I have a very vivid imagination :)

Keep it safe! There are ways in which you can include safe sex in your sexual fantasy and make the safer versions work out better than the others!

Don’t watch porn. Even if it works for you, it would cripple your imagination.

Don’t worry too much about “love”. Yes, it’s better when done with love but, finding love, retaining it, keeping it healthy - all these are very harsh trade-offs. Keeping things simple has worked out much better for me in the last couple of years!

Make breakfast/lunch/tea for your hook-ups. Both of you would feel great when doing it the next time!

There are other things that can be done to enhance your sexual experience but, I won’t go into that due to legal reasons.

Meditation

If there’s one thing you should do after reading this terribly long (but, hopefully useful) post, it should be to take time out to meditate everyday! If you ace this one thing, you can do several others with very less effort! 

There are several ways to reach that “flow” state. I started my journey a few years ago using some random focusing techniques. Then moved to the S N Goenka’s meditation method. I would highly recommend that! There are some neat meditation techs that the Neurosciences department of Stanford often talks about. Check them out as well. Essentially, having a control over your “reptilian brain” should become your life’s purpose! Conquer that and you have everything else within your reach!

I meditate several times a day when I think my mind is going astray.

Meditation just before bed/in-bed is pretty good.

I’ve used the iOS App “Headspace” at times for some guided meditation. The narrator’s voice is seductive :)

I’ve been able to direct my brain to reach orgasmic levels of detachment (le petite mort!) lately! (The first time that happened was when I was camping next to the Parvaty river in Kasol, Himachal Pradesh!)

Hydration

Drink up water! Several times a day!

Water is a good source of Lithium as well, which is responsible for an optimal brain performance! Some important things to keep in mind about hydration:

You body loses water with every breath. That means, your body would require hydration before you feel uncomfortable or dehydrated. Keep a water bottle next to you. Or set a timer!

I drink hot water when I wake up, or before a shower, to heat my body from within. Cold water does the opposite so, I don’t drink any hot fluids before hitting the bed. Remember: Cold body => Prepare to sleep. Hot body => Get ready to work!

Don’t substitute water with carbonated drinks, caffeinated drinks, artificially sweetened fluids.

There are several great hydration strategies that I’m exploring myself. More on that soon.

Social Interactions

Limit your regular social interactions to a small subset of meaningful ones! Very cautiously (and quite ruthlessly at times) remove all negative people from your life. There would be a few that you can’t do much about, try to change them or limit that interaction.

Some people are nasty because they need help! If you are not in a position to provide that help, stay away! Don’t jump into a well of negative thoughts because of these people who are doomed!

Social media is the worst thing to have happened to us! It has become that necessary demon that one cannot stay too far away from. Use very good judgement on every minute you spend on social media (as with people having a negative impact on your life!)

I don’t have Facebook on my phone.

My activity in most social media is on need based - I post but, don’t consume too much. I don’t have notifications turned on for most apps on my phone.

I’m brutally honest with people around me. If they don’t like honestly, they distance themselves from me - saves me the effort to come up with a confrontational encounter.

Speaking of which, I confront people in my lives. A lesson an old friend taught me. Sometime, you just have to tell people what you feel for them to understand you better. Don’t work on assumptions!

Identify people who are beyond repair. Leave them!

Try to improve your Social Intelligence every day. I try to have conversations with strangers I meet on the streets, in the bus, when I go shopping, etc. Smile at people you don’t know! Anxiety needs work, it won’t go away auto-magically!

I use Apps like WoeBot (developed by Professor Andrew Ng’s team) that help you interact with the world outside in a much better way by teaching you terminology to talk about your own behavior and mood!

Engage with people you meet. Don’t just meet them. (You would be forced to do this if you meet them with no alcohol on the table or cigarette in your hands! Work at it!)

I have spent several weeks learning about human behavior from one of the finest people alive - Dr. Robert Sapolsky! His video lecture series (MIT OCW) is available online - It would change your life!

I also, try to understand what the other person is going through so that my words/actions have the effect I want them to have on my interactions with them. It needs practice, I’m practicing! Brutal honesty can help here!

Traditions vs Scientific Method

This is the last topic but, I think it is the most important! There is real wealth in traditional techniques like Fasting, Meditation, etc. But, that’s where you should stop.

Adopting a scientific method to evaluate all aspects of your life would help you make rational choices in life.

If you prefer to go by what your “gut says”, do that by all means but, also try to understand what gives rise to that gut feeling.

There are very valid scientific approaches to living that are being investigated everyday. New evidences are coming to light each hour. Some of it is hopefully going to align with your belief system but, most of it would not! You need to be prepared to let the cognitive dissonances in your life dissolve without much resistance.

As a person coming from a religious family, making peace with my atheistic belief system was the hardest part - luckily, I did that when my Pre-Frontal Cortex was still under development. For a lot of us beyond 25 years of age, fighting these religious ideas, traditions, dogmas would be a great challenge.

A good way to start would be to understand the origins of religion and religiosity. Dr. Sapolsky can come to the rescue here as well. With that in mind, let’s talk about the final topic and probably the most important!

Stress - One Ring to Rule Them All

If you do everything right, follow things to the last detail, be brutally honest about your lifestyle, you could still see yourself going down a spiral of death if you don’t manage stress in your life! Those Glucocordicoids are a friend but, only in limited and well controlled quantities!

One of my previous managers had the belief that “people work better under fear”. Although it sounds stupid, a lot of people work like that. So much so that, we often prefer leaders who treat us like shit! There’s a good read on Creativity and stress by the Harvard Business Review that you might want to check out as well.

Here’s a remarkable lecture on stress and depression: https://www.youtube.com/watch?v=NOAgplgTxfc

If you watch one thing today, watch this! Do this stress business right and everything will fall in place, get this part wrong and everything spirals back into disaster!

Now you know why carefully eliminating stressful people in your life is so important! It’s the people around us that give us stress. If your code does not compile, the machine is not going to complain, your boss would!

Final Words

Each of the bio-hacks I talked about work only if done together. You neglect one and everything else will crumble down. Have a great diet, exercise routine, healthy interactions and drink a glass of wine and, you’ve ruined everything! You might not see the changes because they are very subtle - but hey, we’ve all had that morning after that binge drinking, when we woke up with a swollen face and a demolished spirit! Don’t do that to yourself!

I hope the term “bio-hacking” seems a lot less of a “hack” to you now. There are absolutely no downsides, only massive gains that work like compound interest and make each day better!

Well, there is one downside, I guess. You’ll have so much energy and focus throughout the day that you’ll spend like, 2 hours writing about it in your blog just because a friend asked you for “a link that would tell me more, most of the things I’m reading online are freaking me out!” So yeah, have fun hacking your biology! Like always, please reach out with comments and suggestions.

[snortingcode]

SBT - Scala’s ((mistaken as) NOT SO) Simple Build Tool

The first thing that one notices when jumping into Scala development is its utterly complicated build tool. Complications arise due to several reasons (and reasons different for different people, I’m sure) but, a consistent gripe that people seems to have is the syntax used in the build definition file (typically build.sbt although, any *.sbt would work at the root of the project) for people not very familiar with some operators/constructors that the Scala community heavily relies on and, the monstrous amount of time it takes to get the first build started. The syntax seems strange at times even for Scala developers not familiar with sbt, that is because sbt is a DSL (Domain Specific Language). Let’s dive in to understand the essential basics!

[When I first ran “sbt run”, I wasn’t sure if things were even working at all, stared into the terminal for few 10s of minutes for several hours and nothing showed up. Google searches showed that it was a common problem! Although, in my case it was a conflict between the versions of Scala and sbt that were individually installed on my machine!]

With that welcoming note, let’s dive into the sbt world - it is bound to make your life easier if you know what you are doing. And it is an extremely powerful build tool.

I highly recommend that you read the sbt getting started guide completely before starting your project (this post is not a replacement as there are several details that are important and won’t be discussed here). I’m assuming that you’ve installed a recent version of sbt already. I’m writing this to provide a quick start (which might be a lot quicker than the “Quick Start Guide” that sbt documentation provides.)

Getting Started With “sbt new”

The recent versions of sbt (later than 0.13) provide a quick way to kick start your Scala projects  -

$ sbt new sbt/scala-seed.g8

sbt uses g8 templates to create the basic structure of your project. There are tons of useful ones available and you can create your own as well. If you come from the Java world, think of these as Maven archetypes of sorts. The command above would ask you for some project details like the project name, my project’s name is: Whipp. (For reasons unknown to humanity!)

The previous command should set up a simple Scala project for you with a directory structure that resembles that used by Maven:

src/[main, test]/[scala, java, resources]

$ tree -L 2 . ├── build.sbt ├── project │   ├── Dependencies.scala │   ├── build.properties │   ├── plugins.sbt │   ├── project │   └── target ├── src │   ├── main │   └── test └── target    ├── scala-2.12    └── streams9 directories, 4 files

The file that would the focus of our discussion in this post is the build.sbt file - the build definition.

(Don’t forget to add target/ to your .gitignore before you make your initial commit.)

Running the project:

Building and running the project is simple:

$ sbt #takes you to the sbt shell from where you can issue commands like run [info] sbt server started at 127.0.0.1:5522 sbt:Whipp> run #The task we wish to execute.

This would compile the project files and start the project. This could take a while if you are running your first project. Patience trumps hate!

Some cool things about the sbt shell are its tab-completion feature and history. It essentially behaves like your terminal’s shell. Pretty cool huh?

You could also provide tasks in a “batch mode” like this:

#takes sbt:Whipp> clean compile run

sbt provides a continuous build option. Append a task with ~ and sbt would watch for changes in your files, auto compile them and, execute the task - for instance, ~run would add efficiency to your development setup!

Here’s a list of all the commands available for you to use: https://www.scala-sbt.org/1.x/docs/Command-Line-Reference.html

The build definition - build.sbt

This is the most important and interesting section of this post so grab your coffee and read on! The key element in understanding the build.sbt file is that it is not just a properties file but, valid Scala code (well, valid sbt DSL code to be precise but, you get the drift!)

Read the generated *.sbt file with that perspective first and see what you can gather from it:

import Dependencies._ lazy val root = (project in file(".")).  settings(    inThisBuild(List(      organization := “com.snortingcode”,      scalaVersion := "2.12.3",      version      := "0.1.0-SNAPSHOT"    )),    name := “Whipp”,    libraryDependencies ++= Seq(      scalaTest % Test)  )

(Depending on which seed project you start with, this might look a little different. Hold on for a while and it would make a lot of sense to you!)

Let’s first look at what the below line is all about:

“lazy val root = (project in file(“.”))”

We are declaring a lazy value ‘root’ which happens to be everything followed by “lazy val root” in this case. Your build file is composed of several “val” and/or “lazy val”, these would typically fall under these 3 categories:

SettingKey[T]

TaskKey[T]

InputKey[T]

In fact, if you take a look at the config-classes under the target/ directory, you would find a cache file that has all the lazy val and val that you defined in your build.sbt file. You can also use the sbt shell to see what these keys are.

With that in mind, if you look at the build.sbt again, you’d see that we have a lazy val called root that contains the settings for each “project” in the current directory [project in file(“.”)]

Some of these are generic properties like the name of the project, organisation, etc. These are unlikely to change (it is a common practice to extract them out to a val and add it to the project.settings.)

In that sense, most of the build definition is just a bunch of key-value pairs => some help define things, some do interesting stuff. The value “name” is an example of a key that defines something.

A quick digression: If you try to assign a non-String value to “name”, you would get a typeMismatch error. The key “name” (which we did not define ourselves) comes along with the implied imports => sbt._, Keys._

Let’s try to add some interesting key that does something. Naturally, we would turn to a TaskKey[T]. Add these lines to your build definition:

lazy val doSomething = taskKey[Unit](“I do something cool!”)

Now, add this key to your settings. The updated build.sbt might look something like this:

import Dependencies._ lazy val root = (project in file(".")).  settings(    inThisBuild(List(      organization := “com.snortingcode”,      scalaVersion := "2.12.3",      version      := "0.1.0-SNAPSHOT"    )),    name := “Whipp”,    libraryDependencies ++= Seq(      scalaTest % Test),   doSomething := {println(“Look what I did!”)}  ) lazy val doSomething = taskKey[Unit](“I do something cool!”)

We defined a TaskKey[Unit] named doSomething using the method in the sbt DSL - taskKey[T]. Once you enter the sbt console (by restarting the sbt shell), you would notice that you can call this new task from the shell, simply by typing the task’s name - doSomething.

sbt:Whipp> doSomething Look what I did! [success] Total time: 1 s, completed 8 Feb, 2018 1:59:01 PM

You can thus, create a DAG (Directed Acyclic Graph) using these tasks and create a workflow for your project! Sweet!

A neat way to see what a task is doing is to “inspect” it:

sbt:Whipp>inspect compile [info] Task: xsbti.compile.CompileAnalysis [info] Description: [info] Compiles sources. [info] Provided by: [info] {file:/Users/prasoonjoshi/code/whipp/whipp/}root/compile:compile

(The actual output is rather long, so I’ve truncated it. Each [info] tells you important information about that task. (Try this out with the doSomething task you created just now and see if everything makes sense!)

We talked about the SettingKey[T] and TaskKey[T] a little bit, what is this InputKey[T] all about? Well, for now, you can just think of the InputKey as a method to take input from the user to create tasks on the fly (simple and powerful - we’ll talk about this in a future post soon!)

Let’s jump to the libraryDependencies

Yes, you know what they are! The essential information about this section of your build definition’s settings are:

+= is a method to append the provided dependency to the old libraryDependencies and return the new one!

libraryDependencies ++= Seq[<your list of dependencies>] is another way you’ll see the dependencies often defined in the build definition. (This is what our example build definition has.)

sbt uses Ivy for dependency resolution. The % that you see in the syntax below are sbt DSL’s methods that construct the Ivy module Id.

To define a new libraryDependency, you need to define the dependency first, then add it to the list of dependencies.

Here’s an example that adds mongo awesomeness to your project:

val mongo = "org.reactivemongo" %% "reactivemongo" % "0.12.6" libraryDependencies += mongo# # This line is part of my plugins.sbt file under the project directory. addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.6.7") libraryDependencies += ws

Of course, you can add the val directly to the settings without declaring it first. See what works best for you!

I promised earlier that everything in your build definition is just a key (SettingKey, TaskKey, InputKey). See what the libraryDependencies key shows you on the sbt shell and see if that makes sense :)

A final note on the evaluation order in your build definition file: Since the build definition is just a DAG which is interspersed with “lazy val”, you should expect a sequential processing, with lazy val(s) evaluated only when they are required.

A point to keep in mind is that the SettingKeys are evaluated once, TaskKeys are evaluated each time the task is executed (these would usually have side-effects as you can see).

There are several beautiful aspects of sbt that we can’t explore here in this quick guide. I would encourage you to check out the documentation and Josh Suereth’s great talk on YouTube.

[Please do reach out to suggest improvements or corrections to this post :) ]

[snortingcode]

Higher Order Functions

[This is part of a series of posts on Scala and Functional Programming: Part 1: Scala Tricks - tupled/untupled

Part 2: Higher Order Functions

Part 3: Cooking yummy Functional code - (partial, curry, compose)!

For a basic introduction to Scala, syntax, etc. you can check out Scala Fast Lane ]]

Unless you’ve been living under a rock, you’ve heard about Higher Order Functions already. You might have even used them in your favorite programming languages. Hopefully, I’ll give some insights here that would cement your understanding of HOFs and motivate you to try a Functional Programming language that does a lot more with HOFs than your favorite programming language can do.

Quite simply, a Higher Order Function is one that takes another function as a parameter or returns another function as a result. But, let’s take a couple of steps back and look at what we mean by a “function”. The best way to understand this is to forget all you know about functions from a programming perspective and think of that old friend, Mathematics.

So, in mathematics, functions define a mapping between a domain and a co-domain. For instance, let’s consider a function that takes two number and returns the product of the two numbers:

scala> def mult(x: Int, y: Int): Int = x * y mult: (x: Int, y: Int)Int

This should be fairly simple to understand but, let me add a quick explanation just to add rigor to the post: the function “mult” takes 2 Integer arguments in a single argument list (yes, we can have multiple argument lists to a function which, as it turns out, is very useful and we’ll explore it in a future post) - x and y, and returns another Integer that is the product of the two - x*y. (Yes, the return type should not be Int but, we’ll use this ‘problem’ to illustrate an important point in a future post - Scala types and pattern matching).

So, this stuff is easy. Running this code in a scala Repl would give you a function object that has a type: (Int, Int) => Int Meaning, it takes two Integers and returns another Integer. Simple stuff so far.

Functions are just that, a relationship that converts one type to another using some defined transformations. Since, these functions are only defining the relationships that exists between two different ‘types’, we have a sense of “purity” when we talk about such functions - they don’t change over time and/or with parameters passed to them. The pure functions are like a Knight with a clear conscience - they always produce the same result for the same set of parameters!

There are 2 intuitions that might help in understanding “Pure Functions”:

Functions are relationships (also called “Morphisms” or simply “arrows” in Category Theory) between two types.

Functions try to define “what things are” not “how to perform a task”.

With that, let’s talk about functions in an imperative language like Java. In a language like Java/Python, etc. the programmer needs to define what to do with each parameter and how to produce the result. Functional Programming takes a different (better!) approach, it defines what things are and how to “make one given the other” and how to compose bigger ideas from smaller ideas.

Since a function, thus defined, is just an abstraction over a Type, we can tinker with it the way we like without disturbing its identity, by composing/decomposing functions together that don’t violate the restrictions that the Types impose on them (It is these restrictions that the functions must honor that give the teeth to these types!)

So, a good way to look at functions is by visualizing the whole code-base as a network where, contrary to a similar analogy in other languages, the functions are the edges of the graph and, the data (Types) flow via these edges from one Type to the other (or same like in our example above) bringing about the necessary transformations. This analogy, I think, is a great way to look at functional code and helps to design bigger, complicated systems.

Let’s look at how a higher order function would fit into this understanding. We know that functions can exist on their own (functions are first-class citizens in a functional programming language) or be part of an Object, in which case they are called methods (There’s a neat trick that converts a method to a function that we’ll talk about in a bit - lifting).

The compose-ability of pure functions (and the way they preserve the type system by definition) is what gives rise to the need (and use) of HOFs. Let’s expand on our tiny example a bit:

Let’s assume that the function mult (Int, Int) => Int is not just multiplying 2 Integers, but these Integers mean something. Let’s just say that this function mult defines the relationship between variables of a term in a linear equation:

def hypothesis(A, X) = ∑ (a*x) , for a in A and x in X

Here, A is a Vector of coefficients and X is a Vector representing some variables. More precisely, let’s define a few Types here to help the intuition we are trying to build:

scala> type A = Int defined type alias A scala> type B = Int defined type alias B scala> def hypothesis(a: Seq[A], x: Seq[X]) = ??? hypothesis: (a: A, b: B)Int

We’ve just defined some type alias. We can now use A, B for Int. Simple.

Our hypothesis(Seq[A], Seq[X]) is now a valid Scala function. We’ve not defined the implementation of this function yet, but, this is valid since function definitions are evaluated when they are called (every time they are called!)

Let’s try to implement this function and see how things go. We’ll start in a (mostly) imperative way and try to improve as we go along:

scala> def hypothesis(a: Seq[A], x: Seq[X]): Int = | (for ((a, x) <- (a zip x)) yield (a * x)).sum

This might look a bit disturbing if you come from an imperative world (which you most likely do) but, try spending a few minutes reading the code and see if it makes any sense to you before reading further.

The function hypothesis takes two Sequences as parameters (Think of Seq for now as a collection with an Iterator and Order between elements), zips them together forming a collection that has pairs (tuples), each of which has the first element from ‘a’ and the second element from ‘b’.

scala> val a = Seq(1, 2, 3, 4) a: Seq[Int] = List(1, 2, 3, 4) scala> val x = Seq(1, 2, 3, 4) x: Seq[Int] = List(1, 2, 3, 4) scala> a zip x res12: Seq[(Int, Int)] = List((1,1), (2,2), (3,3), (4,4))  

Alright, neat!

The construct:

for (e <- elems) yield (doSomething(e))

takes elements from ‘elems’ one at a time as ‘e’ and do something with that (any functional transformation, etc.) This would work well and is fairly easy to read and expressive enough to be understood in a shared code-base. The function ‘sum’ chained at the end would give us the summation that we required.

Let’s say the relationship between A and X changes and is not defined by a simple product but, some other function complexProd. This change would require us to re-write the definition of the hypothesis function and would lead to code duplication and in general would be a nasty thing to do! This is where HOFs would come to our rescue. Let’s see how:

scala> def betterHypo(a: Seq[A], x: Seq[X], f(a: A, x: X): Int): Int =     | (for ((a, x) <- (a zip x)) yield f(a, x)).sum betterHypo: (a: Seq[A], b: Seq[B], f: (Int, Int) => Int)Int

We’ve passed a function, ‘f(a: A, x: X): Int’ to this betterHypo function so that the way we define the relationship between A and X is abstracted out of the function. We can stretch this idea and think of functions that return functions that can later be applied to some parameters in another higher order function.

Just think of functions as an immutable system of arrows from one Type to the other which doesn’t care to say much about the individual elements of the Types alone but, about the Type itself. This is a good intuition to build and a nice parallel emerging from Category Theory.

In the next installment of these set of posts on Scala/FP, I would talk more about Partial Functions, partial application of functions and Currying. Go functional :)

[snortingcode]

Scala Tricks - tupled/untupled

[This is part of a series of posts on Scala and Functional Programming: Part 1: Scala Tricks - tupled/untupled Part 2: Higher Order Functions]

I started learning functional programming a couple of years ago. Started with this brilliant book called “Learn you a Haskell for Great Good”. I don’t think I can recommend a beginner book on a complex topic like FP more than this beauty. It doesn’t claim to make you an expert in FP or explain the underlying Category Theory as such but it does attempt to make the subject less scary and adds a lot of fun during your initial exploration.

The best part about this book, I mentioned, is that it doesn’t try to teach an OOP person how to transition to FP. Instead, it talks about FP in its own right most of the times and it is able to do it because, well, the book is supposed to be fun and possibly not a comprehensive guide to FP (another fun book that changed the way I looked at Python was Why’s “The Poignant Guide to Ruby” - do check it out!)

That said, when you start getting into FP with an intention to have it pay your bills eventually, you might have to switch to languages like Scala. The reason that makes Scala a rational choice to start with FP is the same that makes it a bit silly to programmers new to FP - it let’s you do things the OO way as well.

With that background, let’s look at a particular trick in Scala and explore some aspects of FP along the way:

Functional Programming is all about programming with functions (Duh!) Every other selling point of FP that you hear about is a direct implication of this fact - immutability, code reuse, concise representation of complicated ideas making modeling of the physical world a lot easier. If you keep this as a central idea when you think about a problem, it would help you to write code that works and is efficient.

Let’s consider a function that takes 3 arguments from a Tuple (which is an immutable collection of ‘objects’ with equality of two tuples dependent, not just on the contents of the collection but, also on the relative position of each object in the collection) and adds them together to produce a result:

def addElements(a: Int, b: Int, c: Int): Int = { ... }

Let’s quickly walk through the syntax here:

‘def’ defines a function, the function addElements takes a single list of 3 parameters a, b and c and produces a result. The type of the parameters and the result is given after a “:” following the thing you are providing the types for. (The type system of any FP is very interesting, I’ll write a detailed post on that soon!)

Now, a way to call this method would be to take the Tuple apart and pass each of its 3 elements as 3 separate arguments to this function:

val tup = (100, 200, 300) //’val’ defines an immutable value, a tuple in this case addElements(tup._1, tup._2, tup._3)

This approach certainly works but, looks like a stinking garbage dump to a Functional programmer. Let’s fix it:

val betterAddElements = Function.tupled(addElements _) betterAddElements(tup)

We used the function “tupled” to convert the addElements function to a function that takes the tuple itself and knows exactly how to extract each element of the tuple and produce the same result that we expect from addElements. (The untupled method does the inverse - given a function that takes a tuple, it converts the function to one that takes individual parameters.) You can think of some sort of a magic wrapper around your method that lets you do this but, this magic wrapper is a direct result of any functional programming language design.

There’s a lot of FP magic happening here that I can’t cover in this post because it would make it too long and may confuse the programmer new to FP. The magic can be summed up in a few keywords - higher order functions, partial application, pattern matching (and that curious ‘_’ passed as an argument to Function.tupled)

I’ll write a post about each of them because they require a separate treatment. Stay tuned :)

[snortingcode]

Getting the data-types right or terribly wrong

As I explore the world of Functional Programming more, I can see myself getting obsessed with getting the Type system right for any app that I’m working on. While it is more natural and a lot easier to get the data types right when you work with pure code, it is not impossible to do so in the Object Oriented world either.

Start with defining a ubiquitous language for the domain, design your models based on the domain of the problem you are trying to solve and don’t shy away from refactoring your code if it doesn’t make sense to you after a week of writing it!

Let’s look at some examples of data-types that are easy to get wrong:

    phone_number : Number        book_ISBN : Number       start_date : String        area_code : Number record_identifier : Number      temperature : Number

(The temperature as Number is a bit tricky, since while you can compare 2 different temperature measure, adding them does not make much sense. I would typically create a type alias for such a field.)

Here are some simple rules that are good to keep in mind:

Always assign data-type based on the operations allowed on the values.

Use alias to make things more intuitive, eg. first_name + last_name has no meaning so, define a data-type as such data Name = String This would be a clever thing to do. Most functional languages have a clean way of defining such alias. Doing this for simple data-types in an Object Oriented language might feel like a pain though.

Rule of thumb: Categorical data is never a Number! (caution: 0/1 can be both quantitative data or categorical data.)

Rule of thumb: Look at what the aggregates on the data are possible - classification, averages, nothing. This is a good way to begin to understand what the data-type should be.

If you are working on designing a model after looking at the raw data, do yourself a favor and ask for the meta-data instead. The meta on the data would tell you a lot more about it than the data itself (who would’ve thunk!)

[snortingcode]

Lessons learned from a failed ‘pre-startup’

Recently, a friend has joined the startup wagon armed with a brilliant idea, execution capabilities and potential capital in angel investment money. I was sharing my own experiences with him and could trace out a few that might be worth a 10 minute read for anyone trying to be more productive - something that the act of starting a company certainly requires.

I’ll just put them here in no particular order. Would love to see what you think, what your own experiences have been like.

Work as a company not a person

When you decide to start something tiny with the idea of making it big one day it is easy to still operate in the individual scheme of things. To see yourself (or a few other friends) as a company can be quite a challenge but, it is important that you define those boundaries for your time, resources and expenses. This would help you keep a track of things you are doing that align you more with the future that you visualize. It also helps by giving you a head start in figuring out what works for you as an organization and what does not.

Take designated holidays

Holidays are not just a matter of HR policy, they are required by you to unwind and eventually be more productive. When you define on which days you can drink for long hours with your friends, on which days you can visit your parents, the days you can go on a boat ride, you’ll find that your ‘working’ days are more productive and there are lesser incentives for you to not give your 100% on the days that you are working.

Document everything you do, say, think, talk about your project

Our minds are capable of painting very vivid, entertaining pictures of our ideas for us. Sadly, it is impossible to translate those ideas into tangible artifacts without a great deal of loss of details. The best we can do is make sure we describe our ideas and thoughts in the best possible way and keep doing that till our documentation reaches a maturity that we are satisfied with. Your ideas should be captured in such detail that if one decides to go though them they get a picture as close to what you have in mind as possible. There are caveats to this approach as well - it is possible to get stuck in a documentation quicksand.

More details on effective documentation on some other day but, here’s a quick one - Don’t document thoughts as they come, you’ll throttle the flow by the process of writing. Keep thinking about them and take a break after a few hours before you write them down.

Work with a traceability matrix in your document

Almost everything you want to do would have a target audience and some expected effects that you want to bring about with your idea. So there would be a (measurable) effect that your implementation should bring about. In the frenzy of getting things done fast, it is easy to lose that focus. A simple act of making sure you trace each implementation detail to a point where you don’t lose focus is very important.

Don’t overwork - It gives you reasons to take days off

While working hard is important, what’s more critical is having a routine. If you work very hard, you would need to compensate by taking longer non-working breaks. These breaks do more harm than a regular routine that includes the visit to the badminton courts, a drink with your buddies and a Sunday morning run.

Health

Hopefully, I don’t need to convince you about this one. Health is important! Find an activity that you like to do and do it regularly.

[snortingcode]

16 GB RAM Upgrade - Early 2011 MacBook Pro

The Sierra upgrade had not been very kind on my old laptop, an early 2011 MacBook Pro. It came with an i5 Intel processor but just 4 GB of RAM (and the fact that it has a 320 GB SATA disk didn’t help the cause for a machine that could actually be used for anything productive at all). RAM upgrade was the obvious first choice after viewing the results from vm_stat (that’s vmstat on the Mac terminal) that were showing a very high number of Pageouts - meaning that the RAM was not enough to store all the pages of memory required by the OS and applications which made the OS swap out existing pages into the secondary memory (it being SATA killed speed).

So yeah, long story short, I upgraded to 16 GB (against the guidelines provided by Apple on this issue for my generation of laptop) and I can see 0 pageout and a visibly fast machine. I can switch between open projects in Eclipse and GarageBand, listen to music while editing this post. Might not sound like much to ya’ll but, to me it is close to discovering inter-dimensional space travel portal gun in this age of mediocre earth transportation.

Next saving goes into investing on an SSD storage. I wonder if I can get rid of the monstrous Optical Drive and have an SSD instead. Would make my laptop a lot lighter and lightening fast!

[snortingcode]

Weekend Hack - Face Recognition based Access Control System with Python, OpenCV and Rekognition

My current interest in ML has triggered an urge to revive my Python skills. A good start, I thought, could be experimenting with some Image Processing - mostly to understand the limitations and challenges of tasks like these.

The last time I worked with Image Processing was as part of a PoC for Hadoop. I used HIPI to do the heavy lifting of creating image bundles and preparing them for the underlying HDFS with OpenCV’s java bindings to detect faces using Haar cascades.

The problem statement for this hack extends my experiment with HIPI.

Problem Statement:

Detect human faces on a video feed and compare the captured faces against an image dataset to suggest probable matches to provide access control.

Technology/Tool/Environment Stack:

Python 2.7.x opencv python OSX Sierra AWS Rekocnition

Installation of OpenCV is a major pain! Especially if you wish to work with Videos on Sierra. Most of my time was spent in preparing the workstation with a reliable installation of OpenCV (unsuccessful attempts included installing a pip package that claimed to provide OpenCV but was compiled without ffmpeg). As of now this post is the most reliable source of instructions to install OpenCV on Sierra (and incredibly detailed). I would not reproduce any steps from that guide to avoid redundancy.

You might want to check this guide out if you are new to setting up virtualenv for python development:.

Solution:

The solution revolves around breaking the problem down into smaller tasks that are more or less self contained:

Capture frames of a video (in my case this was a video feed from my internal facetime camera).

Detect human faces in the captured frame.

Have a datastore of known faces (this involves mapping a person with a set of features extracted from their face).

Match the captured frame against the face datastore to grant access or deny it.

Capturing frames from a video feed: Doing this with python is quite straight forward. OpenCV has VideoCapture( ) that gives you a handle on the video. It takes one argument - the identifier for the camera - if you have a single camera, as was my case, just pass a 0.

Detecting human faces in the captured frame: For the purpose of this exercise I was not interested in training the classifier. There are very efficient Haar cascade classifiers that give great accuracy. OpenCV comes equipped with trained classifiers that we would leverage (there are some cool ones that let you identify cats in images too!)

The Haar cascade classifier that I used was: opencv/data/haarcascades/haarcascade_frontalface_default.xml

Working with gray-scale images is faster so we convert the captured frame to gray, create a classifier that can identify a human face in an image (you might have to tweak the classifier to suite your case) and, pass the gray-scale image to it which would return a list of faces it identifies.

I did have to tweak the parameters based on the distance of the camera from the subject, etc. but, it worked well for most parts. It indicated positive results even when the subject was moving fast in front of the camera - it pointed the blurred human face in that case (might not always be what you want - my app breaks in such situations as of now!).

Face datastore: Enter AWS Rekognition!

Rekognition lets you create a Collection of faces that you can match against. The API is straight forward. One important point to note is that AWS does not store the actual image bytes just an incredibly rich set of features that it associates with the face by giving it a faceId. You can keep adding new faces to this collection by calling the rekognitionClient.index_faces( ) API (I’m using the python sdk - boto3).

You would need to map the returned faceId (and other face metadata) to the actual image of the person or some other identifier for the person and persist it.

Match captured face against the face datastore: Rekognition has a face matching API that takes the CollectionId and image frame and returns the metadata associated with the matched face from the collection (or a no match). A list of matched faces with confidence level is returned in decreasing order of confidence.

If more than one face is detected in the input image, only the largest face is matched.

So there you have it! It is ridiculously easy to build this solution as a PoC. Taking this to production should still not be a very daunting task, it would just take more time to visualize all failure scenarios and account for them.

Here’s the code produced as part of this short hack exercise: https://github.com/prasoonj/tzukuru

Places where I was lazy: - The project structure could be a lot better. - I’m using Mac’s ‘say “Hello World”’ terminal command to announce the name of the person matched (also to produce some delay giving better user interaction). This can certainly be done in a much better way. - There are a lot of scenarios where the app fails right now. No exception handling is done. - Unit tests!

[snortingcode]

Notes on DynamoDB’s data model and partitioning strategy

DynamoDB is a fast, scaleable, managed document store from Amazon Web Services. Be sure to check it out if you have not done so already before starting with this post.

Designing a schema for a document store is remarkably different from a traditional relational data store. Most document/NoSQL persistence solutions sell their product with a tag line that often says - “Store anything in any schema” but, the truth is a lot different. While a NoSQL document store does not expect you to follow a particular structure for each record (beyond the presence of a primary key), the reality is that if you don’t think through the design of your tables and the documents stored in them, you won’t be ale to go beyond a trivial {“message” : “Hello World”}.

Within the humble scope of this blog, I won’t be able to cover all design aspects of a document store. Let’s limit our coverage to trying to achieve an understanding of the DynamoDB’s data model and, how the schema design would affect the service availability and cost for your application.

Let’s look at how DynamoDB distributes the throughput that you assign to each of your tables:

If you allocate throughput T to a table (which is what you pay for!) it would be divided evenly over all the partitions that the table has. DyanmoDB is a managed service that promises a robust ‘scale-out’ mechanism to handle increased data load which means that the number of partitions would keep increasing as your table grows to accommodate more data.

Each record for a given table has to have a primary key: a combination of a hashKey or partitionKey and an optional rangeKey or sortKey. The partitionKey (PK) is given to the dynamoDB’s hashing function that decides which partition it would be a part of (and consequently where to look for a particular record). Records with the same PK but a different RangeKey (RK) sit in the same partition in an order defined by the natural ordering of the RK.

The hashing function of DyanmoDB tries to ensure that the records are evenly distributed across all partitions of the table.

In designing your DynamoDB schema, a rule of thumb is:

Data access should be as evenly spread across all partitions as possible!

Let’s consider 2 scenarios:

Scenario 1: You are storing user details for a social networking website where all users are equally likely to request for the user resource over their lifetime of usage of the service. A perfectly valid choice for a PK in this case would be the userId. The hashing function would ensure that users are evenly spread across all partitions and the nature of your application and this particular data has a similar access pattern making your design rather simple.

Scenario 2: You want to store user history which would be used to pick the most recent user interactions to be displayed in a ticker. If we use the same strategy and persist the records with userId as PK, we would end up with a lot of ‘historical’ partitions that are very rarely or never accessed. Can’t be very bad, you’d say.

Let’s look at what happens to the throughput of the table. As we saw earlier, the total throughput assigned to a table is evenly divided between all the partitions - restricting the maximum throughput to T/n on each partition (T := total_throughput, n := total_number_of_partitions). You are essentially wasting a lot of dollars on throughput that you would never use! Your dynamoDB access would show throttling even though you see a lot of unused throughput for your table!

For such ‘time-series’ data a good strategy is to age out old data in tables with limited throughput and provide more throughput to ‘hot data’ that’s contained in time sensitive tables -  create new tables with higher throughput for hot data!

Here’s a quick recap and some important considerations:

Throughput should be spread ONLY across frequently accessed data.

A good way to show that the design has inherent weaknesses is to look for ‘scan’ operations over ‘query’. If your design requires a scan, there’s something wrong!

DynamoDB automatically adds more partitions to your table as the data size grows, dividing the total throughput evenly across all of them.

There might be a need to breakdown requests into smaller sizes - a query/scan that returns a massive response is bound to consume a major chunk of the allotted throughput and as a result throttle the database for other queries.

TL;DR Be very careful when designing a schema for storing time-series data in dynamoDB. Ensure that the assigned throughput is spread across all partitions containing ‘hot-data’ and is not wasted on partitions that are rarely used. Ensure that the data access pattern matches the hashing function - access should be evenly distributed over all partitions of the table. Create new tables with higher throughput in case of data that has a decline in usage as it ages.

[snortingcode]

My First rm -rf /

Credit must be given where due!

Apple, after a series of disappointments, impressed me quite a lot today.

Let’s first talk about the disappointments:

- The ‘phone app’ of my iPhone hangs! I think there’s no place where a company like Apple can hide for something as sloppy as this. A small sample survey that I did on this subject with people around me indicated that the problem is not isolated to my handset. (Apple - 0, LowestOrderSchmuck - 1)

- A teammate lost a day’s worth of work when Sierra’s first update came out. The system crashed and nothing would bring it back other than a complete re-install of the OS. I’ve been forced to stay on El Capitan even after more than 6 months of the latest version being out. I don’t think I have the nerves or the motivation to land myself in muck voluntarily.

Anyway, let’s put that negativity behind us and talk about the uber-cool migration tool that comes with apple by default - no third-party integration, no version mismatch issues, no spilling hot coffee on the vulnerable parts of your body - just the sheer joy of connect-click-migrate!

Just connect your new machine to your old one (over a wired or wireless network), open the Migration-Assistant in both the machines and follow the onscreen instructions. It is as simple as that! The most natural, unadulterated joy I experienced on Apple after a lifetime of pain and disappointment. (In case you are wondering why I am on a Mac in the first place, well, sometimes we have to go down the dirty path for a living!) The whole process took under 40 minutes and for my mac: MacBook Pro (Retina, 15-inch, Mid 2015) 2.2 GHz Intel Core i7 16 GB 1600 MHz DDR3

Oh yeah, before we close, if you migrate from an old machine to a new one and don’t intend to use the old machine, always try:

sudo rm -rf /

(Watching things burn like that is like sitting in a huge room filled with bubble-wrap!)