Nitin Dhar

[nitindhar]

I built duffelapp.com over the weekend with Vercel

First, some background

In my quest to get to a minimalist, clutter-free home, I ran into stuff that had been stowed away in my closet for far too long. I was filled with excitement at the idea that I'd have a clean closet and more storage space to dump stuff that had been lying around my house. As soon as I started, reality hit me. Most of the things I pulled out were items that were relatively unused (new-ish), and at the minimum not worth discarding. The guilt of trashing perfectly functioning items kicked in.

Over the winter holidays, I thought more about it and it struck me that I'd rather not throw these things out. But donating these things also seemed like a lot of effort... find bins, schedule with charities, etc. I couldn't find a new home for my excess this way. Just then I remembered that a friend had been looking for a baby white-noise machine, which I had right in-front of my eyes, collecting dust. I thought to myself "why not share this with my friend?".

Why not share?

There seemed to be a priority order in my head about what to do with each item. First, share with friend. Second, donate to a charity. Third, recycle. Last, trash. I started looking for a way to show my friends all the stuff I had, so they could pick out whatever they needed, but this would require facetime's, in-person hangouts, sharing pictures on iMessage.. there was no easy way to share...

Window shopping

Trying to show my friends my excess items proved to be more difficult than I first anticipated. I thought of putting it all up on a pinterest board, but that didn't make sense either. I wanted a way for them to "window shop" my inventory on their own time. And I wanted to peek in their window shop as well. And I wanted to feel good about giving to friends.

Being trapped in this conundrum, I cooked up duffelapp.com, as a sharing marketplace where my friends and I could list unused items.

Technology

In the past, building a website has always started with a big, initial hurdle of which frameworks to use, configuration, deployment, etc. I chose to build Duffel using Next.js and deploy it with Vercel (the company behind next.js). The app itself is quite simple, but what made the experience exceptional was that I set up the github project under Vercel, and deployed my app in 5 minutes. Roughly speaking here were my steps:

1. Develop site design locally (using Tailwind CSS for UI)

2. Setup the github project under Vercel and hit deploy (the app was available under https://duffel.vercel.app in a few minutes)

3. Setup DNS nameserver records on GoDaddy

With the above setup, every push to the main branch under the git repo auto-deploys to duffelapp.com. Voila! 2 days of work over the holidays (<4 hours in total). Go with Next.js and Vercel.

Check out Duffel here: https://duffelapp.com

[nitindhar]

Meet Gameplan

[nitindhar]

Curriculum - A list of libraries, frameworks, topics, etc for self-learners

I've been managing a bunch of notes for some time that may be of use to others. The first one I've shared on github is called Curriculum. Curriculum is meant to be a running list of libraries, frameworks, technologies, etc (currently focusing on javascript or the "frontend") that I think are relevant in 2017. There have many so many articles about whats new in javascript in 2017 and the overcrowding of this space. Part of my motivation behind this list was to have a 1-2 line understanding of what each of these technologies are and to have such a glossary ready for reference. Hopefully, someone will discover this some day :)

A small snippet of that you can find on my Curriculum:

...

Observables

CycleJs

babel/ES6

systemjs

jspm

ReactJs

Bower

Gulp

Grunt

VelocityJS

HoganJS

TitaniumJS

...

[nitindhar]

Figured that there may be other visual learners out there besides me. This is unidrectional flow of data in Redux visualized along with how container and presentational components fit into the picture. More info here: https://medium.com/@dan_abramov/smart-and-dumb-components-7ca2f9a7c7d0

[nitindhar]

Continuous Delivery: Smoothing

Sometimes it's hard to describe the concept of continuous software delivery to non-technical folks and I'm convinced that the explanation needs to be visual and domain-agnostic. Weirdly, the thought of step farming comes to mind. If you've never seen step farming here it is:

Crops are sowed on incrementally higher/lower steps of land for good water irrigation. That is what a week-by-week release plan looks like, which is consistent, but not on-demand enough for the needs of today. Compare with this:

Notice how the frequency of the steps is significantly higher? That's what continous software delivery looks like. Software is delivered piecemeal and often so that features/fixes are applied real-time for your convenience. From a distance the hill looks smooth:

You can call each of the steps a Releasable Business-Unit, or RBU. RBU's allow you to smooth out the release process, making it less stressful and impacting fewer areas of your codebase. I like to call this effect Smoothing. Drop a new feature on this slippery slope and it quickly slides down to you!

I work for Fevo.com and we're moving to continuous delivery. Ask me a question about it :)

[nitindhar]

Transition (not CSS) ~ from engineering to engineering management

This post has been a long time coming. I've been meaning to write about this topic since I first started managing back in my Gilt days. Though I only managed for ~6 months, a transition like this hits you real fast. The first thing I noticed was how having a plan was so critical to success. Up until that point I had never thought about how managers spend a lot of time creating work tasks, splitting and dividing work amongst the team. As I practiced this art more and more I quickly saw that I was not focusing on that enough, which was leaving my team without a clear direction. I was in execution mode, and didn't have time to answer people's questions, guide them or work with them. That is certainly not team work and certainly not management. When I moved on from Gilt to Host Committee, for a while I was in a solo execution position again. Recently I stepped up to a leadership position, but by now I've had the experience and mentorship to do things the right way. Let me talk about a few things I've learned along the way:

Transition

The biggest aspect of this transition for me has been letting go. Letting go of the code. Letting go of the technology. Certainly not letting go of the goals, though! Watching others make technical decisions on software we want to use, the development style we want to have, etc has certainly been tough. It's a balancing act between hacking to make things work and doing the right thing. One challenge I've felt is to think deeply about the reasons we chose particular libraries or style of development. It's hard to lead a team without having good answers for decisions. Engineers are very logical beings and usually are driven by logic, so to translate human-driven decisions into logical decisions can be tricky. What I mean by that is that sometimes we made decisions because of preference over what was the best thing to do for that use case. I know this is abstract, but that's what makes it tricky. Reasons can be based on intuition, instinct and preference.

Communication

I can't express this more. Communication can make or break a team. Imagine if a team member went rogue to produce code using libraries or patterns that only they can understand... How would the rest of the team feel? Cheated? Angered? All of the above? Yep, probably the last option. It's critical that we're constantly communicating with each other to keep us all informed and on the same page. I think people feel comfortable when things aren't slipping away from them. They feel included when things get communicated, making them feel part of the same team.

You're Not The Expert

I'm no longer the resident expert at Scala or the backend. In fact, I may not be the resident expert at anything at this point. That has hit me the hardest. I'm no longer the person who others come to for advice on how to write cool code. On the flip side, I'm the un-blocker. When my team is stuck on something or need help with modeling a concept I step in. When teammates disagree on an approach I step in. They create frameworks for writing code and producing features. I create the framework for working as a team towards goals.

Prepare to be Challenged

As a solo contributor, you're off on your own. The only times when people care about how and what you're doing is if you're delayed in producing features, when there are bugs or during integration. No one really asks "why you did it this way?" or say "that's not the best way to do this". As soon as you let go and have others making decisions and picking up from where you left off, prepare to be challenged. This phase for me has been a test. Do I have a tough enough skin? Can I move forward and help support my team sincerely? People will ask you questions and you will have to provide answers. In this process, you'll realize that not all of your decisions were the best ones or the right ones. You'll see flaws in your thinking and your logic. Just have to remember that this is a learning phase for everyone. Answer sincerely, learn from your team and take the learning and re-invest in the new experts that they are!

Providing a Decision Framework

How do your team members make tech decisions? What about feature level decisions? I don't have a clear answer on this one yet. It's a work in progress and I will share more thoughts in a later blog post.

Would love to hear your thoughts, especially if you've gone through such a transition recently. Drop a comment and share the link!

[nitindhar]

ScalaJS, WebAssembly & the Future of web development

I'm no ScalaJS or WebAssembly expert and don't claim to be at the fore-front of what's happening in the browser development community, but I've vowed to learn something new each day and talk about it. Today, I dove into ScalaJS, which is a library to allow developers to write Scala code that produces JS code. It was very interesting going through the basic demo, which took me ~1.5 hours to read, code and debug. 

Key learnings were:

It’s great to write code in Scala -- beware of the transpiled output

Scala wrappers to interact with existing JS libraries jQuery might become painful overtime

If we can get into the habit of writing code in Scala, Java, Ruby, etc, etc now that may make the transition to WebAssembly easier whenever that happens!

Here are my notes:

ScalaJS Basic Tutorial (started at 11:12pm and ended at 12:38am)

Code has actually been executed by the Rhino JavaScript interpreter. Rhino is slow, so we set the JS stage to use NodeJS which is fast using FastOptStage

To get source maps install: npm install source-map-support

run fastOptJS to create a single js file from scala code (104KB...yuck)

Use scalajs-dom library to access to the DOM, which uses the %%% operator indicating that a scalajs library is being used

org.scalajs.dom object refers to the window object and dom.document refers to the document object

interesting API methods: document.createElement, document.createTextNode, node.appendChild

Felt like I was already starting to build components and their behavior in Scala =O much wow!

@JSExport makes a method callable from Javascript

Ugh, but you have to depend on scalajs-jquery -- yuck, now we have to wait for these libraries to be made and kept up-to-date/bug-free in addition

additionally we have to include jquery's js file which means that our app now has js libraries and their scala counterparts coupled tightly -- feels like a dependency nightmare.

although the js files can be bundled by setting skip in packageJSDependencies false

feels like Browserify, which produces a bundled js file

Runtime DOM is not available by default. Adding the RuntimeDOM uses PhantomJS behind the scenes to enable access to DOM.

the phantomjs CLI has to be on your PATH for this to work

Without compression the size is huge! (1.4MB)

Run Google's Closure compiler using fullOptJS (after compilation this was down to 17KB!!!!)

Gives you a launcher scripts so you don't have to manually create and start your JSApp like so:

tutorial.webapp.TutorialApp().main();

NEXT STEPS: 

Scala.js single-page-application (SPA) tutorial using Play Framework and ReactJS -- i’m super excited about this one!

REFERENCES:

http://www.scala-js.org/doc/interoperability/export-to-javascript.html

http://www.scala-js.org/doc/project/dependencies.html

http://www.scala-js.org/doc/project/cross-build.html

http://www.scala-js.org/doc/project/building.html

[nitindhar]

Saving arrays in Postgres using JDBI

Last time I wrote about JDBI I spoke to you about creating flexible binders for the JDBI Object API so you could bind fields from custom objects into Postgres table columns.

Today I want to show you how to save a collection to a Postgres column using JDBI. You may think this should be totally trivial, but for anything other than a totally basic use case it isn’t. For example, if you were using the Object API and wanted to save a list of tags to a Posts table, you can’t just do:

The problem is that @Bind operates on the JDBI SQLStatement, which does not accept collections that way. Behind the scenes, JDBI has to map the java collection to a database specific SQL collection. In this case, a SQL Array is needed since Postgres works with that. To accommodate for this, JDBI provides you with bindBySqlType

In our example, name is “tags”, value is a SQL Array that we need to create:

and sqlType is the SQL Array data type from the java.sql package. However, I've skipped telling you about a detail. You don’t have access to SQLStatement in your DAO so you will have to do things differently. Create a binder factory like I showed you in the previous blogpost. Here’s an example:

Now that you have a mechanism to bind a java Post object to its corresponding posts table in Postgres update your DAO:

There you have it! You’ve been able to bind an array from a java object to a postgres column using JDBI. Feel free to ask me questions or drop a comment below.

[nitindhar]

Find a git branch across all local repo’s

Add this to your bash profile:

[nitindhar]

A lightweight Javascript library for the Option Monad

Giftbox aims to provide a wrapper API around absent values and is highly motivated by Scala's Option.

One of big source of bugs in production environments is missing data represented by nulls, undefined, empty strings, etc. Traditionally calling methods on null or 'empty' values causes NPE's, etc. and forces developers to create disorganized utilities with null/empty checks all over the codebase, adding further complexity.

[nitindhar]

Auto-incrementing sequences in MongoDB using Morphia

MongoDB does not offer an auto-incementing sequence solution natively. Instead most app developers will come up with their own. So I did!

Imagine you're building an order system. After processing each order you want to show a number to the customer that they can hold on to and use as reference when contacting customer service.

The idea is to create a sequence collection in Mongo containing a key and a counter:

In this schema key is the name of the collection that the sequence is for and counter holds the next number in the sequence. Once you have the sequence collection you can atomically increment it for each use giving you an pseudo-auto-increment sequence. Using Mongo's findAndModify you can provide a query that will look up the sequence for "orders" and ask Morphia to increment the counter.

Now you can use this in your order processing code to generate an auto-incrementing number which you then save to the order.

The sequence collection could be used to store auto-incrementing sequences for other collections as well.

Hope that for your next sequencing use-case you can leverage something like this. If you have other ideas, do tell!

[nitindhar]

Custom Form Validations in Play Framework 2.3.x

play.api.data.validation.Constraint to the rescue!

Say you have a RegisterForm with a list of predefined cities and an otherCity field and you need either the cities or otherCity to be supplied, i.e., otherCity should be validated if cities is not provided.

You can write a custom Constraint around this:

And your form definition becomes:

In this example emailCheckConstraint and passwordCheckConstraint are additional custom constraints that I defined similar to citiesCheckConstraint. This works in Play 2.2 - 2.3.8. I haven't tested this on more recent versions.

[nitindhar]

Blazzzing fast transactional emails using Play Framework 2.3, MandrillApp & ZURB in 1 day

MandrillApp has created a transactional email service and it is so amazing! Pair that with Play Framework and ZURB and you'll be unstoppable!

Last weekend I started using it for my side project Itemize to send notifications when shared items changed, were restored, were deleted, etc. Most ESPs I’ve worked with in the past have been very bulky so I was dreading working on this at all. BUT, Mandrill was quite a nice surprise. I layered it in using a Scala trait by providing my Play controller a sendMessage api. I created a trait called EmailService

Hold on hold on! Let’s take a step back and talk about Mandrill for a second… They offer a transactional email service free of charge for up to 12K emails/month with an hourly quota, which makes it ideal for small projects. Integrating them in a side project is super easy since their API is fully RESTful and the documentation is beautiful. Once you create an account you can setup a couple of API keys for different environments. I created one for prod and dev environments each. Emails sent from the dev environment are not actually sent out, though their sandbox tool allows you to view the content as it would be sent. As you can see in the snippet above, I just put this key in the Play config and imported it and the API url into my service layer. That’s it! Mandrill ready-to-use…

Creating Templates using Twirl

Originally, I wanted to have the sendMessage method accept a typed template instead of a String. Twirl templates are of type BaseScalaTemplate[...] with TemplateX[...]. The ‘X’ signifies the number of parameters the template accepts (and being a case class it maxes out at 22). The idea was to generic-ify the signature of sendMessage (a work-in-progress) to maintain type-safety yet keeping things flexible, but I wasn’t able to do that easily. In the mean time, it made sense to keep EmailService simple stupid and have it accept the rendered template as a String.

As with most software, the key to moving fast with email templates is to use a framework that provides well tested responsive templates. Thankfully this is where Zurb’s Email Templates came into the picture. Zurb offers 5 template types (I just picked the Basic template and customized it a little) along with tons of resources to make your templates work on different email clients. This saved me a whole lot of time and effort. Thanks Zurb team!!!!

Just to elaborate, since I’m using Play templates, I easily separated out the Zurb template markup and styling into different files (email_layout.scala.html & email_styling.scala.html), with the former importing the latter since email template styling has to be inline. In addition, creating different emails was super easy since I just pass in a HTML content variable into the layout @()(content: Html) ... for each email.

Play WS Client

After configuring Mandrill and setting up the EmailMessage api plus the template I leveraged Play’s WS API to make API calls.

The first step was to use the Play WS client to setup a method that would send json (note: the communication format was set to json using the .json at the end of the url and the Content-Type header) requests to Mandrill endpoints. Next, I hooked in my sendMessage method.

Notice that I use EmailPacket, EmailMessage, EmailRecipient and SendResponse case-classes to handle sending and receiving data from Mandrill. Along with the classes, I wrote Play’s Writes[T] to convert each of them to json using Play’s Json library. One thing to note here is that I didn’t account for failure responses from Mandrill, but those could be baked in a very similar fashion as above and I leave that up to you.

Parting Thoughts

It’s been such a pleasure to work on this using the tri-fecta of Mandrill, Play and ZURB. I was so excited because these tools make it so much easier to build features that were much more tedious to build even 5 years ago. In fact, I built all this in 1 day! I hope you are able to leverage some (if not all) of these tools to build your next email service quickly too!

[nitindhar]

Scala TypeTag, Manifests and erasure - 2 ways for passing type info

The following two are the same:

Firstly, (implicit tag: TypeTag[T]) is considered 'evidence'. It is evidence that provides information about the type of T to the method body so it concretely knows the type.

Secondly, the compiler would write #2 as #1 automatically so it's handy to define methods that way. When writing either of these we're basically asking the compiler to generate evidence to be provided to myMethod so it can be presented to JVM (which doesn't have the notion of generic types) during runtime.

[nitindhar]

Action Composition in Play Framework 2.3.x using ActionFunction

We recently decided to port code over from a Play Framework app written in Java over to Scala for many reasons like Scala’s expressiveness and asynchronous capabilities. Our code was messy, excessively verbose and highly blocking so the plan was to introduce Scala, clean up as much as possible and to write non-blocking code for new pages.

We started with the Corporate site pages like the homepage, about us, contact, etc. These pages were backed by controller actions that looked like:

This was not incredibly bad, but it was not as fluid as we wanted. The code was also making blocking service calls, which was hurting performance so we decided to first make the action declaration read better. This is when we started looking into Play’s Action Composition. We wanted to use verbs (since these were actions) for action function names. For example, we wanted an action that required authentication and notifications to look like:

Our first thought was to validate the request and inject data into it for use in the action. So, in pseudo code this read like:

We explored ActionBuilder[R], which extends from the ActionFunction[R] and is composable. The main functionality is placed in the invokeBlock[A] method. We easily ported the @Authenticated annotation logic here:

Note the use of a custom request case class AuthRequest, which wraps the original request, layering it with additional data. Basically, this builder accepts a request and a function (that accepts our custom AuthRequest and returns a Future[Result]). Since invokeBlock returns a Future[Result] it was very easy to compose async builders like this. Using a case class for AuthRequest was not the only option. A trait would’ve worked too, but case classes offer pattern matching, which could be useful down the line.

Next, we had to create a similar ActionFunction to inject notifications. This time we played with ActionTransformer[R, P], which transforms a request from type R to type P. Here’s what we did:

Again, using the NotificationRequest case class we were able to layer an optional list of the users notifications to the request, except this time the request was an AuthRequest. Here we used an ActionTransformer that transforms the AuthRequest to a Future[NotificationRequest].

So as you can see we can keep adding more ActionBuilders, ActionTransformers, ActionRefiners and ActionFunctions to layer more functionality onto a request before it reaches the core action code to keep it lean and focused on doing one thing: rendering a view, redirecting, etc. Here’s what our action looked like at the end of all this manipulation:

This felt clean and fluid, and is asynchronous as well! In your next project try using these composable functions too and create cool DSLs that read like sentences. In the next post I’ll talk about the other ActionFunctions.

[nitindhar]

engineer your life!

me

[nitindhar]

Learning from Steve Jobs -- Life By Design

Just a week ago, on my way back from Maine, I stopped by a little bookstore in downtown Portland. The first thing that caught my attention was a sketchbook with a saddle-strap and recycled paper. The old-school looking cover that wrapped the leaflets inside was just too appealing to resist. As I explored this notebook, I couldn’t help but love the attention to detail of this product. Everything from the leather smell, to the clasps and the handcrafted stitching was done with great care. I also bumped into a book that I instantly knew I needed to read: "Steve Jobs' Life By Design" by George Beahm.

The sketchbook reminded me of Steve Jobs and how he, as a non-technical visionary of technology, was such a product craftsman. I decided to quickly skim through the book and could not put the book down after browsing through Chapter 1 (“Connecting the Dots”). “Make a dent in the universe,” he used to say all the time. I asked myself: 10 years from now, what would I have to show for myself? What dent would I have made on my world? This turned out to be a hard question to answer. In fact, I didn’t have an answer at all. I don’t know if I have or if I ever will make a noticeable impact, but I continued reading through the book.

Every chapter of the book begins with a lesson we can learn from Steve. The chapters are laid out in the order of 3 stories Steve told at his final lecture at Stanford University’s commencement in 2005 on “Connecting the Dots”, on “Love & Loss” and on “Death”. I took notes on a few ideas that stood out. Here are the 5 lessons that I carry with me:

You have to trust in something

I took this very philosophically. Considering our own mortality, it seems logical to question, “What’s the point?”. Steve’s answer to this is that life is short (#yolo) and we might as well try to be the best version of ourselves possible. We might as well pretend that we will live on through our impact on this world. For me trusting that things will work out, or trusting in “life’s process” as he puts it, is a source of hope. It gives me a motivation that whatever I put my heart into, as irrelevant as it may seem in the moment, will eventually have an impact or is at least a small part of a larger picture. Otherwise, why do anything more than the bare minimum?

Follow the way of your own zeal

If you have a calling or a vision, chase after it. What’s the worst that can happen? Life is short, so you might as well make a move to figure out what you want to do with it. For Steve, this happened with both Pixar and NeXT. As he was pushed out of Apple he never stopped caring about beautiful products. There were times when it was imminent that Pixar would sink, but he just pushed through and paved a path for followers. Then he brought his work on NeXT back to Apple. No matter where he was going to be he was going to work to fulfill his vision. What I’ve learned from Steve is that even though I still struggle with this, I’m not going to stop. I have faint visions of what I want to do and where I want to be and if that’s a path not frequently taken I have to take it nonetheless. I have to “trust in something”.

Imagination is more important that knowledge

Pretend you walked into a toy store and bought a box of Lego. Now, you can follow the instructions inside the box and build the amazing looking building on the box cover, but just as soon as you’ve learned how to build that you’re done. How do you build the next building? You have to use your imagination. Knowing how to do something is definitely important, but using your imagination in combination with your skills is really what sets one apart. If you can’t accomplish that, sorry, its long-term-game over!

Marriage is all about surrendering yourself to a third entity -- the relationship itself

It was quite surprising to find this in the book, but it resonated with how I think about marriage. To me, this is about having faith in your relationship. The sooner you throw your ego out the sooner things will start working well. Surrender yourself to the partnership and trust in its process. I don’t have all the answers, but I know things will work out.

The only way to do great work is to love what you do

I am a 100% believer in this. In the long run, the only way to do great work and have a vision for it is to love it completely. This also applies to other things in life, but that’s why it’s important to keep finding things you absolutely love to do. This is often a cliché that is repeated too often without understanding what it means to love and to be dedicated to a single purpose. Once those two facets are understood, one can be on his or her way to truly “doing what you love.”

In closing, Steve Jobs has left a clear imprint on my mind and I can’t stop thinking about the things that I’ve learned from this book. Places like Portland, Maine are helpful in reflecting on these thoughts. They are quiet places where your mind can breath free of the daily stresses life, so that you can focus your energy on doing great work. I recommend a visit to Portland and Steve Jobs’ Life By Design to everyone!