Ramblings @utilmax-blog - Tumblr Blog

Ambitious

Trends in spending

Here is something I made to analyze my expenses.

I signed up for Mint a long time ago, and for many, Mint provides enough visuals to tell a story with your finances. But I think there are several problems with Mint:

Mint’s method of classifying your transactions relies on an internal dictionary that looks at the description of the transaction to tag it with keywords, but it’s uncertain whether any machine learning is involved, i.e., this dictionary doesn’t evolve as more users categorize their own transactions.

Mint’s main visualization is a pie chart, which is terrible because depending on the variety of your transactions, it can be hard to determine relative slices of the pie.

With Mint, each view takes several clicks to access; I want a single page where I can view everything together instead of going back and forth

Sometime last year, I decided to come up with my own solution for a series of visualizations that would help me get a better handle on my expenses*. I jotted down some questions I could potentially answer with this data:

What are some trends in spending over time?

Does my spending increase around raises, bonuses, or times of stress (when I feel like I need a reward)?

Who are the top vendors getting my money, and how does this change over time?

Can I infer life events from this data, such as career change, single vs. having a girlfriend, etc. (sadly the latter is not possible due to limitation on historical data)

Identify areas where I’m spending much more money than I should

Just in total, how much money am I spending - is this enough to scare me into spending less

And more. I never found time for this, and then life got in the way (new job and such) so I forgot all about it. But I had some time on the plane to Hawaii to finally put this together. I used the following tools:

R - my new go-to language for data manipulation and transformation. I used R to compile all of the spreadsheets I downloaded from Mint into one source, standardize them, and recategorize them by looking for basic keywords in the description. About 30% of my transactions were uncategorized to begin with, so I looked for similarities in vendor and description between an uncategorized transaction and the pool of recategorized transactions. Unfortunately, some transactions were not detailed at all and it was impossible to tell from merchant name alone what they were, so they were excluded.

Tableau - I took my export from R and fed it into Tableau, then started making some quick views using the LinkedIn color palette.

I’m happy with the result and found the following after playing with it for a little:

$64,666 spent from 1/1/2013 - 5/23/2015 (when I last downloaded the data). Wow.

Q3 2014 is interesting because I was unemployed for most of the quarter. Knowing myself, I was bored and had too much free time so I spent too much, especially on eating out.

My spending peaked in Q4 2014, but this makes sense: I was buying furniture for relocation, I was buying all of my flights in advance, I took a trip to Seattle, and spent a LOT of money on Christmas gifts this year.

It’s comforting to know that my expenses have gone down levels identical to before my new job... basically I went through an initial period of spending way too much money because I was happy with my new compensation, now I’m over it. Economists would say I front-loaded my consumption once I had the expectation of higher wages.

Chevron, Amazon, and Southwest are the top 3 merchants overall, with the next few being car related (Uber’s bubble is surprisingly much bigger than expected...)

The peaks in the time series of expenses for the Car category are mostly due to service and mods. Since moving to SF, expenses in this category have dropped, especially Gas & Fuel.

While Car expenses have gone down while moving to SF, what’s increased a lot is Transportation (Uber) and Travel.

I spend slightly more on Alcohol & Bars now, probably because I feel more generous about buying rounds now.

Groceries have been pretty volatile in the past, but have definitely dropped since moving here. This is a tricky one because I’m sure most of my recent groceries have been alcohol.

Food & Dining was very high in the second half of 2014, again, due to seasonality but also because I was unemployed for almost 2 months in Q3. I wish I could say this category has decreased a lot since LinkedIn provides food, but this isn’t true - it looks like knowing that I get free food convinces me to justify eating out at pricier places.

Shopping has peaks in every Q4, which is no surprise given seasonality. I really went all out this past Christmas though.

My favorite one is Coffee. I’ve spent over $2,000 on coffee, and this doesn’t include some of the gear because I sometimes paid for my gear with debit or Paypal for some reason.

You can see the first peak in coffee come in Q3 2013, when I started being really interested in third wave coffee shops. Unfortunately, while I hate Portola’s menu now, it’s at the top of my expenses, with Bar Nine (my favorite place in West LA) coming second.

Q1 2014 is the first quarter after I got a full coffee kit, and from there you can see my consumption rise drastically, as I was buying beans regularly.

There’s a lot of variety in the coffee shops I visited because while unemployed, I made it a goal to try almost every place in LA.

Coffee expenses are much, much lower now because there are coffee bars at LinkedIn.

Still playing around with this to see what other things I’m not paying enough attention to.

Thanks for reading!

* At the same time, I wanted to try and come up with another algorithm for classifying my expenses, and then I figured, shit why don’t I just go all of the way and gather all of the transactions I’ve ever made? This was the toughest part, because banks do not let you access historical transactions online prior to a certain date. Wells Fargo sent me paper statements since inception of my account, but I’ve been too lazy to go through and enter that data.To make the data collection easy, I was able to download transaction data from Mint for accounts I had linked - which turned out to be my two main credit cards. Luckily for me, 90% of my expenses are made with my credit card(s), the other 10% being rent and one-time extremely large expenses (e.g. investing $10k at a time).

“How can I solve this problem at scale?”

This is what I think about with every challenge at work - I wonder if people think I’m just over-analyzing trivial things

Applications to LinkedIn since the start of 2015.

97,391 applications, 57,586 unique candidates, 1,585 unique jobs.

#madethecut

LinkedIn: Month 1

In short, my first month at LinkedIn has been amazing! At times, it still feels like a dream: I sometimes pause and wonder if I really work here.

On my first day, we had our New Hire orientation and I had a chance to speak to an employee about his experience at LinkedIn. His response was simply, "when they talk about culture being the priority... they really mean it." This is something that's been validated in all of my conversations with LinkedIn employees, even before joining the company.

Yes, the perks are outstanding and there are so many that it would take me a long time to explore all of them. The food is delicious, free, and we get fed throughout the day (I don't buy groceries or eat out as much as I used to, which is awesome). There are a lot of tangible things that employees can take advantage of, like all of the free company branded items that come with each new product or program, fitness classes, alcohol on Fridays, cooking classes, movies on campus, laundry, and more.

But aside from all of that, the people at LinkedIn are amazing. Maybe my expectations are low because I came from consulting, which is focused on servicing the client. Here are the major differences I can spot when comparing LinkedIn to BRG (in case you're wondering, I don't think this list is unique to BRG; my entire team came from consulting and they can make the same observations):

1. Credit where credit is due

At career fairs, consulting firms like to claim they have very flat structures. In reality, they don't. Because consulting revolves around the client, it's ultimately a more senior person that makes the final call when it comes to projects and presentation. Your managers are likely to give you the tasks they don't want to do themselves, then they take the credit for it. I understand this makes it easier for the client since they have one reliable contact, but I'm still not a fan of this setup. At BRG, I remember writing a 9 page white paper with tables, charts, and models that I created, only to see my name removed from the title page in the end. That always pissed me off.

Don't get me wrong, I'm not saying I need that attention / need the client to know that I created it. But for my managers to tell me to write the entire paper by myself, and then remove my name from the end result and claim it as their own is just rude.

During my first month at LinkedIn, various managers came asking for me to share my analytical expertise with them to help them make better decisions. For the Campus team, I created a model to estimate the number of interns we should be hiring each year into each business unit in order to meet our employee growth target for the next 5 years. As a complement to that, I created another model estimating the increased strain on our interviewers for each additional intern hired, and analyzed our current interviewer pool to see whether we have the manpower to handle this, or if we need to train more interviewers. Today, these were presented to the execs and I also had the chance to present on the importance of data completion and integrity. For the facilities team, I modeled the growth of our employee base to see which teams will outgrow their spaces before the construction of our new campus is complete. This was all during the first month.

2. Everyone's an owner

At BRG, it was common practice for most managers to hand off the work they didn't want to do to the junior staff, then leave for the day or play games on their computer. Extremely stupid. People who stay in consulting long enough then think they deserve to do the same to the new generation of consultants, and the cycle repeats.

At LinkedIn, nobody has an office unless you're an executive. This means my entire team is sitting next to me, and I've noticed that often times, my manager is working on things that could easily be given to one of the team members. But the fact that he doesn't just hand it off to someone so he can go home, is impressive. I think for managers, it's easy to think that you're too important for certain tasks, but at LinkedIn, becoming a manager just means that in addition to your responsibilities, you are now also responsible for a team. Delegating tasks only happens when someone has a clear comparative advantage (e.g. Chris, since you know this data better than anyone, can you provide me with this information that I can't gather myself), but in no way has delegating occurred simply for the manager to dick around.

3. No jerks

Seriously, the hiring process is designed to only hire candidates who share LinkedIn's values (side note: my interviewers' notes said I was "so LinkedIn" that they had to hire me). Basically, we look for people who get shit done, but have fun while doing it. We look for humor - every meeting, no matter with who, has a moment where everyone breaks out laughing. People understand that we all have lives outside of work, so you have the freedom to work from wherever you want and schedule your day as you see fit (as long as your work gets done, of course). It's totally okay to leave in the middle of the day to attend a coffee demonstration or play some ping pong. I could go on and on about how the quality of people here is just on a different level.

So after a month here, talking to my coworkers about how unbelievably different LinkedIn is, there's one question that makes us wonder... why the hell do people stay in consulting for so long?

On LinkedIn, you see a lot of people sharing posts about going after your passions, working toward that dream job, etc. I'll end this by saying I am extremely grateful that everyday I go into work and am given the opportunity to make a huge impact on the company. I am so glad I've found my home at LinkedIn!

Using LinkedIn to Get a Job at LinkedIn

I think it was back in March that the thought of finding a new job crossed my mind for the very first time.

The culture at BRG wasn't working for me anymore, and I began thinking that the entire industry just wasn't one that I wanted to be in for the rest of my life. A few months later, I finally had the time to gather my thoughts and the first question I had was: where does someone with my background fit in at the big tech companies? Making the move to a tech company is a no brainer. Their awesome pay, benefits, and perks are now enough to poach those in consulting, finance, and investment banking. Many of the companies rank extremely high in employee satisfaction too (LinkedIn was recently named the top place to work, and employees give the CEO an approval rating of 99% on Glassdoor).

But where would I fit at LinkedIn? With extremely tough competition, how could I stand out and get a job there? To answer that, I turned to LinkedIn.

The first step was to fill out my LinkedIn profile and make it more complete. Then, I searched through jobs at LinkedIn, and with my Job Seeker account, was able to look at the skills of other candidates in the applicant pool. I had a good idea of what teams I should focus on, so the next step was finding somebody on that team to learn more about what it's like working at LinkedIn. So I did a search for people at LinkedIn in the roles I wanted, and found one person who had a very similar skill-set, same education, and looked very approachable (this is where the importance of having a profile picture on LinkedIn comes in).

The problem at this point was that we had no mutual connections. So I used LinkedIn to search through my own network and see who could connect me to this guy. Lucky for me, a close friend of mine knew a guy at Xerox who was now working for LinkedIn in New York. He put me in contact with him, and that guy put me in contact with the person I found in my search.

From there, I learned a lot about the team I wanted to be on, what skills were required, and what technologies were being used. The daily updates on LinkedIn through Pulse were very helpful with the entire process too (from preparing for tough interview questions, to tips on negotiating salary).

Then came my break. In late August, I got a referral for a position on the Business Operations and Analytics team, came very close, but didn't get the job. At the same time, I had ongoing interviews with Google, Facebook, and Amazon, but knew that LinkedIn was my first choice. The hiring manager I spoke with at LinkedIn was kind enough to refer me to a colleague of his who was hiring for a nearly identical role, and after getting coffee with him and meeting his team (I have to point out that LinkedIn had the best candidate experience), I'm extremely happy to say that an offer has been extended!

So now that I've experienced the power of LinkedIn, I hope that others will realize how useful it is. And this is coming from a guy who used to absolutely hate networking.

Can't wait to start making an impact at LinkedIn!

Search for these keywords and you'll see one of our cases all over the news.

Long story short, Steve Jobs reached out to CEOs of other large tech companies and asked for an unofficial agreement to stop poaching each other's engineers. By agreeing to stay away from hiring each other's engineers, wages were supposedly suppressed due to the lack of demand.

Adobe, Apple, Google, Intel, Intuit, Lucasfilm, and Pixar formed the 7 main companies in the web of agreements. One of these companies hired us to analyze the compensation of their own engineers over time. My task was managing and analyzing the employee data. We constructed an econometric model regressing real compensation on a multitude of factors such as the duration of the agreement, compensation from the last two years, age, tenure, gender, macroeconomic variables such as employment in Silicon Valley, and firm-specific variables such as number of new hires, number of transfers, and revenue per employee, and found the extent to which wages were (or were not) suppressed.

So if you see a large number being thrown around in the news when it comes to the discussion of under-compensation... I calculated that!

This case took up a lot of my time (I was working 60-70 hour weeks), but it was definitely one of the cooler cases I've worked on. It's satisfying to see my work in the news!

It's time to leave consulting!

Some help would be nice :|

Discrete Choice Modeling Doesn't Apply Here.

Growing up, my favorite video games were RPGs such as Final Fantasy, Chrono Trigger, Star Ocean, Xenosaga, and adventure games such as X-men legends and Champions of Norrath. However, I didn't like Grand Theft Auto and similar games. Since graduating college, I've figured out why - I liked games with a certain level of structure, where I still had some freedom in traveling through worlds, but not so much freedom that would prevent me from making progress because I'm stuck in one place. At the same time, too much structure (games that were too linear, such as the early Mario games) were not to my liking either.

I realize this now because it reflects how I feel about the real world after college. In the first 22 years of your life, you are in school. You start out with a very structured program in elementary school, where everybody is taking the same classes, until you get to middle school where you are first exposed to the idea of elective classes. As you continue through 12th grade, you receive the freedom to pick more of your classes - this is the removal of structure from your life. It is the transition from Super Mario Brothers to Chrono Trigger and then to a modern RPG such as the Final Fantasy games in recent years. Next, you're in college and while most of your schedule is up to you, you have a general idea of what courses you should enroll in because you want to specialize in something. There's more freedom than ever, but there is still some structure to what you are doing. And for me, I was comfortable with that because I always knew what to look forward to and I had an idea of what I could benchmark my performance to.

Then you leave school and you enter the real world, where there are no rules. Want to eat candy for breakfast? You can. Want to live in another country for an undecided number of years? You can. Want to quit your job in finance and find work with a nonprofit organization? You can! You can do anything and live anywhere as long as you don't become a slave to money. That level of freedom is great, and after 20+ years of living a structured life, it is well-deserved.

But when I think about the overwhelming number of choices you can make, how many variables are changing everyday, it is quite intimidating. In textbook-Economics, we use Game Theory to model discrete choices for n number of players and put together models/decision trees to lay out all available actions and contingencies. These were simple models with a finite number of actions for each player. But in the real world, your actions are infinite. To add another layer of complexity to it, not only are they infinite, they are not discrete - your choices are not only "work" or "shirk", but they include all levels of effort (or lack of) in between. This gives the 'game' infinite dimensions.¹

So how do you know that the path you're currently on is the optimal path? The best we can do is pick snapshots in time and compare ourselves at these moments (e.g. telling yourself you want to be a doctor at the age of 30, and evaluating your progress then). Another problem is, modeling this 'game' would show a decision tree growing exponentially. Even a slight change today could greatly affect your available options tomorrow and put you on a completely different path the day after - this is the butterfly effect discussed in Chaos Theory.

A big problem with humans is the feeling of regret. Regret happens because we don't really know exactly which paths are laid out for us at time t=0 and therefore cannot decide which is the optimal path to take. It is only in hindsight that the optimal path is revealed, and that causes regret. When you're deciding whether or not to invest in a stock, you don't have complete information to be able to perfectly forecast its price movement. You may have an educated guess using Statistics and Econometrics, but you can't say with 100% certainty what the next day's price will be because the movement depends on a large number of players and economic factors, none of which you can predict or control. It is only in hindsight that you tell yourself "man, I wish I had bought that stock when I had the opportunity" or "I could have made $10,000 if I bought it!".

Obviously you shouldn't let regret consume you, and the best strategy is to consider your options today and make the best choice. But that's not a foolproof strategy because at the same time you are considering your options, the next day's options are evolving as well. Your paths are being lengthened as time continues, and like movements in stock prices, you can't predict what directions your paths will follow. So you may be able to say that getting a degree in engineering will secure you a high-paying job when you graduate, but nothing in life is certain.

So that's what scares me about the world. There are an infinite number of paths you can take, and these paths are constantly evolving as time continues - evolving in such a way that you can't model your actions with certainty. There's a fear of missing out on opportunities. There is a lot to learn and see and you don't know how today's actions will change tomorrow's actions. You find out that the more you know, the more you don't know. All of the Economic and Statistical modeling in the world can't help you. I don't like that level of uncertainty, while others more courageous than myself relish it.

Going back to the video game example:

I would subconsciously follow a certain strategy with RPGs. Let's say I am in a room with 4 doors to different rooms leading to more rooms, one of which is the "correct" path to fighting the final boss. I would employ a divide and conquer algorithm and explore all possible door combinations to make sure I didn't miss out on any treasure or items. I knew this strategy worked because RPG games back then had a finite number of paths. Granted, I don't think video games have reached that level of complexity where infinite paths are laid out, but after a certain number of available paths I would be too lazy to play the game (that's why I have no interest in Grand Theft Auto).

If only I had a what-if machine, like in Futurama.

--------------------

1. I doubt anybody will think of this counterexample, but some may say that the problem of infinite actions can apply to school as well, because there are varying levels of effort and dedication to school. I will admit that one's effort level today can be an indicator of his effort in the next period, which can appear to be similar to the idea of infinite paths. But in that case, I argue that the grading system ultimately turns these infinite paths into discrete ones (after all, you only need to receive a passing grade to continue, right?).

It's very rare that a case we work on gets publicized to this extent, so I'm happy to be able to share this. This is one of my earlier cases that I worked on after roughly 7 months on the job (I was still a n00bie).

I worked with data from the Office of Statewide Health Planning & Development containing around 15,000 variables (financials, discharges, and a bunch of other things I did not use) on all non-profit hospitals in California. I combined this with population and demographic data from the American Community Survey and Small Area Health Insurance Estimates, on a county level. Using SAS, I built a database from these 3 sources, understood its nuances, and ran through iterations of an econometric model developed by our team. Using these regression results, I created the exhibits you see in this report, estimating the economic damage to hospital services, charity care, state salaries, and taxes in California.

Direct link to the report here. Figures 1 and 2, and Exhibits 1-5 were thanks to me!

Buffett nails it by saying to invest with Jack Bogle.

I've been saying this all along!

A class-action suit by Silicon Valley engineers against companies including Google, Apple and Intel has revealed details of an agreement among them not to cold-call one anotherâs employees.

One of my recent cases in the news, as well as on Reddit.

Arthur Chu has won thousands of dollars on Jeopardy by using game theory.

Awesome.

I have so many topics I want to blog about (seriously, I keep a list on my phone) but I'm too lazy to actually sit down and do it...

A lawsuit against a man who ran websites which linked to episodes of The Simpsons and Family Guy has ended in the most expensive way possible. The judgment, which awards Fox $10.5 million in statutory and punitive damages, is the highest amount ever awarded by the Federal Court in Toronto, Canada. Speaking with TorrentFreak, the target of the lawsuit says that he is now going through a bankruptcy and Fox are chasing

Following my last post's theme... another extreme calculation of damages! However in this situation, I feel less bad for this guy because it's very likely his website was generating revenue through ads.

Earlier this week a torrent site user was hit with a damages claim of $652,000 for uploading one movie to the Internet. With the huge amount undoubtedly still ringing in the 28-year-old's ears, questions are now being raised about how this figure was arrived at. It's an amazing process that shows that sometimes copyright holders may as well just think of a number, double it, multiply it by the day of the week and then add it all to their dog's age.

I came across this on Reddit and had to share it here. A good part of my consulting work includes the calculation of damages in situations such as this. Someone in our office is working on a DRM-related case right now, actually.

On the surface, the number derived by this consultant is ridiculous. It's true that our work is dependent on assumptions - it looks like this consultant's assumptions are a little too extreme. I'd love to read his report... I wonder how many holes are in his analysis.

Maybe I'm being too rough on the guy and his numbers actually do work out. Who knows.

Trending Blogs

Recently Viewed Blogs

Ramblings