Asking better questions
Here is a tough reality for most “data science” positions: nine times out of ten, the question that matters most doesn’t need a cutting edge technology solution. It needs a dashboard, a nifty spreadsheet, or a simple report.
Once upon a time, my data science team worked in a supercomputing environment for a whole year. We had access to petabytes of data to answer whatever questions we could ask around a general theme. I wrote beautiful stuff using the latest and greatest libraries and frameworks. I optimized aggressively, changing my code from Python with a shim to native languages.
At the end of the day, the things that produced the most tangible outcomes for that particular project were some canned jobs and an interface for analysts to write some ad-hoc queries for follow-up on whatever the jobs produced.
I had not initially been asking the right questions.
In a now-moderately-famous tweetstorm, venture capitalist summed up the power of code in a few sentences:
Code and media are permissionless leverage. They’re the leverage behind the newly rich. You can create software and media that works for you while you sleep. An army of robots is freely available - it’s just packed in data centers for heat and space efficiency.
“Leverage” has become a bit of a watered-down term in the past decade or so of corporate-speak. More’s the pity. It’s now commonly used to denote simply “using something well” minus its original implication of “with some sort of amplification beyond what you could do on your own.”
Stock traders have a notion of margin: a trader will take out a loan to purchase additional shares of stock than they could afford otherwise. If the price goes up, they pay back the loan and pocket the extra dollars. If the stock goes down, they’re still on the hook for the loan amount. Their equity position is leveraged. The gains are highly amplified, but the losses are too.
Code is a form of leverage. It doesn’t make ideas good or bad, but it amplifies them both alike.
A bad question with code amplifying it becomes a much worse question.
From 2014 through 2018, Amazon tried to roll their own AI-driven resume evaluator that did not work out the way they intended. Their apparent question: “can we use the resumes for successful candidates over the past ten years to predict which resumes we’ll like going forward?”
The results were unfortunate. From the article:
Top U.S. tech companies have yet to close the gender gap in hiring, a disparity most pronounced among technical staff such as software developers where men far outnumber women. Amazon’s experimental recruiting engine followed the same pattern, learning to penalize resumes including the word “women’s” until the company discovered the problem.
The good questions perhaps a few steps backwards in their decision process might have looked something like this:
- When we look at engineers who have been successful at Amazon over several years, what attributes do they share?
- Are these attributes different across teams?
- Are there any apparent biases in these attributes?
- Do these attributes appear to align with the attributes we think we’ll need over the next 5-10 years, or should we be looking for different ones?
These questions don’t need “artificial intelligence.” They need access to the Amazon performance review database, a little bit of SQL, and some forward thinking on the part of management.
Amazon spent four years and untold amounts of engineering code to answer the question they probably didn’t want to answer: yes, we’ve been biased in our hiring over the past ten years. Not how to hire better engineers going forward.
Princeton computer science professor Arvind Narayanan recently gave a talk on “AI snake oil” where he tangentially touched on the topic of bad questions. He breaks advances in AI into three rough categories:
- Perception (content identification, speech to text, facial recognition)
- Automating judgment (spam detection, essay grading, content recommendation)
- Predicting social outcomes (predicting recidivism, predicting job performance)
In “perception” tasks, AI has advanced to the point of human accuracy. Given enough data (which is itself a challenge) and compute, AI can get the job done. In “judgment automation” tasks, AI can mostly get there, but there is a band where humans may disagree. If I train a content recommendation algorithm on one human, but then try it out on a slightly different human, I may get slightly different answers. That’s not necessarily a problem, but it does mean that the data scientist needs to be concerned with how important that yes/no distinction is.
But with “predicting social outcomes,” we see abundant quantities of snake oil - and therefore bad questions. Dr. Narayanan walks through past experiments in predicting recidivism, showing that artificial intelligence methods prove no better than random scoring. And, more perniciously, it predicts “re-arrest” rather than “recidivism,” because that was what the available data tracked.
So not only does the algorithm not work, but it also starts to equate “re-arrests” with “criminal activity.” It doesn’t ever consider if the arrested individuals were later tried and proven guilty or innocent. It assumes that if they were arrested, they must have committed some criminal activity.
Code as leverage. It doesn’t make a question bad or good, but it does make a bad question worse.
If bad questions can be made into horrible questions with data science, how then do we ask good ones?
Suggestion 1: When considering questions, figure out how to solve them with human intelligence before adding data science.
Consider a data science task where we need to segment customers into groups based on likelihood of responding to a particular marketing message. This is commonly done with some sort of machine learning clustering task, but the need was first fulfilled by humans. You’d do focus groups that represented cross-sections of your populace, show them the marketing materials, then gauge responses. Clustering, at its best, is intended to identify better cross-sections of the population, but the ultimate need here is the same.
A human could perform a clustering task. The human would identify sensible numbers of groups (or, in this example, market segments). They’d consider the data available on each of their sample people and place that person into one of their groups.
We can describe our task alongside a human approach to solving the issue - thus, we have a way to validate if anything we do in data science makes sense. Would a human make the same choices given the same data?
Suggestion 2: Identify the discrete problem and describe an abstract solution before writing a single line of code.
On the surface, our marketing problem looks simple. “Break these people into groups.” But there’s nuance there.
“Break these people into groups” feels rough when we say it out loud, but that’s how a lot of data science tasks actually start without foresight. Segmenting into groups that are all 100 people wide, or segmenting into groups by age alone, or by state of residence, would all meet that task’s objective.
“Okay, okay,” you say. “Break this population into groups based on likelihood of responding to marketing.” But now I think: “does that mean ‘all’ marketing? Television versus radio versus online? What does response mean: they liked the ad or they actually went and bought something?”
“FINE.” you now say. “Segment this population into groups based on their likelihood of purchasing a product after viewing online marketing.” I continue to ask: “any product?” “Yes.” “Can you show me how we can track this through all of our systems now - from the advertising release, on through viewing, on through final sale?” “Sure, it’s like this.”
“Segment this population into groups” –> “Segment this population into groups based on their likelihood of purchasing a product after seeing online marketing.”
I have a discrete question now. What about my abstract solution?
Initially, this problem looked like a clustering problem (break into groups). After we refined the question, though, it starts to look like a classification problem. (Buys/does not buy product) We can start to say “my output should generate a ‘likelihood to buy product’ score for each customer. Then we’ll place all of those on a line and group into 0-20%, 21-40%, etc.
All of that was done and written down without a single line of code. But now we know what we want.
Suggestion 3: Map data to your proposed solution. Confirm that it’s the right data.
If we’ve confirmed our question and loosely confirmed our approach, our next thing to nail down is our data - which, in my anecdotal experience, tends to be where things break down most often.
Let us re-imagine our earlier market segmentation problem. Assume that the organization tracks marketing impressions and clicks, and assume that the sales team captures data at the time of sale. Seems great on our surface, right? But - maybe our marketing team has one system that uses a specific Target ID for each prospect, and the sales system has a completely different numbering scheme for sales accounts.
“FINE.” You say again. “I’ll join them on first name / last name and email.” But the sales team can’t give you that information, because it’s personally-identifiable information, and according to company policy it can’t leave the sales system. Your project might be dead on arrival, but that’s at least something you can find out (and hopefully fix) before your project gets started.
This problem is masked from many junior data scientists because so much pre-work goes in to getting it right. Kaggle competitions come with data. Managers should do plenty of pre-work to make sure their team has what they need to get started. A big leap forward in the data scientist’s career is realizing how to go get the data they need to solve the question that will have the largest impact to the business.
Dashboards and spreadsheets
When I first began writing this post, I started with a simple premise: sometimes a simple dashboard or a spreadsheet is the best way to solve a problem. Sometimes, when we frame our question in the right way, that’s all we might need. And that’s okay. That’s why the data scientists were brought in - to answer the need on the table, not try to rope in the niftiest new tech.
But when we get to the point where knocking through those dashboards and spreadsheets becomes rote, automated, and easy - then we open the door for the really challenging problems.Tweet
You should have a coach / you should be a coach
I became an official manager of people in 2017. At Deloitte, my current employer, this comes with a substantial shift in responsibilities. I began getting included in a lot more strategic discussions with our executives. My performance metrics started to include business development metrics (sales, marketing pieces, etc.) and my corresponding expectations for time spent on client projects on a day-to-day basis went down. This is all fairly comparable for how most professional services firms work as individuals move up in rank.
But with Deloitte, there’s something new added: I also picked up the title of “coach.”
A coach is a person a skip-level above you whose responsibilities are aimed at guiding you and championing you in your career. A coach has several responsibilities:
- to mentor their coachees
- to represent their coachees well during annual performance reviews
- to keep their coachees in line with the strategy of the business
- to expand their coachees’ networks to other leadership
- to make sure their coachees don’t accidentally step into problems due to inexperience
- to slap their coachees’ wrists when they fall out of line with compliance
- to help them grow their careers
This role (in my opinion) is the single greatest talent-growth differentiator we have. Every organization should have a program like this.
Interlude: a brief word on the environment
For some of our coaching structure to make sense, there are two key pieces of structural context you’ll need.
First, Deloitte is a professional services firm, not a product firm. We sell projects: doing taxes, or writing a research report on Belgian pastry law, or building a nifty dashboard for some arcane mainframe software. Most staff will work on 3-6 projects in a performance year, which means it’s entirely possible they may have just as many different managers. That’s different than most organizations where you have a pretty firm idea of who your boss is for a whole year.
Second, our annual performance reviews are decided by a group of our partners, not by an individual project manager. Sure, those project managers weigh in, but the decision is made by partners.
And, finally, Deloitte is a supermassive company with a lot of people spread out over the entire world. What I am describing below is my own experience in my own little corner of the company; to be honest, I’m nearly certain that the experience may differ across regions and countries.
With that context - let’s talk about coaching.
Everybody gets a coach
Everyone has a coach on their very first day. You don’t have to know anyone, you don’t have to even have an idea about your future path yet. Your first coach is assigned to you. That assignment is somewhat haphazard: it will typically be someone in your division and someone who’s running a little light on coachees (so the average coach has six coachees, but one coach recently had two people leave for another job, they’ll be first on the list for the new crop).
Naturally, this matchup might be suboptimal at first. It’s pretty common for a coach/coachee to get to know each other and realize their personalities aren’t a good fit, or that the coachee might be looking for a mentor with a similar (or different) background to their own. Because of that - coach switches are free! You literally email an address that’s on your own personal HR dashboard and say “I want to change my coach to this new person,” and it’s changed. There’s no review process, there’s no approval… it’s just done.
This assigned-but-changeable relationship gives us two perks: it gives the new folks an anchor point into the culture from the beginning, while giving them freedom to adapt as their knowledge and experience with the company grows.
Your coach represents you at your performance review
Recall our peculiar context: annual performance reviews are group settings, taking into account all of the different projects you may have dealt with over your year. The first thing every coachee learns is that their coach is their advocate with those group members. The coach weaves a cohesive narrative throughout the year and helps with some of the intangibles (like: “hey, this particular project was less great than the others because it was her first time doing this, so we should cut her some slack”).
In an ideal scenario, the coach and coachee should be regularly checking in throughout the year so the coach doesn’t have to try to wrangle a good performance story out of an avoidably bad one. There are no forced requirements for this. The one meeting at the start of the performance review and the one meeting at the end are the only “requirements.”
Your coach is there to grow you with the rest of the business
It goes without saying that direct managers have a broader view of the business’s strategy than a practitioner. And their managers have an even broader view of the business. We believe that giving people an avenue into that broader business picture - as they need it - is important for helping them grow. That’s why our coaches are a skip-level above their coachees. It lets coaches bring in some of that broader business throughout conversations with the coachee, letting them see how strategy is being shaped beyond the things they see every single day. And, on the flipside, it’s good for the coach: they get a vantage point into some of the front-line that they might not typically see otherwise.
Your coach is an arbiter between you and your front line managers
I think single points of failure are, as a general rule, not great. And having a single relationship (the manager/staff relationship) as a single point of failure for an individual’s career is not great, nor is it grounded in the reality of how most people’s day-to-day job operates.
Let’s give a hypothetical: we have a talented systems engineer who has worked great with the security teams and system administrators, but she doesn’t really enjoy working with non-technical people. A re-organization happens and our engineer is now on a team led by an MBA type who’s more of a product manager than an engineering manager. They butt heads. MBA type starts marking the systems engineer as difficult to work with in performance reviews. Engineer gets a bad review and no raise. Engineer bounces to go work for another company.
Now, let’s add a coach in to the mix. Remember, the coach is a skip-level rank above - not necessarily in the chain of command but certainly someone our hypothetical manager will likely know and respect.
Coach hears about the MBA vs. engineer personality divide in a check-in with their engineer. There’s the first opportunity to fix the problem: maybe the coach can offer some tips to help the engineer work better with their new boss. Okay, maybe that helps a little, but it’s still not great. So coach starts asking if maybe it’s better if the engineer changes teams to work more with security. Maybe that fixes it! Or maybe it doesn’t - and maybe the issue is that the engineer just isn’t going to be happy with the new regime. That happens too.
But at any rate, we’ve had several opportunities where the coach can both try to course-correct with their coachee, and where the manager will have to at least explain themselves to another person why they’re unhappy with that engineer’s performance.
Your coach is just there for you when you need to vent
Look, things happen. Sometimes your manager has an off day and makes a mistake that means the team has to stay late. Sometimes you get passed up for a promotion that you wanted (but maybe didn’t deserve yet). Working with people can sometimes be frustrating.
And sometimes you need a trusted mentor to just call and gripe to, without fearing that they’re going to judge you or that there will be repercussions for you needing to vent some steam. Coaches are there for that too.
I don’t think there is much debate on whether “coaches/mentors are good.” It’s pretty well-established that they are. But we take the next step and say “and organizationally, we are going to make sure each person has one and align our own processes around it.” That’s the next level and something that organizational leadership has to make a commitment to do. From this person who’s both a coach and coachee - it’s worth it!Tweet
Back in January of 2016, I began a series of monthly blog posts cataloging self-experimentation and meditations that I had experienced over the course of the month. I was capping a three-month binge on Tim Ferriss’s podcasts followed by a thorough reading of The Four-Hour Workweek and a stack of Tim’s blog posts. Something deep in my brain told me that I should begin a similar series of experiments along with associated documentation, and I jumped in. I gave up coffee for the first time. I started waking up at 5:30 for workouts. In my final post (January 2017), I just straight ditched food for a bit, inspired by the work of Dom D’Agostino.
Then, as many of my friends have repeatedly reminded me, I went radio silent.
Oh, sure, I wrote plenty of blog posts, but then I’d scrap them all as being mostly garbage. Nothing seemed to really resonate with me like the monthly posts of that year did. And it’s not like nothing happened, either! Major developments at work, changes in my physical fitness - hell, I even got engaged! Yet the words refused to come in a way that resonated with me. So I kept taking notes, and I waited.
This is a post about self-mastery.
This is not a post about self-determination, or self-experimentation. It is also not a post about skill-mastery.
When I was twenty-one years old, I wanted to control computers. I was unhappy about a lot of stuff - my long-distance relationship, my total lack of a job, the fact that my college and grad school programs weren’t really what I wanted but I was too chicken to take bold steps, you name it, I could find a way to gripe about it. So I got into MIT’s Intro to Computer Science course and I started learning how to control computers. I read TechCrunch diligently and was convinced that if I could just get a job as a developer, I could turn my whole life around. That would fix all of my problems.
Fast-forward to age twenty-five, and someone actually gave me a job as a developer. This is one of the worst things that can happen to you if you started learning how to program in order to change your life. Suddenly, you realize that actual software development can be sometimes really terrible. There is nothing about a career in software development that is actually designed to fix your life. You still have the same problems. This was the start of a pretty nasty downward spiral for me.
Somewhere in this mental morass, I stumbled upon a book list from Jack Dorsey, founder of both Twitter and Square. One of his top four books was The Yoga Sutras of Patanjali, a collection of proverbs on the classical philosophy of yoga. There is no discussion of stretching or postures - this book is about mental state and the philosophy of the self. It’s a religious text (and the translation I got had all the commentary), but it shook me up enough times that I returned to re-read it over and over. The first sutra: “Yoga is the stilling of the mind.” My mind was nowhere near “still”, but Patanjali was at least offering the idea that stillness was something attainable, and you had to practice to get there. It sounded better than the utter hellscape that was my then-current mental state of ambition in six different directions combined with second-guessing and a total inability to focus.
I tried everything. I diligently practiced meditation using whatever app I could get my hands on. I thought that some of Patanjali’s ideas on mindfulness might be easier to use if I saw them in a western Christian context that was similar to how I grew up, so I read Brother Lawrence’s The Practice of the Presence of God. I would try to intensely focus while lifting weights, something that had previously helped focus me in the past. I tried to find focus in code. I got a new engineering job, one that made me feel more “developer-like.” It all culminated in a series of fairly epic panic attacks, and then I discovered Tim Ferriss. (I wrote about all of that here)
Now we’re all caught up to the start of those monthly blog posts, so I can start talking about self-mastery.
Tim Ferriss’s interviewees are experts in their respective fields. They might be major celebrities, or they might be total unknowns, but they are unquestionably experts in something. I hoped to one day be an expert in something, though I didn’t know what. I thought that maybe if I copied what they did, I could be like them. My initial model was Tim himself, who is basically an expert in relentless self-experimentation and writing it all down. Anyone can do that, so I tried it out. Then I took Jocko Willink as inspiration too, so I started waking up super-early. I liked Jamie Foxx’s workout plan, so I started doing lots push-ups and pull-ups. I relentlessly copied EVERYONE.
When you’re trying to learn a skill, you can pretty much take whatever route works best for you to get there. I know that I learn computer science concepts most effectively if I read and take notes as opposed to watching videos. The reverse is true for weightlifting, where I’m better if I just watch a video and emulate it in the mirror than if I read about it. When you’re trying to emulate a person, though, you have to do what they do. “How hard could that be?” I thought. All of the steps are laid out in front of me, I just have to go do them.
Jocko Willink wakes up at 4:30 every day and posts a photo of his watch to Instagram. Jocko sells t-shirts that say “Discipline equals freedom.” Jocko was once asked how to be tougher, and his answer was “if you want to be tougher mentally, it is simple: be tougher. Don’t meditate on it. Just be tougher.” Jocko was asked how to avoid injury during BUDS training and his response was “be tougher.”
I tried and failed to emulate Jocko’s level of discipline. I like drinking beer and playing video games and sleeping in. Jocko does none of these things, because Jocko is constantly preparing and training himself. I said I wanted to act like Jocko, but I didn’t want to give any of those things up.
It’s not just Jocko who constantly reminds me of my total lack of self-mastery, though. My roommate Lauren wakes up every morning to hit the gym before 6:00AM and has typically logged a workout by the time I’m pouring my first cup of coffee. My friend and coworker Brian regularly puts his daughter to bed and then goes into his basement lab to take a Coursera course so he can keep learning, while I am figuring out which new game on Steam I want to buy. My fiancée Linda is able to totally forget about Internet distractions and actually enjoy the world around her, while I can’t stop fidgeting with my phone and regularly have to delete social media apps just to stop playing with them.
I am writing a post about self-mastery because I am not yet a master of myself.
I think I can get there. I think I had to first recognize that there were examples out there and I needed to try to copy them. I needed to fail first to recognize what self-mastery really looked like. And then, after I failed, I needed some processing time to recognize that the strides I took to actually get to failure had put me not too far off from a place I wanted to get to anyway! My relationships are in a happier place. I’m in a much more enjoyable career.
My first steps in processing that failure was to simply wave my hands and say that it was all fine. I would look at my successes, say “can’t be good at everything” and move on. But it didn’t stop the gnawing feeling that I could do better. I’m still frustrated when I lose mastery over myself and sleep in past when I told myself I’d wake up for yoga, or when I grumpily snap at my fiancée because I do not have mastery over which words come out reflexively. Shrugging those types of things off is not acceptable to me, and they are regular reminders of my own lack of mastery.
This is where my head is now, pondering on how to better master myself. I welcome advice and wisdom.Tweet
"And what is it now that you have to give? What is it that you've learned, that you're able to do?" "I can think, I can wait, I can fast." "That's everything?" "I believe that's everything!" "And what's the use of that? For example, the fasting-- what is it good for?" "It is very good, sir. When a person has nothing to eat, fasting is the smartest thing he could do. When, for example, Siddhartha hadn't learned to fast, he would have to accept any kind of service before this day is up, whether it may be with you or wherever, because hunger would force him to do so. But like this, Siddhartha can wait calmly, he knows no impatience, he knows no emergency, for a long time he can allow hunger to besiege him and can laugh about it. This, sir, is what fasting is good for." -- Siddhartha, Hermann Hesse
This month, I fasted for 51 hours. It was awesome. This post is going to be the perfect hybridization of “weird experiments Joe conducts on himself” combined with “Joe might think about his digestive tract a little too much,” so I hope anyone reading this is as excited as I was to write it.
Since the middle of November up until the fast that started the night of January 12, I participated in a dietary experiment called the “slow-carb diet.” In essence, I strike nearly all sugars and carbohydrates out of my diet (though I allow a little bit of fruit), and I replace those calories with extra fat, protein, and vegetables. The idea is to trigger a change in physiology to allow your body to more easily convert fat into energy as needed, rather than storing extra glucose in the body as body fat. I did not do weigh-ins at the beginning of this experiment, but I have experienced a reduction in fat around my midsection and thighs.
The slow-carb diet is a modified version of the ketogenic diet, which is the brainchild of researchers at the Johns Hopkins Epilepsy Center and is being tested for a variety of different conditions. One of its core tenets is that you begin the diet with fasting, which triggers the body to enter ketosis and begin using fat as an energy source more quickly.
Now, I do not suffer from epilepsy, nor do I have an insane desire for weight loss that would provoke me to not eat for a few days. I was curious about the impact fasting would have on my psyche. Would I have the mental discipline to go through with it? What would it feel like to not eat for three days? Would I sleep through most of it or be in hungry pain? There was only one way to find out.
So, from January 9 to January 12, I maintained strict discipline on my extremely low-carb high-fat diet, and at 6:00PM January 12, I ate my last meal in an airport. It was a Cobb salad.
On January 13, I woke up, pounded a bottle of water with some salt in it to make sure my electrolytes stayed up, and started walking around my neighborhood to stretch my legs out. I came back, more water, then had some tea with Brain Octane in it. Brain Octane is essentially coconut oil if you took out everything that wasn’t pure lipids in it. It’s a layer of concentrated oil that’s packed with fatty calories. Over the course of the 13th, I would drink about 1000 calories worth of this weird little fat cocktail in order to prod my body along into ketosis by giving it ample sources of fat to use as fuel.
I was expecting my stomach to be in physical pain for most of Friday, but that turned out to not be the case. I felt typical hunger pangs that intensified until about 3PM. After 3PM, the pangs subsided and were replaced with more of a vague awareness that I hadn’t eaten. I didn’t feel a substantial drop of energy, though the caffeine may have contributed to that. I was able to stay on track with work throughout most of the day.
After 24 hours (around 6PM on Friday) is when things started to get interesting. My mental focus went way up, though it admittedly got redirected into an obsessive desire to win a long game of Civilization V. And when I say long, I mean that I had to turn my computer off at 3AM and make myself go to bed, because I really didn’t have a desire to sleep. I was charged up. I had a manic desire to go see if I could do a distance run that I suppressed. My guess is that this is a primitive instinct to forego sleep in favor of going out to kill a zebra or something, and there is some evidence from online research that this weird shift in circadian rhythm is not unheard of in fasters. No medical evidence as to why, though.
I can’t remember exactly when I dragged myself out of bed the next day, but I’m pretty sure it was around 2PM. I woke up late, sleeping through my planned fasting halfway point of 36 hours. I didn’t really feel like getting out of bed but felt I should at least get the blood pumping. Little bit more tea with Brain Octane. More water - at that point I was wickedly dehydrated from being in bed for almost twelve hours. I kicked the water consumption into high gear, because I was starting to experience an occasional heart palpitation and headache that I thought might have been due to a lack of electrolytes.
Still, at this point, I was bored. I was tired of sitting in my house. I was tired of doing the same lazy activities, which surprised me because I thought I loved lazy introverted activities enough to do them for an entire weekend. This was not true. I didn’t physically feel weird, aside from a slight weakness that I’ve felt before after recovering from an illness. I hit the 48 hour mark, fist-pumped, and thought I was going to cruise through the final 24 hours with ease.
That turned out to not be the case. This is where the “you should probably do this sort of stuff with medical supervision” comes in.
From Hour 48 to Hour 51, I started thinking hard about food. I wasn’t getting hunger pangs, but I couldn’t stay focused on anything because the thoughts of food were all-consuming. That was an interesting sign. My headaches came back and I started drinking more water. I started feeling physically weaker. At Hour 51, I started getting a churning feeling in my stomach like I was about to vomit, and when I burped, there was a taste in my mouth that I didn’t like at all. I decided that was a combination of physical symptoms that I didn’t want to discover was a sign of something bad, so I canceled the experiment. I walked to go get a burrito and a torta and the local burrito shop.
Refeeding was a pretty straightforward affair. I nibbled on the food consistently for about 2.5 hours, as I was a little nervous that it might come back up if I didn’t practice some discipline. I didn’t feel greasy or gross after taking out an entire torta and half a burrito (with rice). I didn’t really feel full. Just sort of contented. I still couldn’t get to sleep before 3AM.
I set my alarm for 9AM the following morning to try to force my circadian rhythm back into some sense. I ate the other half of the burrito. I went to brunch. I had pizza for dinner. The combination of all of these things would normally put me into a coma without some serious exercise. No sense of illness, no sense of bloat. I have not resumed the high-fat, low-carb diet since the fast ended. I have, however, found myself naturally restricting my portions throughout the day and eating larger meals at night. Something I might tinker with in the coming weeks.
So, what have I learned?
- I will absolutely do this again. It was an extremely interesting mental shift and a way to shake up my relationship with food.
- You should definitely not try this if you’re trying to lose weight. For starters, it wasn’t really effective for that. Perhaps more concerning than that, though: you might actually cause a medical trauma to yourself. I picked up the burrito because something didn’t feel right. If you’re trying to lose weight rather than just paying attention to how you feel, you might not make the same choices.
- If you’re trying for weight loss or body recomposition (i.e., less fat in bad places), I highly recommend trying the slow-carb diet over an eight-week period. It worked wonders for me without a lot of struggle.
- If you’re trying the slow-carb diet, you should do your homework and you shouldn’t try to mix and match with other diet plans. I’m mostly speaking about the fat content of the diet here. You cannot, absolutely cannot, listen to the “low carb” portion of this diet and ignore the “eat lots of fat” portion. My breakfast when slow-carbing was four eggs, Greek yogurt, beans, spinach, and whole milk. Do not try to dodge the fat.
- I probably don’t need to eat the portion sizes I do. Or rather, I should make sure to tailor my eating to the amount of exercise I’m actually doing that day rather than the exercise I think I should be doing that day.
- If I don’t eat for an entire day, it will not make me grouchy if I go into it with the right attitude. It does, however, help to plan for that.
I tried to be as detailed as I could about my experience with fasting. If you have any questions about what it was like, or are curious about trying it yourself, I’m happy to answer what I can!Tweet
Closing out the 2016 writings
Any and all writings submitted around this time of the year should have something to say about the year as a whole, and I have therefore struggled to start writing anything. I felt like I had something poignant to say at the end of last year, and I put it out there. It was about traveling a lot and being really excited to explore new territory, and it was written with a sense of gratitude that I was finally getting to hop around the country and the world like I’d always wanted to.
I mean, I could write something like that now. I played on the beach in Rio, camped in Maryland, visited Disney World, spent a lot of time in San Francisco, and watched a soccer game in England. Those things are awesome, and they make me recall almost a decade ago around this time when I was planning my first ever trip abroad (I’ve come a long way since then). But the traveling in 2015 was about growth; it was about stretching my comfort zone. The travel in 2016 was about vacation, so it doesn’t make for as good of a story. There’s no character arc.
What’s different about 2016 is the writing itself.
I never announced it. I just had this idea in my head that I would write a recap every month of 2016 and capture what I’d done and what I’d learned. I’d “publish” those recaps, meaning I’d put them on my social media channels where my friends would see them and I’d feel ashamed if I didn’t write one. The reward for me was that I’d have some push to not be a slouch for an entire month - I didn’t expect anyone would actually use them. But some people have told me they were able to apply a few snippets to their lives, which I’m happy to hear.
I’m reviewing those old posts now and it’s interesting to look at patterns. Here are a few:
- I appear to be more productive as the weather gets colder. Around this time last year and going into the deeper winter, I was hyper-productive. As the spring and summer picked up, I struggled to work because I just wanted to be out and about finding shenanigans. And shenanigans were found.
- My productivity starts to crack if I don’t establish predictable schedules. I can look at a few months where I know that I let someone else control my schedule on an ad hoc basis, and those months were generally ones that I watched more TV and did less actual work.
- My fitness slacking that led to my injury in November looks like it had some warning signs around not making the time for workouts that I’ll have to keep an eye out for.
- I didn’t have a goal most of the time. These posts aren’t a pursuit of a coherent narrative; they’re jumbled.
I don’t have a plan yet for how to respond to these things. What I’m doing now is rebasing.
I got injured in November. It sucked. My first ever lower back injury knocked me right on my tail and I decided I was never, ever going to let that happen again.
And that got me thinking: how much of this do I really enjoy if I’m not at peak physical performance? If I read a whole lot of books but sacrifice my sleep to do it, did I really enjoy the books, or did I just sweat to churn through books? Was going out night after night worth it if I just ended up with an injury due to lack of mobility?
So this month, here are a few steps I’ve taken to address that:
- Running at least twice a week. I’m striving to slowly ease myself back to be able to comfortably do five miles. I do not take shin splints or lower back pain lightly while I’m doing this.
- Resistance training at least twice a week. This has come in the form of kettlebell swings, mace swings, and push-ups. Lots of push-ups. This is not done the same day as the running.
- Mobility training. Apparently everyone I know can do a third world squat with ease, but I can’t. Yet.
- Yoga, every day, for at least fifteen minutes. I use Gaiam’s Yoga Studio app, which is really quite nice. It has all of these canned classes that you can save and play at all levels of training. I typically do a fifteen minute class before resistance training or running, and a half hour class on my resistance training off days.
- Sleep. I set my alarm for 7:00AM, turn my screens off at 10PM and put my phone across the room, and try to be out cold by 11PM. I am experimenting with the optimal amount of sleep here, as I’ve still been feeling sleepy with eight hours of sleep.
- Nutritional supplementation. Cod liver oil (to replace things I don’t get in the winter), magnesium glycinate (as a cortisol attacker), and BCAAs (for muscle recovery) all get daily use.
- Reduction in caffeine to one cup of black tea in the morning and a green tea in the afternoon. I’m trying to work on my anxiety levels, and caffeine wasn’t helping right now.
- A slow-carb diet. I’m seeing noticeable changes in the fat around my midsection in just a few weeks.
None of these things have felt like a huge shift in habits. I slowly stacked them on over the course of the month.
The closing is the hardest part of these posts, so I’m keeping this one short. Happy 2016, everybody. Go get ‘em next year.
P.S. here are some books I read:
- Tools of Titans by Tim Ferriss. Run, don’t walk, to go get this one. No matter what your place in life, it’s packed with incredible stuff.
- Surely You’re Joking, Mr. Feynman! by Richard Feynman. Had a lot of fun reading through this one.
- Fear and Loathing in Las Vegas by Hunter S. Thompson. I try to re-read this often as a reminder that the crazy country in which we live has been crazy for a long, long time.
- American Gods by Neil Gaiman. I got excited as I was typing up my review of it in an earlier post and decided to go through it once more.
- God’s Debris: A Thought Experiment by Scott Adams.
- Beatlebone by Kevin Barry.
- The Obstacle is the Way by Ryan Holiday.
- What If? by Randall Munroe
My top books of 2016
I was blessed to grow up in an environment with parents who encouraged me to read. My parents made me save up my money for toys from a young age - I distinctly remember being told to save up my money from the “tooth fairy” so that I could have enough to purchase a $2.50 Lego toy. I never wanted for books, though. We’d go to the library to look for new things to read, and I always got new books for Christmas. Now, at age 28, my parents buy me Legos for Christmas and I buy my own books. Times have changed.
When I first started writing regularly earlier this year, the first comments I received from friends were often about my book lists. Some people picked up a few of the books themselves. Others asked how I picked things to read. I maintain a running list on Goodreads that I sporadically re-populate with recommendations from people I respect. Bill Gates is one of those. Every year he posts his list of favorite books he’s read during the year, and I was inspired to do the same this year.
So, without further ado, the 2016 list, along with some commentary.
American Gods by Neil Gaiman. What if, when immigrants came to America, they brought their gods with them? What if the spirit of Odin still wandered through the colder parts of America - and what if he wasn’t just a spirit, but a walking, breathing man? That’s the premise of American Gods, which follows a recently released prisoner through a road trip to rally the last of the old gods in a battle against the tyranny of the new ones: technology, media, and overspending. It’s an incredibly fun read and it gets you completely lost in a new world. And there’s a bonus: if you read it now, you can be ahead of the TV show coming out.
High Output Management by Andy Grove. I’ve had this book on my list for a while as it comes recommended by most of the tech industry CEOs and venture capitalists who were at the right age to be mentored by Andy Grove. Grove was a engineer at Intel before rising up through the ranks to take the top job, and he had a lot of fantastic lessons about how to manage others. The big eye-opener for me was the thinking that “my work” is no longer what I produce myself, but what I can empower others to do as well - even if they don’t report to me. It’s been a big maturing step for me as I come to the office every day.
Founders at Work by Jessica Livingston. There are two really amazing things about this book. The first is the stories of entrepreneurs who had major roles in products that have now become commonplace: Steve Wozniak with Apple, Paul Buchheit with Gmail, Max Levchin with PayPal. (added feature: you get to learn about how Elon Musk almost accidentally tanked PayPal, which would have made the world a much different place than it is today) You learn that with a few rare exceptions, most of these founders didn’t really know what they were doing at the time, and many of them spent months or years doing the wrong thing. (PayPal was initially going to be a vendor of souped-up VPN tokens) The second wonderful thing about this book is learning about the company founders who aren’t doing something sexy for the news to pick up, just quietly building successful businesses. Stories like Joshua Schachter, who founded Delicious as a pet project while working full-time at Morgan Stanley, or Joel Spolsky, who just wanted to make a software company filled with people he’d like to work with.
Radical Acceptance by Tara Brach. This was the penultimate book in a lot of thinking and practice I’d done in meditation and mindfulness. When I picked up this book, I had been searching for techniques to quiet my mind for some time. My brain ran wild and typically down horrifying paths that were absolutely no good for my well-being. I’d tried the Calm app, the Headspace app, and countless other books and videos to get some control over my mind. Nothing worked. Brach’s book was the turning point for me: it took away the fear and frustration.
Between the World and Me by Ta-Nehisi Coates. All of the praise for this book - it’s all true. This is a book about a world I didn’t have to grow up in, but my friends did. It’s a book for people who couldn’t get their head around Ferguson or Trayvon Martin because it’s a world I never had to experience. It’s a book for people who only really had racism talked about in the abstract, never as someone who’s experienced it directly walking through their thoughts on the concept and how those thoughts evolved over time. I can’t recommend it enough.
Mastery by Robert Greene. Greene opens the book fairly early on by driving a huge wedge between “financial or business success” with “mastery of an art or craft.” He’s not talking about the former. He’s not talking about the celebrity status now bequeathed to Jiro Ono; he’s talking about the years Ono put into perfecting his craft long before Jiro Dreams of Sushi was filmed. Greene doesn’t promise you’ll be successful, well-liked, or emotionally stable: these things do not concern the people he considers masters. Give it a read. For those of us who think we want to “master” something, Greene has some words on what that really takes.
Honorable mention: Seveneves by Neal Stephenson. First sentence of the book: “The moon blew up without warning and for no apparent reason.” This is the story of how modern-day humans would respond to such a catastrophe. What if we knew a catastrophe was coming, we couldn’t stop it, and we only had two years to prepare? What would happen? Could we survive? I had a great time reading this book, and if you want to learn more than you ever thought you wanted to about orbital mechanics, you’ll have a great time with it too.Tweet
On Thursday, November 3, I sat down on my couch with a curious tinge in my lower back. It was the sort of tinge where you think “huh, little stiff there” and try to take it easy. I told myself it would loosen up and woke up the next morning and went about my day.
On Saturday, November 5, I woke up screaming in pain. My back was in knots. I couldn’t even find a comfortable position to sit down without wincing. I tried to bend over to loosen everything up and almost fell over. I canceled everything I wanted to do for the weekend and relegated myself to the couch to binge on Netflix.
“Old!” I cried. “This is the first sign! I’m going to have to be careful from here on out about any strenuous activity from here on out. What does that mean for my love of weightlifting? Will I be able to run or play basketball again?”
I am prone to melodrama.
On Monday, May 30, I posted my May blog post. I hated it. You can see me talking about stuff I did, and books I read, and things I lifted, but I hated how I felt at the time of writing it. I was struggling to get through it because I felt like I hadn’t really been doing anything productive for weeks. I’d just been playing video games and watching Netflix and making myself work out at home so I wouldn’t feel awful about myself for not going to the gym.
But it didn’t stop there. I proceeded to watch the entirety of Doctor Who (the new seasons) throughout the rest of the summer and fall. Over 120 hours of butt, glued to couch, watching TV. I didn’t read books, not at the volume I had been, anyway. I wasn’t slinging code. I was doing that which was required of me, day in and day out, and nothing else. Nothing at the next level.
On Monday, October 31, I declared “time to start getting myself hardcore in shape again!” I did two-a-day workouts every day that week. I ran. I lifted things. I played basketball for two hours straight against kids who are out there all the time. I haven’t gone running heavily in at least a year. I haven’t played basketball in months. And yet, I was going to do it! All in two-a-day format!
And on Saturday, November 5, with my back in knots, I cried “woe is me, I have grown old.” Took me about five days to put two and two together and say “bro, you’re not old, you’re just out of shape.”
AS IT TURNED OUT TO MY UNENDING SURPRISE AND CONFUSION, one can not stop doing heavy deadlifts and squats, stop running, and stop doing any flexibility training whatsoever and suddenly expect to start doing hardcore two-a-day workouts without injury. Human beings don’t work like that, and my back muscles are perfectly happy to remind me of that.
As I’m prone to do, I started pondering what this meant for other areas of my life. I was so blinded by my insistence that I was a certain person - a person committed to fitness - that I was really scared to admit that I might have lost it. And I have this mental picture of myself as someone with a ton of grit that it really shook me up to write about certain months this year where I didn’t really do a whole lot of productive stuff and played video games instead. In both cases, I couldn’t give myself an honest assessment of my current status - which was exactly what I needed to do to make things better. I needed to get off the mental high of “self improvement” and just start doing things that made me a better person and made me happy.
So this month is about recovery. I’ve been losing ground.
How am I recovering?
First things first, and the number one priority, is getting my physical fitness back in order. This starts with yoga. I’m choosing yoga because my overall strength has gotten out of whack: certain muscles are still reasonably strong, while certain muscles have weakened. All of them are in flexible. That combination means that I can do serious damage to myself when the strong muscles start to compensate for the weaker ones with unexpected movements, which triggers the inflexibility, which puts me on my rear again. I’m also starting to run again, but short distances: one block at a time until I get back to a distance I’m happy with.
I could go to a yoga studio, but I really hate scheduled workouts. Instead, I’ve been using the Gaiam Yoga Studio App to put myself back in order. It’s got a load of things that make it awesome. It’s cheap ($4), it’s got a one-time cost, it lets me download videos ahead of time instead of throwing my mojo off by bad streaming, and it’s got plenty of different workout times while I’m ramping back up. It even lets me follow the fifteen minutes principle. That has translated into doing yoga in some form or fashion every day since November 14, which is fantastic.
Long term plans for physical fitness are still TBD - I’m rebuilding my flexibility first - but I am envisioning something like the workout plans I used to follow back when I was doing P90X. That means two days a week dedicated to yoga, two days a week dedicated to strength, and two days toward cardio. I’d like to get back to being able to run six miles again, and I’d kinda like to be able to do a handstand, too.
For the productivity, that’s about taking out wasted time. I’m defining “wasted” time not simply as unproductive moments, but as moments that are distinctly spent doing something I don’t really care about. If I’m out with my friends, that’s not a wasted moment, even if I don’t get some productive thing done that I’d planned on. If I’m mindlessly scrolling through Reddit, that’s a wasted moment. Yet I’ve never had the ability to effectively block out time-wasters until this month, when my buddy John told me about an app called Freedom. Freedom blocks you from opening certain web sites, but it also allows you to choose certain apps that you want blocked as well. I have a blocklist that includes Reddit, Hacker News, and Instagram, and I flip it on for 30 minutes whenever I need to get something started. (like this blog post!) I use BlockSite for the same purpose on Chrome.
In addition, and at the encouragement of my girlfriend, I’ve taken some of this time to proactively keep my place clean and organized. I’m making sure that laundry is either in the hamper or folded and put away by the time I leave the house in the morning. I take a few more of those fifteen minute intervals to make sure things are in their right place. On Sundays I do a bit of a deeper clean. Does any of this make me inherently more productive? To be honest, I don’t think so. The benefits are mental. There’s no time spent saying “oh, you know, I really should fold that laundry…” and no time spent rushing around stressed because people are coming over and my place is a wreck. It’s always already taken care of. The worst case scenario is now good enough that I don’t sweat it if it happens.
As a closing statement, I know I’m not the only one who feels like they’ve lost ground in something. Maybe it’s your fitness like me, or maybe it’s your work ethic, or maybe it’s something even deeper and more personal. Just remember that point you feel like you’ve lost - you got there somehow, and you backslid somehow, and acting like you’re somewhere that you’re not isn’t going to help you get to where you want to go.
This post is titled “Hypertrophy” because I am choosing to focus on the physiological principle that human beings get stronger when their muscles are forced to hit the limit. There is no growth without somehow getting knocked down.
- Read three books
- Code Complete by Steve McConnell (by the way, this thing is a monster. It’s good knowledge to have and I’m glad I read it as an engineer, but it is the size of the biggest Harry Potter book ever and nowhere near as easy to read)
- Six Easy Pieces by Richard Feynman
- Cracking the Coding Interview by Gayle Laakmann McDowell (not applying for jobs, this is just one of the best refreshers on basic algorithms out there)
- Enjoyed Thanksgiving with 18 of my family members right here in DC
- Went camping in sub-freezing temperatures at the treehouse camp in western Maryland
- Wrote my first full app in Scala and learned a thing or two about functional programming
- Did two-a-day workouts for a week and landed on the couch with a back injury as a result
- Finished Doctor Who
- Watched all of Stranger Things
- Watched all of Luke Cage
- Watched Pacific Rim
- Did at least fifteen minutes of yoga each day for 15+ consecutive days and counting
- Began blocking website and app distractions using Freedom and BlockSite to improve productivity
- Began maintaining a clean house through regular, dedicated small doses of effort
October's Very Own
I had a post for this month that I was really struggling to write. I was pushing through it and pushing through it and thought I almost might have had a good idea… and then my computer restarted for a security update and all of the changes were lost. So this blog post is coming from scratch.
Heck, this whole writing experiment is a reboot from scratch every month. I never know how I’m going to reinvent myself. I had this grand idea at the beginning of the year that I was going to focus on my health in the first part of the year, then focus on my career, then focus on personal education, then focus on saving money. I think that was the order of the goals. I got lost after the first two months and then got a new job and then I forgot about all of those plans and started doing new stuff.
I see tons of people with all of this focus and I wonder how they maintain it. I wonder how much more I could have done if I hadn’t decided to watch every episode of the Doctor Who reboot this year. Do you know how much TV that is? And I binged through all of Jessica Jones back in February, and The Man in the High Castle, and probably some other show that I forgot about. It’s even worse when I think about the amount of hours I logged going for completeness in Skyrim. I could have built an app, or learned to speak a new language, or something.
When I started this year I was at this weird place where I was just accepting what it meant to truly take some responsibility for the outcomes of my life, and that’s where the grand idea with all those different focuses came in. Previously, I’d taken “responsibility” to mean “taking action,” and that’s certainly a component of it. But “taking action” was all I was doing. I was hacking away at stuff as fast as I possibly could, but I wasn’t really taking a step back and thinking about if I was doing the right thing. Checking stuff off of a to-do list made me feel good, and as long as I was feeling good I never stopped to think if the items on the to-do list were the right ones, or even if I cared about doing them at all. And I certainly wasn’t thinking about external consequences. I’d hack on something, and see if it worked, and if it didn’t work, I’d try something new or quit. I quit a lot.
Now I say yes to way less stuff, and I keep my eyes open more. I try to listen more. One of the conditions of my hacking is that I would tend to do it at the expense of my relationships - because if you’re trying to execute on a bunch of random experiments with no clear direction you hardly have time to be paying attention to the people in your life, most of whom are way more complex than the to-dos I was concocting in my head. I’m trying to learn more from the people around me rather than books or guides on the Internet. I’m trying to be a better boyfriend, brother, friend, son, cousin.
I look toward the close of the year with excitement because it’s definitely not where I started the year and definitely not where I expected to be at the end of October. I started the year itching to travel and escape DC. I’m now looking forward to being at home for a bit after spending time in six cities in one month: DC, Houston, New York, London, Liverpool, and Edinburgh. I started the year feeling stuck in my job, and now I’m ending the year feeling like I’ve experienced insane career and knowledge growth in the past six months. I feel like I have better relationships with more people than I did this time last year.
I look back and I know I accomplished all of this because I was really fortunate to have people around me who put up with my insanity and pardoned me when I forgot to answer their text messages or said something really rude unintentionally. Most of you probably don’t even know that you helped, because you were just off being yourselves and I was scrambling trying to figure out how to be more like you. So if you’re stuck in any way - whether you’re feeling like you don’t know where your life’s going or what to do with your job or you’re trying to crack the next level in your career or you’re just looking for a good book to read - I’d love to try to help. I’ve learned a lot from all of you and maybe I have something to share.
This blog post was a bit rambling, and I’m sorry about that. I didn’t have a clear thought, just a whole lot of gratitude.
But I also have lists!
- Traveled to:
- Watched a Liverpool FC game in Anfield
- Read one book
- Did a bootcamp workout in Hyde Park in London
- Hosted my friend Sean in DC
- Dressed as the Doctor from Doctor Who for Halloween
- Began studying Network+ videos from Cybrary
September, or fifteen minutes
I am obsessed with time.
It’s a fascinating notion, time. I used to start stressing out if my web applications took longer than 300 milliseconds to respond to an incoming request, which is about as much time as it takes for you to blink your eye. I know that taking one of the 50-series buses back to my house will take about ten minutes longer than one of the 60’s (but there are less frequent 60’s), so I will recalculate public transportation options based on when each bus is coming at the time I arrive at the bus stop. I am constantly re-evaluating and optimizing how I spend my day, because certain things (commuting) are a waste of time, others are a necessary evil (cooking, laundry), others are valuable and enjoyable (writing), and some are just plain fun (watching Doctor Who, a show about a time traveler that I’ve been binging on this year).
I find large chunks of future daunting to evaluate. I tend to optimize for minutes, not days or weeks, and so I struggle to truly focus on the long term and am constantly going for short-term victories. Yet I want to accomplish so much - I want to build things, I want to create great ideas, I want to develop wonderful relationships. I want to do a lot of things that aren’t accomplished in minutes, they’re accomplished over weeks and months and lifetimes. For years I’ve struggled to reconcile my desire for big things with my tendency toward little thinking.
An idea to resolve this conflict began to take root in my mind when I stumbled upon a Reddit post about 15 push-ups. It’s a simple idea:
“LPT: There is a visible difference between not working out at all and doing 15 pushups every day. Make 15 push ups your new ‘not working out’.” - source
I originally started chewing on this idea from a purely fitness-related perspective, and I liked it. If you’re doing 15 push-ups a day, you’re not going to turn into Arnold Schwarzenegger, but you’re moving the needle forward. You’re doing something positive in a short window of time that will train your body into something stronger than it would have been otherwise. That step forward is important mentally and physically.
Now, fast forward a few weeks, and I’m reading the book Founders at Work while on a trip to Charleston, SC. In Founders at Work, Jessica Livingston conducts a series of interviews with business founders to discuss the early days of their businesses. The results are fascinating. There are some archetypal stories in there (Apple, PayPal) as well as some slightly off the wall ones (Hot or Not). But one of the ones that really stuck with me was the tale of the early days of del.icio.us, the social bookmarking site that eventually sold to Yahoo for an undisclosed amount.
When Joshua Schachter founded del.icio.us, he wasn’t trying to become a startup founder. He was working as a quant for Morgan Stanley full-time: a job notorious for long, stressful hours. But here he is with del.icio.us, a hack project that was realized to the world and was getting some 30,000 users. Here was how he described those early days:
**Livingston:** When you were doing this in your spare time, did you ever say, "Ugh. This is too much work"? **Schacter:** Not really. I was always very careful...to structure the code...such that I could come in and look at it, figure out what I'm doing, do it, and be done for the day in 15 minutes. So if I could get one thing done a day, I was happy. A lot of stuff, if I could spend more time, I did, but as long as I could get one or two things done a week total, if I didn't have time, I didn't have time. So it moved pretty slowly. I worked on it for years.
That passage is fascinating to me. Fifteen minutes a day! While working as a quant for Morgan Stanley!
So this month, I started tinkering around with fifteen minutes. When I’m feeling daunted by the scale of a task, I set my timer for fifteen minutes and crank. I’m moving the needle forward. I’m getting my fifteen push-ups in.
There’s a surprising amount of stuff you can do in fifteen minutes, like:
- Work out
- Fold your laundry
- Straighten your room up
- Write a single code function
- Pay your bills
- Purchase and initialize a server
- Learn how to send a custom HTTP method in Python
- Read a few pages of a book
- etc., etc.
You keep stacking those fifteen minutes up and up and suddenly you’re staring at something you’re proud of. I built an app called “FutureSelf,” which plays on my obsession with time and allows me to send picture messages to myself in the future. I wrote a Python library that lets me analyze log files in Spark. I set up my own personal, hosted version of Dropbox.
These are things I wanted to accomplish, and the principles are the same for anyone out there reading this. Set your time for fifteen minutes. Crank. Move the needle forward just a little bit whenever you can. You’ll be shocked at what you can get done.
- Founders at Work by Jessica Livingston
- The Song Machine: Inside the Hit Factory by John Seabrook
- So Good They Can’t Ignore You by Cal Newport
- FutureSelf [non-public]
TV shows watched!
- Twilio - Security
- Certbot Documentation
- The Road Coder’s Survival Kit
- Ask HN: What’s your favorite HN post?
- Advancing in the Bash shell
- Machine Learning in a Year
- Seven Languages in Seven Weeks
Spearman's rank-order correlation
This is a mini-post: something I learned about today at work that I thought was awesome in and of itself. It’s called “Spearman’s rank-order correlation,” and it tells you if two data sets move in the same direction.
Two sets of data can have what’s called a “monotonic” relationship, which basically means that if one value goes up, the other value goes up; and if one value goes down, the other goes right down with it. Monotonicity doesn’t necessarily care about the direction of that relationship. It just cares that the two go together.
Let’s look at something topical. Specifically, let’s see how Kirk Cousins’s and Eli Manning’s passing yards compare over a given sixteen-game season. If Cousins and Manning had a strong Spearman rank-order correlation (shortened as Spearman’s rho, because mathematicians like Greek), we would Manning’s passing yards go up if Cousins has a great game, and if Manning only throws for a hundred yards, Cousins will have a bad game too. I’m going to ignore bye weeks in this example so each will have exactly sixteen games.
from scipy.stats import spearmanr kirk = [196, 203, 316, 290, 219, 196, 317, 217, 324, 207, 302, 219, 300, 319, 365, 176] eli = [189, 292, 279, 212, 441, 189, 170, 350, 213, 361, 321, 297, 337, 245, 234, 302] spearmanr(kirk, eli) # SpearmanrResult(correlation=-0.19602068697550504, pvalue=0.46686779279825008)
Okay, so not really correlated… in fact, they’re actually slightly negatively correlated, which would imply that Cousins has bad games when Manning has good ones and vice versa. But it’s a loose relationship, so we won’t compare that too much.
This isn’t much of a fair comparison, though. If the Giants were playing the Patriots on the same week that Washington was playing Philadelphia, we’d expect Manning to have a bad game while Cousins has a great one. So let’s compare how they fared when facing similar teams and see if the correlation improves.
So now if we run our calculation:
kirk_vs_teams = [290, 219, 196, 317, 217, 324, 207, 219, 319] eli_vs_teams = [302, 292, 297, 213, 361, 350, 245, 170, 212] SpearmanrResult(correlation=-0.10041928905068677, pvalue=0.7971388092372127)
Okay.. almost no correlation at all! Good to know for fantasy purposes, that if you want to hedge your QB bets, put Cousins and Manning on the field together!
That’s all I’ve got, that’s Spearman’s rho. Go have fun with it!Tweet