a user sums it up

    Create an open source project, they said.

    It’ll be great for your resume, they said.

    Part 0: Wherein we provide context

    The year was 2013, Meteor was the hip new kid on the block and Coffeescript was a reasonable JS dialect choice. We were fresh out of college. Meteor was hosting their first ‘Summer Hackathon’ in San Francisco on 10th and Minna, and we figured this was our shot.

    Sequestering ourselves on the second floor of the hackspace, a couple of college friends and I acquired a whiteboard and some markers, and jotted down our big idea list. Not lacking for ambition, the first thing we figured we would fix was server-side rendering. Turned out this seemingly simple task was already on Meteor’s Trello Roadmap, so, what the heck, we figured we’d pitch in.

    I inquire about server-side rendering

    Speaking with the actual development team a few hours later, we learned that server side was (1) hard and (2) coming soon anyway1. Fair enough - we’d solve our other painpoint, the lack of a proper admin DSL.

    Part 1: introducing Z, the Mongo Admin

    Coming from Django land, one of my favorite Framework features had always been being able to describe Admin UIs with a very high level DSL - stuff like

    from django.db import models
    
    class Author(models.Model):
        name = models.CharField(max_length=100)
        title = models.CharField(max_length=3)
        birth_date = models.DateField(blank=True, null=True)

    leading to auto-generated pages that look like:

    django admin screenshot, from tutorial

    Meteor didn’t have an admin DSL2 yet. I wanted very much to build one, and a zero-config one (IE, automatically discover your models) at that. I kept suggesting this idea until the others agreed. Leadership.

    We cranked out a desired feature-list on a whiteboard. It looked like this.

    The List

    • document view
    • dealing with arrays
    • dealing with nested objects
    • boolean fields
    • integer fields
    • editing
    • auto-discovery & Schema
    • collection (table) view
    • home page view
    • doesn’t look terrible
    • wrapping the admin into a package.

    Then we got started. I hid in a corner and tried to figure out how to get a list of Collections, and the fields in each collection, out of an arbitrary Meteor app. It turned out there was no official way but if I poked under the covers of the undocumented Meteor._RemoteCollectionDriver.mongo.db.collections call.

    It turned out you had to ‘warm up’ the _RemoteCollectionDriver (by creating an arbitrary collection) to get the collections loaded. This was the way things worked around Meteor 0.6.

    Package loading was the other relatively painful thing - since we couldn’t figure out quite how to get our package loaded last (since we relied on a router and a number of other packages), Yefim solved the problem the pragmatic Hackathon way, and named the package Z-Meteor Admin. This way, unless we relied on a package that decided to start with two Z’s, Meteor would load us last.

    Launching with the MVP

    Surprisingly, it kind of worked.

    an early version of the UI

    We demoed at the Hackathon and even won the ‘Award for Most Technically Impressive’.

    Greg and I kept working on the project throughout the summer and fall of 2013. We cleaned it up and released 1.0 in December 2013, after gaining over 200 stars on Github.

    For our 1.0 release, I demoed a slightly-less-hacky-and-now-renamed Houston Admin at Meteor Devshop 10:

    Here’s the blog post I wrote to celebrate the event.

    Part 2: A primer on gas tank emptying

    Between the 1.0 in December 2013 and roughly summer 2014, our enthusiasm for working on the project waned. There were many reasons, but here are some:

    1. We had shit to do.

    Greg finished school and was working for Gumroad, not in Meteor. I was doing consulting/startup prototyping work, also mostly not in Meteor. Yefim & Ceasar made the wise decision of not doing much contributing work past the initial release.

    2. We were no longer scratching our own itch.

    Originally, Yefim and I used Houston for the Intern Dinners we were running that summer. It was pretty helpful, and if we needed more UI stuff, we’d just add it. That was summer 2013.

    Even though many of the feature requests we were getting were quite sensible (and we implemented a bunch of them), we really weren’t scratching our own itch after summer 2013.

    3. ‘How do I even’-esque Github Issues

    Actually, having gone through some of the ~270 Github Issues on the project, I’m surprised by how good many of them were and how attentive and friendly we were about them. Still, there were a few bad apples, like css not rendering, how to upload files and Cannot login on Heroku app that either failed to provide enough context and/or were coming from folks who were not otherwise yet competent Meteor (or even JS) developers.

    4. Meteor doesn’t support Mounting & hacky CSS.

    Meteor bundles all CSS & Javascript together when it compiles. This is not the ideal behavior for UI libraries, since if the parent app has some logic that say, for example $('.save').disable() and later the user goes to Houston, all of a sudden all of our ‘save’ buttons are disabled and its unclear why. Likewise, any global CSS rules the parent app chooses to use, like (say) making table columns 200 pixels high, will also make our table columns 200 pixels high. Greg did some crazy things to avoid the namespace collissions.

    Django solves this problem through URL namespaces, allowing the ‘admin’ to behave largely like its own app. Express.js allows for ‘mounting’ of multiple sub-apps on certain directory paths - in either case, no shared CSS/JS is bundled for these cases, avoiding collissions like the above.

    Meteor did not support Mounting out of the box. I bugged the devs about it at a later Devshop, and got an off-hand note that this was not a priority for now. Later on we tried to add Mounting to Meteor and/or host the Admin as a separate app that shared only the database with the parent app, but by then lacked the enthusiasm to bring the projects to completion.

    5. Testing is tough!

    Reactive apps with not a ton of business logic are tough to test! Perhaps we simply lacked the experience here, but when Greg and I tried to add proper integration tests to the app, we spent tens of hours being our heads against the wall, time that would have been better spent actually fixing bugs.

    6. Router Wars

    When we created Houston in summer 2013, there was a router that I believe was just called Router. Later, Iron-Router becames the default router (perhaps this was a rename?) - and even later, Flow-Router became the preferred router. Here’s a post on the state of routing as of Summer 2015.

    The point was, you couldn’t really use both routers and so we would need lots of clever work to see if we could support both paths. We discussed this in a Github Issue and I wrote up a prototype but ultimately just didn’t have enough time/energy to ship a supports-both solution.

    7. When Undocumented Internal Dependencies Change

    Remember that _RemoteCollectionDriver hack? In most every release (0.7, 0.8) my ‘whatever, go ask mongo what collections there are’ hack kept breaking and I had to re-implement a new hack based on whichever refactor the Meteor Development Group implemented internally.

    Not particularly taxing, but just a drain.

    8. We never stepped back and thought about architecture.

    Things were in slighly better shape than they were back during the hackathon, but there was never a “how should this package be properly architected” conversation. As a result, in later months as I tried to go back and implement changes, it was never clear to me what was where or why, or what edge cases I would need to consider and support. This made “weekend-a-month” type support less useful over time as I knew the codebase less and less well.

    Part 3: Where we go to Costa Rica and try to turn things around

    Costa Rica: the plan

    Around Summer 2014, a year after we started, things were going kind of slumpey. As an effort to regroup, I invited Greg to Costa Rica where I was staying for a few months: Hey Greg, come to Costa Rica!

    The plan was basically to unbreak the critical stuff and make a plan for what we wanted to do next, and maybe also play some soccer on the beach.

    Costa Rica: what actually happened

    I got bogged down on a consulting project that was a bit behind schedule as Greg visited. So we got maybe 1-2 days of work done, which instead of dealing with any of the issues on my part was shipping a new feature I decided I wanted, called Custom Actions, which were kind of cool but not at all the problem.

    Custom Actions

    We still played some soccer, though. Greg was way better at soccer than the rest of us.

    Get other maintainers, they said

    We had two pretty helpful maintainers turn up: First, Roger decided to redesign our UI from basic boostrap to the line-green version you see today. Second, sometime around late 2014, this Swiss dude named Matteo showed up and fixed a bunch of things. That was awesome, and we talked to him on Skype and gave him write access as somebody had suggested for growing your maintainers.

    Later, in 2015, my friend Sam came in and added some proper tests, but development had largely ceased by then.

    Still, for whatever reason, nobody stuck around and answered issues, etc, for the long term, so it was still up to us.

    Part 4: Decline and Fall of the Houston Admin

    Meteor 1.0 is released!

    Coinciding with Greg’s visit, Meteor finally released 1.0 in October 2014. Perhaps we’ll no longer have to adjust the incompatible hacks we use to figure out which collections are to be added and we can focus on the good stuff!

    Blaze vs React

    If only. A year later, the big question up in the air becomes whether to use Meteor’s original Blaze templating or switch to a react-based front-end. React is great and all, but we for one are not up for yet anothe rewrite of this thing we haven’t really gotten to use for our own stuff in like, two years.

    1.3

    To add insult to injury, The Meteor Development Group ignores the principles of Semantic Versioning to release 1.3 in April 2016 with Breaking Changes - and our package is broken yet again. Maybe.

    Part 5, an Epilogue

    Perhaps the most poignant symptom of the sort of stagnancy that the project fell into is that I had an idea to write this blog-post in early 2015, and it took two years to even get the post-mortem for this project done.

    I think Greg may have fixed the 1.3 issue, but the last change I can see in the codebase is in March 2016 and frankly I just don’t care anymore. On the bright side, it looks like there’s a half-decent competitor that people who want an admin can use3. Also, the Meteor Development Group looks like it’s less interested in Meteor and focusing on GraphQL tools these days, so maybe nobody needs this. On the bright side, 880 stars on Github, so that’s cool!

    What have we learned?

    • Open-source maintenance is hard, especially if your only remaining motivation is altruism. Richard Schneeman gave a great talk about Saving Sprockets last year which nailed it.
    • Perhaps we should have quit while we were ahead and put an “not actively maintained, looking for maintainers” on the README back in 2014. That would have been a bit more responsible.
    • On the bright side, building a key piece of infrastructure for a hip new development framework turned out to be a great way to get a lot of users for your open source project.

    …one more thing

    I leave you with the contributions over time graph, which tells the whole story, but with graphics. contributions over time

    Thank you

    • To Greg, the prospect of collaborating with whom kept me in the project as long as it did.
    • The Yefim, Ceasar, Matteo, Roger, Sam and other contributors!
    • The the Meteor Development Group folks for their contributions to realtime web development, whose ideas will live on independent of Meteor’s future, and also for the free snacks and t-shirts and the Pebbles we won at that hackathon back at 2013.

    1. as far as I know, Meteor still doesn’t have proper Server-Side rendering. Arunoda wrote a community library in 2015, but given the Blaze/React split, I’m uncertain how much of this has come to pass.

    2. Ironically, the closest thing to an Meteor Admin at the time was Genghis, a standalone one-page PHP-based Mongo editor by Justin Hileman, whose younger brother I had somehow hired the year prior. Silicon Valley is a small place.

    3. though it too looks like it has not been updated in 10 months :-/.

    Published: May 06 2017

    For most1 engineering tasks, I prefer to avoid TDD, or Test Driven Development.

    Yet when I job-hunt, I use TDD religiously2. It has been tremendously helpful.

    Hmpf. Why?

    Most interview challenges actually come in two parts.

    1. Can you take a broadly-worded problem and nail it down into something unambiguous?
    2. Can you implement a challenging but unambiguous spec in the allotted time?

    TDD forces you into the ideal mindset for nailing down (1) problem definitions. There’s no better way to properly grok a problem than to have to think through all the fun ways an implementation could be slightly off.

    The implementation itself (2) also gets easier since you no longer have to assume that your code “probably” works, or fiddle with a REPL each time you’re ready to check.

    So how do I do it?

    As soon as you hear the problem, resist the urge to rush straight to the solution.

    Instead, force yourself to fully think through the test cases for an arbitrary implementation. Make it a dialog - “what if the same element is in the array multiple times?” Get the interviewer engaged. Show you care.

    By the time you’ve written good tests and the interviewer agrees, you’re all set. Even though you tried not to, you probably already have a decent solution in mind. Your tests will let you know you’re done. They’ll also give you the comfort that your subsequent clean up / refactoring doesn’t inadvertantly break your implemenation.

    Yeah, but I can always just write tests later.

    When I conduct interviews, I keep running into candidates clever enough to power through engineering challenges on raw talent3. The candidate implements an optimal solution within fifteen minutes. Then it’s time for tests and she just cannot think of cases beyond the common path, which makes her look like a cowboy.

    I blame the curse of knowledge - once you’ve solved the problem, it’s harder to think about what wrong turns you could have taken. It all feels trivial.

    Seriously though, are you going to make me import a testing framework? Like, Mocha or Cucumber? Ugh.

    ¯\_(ツ)_/¯, I just use asserts and print statements. If I’m feeling fancy, I’ll create an array of test case scenarios and run them like

    test_cases = [
      [[2, 2], 4],
      [[1, 2, 3, 4], 10],
      [[], 0],
      [[-3, 2], 1],
      [[100, 2], 102],
      [["banana"], None]
    ]
    
    for test_case in test_cases:
      args, expected_answer = test_case
      answer = quantum_addition(args)
    
      print("called add({}) and got {}, expected {}"
        .format(args, answer, expected_answer))
    
      if answer != expected_answer:
        print("UH OH!")
    
      # (or you could just do: assert answer == expected_answer)
    

    If you insist on being fancy, you can do that thing where you ask your interviewer if they’d like you to use a proper testing framework first, and of course they don’t, but now it looks like you could and you’re just doing them a favor by using asserts, cause you’re like, super assertive.

    Ok so now what?

    Well obviously you should come work at Opendoor with me! We have the good coconut water and don’t run out of it until like noon, 12:30 on good days. Plus, there’s a decent chance you’ll run into me on your phone screen and we can have that awkward interaction of trying to figure out why my name sounds familiar.


    Thanks to Vaibhav, Joe, Nick, Gayle, Tess, Charlie, Kevin, Zain, and Yahel for giving feedback on earlier drafts that I incorporated.

    1. Don’t get me wrong, TDD has its uses, but we often mistake what Kent Beck meant by ‘unit’. Testing ‘just right’ is hard.

    2. To be clear, TDD won’t help you with interview where you don’t actually code (IE, whiteboard interviews) You’ll still have to start by asking questions, though.

    3. I used to have that problem earlier in my career. I’ve addressed it, largely by becoming dumber.

    2015 and 2016 could not have been more different.

    2015 was mostly spent living out of a carry-on bag, traveling from Estonia and Berlin to Tokyo and Taipei helping run trips for, and building, Hacker Paradise.

    It was great. Wanderlust: satisfied.

    2016 found me receiving an H1-B visa, moving back to the Bay Area, and obtaining honest-to-goodness employment. Different.

    Here’s how becoming sedentary and employed has turned out so far, relative to expectations.

    The Good

    Basically, stability.

    Engaged

    My partner Mari1 and I are (at long last) engaged! After 5 years of being together, we’re back to living together and have a place in Bernal Heights that feels like our own. We’re so domestic we even got an L-shaped couch and the kind of mattress they advertise on podcasts.

    There’s something very warm and comforting in having a good sense of who you are going to spend the rest of your life with. Mari has been an anchor, making the disruption in other parts of my life that much more manageable.

    Friendship & Community

    It’s also been rewarding to be able to engage and re-engage in meaningful friendships. One of the toughest parts of traveling full-time turned out to be meeting new people every month (great people, mind you) and forging friendships with them. At some point, the task became exhausting. I yearned for the comfort of spending time with the sort of friend that already knew my jokes were overly-complicated and terrible and would laugh anyway, if only on the inside.

    I’m still not 100% back to feeling integrated or satisfied with the depth of my relationships back in San Francisco, but compared to full-time travel, the improvement has been remarkable and the last few months of 2016 felt much better than the months prior.

    J.O.B.

    I joined Opendoor in September. Part of the motivation of being an employee was to have co-workers I would learn from & enjoy working with. This came true. Another part, which I was more surprised to learn, was that “Growth Engineering” - a term I would have thought ridiculous back in college, and as a sell-out-sort-of-a-thing-to-do, is actually quite engaging and satisfying. It feels like I’m doing behavioral economics research again, but with higher stakes. I’m very grateful to Remi Gabillet, for first identifying this role for me.

    Opendoor has easily met expectations in terms of the company’s growth, the quality of my coworkers, my faith in the executive team, and my ability to have an impact as an individual contributor joining a 150+ person company.

    The Bad

    There’s no better crutch for procrastination than employment.

    Balance

    A problem that I’ve seen a fair bit with engineers and especially founders in tech is this mission-at-all-costs mentality. Like political operatives on a critical campaign or soldiers in the middle of a war, it’s easy to say “this thing I’m part of is more important. I can postpone [personal thing that I should get done] for now, because if we win we can always deal with it later.”

    And then the mission comes to an end and we’re stuck figuring out that we’ve got to find a way to do our own laundry and maybe go to the gym more often and our last annual check-up was in the last decade. Maybe you deal with it, but often you start hoping and searching for another mission to come along because it’s easier and you don’t have to do boring, mundane things like take care of yourself.

    One of the reasons for deciding to get a job rather than starting another company was that I could use the superior work-life balance to get the rest of my shit in order, and avoid getting caught in the mission-at-all-costs mentality.

    That hasn’t really happened.

    It did, a little bit, at Sprig. But at Opendoor, working harder and staying longer has been too gratifying, too tempting to be easily avoided, and I found myself slowly falling back into using mission-at-all-costs excuses2.

    Consequences

    As a result, I’ve definitely gained some weight (something that did not happen while traveling and eating at restaurants full-time for a year and a half). I’ve ignored a doctor’s note to start watching my cholesterol. I missed a dentist appointment. I bike to work but have failed to follow a regular gym schedule.

    I also have yet to deal with setting up some nuances relating to my Health Savings Account. I have an accident claim from a rental car that got broken into that has been pending since September.

    I planned to set concrete mental health goals around building resilience and good habits, and meet with a therapist regularly3. Still TBD.

    Instead, I’ve reverted to occasionally staying up way later than I should and then being less productive at work the next day - essentially a high-school version of myself but with more severe impact to my productivity, given I’m not sixteen anymore.

    My lifestyle has me eating less healthily and drinking more than when I was traveling, on “vacation”.

    I find myself easily falling into many of the typical privileged-engineer-at-a-legit-startup-in-the-bay-area traps. It’s funny but also a little disappointing, since I was well aware of (and caution others) against this kind of self-neglect. I’d rather be making new mistakes.

    All in all

    Some good, some bad. As far as new year’s resolutions go, I’m going to need to make sure to be hitting the original goals I had in mind when I moved back to San Francisco.

    2017 Goals

    Sleep Maintain a healthy sleep schedule (7+ hours of sleep and in bed by 1AM, 95% of the days of the year).

    Exercise Do something (bouldering, gym, or other form of exerfise) 3+ days a week, 95% of weeks except vacation. Reach and maintain target weight of around 10 pounds less than Jan 1st, 2016.

    Mental Health Build mental resilience (either follow a book or hire a therapist by Jan 30; create a growth plan; follow said plan).

    Physical Health - Schedule & attend follow-up physical by Jan 30; follow recommendations. - Schedule & attend dentist appointment by Feb 30.

    Writing (AKA, other interests) - Publish a 500+ word piece once a month, including receiving quality feedback from friends.

    Fingers crossed.


    1. I have to say I’m pretty stoked to stop explaining why I use “partner” and just saying “fiancee”.

    2. More caveats: Opendoor is a great place to work and if I haven’t tried convincing you to join yet, ping me.

    3. I assume that by 2017 this is not a controversial position. To be explicit: I’m not in any particular trouble, other than the sort of typical garden-variety depression and anxiety you’ll often see in ambitious over-achievers. You don’t wait until you’re overweight to start exercising; you shouldn’t wait to be struggling emotionally to work with a professional to build resilience and maintain mental health.

    Published: March 10 2016

    Giving back to open source projects you benefit from is one of those obviously-good-in-theory-confusing-in-practice ideas, like eating healthy or being carbon-neutral.

    Many services - flattr, bountysource and gratify, among others, offer widgets that developers can put on their projects to encourage subscription donations. None has been a breakout success.

    There needs to be a fast and straightforward tool to empower an engineering manager to say “sure, let’s do it.”

    Dependonate

    Here’s how it would work.

    Dependency File

    Most projects I work on these days have a file with a list of package dependencies - Gemfile for ruby, requirements.txt for python, package.json for node, and plenty more.

    List of Dependencies

    Piggybacking on package managers files, Dependonate would take a project’s dependency file and generate a list of direct and indirect projects. Each of those projects can then be parsed to see whether their Readme file includes links to any of the popular donate options (flattr, bountysource, gratify, paypal, etc).

    Actionable Next Steps

    Finally, Dependonate would open up the various URLs that would make donating as registering and entering your credit card on the services above, the lists of desired projects to contribute to having been auto-filled.

    Dependonate would also generate a pretty report for the company to put up on their website as proponents of Open Source, somewhere near the engineering hiring page.

    Hit me up if you build something like this, or if your company would be up for donating through it.

    I found myself planning a series of flights for Hacker Paradise, flying from Tel Aviv –> New York –> San Francisco –> Tokyo –> Tel Aviv with a few days of flexibility for when to take each flight. I spent about half an hour on Skyscanner and Google Flights, and couldn’t find flights for less than about $3,000. Which is a lot. At roughly the same time, Vlad from Flystein reached out and offered to help Hacker Paradise participants book flights. So I tried the service out.

    Flystein Homepage

    What’s Flystein?

    Glad you asked. Flystein, it turns out, is what a value travel agent looks like in the “digital age”. Instead of making money by taking a percentage of your booking fare from airlines, Flystein helps you find the cheapest possible fare online that meets your needs and then charges a flat fee for doing so. You then book the flights yourselves, via links & booking instructions Flystein provides.

    Taking money for providing value. Who knew.

    How did it go?

    I gave Flystein my itinerary and neurotic set of flight cost preferences.

    My flight itinerary

    A day later, they came back with flights totalling $1934. A ~$1,150 difference between a “reasonably competent” online flight shopper like me and a bona fide flight expert.

    Proposed Routes

    How did things get so much cheaper?

    Throwaway Ticketing. My Tel Aviv to New York flight includes a roundtrip flight back to Copenhagen in February. I’m probably not going to take it, but who knows. Also, who cares – it was $200 or so cheaper than the equivalent one-way flight. I would have never known to check.

    Hidden City Ticketing. My San Francisco to Tokyo flight included a day-long layover in Tokyo and then continued on to Hong Kong. I, however, never made it on the connecting flight. One day, Hong Kong, one day.

    The Economist has a good write-up of the hows and whys and economics of flight hacks. The point is, you can save a considerable amount of money on flights if you know what to look for and are willing to dedicate the time required. Or you can use Flystein.

    How was booking?

    Mostly fine. One of my flight legs was expired by the time I looked (you have to book quickly) and I ended up calling British Airways, only to be told that I needed to pay thousands of dollars more.

    So I let Flystein know. After cursing about British Airways for a bit, Vlad found a flight that worked perfectly only a few minutes later. The new flight cost a whole $20 more.

    The whole booking episode could have been avoided if I had the ability to go full “travel agent” and just given Flystein permission to book on my behalf. It turns out there are legal reasons why doing so is not trivial, but they’re on it.

    Was it worth it?

    Flystein saved me over $1,000. Research fees for a trip like mine cost <$100 (a simpler flight starts at $49). So, yes. 90%+ of the savings for near 0% of the frustration in finding & booking flights. Value….but what if they don’t find anything? Fair question.

    It’s possible you’ve already found the cheapest rate you’re going to get. Flystein has a beat-my-price option where you only get charged if Flystein actually saves you more than their fees.

    Flystein. Use it when booking flights of non-trivial complexity (international flights over $500). It’s a thing.