2013-11-24

Do you even lift?

If you want to do important things, you should find something that is either on, or beyond, our current capabilities and do what was borderline impossible up until that point.  If you are more concerned with making money, you should lower your ambitions and shamelessly pump out highly derivative work in the hope that something is going to stick.

I have been in the software industry for a few decades now and it appears to be a constant that at any given time the vast majority of the industry is stuck in a group-think.  Everyone agrees on what is important.  In the 80s it was 4-gen systems.  In the 90s it was OO-everything.  In the 00s it was pragmatism (as a backlash to the insanely tedious process oriented nonsense of the mid to late 90s).

The current craze appears to be a mish-mash of business development ideas applied to software engineering in order to squeeze tons of cash out of shallow ideas that have a limited life-span.

Sure, it is exciting when people spend 3-6 months building something they can sell for a cubic meter of Benjamins.  I'd love to do that.  Mostly because it would mean I could pay people to do things I find important without having to convince anyone -- which is a tedious, undignified affair.  But I think it is fundamentally unhealthy when this is what we aspire to. Because it has more in common with winning the lottery than accomplishing anything really important. The ugly truth is that most startups do not get acquired for a billion dollars, most shallow products are just worthless nonsense and most of the people who build them aren't really that brilliant.  They are just lucky.

Beware of having lucky people trying to rationalize their luck into repeatable process postfact.

One of the greatest, truly worthless ideas of its time was Pointcast.  In essence a service that pushed advertising to screen-savers.  For a while in 1996, this was the biggest idea around and everyone was terribly excited.  At some point it was even one of the biggest traffic generators on the Internet -- pushing terabytes of advertising to screens that nobody was looking at.  I remember lots of people building copycat products and a guy I was working with felt we should drop everything we were working on and bet the farm on this.

Pointcast was, at one point, valued at a quarter of a billion dollars.  Then 100 million.  Then 10 million. And then nothing.

Pointcast wasn't important.  Anyone with half a brain back then should have been able to see that the product was bullshit and that it brought us absolutely no breakthroughs of any kind and it was trivial to rip off.  But most people didn't.  Most people were unable to tell a win in the lottery of Internet fads from creation of lasting value.

During the same years, Page & Brin wrote bunch of papers(1) on data-mining and IR and eventually built a search engine.

And before people who know sod-all about what Page & Brin did:  there was nothing easy about what they did.  They didn't start off with a minimum viable product and then iterate it into something that was successful.  They started by solving a handful of hard problems.  And then they solved a bunch more.  Including problems that every single one of their competitors found to be outside the scope of what they were doing.

(One example is their internal infrastructure from the early 00s which allowed them to speed up innovation.  Much of which was very un-traditional and required engineers and leaders with a flexible mind to do.  But also in terms of things like AdWords and AdSense -- which was invented independently multiple places at roughly the same time, but which was in some cases shouted down as "irrelevant distractions" by leaders without vision).

At the time there was no shortage of people who were ready to piss on what they were doing.  I can remember much of what was said about Google in the early years by people in the industry.  How pompous executives made fun of them because they didn't conform to the orthodoxy of the time -- which was portals.  One stop shops to capture and hold audiences.   Horribly ugly things filled with desperate salesmanship and terribly packaged content.

Everyone knew this was the way to do Internet services.

You can, as I mentioned earlier, make a lot of Benjamins by winning the Internet lottery and become an overnight hit.  With audiences numbering in the hundreds of millions, and sometimes billions, things that become popular can become very valuable.  But it is a lottery.  And we tend to focus only on the winners -- not the thousands upon thousands of losers.

And we do not focus at all on the substance these "winners" leave behind.

If you are a young developer I would advise you to seek out deep challenges.  Solve hard problems. Use hard science, or at least apply your education to practical problems where you might have an edge.  Sure, you should buy the occasional lottery ticket, and sure you can have some commercial success by timing Internet fads, but if you can, you should try to do some heavy lifting every now and then.

(1) The first time I heard about Sergey Brin was in 1997 when I read his paper on "Dynamic Itemset Counting": http://www2.cs.uregina.ca/~dbd/cs831/notes/itemsets/DIC.html . At the time a friend of mine was building an online book-store and I spent a weekend figuring out how he might implement efficient recommendations for he bookstore.  Of course, he never did, but at least I learned something by writing a people-who-bought-this-book-also-bought-these-books recommender based on what I learned from reading those papers.


2013-11-09

Iterative development and physical objects.

Decades ago, programming a computer consisted of punching small holes in paper cards and feeding them into a reader.  Admittedly I am old, but I am not old enough to have experienced what must have been a frustratingly tedious process.  I'm not sure I would have become a programmer if I had been confronted with the tedious business of writing my programs out on paper and then punch out cards. Writing programs back then required a lot of up front planning -- and since computers were expensive and rare, everyone I have spoken to who programmed computers in this manner emphasise that computer time was a limited resource.

It wasn't exactly a process that invited iteration.  You actually needed a fairly good grasp of what you were going to do long before you got around to any of the doing.

Fast forward a few decades and complete religions are erected around the idea of not having the faintest grasp of what you are doing -- but doing it anyway.   You just start writing code, take stock of where you are and then you massage the code some more, iteratively, until it does what you want it to -- or perhaps even something useful you didn't originally think of, but which you ended up doing because the road just took you there.

It really took off a little over a decade ago when all the frightfully boring process oriented people were overrun by the unwashed pragmatists.  People came up with macho names for this recklessness, such as "extreme programming", or worse.  (This is what happens when pale geeks with no upper-body strength read too many fantasy novels).

The 2000s software engineering methodologies can be summarized about "how about we stop talking and just do stuff until it works".  It was a glorious time.

Eventually we arrived at ways of working that departed completely from the realm of computer engineering and landed squarely in business development.  The hip people don't prototype.  They make "minimum viable products".  Which is great if you have absolutely no idea what you are doing business-wise, but appears to lead to questionable software.  Mainly because the focus is on "minimum" and "product" and not so much "viable".  The "viable" is more of a business development aspect.

And that is okay when you are prepared to throw away all your software and start over once your product achieves traction, but sadly, management cultures are striving to catch up.  Managers love the speed with which the illusion of a product arrives -- but few are prepared to face the harsh reality that sets in once you try to "iterate" over a wobbly, chaotic, slapped together foundation.  Management culture is always last to catch up I guess.

But I digress.  I was going to talk about physical objects.

Up until this point, designing physical objects was a bit like dealing with those tedious punch-cards. You needed a plan.  You needed to know what you were doing.  You needed to commit to something and then work long and hard and spend lots of cash to get from idea to mass production.  Because you had to make lots of widgets to eat up the tooling cost -- nobody makes a toilet roll holder with an FM radio and a telephone just to manufacture five of them.

Up to this point, physical objects made from plastics or metal were largely made by people who knew how to design things and who had access to lots of special and very costly toys.  Expensive CAD systems, horribly expensive CAM systems to compute optimum toolpaths, obscenely expensive 5 axis CNC machines to create the molds, materials experts that can pick the right material, plants to do injection molding etc etc.

Process heavy.  Not exactly processes conducive to rapid iteration.

Until affordable 3D printers and 3D design software started popping up.

Sure, 3D printers have existed for a couple of decades now, but their migration into the hands of the unwashed pragmatists has taken a while -- for a number of reasons that are material for several good, long rants.   And while monied engineering firms have used them to do rapid protyping and iteration, these people were largely ... well, engineers.  People who knew what they were doing in the first place.  (Well, that's the general idea anyway).

What we are seeing now is people who have no idea what they are doing rapidly iterating physical designs.  You no longer need to plan when you are going to make something. You no longer need a fully formed idea before starting to build something.  You just do it.  You just start with a flimsy idea and you go from there.  If it doesn't work, you do it again.  You do and you learn.

Iteration is powerful.  If you can iterate fast, it doesn't matter if you start out without a clue and doing all the wrong things.  Because you can try out a bunch of ideas at very little cost and within a very short time-span.

What has happened to software is happening to physical objects.  The universe of people who will be able to design physical dodads is exploding.

(This blog posting was inspired by a recent blog posting by Hans Jørgen Grimstad.  He is building his own 3D printer and he is doing it by iterating.  The XY-axis of his printer is now at revision 87)