Shot from the Movie AI

Mass production is something that most of us take for granted today. We enjoy the effect of it everyday but rarely think about the dramatic change it has brought to the world. Almost everything we buy today is mass produced.

One thing that’s never been able to be mass-produced is ideas. An idea can be a song, a piece of writing, a software program, or a business plan. The process of idea creation is a personal experience that requires creativity and complex thinking. It seems impossible for machines and science to create ideas like we can.

But times have changed since the day of mechanical, dumb machines that can only handle simple tasks. Recent advancements in AI and data science are giving us signs of truly “intelligent” machines, capable of mass-producing original ideas. How exactly can we do it? To answer this question, we have to understand:

• Fundamental pieces of a mass-production system
• Categories of ideas in the world.

Mass Production System

It’s fairly straight-forward to understand a mass-production system. People have been studying and improving it for centuries, ever since the industrial revolution (one can argue it started a lot earlier than that, but you get the point). Such systems usually involve

• Supply chain to gather raw materials
• Transformation process to change raw materials into generally useful, multi-purpose forms
• Assembly lines that assemble small pieces into functional units, again, usually multi-purpose
• Assembly lines that assemble functional units into final products
• Quality assurance in each step

There are also distribution and fulfillment systems, but let’s focus on the creation process for now.

Categories of Ideas

Categorizing all ideas in the world is much harder. One can turn to cosmology or epistemology, but that tends to get philosophical pretty quickly. It gets even more confusing when the categories of ideas itself is an idea (hmm… did somebody say M.C Escher?) But for arguments sake, let’s group ideas into:

• Metaphysical ideas (music, modern art - ideas for ideas sake)
• Worldly ideas (journalism, a business plan, a piece of computer program - ideas rooted in the physical world)

Mass Producing Ideas

To imagine a mass production system for metaphysical ideas, let’s think about music. Notes/sound are the raw materials. Rhythm, riffs, choruses, movements are the small, functional units. Songs are the final products. The brute-force approach would be generating an infinite number of rhythms, riffs, choruses, create all permutation of these elements and listen to every generated song to find the ones you like. But that would take an infinite amount of time.

The key here is quality control. At each step, we identify the “quality” we care about, and create a filter to only let through what we care about. Machine learning technology can already learn “styles” of music. Rules can be created to structure pieces of music into songs, and music analytics engines + recommendation algorithms can analyze them to determine if you will like the music.

Worldly ideas on the other hand, are harder to create. There are many constraints that are hard to capture using mathematics / rules. How can we generate a piece of original news article? What about a business plan to solve a real-world problem? It seems that we have to teach machines to “understand” the real world before we can generate “analysis” from it.

But short of creating original worldly ideas, we can create derivatives of existing ideas. With sufficient data, machines can “learn” from existing ideas and create similar ideas or other forms of the same ideas. The process varies depending on the algorithm, but usually statistical analysis and natural language processing is involved. For example, text analysis engines like the SRI Internaional are able to take existing text and create short summaries from them. Diagnostic engines are able to use patient data and symptoms to create diagnosis.

Conclusion

The goal of mass production is a very simple economic principle: dramatic reduction on cost and dramatic increase on supply. We are still far away from creating music as good as Bach, or generating news reports worthy of the New York Times, but technologies that create ideas have fundamental impacts on society.

With SRI’s text summary technology, it now takes 10% of the time it did before to achieve the same breath in the past, which means it increased our reading speed by 1000%. Electronic music lowers the bar for music production by eliminating the necessity of learning to play musical instruments (biggest hurdle of music creation). The amount of new electronic music changed the fundamentals of the publishing side of the music industry. Companies like Game Salad try to lower the threshold for creating games, and the effect has been the commoditization of game creation. With the same level resources, my friends are way more likely to create games I want to play than Zanga ever will.

Technology is at a point to challenge the traditional method of idea creation. With the right application, we will be able to create more new ideas than ever before. And maybe one day, we will have a robot scientist making things for us.

I really like the way Andreessen describe the Internet. While the internet alone is not enough to “save innovation”, it will be the information backbone for almost any new innovation going forward. Self-driving cars, wearable smart devices, or even cancer treatment will rely on it. At its core, the internet is an information management / distribution system, supported by governments and some of the biggest corporations in the world. It takes in information and sends it to the people who want it with incredible speed and accuracy. Since almost everything with significant impact needs management and distribution, the internet will be the most important piece of support infrastructure for innovation in the foreseeable future.

One of the downsides of the internet infrastructure as a distribution system is the restriction of wired networks. Since we haven’t figured out how to economically scale satellite data transmission, the next best thing is local/regional wireless transmission. To that extend, mobile will be a larger infrastructure than the internet today. For one reason or another, mobile hardware did not pan out like the open architecture of PCs. This places a lot of trust on companies for an open ecosystem to create enough room and incentive for innovation. The iPhone might not be the best suited for connecting self-driving cars, but they have the best distribution and that’s valuable. It’s too great of a hurdle for someone to create a self-driving car control panel that plugs into the iPhone, when Apple is charging developers ridiculous fees just to connect through the data port. Of course they are not trying to make a buck here, it’s a “more acceptable” way to limit competition and that’s why almost all hardware innovations on top of the iOS ecosystem has been low-data-transmission devices like Square or Kinsa that go through the headphone jack. Open access to widely distributed hardware will be important to the mobile ecosystem and sustaining the rate of innovation.

It’s exciting to think about innovation in the fields of energy, raw material, transportation, and bio tech. Andreessen and Thiel talked a lot about the obstacles in these fields, but I’d like to think about the ideal situation. For example, energy is a highly controlled and regulated field. Most people make money in energy because of the lack of innovation. It’s all about supply chain control and price control. Fracking is the “latest and greatest” in oil, but it was first invented and commercialized in the 40s. It took close to 70 years and multiple wars for it to be as big as it is today. What will it take for there to be as many energy hackers as there are javascript hackers? It’s not because there are less energy / chemical engineers. It’s because of the (negative) hollywood effect and the lack of clear path to success. Being a code hacker means you will at least be financially self-sufficient, with the possibility of creating the next Napster or Facebook and have huge impact. Being an energy hacker means you will probably be jobless, have no prior examples to follow, and eventually be acuqa-hired by one of the 3 giant corps in your industry for pennies while they go on and use your technology to make millions.

Equally as important is the business side of these industries. Not just venture capital, but also “business guy” founders, marketers, community management, etc. It takes a few successful examples and some marketing to set a trend, but it will be hard for these examples to come from within the industry because of its ingrained culture. University research groups / outsiders might be better positioned to create new innovation in these industries, and software startups have created a great model of operation. The key here is for the ecosystem to provide enough support so these young companies can survive on their own. Even though the constraints are different, energy / transportation / bio tech startups can take a lot of the same principles when it comes to figuring out product-market fit, creating communities, human-centric design, etc.

Btw, who do you think won the debate? Personally I liked Thiel’s arguments better - more quantitative and less anecdotal, but I agree with Andreessen that innovation is happening faster than ever before.

Flying cars and meals in a pill are figurative imagination from the past, and we have embodiments of these concepts today. Think Google self-driving cars and new dietary movement (juice, protein shakes, multi-vitamin pills and other meal supplements) With the macro trends like the hardware renaissance and biohacking, I’m very excited to see more entrepreneurs in other engineering disciplines.

To Lay the Ground Work

On the most fundamental level, every small action we do on a webpage is triggered by an intent. These are not “want to buy a car”, “want babies” intent. These are more like “click on name link”, “navigate to home page”, “sign up”, etc. Micro-view, super short-term, knee-jerk reaction type of stuff. From that we can define an “Intent” as “the desire of performing an action”, which really is the action itself + some meta data (like time, location, etc).

Now that’s quite a simplistic and lower-value view of Intents. What people really want is to derive “insights” from “intents”, which are the more “meta” intents people talk about in the advertising world (“want to buy a BMW”, “want to go to Daft Punk concert”, etc).

Let’s call the more insightful intents “meta-intents”, and the lower-level, action-representations “simple-intents”.

The Problem Of Predicting Intent

So what does it take to arrive at the “meta-intents” from a bunch of recorded “simple-intents”? When we translate it to “data science” terms, the question becomes “Given a series of prior simple actions, how likely is it for a person to take a particular action I care about in the immediate future?”

Statisticians have long studied this problem. Countless research studies have been conducted, ranging from brand loyalty, purchase behavior, to World of Warcraft and condom usage. Many regressions and latent-models and collaborative filters later, they all have one thing in common: the data has to be high-signal and low-noise. Said in another way, we have to know all of the relevant actions and at the same time filter out actions that are not relevant.

To put it in an example: in a cookie-centric world, an advertiser typically have about 10%-25% of the data. This means if I visited 100 sites this morning, they know about 10-15 of them. How high of a signal is that? How accurately can they predict what I’m trying to do?

In Real Life

Simple intents and meta intents inform one-another. We can derive simple-intents from meta-intents to guide users down to specific paths, and we can observe users’ simple-intents to derive meta-intents. But these 2 operations require different amounts of efforts. “Meta-to-simple” can be achieved almost automatically (mostly driven by algorithms), “simple-to-meta” is much more manual (data scientists or analysts validating assumptions on top of hadoop clusters). On by the way, it’s not obvious what those meta intents are so the assumptions are hard to make. Think the book Freakonomics.

From a cold-start, since you don’t have enough data to study any patterns of simple intents, ‘meta-to-simple’ is the only approach. There might still be room to use public data sets to generate meta-intents, but that’s a timed window as data science becomes commoditized over time. A data-driven product needs to have enough data science DNA in-house to continually experiment with new assumptions, and that’s how you build true advantages that no one else can copy.

There are a lot more topics to think about, like personal profile and short-term vs. long-term intent. We’ll talk about that in another post.

People join startups for different reasons, but almost everyone does it partially for the upside of making lots of money through the equity plans. This is probably the biggest myth for startup employees. The chance of your startup turning into Google or Facebook is practically zero. So the first question you should ask is:

Why should I join a startup if it’s not for the money?

This means completely rule out the money factor, and this is where people’s opinions diverge. For some people it’s the thrill of building something from the ground up, for some people it’s the prospect of taking on a wide range of responsibilities. Whatever it is for you, this is not a matter to be taken lightly. Here are some questions I see people often have and my thoughts about them:

Q: I want to get in early so when this company goes big I’ll be on top. Should I join a startup?
A: If this is your main motivation, it’ll be a long and treacherous process for you. The benefit is far out-weighted by the risk here, you’d have to be a really irrational person to do it. I find the carrot is usually not enough incentive to carry people over the hump. You will be exposed the ugly truths as the company figures out a path to survival, and it’s sometimes soul-crushingly painful. At times you will feel like you’ve just wasted the last 6 month, year, 2 years making something that is doomed to fail, and your friends who work at the big-co next door is making 150k salary while cruising through life. And you will get discouraged. But unlike the founders, you are not nearly as invested in the mission, so you will quit.

Q: I’m not sure about the founders, but I love the mission of this startup. Should I join?
A: From my experience, the is almost a sure death. As proven by history time and time again, ideas aren’t worth much. Ten years ago, people use to value ideas alot more, partially because the startup culture back then was less open. But today startups have become more agile, and the startup landscape is changing more rapidly than ever. Don’t join someone you can’t completely respect.

Q: I don’t know a lot about this industry, but I’m super impressed by the founders. Should I join?
A: Opposite of the last one, and much more promising. Sometimes it’s difficult to evaluate the quality of an idea, especially if you haven’t been working in that industry for more than a few years. Valuable companies are the ones able to adapt and execute, which makes the original idea less valuable. This puts more emphasis on the founders. How do you evaluate a founder? A great track record helps, but not having it doesn’t mean they cannot succeed. They have to be intelligent, genuine, and most importantly, super resourceful. If you find someone who check off all of these marks, I’d say go for it. Good founders have the tendency of getting stuff done come hell or high water. Even if the startup ends up failing, having worked with great people leads you to other great opportunities.

Q: I’m ready to move on, but my team now needs me. Plus, my boss would kill me if I leave.
A: We’ve all been there. You don’t want to let your teammates down by leaving. But this is a 2-way street. This is what happens if current job is not doing it for you and you try to be a “nice guy” by staying: you still don’t enjoy working there, your current work will go as it’s always been, and the startup will suffer. No one really wins in this situation. Give your 2-weeks notice, be respectful, and move on.

Q: I feel like I’ve worked in the corporate environment, it’s time to move onto a new challenge with more responsibilities. Should I join?
Q: I want to start my own company some day, so I’m here to learn. Should I join?
A: These are what I consider “the right reasons” to join a startup. To me, startup is a lifestyle. It’s really exciting at times, pretty fun and interesting most of the time, and really stressful and devastating occasionally. What carries you through the tough times is your personal resilience, passion and curiosity. You will get the opportunity of doing a little bit of everything, taking something by the horn and running with it. You get rewarded for taking risks and learn from making mistakes. The right attitude here takes you a long way.

Q: I can always go work at another startup if this one doesn’t work out right?
A: True. But you want to be careful here. Think of joining a startup as a 2-3 year career investment. How many 2-3 year chunks do you have? How many are you willing to invest?

Here are some day-to-day stuff to think about:

Q: I hear startup people work crazy hours.
A: That’s true somewhere. But if a company is well-ran, you shouldn’t be working more than 65 hrs a week on average. I’d say I worked a 60 hr week on average at Hyperpublic.

Q: I’ve made a lot of friends at my old job. Will I do the same at a startup?
A: It really depends on the company. Some companies are really social and some keep it professional. If you really like a social work environment, you’ve gotta make sure they have that.

Q: I hear you meet a lot of cool people (outside of your own company) by working at a startup. Is that true?
A: It’s true, but it doesn’t come for free. There are plenty of “networking” events, going to all of them can suck up a lot of time. You don’t want to talk to the people who are JUST NETWORKING. My advice is to pick events hosted by good startups, and pick the ones that will teach you something. Getting out of a conversation politely is a good skill to learn. Remember to always be respectful.

Q: I’m a PHD in (Machine Learning | NLP | PL | Cryptography| etc…), what’s it gonna be like for me at a startup?
A: Pretty different than your academic research life for sure. I don’t have a PHD but I’ve done a few years of academic research and have worked with many PHDs both in research settings and startup settings. Sometimes you are gonna have to write that simple CRUD app or that piece of front-end javascript just because everyone else is busy and the company really needs it. But you’ll also have a lot more control in the work you do, and that’s utilizing science in a truly practical manner.

Q: What’s the most exciting part about joining a startup?
A: I can only speak from my own experience. The most exciting thing is that I’ve grown so much faster in the past 14 months. I find myself becoming more resourceful. I have more ideas when it comes to getting stuff done, and I learned it through getting stuff done everyday in the last 14 months.

Q: What’s the suckiest part about joining a startup?
A: As a startup, there is no room for HR mistakes. That’s why when there is a misfit, you have to execute quickly. This is not personal, and it’s not even a statement on the person’s ability. It’s simply a “misfit”. But it still sucks to see people who have become your close friends being let go.

Q: Just how much WILL I make if all goes well?
A: Let’s just do a back-of-the-envelope calculation: let’s say you get 3% of the company (This is completely arbitrary. See PG’s essay for an idea on how much you should ask for). After a few rounds of funding (and most likely 2-3 years), you will be diluted to less than 2%. This means if your company gets sold at 100million, you will make less than 2million. Think about how many companies you know that’s sold for 100million, then think about how many launch announcements you hear per week. I think you get the picture. Also see here and here

Q: I’ve considered everything. Still not sure about this.
A: Try joining on trial basis. This is a great idea for both the startup and the employee if executed correctly. If you decide to do this, do some research about the startup and make sure they don’t have a bad track record of trying to get “cheap” work out of people this way. Do it on a project-basis and set the terms for both cases (hire or no hire) before you start.

Conclusion

Each of these points can be its own blog post, and I might pick specific topics to go into depth in the future. Good luck with this decision. It can be the best thing you’ll ever do for your career.

The New Yorker published a great article this week titled “Group Think: The Brainstorming Myth”. The article tries to prove that “brainstorming” is not as efficient as “casual conversations”, mainly because it discourages criticism. Also see the linkbate on Hacker News “Brainstorming Doesn’t Really Work”.

The author makes great points about some downsides of brainstorming, backing it up with experiments and data. For example, the 2003 UC Berkeley experiment suggests that, groups who arrive to results through a “debate” process is often more productive than groups who use a “brainstorm” process, even though at times debating is unpleasant. On a slightly tangential topic, the author also brings up the point that “unfamiliarity” can increase productivity. (Backed by the Broadway production experiment conducted by Brian Uzzi at Northwestern) The article goes through many successful examples of modern work space setups that encourage casual conversations between multiple disciplines.

I totally disagree with what the linkbate suggests. (Although the article itself is quite interesting) Brainstorming and casual conversation have very different roles in the creative process. The points presented in the article are all valid, but the comparison is inappropriate. In a modern-day work environment, we need both.

Brainstorming is an organized activity. While it’s difficult to administor (therefore most brainstorming sessions are pretty poorly ran), it’s quite effective if implemented right. Here is why:

• It gets all the shitty ideas out of your system. Everyone has them, it doesn’t matter how smart/creative you are. Getting all the shitty ideas out of your head frees more mental space for you to focus on the good ones.
• It’s highly collaborative. It’s difficult to have a 5-way casual conversation. But having 5 different perspectives is super valuable. Sometimes 1 small idea changes the whole game.
• Most brainstorming sessions involve Whiteboards. This is good. Laying things out visually can again, free up more energy to focus on the hard problems.
• Breath-first vs. depth-first. For those who are not familiar with graph theory, BFS means exploring all of the adjacent points first, and DFS means go as deep as possible on one idea before exploring other ideas. Brainstorming tends to be on the BFS side, and that’s important especially at the early stage of a product.

I’m personally a huge fan of casual conversations. It’s a pressure-free and natural way to refine ideas and solve problems from other angles. These mini sessions can be extremely productive, but it’s kind of a hit-or-miss. To maximize the probablility of productive casual conversations here at Hyperpublic, we essentially sit in one big circle with the snack table in the center. Some of our best work come from casual conversations. Coffee trips is another good source of interesting discussions. When you are solving a technical problem, being able to talk to someone else while the problem is fresh in your head can be very productive.

Brainstorm and casual conversation are both important. They compliment each other in various ways, and the right practice of both can increase our productivity by leaps and bounds.

Taken by Justin Merriman, Tribune-Review

Last weekend I was fortunate enough to speak at the Build18 hackathon at my Alma Mater, Carnegie Mellon University. Unlike many of the hackathons I’ve seen in the “real world”, most projects here were not “business ideas”. They were true “hacks” - ideas created by the intellectual exhaust, experiments that exists for no purpose other than existance itself. I was super impressed by the technical depth of the projects.

At the awards dinner, standing in front of a room full of sleep-deprived college engineering students, I couldn’t help but to think back to 4 years ago. We were sitting in the same seats, filled with gumption and no particular direction, hoping to make a dent in the world. We wanted to be important, special, even rich, because we have earned the rights through the endless pencil-smeared pages of equations and the caffine-buzzed lab hours. “My heart is in the work”, says Andrew Carnegie. All you have to do is to keep your head straight, work hard, and eventually the dent will be made. Right?

As it turns out, direction and course correction is more important than hard work. Just as a car with a powerful engine but no steering wheel has very little chance of getting to its destination, effort is the pre-requisite for doing anything significant.

I had planned to share a slice of the startup life, maybe even inspire some of them to start or join a startup. But given the particular setting and timing (they were probably all cracked out from a week-long hacking session), the most efficient way to get my points across was to be succinct. I compiled everything I wanted to say into the following 8 bullet points.

• If you haven’t read PG’s essays or don’t already read hacker news, just do it.
• Iterate - It’s quite a remarkable concept. I alaways get a feeling of comfort when I know that I can just keep doing the same things over and over again, and my product will magically become better and better.
• Launch early, launch often - Be lean. It’s something to strive for, but don’t kill yourself over it. It’s much more important to know why you should be lean than how to be lean.
• Over-share your idea - Don’t be afraid that other people will steal your startup idea. The risk of hiding your idea and no learning from other people is much greater than the risk of some douche stealing your idea. Ask people who have done this before, ask people who you really respect, ask random people in coffee shops.
• Always find ways to validate your idea - Think of your startup idea as the hypothesis, and your goal is to test your hypothesis through as many lenses as possible. The power of statistical approaches is much greater than statistics itself.
• Define some metrics - Metrics are great. They keep you focused on the most important things. Find the thing that will kill you and do that first.
• Avoid going to networking events - Also don’t think about raising money too soon. Both are HUGE distractions from building your product. If you are trying to be “one of the cool kids”, you are doing it wrong.
• Don’t be a dick - The people you work with are your greatest assets, so be very careful of choosing who you want to work with. Once you’ve made that decision, treat them like a brother (or sister).

Startup is definitely a learn-by-doing experience, which means the students probably won’t do any of the above 8 things. But hopefully these tips will guilde them back towards the right path when they make a mistake.

Here is a step-by-step tutorial on how I build this site:

Install Jekyll

You should first install jekyll. There is a good tutorial on the Github wiki. But If you don’t want to follow the official tutorial, here is what I did.

Install the Jekyll gem by running:

gem install jekyll

If you want syntax highlighting for code blocks, you have to install a python module called Pygments.

sudo easy_install Pygments

Of course you have to have python-setuptools installed.

First blog entry

After installing jekyll, I created my site structure that looks like this:

      .
      |-- _config.yml
      |-- _includes
      |-- _layouts
      |   |-- default.html
      |   `-- post.html
      |-- _posts
      |   |-- 2012-01-20-some-post-I-wrote.textile
      |   `-- 2012-01-21-how-I-built-this-blog.textile
      |-- _site
      `-- index.html

The best way to get started is to take a look at someone else’s project. Since I’m terrible at anything css/html related, I borrowed a lot of code from User Obsessed.

The _config.yml file contains configuration settings of the site. Here is what mine looks like:

    url: "http://www.erictang.org"
    pygments: true
    exclude: [/bin, Gemfile, Gemfile.lock]
    markdown: kramdown
    permalink: pretty
    paginate: 10

    kramdown:
      entity_output: symbolic
  

The index.html file is your homepage. The layouts directory contains the formating of your pages, and the posts directory contains individual posts.

To get a simple site up and running, I created config.yml, index.html, layouts/default.html and _posts/2012-1-21-sample-post.textile

After that, run

    jekyll --auto --server
  

You should see a server starting up. Go to http://localhost:4000 to make sure it’s working.

There are many other things you can do with this barebone site, refer to the Wiki for details.

Disqus

Comments are important for blogs. There are many different ways to add comments to your Jekyll-powered blog, I chose to use Disqus because it’s easy to integrate.

Luckily Disqus has make this process SUPER easy. Here is what you have to do:

• Sign up for a disqus account, register your domain.
• Copy the embeded code to your _layouts/posts.html. Remember to replace “example” with your own shortname.
• Reload the page.

That’s it!

That’s it! You’ve set up your first jekyll blog!