Posts

Posted on

Repost: What to do about COVID-19?

It’s about your psychological health and survival.

As the internet gets inundated with graphs and viral misinformation about COVID-19, and the many ramifications, it’s easy to go down a rabbit hole of fear and start imagining a veritable apocalypse.

(It’s also just as easy to err in the other direction and erroneously say that everyone is just over-reacting to a flu bug).

But what’s hard to do in these times of high uncertainty and high anxiety, when the ground is shifting under us and very little is the same as it was two weeks ago, is to manage your internal state.   

So here is a Five Step Action Plan to help you gain some equanimity during this unprecedented time:

Step One: Prepare, don’t panic.   

There is a big difference between preparingand panicking.

If you’re preparing, you’re logically evaluating a situation, taking the necessary steps keep yourself safe, and monitoring for any major developments.   

If you’re panicking, you’re probably not prepared, and likely won’t be able to prepare intelligently either. You become a liability to yourself and others. 

So avoid the anxiety of feeling unprepared by actually preparing. Do your research, make a list of what you need, obtain those items, and create protocols to keep your home and surroundings (including workplace if applicable) a safe zone at all times. 

Once you have this set up, and you can do it in half a day or less, you can move on to the next step:

Step Two: Operate in reality, not anxiety. 

Anxiety is an exercise in future projection, according to the leading neuroscientist Joseph LeDoux. 

It takes place in a different part of the brain than fear. 

Fear is an instant response to a present danger. 

Anxiety is an exercise of the imagination.

And some people are imagining the worst. 

So here’s a fact that may help those who have major concerns about the economy, which is likely just about everyone. 

During the Great Depression, there was no uptick in deaths. In fact, mortality rates actually improved. So people survived financial devastation. And we will survive the economic consequences of this. 

However, there was one distinct category during the Great Depression where there were more deaths: Suicide. 

Some people thought they lost everything when what they really lost was their savings, their investments, their businesses. But if you have your self, the people you love, and you all manage to stay healthy through this, you have everything.

The rest, while painful, can be rebuilt, just as it was built from nothing in the first place.

If life gets very tough and we have to struggle, it will make us stronger, it will make us more grateful, and we will bounce back.

The number one priority is to care for your health and your immunity, and adding unnecessary stress hurts both of those. 

So do the inner work to have inner calm during all this. 

Remember: 

You cannot control every event in the outer world, but you can control your reactions to them. 

This is an equation from former Brian Kight…

Event + Reaction = Outcome 

One half of that equation is fixed. Now is the time to dial in the other half of it, work on your reactivity, and balance your inner world. 

This leads us to: 

Step Three: Create a self-care regime 

While we are disconnecting from others during this time, the social implications of which are going to be staggering, what is going to come up is a lot of the feelings that we medicate through work, sex, the gym, socializing, and generally keeping busy. 

The biggest challenge for many is to sit alone with themselves; for others, it’s to sit cooped up with their families pressing the buttons they haven’t worked to remove yet. 

So your main job is to really take care of yourself, inside and out. 

Make yourself a schedule and stick to it. 

Don’t have a morning routine? Time to start one.

Don’t eat healthy meals and supplementation? Time to be the healthiest you’ve ever been. 

Don’t have a meditation practice? Time to find one. 

Don’t exercise? Find a great resource online and do it daily. And get outside or get fresh air while still social distancing.

Feeling depressed, alone, or anxious? Feel deeply into it, notice it, observe it, and get curious about it. When have you felt it before? What’s it really about? Our neuroses will attach to this new situation. It is a great time to feel in order to heal.

News getting you worked up? Download Freedom or a similar app, and only give yourself access to news and/or social media sites for two small windows of time during the day. 

The real goal here is not to sink into malaise or lethargy, but to leap into motivation.   

So many people I talk to say that their job gets in the way of their passions. Well, here’s your chance. Many of you can use this time to work on living those passions. 

And I can guarantee one other thing: if you do not use this time productively, but instead fritter it away compulsively checking Twitter and the news, you will deeply regret having wasted a precious opportunity. 

And before a few of you email saying that’s not possible for you because of this or that reason, read the next step. 

Step Four: Adapt and Flourish

I’ve seen two responses to the shift we are all experiencing. 

Response One: All these things are gone now, I’ll have to live without them and wait it out. 

Response Two: All these things are gone now, I’ll do other things instead. 

The first response is fixed mindset, the second is growth mindset. 

The world we once knew is gone for now. We are in a new reality. 

And to survive it, we must use the trait that has enabled our species to survive for so long: adaptability. 

If your business model is suffering, pivot to something that people need now. If you lost your job, look at where the needs are in society right now. Now’s the time to get proactive and start a new life. Amazon, for example, just announced it was hiring 100,000 people. 

If you’re upset that you have to put your social plans on hold, then figure out how to socialize in this new world. My in-person game night became a fun game night using multiple-player app-games with everyone on video conference. 

Can’t go on dates? People are lonely. Now’s the time to build a deep connection with someone new, and just treat it like a long-distance relationship.

Switch from the fear and scarcity brain to the problem-solving and abundance brain. 

When your back is against the wall, it’s a good time to find a new door.

But always remember…

Step Five: Service 

A lot of people are suffering, and more will likely suffer and worse. Part of this opportunity for you is service. A chance to give back.

If you have the economic means, see if there’s a helpful initiative you can donate to for your community, or the elderly and other high-risk populations. Is there anything materially or physically you can do? Spend a day thinking of ways to be of service, then add that to your self-care regime.

After all, one of the best ways to get out of your own emotional lows is to help others with their lows. Love, compassion, and community is the best medicine. It lasts longer than laughter.

P.S. I’ll close this with a poem that went viral, in case you haven’t seen it. As with all things viral, the attribution may not be correct:

And the people stayed home. And read books, and listened, and rested, and exercised, and made art, and played games, and learned new ways of being, and were still. And listened more deeply. Some meditated, some prayed, some danced. Some met their shadows. And the people began to think differently.

And the people healed. And, in the absence of people living in ignorant, dangerous, mindless, and heartless ways, the earth began to heal.

And when the danger passed, and the people joined together again, they grieved their losses, and made new choices, and dreamed new images, and created new ways to live and heal the earth fully, as they had been healed.

–Kitty O’Meara

Posted on

Book Review – Hello World: Being Human in the Age of Algorithms – Part 1

Introduction
I often think of AI as something separate from traditional computer programming, something transcendent. However, most of the advances in modern AI are not the result of revolutionary new concepts or fields of study but rather the application of previously developed algorithms to significantly more powerful hardware and massive datasets.

Hannah Fry’s take on the world of AI covers topics ranging from justice to autonomous vehicles, crime, art and even to medicine. While the author is an expert in the field, she does a great job distilling the topics down to a level understandable by a layperson, but also keeps it interesting for someone with more background in programming and AI.

My favourite quote from the first part of the book comes on page 8, where Hannah succinctly describes the essence of what an algorithm is in only one sentence:

An algorithm is simply a series of logical instructions that show, from start to finish, how to accomplish a task.

Fry, Hannah. Hello World: Being Human in the Age of Algorithms (p. 8). W. W. Norton & Company. Kindle Edition

Once you read it, it seems obvious, but trying to describe to a first-year computer science student what an algorithm is can be a challenging task. The author manages this well. Despite the complexity and depth of the subject matter, Fry is able to bring context and relevance to a broad array of topics. The remainder of my review will speak to some of the book’s many sections and how someone with a business-facing view into the topics sees them.

Data
This section covers some of the unknown giants in data-science including Peter Thiel’s Palantir. The section also touches on some very public examples where analytics has played a negative role – Cambridge Analytica’s use of private user data during the 2016 Presidential Elections.

The story here is about data brokers. Data brokers are companies who buy and collect user data and personal information and then resell it or share it for profit. A surprising fact is that some of these databases contain records of everything that you’ve ever done from religious affiliations to credit-card usage. These companies seem to know everything about just about everyone. It turns out that it is relatively simple to make inferences about a person based on their online habits.

The chapter converges to one of the major stories of 2018, the Cambridge Analytica scandal. But it begins by discussing the five personality traits that psychologists have used to quantify individuals’ personalities since the 1980s: openness to experience, conscientiousness, extraversion, agreeableness and neuroticism. By pulling data from users’ Facebook feeds, Cambridge Analytica was able to create detailed personality profiles to deliver emotionally charged and effective political messages.

Perhaps the most interesting fact though, is how small of an impact this type of manipulation actually has. The largest change reported was from 11 clicks in 1000 to 16 clicks in 1000 (less than 1 percent). But even this small effect, spread over a population of millions can cause dramatic changes to the outcome of, say, an election.

That’s the end of part 1 of this review. In Part 2, I’ll touch on some of the other sections of the book including Criminal Justice and Medicine.

Posted on

AI Everything

These days it seems like businesses are trying to use AI to do everything. At least for startups, that isn’t far off. Anywhere there is a dataset remotely large enough and an answer that is vaguely definable, companies are putting together a business model to use machine learning to solve the problem. With some incredible successes in areas like image classification and defeating humans at video games, its hard not to be impressed.

One of the best channels for following recent breakthroughs in AI is the 2 Minute Papers YouTube Channel, started by Károly Zsolnai-Fehér, a professor at the Vienna University of Technology in Austria. Károly’s videos combine interesting clips of the programs in action with well-delivered summaries of recent papers illustrating advances in artificial intelligence.

In one of his latest videos, he covers an AI that not only can copy the most successful actions that humans take in video games but can actually improve on those actions to be better than the best human players. So does that mean that AI will be displacing office workers once it learns how to do their jobs better than them? Probably, yes. But maybe not quite how you think it might.

As much of a ‘black-box‘ as AI has been in the past, modern systems are becoming better and better at explaining how they arrived at an answer. This gives human operators predictive capabilities that we didn’t have with systems of the past that could spit out an answer but gave us no indication of how that answer was formulated.

This Forbes article on Human-Centric AI provides some examples of how modern AI systems can be implemented to train employees to do their jobs better and even enjoy their jobs more while doing it! If that doesn’t sound incredible to you, you may be a machine who is only reading this page to improve your search algorithm.

So what does this all mean? A lot of research is showing that AI is actually creating many more jobs than it destroys. So, as long as you’re willing to try and understand the systems that will one day be our overlords, you should be able to upgrade your career and stay employed.

Whether you still want the job that remains is another question entirely.

Posted on

Turning Your Selfie Into a DaVinci

Transfer learning. It’s a branch of AI that allows for the style transfer from one image to another. It seems like a straightforward concept: take my selfie and make it look like a Michelangelo painting. However, it is a fairly recent innovation in Deep Neural Networks that has allowed us to separate the content of an image from its style. And in doing so, to combine multiple images in ways that were previously impossible. For example, taking a long-dead artist’s style and applying it to your weekend selfie.

Just to prove that this is pretty cool, I’m going to take my newly built style transfer algorithm and apply it to a ‘selfie’ of my good dog, Lawrence. Here’s the original:

And here’s the image that I’m going to apply the style of:

That’s right, it’s Davinci’s Mona Lisa, one of the most iconic paintings of all time. I’m going to use machine learning to apply Davinci’s characteristic style to my iPhone X photo of my, admittedly very handsome, pupper.

If you’re interested, here’s a link to the original paper describing how to use Convolutional Neural Networks or CNNs to accomplish image style transfer. It’s written in relatively understandable language for such a technical paper so I do recommend you check it out, given you’re already reading a fairly technical blog.

So what is image content and style and how can we separate out the two? Well, neural networks are built in many layers, and the way it works out, some of the layers end up being responsible for detecting shapes and lines, as well as the arrangement of objects. These layers are responsible for understanding the ‘content’ of an image. Other layers, further down in the network are responsible for the style, colors and textures

Here’s the final result next to the original.

Pretty striking, if I do say so myself.

Using a pre-trained Neural Network called VGG19 and a few lines of my own code to pull the figures and what’s called a Gram Matrix I choose my style weights (how much I want each layer to apply). Then using a simple loss function to push us in the right direction we apply the usual gradient descent algorithm and poof. Lawrence is forever immortalized as a Davinci masterpiece.

Impressed? Not Impressed? Let me know in the comments below. If you have anything to add, or you think I could do better please chime in! This is a learning process for me and I’m just excited to share my newfound knowledge.

Here’s a link to my code in a Google Colab Notebook if you want to try it out for yourself!

Posted on

How Data Will Be Used To Decide Your Future

Welcome to 2019! Let’s start the year off by discussing how your ability to exercise your free will could be directly impacted by who you follow on social media, or who you pass by on the street.

In the dystopian Netflix series Black Mirror, there is an episode called Nosedive where a person’s ability to ride a plane, work at their job or even be served food at a restaurant is decided by a crowd-sourced rating system. Much like you rate your Uber driver at the end of your trip (and your Uber driver rates you), in this episode, every social interaction is followed by both parties giving each other a star-rating with an app on their phone. Everyone’s rating is public and it seems like you can downvote someone at any time. People with higher overall scores have more influence on the scores of people that they interact with. Inevitably, the episode’s main character encounters a string of bad luck and unfortunate encounters that lead to her score tumbling to almost nothing. She ends up losing her job, her home and her ability to travel in a matter of days based on nothing more than a few left swipes from some strangers and a co-worker.

Image result for black mirror rating episode

Everyone has bad days, so it really hits home when you see the protagonist lose her life over a few social mishaps. She doesn’t actually die, but she is locked in a plastic box because of her low social score.

How far are we from experiencing this kind of scenario in our lives?

The Chinese government has announced plans to implement a Social Credit Score for its citizens, that tracks all online and in-person activity, including who your friends are, who you talk to and who your acquaintances talk to. This score will decide if you can get a loan on a house, similar to a credit score in the United States or Canada, but it will also decide what schools you can go to, what businesses you can shop at and even whether or not you can leave the country. Going live in 2020, literally next year, all 1.6 billion Chinese residents will be subject to this Social Credit System; a system that is eerily similar to the one described in the Black Mirror episode.

I live in North America. Calgary, Alberta, Canada, specifically, so this won’t affect me, right?

All around the world, people have free and easy access to instant global communication networks, the wealth of human knowledge at their fingertips, up-to-the-minute information from across the earth, and unlimited usage of the most remarkable software and technology, built by private companies, paid for by adverts. That was the deal that we made. Free technology in return for your data and the ability to use it to influence and profit from you. The best and worst of capitalism in one simple swap. We might decide we’re happy with that deal. And that’s perfectly fine. But if we do, it’s important to be aware of the dangers of collecting this data in the first place. We need to consider where these datasets could lead – even beyond the issues of privacy and the potential to undermine democracy (as if they weren’t bad enough). There is another twist in this dystopian tale. An application for these rich, interconnected datasets that belongs in the popular Netflix show Black Mirror, but exists in reality. It’s known as Sesame Credit, a citizen scoring system used by the Chinese government. Imagine every piece of information that a data broker might have on you collapsed down into a single score. Everything goes into it. Your credit history, your mobile phone number, your address – the usual stuff. But all your day-to-day behaviour, too. Your social media posts, the data from your ride-hailing app, even records from your online matchmaking service. The result is a single number between 350 and 950 points. Sesame Credit doesn’t disclose the details of its ‘complex’ scoring algorithm. But Li Yingyun, the company’s technology director, did share some examples of what might be inferred from its results in an interview with the Beijing-based Caixin Media. ‘Someone who plays video games for ten hours a day, for example, would be considered an idle person. Someone who frequently buys diapers would be considered as probably a parent, who on balance is more likely to have a sense of responsibility.’ If you’re Chinese, these scores matter. If your rating is over 600 points, you can take out a special credit card. Above 666 and you’ll be rewarded with a higher credit limit. Those with scores above 650 can hire a car without a deposit and use a VIP lane at Beijing airport. Anyone over 750 can apply for a fast-tracked visa to Europe. It’s all fun and games now while the scheme is voluntary. But when the citizen scoring system becomes mandatory in 2020, people with low scores stand to feel the repercussions in every aspect of their lives. The government’s own document on the system outlines examples of punishments that could be meted out to anyone deemed disobedient: ‘Restrictions on leaving the borders, restrictions on the purchase of . . . property, travelling on aircraft, on tourism and holidays or staying in star-ranked hotels.’ It also warns that in the case of ‘gravely trust breaking subjects’ it will ‘guide commercial banks . . . to limit their provision of loans, sales insurance and other such services’. Loyalty is praised. Breaking trust is punished.

Fry, Hannah. Hello World: Being Human in the Age of Algorithms (pp. 44-46). W. W. Norton & Company. Kindle Edition.

Science fiction mirrors reality and reality reflects science fiction. While this type of system is only going live in China, it is unclear how long we have before something similar arrives in the West.

Posted on

How Afraid Should You Be of AI?

A friend sent me a video today. It started off rather innocuously, with a program called EarWorm, designed to search for Copyrighted content and erase it from memory online. As many of these stories do, it escalated quickly. Within three days of being activated by some careless engineers with no backzground in AI ethics, it had wiped out all memory of the last 100 years. Not only digitally, but even in the brains of the people who remembered it. Its programmers had instructed it to do so with as little disruption to human lives as possible, so it kept everyone alive. It might have been easier to just wipe humanity off the map. Problem solved. No more Copyrighted content being shared, anywhere. At least that didn’t happen. Right?

This story is set in the year 2028, only ten years from now. These engineers and programmers had created the world’s first Artificial General Intelligence (AGI) and it rapidly became smarter than all of humanity, with the computing power and storage capacity surpassing what had been available previously though all of human history. Assigned a singular mission, the newly formed AGI sets out to complete its task with remorseless efficiency. It quickly invents and enlists an army of nanoscopic robots that can alter human minds and wipe computer memory. By creating a mesh network of these bots that can self-replicate, the AI quickly spreads its influence around the world. It knows that humans will be determined to stop it from accomplishing its mission, so it uses the nanobots to slightly alter the personalities of anyone intelligent enough to pose a threat to its mission. Within days it accomplishes its task. It manipulates the brains of its targets just enough to achieve the task while minimizing disruption. It does this by simply reducing the desire of the world’s best minds in AI to act. It creates apathy for the takeover that is happening right in front of them. By pacifying those among us intelligent enough to act against it, its mission can proceed, unencumbered by pesky humans.

Because it was instructed to accomplish its task with ‘as little disruption as possible’ the outcome isn’t the total destruction of humanity and all life in the universe, as is commonly the case in these sorts of AI doomsday scenarios. Instead, EarWorm did as it was programmed to do, minimizing disruption and keeping humans alive, but simultaneously robbing us of our ability to defend ourselves by altering our minds so that we posed no threat to its mission. In a matter of days, AI drops from one of the most researched and invested-in fields to being completely forgotten by all of humanity.

This story paints a chilling picture (though not as chilling as many ‘grey-goo’ scenarios, which see self-replicating, AI-powered nanobots turning the earth, and eventually the entire universe, into an amorphous cloud of grey goo). It is a terrifying prospect that a simple program built by some engineers in a basement could suddenly develop general intelligence and wipe an entire century of knowledge and information from existence without a whimper from humanity.

How likely is it? Do we need to worry about it? and What can we do about it? are some of the questions that sprang to mind as I watched the well-produced six-minute clip.  It is a scenario much more terrifying and unfortunately, more plausible than those of popular TV and films like Terminator and even Westworld. There are a lot of smart people out there today who warn that AI, unchecked, could be the greatest existential threat faced by humanity. It’s a sobering thought to realize that this could happen to us and we wouldn’t even see it coming or know it ever happened.

Then, the real question that the video was posing dawned on me: Has this already happened?

We could already be living in a world where AI has already removed our ability to understand it or to act against it in any way…

I hope not, because that means we’ve already lost.

Here’s the video if you’re interested

Posted on

We’re Hiring!… And Raising Money!

My company, InOrbis Intercity, is doing a capital raise. We’ve been operating profitably for nearly two years now and we’re growing at a steady 10-20% per month. It’s exciting, to say the least, but this is the first time I’ve raised outside money. I have read book after book on Venture Capital, Angel Investment, Private Equity, and even raising money via an ICO (Initial Coin Offering). I have pitched our company several times, but the interest has been limited in this neck of the woods (Alberta, Canada). I think it’s time to start investigating alternative funding sources.

I know this business can be a billion dollar company, but we need to build scalable software to allow us to operate in more markets. We have developers, but we need more, and we need to hire some of the best in the business to work on our team. We’re building something great, applying AI to make intercity travel a better experience for millions of people. 

Raising money is a battle, only about 0.01% of companies actually manage to raise the funds for a billion-dollar valuation. There are hundreds of unreported examples where companies raise multiple-millions, the company grows and is sold for tens of millions but the founders are left with nothing.

It is a monumental risk, to be sure, but at this point, it seems like the best option we have available to grow the business.

If you’re interested in being part of something special, the future of autonomous, sustainable, intercity transportation, and you are a passionate, intelligent developer, or software engineer, willing to risk it all on a moonshot (I know, there must be at least one or two out there), send me a message and we’ll meet for coffee.

On the flip side, if you’re an Angel or VC and you’d like to learn more, drop us a line. I’d be happy to pull out our pitch deck and wow you with this vision. We will make it happen, but your help could make it a reality that much sooner.

Cheers,

Rosario

Posted on

On AI and Investment Management

Index funds are the most highly traded equity investment vehicles, with some funds like ones created by Vanguard Group cumulatively being valued at over $4 Trillion USD. Index funds have democratized investing by allowing access to passive investments for millions of people. But what are they?

An index fund is a market-capitalization weighted basket of securities. Index funds allow retail investors to invest in a portfolio made up of companies representative of the entire market without having to create that portfolio themselves. Compared to actively managed funds like mutual funds and hedge funds, index funds tend to have much lower fees because the only balancing that happens occurs based on an algorithm to keep the securities in the fund proportional to their market cap (market capitalization, or market cap, is the number of shares that a company has on the market multiplied by the share price).

Starting in the 1970s, the first ‘index funds’ were created by companies that tried to create equally weighted portfolios of stocks. This early form of the index fund was abandoned after a few months. It quickly became apparent that it would be an operational nightmare to be constantly rebalancing these portfolios to keep them equally weighted. Soon companies settled on the market capitalization weighting because a portfolio weighted by market cap will remain that way without constant rebalancing.

With the incredible advancement of AI and extraordinarily powerful computers, shouldn’t it be possible to create new types of ‘passively managed’ funds that rely on an algorithm to trade? What that could mean is that index funds might not have to be market cap weighted any longer. This push is actually happening right now and the first non-market cap weighted index funds to appear in over 40 years could be available to retail investors soon.

But this means that we need to redefine the index fund. The new definition has three criteria that must be met for a fund to meet:

  1. It must be transparent – Anyone should be able to know exactly how it is constructed and be able to replicate it themselves by buying on the open market.
  2. It must be investable – If you put a certain amount of money in the fund, you will get EXACTLY the return that the investment shows in the newspapers (or more likely your iPhone’s Stocks app).
  3. It must be systematic – The vehicle must be entirely algorithmic, meaning it doesn’t require any human intervention to rebalance or create.

So, what can we do with this new type of index fund?

“Sound Mixer” board for investments with a high-risk, actively traded fund (hedge fund) on the top and lower risk, passively traded fund (index fund) on the bottom.

We can think of investing like a spectrum, with actively managed funds like hedge funds on one side and passively managed index funds on the other and all the different parameters like alpha, risk control and liquidity as sliders on a ‘mixing board’ like the one in the image above. Currently, if we wanted to control this board, we would have to invest in expensive actively managed funds and we wouldn’t be able to get much granular control over each factor. With an AI-powered index fund, the possibilities of how the board could be arranged are endless. Retail investors could engage in all sorts of investment opportunities in the middle, instead of being forced into one category or another.

An AI-powered index fund could allow an investor to dial in the exact parameters that they desire for their investment. Risk, alpha, turnover, Sharpe ratio, or a myriad of other factors could easily be tuned for by applying these powerful algorithms. 

The implications of a full-spectrum investment fund are incredible. Personalized medicine is a concept that is taking the industry by surprise and could change the way that doctors interact with patients. Companies like Apple are taking advantage of this trend by incorporating new medical devices into consumer products, like with the EKG embedded into the new Apple Watch Series 4.

Personalized investing could be just as powerful. Automated portfolios could take into account factors like age, income level, expenses, and even lifestyle to create a portfolio that is specifically tailored to the individual investor’s circumstances.

So why can’t you go out and purchase one of these new AI managed, customizable index funds?

Well, unfortunately, the algorithms do not exist, yet. The hardware and software exists today to do this but we’re still missing the ability to accurately model actual human behaviour. Economists still rely on some pretty terrible assumptions about people that they then use to build the foundations of entire economic theories. One of these weak assumptions is that humans act rationally. Now, there is a lot of evidence to suggest that many people act in the way that we are programmed to by evolution. The problem is, a lot of what allowed us to evolve over the last 4 billion years of life on earth, is pretty useless for success in 2018-era financial planning and investment.

All hope is not lost, however. New research into the concept of bounded rationality, the idea that rational decision making is limited by the extent of human knowledge and capabilities, could help move this idea forward. One of the founding fathers of artificial intelligence, Herbert Simon,  postulated that AI could be used to help us understand human cognition and better predict the kinds of human behaviours that helped keep us alive 8,000 years ago, but are detrimental for wealth accumulation today. 

By creating heuristic algorithms that can capture these behaviours and learning from big data to understand what actions are occurring, we may soon be able to create software that is able to accentuate the best human behaviours and help us deal with the worst ones. Perhaps the algorithm that describes humanity has already been discovered.

Posted on

I Built a Neural Net That Knows What Clothes You’re Wearing

Okay, maybe that is a bit of a click-baitey headline. What I really did was program a neural network with Pytorch that is able to distinguish between ten different clothing items that could present in a 28×28 image. To me, that’s still pretty cool.

Here’s an example of one of the images that gets fed into the program:

Yes, this is an image of a shirt.

Can you tell what this is? Looks kind of like a long-sleeve t-shirt to me, but it is so pixelated that I can’t really tell. But that doesn’t matter. What matters is what my trained neural-net thinks it is and if that’s what it actually is.

After training on a subset of images like this (the training set is about 750 images) for about 2 minutes, my model was able to choose the correct classification for any image that I fed in about 84.3% of the time. Not bad for a first go at building a clothing classifying deep neural net.

Below I have included the code that actually generates the network and runs a forward-pass through it:



class Network(nn.Module):

    def __init__(self, input_size, output_size, hidden_layers, drop_p=0.5):

        ''' Builds a feedforward network with arbitrary hidden layers.

        

            Arguments

            ---------

            input_size: integer, size of the input

            output_size: integer, size of the output layer

            hidden_layers: list of integers, the sizes of the hidden layers

            drop_p: float between 0 and 1, dropout probability

        '''

        super().__init__()

        # Add the first layer, input to a hidden layer

        self.hidden_layers = nn.ModuleList([nn.Linear(input_size, hidden_layers[0])])

        

        # Add a variable number of more hidden layers

        layer_sizes = zip(hidden_layers[:-1], hidden_layers[1:])

        self.hidden_layers.extend([nn.Linear(h1, h2) for h1, h2 in layer_sizes])

        

        self.output = nn.Linear(hidden_layers[-1], output_size)

        

        self.dropout = nn.Dropout(p=drop_p)

        

    def forward(self, x):

        ''' Forward pass through the network, returns the output logits '''

        

        # Forward through each layer in `hidden_layers`, with ReLU activation and dropout

        for linear in self.hidden_layers:

            x = F.relu(linear(x))

            x = self.dropout(x)

        

        x = self.output(x)

        

        return F.log_softmax(x, dim=1)

After training the network using a method called backpropagation and gradient descent (code below), the network successfully classified the vast majority of the images that I fed in, in less than half a second. Mind you, these were grayscale images, formatted in a simple way and trained with a large enough dataset to ensure reliability.

If you want a good resource to explain what backpropagation actually does, check out another great video by 3 Blue 1 Brown below:

So, what does this all look like? Is it all sci-fi futuristic and with lots of beeps and boops? Well… not exactly. Here’s the output of the program:

Output of my clothing-classifier neural net. Provides a probability that the photo is one of the 10 items listed.

The software grabs each image in the test set, runs it through a forward pass of the network and ends up spitting out a probability for each image. Above, you can see that the network thinks that this image is likely a coat. I personally can’t distinguish if it is a coat, a pullover or just a long-sleeve shirt, but the software seems about 85% confident that it is, in fact, a coat.

Overall, it’s pretty awesome that after only a few weeks of practice (with most of that time spent learning how to program in python) I can code my very own neural networks and they actually work!

If you’re interested, here’s a video of the neural network training itself and running through a few test images:

If you’d like to test out the code for yourself, here’s a link to my GitHub page where you can download all the files you need to get it running. Search Google if you can’t figure out how to install Python and run a Jupyter Notebook.

That’s all for now! See you soon 🙂

Posted on

Facebook Made The Best Tool For Creating Neural Networks

It’s called PyTorch. And it’s a tool designed to work perfectly with Python libraries like NumPy and Jupyter Notebooks to create deep neural networks. As it turns out, it is much easier to use and more intuitive than Google’s TensorFlow packages. In fact, I have been trying to get TensorFlow working on my Mac laptop for about a month, each time I run it, I get a new error, and when I fix that error, I encounter another, and another, until I eventually resign myself to never being able to train a neural network on my laptop.

Fortunately, compatibility is not the only thing that PyTorch has going for it. After its release in 2017, it has been adopted by teams around the world in research and in business. It is extremely intuitive to use (for a high-level programming language targeted mostly at people with PhDs in Computer Science and Mathematics, admittedly). But seriously, it is designed with the structure of neural networks in mind, so the syntax and structure of your code can match the logical flow and linear algebra model that a neural network has conceptually.

A neural network with two hidden layers, as shown above, can be coded in PyTorch with less than 10 lines of code. Quite impressive.

All the squishification functions are built in to the PyTorch library, like

  • Sigmoid: S(x) = 1/(1+e^{-x}) ,
  • ReLU: f(x) = max(x,0) , and
  • Softmax: \sigma(z)_j = {e^{Z_j}}/{\sum_{k=1}^K*e^{Z_k}} .

On top of that, you can define a 756 bit input multi-dimensional matrix (AKA ‘Tensor‘) with one line of code.

Here’s the code for the above neural network that I created. It takes a 784-bit image file, pumps it through two hidden layers and then to a 10-bit output where each one of the output nodes represents a digit (0-9). The images that are in the training set are all images of handwritten numbers between zero and nine, so, when trained, this neural network should be able to identify the number that was written automatically.

Jupyter notebook code for a neural network with 784 input bits, two hidden layers and a 10 bit output

This few lines of code, executed on a server (or local host) produces the following output:

The neural network trying to classify a handwritten 5, notice that the probability distribution is fairly even, that’s because we haven’t trained the network yet.

See how cool that is!? Oh… right. The network seems to have no clue which number it is. That’s because all we’ve done so far is a feedforward operation on an untrained neural network with a bunch of random numbers as the weights and zeroes as the biases.

In order to make this neural network do anything useful, we have to train it, and that involves another step, back-propagation. I’ll cover that in my next blog. For now, we’re left with a useless random distribution of numbers and weights and no idea what our output should be, enjoy!