The platforms that these companies have created are essentially quasi economies. They comprise of owners of assets such as cars or houses that make up supply and people that want to use these assets in some way or form that make up demand. Prices are set, much like any other economy, through the interaction of demand and supply. Liquidity is the lifeblood of an economy, but these platforms aren’t born with liquidity. Liquidity is purchased through an elaborate and expensive scheme of subsidizing both the supply side and demand side of these markets, forcing fruitful economic exchange and changes in consumer behavior.

The upside of spending so much money for growth is the theory that once some liquidity is infused in these markets, network effects take over and attract more people to these marketplaces. These companies are brokers of the digital age, they own none of the wares that they ply. Revenue and profitability are generated by taking a cut of every economic transaction that these platforms enable and costs are kept to a minimum. Fixed costs mostly amount to the salaries of full-time employees and office leases and any other costs are highly variable, fixed assets that require debt financing are close to non-existent.

The biggest assets that these companies have, in addition to the technology behind their platforms and the brands that they have built, is a highly liquid platform where economic transactions take place. For these companies, the platforms or more importantly the liquidity in the platforms is the proverbial golden goose, the hard-fought and expensive gift that is meant to keep giving. The cost of building the platform and artificially creating liquidity was meant to be a one time expense, an exercise in deploying growth capital to build an economic flywheel that would generate revenue at very low marginal costs. An attempt at applying the software playbook of platformization and zero marginal cost growth to the real world, if you will.

The impact of COVID-19 has resulted in not only the actual economy grinding to a halt but a complete erosion in demand in many of these quasi economies. Food delivery seems to be a holdout, Uber Eats has seen a 30% jump in new sign-ups. Uber’s ride-sharing business, on the other hand, has seen a 70% decline in bookings on its platform in Seattle, a city especially hit hard by the virus. Airbnb has also been hit hard, bookings have fallen off a cliff with Europe seeing an 80% decline. The very same structures of these companies that allow them to bring costs down dramatically with a decline in demand present a unique set of challenges when there is an inevitable economic recovery.

Unlike a conventional asset-heavy company like a hotel that owns or leases all its supply, these platform companies don’t own any of the supply and as a result, can’t guarantee its existence on the platform when demand eventually recovers. The assets that make up the supply on the platform, while not on the balance sheet of these companies, reside on someone’s balance sheet. In the case of Uber, these assets are typically cars that individuals have bought with the explicit goal of using on the platform to make a living. In the case of Airbnb, these are homes bought by individuals to rent out to either make their primary living or to make supplemental income. None of these car loans or mortgages are going to pay themselves, they still need to be serviced. This has already set in motion an erosion of supply from these platforms.

In the case of Airbnb, there seems to be clear evidence of a flight of supply from the platform. Many major cities have seen a spike in suspiciously well done up fully furnished long term rentals suddenly appear onto the market. Uber and Lyft drivers are reporting total losses in income from the platform and while there is no clear evidence of them finding other employment (maybe it doesn’t exist) as in the case of Airbnb owners, it is inevitable that some of the supply disappears from Uber and Lyft’s platform too. The simultaneous drawdown of demand and supply is a death knell for these platforms, it can lead to a death spiral that is the end of the highly-priced liquidity that is so important to the economic flywheel that these companies have built.

While demand may take its own time to come back, supply can be preserved on these platforms, albeit at a cost. The cost is giving the actual asset owners economic relief - money that they would have otherwise made on these platforms that allow them to sit on the sidelines and wait until demand returns to these platforms. These platforms are currently looking to the government and lobbying it to rescue its distressed supply. However, any action from the government is a broad response geared towards rescuing the economy as a whole. No doubt, some of these actions will help these asset owners. But what these platforms need is a targeted response.

These platforms owe their liquidity to a dedicated set of individuals that make their primary living off these platforms. While these companies might claim otherwise, it is an open secret that these individuals make up the backbone of their supply. These individuals are hurting the most as a result of the current crisis. For most of them, these assets reside on their personal balance sheets, they can’t hide behind an LLC and shield themselves from filing for personal bankruptcies. Unfortunately for them, the government isn’t handing out stimulus checks based on how much outstanding debt each individual has. These individuals - the most loyal supply on these platforms - will be wiped out by this. These companies need to utilize the reams of data that they have to identify this supply and transfer money from their cash reserves to keep them afloat. This would preserve their shared economic future. In the absence of this targeted bailout, they risk killing the golden goose and ending up with empty marketplaces devoid of any liquidity.

Airbnb seems to have woken up to the fact that it needs to step in to preserve supply on the platform or risk losing liquidity forever. It has pledged $250 million to pay hosts for missed or canceled bookings. Furthermore, it has pledged an additional $10 million to help ‘super hosts’ on its platform - these are presumably the owners of the backbone of supply on its platform. Admittedly, Airbnb is the first to do this because, given the nature of the rental market, it will be much harder for Airbnb to regain lost supply than it will be for a platform like Uber.

It is, of course, unreasonable to expect these companies to own the entire risk on all of the supply on their platform in economic downturns. This would erode most of the advantages of their current business model and cost structures. A fine balance has to be struck - targeting the right supply with the right amount of money to preserve the right amount of liquidity. How they might arrive at this balance, is a question for another day. The fact remains, however, that they need to think about and act on a targetted bailout of supply.

These companies though, are here to stay. They still have tons of unrestricted cash on their balance sheets - Uber has $10billion and Airbnb has $3 billion. They will surely weather this crisis that, from the looks of it, is far from over. The crisis, however, has exposed more chinks in the armors of these companies. The long-touted cost advantages over conventional asset-heavy businesses that have led to almost SaaS like multiples on valuations are looking, if not already, all the more suspect.

Much like actual economies have central banks that look after the health of the economy, provide liquidity and backstop debt to prevent a complete implosion in troubled times, these platform businesses will have to step in to do the same for their quasi economies. Fortunately for central banks, they have a whole bunch of tricks up their sleeve to generate the money to help during times of crisis, one of which is literally printing more money (The United States Federal Reserve is the only central bank that can do this without much consequence). Unfortunately, platform companies aren’t as fortunate. Helping to preserve liquidity comes at the cost of reduced profitability. This only means one thing for these companies, most of which have never been profitable ever - the long road to profitability just got a little longer.

These platforms have used technology to upend the incumbent taxi industry and have successfully rewritten the rules of urban commuting. Firstly, by not owning any of the cars on their platforms they reduce their operating costs significantly and brand themselves as technology companies. This asset-light model that they use has allowed them to scale rapidly and enter multiple cities in quick succession. Secondly, they take pride in using algorithms that use price as a lever - ‘surge pricing’ in Uber-speak - to efficiently match supply and demand. This ensures that passengers have access to rides in locations or at times that are unfavourable to the drivers.

These multi-side sharing economy platforms have not only transformed the taxi business, but have been making waves in other industries as well. Airbnb is a sharing economy platform that operates in the hospitality space. TaskRabbit matches people willing to perform tasks to people who want these tasks done on demand. Kickstarter, a fundraising platform, matches people looking for funds to execute projects with people willing to back these projects.

The widespread impact that these platforms have had and the scale at which they operate now have made them, increasingly, subject to regulatory scrutiny. This has led to people asking questions about the societal impact of these platforms. With respect to ride sharing, local governments of large urban areas that are plagued by traffic congestion are interested in whether the introduction of these platforms contributes to an increase in traffic congestion.

There are two differing perspectives on this issue; proponents of ride-sharing services argue that the low cost of these services and ease of access will provide a convenient alternative to driving alone and thus reduce traffic congestion and car ownership in the long run. Critics of ride-sharing services, on the other hand, argue that the low cost of ride-sharing services will divert trips from other modes of public transit or non-motorized transport to these services and perhaps, introduce new trips altogether.

Anecdotally, there is city specific evidence that ride sharing platforms do, in fact, contribute to an increase in traffic congestion. However, this is far from conclusive. Anecdotal evidence is often unreliable because it is affected by false causality and the availability bias. For example, if you lived in a city for a period of time, various things that affect traffic congestion could be happening concurrently. If you suffer from the availability bias - most of us do - you might attribute the total effect of all the factors affecting traffic congestion to the introduction of ride sharing, just because it is in the news and people are talking about it. There could, however, be a correlation between the introduction of ride sharing and a rise or decrease in traffic congestion. But correlation does not imply causation.

Econometrics without the equations

The best shot are a rigorous answer to this question of isolating the causal impact of ride sharing platforms is by using data of traffic congestion across various urban areas in the United States cities. This brings us into the realm of experimental design and econometrics.

Breaking this down, it’s helpful to first identify what conditions are needed for causal inference in an contrived experimental setting. The key element in any study that seeks to identify causal effects is random assignment of the treatment and control. For example, in the case where the efficacy of a drug is being tested, for accurate causal inference the drug and the placebo have to be given at random to people. If the drug were given only to people who had a disease and the placebo to another group with different characteristics, it would mean that the actual effect of the drug tested against the control would be skewed. In addition to this, an accurate control group that the counterfactual is built on is also required, this is also taken care of by the random assignment. The assumption that is being made here in the case of the drug is that the control group would react in a similar manner to the treatment in the event that their roles were switched.

In the case of real world examples like estimating the impact of ride sharing on traffic congestion, it’s impossible to construct such experiments. Does this mean that it is impossible to perform causal inference? Fortunately not. What can be exploited to draw causal inference is what is known as a natural experiment, these are naturally occurring events that mimic the characteristics of a contrived experimental setup with random assignment and an accurate control group on which the counterfactual is based. In the case of ride sharing, specifically the entry of the low cost ride sharing product from Uber, Uber X, there is an excellent natural experiment that lends itself to causal inference.

The primary method of exploiting natural experiments to draw causal inference is known as the ‘difference-in-difference’ technique in Econometrics lingo. It rests on the assumption that in the absence of an intervention on the treated group, the treatment group and the control group would behave in the same manner - known commonly as the parallel trends assumption. If this is the case, post the intervention, the causal effect of the treatment is simply the difference between - what would have been in the absence of the treatment, the counterfactual and what happened after the treatment.

The entry of Uber X into various urban centers in America is conveniently varied across time, which gives a natural experiment type setting to investigate the causal impact of Uber on traffic congestion. The data that enables this study comes from various sources; firstly, traffic congestion data is obtained from the Urban Congestion Report (UCR), secondly, the dates that Uber entered various cities are captured from blog posts that Uber puts out before they launch in any new city. Finally, data on other factors that might affect traffic congestion such as population and GDP are obtained from the Bureau of Economic Analysis.

This estimation of the effect of the entry Uber X on traffic congestion is based on a regression design of the ‘difference-in-difference’ estimation technique. The basic idea that underpins this design is identical to what is described above. It requires that the entry of Uber is random with respect to traffic congestion and that the effect of the entry of Uber on traffic congestion is the same across all cities. If these were not met, it would violate the random assignment and accurate control assumptions and not lead to causal inference. The advantage of this design is the fact that other factors that affect traffic congestion can be controlled for, leading to a more accurate estimation.

Results

The rigor of Econometrics requires that results are accepted as robust only if they conform to a 95% level of confidence - that is, the specific results occur only 5% of the time due to random chance. Even with great data and a good Econometric design, this is hard to achieve. Unfortunately, as is the case with many Econometric studies, in this case, it turned out that there was small increase in traffic congestion that could be causally attributed to the introduction of Uber X, but it lacks the statistical significance to make a robust claim.

Potential reasons for these results

• Lack of Data

The data that this study is based on is limited, in that it doesn’t account for some large cities that could drive the results one way or another. New York, a large urban center with heavy usage of Uber is not represented in this study. Including these results could perhaps, tell a different story or add statistical significance to these results.

• Time effects, the effect that it takes for consumer behaviour to change is longer

There is a time lag between widespread adoption of a new product of service and the launch of a product or service. The behaviour of people needs to change, less people would likely be using Uber X right when they launched as opposed to a couple of years in. These results could be explained by the fact that the effect of Uber X has not had time to fully manifest itself. The story could very well be different given data over a longer timeframe.

• Carpooling services - Uber Pool and Lyft Line

A significant limitation of this study is that it considers only the entry of Uber X, a product that matches one rider to one driver. The more interesting case, however, is the concept of carpooling that these services enable. Products like Uber Pool and Lyft Line are where the greatest gains from a reduction of traffic congestion stand point may be, these services group users travelling in the same direction and match one driver to multiple users. These services, however, are new and there isn’t sufficient data post their introduction to run a good difference-in-difference study because of the time effect of the intervention mentioned above.

The Path Ahead

It is clear that irrespective of their impact on traffic congestion, ride sharing services are here to stay. This fact alone warrants further investigation into its societal impact in a data driven manner. Unfortunately, in the case of ride sharing the most valuable data is with the ride sharing companies themselves. This data has information on a fine granular level that could unearth the answers about the effect of ride sharing on traffic congestion and other societal issues. Local governments have begun requesting this data - it remains to be seen what the analysis of this data points to. In addition to this, adding more controls to the difference-in-difference design could produce better results, for example, the intensity of public transportation in a given urban area, if available, will make a very good control.

On a final, albeit tangential, note. We are currently in the initial phases of a revolution in transportation, self-driving cars are going to be a reality much sooner than most of us imagine. Ride sharing companies are investing heavily in this technology, the real gains from algorithmically managed ride sharing services might actually be in the future when self driving cars are a reality and commonly used. The widespread adoption of this technology could lead to efficiency gains through efficient allocation that alleviates traffic congestion in most urban areas. If this becomes a reality, it could be that app based ride sharing, while currently contributing to traffic congestion could eventually be the solution to congested cities.

Here is a link to the actual paper.

At the end of the last post I made a case of why Econometric techniques could come in handy when A/B testing can’t be carried out. This post will deal with an explanation of what these Econometric techniques for causal inference are.

These techniques and examples may seem very specific to just questions that economists ask. But, if you really think about it from a statistical perspective these techniques to establish causal inference are broadly generalizable to other contexts as well, such as in a product setting where A/B testing isn’t feasible.

Difference in Difference

Among other things, the primary condition that is required for causal inference is an accurate counterfactual that can act as the control. A counterfactual is the what if condition - it answers the question of what would have happened if the treatment had not occurred. In an experimental setting the control group performs this role.

The difference in difference method exploits what economists call a ‘natural experiment’ to construct this counterfactual. This works by identifying two similar groups, one that is randomly affected by the treatment and one that is not affected by the treatment. The group that is not affected by the treatment serves as the counterfactual for the group that is affected by the treatment. The difference between the effect on the group affected by the treatment and what would have happened if the same group wasn’t affected by the treatment is the causal effect of the treatment. The ‘what if’ scenario is constructed using the counterfactual.

A classic case where this is used in economics is in evaluating the causal effect of policy changes. It is often the case that policies are enacted on a state level, this results in the state where the policy change is enacted acting as the treatment and the neighbouring state or a demographically similar state acting as the control.

This can visually be illustrated easily using the following picture:

Source: Mostly Harmless Econometrics An Empiricist’s Companion by Joshua Angrist and Jorn-Steffen Pischke

Given the fact that this is an experiment that has not been constructed, it makes one big assumption. The fact that both groups, in the absence of the random treatment, would behave in the same manner over time. This is known as the ‘parallel trends’ assumption. While exploiting a natural experiment to construct a difference in difference study, it is crucial to try and validate this assumption.

In addition to this, the difference in difference can also be modeled using a regression framework. In this case, the independent variable is the metric of interest and the dependent variable is a dummy variable that takes on a 0 for when the treatment doesn’t exist and 1 for when the treatment exists. The advantage of this design is two-fold. First, this allows for multiple states or groups that serve as both control and treatment, as long as the introduction of the treatment is randomly varied across time. Second, it allows for the controlling of confounding variables. Confounding variables are other things independent of the treatment that can affect the dependent variable. Controlling for these factors strengthens the statistical significance of the causal interpretation.

Instrumental Variables

In a non experimental setting an important consideration when it comes to causal inference is the omitted variable bias. Omitted variables are variables that a regression setup does not account for that are captured by the error term. This breaks down a key assumption of causal inference that the independent variables and the error term must be uncorrelated.

A classic example from economics is estimating the effect of schooling on wages. In addition to schooling, wages are also influenced by the ability of an individual which is likely correlated with the amount of schooling an individual receives. Students with higher ability who do better at school tend to stay longer in school. If wages are regressed on schooling, the error term would likely represent ability and cause the estimate of the effect of schooling on wages to be off. In other words, the error term will be correlated with the dependent variable.

The tool that economists use to correct this is called the Instrumental Variable, sometimes just referred to as the Instrument. The idea is pretty simple: it requires the identification of a dependent variable that is correlated with the original dependent variable of interest but not the error terms. This new variable acts as a proxy for the original dependent variable, this allows for causal inference and a solution to the omitted variable bias problem.

In a treatment and control setting such as the cases observed in an A/B test, the instrumental variable comes in handy in a situation where the treatment is not randomly assigned. Non random assignment of a treatment causes selection bias, which means that the estimation of the causal effect could either be overstated or understated.

An instrumental variable is valid if it satisfies three conditions. First, the instrument itself is randomly assigned; i.e., there is no selection bias in the case of the instrument. Second, it affects the actual outcome that is being measured via the treatment of interest. Third, and probably the most difficult assumption to justify, it affects the outcome that is being measured only via the original treatment and in no other way. This final assumption is known as the exclusion restriction.

The important question here, however, is where to find these instruments. Finding a good instrument requires some creativity and a good understanding of the problem at hand. A good example of instrumental variable strategy is from the 1991 paper by Ashenfelter and Krueger. They use the season of birth of an individual, which is likely randomly assigned, combined with compulsory schooling laws as an instrument for constructing treatment and control for an extra year of schooling. They were able to use this because the season of birth varied, by a year, when an individual on the margin of the enrollment deadline could drop out of school.

Regression Discontinuity

There are many instances of deterministic rules based on arbitrary cutoff criterion that govern things that we do on a daily basis. For example, if the cut off score on a test for admission into an exclusive school is 80 points, there could be people who got 79 points and people who got 81 points. And if you think about it, these two groups of people may not be all that different, these scores that they got around the cut off could be based on random luck on the day of the test. This situation of individuals in the neighborhood of the cut off can be interpreted as having been assigned to the treatment group or control group in a random manner.

In order to draw causal inference from these kinds of situations a sharp regression discontinuity approach is used. A regression line is fit that measures the outcomes of both groups, the difference in the intercept of the regression line on the margin of this cut off is interpreted as the causal effect of the treatment. Visually this effect can be seen from the following image. The difference in the intercept at the dotted line is the causal effect of the treatment.

Source: Mostly Harmless Econometrics An Empiricist’s Companion by Joshua Angrist and Jorn-Steffen Pischke

The primary assumptions in this case are twofold. First, the status of the treatment is a deterministic function of the cut off. This means that there is no way to get around the cutoff, it is the sole determinant of access to the treatment. Second, the treatment is discontinuous. This means that even if an individual is arbitrarily close to the cut off on either side, the individual will clearly fall into one group or the other.

Since things aren’t perfect, especially so in the world of causal inference, cases exist where there is no sharp regression discontinuity. This happens because people who get selected in either the treatment or control group around the neighborhood of the cutoff might be non compliant and find a way to move to the other group. As a result, the treatment and control are not cleanly assigned to two groups as in the case of the sharp regression discontinuity. However, what does happen and can be used to draw causal inference is the fact that the probability of being treated is higher on one side of the cut off and lower on the other side of the cut off. Economists call this situation a fuzzy regression discontinuity.

To estimate the causal effect of the treatment an instrumental variable design using two stage least squares is used. The instrument in this case is a dummy variable (0 or 1) assigned to each individual according to the actual cutoff. The estimation of the causal effect is in two stages. The first stage regresses the actual treatment that ends up being assigned after non compliance on the dummy variable assigned by the cutoff. This estimates, in loose terms, the probability of receiving the treatment given the actual cutoff. The second stage regresses the outcome variable of interest on this predicted value of the the intensity of the treatment. The coefficient on this is interpreted as the causal effect of the treatment.

A good question to ask is why the cutoff dummy is a valid instrument for the treatment. This can be argued based on two facts about the cutoff that satisfy the selection conditions of a valid instrument. First, the cutoff is a randomly assigned or arbitrary, there is no selection problem in the assignment of the cut off. Second, the cutoff as an instrument also satisfies the exclusion restriction; i.e., the only way in which it affects the outcome variable of interest is via the treatment.

If you’re interested and getting deeper into the mathematics of how these ideas work, the go to book is Mostly Harmless Econometrics An Empiricist’s Companion by Joshua Angrist and Jorn-Steffen Pischke. For a more math-light treatment of the same ideas, the book to refer is Mastering ‘Metrics: The Path from Cause to Effect by the same authors.

A/B testing is one of the primary tests tech companies use before they roll out new features or changes to their products. If you pay close attention you might be able to point out that you’re part of an A/B test by noticing a subtle change on an app or website that you use. At its core an A/B test is nothing but a controlled experiment. A fraction of users are randomly assigned to a control and another fraction randomly assigned to a treatment. The control is the status quo, where there is no change in the product. The treatment is the case where the users see a change in the product. The difference in the metric of interest, known more formally as the outcome variable, between the control group and the treatment group is the causal effect of the change.

The goal of any A/B test is to quantitatively ascertain, with statistical significance, the causal effect of a change measured by some pre defined metric - known in some cases as a KPI (Key Performance Indicator). For example, a company may be interested in quantifying the change in click through rate by altering the position of a button on its website or app. Companies also test more complex things like sign-up flows, landing page designs. etc with a focus on tracking conversion via a specific pre defined metric.

The advantage of an A/B test is the fact that the design allows for causal identification. Causal identification is important because correlation doesn’t imply causation. For example, if a company were to simply make a change on a product and after a certain amount of time look at the value of the pre defined metric, what they will be picking out is a correlation between the the change and the predefined metric. The reason for this is the fact that multiple things are likely happening at the same time that could affect the metric. However, the strategy of control and treatment uses the control as a benchmark to estimate the effect of this change on the treatment.

From the perspective of identifying the causal effect of a change in a product setting, the A/B test is the gold standard. Tech companies typically have users in the millions, therefore the sample sizes for these tests are easily large enough to get an answer with a high level of statistical significance. In addition to this, random assignment is easy - this prevents selection bias and allows for true causal inference.

A/B testing, while admittedly the gold standard, isn’t always something that is feasible. There are ethical concerns that come up when running an experiment and more pertinent to the case of a product company is actually alienating users through these experiments. The other case where A/B tests can’t help are looking at something retrospectively or trying to see if something out of the control of the product team affects usage - you can’t run experiments on things that you can’t control. This begs the question: Are there other ways to ascertain causal inference when experiments are not feasible?

The answer to this question lies in the Econometricians tool box. Econometrics as a field is primarily focused on causal inference and a lot of the questions that economists ask can’t be tested by running an experiment. Experiments run in the real world are very expensive and if you’re asking questions like - what the causal effect of an extra year of schooling is on wage earning - there is no good way to construct an experiment to test this. Randomly assigning students who want to complete school to a control group where they are forced out of school one year before the treatment group is simply infeasible.

Econometricians, however, have designed tools to answer these questions. The successful use of these tools is by using what are known as ‘natural experiments’ to approximate conditions that would have resulted from designing and running experiments.

In the schooling example that I cited earlier economists Angrist and Krueger used a creative method where they used the season of a child’s birth and state laws that mandate compulsory school attendance to identify the causal effect of an extra year of schooling. The season of a child’s birth influences when a child can start school which in turn, many years down the line, alters the date that the same child can drop out of school by a year. This provides them with the control group that were born in the same year, within months of each other, but could drop out one year before the treatment group. An impossible condition to achieve in an experimental setting.

These methods are more widely applicable to answer other questions around causal inference where A/B testing is infeasible. These methods, while not as robust as an experiment, do a great job of identifying causal effects. They require a bit of creativity and some simplifying assumptions, but they work when an experiment is infeasible.

Part 2 of this post focuses on an explanation of some of these methods.

References

Angrist, J. D., and A. B. Krueger. “Does Compulsory School Attendance Affect Schooling and Earnings?” The Quarterly Journal of Economics, vol. 106, no. 4, Jan. 1991, pp. 979–1014., doi:10.2307/2937954.

Angrist, Joshua D., and Pischke Jörn-Steffen. Mostly Harmless Econometrics: an Empiricist’s Companion. Princeton University Press, 2009.

Airbnb has emerged as a significant threat to the incumbent hotel industry, which is tightly regulated. One study has shown that the introduction of Airbnb has estimated that the causal impact of the introduction of Airbnb on hotel revenue is in the range of 8-10%. In addition to this, there are concerns about the impact that Airbnb has on rental markets which are already very tight in large cities.

These concerns, public demand as well as demand from the incumbent hotel industry has led local governments globally to try and regulate Airbnb. These regulations have taken various forms, some governments require special licensing for properties listed on Airbnb others have put in place rules that prevent apartments from being rented out for extended periods of time arguing that this puts added pressures on rental markets.

In light of these facts, an interesting question to answer is the extent to which Airbnb is a pure peer-to-peer sharing economy market place where excess capacity is shared or if it is, as critics point out, a platform the allows businesses to operate on it. Whether the critics are right in demanding the sort of regulation that could impede Airbnb’s growth is another debate in itself, one that I will not get into. Enough talking, I will let the data speak for now.

I use data from San Francisco that is available on insideairbnb.com, a website that regularly scrapes the Airbnb website to collect data on listings to construct visualizations to try and answer this question. The data is fairly detailed and includes the characteristics of all the listings on the platform including free text reviews of properties. I used Tableau to build the visualizations along with Python and good old Excel to wrangle the data. I built these visualizations for a data visualization class that I took, the goal was to incorporate principles from data visualization to tell rich multivariate stories in an easily accessible manner.

The visualization above tells a very interesting story. It is quickly apparent that about 14% of hosts on Airbnb control about 31% of properties listed in San Francisco. These hosts with multiple properties listed are more likely to be using Airbnb as a business platform as opposed to sharing excess capacity. In addition to this, the color encoding shows that there is a difference in the ratings that hosts with multiple properties get versus hosts with a single property listed.

The results for this are revealing, the personal connect that an individual host likely has as opposed to a professional Airbnb host results in a variation of about 3 points in the average review received. While on a 100 point scale the difference seems small, something to put this in perspective is the fact that a significantly large percentage of Airbnb reviews are greater than 90 out of a 100.

Another way to segregate hosts that share excess capacity from hosts that use Airbnb to permanently list properties is by investigating the number of days a property is listed on the platform in a year.

This visualization reveals a ‘U shaped’ distribution, there are a large number of properties that are listed for short term rentals, less than 75 days in a year. And on the other end a large number of properties that are listed for greater than 270 days in year. In terms of the review scores, again encoded using color, it shows that the properties listed for shorter durations, where presumably hosts personally manage the experience, have on average better ratings than properties listed for long periods of time.

As a strong believer in the promise of the sharing economy, I am of the opinion that while regulation of these platforms is required, regulation that is reactionary to incumbent business interests and not data driven is harmful. Policymakers should leverage data and arrive at data driven conclusions that inform regulatory action. With respect to Airbnb, the debate is far from settled. Questions such as the causal impact that Airbnb listings have on rental markets are still to be answered.

Two things about these ratings jump out at me. First, we don’t put too much thought into assigning these ratings. Second, we almost always end up assigning a rating on the higher end of the scale that is available.

These ratings, which I guarantee we don’t think too much about, are in fact, an important metric that the governance of these these platforms are based on. For example, in the case of a hotel, the quality of the room is directly managed by the hotel management. However, in the case of an Airbnb the strongest indicator of the quality of the room or apartment is an average of this arbitrary rating that we assign post usage. The logic behind these ratings is that on an aggregate level by being revealing about the quality of service they serve as one of the tools to help build the all important ‘trust’ factor in the sharing economy.

To the credit of these sharing economy companies, they have been trying to improve this system and build better ways of capturing these ratings. Uber has us input the reason behind a specific rating, Airbnb has us rate the service on a variety of parameters. However, while these additional data points help build a richer picture of the ‘star-rating’, it still suffers from the fact that there is no good way to judge how to rate the level of service and most of us end up rating these services on the higher end of the scale that is available.

This could mean two things, first, people are very satisfied with the service or second, like discussed before people by lacking a good mental model of what these ratings mean are predisposed to rating the service on the higher end of the scale.

Airbnb fortunately collects another form of data on the users rating, a free text review that users can give to a property that they stayed at. This presents an interesting case for using NLP techniques to try to analyze the average sentiment of reviews for these properties and seeing how they map to the numeric rating that users assign. It will show if users are revealing things in free text reviews that they don’t know how to map to a numeric rating.

What is Sentiment Analysis?

The sentiment of a bunch of free text is a score assigned between -1 and 1, where -1 is most negative and 1 indicates the most positive a sentence can be. This is calculated by looking at each of the adjectives in the sentence individually and assigning a score based on the average of the polarity of each of the words.

This method brings up obvious questions about how context is handled, there are more advanced tools and methods used that handle context, the tool used for this analysis ignores context. This is a limitation of the sentiment that is calculated for these reviews. But nonetheless, a visualization using a scatter plot of the results are blow.

The first thing that is apparent is that the distribution of ratings of listings are clustered in the top of the visualization, confirming the assumption that I made earlier that we tend to give pretty high ratings. In addition to this, it is also apparent that the polarity of sentiment is clustered around 0.4, which is moderately positive even when ratings are very high. Interestingly, it can be seen for properties with a rating of 100 that the polarity is across the spectrum, again indicating that the ratings alone do not capture the full sentiment of an experience with Airbnb.

The question that prompted the idea was simple, why should everyone that lives in an apartment complex own a vacuum cleaner ? Or anything for that matter, that is used rather infrequently. I was fully sold on the idea that access is greater than ownership. The idea for the product was to build an ad supported, hyperlocal neighborhood marketplace where users could list these products and rent, buy/sell or share them. We called it neighbourbase.com.

Long story short, it didn’t succeed for a variety of reasons. I went on to grad school, learnt a whole bunch more and I’m glad for that. But I learnt a lot through the process of trying to build Neighbourbase. The kind of lessons that you read about a lot, but only fully understand when you get your hands dirty. This post is a summary of the things I learnt, organized in no particular order of importance.

You will not build a perfect product the first time around - Don’t even try.

A good approach to building any tech product is to build a functional yet scrappy quick first iteration of the product with a minimum set of features that make it usable, give the product to users and iterate based on feedback. Most successful startups start this way, don’t believe me, check this out. Our approach based on heavy doses of naivete was to build a fully functional product, with all the features that we envisioned (This list kept growing) combined with a perfect UI and UX as our first iteration of the product.

In retrospect, this approach was problematic for two reasons. First, the initial set of user stories for any product are a set of untested assumptions about what you think the user might want. The only way to validate these assumptions in quick time at a low cost is to build a first iteration, get this in the hands of real users and test these assumptions. This will help you figure out what users like, what they don’t and what the next iteration should look like. Second, isolated from any real feedback, the list of features that you can imagine can keep growing. Soon you find yourself building things that were never in your original plan and users might never really care about or want.

Companies don’t run on air - Think about how you want to make money.

In terms of thinking of a business model, we decided that we’d have a free ad-supported tier as well as a premium tier that allowed apartment complexes to pay a monthly subscription for a private sharing network. The problem with building an ad supported website is that, you’re only attractive to advertisers when you’re at scale. In the case of a platform product this is compounded by the fact that achieving scale is tricky because you need to grow both sides of the market. Borrowers will be on the platform if there are lenders and vice versa. Advertising isn’t going to make you money from day one, so if the ad supported business model is what you’re going for, you need to make sure you have enough runway until you’re at scale

In getting people to pay for a product, you need to reduce the amount of friction in the transaction.For our premium tier, the sign up process was complicated and for a product that a user didn’t yet know could be useful, I doubt they would have taken the trouble.

Think about Data from Day 1

Any multi side platform will generate a lot of data, there is data about listings, user behaviour etc, and in thinking about your product and strategy it’s crucial to also think about ways that you want to capture this data and put it to use creatively.

We were so focused on building the perfect product that we paid little attention to these aspects of strategy. Building a product always has to have a data first strategy, there is the user interface for that the users get to see and on the back end what is equally important is smart dashboards to get a snapshot of the data that is being collected.

The next two lessons are more people specific.

People you work with may not be as excited about your idea as you are

When you come up with an idea for a product, it is going to end up that you are more excited about it that than the people you work with. I think a large part of success depends on how good you are at getting a team motivated and as excited about the idea as you are. The difference in energy, if you don’t, is palpable.

Experience matters a great deal

Finally, this was something that was humbling on a personal level and I think a significant take away for me personally. It’s easy to think you know a lot and dive right into doing things, however, there are a lot of things that you only learn through experience. Trying to build a product and start a company is an ambitious goal, the odds of success are certainly higher with some experience under your belt.