Everything you need to know about Santa Clara County's housing crisis: the causes, the mistakes,the way out.

In his exclusive report for Opportunity Now, Randal O'Toole of the Thoreau Institute explores how New Urbanist thinking and misguided anti-market policies created one of the most unaffordable housing markets in the world--and how to fix it.

High-density, mid-rise and high-rise housing is more costly to build and most homebuyers and renters consider it less desirable than single-family homes. Yet San Jose and other cities in Silicon Valley are eager to promote the construction of such high-density housing projects. San Francisco Senator Scott Wiener has introduced bills in the state legislature that would abolish any limits to such projects in major transit corridors which, maps indicate, pretty much includes all of San Francisco, Oakland, Berkeley, and other Bay Area cities.

There are at least two reasons why government officials favor high-density housing projects. The first has to do with tax policy: a fifty-unit condominium will generate less taxes than fifty single-family homes, but the taxes generated per acre will be much higher.

A more important reason has to do with the sixty-year-old debate over urban sprawl and the laws that were passed in response to that debate. Unless these laws are repealed, Silicon Valley effectively has no choice but to build high-density housing projects even though those projects mean housing will simultaneously be less affordable and less desirable to potential residents

The Debate over Annexation

Though San Jose is California’s oldest city, it was quickly eclipsed by San Francisco, which by 1950 had eight times as many people while Oakland had four times as many. In that year, the San Jose city council hired a former automobile executive named A.P. “Dutch” Hamann to be city manager. Some say Hamann set a goal of making San Jose the biggest city in the Bay Area, a goal that was eventually reached though not in his lifetime.

The first thing he did was to build a sewage treatment plan that had much more capacity than San Jose needed. Then he went on an annexation spree, growing the city from 17 to 136 square miles by persuading property owners to join the city so they could receive (and help pay for) sewer and other city services.

The annexations were mostly voluntary but became controversial because San Jose’s aggressive annexation led it to take land that other cities believed should be within their spheres of influence. The other cities complained to the legislature and in 1963 the legislature passed a law creating Local Area Formation Commissions (LAFCos) in each county. Commissions consisting of representatives of various local governments had the power to approve or reject annexations, the incorporation of new cities, and the creation of new service districts such as sewer and water districts.

Initially, LAFCos did not slow down San Jose’s growth. In fact, the city did more annexations in the 1960s than the 1950s. But the legislature’s creation of LAFCos unintentionally created the vehicle that slowed San Jose’s growth in the 1970s.

The Debate over Urban Sprawl

At the same time the legislature was debating annexation, some San Jose residents were beginning to raise concerns about urban sprawl. The late 1950s and early 1960s saw a backlash against the suburbs with publication of books like The Crack in the Picture Window (1956) by John Keats and the popularization of songs like 1962’s Little Boxes (which, ironically considering the song’s references to “doctors and lawyers and business executives,” was inspired by a then-affordable working-class neighborhood in Daly City). A growing environmental movement worried that sprawl was destroying valuable farmlands and open space. In addition, taxpayers worried that they were being forced to subsidize growth.

In 1962, Virginia Shaffer became the first dissenter to A.P. Hamann’s vision to be elected to the San Jose city council. The 1968 election saw advocates of slower growth win the majority of the council. When they took their seats in 1969, Hamann gracefully resigned and later died in the 1977 Tenerife air disaster.

Before the 1963 law creating LAFCos, San Jose had no authority over land outside of its borders. If the city council imposed strict growth limits, such as the growth controls adopted by Petaluma in 1972, then growth would simply take place outside of the city limits. This created a tension: cities wanted the tax revenues that came from growth, but the city council didn’t want the environmental impacts of that growth.

LAFCos enabled them to resolve that tension. In 1974, the region drew an urban-growth boundary around all of the cities in Silicon Valley, enforceable via the Santa Clara LAFCo. That boundary forced most or all new growth, and the taxes it generated, to take place within the borders of San Jose and other cities.

In 1970, the legislature had passed the California Environmental Quality Act, which required preparation of draft and final environmental impact reports, with public review, for any action that could have a significant environmental impact. A subsequent court decision found that any boundary changes approved by a LAFCo would require such an environmental impact report, and other legal decisions made the preparation of such reports extremely costly. The cities weren’t interested in paying this cost, so few expansions to the boundary were ever considered.

One such proposed expansion was of one of the last areas annexed into San Jose by Hamann, Coyote Valley, an agricultural area in the southern part of the city. Even though it was in the city limits, the city excluded it from the urban-growth boundary in the 1974 General Plan, calling the valley an “urban reserve” that would be added to the boundary when necessary and when “the City’s fiscal condition is stable, predictable and adequate” to support development of the valley.

Nearly three decades later, a developer proposed to build homes in Coyote Valley and spent $17 million on the draft environmental impact report that would be necessary to expand the urban-growth boundary. The Sierra Club announced it would support the expansion only if the developer put up $100 million to preserve land from development elsewhere. The developer couldn’t afford that and, since the Sierra Club seemed to have the ears of the San Jose city council, it gave up and never wrote a final environmental impact report.

The Rise of New Urbanism

In short, Silicon Valley has seen nearly five decades of growth with no increase in the amount of land available for that growth. This has been justified by recent planning movement called the New Urbanism that argued cities would be better off growing “up, not out,” that is, growing denser rather than sprawling across the landscape. Such density was supposed to be particularly important in transit corridors as the construction of mixed-use (residential and commercial) developments near transit stations would allow people to walk to shops and take transit to work, thus saving them the cost (including environmental impacts) of driving an automobile.

New Urbanists would ask people, “Would you rather live in a compact neighborhood where you can walk to a Starbucks and a grocery store and take a light-rail train to work, or a single-family home in a suburban neighborhood where you would have to drive everywhere you go?” Many people would answer the former, leading New Urbanists to conclude that there was a “pent-up demand” for high-density housing.

In fact, there is and was no such pent-up demand, which New Urbanists would have learned if they had asked the question honestly: “Would you rather spend $400,000 for a 1,100-square-foot condominium within walking distance of a high-priced, limited-selection grocery store or $250,000 for a 2,200-square-foot, four-bedroom, two-and-one-half bath home within easy driving distance of three major supermarket competing for your business on the bases of both price and selection?”

Despite this, urban plans that called for denser housing and limits on low-density development came to be known as “smart growth.” Anyone who favored low-density development was castigated as favoring “dumb growth.”

High housing prices were the unforeseen result of San Jose’s slow-growth plan. Any economist would predict that limiting the supply of land for development would increase housing prices, but urban planners insist to this day that Silicon Valley’s high housing prices are a function solely of demand and have nothing to do with the growth boundary.

A standard measure of housing affordability is the value-to-income ratio, that is, median home prices divided by median family incomes. Under standard mortgage rules that limit loans to 30 years and limit spending on housing to 30 percent of a homebuyer’s income, a family can easily afford a house that costs three times their income. Depending on the interest rate, it is more difficult to buy a house that costs four times the family’s income. A family cannot get a mortgage for a house that costs five times its income unless it pays for most of that house with a hefty down payment.

In 1969, the median price of homes in the San Jose urban area was 2.2 times median family incomes, which was actually more affordable than the national average of 2.4. The establishment of the urban-growth boundary drove prices to more than 4.0 times incomes in 1979 and more than 5.3 times incomes in 1989. By 2005, when the city was considering expanding the growth boundary to include Coyote Valley, they were nearly 7.6 times incomes. 

Although prices fell after 2006, they have since recovered and today are more than 7.6 times incomes.

Density advocates say that the solution is to build more high-density housing. This will increase the housing supply and make housing more affordable. The problem with that is that mid-rise (four- to six-story) and high-rise (seven stories or more) housing projects cost a lot more per square foot than single-family homes. Such projects require a lot more structural steel and concrete. They also require large common areas including hallways, recreation areas, and in some cases parking structures whose costs must be counted against the residential units. They also require elevators, whose cost per housing unit can be especially high in mid-rise developments.

In testimony before the San Francisco Bay Area Metropolitan Transportation Commission, Bay Area developer Nicholas Arenson reported that mid-rise housing cost three to four times as much to build per square foot as low-rise, while high-rise cost 5.5 to 7.5 times as much per square foot as single-family homes. Contrary to claims that there is a pent-up demand for such housing, he also noted that such high-density housing “sells at a discount” to single-family dwellings.

The Los Angelization of San Jose

In 2001, former San Jose Mayor Janet Gray Hayes, who was on the city council when it adopted the 1974 growth boundary, called proposals to expand the urban-growth boundary “the Los Angelization of San Jose.” The Sierra Club has called Los Angeles “the epitome of sprawl” and considers its growth pattern as something to be avoided.

In fact, close scrutiny reveals that Los Angeles is the epitome of smart growth, not of sprawl. The Los Angeles urban area, which includes Pasadena, Anaheim, and other parts of Orange County, is the densest urban area in the United States, with more than 7,200 people per square mile. The New York urban area, which includes northern New Jersey, most of Long Island, and parts of Westchester County, has less than 5,500 people per square mile.

Rather than preventing the Los Angelization of San Jose, the smart-growth plans for the region ensured it. In 1970, the San Jose urban area had less than 3,700 people per square mile while the San Francisco-Oakland urban area had under 4,400 people per square mile and the Los Angeles region, at 5,300 per square mile, was less dense than the New York urban area. Since then, urban-growth boundaries and similar policies have boosted the density of San Francisco-Oakland to more than 6,800 people per square mile and San Jose urban area to more than 6,300 people per square mile, making them the second- and third-densest urban areas in the United States.

There is no evidence that this density has made housing more affordable. Instead, affordability has declined as these and other regions became denser. On a nationwide basis, there is a strong inverse correlation between densities and price-to-income ratios: denser regions are less affordable. Among the nation’s sixty largest urban areas, the correlation coefficient between density and the price-to-income ratio is 0.82, where 1.0 is perfectly correlated and 0.0 is totally random.

In addition to the higher cost of building mid-rise and high-rise housing, urban-growth boundaries increase land costs. A 2017 study published by the National Bureau of Economic Research found that the average cost of an acre of land in the San Jose urban area was $2.3 million, compared with costs of around $300,000 per acre in fast-growing regions with no growth boundaries such as Atlanta, Dallas, and Houston.

The growth boundaries also allow cities to impose huge impact fees and an onerous approval process on home construction since they know developers can’t go elsewhere. The highest impact fees are in Fremont, which charges $75,000 per dwelling unit, but fees are also high in San Jose and other cities in the region. By comparison, the only fees in Houston are about $2,000 per unit to hook up to sewer and water systems. In Houston, it’s also possible to buy land, get all permits, build a home, and move in within a few months; in San Jose, it can take years.

A 2002 study quantified each of these costs by comparing home construction in San Jose vs. Dallas. A 2,400-square-foot lot in San Jose cost $200,000 more than a 7,000-square-foot lot in Dallas. Impact fees in San Jose at the time were $24,000 per home than in Dallas. Because it cost more to live in San Jose, labor costs were higher, adding another $143,000 to the cost of the house. Finally, the additional cost of getting a permit—and the risk that permit applications would be denied—added $90,000 to the cost in San Jose. All of this explains why the median home price in San Jose at that time was about $500,000 compared with under $120,000 in Dallas.

All of these factors have only gotten worse since 2002. The Census Bureau estimates the median home price in the San Jose urban area was $1.2 million in 2019 while in the Dallas-Ft. Worth urban area it was under $250,000. Zillow reports about the same median home values in 2020. To add insult to injury, San Jose now charges an “affordable housing fee” of $17 per square foot of new residential construction, thus making existing housing that much less affordable.

High housing costs have greatly limited the growth of Silicon Valley. Despite the fact that it is one of the wealthiest urban areas in history, the population of the region has grown by only about 25 percent since 1990. In the same time period, affordable regions such as Atlanta, Charlotte, Dallas-Ft. Worth, Houston, and Phoenix have grown by 80 to 140 percent or more.

Density and Taxes

The Santa Clara County tax assessor bases the value of residential properties on the value of the land plus the value of the improvements. Under proposition 13, which was passed in 1978, property values are reassessed whenever the property is sold. Between sales, assessed values can only increase at the rate of inflation but not more than 2 percent per year. Since urban-growth boundaries have led to prices increasing much faster than 2 percent per year, people who have recently purchased homes end up paying much higher taxes than people who have owned their homes for many years.

To see how density influences tax revenues, I compared the assessed values of single-family homes and condominiums of similar sizes that were most recently sold in the same years. In general, I found that, for properties sold in the same year, the assessed value of the improvements per square foot was about the same whether it was a single-family home or a multifamily condominium. However, the values of the land under the single-family homes were much greater than the land values for the condominiums. Thus, single-family homes bring in more revenue per dwelling unit than condominiums, but condominiums bring in more revenue per acre.

I also found that more than 200 multifamily housing projects, with more than 21,000 apartments or condominiums, have been built in Silicon Valley with low-income housing tax credits and other affordable housing subsidies since 1990. While I haven’t checked them all, the ones that I checked were almost completely exempted from property taxes as a part of their affordable housing subsidies. Many of the housing units in these developments are open to any family whose income is less than the median family income of the region. In 2019, the median family income in the San Jose urban area was more than $150,000, so this tax-exempt housing is not necessarily serving people with very low incomes.

The fears expressed in the 1960s that urban sprawl didn’t pay for itself or that it threatened farms and open space are unwarranted. The United States has 1.1 billion acres of agricultural land and only uses about a third of it for growing crops. The number of acres used for growing crops has declined not because of urban sprawl but because per-acre yields of most major crops are growing faster than the nation’s population. Only 3 percent of the nation and 5 percent of California has been urbanized, and urbanization is no threat to open space.

Santa Clara County, for example, was only 26 percent urbanized in 2010. Rural areas include hundreds of thousands of acres of low rolling hills, many of them immediately east of San Jose. These hills might be used as range for cattle but are far from prime farmland. Supposedly, the hills were excluded from the boundary because they are steep, but most of them are far less steep than fully developed hills in San Francisco. The highest and best use of these hills would be to open them up for housing and other urban development.

The question of whether growth pays for itself is complex because it depends on state and local tax rules (which means proposition 13 influences the answer) and numerous other factors. The most recent “costs-of-sprawl” study found that low-density housing imposed about $11,000 more in infrastructure costs per dwelling unit on communities than high-density housing. Since San Jose’s density policies have added hundreds of thousands of dollars to the cost of a residence, a mere $11,000 would be considered far more affordable.

Studies that claim that new residential development doesn’t pay for itself, especially in light of proposition 13, usually ignore the fact that such developments are inevitably accompanied by new retail, commercial, and industrial developments. The biggest community cost of new residences is schools, and the taxes paid by owners of those residences may not cover school costs. But retail, commercial, and industrial developments pay taxes too even though they don’t send any children to schools. When the tax revenues from all of these developments are considered, growth does pay for itself. In fact, taxes per household were far lower in 1970 than they are today.

It is useful to compare California with Texas, where counties aren’t even allowed to zone, much less impose urban-growth boundaries. Texas developers subdividing unincorporated land frequently design master-planned communities that include single- and multifamily homes, parks, schools, retail, and commercial areas. To provide the infrastructure needed for these communities, they often create municipal utility districts that sell bonds to pay for roads, sewer, water, and other utilities. Homebuyers and other property owners then pay an annual fee to repay these bonds. Irvine, California is an example of such a master-planned community, but it would be nearly impossible for a developer to build such a community in the San Francisco-San Jose region today.

Conclusions

Given the urban-growth boundary and the demand for more workers in Silicon Valley, the region appears to have little choice but to build high-density developments. But Hamann’s policy of allowing and annexing low-density housing produced more tax revenues for the city and much more affordable housing.

Fixing this problem will require action at both the state and local level. The state legislature, at the very least, should exempt changes in urban-growth boundaries from California 

Environmental Quality Act requirements for environmental impact reports. It would also help to abolish LAFCos so that cities, whose views of rural development are biased by their desire for tax revenues, have no control over rural home construction.

Once the region is allowed to change growth boundaries, it should abolish rather than simply expand the boundary. Mere expansions will lead to rapid speculative bidding on all of the land within the boundary which will keep land prices high. Elimination of the boundary will allow land prices to fall to levels determined by supply and demand rather than by artificial constraints.

Silicon Valley is one of the greatest generators of wealth the world has ever seen, and that wealth generation does not depend in any way on policies that have made housing unaffordable to most people. Building more high-density housing will not make housing more affordable both because of high land costs and because construction costs of mid-rise and high-rise housing are greater than for single-family homes. Eliminating the laws, ordinances, and regulations that restrict new developments of single-family homes will make the region’s housing much more affordable and significantly add to the net wealth generated by the region.

Follow Opportunity Now on Twitter @svopportunity.