Do correct me if I’ve got any of the history wrong here:
1. It seems pretty likely that a big part of the reason why Amazon’s website works so well and Obamacare’s website works so poorly is that Obamacare, unlike Amazon, is not subject to the discipline of the market (and therefore, for example, employs coders with no equity in the enterprise).
2. A whole lot of people predicted that the Obamacare bureaucracy would not work well because it would not be subject to the discipline of the market. I’m not sure anyone pointed to the webpage as a particular point of vulnerability, but plenty of people made the general observation that large government bureaucracies don’t work well and that this was a reason to be skeptical of Obamacare.
3. Paul Krugman pooh-poohed those concerns.
4. Paul Krugman reminds us approximately 914 times per month that only a very bad person would fail to acknowledge accurate predictions of his adversaries. (It’s true that in approximately 914 of those 914 cases, the vindicated adversary is Paul Krugman. But he has indicated support for the general principle.)
5. Paul Krugman has made no attempt to acknowledge that his adversaries made a correct prediction about Obamacare.
6. I leave it to my readers to draw the logical conclusion.
Technically, that’s 5 observations and a prediction we will draw the logical conclusion…
Interesting observation on the health care website: http://www.alexmarchant.com/blog/2013/10/22/healthcare-dot-gov-lines-of-code-comparison.html
Pretty darn lazy post even for an economist.
Better post would have included examples of the predictions and the pooh-poohing.
And someone who wasn’t just a butthurt gotcha playing jerkoff would email Krugman and ask him about it, which might alert him to his past predictions and how he got them wrong.
Well, there are many other government websites that are not subject to the discipline of the market, but which work better than the Obamacare website. So that’s unlikely to be the biggest factor.
I think the relevant difference is that Amazon’s audience grew slowly (giving them time to ramp up accordingly), not in a huge jump like the Obamacare site.
Bennett Haselton (2) is absolutely correct. This has nothing to do with discipline of the markets and everything to do with creating a large IT infrastructure at once.
We see this almost every time a new computer game comes out that requires a connection to the publishing company’s servers.
SimCity: http://kotaku.com/5991077/your-complete-guide-to-the-simcity-disaster
SimCity (again, when it launched on Mac): http://www.gamesindustry.biz/articles/2013-08-30-simcity-mac-launch-runs-into-problems
Star Wars: The Old Republic: http://www.gamespot.com/articles/star-wars-the-old-republic-server-queues-upset-players/1100-6347847/
Diablo III: http://www.pcadvisor.co.uk/news/game/3358011/diablo-iii-players-angry-as-hell-at-launch-chaos/
All of these were subject to the discipline of the market. All of them had launch issues.
It’s very easy to predict that a large scale IT project will have issues upon release. That doesn’t prove it’s because of a lack of market discipline.
The NSA isn’t subject to market discipline but seems to have very effective IT in place.
Point 1 is speculation. There are other possible reasons.
@Landsburg,
I agree that the website is flawed but that it probably has more to do with the way the government conducts auctions than market efficiency. One would have thought that given the scope and complication of the website, google would have been employed, instead we got some random terrible newbie company.
Anyway, I don’t think anyone expects the government to be good at website creation. In fact this is exactly the sort of thing that we know the private sector does better. This is not the same as insurance and transfer payments which the government arguably does better than the private sector.
I also think Obamacare has created some bad incentives that wouldn’t have been a problem in a single-payer system. This is what we get for taking heritage’s plan. On net though I still believe that it’s a better system then we had previously.
1. Amazon is subject to the discipline of the marketplace — you mean, like, the need to turn a profit? Since when?
Reviewing Amazon’s financial performance, some commentator concluded that Amazon appears to be a charity run by the investor class for the benefit of consumers.
No, Amazon does NOT turn a profit. But Amazon has the POTENTIAL to turn a profit in the future. And, depending on which forecasts you review and which things you value, so does Obamacare.
2. I eagerly await Landsburg’s review of how Amazon’s development conformed to its early business plan. Because the HHS reported years ago that developing the web site would be more expensive than expected; the project was undercapitalized. And what did Congress do with that information? Being filled with people who wanted nothing more than to see the project fail, Congress declined to authorize more funds.
Landsburg’s moral: Republicans predict government will fail, and when they get their hands on the levers of power this becomes a self-fulfilling prophecy.
Lack of exposure to the market may have something to do with the Obamacare website being broken, but only in the same sense that a lot of major online releases from big video game companies are broken on their release dates. With systems so complicated, you’ll never succeed in testing everything before implementation. I think it would be more accurate to suggest that the Obamacare online service will be fixed more slowly than a commercial online service due to its lack of exposure to the market.
The game analogy is good – these things are tricky from an IT standpoint because your peak load is day one, but you don’t want to overcommit resources. They should have done beta tests for interest early adopters and spread the rollout state by state
The NY Times has on Op-Ed today on this subject: http://www.nytimes.com/2013/10/23/opinion/emanuel-how-to-fix-the-glitches.html?hp&_r=0
It points to three main drivers of the issues with the website:
1. The Obama Administration finalize the exchange rules and regulations too late to allow sufficient time to implement and thoroughly test the website.
2. The leaders managing the website development were not experienced in large technology program management, which is necessary when attempting to rapidly deploy new technology.
3. As Bennett Haselton points out, the government did not have the opporunity to deploy a pilot system and then expand over time, which is how business usually wants to implement new systems.
Deploying large new technology systems is what I do for a living, and I agree that the A.C.A. Exchange websites appear to have these failings.
Re. the NSA, I don’t know how they manage their IT today, but up until the early 80s, they outsourced their IT development to Bell Labs. Bob Morris Sr. led that project for many years on behalf of Bell Labs.
nobody.really, the person who described Amazon as a “charity” is Matthew Yglesias. He also says Amazon never needs to turn a profit because it has plowed revenues that would normally be profit into expanding the business, and those assets are still valuable. If Amazon had created a lousy website nobody could use, it would have failed the market test and gone under.
Re observation #2, the widely-published Megan McCardle pointed to the webpage as a particular point of vulnerability in this article in late September 2013:
http://politicsinminnesota.com/2013/09/megan-mcardle-it-is-one-of-obamacares-weakest-links/
I look forward to Paul Krugman acknowledging her prescience.
Tom(UK) pretty much got it. I too can think of the number of times a for profit company like Blizzard still has problems with it’s online game users even a year or two after the launch due to glitches.
This is a lazy post…
I don’t have any special insight into the federal government’s website building business but I know that I am constantly bombarded by government “public service announcements” and that each government agency has a website.
Can the numerous defenders of the rollout (RJ #15, Tom #5) name a single private website that cost this much to develop and still had problems (not glitches or bugs, but was dysfunctional) nearly 3 weeks into launch? People site games of all things that cost a tenth of what this cost as if that were equivalent. Three things, A) I don’t need to buy or care about a game, B) When a game botches a launch, it isn’t with tax payer dollars, C) The company normally responds quickly and the problem is resolved in response to market pressure.
“Pretty likely” is straining under the weight you’re putting on it in that paragraph. Indeed, I think I heard it mutter “mama!” under its breath.
Amazon.com is well over 15 years old at this point, started out selling far less of a far simpler product, had regular site-wide outages for the first year of its commercial existence, and consumed several hundred million dollars of investor money before promptly going bankrupt. That the company still exists at all is a testimony to the overly generous design of American bankruptcy laws (and the indefatigable and frankly bizarre willingness of AMZN’s institutional investors to accept year upon year of minimal profits) than to “the discipline of the market” producing by magic an instantly stable website.
Not sure the programmer’s incentives have as much to do with it as the overall process.
Amazon.com has been evolving for a long time. It has the luxury of having control over roll-out dates which allows it to test. Even then, I imagine they’ve had their issues over the years, but can fix them without the political microscope.
But, the key is that Amazon.com lives in an environment that makes it easy to respond to its customers and evolve. Obamacare does not.
Daniel@7: “One would have thought that given the scope and complication of the website, google would have been employed.”
Google does not have any sort of professional services / government contracting / software consulting practice, nor would it be possible to staff one up from scratch in under a year. This is approximately like saying “I need a new combine harvester; I’d better call Ferrari.”
Point #1 suffers from survivorship bias. You know about Amazon because it’s solid. For every Amazon there are many large failed IT projects that were subject to the discipline of the market. You don’t know about those, because they failed.
@ Doctor Memory,
Okay, you’re obviously more knowledgeable about the tech field than I am. The articles I read seemed to suggest that the tech company that was contracted was a very new and untested company. I could have misread though, so you can probably give us more insight into this.
Lack of market discipline is perhaps half the story. The other half is the presence of political discipline, which forces politicians to do things that make it harder for government programs to work well. For instance, the regulations were delayed until after the 2012 elections, lest they give Republicans ammunition for the campaign. So real work couldn’t start as early as it should have. Also, the reason the mandated insurance has to cover all sorts of things, like chiropractors and acupuncture, is that politicians get rewarded by the providers of those things for putting them i. Something like Obamacare might have worked reasonably well, and at much lower cost, if it had been limited to catastrophic illnesses, with a high deductible. But who will make campaign contributions to a politician who favors that?
‘And someone who wasn’t just a butthurt gotcha playing jerkoff would email Krugman and ask him about it, which might alert him to his past predictions and how he got them wrong.’
Back in 1998, when Lee Gomes of the Wall Street Journal tracked Krugman down and got him to admit (sort of) that he’d been taken in by Paul David and Brian Arthur over ‘The Economics of QWERTY’, I did just that. I e-mailed him and asked him about some specifics of that chapter in ‘Peddling Prosperity’.
I got no answer. And, in Krugman and Wells (2006), students are still reading about ‘The QWERTY Problem’, which they define as;
“Government can play a useful role both in helping an industry establish a standard and helping it avoid getting trapped in an inferior standard known as the QWERTY problem” (p.536) and “in principle government intervention might be useful in moving an industry to a superior standard” (p.534).
Why don’t you e-mail him and ask about the contradiction. I’m sure Professor Landsberg would be happy to post his reply.
‘Something like Obamacare might have worked reasonably well, and at much lower cost, if it had been limited to catastrophic illnesses, with a high deductible.’
Yeah, you might even be able to sell it in the marketplace!
Daniel: I can’t say I have a lot of insight into why the company in question was chosen or whether a different one might have done a better job, and I certainly wouldn’t claim that the current status quo was an acceptable one.
But yes, I work in this field (including 4 years at Google, for whatever it’s worth), and I’m always amused at the amount of magical thinking and ex-post-facto rationalizations that go on from people unacquainted with the interior of the sausage factories. :)
To wit, a few thoughts:
– The fainting shock of the chatterers that a huge web application which requires interfacing with dozens of disparate legacy systems on the backend and which had little or no flexibility in its first ship date might have some crippling bugs in the first release… well, at best I’ll chalk it up to naiveté. At worst I would call it politically motivated disingenuousness.
(And when I say ‘legacy’ I mean it: insurance companies are usually sitting on mainframe technology from the 1960s at best.)
– I’m leery of the claims that the integration company chosen was too small / too new. If the contract had been handed to IBM or HP/EDS, we would be instead hearing complaints about how obviously a nimble tech startup should have been given the job rather than a dinosaur behemoth. IBM and the other consulting giants have in their own history regularly delivered nonfunctional turkeys, and often over budget and behind schedule. There is no magic bullet here: some projects are authentically difficult, and indeed impossible to get right on the first try.
– I would go so far as to assert that in huge integrative IT projects like this, failure-at-launch isn’t just normal, it’s the norm. (I’ve certainly been involved in my share.) Frankly, if your 1.0 release performs perfectly, your product is probably a trivial one. Based upon what admittedly sketchy knowledge I have, I would classify healthcare.gov’s problems as “somewhat worse than average” but not surprisingly so given the scope of the project.
…all of which is not meant as any sort of excuse for what is for a lot of people a completely non-functioning product. If it’s still just as broken eight weeks from now, it’s authentically a disaster. But if it’s not, nobody will remember the bumpy launch by next year.
@ Doctor Memory,
Thanks, I’ve been worried about this for weeks (mainly because I felt really under informed), but your explanation helps to put things in perspective.
@Daniel. De Nada.
I do want to caution that I am really not trying to put a positive spin on things. Healthcare.gov is, full stop, broken right now, and there’s not any getting around that– now we get to find out if the people working on fixing it are worth their salaries, and there is no guarantee that the answer will be ‘yes’. I’m just allergic to airy assertions about how “the discipline of the market” would somehow have prevented this or made it better.
Some people here are mistakenly under the misapprehension that this post is about obamacare, healthcare.gov, and competitive pressure. It is, of course, snarkiness directed at the mendacity of Nobel prize winning polemicist.
@suckmydictum,
Steve’s general point about Krugman is not a very interesting one. Krugman said that Obamacare will work, and so far it hasn’t, therefore he should unequivocally admit he was wrong, just doesn’t seem like a very good argument to me. Most of Krugman’s points about why Obamacare would work had to do with the general structure of the program and the logic behind the individual mandate. A dysfunctional website does not prove that logic wrong, so I think Krugman would like to wait for more evidence before he admits that he was flat-out wrong about Obamacare.
I’m interested that Steve chose to pick on the website rather than the other flaws with Obamacare that are potentially more fatal. There’s the disposable income loss between 400%-401% of the poverty level, the marriage penalty for low income individuals, and the employer mandate. All of these would have been better targets than a website that hasn’t worked right in the first 22 days. Instead I feel that Steve is engaged in a gotcha war with Krugman who isn’t paying him the least of attention.
@Daniel,
Yes, just because SL is (or ought to be) above PK’s crap does not mean PK’s crap doesn’t exist.
Can the numerous defenders of the rollout (RJ #15, Tom #5)
…as he names two.
name a single private website that cost this much to develop and still had problems (not glitches or bugs, but was dysfunctional)
What the frag do you think glitches or bugs cause, a functioning website?
People site games of all things that cost a tenth of what this cost as if that were equivalent. Three [irrelevant] things…blah blah blah
Except we weren’t speaking of the costs. It was mentioned, and illuminated by Doctor Memory, that large scale IT projects have numerous problems when rolled out and I mentioned how they still continue. To argue that market discipline would have remedied these problems from the get-go is lazy.
IMHO, the most relevant and accurate points here are Alan Gunn #23 and Doctor Memory #26. One of the principles of IT development is the Iron Triangle: cost, schedule, and scope – pick two. What it means is that when managing a large IT project, you can control two of those variables, but not all three. In this case, all three were constrained, so what usually happens is that the schedule gives, by reducing the time spent on different activities. In this case, I would guess that design and testing were skimped on.
An IT deployment certainly CAN deliver a 100% perfect solution out of the box, and some do. But only when failure is not an option. I would bet that Boeing has very few defects in the systems that control its jets at their first flight. If you read the license agreement for Java (a programming language), it states that Java is not certified for use in air traffic control or nuclear power plant control, because it is hard to write a 100% defect-free application in Java. healthcare.gov is a javascript application on the front end.
The problem with creating a defect-free application is that there are diminishing returns in testing. The old 80-20 rule applies: 80% of the defects are found in the first 20% of testing. Thus, if I plan on 3 testing cycles, I can expect to find 99% of the defects. Where I work we are usually satisfied with 2 testing cycles, identifying 96% of the defects. To get close to perfection, we’d have to spend 2.5 times as much on testing, and it is not worth the money. The result is that we “test in production”. That’s life.
My take is similar to Doctor Memory’s. The HealthCare.gov debacle strikes me as a plain old large-scale software disaster. Private industry has been having them for decades. (This used to be called the “software crisis” by software engineers who despaired of ever reducing the failure rate.)
Most successful large software systems that we’re familiar with these days (Amazon,Google,Facebook) started out as successful small software systems, which were then incrementally grown — adding both features and user volume — to become what we see today. The industry has developed a variety of management practices (and a large amount of software capital) to support this methodology, and it works fairly well for the types of systems that can use it.
Unfortunately, HealthCare.gov was not one of those types of systems: The requirements and release date were set in stone by Act of Congress, so incremental growth was not really an option. So they tried to build the whole thing at once, broken into several large pieces that had to be integrated at the end.
They also had to integrate with many pre-existing systems, which adds to the complexity. I’m pretty sure that the New York Time’s estimate of 500 million lines of code must be including all the pre-existing systems. Principle coding started this year, so there’s no way it’s all new. And benefit enrollment websites are just not that complicated. I’ve worked on a couple myself.
If there is a lack of market discipline implicated here, it’s in the form of inflexibility of scope and schedule, coupled with delay in awarding contracts and finishing requirements specification.
My Dad worked on the design for the U.S. ICBM guidance systems in the 60s, and then again on President Reagan’s “Star Wars” program in the 80s. I asked him once why the systems development for “Star Wars” was such a miserable failure, while the U.S. ICBM development was such a success.
His response was that the men (and they were all men) working on the guidance systems in the 60s thought their children’s lives depended on their doing a good job. “Star Wars” was just pork barrel, and nobody cared.
@JamesP
Did they pay them by lines of code produced?
@RJ 32
Come on, “glitches” or “bugs” has the connotation of minor things wrong but with the overall system being largely functional, that’s why the left (less so than before, but still many) continue to choose those words.
You act as if Amazon or Facebook, if creating an similar version of this scale system with the same amount of funding but for a private, market purpose, wouldn’t result in the same kind of debacle 3 weeks into launch. It surely might but there’s no prior evidence for it and citing a gaming company with a tench of the funding does not convince me (who also wouldn’t take 3 weeks btw and still have a broken product). I’m sure those companies would laugh at having an all-in rollout date to begin with versus a beta period or state-by-state launch but no blame to go around to the feds for that.
“And someone who wasn’t just a butthurt gotcha playing jerkoff would email Krugman and ask him about it, which might alert him to his past predictions and how he got them wrong.”
I can’t imagine why anyone would want to e-mail Krugman and ask him about it.
A three-part series in the Huff Post documenting mistakes that Krugman won’t admit is a different story, though:
http://www.cyniconomics.com/2013/10/23/niall-ferguson-shatters-paul-krugmans-delusions/
With Niall Ferguson’s HuffPo approach, you may at least get through to a few people who were duped by Krugman’s boasts. The record shows that you won’t get through to Krugman himself. He’s dug himself too far into his false self image as “Krugton the Invincible” (not to mention bad policy positions).
Come on, “glitches” or “bugs” has the connotation of minor things wrong but with the overall system being largely functional, that’s why the left (less so than before, but still many) continue to choose those words.
Perhaps to an naive layperson who still makes ‘left-right’ that have no purpose.
You act as if Amazon or Facebook, if creating an similar version of this scale system with the same amount of funding but for a private, market purpose, wouldn’t result in the same kind of debacle 3 weeks into launch.
I do? I’m pretty sure that’s not what I said.
It surely might but there’s no prior evidence for it
Then you didn’t read Tom’s links or are an avid gamer. Bugs and errors persist all the time because they’re hard to correct or they weren’t able to be corrected before a launch deadline. That’s why there’s persistent ‘patches’ for certain games to fix compatibility problems.
Online gaming is just one example.
and citing a gaming company with a tench of the funding does not convince me (who also wouldn’t take 3 weeks btw and still have a broken product).
And your evidence is speculation or gut instinct? Good one!
You realize I’ve no real desire to actually convince you right? I didn’t take you seriously from the beginning and I don’t plan to.
I’m sure those companies would laugh at having an all-in rollout date to begin with versus a beta period or state-by-state launch but no blame to go around to the feds for that.
Wtf are you even talking about?
I’ve had a lot of health insurance companies. (I tend to switch annually, because they think their hikes outweigh the massive timesink transaction costs of switching. Fools! Joke’s on them!)
All of their websites have been terrible, and often inoperable. None of them were spun up nearly so quickly as Obamacare’s. None of them, for that matter, allow online application at all.
If you are looking for counter examples to (1):
– The California DMV sites works very well.
– I pay quarterly taxes on the federal tax site and have never had an issue.
I would be curious to know what government sites that have existed for as long as Amazon (or even existed for at least 5 years) have issues.
Doctor Memory has made a lot of the points I might have, but I wanted to add one: in my experience, most large websites these days are written by consultants, rather than in-house developers (my experience being mostly with large e-commerce sites), as this one was. Those consultants have the same amount of “equity in the enterprise” for those projects as they did for this one: if they don’t do a good job, and the client complains, they may get demoted, fired, and so on. So I think your first point is off the mark.
Also, Daniel #7: I’m not sure which company you’re referring to as a “newbie”, but the main company involved, CGI, was founded in 1976 and is one of the world’s biggest IT consultancy firms. I agree with Doctor Memory that they were in a no-win situation here.
@Rowan,
You’re right, I got some bad information from a bad article it appears.
Point 1: I don’t think this point is valid because there probably are govt websites that work well and companies’ websites which work poorly. For example Essex County Council’s library website works well. One of your posts was about something stupid in your Mircosoft’s Hotmail account. I wouldn’t draw the conclusion that if I want an IT project completed I’m better off asking Essex County Council than Microsoft!
Point 2: I find it hard to believe that no-one predicted the website would have problems. In the UK everyone knows that government IT projects have enormous problems (apart from library services). However if it is true that no-one did predict these problems one has to ask why. It is reasonable to assume that the US government has had many large IT projects. If every previous US govt IT project has performed well it is believable that no-one would predict problems with this particular IT project. If previous US govt IT projects have had problems then people would have predicted similar problems with this IT project. The fact that no-one predicted problems with the website can only mean that other govt IT projects / websites have not had problems. Therefore this point actually proves the success of govt websites which is the opposite of your intention!
Point 3: Krugman can not have pooh-poohed concerns regarding the website since there were no concerns raised about the website. I haven’t read Krugman’s opinions and you provide no quotes but I would guess that Krugman believes that Obamacare will be successful but it is best to judge in a year or two. If this is the case points 4 and 5 are not valid criticisms of Krugman, however the might be in the future.
I would echo Doctor Memory 26 Nd WindyPundit 34, but I want to emphasize that Windy’s last para is spot on. But naming it is not to deny it. It supports Steve’s point, not as Windy seems to imply, undercuts it.
Yeah I think you’re stretching things a bit Steve. We will see how things go once we start seeing major claims. In other news, Greg Mankiw has posted this:
http://investigations.nbcnews.com/_news/2013/10/29/21222195-obama-administration-knew-millions-could-not-keep-their-health-insurance
Quite frankly, I saw this coming but don’t really care. It sucks that this means ether: The Whitehouse staff lied, didn’t know about it, or the president lied, but it won’t really matter to me if the new healthcare law can provide better healthcare coverage overall at a lower cost.
46 – If the reason people are getting dropped is because the ACA says their plans don’t cover enough, how could this possibly result in lower cost? (Even if more care meant “better” care.)