What did I get right and wrong about the first quarter?

I analyze my performance every quarter to see if there is any pattern of error in my analysis. I devised a scoring method analogous to the grade point average system used in most US schools. You can read about the fourth quarter here.

This quarter I made one public release of data here and a semi-public update on the iPhone shipments here.

The resulting scores for both early and late estimates are shown in the table below:


Overall performance improved with the later estimates but not enough to move the letter grade. In fact, the letter grade of B+ is the same as it was for the fourth quarter.

The best performance was in (late) iPhone prediction and the worst performance was in Mac prediction. The Mac error was due to the relatively strong quarter the year before (which grew over 30%) and the lack of new product introductions. I did not think enough about these two conditions.

iPod error was in the direction of underestimation. The iPod grew better than expected through the average price seems to have dropped.

iPad error was quite modest at 3.4%. We found out after earnings that Apple was supply constrained. They indicated that they will sort out the supply/demand balance in the current quarter.

Gross margin also came in well above estimates, reaching a record high. This is difficult to predict as many factors come into play.

Overall the average error was less than 6% across all categories (late error). This means there was little wrong with the process of prediction. The seasonally adjusted growth for the iPhone and iPad remained consistent. Next, I will look at the scores for all the other public forecasts.

  • I’d read you even if your error rate was higher, I like your analysis and you have a bigger world view than many. Keep up the great work.

    • nangka

      yeah. apart from the great analytical posts, horacle also analyzes his performances which I don’t, or have yet to, read that another analyst, pro or blog, done it. it’s like learning, and learning again from the great one.

  • Don’t forget to give yourself an E for works well with others. 😉

  • oases

    Your average is only less than 1% if you allow the negativity and positivity to cancel eachother out. Your average distance from correct was 5.6% …but I don’t know if that’s a better way of looking at it; I’m not a mathematician.

  • Remind me not to take your class. You are a hard grader.

  • Neil Kosterman

    Common “best” means of measuring forecasting performance is MEAN ABSOLUTE PERCENT ERROR. Direction of error specifically not taken into account, intentionally.

  • Jamie Barnett

    Yep, I’m a fan too. Always learn a lot from you.

  • Guest

    The line between A and B is drawn at 3.5.  You should up your 3.63 B+ grade to A-.

  • Gondare

    Hey Horace,

    You should really take the absolute values of the errors and then take the average of the absolute values. When you do that, the average error for early prediction comes to 7.6% and 5.6% for late prediction. Still pretty accurate prediction, but the stats should be done right.

  • chano1

    At some point it’s important to set such pursuits aside and do real stuff. If I were to consider attempting to forecast AAPL results, whether for work or as a hobby, I would likely desist after a very short while.
    Because I would keep asking myself, will the result of all my efforts really matter, to anyone, after a few days?
    That question is a real perspective setter, in my experience.

    I’m not dismissing your work on this Horace, but I have to ask, does it matter?
    Your mainstream work is far more interesting, thought-provoking and often of enduring value.
    Life imitates art here. In this game, well, you win some, you …… 

    The one to watch perhaps is Peter Oppenheimer. He is the only one who understands the game and he has grown rich playing it, and the market.

    • There is only one person to whom the results of your test need to matter. You.

  • Craig Yoshioka

    What’s other people’s take on the manipulation of Apple’s stock?  I’m not sure if it’s a conscious manipulation, or if it’s an emergent phenomena of HFT, but my current crackpot hypothesis is, that the fact the stock is hovering around 600 then dropping far below, then bouncing back seems fishy.  As if something is exploiting the fact that 600 is a nice round number and they might trigger a statistically higher number of sells if they dump enough stock while above 600, then buy it back later?  Dunno, but I’d predict the stock is going to jump up past 600 again in the next day or two.

    • I don’t think manipulation at this scale is possible.

      • That’s what they said about citrus futures, but the Duke Brothers proved them wrong!

        But seriously, aren’t we looking a “emergent behavior”. The interesting question, IMO, is what triggers this behavior?

      • In the case of HFT algorithms, it sure looks like part of the emergent behavior is related to some sort of momentum metric. I was just looking at some of the commentary that reported the government flash crash analysis, and it definitely looks like they react to strong momentum signals (and sometimes amplify them).

        If the algorithms interact the way I’d expect them to, the question of “what triggers the behavior” may be irrelevant… it may be more a matter of “what *doesn’t* trigger this behavior”. Chaotic feedback systems exhibit all kinds of weirdness. I’m frankly amazed that there’s anything resembling a stable state.

      • So, everyone gets bitchslapped by the invisible hand? 🙂

        Given that there might be any number of competing/latent/submerged behaviors waiting to emerge, perhaps it is just that the stable state is the likeliest at any given moment. Different inputs cold nudge it temporarily to different unstable states, or it might reach a new, heretofore unseen, stable state.

        Ugh, I’m not even sure what I’m talking about anymore! My brain is resisting. I did my best to avoid using the term “quantum”. Doh!

      • Speaking as someone who spent a lot of time trying to figure out why distributed systems behaved in weird ways, I think I’ll just stick my head in the sand and not watch….

        And no, it’s not quantum-bad, I’ve had to play with that sort of stuff too, and it’s a whole different kind of weird.

    • I am very suspicious that there are major emergent interactions between the different HFT operations. You have a set of different algorithms that are cross-coupled, and their actions affect the actions of both themselves and their peers. I’ve also heard comments that some of the “algorithms” are learning neural-network designs. Add all that together and you have a perfect recipe for a chaotic oscillator….

      While I haven’t worked on financial market algorithms, I do have a lot of experience with various large-scale parallel algorithms and systems, including Internet routing.

      Such systems are *very* prone to various kinds of instabilities, especially when connected with varying delays (e.g. network links) so that their view of the universe isn’t entirely consistent. It took a long time to develop stable and effective Internet routing algorithms, and that was a system where everyone was actually cooperating on doing the same thing. With the secret trading algorithms of multiple firms each trying to one-up another… I see this as a fertile breeding ground for a *lot* of chaotic emergent behavior.

      I’m also concerned that if there are neural network components, these components may be learning to manipulate the market to advantage, even though they aren’t *explicitly* programmed to do so. The SEC has rules that stop human traders from manipulating markets, but can it be shown that the HFT algorithms abide by these rules? It is *very* difficult to understand why a neural network is doing something, or to predict what it will do in novel circumstances. Frankly, I’d hope this sort of technology would be outlawed in computer trading, but as far as I know it’s allowed.

      • Craig Yoshioka

        Yeah, and indeed “learning” behavior like the one I described is exactly what a wiley programmer would want their HFT algorithm to do… i.e. exploit human tendency (or even feedback from other HFT systems) to trigger decisions based on nice round numbers etc..  There is no reason why HFT might not be done with the intent to try and manipulate a stock if an equation determined that the HFT had access to enough monetary resources, and the stock was sufficiently close to an unstable price point, etc. It’s what I would try to do if I programmed these things.

      • I don’t think an explicit human directive is even necessary, if there is an optimization feedback loop that drives tweaks to the algorithm by rewarding successful performance. “Cornering the market” is a very successful strategy for profit, just not a legal one. But if the algorithm hits upon such a strategy without (obvious) deliberate human intent other than “perform as well as you can”….

        Another potential problem emerges if Algorithm A can predict Algorithm B’s response to Algorithm A, at which point Algorithm A can enlist B as an unwitting stooge to help manipulate the market price. And these algorithms are written *antagonistically* relative to each other, so they may indeed be cued from unintentional signals from other algorithms.

        The human managers and programmers could rightfully (or wrongfully) disclaim any intent to behave illegally, and they might not even be able to tell if the system hit on an illegal behavior on its own, if it’s a neural net or other self-optimizing adaptive system.

        The potential for shenanigans, both machine and human, are scary.

  • Horace the Grump

    Seasonally adjusted?  Are you sure?  First you don’t have anywhere near enough data points (usually 5 years of monthly data (60 data points) is the minimum) and you are looking at a time series that is heavily influenced by factors other than seasonality – growth in China, new product launches (slight Osbourne effect there I think as well), and supply chain issues.

    Technically you could do it – but I think the estimates would be pretty much meaningless given the above factors and the influence they have on the numbers.

    One interesting way of looking at this issue would be to estimate the relative contribution of these factors in a quarterly result – would pretty much by guesstimates, but could be quite instructive nonetheless.

  • kankerot

    You have been overly positive on Revenue, Iphones, Ipads, Macs and overly negative on EPS, Ipods and GM.

    It stands to reason if you are overly positive on Revenue that you will be overly positive on the 3 big revenue drivers – Iphones, Ipads and Macs.

    You have underestimated Apples ability to squeeze its suppliers and manufacturers and costs and thus its reflected in better GM and EPS than you estimated.

    The error on the Ipods is not that relevant as it is a product with declining sales. Its the error oon the Macs – so not sure whether this is a seasonal one off or portending to something else – perhaps Ipad sales eating into Mac sales?

    • I’d bet the Mac numbers are a lot like the iPhone numbers last year before the 4S was released — deferred sales due to Macbook refresh expectations. The refresh seems to be pushed out a bit later this year due to delays in the new Intel chips becoming available, which may have led to an extended wait on the part of potential buyers. Certainly if I were in the market for a new Macbook right now, I’d be waiting until the new model comes out, which is predictably soon if you’re a veteran Mac buyer.

      I don’t know how many Mac buyers actually pay attention to the rumor mill, but it certainly seemed to affect iPhone buyers last year, and I’d expect the Mac buyers to be more likely to know the usual schedules than iPhone buyers.
      That said, there’s probably a some iPad cannibalization as well, Tim Cook said as much last quarter, I think.

    • Revenue is a function of units and price. One can have a high estimate on revenue while having a low estimate on units and vice versa.
      Management discussed the Mac business and explained their take on why it was growing more slowly in the last quarter during the conference call.
      Richard Gardner – Citigroup Inc, Research Division

      I recognize that you outgrew the market in Macs yet again. But I did want to get your views on the slowdown in year-over-year growth rate within the category and what factors you would attribute that to. Are we — is it product transition related? Were the inventories down quarter-over-quarter? iPad cannibalization? Any other color that you can provide would be helpful.
      Timothy D. Cook

      Yes, Rich, it’s Tim. As you said, we did outgrow the market. Macs grew at about 7% where the market grew about 2%. And so this is the 24th straight quarter that we’ve outgrown the market, and so we’re extremely pleased with that. The compare to last year is largely affected by the fact that we changed the bulk of our portable line, or the MacBook Pros, in the February time frame of 2011. And so it’s a very tough compare. Specifically, the portables last year were up 53% year-on-year. And obviously, that — so that compare was very difficult. If you look at it sequentially, you also have to factor in that we had 14 weeks in the December quarter. And so the 26% year-over-year growth in December quarter is probably more like 17% when you factor out that 14th week. And so yes, I think there was some cannibalization from iPad and the market is slow. But the much, much larger factor and in fact, in — it might be the vast majority of the difference is the compare to a year ago.

  • mt23

    Why did you lower iPhone shipment due to an earlier CNY?  CNY fell in Q1 both this year and last year.  It’s a great call…would like to understand your insight.”I had adjusted iPhones earlier to 35.4 million because of the relatively early timing of Chinese New Year this year (January 23) vs. previous year (February 3).”

    • My assumption was that there would be fewer shopping days before the holiday. However, I was told after the event that most people receive cash as a gift so the shopping happens after the event. However, there were probably fewer units available for sale early in the season so the demand/supply balance occurred later in the quarter. As it turns out my adjustment was correct but for the wrong reasons.

  • Horace, I think using a GPA system is weird. Getting 90% in a course in school is different from being 10% off in a prediction. This is really an Apples/Oranges way of grading yourself.

    I mean when you look at this list 

    with the worst predictions being 15% off, even that analyst would get an overall B-, meaning that all you guys were in a range from A+ to B- 

    You can’t consider that to be what in Europa would be called “good minus”. 

    Being 5% off in the financial world is huge. All analysts being 15% off would send the stock price into turmoil.

    But I understand that this is a way to make numbers, which to me are fascinating on their own, and your predictions, more easily understandable for your readers.

    • The choice of the letter grade system is to make it understandable to a large audience. The trick is in getting the range of error matching a letter grade. I use 15% as about a C level and 33% as a failure. There are many analysts who are getting D or F grades. You can see the historic performance here:

      • Thanks for the reply and the link Horace, much appreciated.

        I perfectly understand that the letter grade is just a rough measure to make the numbers game more readily understood for the layman. And I agree with the grades you gave yourself. To me, you were only off on the Mac numbers and those didn’t really matter in the overall picture. (oh and I would never say I am perfect with numbers or consider myself a “pro” like you)

  • Rudolf Charel

    Predicting the future is fraught with difficulties. Horace did OK, but could do better.

    I let you off the hook as I have no money to invest in Apple and only use their products. Still, I find this blog fascinating and informative.

    Keep up the good work.

  • Sacto_Joe

    The iPad is predictably always going to be production constrained in its first quarter of release as long as they keep the release date so close to the holiday season. The have massive sales of the old iPad to fulfill before they can begin to shift production to the old iPad.

  • ftaok

    In regards to the iPods numbers going up, but the iPod revenue going down.  Could it be that the recall program for the 1G nano is the reason?  I surmise that lots of folks who traded in a broken/unused 1G nano and received a brand-new 6G nano were so delighted that they went out and bought one for their spouse, children, or significant others.  That could account for the numbers going up, but the average price going down.

    Does anyone know if the ratio of iPod touches to other iPods decreased this quarter?

  • Jeff

    Just a side note. It looks like the relationship you’ve plotted in the past, regarding share price as a function of cash is still holding up well.

    Share price is approximately =

    5x nominal cash equivalents/1 billion

    So, at $110 billion cash equiv. share price would equal $550 (it is $585 as I write)

    How much cash might they be expected to add by year end? Within 12 months? Etc?