I think the single biggest improvement Apple could (and really must) make to Siri is to make it faster. And that’s exactly the sort of thing Apple has never really shown the chops for.
Credit to John Gruber for calling out Apple on a clear competitive disadvantage it has compared to Google Now. You should also check out ATP for some really geeky discussion each week. I doubt The Beard would write something like this as openly on The Loop, but I’m sure he agrees.